Intelligent obstacle avoidance method and system

Abstract

An intelligent obstacle avoidance method is suitable for a mobile vehicle. An operation device is used to perform the following steps: obtaining a plurality of obstacle sensing data and vehicle sensing data; obtaining an obstacle determination result according to the obstacle sensing data and the vehicle sensing data; determining whether the obstacle affects a global path to be passed by the mobile vehicle according to the obstacle determination result; when it is determined that the obstacle affects the global path, generating a local obstacle avoidance strategy by using a behavior tree according to the obstacle determination result; updating the global path by using a path planning algorithm according to the local obstacle avoidance strategy; and generating an obstacle avoidance movement instruction by using a motion control algorithm according to the local obstacle avoidance strategy and the updated global path.

Description

Technical Field

[0001] This invention relates to an obstacle avoidance method and system, and more particularly to an intelligent obstacle avoidance method and system suitable for mobile vehicles. Background Technology

[0002] Automated Guided Vehicles (AGVs) are wheeled mobile robots that typically move along pre-set, fixed paths by recognizing guide markers on the ground. AGVs are mostly used for transporting goods and are now widely used in industry and warehousing. However, because AGVs usually rely on pre-set, fixed paths, changes in the external environment often require reconfiguring the AGV system and rearranging the guide markers on the ground. This makes it difficult for AGVs to adjust their paths in real time and flexibly to avoid obstacles that suddenly appear on their path. This poses a significant challenge for applying AGVs in complex environments requiring precise movement, such as medical settings. Furthermore, medical environments often contain numerous obstacles, such as patients, medical staff, and medical equipment. Failure to avoid people appearing on the path in time not only threatens personal safety but also causes financial losses to hospitals if they collide with delicate medical equipment. Therefore, improving the accuracy of AGV obstacle avoidance is crucial.

[0003] Therefore, optimizing the obstacle avoidance capabilities of AGVs to adapt to complex environments with multiple obstacles has become one of the issues that related technical fields seek to address. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent obstacle avoidance method and system that can overcome at least one drawback of the prior art.

[0005] The present invention provides an intelligent obstacle avoidance method applicable to mobile vehicles equipped with vehicle sensors and multiple different obstacle sensors. One of the obstacle sensors is a camera module. The intelligent obstacle avoidance method is executed using a computing device and includes the following steps: (A) obtaining multiple obstacle sensing data corresponding to the obstacle sensors using the obstacle sensors, and obtaining vehicle sensing data using the vehicle sensors; each obstacle sensing data indicates the positional relationship between multiple obstacles within the sensing range and the mobile vehicle; the vehicle sensing data indicates the motion state of the mobile vehicle, and one of the obstacle sensing data is image data obtained by the camera module; (B) obtaining information about the obstacle sensing data and the vehicle sensing data based on the obstacle sensing data and the vehicle sensing data. (C) Based on the obstacle determination result, determine whether the obstacle affects the global path that the mobile vehicle intends to travel; (D) When it is determined that the obstacle affects the global path, generate a local obstacle avoidance strategy using a behavior tree based on the obstacle determination result; the local obstacle avoidance strategy includes at least one of obstacle avoidance trajectory information and obstacle avoidance speed; (E) Update the global path using a path planning algorithm based on the local obstacle avoidance strategy; and (F) Based on the local obstacle avoidance strategy and the updated global path, generate an obstacle avoidance movement command for the mobile vehicle to avoid the obstacle using a motion control algorithm.

[0006] In some embodiments, step (B) includes the following sub-steps: (B-1) fusing the obstacle sensing data and the vehicle sensing data using multi-sensor fusion technology to obtain a fusion result, wherein the obstacle sensing data and the vehicle sensing data use the same coordinate system and are time-synchronized; (B-2) obtaining fusion data for each obstacle using at least target detection technology and target tracking technology based on the fusion result and at least one previously obtained fusion result, wherein each fusion data includes at least the motion trajectory information of the corresponding obstacle and the object category information of the corresponding obstacle; (B-3) classifying each obstacle into one of dynamic obstacles and static obstacles based on the fusion result and the fusion data of each obstacle; (B-4) for each obstacle, obtaining the distance category to which the distance between the obstacle and the moving vehicle belongs based on the fusion result and the fusion data of the obstacle, wherein the distance category is one of safe distance, warning distance, and danger distance; and (B-5) generating the obstacle determination result based on the distance category of each obstacle.

[0007] In some embodiments, between sub-steps (B-2) and (B-5), the following sub-step is further included: (B-6) determining whether there is at least one person obstacle among the obstacles based on the object category information of the fused data of each obstacle, wherein each person obstacle is an obstacle that indicates the person category according to the corresponding object category information; and (B-7) when it is determined that there is at least one person obstacle among the obstacles, for each person obstacle, obtaining a human posture recognition result corresponding to the person obstacle using human posture estimation technology based on the fusion result.

[0008] In some embodiments, in sub-step (B-5), the obstacle determination result is generated based on the distance category of each of the obstacles and the human posture recognition result of each of the at least one person obstacle.

[0009] In some embodiments, sub-step (B-2) includes the following sub-steps: (B-2-1) obtaining target detection results using target detection technology based on the fusion result, the target detection results indicating the object bounding box of each obstacle and the object category information of each obstacle; and (B-2-2) obtaining the motion trajectory information of each obstacle using target tracking technology based on the fusion result and the target detection result.

[0010] In some embodiments, the obstacle sensor is a three-dimensional optical radar, an ultrasonic sensor, a two-dimensional laser scanner, and the photography module, and the vehicle sensor is an inertial measurement unit. In step (A), the obstacle sensing data is three-dimensional point cloud data obtained by the three-dimensional optical radar, distance data obtained by the ultrasonic sensor, two-dimensional point cloud data obtained by the two-dimensional laser scanner, and the image data.

[0011] In some embodiments, between steps (A) and (C), the following steps are further included: (G) updating a three-dimensional environment model of the sensed range and the obstacles therein using simultaneous localization and mapping technology based on the fusion result; and (H) obtaining the global path using another path planning algorithm based on the three-dimensional environment model.

[0012] The present invention provides an intelligent obstacle avoidance system suitable for mobile vehicles, and includes a vehicle sensor, multiple different obstacle sensors, and a computing device.

[0013] The vehicle sensor is used to generate vehicle sensing data. The vehicle sensing data indicates the motion state of the moving vehicle.

[0014] The obstacle sensor generates multiple obstacle sensing data points, each corresponding to one of the obstacle sensors. Each obstacle sensing data point indicates the positional relationship between multiple obstacles within the sensing range and the moving vehicle. One of the obstacle sensors is a camera module, and one portion of the obstacle sensing data is image data acquired by the camera module.

[0015] The computing device is connected to the vehicle sensor and the obstacle sensor, and is used to receive vehicle sensing data transmitted by the vehicle sensor and obstacle sensing data transmitted by the obstacle sensor; obtain an obstacle determination result regarding the obstacle based on the obstacle sensing data and the vehicle sensing data; determine whether the obstacle affects the global path that the mobile vehicle intends to travel based on the obstacle determination result; when it is determined that the obstacle affects the global path, generate a local obstacle avoidance strategy using behavior tree based on the obstacle determination result, the local obstacle avoidance strategy including at least one of obstacle avoidance trajectory information and obstacle avoidance speed; update the global path using path planning algorithm based on the local obstacle avoidance strategy; and generate an obstacle avoidance movement command for the mobile vehicle to avoid the obstacle using motion control algorithm based on the local obstacle avoidance strategy and the updated global path.

[0016] In some embodiments, the computing device uses multi-sensor fusion technology to fuse the obstacle sensing data and the vehicle sensing data to obtain a fusion result. In the fusion result, the obstacle sensing data and the vehicle sensing data use the same coordinate system and are synchronized in time. Based on the fusion result and at least one previously obtained fusion result, the computing device uses at least target detection technology and target tracking technology to obtain fusion data for each obstacle. Each fusion data includes at least the motion trajectory information and object category information of the corresponding obstacle. Based on the fusion result and the fusion data of each obstacle, the computing device classifies each obstacle into either a dynamic obstacle or a static obstacle. For each obstacle, based on the fusion result and the fusion data of the obstacle, the computing device obtains the distance category between the obstacle and the moving vehicle, which is one of a safe distance, a warning distance, and a danger distance. Based on the distance category of each obstacle, the computing device generates the obstacle determination result.

[0017] In some embodiments, the computing device determines whether there is at least one person obstacle among the obstacles based on the object category information of the fused data of each obstacle, wherein each person obstacle is an obstacle that indicates the person category according to the corresponding object category information; and when it is determined that there is at least one person obstacle among the obstacles, for each person obstacle, the computing device obtains a human posture recognition result corresponding to the person obstacle using human posture estimation technology based on the fusion result.

[0018] In some embodiments, the computing device generates the obstacle determination result based on the distance category of each of the obstacles and the human posture recognition result of each of the at least one person obstacle.

[0019] In some embodiments, the computing device obtains target detection results using target detection technology based on the fusion result, the target detection results indicating the object bounding box of each obstacle and the object category information of each obstacle; and the computing device obtains the motion trajectory information of each obstacle using target tracking technology based on the fusion result and the target detection result.

[0020] In some embodiments, the obstacle sensor is a three-dimensional optical radar, an ultrasonic sensor, a two-dimensional laser scanner, and the photography module, the vehicle sensor is an inertial measurement unit, and the obstacle sensing data is three-dimensional point cloud data obtained by the three-dimensional optical radar, distance data obtained by the ultrasonic sensor, two-dimensional point cloud data obtained by the two-dimensional laser scanner, and the image data.

[0021] In some embodiments, the computing device updates a three-dimensional environment model of the sensing range and the obstacles therein based on the fusion result using simultaneous localization and mapping (SMR) technology; and the computing device obtains the global path based on the three-dimensional environment model using another path planning algorithm.

[0022] The beneficial effects of this invention are as follows: First, by obtaining obstacle sensing data from different obstacle sensors and vehicle sensing data, it is helpful to comprehensively collect information about the obstacle, so as to obtain the obstacle determination result more accurately. Based on the obstacle determination result, it can be determined whether the obstacle affects the overall path. Second, by generating the local obstacle avoidance strategy based on the obstacle determination result when it is determined that the obstacle affects the overall path, the mobile vehicle can perform more precise obstacle avoidance based on the overall path, so as to better avoid and bypass the obstacle, thereby achieving the effect of flexible obstacle avoidance in complex environments. Attached Figure Description

[0023] Other features and effects of the present invention will be clearly presented in the embodiments with reference to the accompanying drawings, wherein:

[0024] Figure 1 This is a flowchart illustrating an embodiment of the intelligent obstacle avoidance method of the present invention.

[0025] Figure 2 This is a block diagram illustrating, exemplarily, the mobile vehicle and intelligent obstacle avoidance system used in the embodiments described;

[0026] Figure 3 This is a perspective view, exemplarily illustrating the mobile medical vehicle used as the mobile carrier in the described embodiment;

[0027] Figure 4 This is a perspective view illustrating, exemplarily, the mobile medical vehicle used in the described embodiment;

[0028] Figure 5 This is a block diagram illustrating, exemplarily, an implementation of the intelligent obstacle avoidance system used by the mobile medical vehicle described in the embodiment.

[0029] Figure 6 This is a flowchart that exemplarily illustrates how the computing device of the intelligent obstacle avoidance system in the described embodiment obtains the obstacle determination result. Detailed Implementation

[0030] Before the invention is described in detail, it should be noted that similar elements are represented by the same numbers in the following description.

[0031] See Figure 1 and Figure 2 An embodiment of the intelligent obstacle avoidance method of the present invention is applicable to a mobile vehicle 1. The mobile vehicle 1 is equipped with an intelligent obstacle avoidance system 2, which includes a vehicle sensor 21, a plurality of different obstacle sensors 22, and a computing device 23 connecting the vehicle sensor 21 and the obstacle sensors 22.

[0032] The vehicle sensor 21 is used to generate vehicle sensing data, which indicates the motion state of the moving vehicle 1 (e.g., angular velocity and acceleration).

[0033] The obstacle sensor 22 is used to generate multiple obstacle sensing data points, each corresponding to the obstacle sensor 22. Each obstacle sensing data point indicates the positional relationship between multiple obstacles within the sensing range and the mobile vehicle 1.

[0034] In this embodiment, the mobile vehicle 1 is, for example, Figure 3 and Figure 4 The mobile medical vehicle shown is an AMiS-50E. The intelligent obstacle avoidance system 2 installed on the mobile medical vehicle is as follows... Figure 5 As shown, the vehicle sensor 21 includes an inertial measurement unit (IMU), and each of the obstacle sensors 22 includes a three-dimensional LiDAR (LiDAR) 22a, an ultrasonic sensor 22b, a two-dimensional laser scanner 22c, an infrared sensor 22d, and a photography module 22e. The computing device 23 included in the intelligent obstacle avoidance system 2 installed in the mobile medical vehicle can be implemented as a processor installed on the mobile medical vehicle, a server independent of the mobile medical vehicle, or a combination of both. The computing device 23 is electrically or communicatively connected to the vehicle sensor 21 and the obstacle sensor 22. Since the medical environment is filled with high-precision medical instruments and dynamic objects and people, the three-dimensional optical radar 22a can obtain high-precision perception of three-dimensional space, the ultrasonic sensor 22b can be used to detect obstacles at close range, the two-dimensional laser scanner 22c can scan planar obstacles around the mobile medical vehicle to avoid collisions with low objects or people's feet, and the infrared sensor 22d can detect surrounding heat sources or obstacles in low light or no light conditions.

[0035] The following will combine Figure 1 , Figure 2 and Figure 5 Each step of the described embodiment will be explained in detail below.

[0036] In step S1, the computing device 23 receives vehicle sensing data transmitted by the vehicle sensor 21 and multiple obstacle sensing data transmitted by the obstacle sensor 22. In this embodiment, the vehicle sensing data is attitude data obtained by the inertial measurement unit, and the obstacle sensing data are three-dimensional point cloud data obtained by the three-dimensional optical radar 22a, distance data obtained by the ultrasonic sensor 22b, two-dimensional point cloud data obtained by the two-dimensional laser scanner 22c, infrared signal data obtained by the infrared sensor 22d, and image data obtained by the photography module 22e.

[0037] In step S2, the computing device 23 obtains an obstacle determination result regarding the obstacle based on the obstacle sensing data and the vehicle sensing data. More specifically, see [link to relevant documentation]. Figure 6 The computing device 23 obtains the obstacle determination result by performing the following sub-steps.

[0038] In sub-step S21, the computing device 23 uses Multi-Sensor Fusion (MSF) technology to fuse the obstacle sensing data and the vehicle sensing data to obtain a fusion result. In the fusion result, the obstacle sensing data and the vehicle sensing data use the same coordinate system and are time-synchronized. More specifically, the computing device 23 may, for example, use a Precision Time Protocol (PTP) algorithm or a GPS-based time synchronization algorithm to synchronize the timestamps of the obstacle sensing data and the vehicle sensing data. The vehicle sensing data (i.e., the attitude data in this embodiment) is then used to calibrate and correct the pose deviations between the obstacle sensors 22 using methods such as Extended Kalman Filter (EKF), Levenberg-Marquardt least squares, or Hand-Eye Calibration, or a combination thereof, so that the obstacle sensing data can be represented in the same coordinate system. Then, fusion algorithms such as Kalman Filter, EKF, Unscented Kalman Filter (UKF), or Particle Filter are used to fuse the obstacle sensing data and vehicle sensing data after the aforementioned processing, thereby obtaining the fusion result. Alternatively, a neural network based on multimodal learning can be used to extract features from the sensing data of different sensors and then perform deep fusion; or a temporal fusion algorithm based on Long Short-Term Memory (LSTM) networks can be used to fuse the data based on the temporal correlation of each sensing data point; or a transformer-based fusion model can be used to weight the sensing data using an attention mechanism to achieve fusion. By integrating the data from each sensor using MSF to obtain the fusion result, the strengths of the sensing data collected by each sensor are combined to compensate for their weaknesses, which helps to subsequently use the fusion result to build a more accurate 3D environment model and to make more precise plans for obstacle avoidance strategies and paths.

[0039] For example, in this embodiment, the inertial measurement unit obtains the attitude data at a frequency of 50 Hz, the three-dimensional optical radar 22a obtains the three-dimensional point cloud data at a frequency of 10–30 Hz, the ultrasonic sensor 22b obtains the distance data at a frequency of 10 Hz, the two-dimensional laser scanner 22c obtains the two-dimensional point cloud data at a frequency of 10–20 Hz, the infrared sensor 22d obtains the infrared signal data at a frequency of 5–10 Hz, and the imaging module 22e obtains the image data at a frame rate of 30 FPS. During MSF (Mean Squared Function), the obstacle sensing data and the vehicle sensing data are fused using YCi every 50 milliseconds to generate the fusion result.

[0040] It should be added that the "timestamp synchronization" described in this embodiment can refer to the complete alignment of the timestamps of each sensing data. At the same time, since the data collection frequency of each sensor is not consistent, the data parts with the closest timestamps in each sensing data can also be aligned to achieve the effect of "timestamp synchronization".

[0041] It is worth mentioning that when fusing the obstacle sensing data and the vehicle sensing data, a low-pass filter or a Kalman filter is used to filter out noise and interference in each sensing data to improve data quality.

[0042] In sub-step S22, the computing device 23, based on the fusion result and at least one previously obtained fusion result, uses object detection and object tracking techniques to obtain fused data for each obstacle. Each fused data includes at least the motion trajectory information and object category information of the corresponding obstacle. More specifically, sub-step S22 includes the following sub-steps: (i) the computing device 23, based on the fusion result and the at least one previously obtained fusion result, uses object detection techniques to obtain object detection results. The object detection results indicate the object box of each obstacle and the object category information of each obstacle; and (ii) the computing device 23, based on the fusion result, the at least one previously obtained fusion result, and the object detection results, uses object tracking techniques to obtain the motion trajectory information of each obstacle. For example, object detection can be implemented using a YOLO model or a Faster R-CNN model, and object tracking can be implemented using the DeepSORT algorithm, for example. Since target detection and target tracking technologies are common knowledge to those skilled in the art, detailed explanations are omitted here for the sake of brevity. By considering not only the fusion result but also the previously obtained at least one fusion result when obtaining the fusion data for each obstacle, the accuracy of the motion trajectory information of each obstacle and the reliability of the object category information determination of each obstacle can be improved. At the same time, it can also effectively avoid the interference of sensor noise and the impact of target occlusion during target tracking.

[0043] It should be added that, in this embodiment, since the obstacle sensing data includes the three-dimensional point cloud data, the distance data, the two-dimensional point cloud data, the infrared signal data, and the image data, the fusion result has the information encompassed by each obstacle sensing data. The fusion data of each obstacle generated according to the fusion result also has the information encompassed by each obstacle sensing data. That is, each fusion data may also include, for example, the three-axis spatial coordinates (position), three-axis velocity, three-axis acceleration of the corresponding obstacle at each time point, and the obstacle distance indicating the distance between the corresponding obstacle and the moving vehicle 1.

[0044] In sub-step S23, the computing device 23 classifies each obstacle into either a dynamic obstacle or a static obstacle based on the fusion result and the fusion data of each obstacle. For example, the static obstacle may be a hospital bed, medical equipment, or a wall, while the dynamic obstacle may be a moving medical person, patient, or medicine cart. In this embodiment, sub-step S23 includes the following sub-steps: (I) The computing device 23 obtains a point cloud segmentation result indicating the static background portion and the dynamic object portion within the sensing range using point cloud segmentation technology based on the three-dimensional point cloud data in the fusion result; and (II) Based on the point cloud segmentation result and the fusion data of each obstacle, classifies each obstacle into either a dynamic obstacle or a static obstacle. For sub-step (I), point cloud segmentation can be achieved using, for example, the Random Sample Consensus (RANSC) algorithm or PointNet++. This can be combined with methods such as background subtraction (e.g., VoxelGrid Filter and Statistical Outlier Removal), optical flow, 3D object detection models like SECOND (Sparsely Embedded Convolutional Detection) and PointRCNN, and two-stream network algorithms to separate the static background and dynamic object portions and perform dynamic / static classification of objects in the 3D point cloud data to obtain the point cloud segmentation result. For sub-step (II), methods such as Gaussian Mixture Model (GMM), spatio-temporal analysis, and multimodal fusion can be combined to classify each obstacle based on the point cloud segmentation result and the fused data.

[0045] By classifying each obstacle into a dynamic obstacle or a static obstacle, it is helpful to select the most suitable path planning algorithm for different types of obstacles during subsequent path planning. For example, the Dynamic Window Approach (DWA) algorithm can be used for dynamic obstacles, and the A* search algorithm can be used for static obstacles.

[0046] In sub-step S24, for each obstacle, the computing device 23 obtains the distance category of the distance between the obstacle and the mobile vehicle 1 based on the fusion result and the fusion data of the obstacle. The distance category is one of safe distance, warning distance, and danger distance. More specifically, in this embodiment, for example, the distance category corresponding to the obstacle can be determined based on the obstacle distance in the fusion data of each obstacle. For example, when the distance between the obstacle and the mobile vehicle 1 is greater than 100 cm, the distance category of the obstacle is the safe distance; when the distance between the obstacle and the mobile vehicle 1 is greater than or equal to 50 cm and less than or equal to 100 cm, the distance category of the obstacle is the warning distance; when the distance between the obstacle and the mobile vehicle 1 is less than 50 cm, the distance category of the obstacle is the danger distance.

[0047] In sub-step S25, the computing device 23 determines whether there is at least one person obstacle among the obstacles based on the object category information of the fused data of each obstacle. Each person obstacle is an obstacle indicating the person category for its corresponding object category information. When it is determined that there is at least one person obstacle among the obstacles, the process proceeds to sub-step S26; when it is determined that there are no person obstacles among the obstacles, the process proceeds to sub-step S28.

[0048] In sub-step S26, for each person obstacle, the computing device 23, based on the fusion result, uses human pose estimation technology to obtain a human pose recognition result corresponding to the person obstacle. For example, the human pose recognition result may indicate a stationary state where the person obstacle is stationary, a pushing state where the person obstacle is pushing an object, or a moving state where the person obstacle is moving. Since human pose estimation technology is common knowledge to those skilled in the art, a detailed explanation is omitted here for brevity.

[0049] It is worth mentioning that the determination of the distance category of each obstacle and the determination of whether there are people or obstacles are not limited to the execution order in this embodiment. Their execution order can be exchanged, or both can be executed simultaneously.

[0050] In sub-step S27, the computing device 23 generates the obstacle determination result based on the distance category of each obstacle and the human posture recognition result of each of the at least one human obstacle. The obstacle determination result includes the distance category of each obstacle and the human posture recognition result of each of the at least one human obstacle.

[0051] In sub-step S28, the computing device 23 generates the obstacle determination result based on the distance category of each obstacle. The obstacle determination result includes only the distance category of each obstacle.

[0052] In step S3, the computing device 23 updates the 3D environment model regarding the sensing range and the obstacles therein based on the fusion result using Simultaneous Localization and Mapping (SLAM) technology. For example, SLAM algorithms such as Gmapping, Hector SLAM, or Cartographer can be used to construct and update the 3D environment model. Since using SLAM technology to build the 3D environment model based on the fusion result is not a feature of this invention and is common knowledge to those skilled in the art, detailed descriptions related to SLAM are omitted here.

[0053] In step S4, the computing device 23 obtains a global path based on the three-dimensional environment model using a path planning algorithm. For example, the path planning algorithm may be a Dijkstra's algorithm or an A* search algorithm. It is worth noting that the computing device 23 also uses smoothing algorithms, such as Bézier curves or cubic splines, to smooth the path, avoiding overly straight lines or sharp turns, ensuring that the operation of the mobile vehicle 1 is more natural and safer.

[0054] In step S5, the computing device 23 determines whether the obstacle affects the overall path that the mobile vehicle 1 intends to travel based on the obstacle determination result. When it is determined that the obstacle affects the overall path, the process proceeds to step S6; when it is determined that the obstacle does not affect the overall path, the process proceeds to step S9. For example, the computing device 23 determines whether the obstacle affects the overall path by determining whether there is an obstacle in the obstacle determination result whose corresponding human posture recognition result is not in the stationary state or an obstacle whose corresponding distance category is not the safe distance. When it is determined that there is an obstacle whose corresponding human posture recognition result is not in the stationary state or an obstacle whose corresponding distance category is not the safe distance, the obstacle affects the overall path; when it is determined that there are no obstacles whose corresponding human posture recognition result is not in the stationary state and no obstacles whose corresponding distance category is not the safe distance, the obstacle does not affect the overall path.

[0055] In step S6, the computing device 23 generates a local obstacle avoidance strategy based on the obstacle determination result using a behavior tree. The local obstacle avoidance strategy includes at least one of obstacle avoidance trajectory information and obstacle avoidance speed. For example, the behavior tree may have the following behavior rules: (a) for obstacles whose corresponding distance category is the safe distance, the obstacle avoidance speed of the mobile vehicle 1 is to maintain the original speed; (b) for obstacles whose corresponding distance category is the warning distance, the obstacle avoidance speed of the mobile vehicle 1 is to gradually decelerate to the safe speed; (c) for obstacles whose corresponding distance category is the danger distance, the obstacle avoidance speed of the mobile vehicle 1 is zero, and the mobile vehicle 1 stops immediately; (d) for obstacles whose corresponding human posture recognition result is the stationary state, the obstacle avoidance speed of the mobile vehicle 1 is to maintain the original speed; (e) for obstacles whose corresponding human posture recognition result is the pushing object state, the motion trajectory information of the object pushed by the obstacle (the object is also an obstacle) is obtained as the obstacle avoidance trajectory information; and (f) for obstacles whose corresponding human posture recognition result is the moving state, the motion trajectory information of the obstacle is obtained as the obstacle avoidance trajectory information.

[0056] In step S7, the computing device 23 updates the global path using another path planning algorithm based on the local obstacle avoidance strategy. For example, the computing device 23 updates the global path using the DWA algorithm or the Rapidly-exploring Random Tree (RRT) algorithm, based on all obstacle avoidance trajectory information included in the local obstacle avoidance strategy. Updating the global path according to the local obstacle avoidance strategy ensures that the travel route of the mobile vehicle 1 can adapt to changes in the surrounding environment and obstacles in real time.

[0057] In step S8, the computing device 23, based on the local obstacle avoidance strategy and the updated global path, uses a motion control algorithm to generate an obstacle avoidance movement command for the mobile vehicle 1 to avoid the obstacle. In this embodiment, the motion control algorithm is implemented, for example, using a proportional-integral-derivative (PID) controller or a model predictive control (MPC) controller.

[0058] In step S9, the computing device 23 generates another obstacle avoidance movement command based on the global path and the motion control algorithm.

[0059] In summary, firstly, by obtaining obstacle sensing data from different obstacle sensors 22 and vehicle sensing data, comprehensive information about the obstacles can be collected. Secondly, by integrating the data from each sensor using MSF to obtain the fusion result, the strengths of the sensing data collected by each sensor can be combined to compensate for the weaknesses, which helps to build a more accurate 3D environment model using the fusion result and to make more precise plans for obstacle avoidance strategies and paths. Next, by analyzing the distance between each obstacle and the mobile vehicle 1 and the actions of each person and obstacle, the obstacle determination result is obtained, thereby generating the local obstacle avoidance strategy. This allows the mobile vehicle 1 to perform more precise obstacle avoidance based on the global path, so as to better avoid or bypass the obstacles, thereby achieving flexible obstacle avoidance in complex environments. Therefore, the purpose of this invention is indeed achieved.

[0060] The above description is merely an embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any simple equivalent changes and modifications made in accordance with the claims and description of the present invention shall still fall within the scope of the present invention.

Claims

1. An intelligent obstacle avoidance method, applicable to mobile vehicles equipped with vehicle sensors and multiple different obstacle sensors, one of which is a camera module, the intelligent obstacle avoidance method being executed using a computing device, characterized in that, The intelligent obstacle avoidance method includes the following steps: (A) Obtain multiple obstacle sensing data corresponding to the obstacle sensor using the obstacle sensor, and obtain vehicle sensing data using the vehicle sensor. Each obstacle sensing data indicates the positional relationship between multiple obstacles within the sensing range and the moving vehicle. The vehicle sensing data indicates the motion state of the moving vehicle. One of the obstacle sensing data is image data obtained by the photography module. (B) Obtain an obstacle determination result regarding the obstacle based on the obstacle sensing data and the vehicle sensing data; (C) Based on the obstacle determination result, determine whether the obstacle affects the global path that the mobile vehicle intends to travel; (D) When it is determined that the obstacle affects the global path, a local obstacle avoidance strategy is generated using behavior tree based on the obstacle determination result. The local obstacle avoidance strategy includes at least one of obstacle avoidance trajectory information and obstacle avoidance speed. (E) Update the global path using a path planning algorithm based on the local obstacle avoidance strategy; and (F) Based on the local obstacle avoidance strategy and the updated global path, a motion control algorithm is used to generate an obstacle avoidance movement command for the mobile vehicle to avoid the obstacle.

2. The intelligent obstacle avoidance method according to claim 1, characterized in that, Step (B) includes the following sub-steps: (B-1) The obstacle sensing data and the vehicle sensing data are fused using multi-sensor fusion technology to obtain a fusion result. In the fusion result, the obstacle sensing data and the vehicle sensing data use the same coordinate system and are synchronized in time. (B-2) Based on the fusion result and at least one previously obtained fusion result, at least using target detection technology and target tracking technology, obtain fusion data for each obstacle, wherein each fusion data includes at least the motion trajectory information of the corresponding obstacle and the object category information of the corresponding obstacle; (B-3) Based on the fusion result and the fusion data of each obstacle, classify each obstacle as either a dynamic obstacle or a static obstacle; (B-4) For each obstacle, based on the fusion result and the fusion data of the obstacle, the distance category to which the distance between the obstacle and the mobile vehicle belongs is obtained, and the distance category is one of safe distance, warning distance and danger distance; and (B-5) Generate the obstacle determination result based on the distance category of each of the obstacles.

3. The intelligent obstacle avoidance method according to claim 2, characterized in that, Between sub-steps (B-2) and (B-5), the following sub-steps are also included: (B-6) Based on the object category information of the fused data of each obstacle, determine whether there is at least one person obstacle among the obstacles, wherein each person obstacle is an obstacle indicating the person category according to the corresponding object category information; and (B-7) When it is determined that at least one person obstacle exists among the obstacles, for each person obstacle, according to the fusion result, human posture estimation technology is used to obtain the human posture recognition result corresponding to the person obstacle.

4. The intelligent obstacle avoidance method according to claim 3, characterized in that: In sub-step (B-5), the obstacle determination result is generated based on the distance category of each of the obstacles and the human posture recognition result of each of the at least one person obstacle.

5. The intelligent obstacle avoidance method according to claim 2, characterized in that, Sub-step (B-2) includes the following sub-steps: (B-2-1) Based on the fusion result, target detection technology is used to obtain target detection results, which indicate the object box of each obstacle and the object category information of each obstacle; and (B-2-2) Based on the fusion result and the target detection result, the motion trajectory information of each obstacle is obtained using target tracking technology.

6. The intelligent obstacle avoidance method according to claim 2, characterized in that: The obstacle sensors are a three-dimensional optical radar, an ultrasonic sensor, a two-dimensional laser scanner, and the photography module. The vehicle sensor is an inertial measurement unit. In step (A), the obstacle sensing data are three-dimensional point cloud data obtained by the three-dimensional optical radar, distance data obtained by the ultrasonic sensor, two-dimensional point cloud data obtained by the two-dimensional laser scanner, and the image data.

7. The intelligent obstacle avoidance method according to claim 2, characterized in that, Between step (A) and step (C), the following steps are also included: (G) Based on the fusion results, update the three-dimensional environment model regarding the sensing range and the obstacles therein using simultaneous localization and mapping technology; (H) Based on the three-dimensional environment model, another path planning algorithm is used to obtain the global path.

8. An intelligent obstacle avoidance system, suitable for mobile vehicles, characterized in that, The intelligent obstacle avoidance system includes: Vehicle sensors are used to generate vehicle sensing data, which indicates the motion state of the moving vehicle; Multiple different obstacle sensors are used to generate multiple obstacle sensing data points corresponding to the obstacle sensors, each obstacle sensing data point indicating the positional relationship between multiple obstacles within the sensing range and the mobile vehicle, one of the obstacle sensors being a camera module, and one of the obstacle sensing data points being image data obtained by the camera module. and A computing device, connected to the vehicle sensor and the obstacle sensor, is used to receive vehicle sensing data transmitted by the vehicle sensor and obstacle sensing data transmitted by the obstacle sensor. Based on the obstacle sensing data and the vehicle sensing data, an obstacle determination result is obtained regarding the obstacle. Based on the obstacle determination result, it is determined whether the obstacle affects the overall path that the mobile vehicle intends to travel. When it is determined that an obstacle affects the overall path, a local obstacle avoidance strategy is generated using a behavior tree based on the obstacle determination result. The local obstacle avoidance strategy includes at least one of obstacle avoidance trajectory information and obstacle avoidance speed. Based on the local obstacle avoidance strategy, the global path is updated using a path planning algorithm; and Based on the local obstacle avoidance strategy and the updated global path, a motion control algorithm is used to generate obstacle avoidance movement commands for the mobile vehicle to avoid the obstacles.

9. The intelligent obstacle avoidance system according to claim 8, characterized in that: The computing device uses multi-sensor fusion technology to fuse the obstacle sensing data and the vehicle sensing data to obtain a fusion result. In the fusion result, the obstacle sensing data and the vehicle sensing data use the same coordinate system and are synchronized in time. The computing device, based on the fusion result and at least one previously obtained fusion result, uses at least target detection technology and target tracking technology to obtain fusion data for each obstacle. Each fusion data includes at least the motion trajectory information of the corresponding obstacle and the object category information of the corresponding obstacle. The computing device classifies each obstacle into either a dynamic obstacle or a static obstacle based on the fusion result and the fusion data of each obstacle. For each obstacle, the computing device obtains the distance category of the distance between the obstacle and the mobile vehicle based on the fusion result and the fusion data of the obstacle. The distance category is one of safe distance, warning distance, and danger distance. and The computing device generates the obstacle determination result based on the distance category of each of the obstacles.

10. The intelligent obstacle avoidance system according to claim 9, characterized in that: The computing device determines whether there is at least one person obstacle among the obstacles based on the object category information of the fused data of each obstacle, and each person obstacle is an obstacle that indicates the person category according to the corresponding object category information; and When it is determined that at least one person obstacle exists among the obstacles, for each person obstacle, the computing device uses human posture estimation technology based on the fusion result to obtain the human posture recognition result corresponding to the person obstacle.

11. The intelligent obstacle avoidance system according to claim 10, characterized in that: The computing device generates the obstacle determination result based on the distance category of each of the obstacles and the human posture recognition result of each of the at least one person obstacle.

12. The intelligent obstacle avoidance system according to claim 9, characterized in that: The computing device obtains target detection results using target detection technology based on the fusion results. The target detection results indicate the object bounding box of each obstacle and the object category information of each obstacle. and The computing device obtains the motion trajectory information of each obstacle using target tracking technology based on the fusion result and the target detection result.

13. The intelligent obstacle avoidance system according to claim 9, characterized in that: The obstacle sensors are a 3D optical radar, an ultrasonic sensor, a 2D laser scanner, and the photography module. The vehicle sensor is an inertial measurement unit. The obstacle sensing data are 3D point cloud data obtained by the 3D optical radar, distance data obtained by the ultrasonic sensor, 2D point cloud data obtained by the 2D laser scanner, and the image data.

14. The intelligent obstacle avoidance system according to claim 9, characterized in that: The computing device, based on the fusion result, uses simultaneous localization and mapping (SMR) technology to update the three-dimensional environment model regarding the sensing range and the obstacles within it; and The computing device obtains the global path based on the three-dimensional environment model using another path planning algorithm.

Images