Obstacle avoidance system and obstacle avoidance method for unmanned system

Abstract

The application provides an obstacle avoidance system and method for an unmanned system, the obstacle avoidance system comprising: a visual perception module, an obstacle avoidance decision module and a motion control module, the visual perception module being used for detecting the category, pixel position and depth of obstacles for the unmanned system, performing fusion calculation, and establishing a non-uniform equivalent depth grid to represent the obstacle state; the obstacle avoidance decision module being used for projecting the geometric shape of the unmanned system to an image plane to form a core area, constructing a repulsive force field to calculate the resultant repulsive force of the core area, and providing an obstacle avoidance decision according to the resultant repulsive force; and the motion control module being used for warning potential dangerous obstacles, and executing an obstacle avoidance action according to the motion model of the unmanned system and the obstacle avoidance decision. The application has the advantages of low cost, light weight and low delay, can detect obstacles such as pedestrians and vehicles in real time, and can guide the control of the unmanned system to effectively and reliably avoid obstacles.

Description

Technical Field

[0001] This invention relates to the field of obstacle avoidance technology for unmanned systems, and more particularly to an obstacle avoidance system and method for unmanned systems. Background Technology

[0002] Typically, intelligent unmanned systems are equipped with various sensors to acquire data about their surrounding environment. They need to use obstacle avoidance algorithms to obtain real-time status information of obstacles, which is then transmitted to the decision control system for path planning and real-time obstacle avoidance. Therefore, effective obstacle avoidance strategies ensure the safety of unmanned systems during their operation and are of great research significance.

[0003] Some obstacle avoidance algorithms require the use of active sensors such as lidar, structured light, or sonar to obtain accurate obstacle depth. However, such sensors are expensive, heavy, have a small ranging range, and consume a lot of power, making it difficult to meet the application requirements of real-world scenarios.

[0004] In recent years, with the development of deep learning theory and hardware, image depth features adaptively extracted using deep neural networks have been proven to significantly outperform manually designed features such as SIFT, HOG, and LBP. They have achieved superior performance in visual tasks such as object detection, segmentation, and depth prediction, and have also made some progress in obstacle avoidance design for unmanned systems. Existing image-based obstacle avoidance methods typically combine the results from object detection and semantic segmentation algorithms to find passable regions and achieve obstacle avoidance.

[0005] Existing traditional obstacle avoidance methods can be divided into two categories: global path planning and local collision avoidance. The former assumes that global obstacle information is known and then plans a reachable path from the starting point to the target point, such as heuristic algorithms like genetics, intelligent swarms, and A* / D*, which are suitable for avoiding static obstacles. The latter does not rely on global information and is a class of real-time, local collision avoidance algorithms. Common methods include guidance rate-based, velocity-based obstacle avoidance, and artificial potential field-based methods.

[0006] While these methods can accomplish obstacle avoidance tasks in specific scenarios, their stability and applicability are easily affected by the environment, exhibiting poor robustness and complex computational processes. Image-based deep learning-based obstacle avoidance methods do not consider the geometry of the unmanned system, limiting their practicality. Deep reinforcement learning-based obstacle avoidance methods, such as deep Q-learning networks, are based on a reward and punishment mechanism, learning obstacle avoidance decisions based on feedback from issued action commands. This gradually endows unmanned systems with global planning and path optimization capabilities without relying on human experience or prior rules. However, a reasonable reward function needs to be designed based on the actual task and sensor data. Limited by sample size and learning efficiency, most current research focuses on simulation environments, and further research is needed to extend its application to real-world environments. Summary of the Invention

[0007] The technical problem to be solved by this invention is how to achieve reliable obstacle avoidance in unmanned systems. This invention proposes an obstacle avoidance system and method for unmanned systems.

[0008] An obstacle avoidance system for an unmanned system according to an embodiment of the present invention includes:

[0009] The visual perception module is used to detect the type, pixel position, and depth of obstacles for unmanned systems, and to perform fusion calculations to establish a non-uniform equivalent depth grid.

[0010] The obstacle avoidance decision module is used to project the geometry of the unmanned system onto the image plane to form a core area, construct a repulsive field to calculate the resultant repulsive force on the core area, and provide obstacle avoidance decisions based on the resultant repulsive force.

[0011] The motion control module is responsible for providing early warnings of potential hazards and, based on the unmanned system's motion model, executing obstacle avoidance actions according to obstacle avoidance decisions.

[0012] The obstacle avoidance system for unmanned systems according to embodiments of the present invention has low component module cost and enables unmanned systems to effectively and reliably achieve obstacle avoidance.

[0013] According to some embodiments of the present invention, the visual perception module includes: an object detection network and a depth estimation network.

[0014] In some embodiments of the present invention, the visual perception module uses the YOLOv5s object detection network to detect the category and pixel position of obstacles in a monocular image, and the visual perception module uses an open-source monocular depth estimation tool to predict the depth of obstacles.

[0015] According to some embodiments of the present invention, the monocular depth estimation open-source tool is used to perform transfer learning on a preset dataset, enabling the visual perception module to predict the depth of obstacles based on monocular images.

[0016] In some embodiments of the present invention, the unmanned system includes an unmanned vehicle, and the obstacles include pedestrians, vehicles, and traffic lights.

[0017] According to an embodiment of the present invention, an obstacle avoidance method for an unmanned system employs the obstacle avoidance system for unmanned systems described above to achieve obstacle avoidance for the unmanned system. The obstacle avoidance method includes:

[0018] S10 detects the type, pixel location, and depth of obstacles, and performs fusion calculations to obtain a non-uniform equivalent depth mesh;

[0019] S20 projects the driverless vehicle into the imaging space to establish the core area;

[0020] S30, construct a repulsive field function with respect to the obstacle depth and the area of ​​the core area, and calculate the magnitude and direction of the repulsive force exerted by the obstacle on the center of the core area;

[0021] S40, calculate the resultant repulsive force of all obstacles on the core area, and determine the obstacle avoidance decision based on the resultant repulsive force;

[0022] S50, based on the motion model of the unmanned system and combined with the resultant force of repulsion, uses an algorithm to provide yaw angle and linear velocity to guide the unmanned system in dynamic obstacle avoidance.

[0023] The obstacle avoidance method for unmanned systems according to embodiments of the present invention has the advantages of low cost, lightweight design, and low latency. This invention integrates two visual perception networks—target detection and monocular depth estimation—to determine the category, pixel position, and distance of obstacles. Based on the pinhole imaging principle, it rigorously derives the constraint relationship between three-dimensional space and the imaging plane space. Based on this, it improves upon the traditional artificial potential field method by establishing a repulsive force field on the image plane. Finally, it comprehensively analyzes the perception results to guide the unmanned vehicle in completing obstacle avoidance tasks against vehicles, pedestrians, and other targets.

[0024] According to some embodiments of the present invention, step S10 specifically includes:

[0025] S11, using the Yolov5s object detection network to detect the category and pixel location of obstacles in monocular images;

[0026] S12 uses a depth estimation network to predict pixel-level monocular dense depth maps;

[0027] S13, a non-uniform grid is established in the image with the detection box of the obstacle as the boundary, and the equivalent depth of the obstacle is determined in each network region to achieve the fusion of obstacle position and depth information.

[0028] In some embodiments of the present invention, in step S20, a projection formula is derived based on the camera pinhole imaging model, and the geometry of the unmanned system is projected onto the imaging space using the projection formula.

[0029] According to some embodiments of the present invention, in step S40, when the magnitude of the repulsive resultant force is not zero and the direction is horizontal to the left, the unmanned system is controlled to turn to the left to avoid the obstacle.

[0030] When the magnitude of the repulsive net force is not zero and its direction is horizontal to the right, the unmanned system is controlled to turn right to avoid the obstacle.

[0031] When the magnitude of the repulsive net force is zero, the area s of the grid with an equivalent depth less than the safe distance within the core area is determined. If s is greater than zero, the unmanned system applies braking; if s is equal to zero, the unmanned system maintains its current motion state.

[0032] According to some embodiments of the present invention, in step S50, the yaw angle can be calculated from the maximum horizontal distance between the obstacle in the core area and the boundary of the core area, and the linear velocity is determined by the braking force and the horizontal repulsive force. The unmanned system is then returned to the globally planned route after the obstacle avoidance is released.

[0033] In some embodiments of the present invention, the obstacle avoidance method is used for real-time dynamic obstacle avoidance of the unmanned system. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of an artificial potential field according to an embodiment of the present invention;

[0035] Figure 2 This is an overall framework diagram of an obstacle avoidance method for an unmanned system according to an embodiment of the present invention;

[0036] Figure 3 This is a structural diagram of YOLOv5s according to an embodiment of the present invention;

[0037] Figure 4 This is a schematic diagram illustrating the safe distance control of the obstacle avoidance algorithm according to an embodiment of the present invention;

[0038] Figure 5 This is a schematic diagram of the non-uniform equivalent grid and core region according to an embodiment of the present invention;

[0039] Figure 6 This is a schematic diagram of a vehicle steering according to an embodiment of the present invention;

[0040] Figure 7 This is a schematic diagram of the original monocular image according to an embodiment of the present invention;

[0041] Figure 8 This is a schematic diagram of the detection results of Yolov5s according to an embodiment of the present invention;

[0042] Figure 9 This is a schematic diagram of monocular depth estimation results according to an embodiment of the present invention;

[0043] Figure 10 This is a schematic diagram of filtering distant targets according to an embodiment of the present invention;

[0044] Figure 11 This is a schematic diagram of the core region and equivalent depth grid according to an embodiment of the present invention. Detailed Implementation

[0045] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the present invention will be described in detail below with reference to the accompanying drawings and preferred embodiments.

[0046] The steps described in the specification and the flowcharts in the accompanying drawings of this invention are not necessarily to be strictly followed according to the step numbers; the execution order of the steps can be changed. Furthermore, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be broken down into multiple steps.

[0047] To address the limitations of existing image-based obstacle avoidance algorithms, such as those for estimating the geometric dimensions of unmanned platforms, this invention studies a dynamic obstacle avoidance method that integrates deep neural networks and improved artificial potential fields. Based on monocular vision, the deep neural network and the improved artificial potential field are fused to design an obstacle avoidance system and a real-time dynamic obstacle avoidance method suitable for intelligent unmanned systems.

[0048] like Figure 2 As shown, the obstacle avoidance system for unmanned systems according to an embodiment of the present invention includes: a visual perception module, an obstacle avoidance decision module, and a motion control module.

[0049] Among them, the visual perception module is used to detect the type, pixel position and depth of obstacles for the unmanned system, and to perform fusion calculations to establish a non-uniform equivalent depth grid;

[0050] The obstacle avoidance decision module projects the geometry of the unmanned system onto an image plane to form a core region, constructs a repulsive field to calculate the resultant repulsive force on the core region, and provides obstacle avoidance decisions based on the resultant repulsive force. For example, obstacle avoidance decisions may include guiding the unmanned system to release brakes or steer.

[0051] The motion control module is responsible for providing early warnings of potential hazards and, based on the motion model of the unmanned system, executing obstacle avoidance actions according to obstacle avoidance decisions.

[0052] The obstacle avoidance system for unmanned systems according to embodiments of the present invention has low component module cost and enables unmanned systems to effectively and reliably achieve obstacle avoidance.

[0053] According to some embodiments of the present invention, the visual perception module includes: an object detection network and a depth estimation network.

[0054] In some embodiments of the present invention, the visual perception module employs a lightweight YOLOv5s object detection network to detect the category and pixel position of obstacles in monocular images, and the visual perception module uses an open-source monocular depth estimation tool to predict the depth of obstacles.

[0055] It should be noted that monocular cameras are low in cost and flexible in deployment. By combining them with deep neural networks to design obstacle avoidance algorithms, the cost of unmanned systems can be reduced to a great extent, while improving the system's perception capabilities.

[0056] This invention is an improvement upon the traditional artificial potential field method, as follows: Figure 1 As shown, the obstacle, the autonomous vehicle, and the target point are all located in the plane of the direction of travel. The target point exerts an attractive force on the vehicle, always pointing towards the target point, and its magnitude is directly proportional to the distance. The obstacle, within a certain range, exerts a repulsive force pointing towards the vehicle, and its magnitude is inversely proportional to the distance. The artificial potential field method calculates the magnitude and direction of acceleration based on the resultant force of the repulsive and attractive forces on the vehicle, guiding the vehicle to complete obstacle avoidance.

[0057] The target point generates a gravitational potential field on the car:

[0058]

[0059] In equation (1), η is the gravitational gain coefficient; ρ(q, q g The magnitude is the distance between the car and the target point, and the direction is towards the target point. The gravitational function is obtained by taking the negative gradient with respect to the potential field.

[0060]

[0061] Figure 1 A repulsive field is generated at the obstacle point:

[0062]

[0063] In equation (3), k is the repulsive force coefficient, P(q, q0) represents the distance between the car and the obstacle, and its direction is towards the car. ρ0 is a constant representing the maximum effective range of the obstacle. Therefore, the corresponding repulsive force function is:

[0064]

[0065] This method, based on the YOLO-v5s algorithm and a lightweight depth estimation network, designs a local real-time collision avoidance algorithm applicable to both static and dynamic obstacles. The main idea is to project the position and depth information of surrounding obstacles, along with the dimensions of the unmanned system itself, into the image space, and establish a repulsive potential field of the obstacles within the image plane to guide the unmanned system in real-time obstacle avoidance. Compared to traditional artificial potential fields, this model is more consistent with human vision.

[0066] According to some embodiments of the present invention, a monocular depth estimation open-source tool is used to perform transfer learning on a preset dataset, enabling the visual perception module to predict the depth of obstacles based on monocular images.

[0067] In some embodiments of the present invention, the unmanned system includes an unmanned vehicle, and obstacles include pedestrians, vehicles, and traffic lights. It should be noted that the detected obstacles depend on the data used by the target detection network. The present invention can be trained on large public datasets such as COCO and VOC, and is capable of detecting dozens of targets including people, vehicles, public facilities, and animals.

[0068] According to an embodiment of the present invention, an obstacle avoidance method for an unmanned system employs the obstacle avoidance system described above for unmanned systems to achieve obstacle avoidance for the unmanned system. The obstacle avoidance method includes:

[0069] S10 detects the type, pixel location, and depth of obstacles, performs fusion calculations, and establishes a non-uniform equivalent depth mesh;

[0070] S20 projects the driverless vehicle into the imaging space to establish the core area;

[0071] S30, construct a repulsive field function with respect to the obstacle depth and the area of ​​the core area, and calculate the magnitude and direction of the repulsive force exerted by the obstacle on the center of the core area;

[0072] S40, calculate the resultant repulsive force of all obstacles on the core area, and determine the obstacle avoidance decision based on the resultant repulsive force;

[0073] S50, based on the motion model of the unmanned system and combined with the repulsive and resultant forces, determines the best course of action (such as braking, turning left or right) for the next moment, and provides the yaw angle and linear velocity by the algorithm to guide the unmanned system to perform dynamic obstacle avoidance.

[0074] The obstacle avoidance method for unmanned systems according to embodiments of the present invention has the advantages of low cost, lightweight design, and low latency. This invention integrates two visual perception networks—target detection and monocular depth estimation—to determine the position and distance of obstacles. Based on the pinhole imaging principle, it rigorously derives the constraint relationship between three-dimensional space and the imaging plane space. Based on this, it improves the traditional artificial potential field method, establishing a repulsive force field on the image plane. Finally, it comprehensively analyzes the perception results to guide the unmanned vehicle in completing obstacle avoidance tasks for typical targets such as vehicles and pedestrians.

[0075] According to some embodiments of the present invention, step S10 specifically includes:

[0076] S11, using the Yolov5s object detection network to detect the category and pixel location of obstacles in monocular images;

[0077] S12 uses a lightweight depth estimation network to predict pixel-level monocular dense depth maps;

[0078] S13, a non-uniform grid is established in the image with the detection box of the obstacle as the boundary, and the equivalent depth of the obstacle is determined in each network region to achieve the fusion of obstacle position and depth information.

[0079] In some embodiments of the present invention, in step S20, a projection formula is derived based on the camera pinhole imaging model, and the geometry of the unmanned system is projected onto the imaging space using the projection formula.

[0080] According to some embodiments of the present invention, in step S40, when the magnitude of the repulsive resultant force is not zero and the direction is horizontal to the left, the unmanned system should turn to the left to avoid the obstacle.

[0081] When the magnitude of the repulsive net force is not zero and its direction is horizontal to the right, the unmanned system should turn right to avoid the obstacle.

[0082] When the magnitude of the repulsive net force is zero, the area s of the grid with an equivalent depth less than the safe distance within the core area is determined. If s is greater than zero, the unmanned system applies braking; if s is equal to zero, the unmanned system maintains its current motion state.

[0083] According to some embodiments of the present invention, in step S50, the yaw angle can be calculated from the maximum horizontal distance between the obstacle in the core area and the boundary of the core area, and the linear velocity is determined by the braking force and the horizontal repulsive force. The unmanned system is then returned to the globally planned route after the obstacle avoidance is released.

[0084] In some embodiments of the present invention, the obstacle avoidance method is used for real-time dynamic obstacle avoidance in unmanned systems.

[0085] The obstacle avoidance system and method for unmanned systems according to the present invention will now be described in detail with reference to the accompanying drawings and a specific embodiment. It should be noted that the following description is merely exemplary and should not be construed as a specific limitation of the present invention.

[0086] This invention employs a monocular camera-based obstacle avoidance scheme, combined with an image-based deep neural network to achieve environmental perception and obstacle avoidance. The technical challenge is how to rigorously unify the geometry of the unmanned platform with the visual perception results within the same reference frame, and determine its positional relationship with obstacles, thereby achieving reliable obstacle avoidance with low-cost investment. This invention proposes an obstacle avoidance system and method for unmanned systems, with the following steps:

[0087] Step 1: Build a YOLO-v5s object detection network to detect obstacle targets;

[0088] YOLOv5 (You Only Look Once) is a high-performance single-stage object detection algorithm. Its releases include a series of algorithms such as YOLOv5s, YOLOv5m, YOLOv5l, and YOLOv5x. It supports balancing accuracy and speed according to different tasks, thereby selecting the most suitable method.

[0089] like Figure 3 Considering the real-time requirements of obstacle avoidance algorithms, this method selects Yolov5s, which has a small model structure and fast inference speed, to build a detection network. It is trained on large datasets such as COCO and VOC to achieve real-time detection of pedestrians and vehicles in driving environments.

[0090] Step 2: Build a monocular depth estimation network to predict the depth and distance of obstacles;

[0091] This method uses the open-source Monocular Depth Estimation Toolbox (MDET) to predict the depth of obstacles. MDET is developed based on the PyTorch framework and provides effective support for monocular depth estimation and visualization of calculation results.

[0092] Currently, MDET offers users five monocular depth estimation methods: BTS, Adamins, SimIPU, DPT, DepthFormer, and BinsFormer. These methods support training and testing on four large-scale depth datasets: KITTI, NYU-Depth-V2, SUN RGB-D, and Cityscapes. Furthermore, it has released various backbone network pre-trained model libraries, such as ResNet, VisionTransformer, and the lightweight network EfficientNet, for users to choose from. Both DepthFormer and BinsFormer networks utilize advanced Transformer technology, resulting in good depth prediction accuracy. Their encoders can be further replaced with lightweight structures for faster inference speeds.

[0093] By utilizing the MDET open-source tool to perform transfer learning on the KITTI and NYU-Depth-V2 datasets, the unmanned system can perform depth prediction of typical obstacles in indoor and outdoor scenes, such as pedestrians and vehicles, using only a monocular camera, providing key target distance information for obstacle avoidance.

[0094] Step 3: Establish a non-uniform equivalent depth grid to fuse target location and depth information;

[0095] This method uses the four coordinates (x, y, x) of the obstacle corner points detected by Yolov5s. min x max y min and y max To define the boundaries, a non-uniform grid is constructed to determine the optimal partitioning method and distinguish obstacle regions. A monocular depth estimation network predicts pixel-level dense depths, and different depth sets D are determined by the grid boundaries. depth ={D1,D2,…,D n}

[0096] Since obstacle avoidance algorithms are concerned with the shortest distance to obstacles, the minimum value of each set in the region where the target is located is taken as the equivalent depth, while the average depth in regions without the target is taken as the equivalent depth, thus obtaining the equivalent depth set.

[0097] Step 4: Project the core area of ​​the driverless vehicle;

[0098] This method introduces this idea into image plane space, establishing a virtual potential field with respect to depth and area for the non-uniform grid. This generates a horizontal repulsive force acting on the vehicle, achieving the goal of pushing the grid containing the obstacle target out of the core area. As shown in the figure, first, assume a safety distance of d. safe This refers to the minimum distance at which the autonomous vehicle takes evasive action. Now, we only need to project the actual size of the autonomous vehicle at this distance when it collides, which can be divided into... Figure 4 There are three cases: (a), (b), and (c).

[0099] (a): Reaching d safe At the moment of collision, the gravity lines located at the lower boundary of the camera's ray cone, the ground, and the front of the vehicle intersect at the same point;

[0100] (b): Reaching d safe At the point of collision, the intersection of the lower boundary disappearance point of the view cone and the ground is farther than d. safe Place;

[0101] (c): Reaching d safe At the time of the collision, the intersection of the lower boundary disappearance point of the view cone and the ground is close to d. safe Place.

[0102] For case (b), the corresponding d of the core area height safe The actual height is H′ car <H car This method cannot guarantee that targets outside the core area will not collide with the vehicle, rendering it ineffective. Therefore, d should be adjusted during the overall system design phase. safe This situation is avoided by adjusting the camera's longitudinal field of view (VFOV). Now consider a more general case (c), assuming the camera's equivalent focal length is f. To determine the height and width of the core region in the image, we can obtain the following based on the similarity triangulation relationship of the camera pinhole imaging model:

[0103]

[0104]

[0105] In equations (5) and (6), h pixel w pixel These are the height and width of the core area, in pixels; H car W car This is the actual height of the car; d safe For safe distance; f is the camera focal length; S V S HThese represent the actual lengths of each pixel at point f (within the image plane) in the vertical and horizontal directions, respectively. Without losing sight of the influence of the field of view angle, we have:

[0106]

[0107]

[0108] In equations (7) and (8), α VFOV β HFOV H represents the vertical field of view and the horizontal field of view, respectively; img W img These are the image height and width, in pixels. Substituting them into formulas (5) and (6), we get:

[0109]

[0110]

[0111] The bottom edge of the core area may have a certain distance from the bottom edge of the image. In case a, this height should be 0; in case c, the height is assumed to be l. pixel The unit is pixels. Here, we first need to determine the value of each pixel in both the vertical and horizontal directions (d). safe The actual length at the point can be obtained by simply replacing f with d in formulas (7) and (8). safe That's it. pixel It can be written as:

[0112]

[0113] Non-uniform equivalent grid and the projection of the car core area, such as Figure 5 As shown, the core area can be determined by formulas (9), (10), and (11).

[0114] Step 5: Construct a repulsive field in the pixel plane;

[0115] This method introduces this idea into the image space, and establishes a virtual repulsive potential field in the imaging plane based on the obstacle depth and pixel area, so that the target entering the core area generates a horizontal repulsive force on the car within a certain distance range, and the direction of vehicle movement is planned under the action of the resultant force.

[0116] To simplify the conditions and disregard the influence of the target point, the distance element in formula (10) is replaced by the definite integral of depth in the image plane, V(x, y) = ∫∫d(x, y)dxdy, with the direction horizontally to the left or right. d(x, y) represents the depth density in the image plane, which is replaced by an equivalent depth and can be written as: Then we have:

[0117]

[0118] In formula (12) Let be the minimum depth of the grid region, and s be the intersection-to-unity ratio (IOU) between the grid and the core region. This determines the magnitude of the repulsive force. If the repulsive force is to the left of the core region's center, it is horizontal to the right, and vice versa. When the equivalent depth of the grid enters d... safe If the range is within 0 and s≥0, the driverless car will take obstacle avoidance measures.

[0119] Step 6: Develop obstacle avoidance decisions based on the resultant force of repulsive forces;

[0120] The autonomous vehicle should take different obstacle avoidance measures based on the force conditions in the core area, mainly including three scenarios:

[0121] a. The first type is when the resultant force of the horizontal repulsive force is not zero and its direction is to the left. In this case, the unmanned vehicle should be controlled to turn left to avoid the obstacle.

[0122] b. The second type is when the resultant force of the horizontal repulsive force is not zero and its direction is to the right. In this case, the unmanned vehicle should be controlled to turn right to avoid the obstacle.

[0123] c. The third scenario is that the resultant force of the horizontal repulsion is zero. In this case, the depth of the core region should be less than d. safe The grid area s is given. If s is greater than zero, the autonomous vehicle should brake. If s is equal to zero, the autonomous vehicle should maintain its current motion.

[0124] Step 7: Obstacle avoidance motion control;

[0125] This step uses an algorithm to provide real-time yaw angle and linear velocity guidance for the autonomous vehicle to avoid obstacles. The autonomous vehicle chassis has a stable control system that can control the yaw angle and rate based on decision commands. The motion model is as follows: Figure 6 As shown. When taking obstacle avoidance actions, the obstacle avoidance algorithm needs to calculate the yaw angle θ and the speed of the vehicle in real time.

[0126] The autonomous vehicle's steering direction is consistent with the resultant force direction. After determining the steering direction, the yaw angle θ can be calculated from the maximum distance ΔL between obstacles within the core area and the edge of the core area.

[0127]

[0128] The linear velocity is determined by both braking force and horizontal repulsion force. The obstacle avoidance algorithm continues until the obstacle disappears from view, at which point it reverts to the globally planned flight path. The main logic of the obstacle avoidance algorithm is as follows:

[0129]

[0130]

[0131] like Figure 7As shown, typical images from real-world scenarios were selected for testing, and the results of the Yolov5s object detection algorithm are as follows. Figure 8 As shown, this algorithm boasts advantages such as lightweight design and low latency, meeting the real-time detection requirements for targets in scenarios such as pedestrians and vehicles. Pixel-level depth is predicted using open-source depth estimation tools, and the results are as follows: Figure 9 As shown. Targets at greater distances are filtered to reduce computational load; the results are shown below. Figure 10 The position and depth information of obstacles are fused using a non-uniform equivalent grid, as illustrated in the diagram. Figure 11 As shown, the desired yaw angle and speed are then given to achieve obstacle avoidance.

[0132] Through the description of specific embodiments, a more in-depth and specific understanding should be gained of the technical means and effects adopted by the present invention to achieve the intended purpose. However, the accompanying drawings are only provided for reference and illustration and are not intended to limit the present invention.

Claims

1. An obstacle avoidance system for unmanned systems, characterized in that, include: The visual perception module is used to detect the category, pixel position, and depth of obstacles for the unmanned system, and to perform fusion calculations to establish a non-uniform equivalent depth grid. The visual perception module uses a Yolov5s object detection network to detect the category and pixel position of obstacles in a monocular image, uses a depth estimation network to predict a pixel-level monocular dense depth map, establishes a non-uniform grid in the image with the detection box of the obstacle as the boundary, and determines the equivalent depth of the obstacle in each network region, thereby realizing the fusion of obstacle position and depth information. The obstacle avoidance decision module is used to project the geometry of the unmanned system onto the image plane to form a core area, construct a repulsive field to calculate the resultant repulsive force on the core area, and provide obstacle avoidance decisions based on the resultant repulsive force. The motion control module is responsible for providing early warnings of potential hazards and, based on the unmanned system's motion model, executing obstacle avoidance actions according to obstacle avoidance decisions.

2. The obstacle avoidance system for unmanned systems according to claim 1, characterized in that, The visual perception module includes an object detection network and a depth estimation network.

3. The obstacle avoidance system for unmanned systems according to claim 1, characterized in that, By using an open-source monocular depth estimation tool to perform transfer learning on a preset dataset, the visual perception module can predict the depth of obstacles based on monocular images.

4. The obstacle avoidance system for unmanned systems according to any one of claims 1-3, characterized in that, The unmanned system includes unmanned vehicles, and the obstacles include pedestrians, vehicles, and traffic lights.

5. An obstacle avoidance method for an unmanned system, characterized in that, The obstacle avoidance method employs the obstacle avoidance system for unmanned systems as described in any one of claims 1-4 to achieve obstacle avoidance for the unmanned system, and the obstacle avoidance method includes: S10 detects the type, pixel location, and depth of obstacles, and performs fusion calculations to obtain a non-uniform equivalent depth mesh; S20 projects the driverless vehicle into the imaging space to establish the core area; S30, construct a repulsive field function with respect to the obstacle depth and the area of ​​the core area, and calculate the magnitude and direction of the repulsive force exerted by the obstacle on the center of the core area; S40, calculate the resultant repulsive force of all obstacles on the core area, and determine the obstacle avoidance decision based on the resultant repulsive force; S50, based on the motion model of the unmanned system and combined with the resultant force of repulsion, uses an algorithm to give the yaw angle and linear velocity to guide the unmanned system to perform dynamic obstacle avoidance; Step S10 specifically includes: S11, using the Yolov5s object detection network to detect the category and pixel location of obstacles in monocular images; S12 uses a depth estimation network to predict pixel-level monocular dense depth maps; S13, a non-uniform grid is established in the image with the detection box of the obstacle as the boundary, and the equivalent depth of the obstacle is determined in each network region to achieve the fusion of obstacle position and depth information.

6. The obstacle avoidance method for an unmanned system according to claim 5, characterized in that, In step S20, the projection formula is derived based on the camera pinhole imaging model, and the geometry of the unmanned system is projected onto the imaging space using the projection formula.

7. The obstacle avoidance method for an unmanned system according to claim 5, characterized in that, In step S40, when the magnitude of the repulsive net force is not zero and the direction is horizontal to the left, the unmanned system is controlled to turn left to avoid the obstacle. When the magnitude of the repulsive net force is not zero and its direction is horizontal to the right, the unmanned system is controlled to turn right to avoid the obstacle. When the magnitude of the repulsive net force is zero, the area s of the grid with an equivalent depth less than the safe distance in the core area is determined. If s is greater than zero, the unmanned system applies braking; if s is equal to zero, the unmanned system maintains its current motion state. In step S50, the yaw angle is calculated from the maximum horizontal distance between the obstacle in the core area and the boundary of the core area, and the linear velocity is determined by the braking force and the horizontal repulsive force. After the obstacle avoidance is released, the unmanned system is made to return to the globally planned route.

8. The obstacle avoidance method for an unmanned system according to any one of claims 5-7, characterized in that, The obstacle avoidance method is used for real-time dynamic obstacle avoidance in the unmanned system.

Images