Binocular vision navigation method for unmanned vehicles based on improved artificial potential field method

Abstract

The invention belongs to the field of artificial intelligence technology, and specifically relates to a binocular vision navigation method for an unmanned vehicle based on an improved artificial potential field method. Firstly, a semi-global stereo matching algorithm in binocular vision is used to realize real-time perception of depth information of a scene, an FPGA implementation framework of the semi-global stereo matching algorithm based on block comparison is designed, the semi-global stereo matching algorithm is modularized, parallel operation is adopted within the module, and pipeline operation is adopted between modules to accelerate it, thereby reducing resource usage and improving the real-time performance of the stereo matching algorithm. Then, binarization, median filtering, connected domain detection and other operations are performed on the current scene, and the obstacle distance of the current scene is output in combination with a disparity map. Finally, in view of the local minimum value problem in the traditional artificial potential field method, the invention effectively solves the local minimum value problem by autonomously establishing a virtual target point, and realizes the path planning of the unmanned vehicle. The method of the invention is verified on a built unmanned vehicle system, and experimental results show the feasibility and superiority of the method of the invention.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a binocular vision navigation method for an unmanned vehicle based on an improved artificial potential field method, in particular to a low-cost real-time path planning method for an unmanned vehicle in an unknown environment. Background Art

[0002] With the continuous development of science and technology, artificial intelligence has ushered in the fourth wave; the intelligent unmanned vehicle system is an autonomous mobile vehicle that integrates functions such as obstacle perception, path planning and automatic control. It is widely used in traffic safety, manufacturing industry, social life, national defense and military fields; among them, real-time perception of obstacles and path planning are the key technologies of the unmanned vehicle system.

[0003] Binocular stereo vision has the advantages of high accuracy, rich information, low power consumption and high cost performance. It is often combined with path planning algorithms for navigation and obstacle avoidance of unmanned equipment; Jiang Shelu et al. [1] The semi-global stereo matching algorithm (SGM) and A* algorithm were used to achieve vehicle navigation in a known environment. The large amount of computation of the SGM algorithm resulted in the system taking 10 seconds to process each image, and the system had poor real-time performance. Sourish et al. [2] The Sum of Absolute Differences (SAD) matching algorithm and Rapidly-exploring Random Trees (RRT) were used to achieve joint perception and path planning for effective obstacle avoidance of unmanned ground vehicles; although unnecessary full 3D reconstruction was avoided, it was relatively blind and inefficient, and was not suitable for dynamic path planning scenarios with high real-time requirements; Radouane et al. [3] proposed an obstacle avoidance method for autonomous quadrotor micro-aircraft based on simultaneous localization and mapping (SLAM) and RRT* algorithm. This method solves the map creation and safe trajectory planning of unknown environments, but it needs to run dense stereo matching on the central processor to estimate the disparity map to obtain the corresponding depth information, which has the disadvantages of huge computational complexity, poor real-time performance and high cost; Akhil et al. [4] Directly use the depth camera to obtain the scene disparity map, and then use the K-means algorithm to complete the obstacle area division and the A* algorithm to complete the robot's path planning; however, this method requires global information and is not suitable for the movement of unmanned equipment in unknown environments; Zhang Chao et al. [5]Based on the disparity map output by the SAD algorithm, the Bezier curve is introduced to generate the obstacle avoidance path, and the drone navigation based on stereo vision is realized. The speed of this method on the NVIDIA TX1 platform is 10fps, and the real-time performance is poor. At the same time, in order to meet the real-time performance of path planning, a local stereo matching algorithm is used, and the matching effect is poor; Zheng Yixiao et al. [6] The SAD algorithm and improved A* algorithm are also used to complete the environmental perception and path planning of the mobile robot respectively. Although reference [6] can complete the obstacle detection task and provide a better path, it still has the problems of being unable to use the unknown environment in reference [4] and reference [5] and the poor matching effect of the local stereo matching algorithm.

[0004] In unknown environments, Artificial Potential Field (APF) [7] It is one of the commonly used path planning algorithms. It has been widely used because of its simple principle, easy implementation and good real-time performance. It meets the real-time dynamic planning requirements of the system. However, the algorithm is limited by mechanical factors and has the problems of unreachable targets and local minima. [8] ; For the problem of unreachable target, the distance between the current point and the target point can be introduced into the repulsive field function to make the target point the global minimum potential energy point [9] ; For the local minimum problem, Zhao Liangbo et al.

[10] Introducing the rotation matrix into the gravitational field function overcomes the local minimum problem to a certain extent, but the resulting path has a local oscillation problem; Xiao Ma et al.

[11] By adding a random perturbation potential energy adjustment mechanism to fill the potential energy depression of the local minimum point, the local minimum trap can be escaped; this method solves the local minimum problem to a certain extent, but the addition of perturbation potential energy is random, which makes the algorithm result uncertain; Lantian Shangguan et al.

[12] Combining the APF algorithm with fuzzy logic control overcomes the defects of the APF algorithm, but increases the amount of algorithm calculation.

[0005] The following are the references given by the applicant:

[0006] [1]Jiang S L,Hong Z H,Zhang Y,et al.Automatic path planning andnavigation with stereo cameras[C] / / 2014Third International Workshop on EarthObservation and Remote Sensing Applications.IEEE,2014:289-293.

[0007] [2]Ghosh S,Biswas J.Joint perception and planning for efficientobstacle avoidance using stereo vision[C] / / 2017IEEE / RSJ InternationalConference on Intelligent Robots and Systems.IEEE,2017:1026-1031.

[0008] [3]Ait-Jellal R,Zell A.Outdoor obstacle avoidance based on hybridvisual stereo SLAM for an autonomous quadrotor MAV[C] / / 2017 EuropeanConference on Mobile Robots.IEEE,2017:1-8.

[0009] [4]Shetty A A,George V I,Nayak C G,et al.Disparity maps based pathplanning algorithm for autonomous robot navigation[C] / / 2019 3rd Internationalconference on Electronics,Communication and Aerospace Technology.IEEE,2019:669-675.

[0010] [5]Zhang C, He T, Zhan QZ, et al.Visual Navigation Based on StereoCamera for Water Conservancy UAVs[C] / / 2019 9th International Conference onInformation Science and Technology.IEEE,2019:158-183.

[0011] [6] Zheng Y,

[0012] [7] Mancini M, Bloise N, Capello E, et al. Sliding mode control techniques and artificial potential field for dynamic collision avoidance in rendezvousmaneuvers[J]. IEEE Control Systems Letters, 2020, 4(2): 313-318.

[0013] [8] Han Yao, Li Shaohua. UAV trajectory planning based on improved artificial potential field method[J]. Systems Engineering and Electronics, 2021, 43(11): 3305-3311.

[0014] [9] Xiong Chao, Xie Wujie, Dong Wenhan. Obstacle avoidance path planning for UAV based on improved artificial potential field with collision cone[J]. Computer Engineering, 2018, 9: 314-320.

[0015]

[10] Zhao LB, Zhu GS, Zhuang L, et al. Improved Research on TargetUnreachable Problem of Path Planning Based on Artificial Potential Field for an Unmanned Aerial Vehicle[C] / / 2021IEEE 7th International Conference onControl Science and Systems Engineering.IEEE,2021:136-142.

[0016]

[11] Ma

[0017]

[12] Shangguan L, Thomasson JA, Gopalswamy S. Motion planning for autonomous grain carts[J]. IEEE Transactions on Vehicular Technology, 2021,70(3):2112-2123. Summary of the invention

[0018] 1. Technical issues to be resolved

[0019] The technical problem to be solved by the present invention is: how to provide a binocular vision navigation method for an unmanned vehicle based on an improved artificial potential field method, solve the problem of poor real-time performance of a system that uses an SGM algorithm in binocular stereo vision to perceive the surrounding environment and the local minimum problem of a traditional artificial potential field method, and realize real-time path planning for unmanned vehicles in unknown environments.

[0020] (II) Technical solution

[0021] In order to solve the above technical problems, the present invention provides a binocular vision navigation method for an unmanned vehicle based on an improved artificial potential field method. The method adopts an FPGA implementation framework of a semi-global stereo matching algorithm based on block comparison, which is divided into a matching cost calculation module, a cost aggregation module, a disparity calculation module and a disparity optimization module; the modules are parallelized and pipelined between modules, thereby realizing real-time perception of depth information while reducing the use of FPGA hardware resources.

[0022] (III) Beneficial effects

[0023] Compared with the prior art, the present invention aims at the problem of real-time path planning in unknown environments and proposes a binocular vision navigation method for unmanned vehicles based on an improved artificial potential field method. Firstly, the hardware acceleration of the SGM algorithm is completed by using a field programmable gate array (FPGA) to realize fast real-time perception of scene depth information. Secondly, the current scene image is binarized, median filtered and connected domain detected to realize obstacle recognition, and obstacle distance information is output in combination with the scene depth information. Finally, on an advanced reduced instruction processor (ARM), with the obstacle distance information within the perception range as a constraint condition, an improved artificial potential field method is used to search for a collision-free path from a starting point to a target point to realize unmanned vehicle navigation.

[0024] The method of the present invention was experimentally verified on a built unmanned vehicle system; the experimental results show that the FPGA implementation framework of the semi-global stereo matching algorithm based on block comparison in the present invention has good real-time performance, and the unmanned vehicle binocular vision navigation method based on the improved artificial potential field method improves the navigation performance of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a structural block diagram of the unmanned vehicle system of the method of the present invention;

[0026] Figure 2 This is the result image of the left camera of the binocular camera after grayscale conversion and correction according to the method of the present invention;

[0027] Figure 3 This is the result image of the right camera of the binocular camera after grayscale conversion and correction according to the method of the present invention;

[0028] Figure 4 A physical diagram of the unmanned vehicle system of the method of the present invention;

[0029] Figure 5 This is a block diagram of the matching cost calculation FPGA implementation of the method of the present invention;

[0030] Figure 6 The right figure of the method of the present invention is a schematic diagram of rightward shift in units of columns;

[0031] Figure 7 The hardware implementation block diagram of the window taking method of the present invention is as follows;

[0032] Figure 8 The method of the present invention is Figure 2 and Figure 3 is the matching cost result when the input disparity is 0;

[0033] Fig. 9 This is a schematic diagram of the cost aggregation direction of the method of the present invention;

[0034] Fig.10 A block comparison implementation method for cost aggregation of the method of the present invention;

[0035] Fig.11 It is a two-dimensional register structure diagram of the method of the present invention;

[0036] Fig.12 It is a timing diagram of cost aggregation iteration of the method of the present invention;

[0037] Fig.13 The method of the present invention is Figure 2 and Figure 3 The cost aggregation result when the parallax obtained by cost aggregation is 0;

[0038] Fig.14 This is a block diagram of the hardware implementation of the parallax calculation method of the present invention;

[0039] Fig.15 The method of the present invention is Figure 2 and Figure 3 An initial disparity map obtained by performing disparity calculation;

[0040] Fig.16 The hardware implementation diagram of the median filter method of the present invention is as follows

[0041] Fig.17 The method of the present invention is Fig.15 A final disparity map obtained by performing disparity optimization;

[0042] Fig.18 It is the RGB image of the left camera of the binocular camera of the method of the present invention;

[0043] Fig.19 In the method of the present invention Fig.18 Corresponding binary result map;

[0044] Fig. 20 In the method of the present invention Fig.19 Corresponding median filtering result graph;

[0045] Fig.21 In the method of the present invention Fig. 20 Corresponding obstacle detection results;

[0046] Fig. 22 In the method of the present invention Fig. 20 and Fig.17 Corresponding obstacle disparity map;

[0047] Fig.23 Schematic diagram of the angle formed by the resultant force of the j-1th movement and the direction of the resultant force of the jth movement in the method of the present invention;

[0048] Fig.24 A schematic diagram of setting a virtual target point for the method of the present invention;

[0049] Fig.25 In the method of the present invention Fig.21 Schematic diagram of the columns;

[0050] Fig.26 A schematic diagram of a collision-free path of the method of the present invention;

[0051] Fig. 27 The method of the present invention compares blocks and directly compares resource usage histograms. DETAILED DESCRIPTION

[0052] In order to make the purpose, content, and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the accompanying drawings and embodiments;

[0053] The unmanned vehicle binocular vision navigation method based on the improved artificial potential field method in the present invention adopts an FPGA implementation framework of a semi-global stereo matching algorithm based on block comparison, which is divided into a matching cost calculation module, a cost aggregation module, a disparity calculation module and a disparity optimization module; parallelism within the modules and pipeline operation between modules realize real-time perception of depth information while reducing the use of FPGA hardware resources; the present invention uses an improved artificial potential field method based on virtual target points to effectively solve the local minimum problem existing in the traditional artificial potential field method.

[0054] The structure diagram of the unmanned vehicle system of the present invention is as follows: Figure 1 As shown in the figure, firstly, the ATK-OV5640 binocular camera 1 is used to collect images of the environment in front of the unmanned vehicle (the left and right images after grayscale and correction are shown in Figure 1). Figure 2 and Figure 3As shown in the figure, the ATK-IMU901 inertial measurement unit 2 (IMU) is used to collect the state information of the unmanned vehicle (azimuth, acceleration and position, etc.); then, the core controller ZYNQ-7020 composed of ARM 3 and FPGA 4 is used to realize the detection of obstacles on the road ahead of the unmanned vehicle, and the path planning technology and the position information of the unmanned vehicle are combined to complete the path planning and control strategy output of the unmanned vehicle system; finally, according to the control strategy, the driving of the RS380SH DC motor 5 of the unmanned vehicle and the steering control of the SRM102 steering gear 6 are completed, so that the unmanned vehicle finally arrives near the target point, as shown in the corresponding physical figure. Figure 4 As shown;

[0055] Step 1: Semi-global stereo matching (SGM) programmable hardware implementation

[0056] The semi-global stereo matching algorithm in the method of the present invention is implemented based on the hardware of a programmable logic device FPGA, and the specific steps are as follows:

[0057] 1.1 Implementation of Matching Cost Calculation

[0058] The matching cost calculation mainly consists of two steps: Census change and Hamming distance calculation. The FPGA implementation process is as follows: Figure 5 As shown, assuming that the resolution of the left and right images is Length×Width, d min and d max Respectively represent the minimum and maximum values ​​of the disparity d search range; Figure 5 The Census transformation of the left and right images is expressed by equations (1) and (2):

[0059]

[0060] Where p and q represent the central pixel of the matching window and any other pixel in the matching window except the central pixel p, respectively. ζ(I(p), I(q)) represents the comparison result of the grayscale values ​​of the pixel points p and q. I(p) and I(q) represent the grayscale values ​​of the pixel points p and q, respectively.

[0061]

[0062] Where T(p) represents the bit string of point p (length w 2 -1), N p Represents the neighborhood set consisting of all pixels except the central pixel p in the Census window of size w×w. Represents a binary join operation;

[0063] After the Census codes (bit strings) of the left and right images are calculated, the right image is shifted right by columns based on the left image, and the first column of the right image is used to fill the blank columns on the left. The schematic diagram is as follows: Figure 6 As shown, the calculation process of the Hamming distance under different parallaxes is as follows:

[0064]

[0065] Where C(p,d) represents the parallax d∈[d min ,d max ]The matching cost of point p, L i (p) represents the i-th binary number in the bit string at point p in the left figure, R i (p, d) represents the i-th binary number in the bit string at the right image point p under disparity d. The symbol represents an XOR operation; the Census transformation and the Hamming distance calculation under different parallaxes are both calculated simultaneously using the parallel computing characteristics of the FPGA; the Census transformation and the Hamming distance calculation are implemented using the pipeline characteristics of the FPGA to output the matching cost values ​​under different parallaxes;

[0066] The more complicated part in the FPGA hardware implementation process of matching cost calculation is w×w window extraction. The present invention uses line cache and window cache to implement window extraction. The hardware implementation process of window extraction is as follows: Figure 7 As shown in the figure, the pixel values ​​of the left and right images are combined and stored. The line memory is composed of w-1 pseudo dual-port random access memories (RAMs). The depth of each RAM is consistent with the image length, and the width is 16 bits, where the high 8 bits and the low 8 bits represent the grayscale values ​​of the left and right pixels respectively. Figure 7 RAM1 to RAMw-1 are used to store the first w-1 rows of pixel data of the current pixel point. When a new row of data arrives, the data in RAMw-1 is read out, and the data in RAMw-2 is written into RAMw-1, and so on, until the data in RAM1 is written into RAM2, and finally the new row of data is written into RAM1, so as to realize the output of "w rows and one column" row cache data; then the row cache data is sequentially stored in the memory composed of w 2 The registers form a w×w matrix to realize the extraction of w×w window;

[0067] When the window is taken, the output data DATA of the window buffer is divided into the grayscale values ​​of the left and right images to form two w×w matrices; then the grayscale values ​​of the center points of each window are simultaneously combined with N p In the collection 2 -1 data to compare and generate (w 2-1) bit Census code, and then output the Hamming distance under different parallaxes according to formula (3);

[0068] In the embodiment, the left and right image resolutions are set to 512×300, the disparity search range is d∈[0,69], and the Census window size is 5×5, that is, Length=512, Width=300, d min =0,d max =69, w=5; Figure 2 and Figure 3 The matching cost is calculated for the input. When the disparity is 0, the matching cost calculation result is as follows Figure 8 As shown in the figure, the darker the color, that is, the smaller the Hamming distance, the higher the similarity between the pixels of the left and right images under the current parallax;

[0069] 1.2 Implementation of cost aggregation

[0070] The present invention uses Fig. 9 The cost is aggregated in the three directions of "0", "4" and "1". The cost calculation formula for a pixel point p along direction r is as follows:

[0071]

[0072] Among them, L r (p, d) represents the cost value of pixel p in the r direction at the disparity d; L r (pr, d) represents the cost value of the previous pixel in the direction of the current point r under disparity d, that is, when the direction is "0", "4" and "1", L r (pr, d) represents the cost value of the pixel to the left, the pixel to the upper left, and the pixel directly above the current pixel p respectively; if the difference in disparity between the pixel p and the previous pixel in the r direction under the current disparity d is 1, a penalty coefficient Q1 is added, which is a given constant; if the difference in disparity is greater than 1, a penalty coefficient Q2(p) is added; in order to reduce disparity mutations and discontinuities, the Q2(p) value is determined by the grayscale gradient of the current point p in the left image, as shown in formula (5):

[0073]

[0074] Among them, Q'2 is a given constant, I L (p) and I L (pr) represents the grayscale value of the previous pixel in the direction of the pixel p and the current point r in the left image, that is, when the direction is "0", "4" and "1", I L (pr) represents the grayscale value of the pixel to the left, the pixel to the upper left, and the pixel directly above the current pixel p respectively;

[0075] The cost value of the current point p in all directions is accumulated to obtain the cost aggregation value C under different parallaxes sum (p,d):

[0076]

[0077] In formula (4), the cost value L r During the (p,d) calculation process, the most resource-consuming part is and Calculation of two items; used in the present invention Fig.10 The main idea of ​​the block comparison method to calculate min_i is to merge repeated comparison parts as much as possible; Fig.10 The columns in represent the cost values ​​of different disparities required for calculating min_i for pixel p under a certain disparity. The square blocks in the figure represent the merged repeated comparison parts. For example, square blocks ① are repeated comparison parts of disparities 0 to 34 from left to right, that is, to calculate the min_i value of disparities 0 to 34, it is only necessary to calculate the minimum value in square block ① once. The principle of other square blocks is similar. After the min_i value is calculated, min_k can be obtained by comparing min_i with the adjacent disparity of the current pixel and the cost value under the current disparity. It can be seen from formula (4) that the cost value L of pixel p in the r direction is r (p,d), by L r (pr, d) values ​​are iterated; for the cost value calculation of direction "0", the cost value under all disparities of the current row of pixels is required, and for directions "4" and "1", the cost value under all disparities of the previous row of pixels is required; therefore, a two-dimensional register is required to implement the storage of the cost value, and the two-dimensional storage structure is as follows Fig.11 As shown, the depth is Length+2 and the width is the maximum cost value L r (p, d) corresponds to the number of binary digits and the number of parallaxes (d max -d min +1); in the figure, L0, L4 and L1 represent the cost values ​​in the directions of "0", "4" and "1" respectively, and the cost values ​​corresponding to all disparities of a certain pixel in a certain direction can be read out within one clock to facilitate the iterative update of the cost value;

[0078] The present invention uses Fig.12 The cost aggregation is performed using the time-sharing processing method, that is, the cost values ​​in the three directions of "0", "4" and "1" are calculated in sequence at different times to complete the cost aggregation operation of the current point; Fig.12 Where C(p,d) is calculated by equation (3), Q2(p) is calculated by equation (5), and L r (pr,d) from Fig.11Then, the cost value L in different directions of the current point is obtained by using formula (4): r (p, d), and store it in a two-dimensional register; finally, use formula (6) to output the cost aggregation value of the pixel under different disparities; Fig.12 One clock corresponds to the cost value of pixel p in one direction, that is, the output of the cost aggregation value of one pixel is completed every three clocks, and the cost aggregation values ​​of pixels under different parallaxes are calculated simultaneously to reduce the running time of the algorithm;

[0079] In the embodiment, Q1 in formula (4) and Q'2 in formula (5) are respectively 4 and 300; Fig.11 The two-dimensional register is implemented using a pseudo dual-end RAM with a depth of 514 and a width of 9 bits × 70 = 630 bits. Figure 2 and Figure 3 Perform cost aggregation. When the disparity is 0, the cost aggregation result is as follows: Fig.13 As shown in the figure, similar to the matching calculation results, the darker the color in the figure, the higher the similarity between the pixels of the left and right images under the current disparity;

[0080] 1.3 Implementation of parallax calculation

[0081] When the cost aggregation value of pixel p under all disparities is calculated, the cost aggregation values ​​of the current point under all disparities are compared, and the disparity value corresponding to the minimum cost aggregation value is output as the initial disparity of the current point. The calculation process is as follows:

[0082]

[0083] The present invention adopts a pairwise comparison method to find the minimum cost aggregation value under all disparities. Each comparison needs to record the disparity value corresponding to the current minimum cost aggregation value as the final disparity value of the current pixel point. The hardware implementation process of FPGA pipeline disparity calculation is as follows: Fig.14 As shown in the figure, the comparisons at the same level are performed simultaneously, and the calculations between different levels are pipelined;

[0084] In the embodiment, the Fig.14 The method in finds the minimum cost aggregation value under 70 disparities. It needs to perform 7 levels (70→35→18→9→5→3→2→1) of comparison to output the minimum value. Figure 2 and Figure 3 Perform parallax calculation and obtain the initial parallax as Fig.15 As shown, it can be seen that there are isolated points and stripe noise in the figure;

[0085] 1.4 Implementation of parallax optimization

[0086] Since the disparity map obtained by the SGM algorithm based on the dynamic programming idea has the phenomenon of stripes and discrete points, the present invention uses median filtering to optimize the initial disparity map. The basic principle of median filtering is to select a median filtering window of size w1×w1 (w1 is an odd number) and replace the central pixel disparity value with the median of the initial disparity of all pixels in the window. The windowing principle is the same as Figure 7 Similarly; after the w1×w1 median filter window is extracted, use Fig.16 The median in the window is extracted by pipeline operation. Here, a 3×3 window is used as an example. First, each row of data in the window is compared to solve the maximum, minimum and median of each row of data. Then, the maximum, minimum and median of different rows are compared to solve the minimum value data1 of all row maximum values, the median value data2 of all row medians, and the maximum value data3 of all row minimum values. Finally, data1, data2 and data3 are compared. If (data1>=data2&&data1<=data3)||(data1>=data2&&data1<=data3)||(data2>=data3)||(data3>=data1)||(data1>=data2)||(data2>=data3)||(data3>=data3)||(data1>=data2 ... When the condition (data1>=data3&&data1<=data2) is met, Mid_data=data1; when the condition (data2>=data1&&data2<=data3)||(data2>=data3&&data2<=data1) is met, Mid_data=data2; otherwise, Mid_data=data3; Mid_data is the median of all data in the 3×3 window, which is the final output result; in the above conditions, "&&" represents the logical AND operation, and "||" represents the OR operation;

[0087] In the embodiment, the median filter window size is set to 3×3, that is, w1=3. Fig.15 The initial disparity map in is used for disparity optimization, and the final disparity map is as follows Fig.17 As shown in the figure, the brighter the color, the larger the parallax, which means the smaller the distance between the object and the camera;

[0088] Step 2: Obstacle distance calculation

[0089] After the scene depth information is calculated, it is necessary to detect obstacles in the current scene to calculate the obstacle distance; the specific steps are as follows:

[0090] 2.1 Binarize the left image

[0091] In the left image of the binocular camera, set the pixel data whose red component is greater than φ1, whose green component is less than φ2, and whose blue component is less than φ3 to 1 (obstacle pixel), and set the pixel data whose red component is greater than φ1, whose green component is less than φ2, and whose blue component is less than φ3 to 0;

[0092] In an embodiment, the corrected left image obtained by the binocular camera is as follows: Fig.18As shown, set the above φ1, φ2, and φ3 to 70, 75, and 75 respectively, and obtain Fig.19 The binary image shown in the figure shows that the white part is the obstacle;

[0093] 2.2 Perform median filtering on the binary image obtained in step 2.1

[0094] Set the median filter window to w2×w2 (w2 is an odd number), add the values ​​of all pixels in the window, and if the result is greater than (w2 2 -1) / 2, that is, if the number of pixel values ​​in the window is "1" is greater than the number of "0", then "1" is output instead of the center pixel value in the window; otherwise, "0" is output;

[0095] In the embodiment, w2 is set to 5. Fig.19 The result of median filtering is as follows Fig. 20 As shown, from Fig. 20 It can be seen Fig.19 A large amount of noise is filtered out, but there are still a few isolated noise points;

[0096] 2.3 Obstacle identification based on the median filtering result obtained in step 2.2

[0097] Since the first few rows of image data are usually the background part of the current scene, the corresponding distance is far, so it is set that when the number of rows where the pixel point is located is greater than the valid row threshold λ1, the pixel point is valid, that is, the first λ1 rows of pixels are invalid; then it is connected The domain detection is performed on it, and each pixel point in the image is traversed. First, it is determined whether the value of the current pixel point is "1". If not, it jumps to the next pixel; if so, it is determined whether there is a point with a pixel value of "1" in its 8-neighborhood. If so, it is considered that the pixel points with a value of "1" are connected, and the same label as the connected pixels is assigned; if not, a new label is assigned to the current point; finally, the pixels with the same label are regarded as the same connected domain; the number of connected domains corresponds to the number of obstacles in the current scene;

[0098] Then, the number of pixels in each connected domain in the image is counted, that is, the area of ​​the connected domain; when the area of ​​the connected domain is greater than the set threshold λ2, the obstacle corresponding to the connected domain is regarded as a valid obstacle. This operation can effectively remove the influence of noise or tiny objects;

[0099] In the embodiment, the valid row threshold λ1 is set to 30, and the area threshold λ2 is set to 100; Fig. 20 The corresponding obstacle detection results are as follows: Fig.21 As shown, it can be seen that there are two obstacles in the figure;

[0100] 2.4 Obstacle distance calculation

[0101] Combine the effective obstacle result obtained in step 2.3 with the disparity map output by stereo matching in the first step, and calculate the average disparity of the effective pixels in the effective obstacle area (when the disparity corresponding to the pixel is greater than the set threshold λ3, the pixel is valid) arg , and through formula (8)

[0102] Convert parallax into distance information and calculate the distance of obstacles;

[0103] z=Bf / d arg (8)

[0104] Where z is the obstacle distance, B is the baseline distance of the binocular camera, and f is the focal length information of the camera;

[0105] In an embodiment, the effective disparity threshold λ3 is set to 10, combined with Fig.17 The scene depth information and Fig.21 The obstacle detection result is obtained by Fig. 22 As shown; then, using formula (8), the distance between obstacle 1 and the unmanned vehicle is calculated as z1 = 31.1991 * 863.6526 / 34.9424 = 77.1301 cm, and the distance between obstacle 2 and the unmanned vehicle is z2 = 31.1991 * 863.6526 / 51.8875 = 51.93 cm;

[0106] Step 3: Path planning for unmanned vehicles

[0107] Taking the obstacle distance information in the second step as the constraint condition, the improved artificial potential field method is used to realize the path planning of the unmanned vehicle;

[0108] 3.1 Calculate the target point P g The gravitational force F on the autonomous vehicle P att (P):

[0109]

[0110] Among them, k att represents the gravitational gain coefficient, dist(P,P g ) represents the distance between the unmanned vehicle P and the target point P g The distance between Represents the unit vector pointing from the unmanned vehicle to the target point;

[0111] 3.2 Calculate the i-th obstacle P o (i) Repulsive force F on the unmanned vehicle P rep (P,i):

[0112]

[0113] in

[0114]

[0115]

[0116] k rep represents the repulsion gain coefficient, dist(P,P o (i)) represents the unmanned vehicle P and the obstacle P o (i) distance between Indicates that the obstacle P o (i) The unit vector pointing to the unmanned vehicle, c is a real number greater than 0, ρ0 is the obstacle distance threshold, and when dist(P,P o (i))>ρ0, the repulsive force exerted by the obstacle on the unmanned vehicle is 0;

[0117] 3.3 Calculate the resultant force F of the unmanned vehicle in the scenario with m obstacles sum (P) As shown in formula (11), the unmanned vehicle always moves in the direction of the resultant force during the path planning process;

[0118]

[0119] 3.4 When the unmanned vehicle does not reach the target point, the angle formed by the resultant force of the j-1th movement and the direction of the resultant force of the jth movement is greater than the set value or the current resultant force is 0, the unmanned vehicle will stay at a certain position or oscillate back and forth near a certain position, as shown in the schematic diagram Fig.23 As shown, in this case, the unmanned vehicle is considered to be trapped in a local minimum;

[0120] At this time, the present invention establishes a virtual target point to make the unmanned vehicle escape from the local minimum trap; when the unmanned vehicle reaches the vicinity of the virtual target point, the virtual target point is cancelled and the vehicle continues to move in the direction of the resultant force in formula (11) until it reaches the vicinity of the target point; the virtual target point P ng (x ng ,y ng ) is set according to formula (12), and its schematic diagram is shown as follows Fig.24 As shown;

[0121]

[0122] Where x and y represent the current coordinates of the unmanned vehicle. is the heading angle of the unmanned vehicle (detected in real time by IMU), d Gis the distance between the unmanned vehicle and the nearest obstacle when it falls into the local minimum (obtained from step 2.4), α is the angle between the virtual target point and the current direction of the unmanned vehicle, and is the set angle, which can be adjusted in real time according to the location of the obstacle in the current scene; the binary image corresponding to the obstacle detection result of step 2.3 is evenly divided vertically, and the angle of each column of the divided area is set; when the position occupied by the obstacle in the left half image is greater than or equal to the obstacle in the right half image, the unmanned vehicle turns right, α<0; conversely, when the position occupied by the obstacle in the left half image is smaller than the obstacle in the right half image, the unmanned vehicle turns left, α>0; the column processing of the image can adjust the size of α in real time according to the location of the obstacle in the current scene to complete the autonomous establishment of the virtual target point;

[0123] In an embodiment, the third step of improving the artificial potential field method is combined with the obstacle distance information of the second step, and Figure 4 The ARM end of the ZYNQ-7020 controller on the unmanned vehicle platform in the figure is completed to realize the navigation of the unmanned vehicle; establish a navigation coordinate system (the direction of the front of the vehicle is the positive direction of the longitudinal axis, and the positive direction of the horizontal axis is perpendicular to the vehicle body and faces right), the starting coordinates of the unmanned vehicle are set to (0m, 0m), the coordinates of the target point are set to (0m, 10m), the obstacle action distance ρ0, and the gravity gain coefficient k att , repulsion gain coefficient k rep and constant c are set to 0.9m, 15, 10 and 2 respectively; α in the virtual target point coordinates is adjusted by the unmanned vehicle in combination with the current scene, and the image is divided into 10 columns, as shown in the schematic diagram. Fig.25 As shown in Table 1, the relationship between the obstacle position and the α value in the column image is shown in Table 1. In the table, "0" represents that there is no obstacle in the column, "1" represents that there is an obstacle in the column, and "×" represents that the column can be "0" or "1". Except for the 13 cases described in the table, α=35° is given in the rest of the cases;

[0124] Table 1 Relationship between obstacle position and α value in column image

[0125]

[0126]

[0127] exist Fig.26 The figure shows the process of the unmanned vehicle navigation experiment in a certain scenario. There are five obstacles in the figure. Since the first two obstacles are on the same horizontal line, the unmanned vehicle avoids obstacles four times. Fig.26In the middle position ①, the unmanned vehicle is at the starting position; position ② corresponds to the first obstacle avoidance. Through the image column processing, it is judged that the distance between the two obstacles in the current scene is passable, so it goes straight; position ③ corresponds to the second obstacle avoidance. Since the obstacle in front of the unmanned vehicle is located between the target point and the current point, the obstacle avoidance is completed by establishing a virtual target point; position ④ corresponds to the third obstacle avoidance. Since the target point is located in the left front of the unmanned vehicle, and the binocular camera detects that the obstacle is located in the right front of the unmanned vehicle in the current scene, the unmanned vehicle turns left; position ⑤ corresponds to the fourth obstacle avoidance. Since the binocular camera detects that the obstacle is located in the left front, the unmanned vehicle turns a small angle to achieve obstacle avoidance; position ⑥ is the final arrival position of the unmanned vehicle. The coordinates of this point are (0.04m, 9.91m). Since there is an error in the measurement data of the IMU sensor, it can be considered that the unmanned vehicle has arrived near the target point at this time; The experimental results show that the unmanned vehicle can plan a collision-free path from the starting point to the target point, which proves the feasibility and practicality of the method of the present invention;

[0128] The advantages of the method of the present invention are: 1) an FPGA implementation framework of a semi-global stereo matching algorithm based on block comparison is proposed, with parallelism within the module and pipeline operation between modules, which realizes real-time perception of the depth information of the current scene while using as few FPGA hardware resources as possible; 2) an improved artificial potential field method based on virtual target points is proposed, which can effectively solve the local minimum problem existing in the traditional artificial potential field method;

[0129] The resource usage and obstacle ranging results of the SGM algorithm FPGA implementation method based on block comparison are given below;

[0130] Resource usage

[0131] In the first step of cost aggregation, if the pairwise comparison method is used directly to obtain the min_i value under each parallax, it will consume 765 lookup tables (LUTs) and 811 flip-flops (FFs) for one calculation, and it takes 7 clocks. If this method is used directly to simultaneously solve the min_i values ​​under 70 parallaxes using the parallel characteristics of FPGA, 70×765=53550 LUTs and 70×811=56770 FFs are required, which consumes a lot of FPGA hardware resources. Fig. 27 The histogram in the figure shows the resource usage of direct comparison and block comparison; if the present invention is used Fig.10 The block comparison method in the proposed method only uses 7922 LUT resources and 4365 FF resources, which reduces 85.21% of LUT and 92.31% of FF resources compared with the direct comparison method, greatly reduces the use of hardware resources, improves the utilization of system resources, and consumes only 3 clocks, which improves the real-time performance of the algorithm.

[0132] The present invention uses Verilog HDL language to complete the programmable hardware implementation of the semi-global stereo matching algorithm in the first step above; the image resolution is 512×300, the speed is 40fps, and the resource usage of the four steps of the SGM algorithm is given in Table 2; it can be seen that after the matching cost calculation, cost aggregation, disparity calculation and disparity optimization are encapsulated as the SGM intellectual property (Intellectual Property, IP) core, the resource usage is: 21515 LUTs, mainly used as logic function generators and storage units; 11001 FFs, belonging to sequential logic; 33 embedded block random access memories (Block Random Access Memory, BRAM), each block can store up to 36KB of information; it can also be seen from Table 2 that the cost aggregation step consumes the most resources among the four steps of the stereo matching algorithm;

[0133] Table 2 Resource usage of SGM algorithm

[0134]

[0135] The total resource usage of the unmanned vehicle system is shown in Table 3. It can be seen that a total of 40,763 LUTs are used, accounting for 76.62% of the total resources; 764 unintegrated LUTs are used as distributed RAM (LUTRAM); 33,401 FFs; 60 BRAMs; 96 digital signal processor (DSP) resources; 62 input / output (IO) ports, which mainly include binocular camera IO pins, LCD display IO pins, PWM signal output pins, etc.; 1 Mixed-Mode Clock Manager (MMCM) is mainly used to drive the clock of the device logic;

[0136] Table 3 Total resource usage of unmanned vehicle system

[0137]

[0138] Ranging results and analysis

[0139] Use FPGA to complete the first and second steps above to measure the obstacle distance. Place the unmanned vehicle at rest and place the obstacle in front of it. The measurement range is 40 cm to 200 cm. The obstacle moves back 5 cm each time. The measurement results are shown in Table 4:

[0140] Table 4 Obstacle distance measurement data

[0141]

[0142]

[0143] In Table 4, the absolute error represents the difference between the measured distance value and the true distance value, and the relative error represents the ratio of the absolute error to the true distance value. It can be seen from the table that when the distance between the unmanned vehicle and the obstacle is within 2m, the maximum relative error of the obstacle distance measurement is 4.5538%, and the average absolute value of the relative error is 1.2630%. When the obstacle distance is between 40cm and 170cm, the absolute error of the measurement result is within ±2.5cm. When the distance is greater than 170cm, the absolute error increases significantly. The main reason for this phenomenon is that as the parallax decreases, the distance corresponding to the parallax increases significantly.

[0144] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A binocular vision navigation method for an unmanned vehicle based on an improved artificial potential field method, characterized in that the method adopts an FPGA implementation framework of a semi-global stereo matching algorithm based on block comparison, which is divided into a matching cost calculation module, a cost aggregation module, a disparity calculation module, and a disparity optimization module; parallel operations are performed within the modules and pipeline operations are performed between the modules, thereby achieving real-time perception of depth information while reducing the use of FPGA hardware resources; The method uses an improved artificial potential field method based on virtual target points, which effectively solves the local minimum problem existing in the traditional artificial potential field method; In the unmanned vehicle system structure involved in the method, first, a binocular camera is used to collect images of the environment in front of the unmanned vehicle to obtain grayscale and corrected left and right images, and an inertial measurement unit is used to collect the state information of the unmanned vehicle, including azimuth, acceleration and position; then, a core controller composed of ARM 3 and FPGA4 is used to realize the detection of obstacles on the road in front of the unmanned vehicle, and the path planning technology and the position information of the unmanned vehicle are combined to complete the path planning of the unmanned vehicle system and the output of the control strategy; finally, the driving of the unmanned vehicle DC motor and the steering control of the servo are completed according to the control strategy, so that the unmanned vehicle finally arrives near the target point; The method comprises: Step 1: Programmable Hardware Implementation of Semi-Global Stereo Matching The semi-global stereo matching algorithm in the method is implemented based on the programmable logic device FPGA hardware, and the specific steps are as follows: 1.1 Implementation of Matching Cost Calculation The matching cost calculation consists of two steps: Census change and Hamming distance calculation. Assuming that the resolution of the left and right images is Length×Width, d min and d max They represent the minimum and maximum values ​​of the disparity d search range respectively; the Census transformation of the left and right images is expressed by equations (1) and (2): in, p and q represent the central pixel of the matching window and any other pixel in the matching window except the central pixel p, respectively. ζ(I(p), I(q)) represents the comparison result of the grayscale values ​​of the pixel points p and q. I(p) and I(q) represent the grayscale values ​​of the pixel points p and q, respectively. Where T(p) represents the bit string of point p, with a length of w 2 -1, N p Represents the neighborhood set consisting of all pixels except the central pixel p in the Census window of size w×w. Represents a binary join operation; After the Census codes of the left and right images are calculated, the right image is shifted right by columns based on the left image, and the first column of the right image is used to fill the blank columns on the left. The calculation process of the Hamming distance under different parallaxes is as follows: Where C(p,d) represents the parallax d∈[d min ,d max ]The matching cost of point p, L i (p) represents the i-th binary number in the bit string at point p in the left figure, R i (p, d) represents the i-th binary number in the bit string at the right image point p under disparity d. The symbol represents an XOR operation; the Census transformation and the Hamming distance calculation under different parallaxes are both calculated simultaneously using the parallel computing characteristics of the FPGA; the Census transformation and the Hamming distance calculation are implemented using the pipeline characteristics of the FPGA to output the matching cost values ​​under different parallaxes; The more complicated part in the FPGA hardware implementation process of matching cost calculation is w×w window extraction. The method adopts row cache and window cache to realize window extraction; the pixel values ​​of the left and right images are merged for storage, and the row memory is composed of w-1 pseudo dual-port random access memories, each RAM depth is consistent with the image length Length, and the width is 16 bits, where the high 8 bits and the low 8 bits represent the grayscale values ​​of the left and right image pixels respectively; RAM1 to RAMw-1 are used to store the first w-1 rows of pixel data of the current pixel point. When a new row of data arrives, the data in RAMw-1 is read out, and the data in RAMw-2 is written into RAMw-1, and so on, until the data in RAM1 is written into RAM2, and finally the new row of data is written into RAM1, so as to realize the output of "w rows and one column" row cache data; then the row cache data is sequentially stored in the memory composed of w 2 The registers form a w×w matrix to realize the extraction of w×w window; When the window is taken, the output data DATA of the window buffer is divided into the grayscale values ​​of the left and right images to form two w×w matrices; then the grayscale values ​​of the center points of each window are simultaneously combined with N p In the collection 2 -1 data to compare and generate w 2 -1 bit Census code, and then output the Hamming distance under different parallaxes according to formula (3); 1.2 Implementation of cost aggregation The method uses the three directions of "0", "4" and "1" for cost aggregation. The cost calculation formula for a pixel point p along the direction r is as follows: Among them, L r (p, d) represents the cost value of pixel p in the r direction at the disparity d; L r (pr, d) represents the cost value of the previous pixel in the direction of the current point r under the disparity d, that is, when the direction is "0", "4" and "1", L r (pr, d) represents the cost of the pixel to the left, the pixel to the upper left, and the pixel directly above the current pixel p. If the difference in disparity between the pixel p and the previous pixel in the r direction is 1 under the current disparity d, a penalty coefficient Q1 is added, which is a given constant. If the difference in disparity is greater than 1, a penalty coefficient Q2(p) is added. Among them, Q'2 is a given constant, I L (p) and I L (pr) represents the grayscale value of the previous pixel in the direction of the pixel p and the current point r in the left image, that is, when the direction is "0", "4" and "1", I L (pr) represents the grayscale value of the pixel to the left, the pixel to the upper left, and the pixel directly above the current pixel p respectively; The cost value of the current point p in all directions is accumulated to obtain the cost aggregation value C under different parallaxes sum (p,d): In formula (4), the cost value L r During the (p,d) calculation process, the most resource-consuming part is and The method uses the block comparison method to calculate min_i, and its idea is to merge the repeated comparison parts as much as possible; the method uses the time-sharing processing method to perform cost aggregation, that is, the cost values ​​in the three directions of "0", "4" and "1" are calculated in turn at different times to complete the cost aggregation operation of the current point; C(p,d) is calculated by formula (3), Q2(p) is calculated by formula (5), and L r (pr, d) is read from the two-dimensional register; then, the cost value L in different directions of the current point is obtained using formula (4) r (p, d), and store it in a two-dimensional register; finally, use formula (6) to output the cost aggregation value of the pixel under different parallaxes; 1 clock corresponds to the cost value of the pixel p in one direction, that is, the output of the cost aggregation value of a pixel is completed every 3 clocks, and the cost aggregation values ​​of the pixels under different parallaxes are calculated simultaneously to reduce the running time of the algorithm; 1.3 Implementation of parallax calculation When the cost aggregation value of pixel p under all disparities is calculated, the cost aggregation values ​​of the current point under all disparities are compared, and the disparity value corresponding to the minimum cost aggregation value is output as the initial disparity of the current point. The calculation process is as follows: The method adopts a pairwise comparison method to find the minimum cost aggregation value under all disparities. Each comparison needs to record the disparity value corresponding to the current minimum cost aggregation value as the final disparity value of the current pixel point; the comparisons at the same level are performed simultaneously, and the calculations between different levels are performed in a pipeline manner; 1.4 Implementation of parallax optimization Since the disparity map obtained by the SGM algorithm according to the dynamic programming idea has the phenomenon of stripes and discrete points, the method uses median filtering to optimize the initial disparity map. The basic principle of median filtering is to select a median filtering window of w1×w1, w1 is an odd number, and replace the disparity value of the center pixel with the median of the initial disparity of all pixels in the window; after the w1×w1 median filtering window is extracted, the median in the window is extracted by pipeline operation; Here we take a 3×3 window as an example to illustrate; first, compare each row of data in the window to solve the maximum, minimum and median values ​​in each row of data; then compare the maximum, minimum and median values ​​of different rows respectively to solve the minimum value data1 among all row maximum values, the median value data2 among all row medians, and the maximum value data3 among all row minimum values; finally, compare data1, data2 and data3. When the condition (data1>=data2&&data1<=data3)||(data1>=data3&&data1<=data2) is met, Mid_data=data1; when the condition (data2>=data1&&data2<=data3)||(data2>=data3&&data2<=data1) is met, Mid_data=data2; otherwise, Mid_data=data3; Mid_data is the median value of all data in the 3×3 window, which is the final output result; in the above conditions, "&&" represents the logical AND operation, and "||" represents the OR operation; Step 2: Obstacle distance calculation After the scene depth information is calculated, it is necessary to detect obstacles in the current scene to calculate the obstacle distance; the specific steps are as follows: 2.1 Binarize the left image In the left image of the binocular camera, the pixel data whose red component is greater than φ1, green component is less than φ2, and blue component is less than φ3 is set to 1, that is, the obstacle pixel, otherwise it is set to 0; 2.2 Perform median filtering on the binary image obtained in step 2.1 Set the median filter window to w2×w2, where w2 is an odd number, and add the values ​​of all pixels in the window. If the result of the addition is greater than (w2 2 -1) / 2, that is, if the number of pixel values ​​in the window is "1" is greater than the number of "0", then "1" is output instead of the center pixel value in the window; otherwise, "0" is output; 2.3 Obstacle identification based on the median filtering result obtained in step 2.2 Since the first few rows of image data are usually the background part of the current scene, the corresponding distance is far, so it is set that when the number of rows where the pixel point is located is greater than the valid row threshold λ1, the pixel point is valid, that is, the first λ1 rows of pixels are invalid; then it is connected The domain detection is performed on it, and each pixel point in the image is traversed. First, it is determined whether the value of the current pixel point is "1". If not, it jumps to the next pixel; if so, it is determined whether there is a point with a pixel value of "1" in its 8-neighborhood. If so, it is considered that the pixel points with a value of "1" are connected, and the same label as the connected pixels is assigned; if not, a new label is assigned to the current point; finally, the pixels with the same label are regarded as the same connected domain; the number of connected domains corresponds to the number of obstacles in the current scene; Then, the number of pixels in each connected domain in the image is counted, that is, the area of ​​the connected domain; when the area of ​​the connected domain is greater than the set threshold λ2, the obstacle corresponding to the connected domain is regarded as a valid obstacle. This operation can effectively remove the impact of noise or tiny objects; 2.4 Obstacle distance calculation Combine the effective obstacle result obtained in step 2.3 with the disparity map output by stereo matching in the first step. For the effective pixels in the effective obstacle area, when the corresponding disparity of the pixel is greater than the set threshold λ3, the pixel is valid, and the corresponding disparity is averaged d arg , and through formula (8) Convert parallax into distance information and calculate the distance of obstacles; z=Bf / d arg (8) Where z is the obstacle distance, B is the baseline distance of the binocular camera, and f is the focal length information of the camera; Step 3: Path planning for unmanned vehicles Taking the obstacle distance information in the second step as the constraint condition, the improved artificial potential field method is used to realize the path planning of the unmanned vehicle; 3.1 Calculate the target point P g The gravitational force F on the unmanned vehicle P att (P): Among them, k att represents the gravitational gain coefficient, dist(P,P g ) represents the distance between the unmanned vehicle P and the target point P g The distance between Represents the unit vector pointing from the unmanned vehicle to the target point; 3.2 Calculate the i-th obstacle P o (i) Repulsive force F on the unmanned vehicle P rep (P,i): in k rep represents the repulsion gain coefficient, dist(P,P o (i)) represents the unmanned vehicle P and the obstacle P o (i) distance between Indicates that the obstacle P o (i) The unit vector pointing to the unmanned vehicle, c is a real number greater than 0, ρ0 is the obstacle distance threshold, and when dist(P,P o (i))>ρ0, the repulsive force exerted by the obstacle on the unmanned vehicle is 0; 3.3 Calculate the resultant force F of the unmanned vehicle in the scenario with m obstacles sum (P) As shown in formula (11), the unmanned vehicle always moves in the direction of the resultant force during the path planning process; 3.4 When the unmanned vehicle has not reached the target point, if the angle formed by the resultant force of the j-1th movement and the direction of the resultant force of the jth movement is greater than the set value or the current resultant force is 0, the unmanned vehicle will stay at a certain position or oscillate back and forth near a certain position. In this case, the unmanned vehicle is considered to be in a local minimum. At this time, this method establishes a virtual target point to make the unmanned vehicle escape from the local minimum trap; when the unmanned vehicle reaches the vicinity of the virtual target point, the virtual target point is cancelled and the vehicle continues to move in the direction of the resultant force in formula (11) until it reaches the vicinity of the target point; the virtual target point P ng (x ng ,y ng ) is set according to formula (12); Where x and y represent the current coordinates of the unmanned vehicle. is the heading angle of the unmanned vehicle, d G is the distance between the unmanned vehicle and the nearest obstacle when it falls into the local minimum, α is the angle between the virtual target point and the current direction of the unmanned vehicle, and is the set angle, which can be adjusted in real time according to the location of the obstacle in the current scene; the binary image corresponding to the obstacle detection result of step 2.3 is equally divided vertically, and the angle of each column of the divided area is set; when the position occupied by the obstacle in the left half image is greater than or equal to the obstacle in the right half image, the unmanned vehicle turns right, α<0; conversely, when the position occupied by the obstacle in the left half image is smaller than the obstacle in the right half image, the unmanned vehicle turns left, α>0; the column processing of the image can adjust the size of α in real time according to the location of the obstacle in the current scene to complete the autonomous establishment of the virtual target point.

2. The unmanned vehicle binocular vision navigation method based on the improved artificial potential field method as claimed in claim 1, characterized in that: In the specific implementation of step 1.1, the resolution of the left and right images is set to 512×300, the disparity search range is d∈[0,69], and the Census window size is 5×5, that is, Length=512, Width=300, d min =0,d max =69, w=5.

3. The unmanned vehicle binocular vision navigation method based on the improved artificial potential field method as claimed in claim 1, characterized in that: In the specific implementation of step 1.2, Q1 in formula (4) and Q'2 in formula (5) are respectively set to 4 and 300.

4. The unmanned vehicle binocular vision navigation method based on the improved artificial potential field method as claimed in claim 1, characterized in that: In the specific implementation of step 1.3, to find the minimum cost aggregation value under 70 disparities, 7 levels are required, i.e., 70 → 35 → 18 → 9 → 5 → 3 → 2 → 1 comparison to output the minimum value.

5. The unmanned vehicle binocular vision navigation method based on the improved artificial potential field method as claimed in claim 1, characterized in that: In the specific implementation of step 1.4, the median filter window size is set to 3×3, that is, w1=3.

6. The unmanned vehicle binocular vision navigation method based on the improved artificial potential field method as claimed in claim 1, characterized in that: In the specific implementation of step 2.1, φ1, φ2, and φ3 are set to 70, 75, and 75 respectively to obtain a binary image, in which the white part is the obstacle.

7. The unmanned vehicle binocular vision navigation method based on the improved artificial potential field method as claimed in claim 1, characterized in that: In the specific implementation of step 2.2, w2 is set to 5.

Images