A Visual Feature Matching and Localization Method for Driverless Yard Trucks in Ports

Abstract

The present invention relates to a vision feature matching and positioning method for driverless container trucks in ports. The method includes the following steps: S1: The vehicle travels on a preset road, acquires a current road condition picture based on an in-vehicle camera, performs image thresholding processing on the current road condition picture, and outputs target picture data information; S2: Based on the target picture data information, deep learning is carried out, feature transformation is performed on the semantic segmentation data matrix, and the relative pose of the vehicle is output; S3: Coupling the high-precision map data information, respectively according to the lane line matching and positioning algorithm and the Berth No. matching and positioning algorithm, the first vehicle pose and the second vehicle pose are output; S4: Coupling the relative pose of the vehicle, the first vehicle pose and the second vehicle pose, the corrected vehicle pose and confidence information are output. The present invention can achieve stable positioning of driverless container trucks in special scenarios such as yards and quay cranes in the port environment, providing a reliable guarantee for the stable operation of driverless container trucks in the port.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned truck matching and positioning, and particularly to a visual feature matching and positioning method for unmanned trucks in ports. Background Art

[0002] Currently, world trade is booming, and the trend of globalization is irresistible. As an important part of global trade, maritime trade is an important means for each country to participate in international trade. Port terminals are the connection points between maritime transportation and land transportation, and their importance in trade transportation is self-evident. Therefore, how to improve the transfer efficiency of port containers, reduce labor costs and operation risks, and thus enhance port competitiveness has always been the goal pursued by ports around the world.

[0003] Whether it is satellite positioning (GNSS) or inertial navigation positioning (INS), the errors of the autonomous driving positioning system are inevitable, and the positioning results usually deviate from the actual position. Introducing map matching can effectively eliminate the system random error, correct the sensor parameters, and make up for the positioning vacuum period where GNSS positioning fails and the INS error increases sharply in urban high-rise areas, tree-lined areas, overpasses, tunnels, yards, gantry cranes, etc.

[0004] Map matching positioning technology refers to the process of matching the longitude and latitude sampling sequence of the driving trajectory of an autonomous vehicle with the road network of a high-precision map. Map matching positioning technology compares the vehicle positioning information with the road position information provided by the high-precision map, and uses an appropriate algorithm to determine the current driving section of the vehicle and its accurate position in the section, correct the positioning error, and provide a reliable basis for autonomous driving path planning. Summary of the Invention

[0005] In view of the above deficiencies of the prior art, the present invention provides a visual feature matching and positioning method for unmanned trucks in ports, which can achieve stable positioning of unmanned trucks in ports in special scenarios such as yards and gantry cranes, and provides a reliable guarantee for the stable operation of unmanned trucks in ports.

[0006] In order to achieve the above object and other related objects, the technical solution provided by the present invention is as follows:

[0007] A visual feature matching and positioning method for unmanned trucks in ports, comprising the following steps:

[0008] S1: The vehicle travels on a preset road, obtains a current road condition picture based on an on-vehicle camera, performs image thresholding processing on the current road condition picture, and outputs target picture data information;

[0009] S2: Based on the target picture data information, perform in-depth learning to obtain a lane line recognition data matrix, a road sign recognition data matrix, and a semantic segmentation data matrix. Perform feature transformation on the semantic segmentation data matrix and output the relative pose of the vehicle.

[0010] S3: Based on the lane line recognition data matrix and the road sign recognition data matrix, couple the high-precision map data information, and respectively output the first vehicle pose and the second vehicle pose according to the lane line matching positioning algorithm and the Beiwei number matching positioning algorithm.

[0011] S4: According to the pre-collected vector map, obtain the vector feature information of the current vehicle position, couple the relative pose of the vehicle, the first vehicle pose, and the second vehicle pose, and output the corrected vehicle pose and the confidence information.

[0012] Further, in step S3, the lane line matching positioning algorithm includes:

[0013] S311: Determine whether the quality of the initial vehicle pose meets the requirements. If not, optimize the initial pose and set the confidence to 0, and output the optimized initial pose. If so, go to step S312;

[0014] S312: Based on the initial vehicle pose, obtain the lane line data information of the road surface at the current vehicle position, filter and retain the lane line data information on the left and right sides of the vehicle, and then output the data matrix of the lane line position points according to the high-precision map.

[0015] S313: Transform the data matrix of the lane line position points to the vehicle body coordinate system through the initial pose, and output the optimal offset of the vehicle based on the least squares optimization algorithm.

[0016] S314: Compensate the optimal offset of the vehicle to the initial pose and output the first vehicle pose.

[0017] Further, in step 313, the calculation formula for transforming the lane line position points to the vehicle body coordinates through the initial pose is:

[0018] , where T is the local compensation pose to be optimized, T0 is the initial pose, (x robot , y robot , 0) is the coordinate point of the lane line position point in the vehicle body coordinate system, (x world , y world , 0) is the lane line position point.

[0019] Further, in step S3, the Beiwei number matching positioning algorithm includes:

[0020] S321: Based on the road sign recognition data matrix and the high-precision map, obtain the bay number information, obtain the Yardnum where the vehicle's current position is located according to the initial pose, determine whether the Yardnum is successfully obtained. If so, proceed to step S322;

[0021] S322: According to the bay number information, obtain the corresponding Yard value, determine the data required by the camera according to the Yard value, and then search for the corresponding Baynum in the high-precision map according to the Yardnum where the vehicle's current position is located, and output the high-precision map Baynum;

[0022] S323: According to the high-precision map Baynum, convert it to the vehicle body coordinate system through the initial pose, and output the offset data information;

[0023] S324: Compensate the offset data information to the initial pose and output the second vehicle pose.

[0024] Further, in step S322, determining the data required by the camera according to the Yard value includes: obtaining the data of the left camera when obtaining an odd Yard, and obtaining the data of the right camera when obtaining an even Yard.

[0025] Further, the Yardnum and the Baynum are in a mapping relationship.

[0026] Further, in step S321, if not, optimize the initial pose and set the confidence level to 0, and output the optimized initial pose.

[0027] Further, the vector feature information includes the vehicle's position, direction, and steering angle information.

[0028] To achieve the above object and other related objects, the present invention also provides a vision feature matching and positioning system for port autonomous container trucks, including a computer device, which is programmed or configured to execute the steps of any one of the above-mentioned vision feature matching and positioning methods for port autonomous container trucks.

[0029] To achieve the above object and other related objects, the present invention also provides a computer-readable storage medium, on which a computer program is stored that is programmed or configured to execute any one of the above-mentioned vision feature matching and positioning methods for port autonomous container trucks.

[0030] The present invention has the following positive effects:

[0031] 1. The present invention can achieve stable positioning of autonomous container trucks in special scenarios such as yards and quay cranes in the port environment, providing a reliable guarantee for the stable operation of autonomous container trucks in the port.

[0032] 2. The present invention corrects the initial pose of the vehicle through a lane line matching and positioning algorithm and a bay number matching and positioning algorithm, in combination with a high-precision map, so that the vehicle can operate with high precision.

[0033] 3. The present invention not only improves the driving precision of the vehicle, but also improves the riding comfort. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic flow chart of the method of the present invention;

[0035] Figure 2 It is a flow chart of the lane line matching and positioning algorithm of the present invention;

[0036] Figure 3 It is a flow chart of the bay number matching and positioning algorithm of the present invention;

[0037] Figure 4 It is a schematic diagram of the lane line matching and positioning result of the present invention;

[0038] Figure 5 It is a schematic diagram of the bay number matching and positioning result of the present invention;

[0039] Figure 6 It is a schematic diagram of the lane line and bay number in the port yard area of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] To describe in detail the technical content, the achieved objectives and the effects of the present invention, the following is described in conjunction with the embodiments and the accompanying drawings.

[0041] Embodiment 1: As Figure 1 shown, a vision feature matching and positioning method for an unmanned port container truck includes the following steps:

[0042] S1: The vehicle is driving on a preset road, and based on the on-vehicle camera, the current road condition picture is obtained, and the current road condition picture is subjected to image thresholding processing to output target picture data information;

[0043] S2: Based on the target picture data information, deep learning is performed to obtain a lane line recognition data matrix, a road sign recognition data matrix, and a semantic segmentation data matrix, and feature transformation is performed on the semantic segmentation data matrix to output the relative pose of the vehicle;

[0044] S3: Based on the lane line recognition data matrix and the road sign recognition data matrix, the high-precision map data information is coupled, and the first vehicle pose and the second vehicle pose are output respectively according to the lane line matching and positioning algorithm and the bay number matching and positioning algorithm;

[0045] S4: Obtain the vector feature information of the vehicle's current position according to the pre-collected vector map, couple the relative pose of the vehicle, the first vehicle pose, and the second vehicle pose, and output the corrected vehicle pose and confidence information.

[0046] As Figure 2 shown, in step S3, the lane line matching positioning algorithm includes:

[0047] S311: Determine whether the quality of the vehicle's initial pose meets the requirements. If not, optimize the initial pose and set the confidence level to 0, and output the optimized initial pose. If so, proceed to step S312;

[0048] S312: Based on the vehicle's initial pose, obtain the lane line data information of the road surface at the vehicle's current position, filter and retain the lane line data information of the left and right lanes of the vehicle, and then output the data matrix of the lane line position points according to the high-precision map;

[0049] S313: Transform the data matrix of the lane line position points to the vehicle body coordinate system through the initial pose, and output the optimal offset of the vehicle based on the least squares optimization algorithm;

[0050] S314: Compensate the optimal offset of the vehicle to the initial pose and output the first vehicle pose.

[0051] Among them, in step S313, the calculation formula for transforming the lane line position points to the vehicle body coordinates through the initial pose is:

[0052] , where T is the local compensation pose to be optimized, T0 is the initial pose, (x robot , y robot , 0) is the coordinate point of the lane line position point in the vehicle body coordinate system, (x world , y world , 0) is the lane line position point.

[0053] As Figure 3 shown, in step S3, the bay number matching positioning algorithm includes:

[0054] S321: Based on the road sign recognition data matrix and the high-precision map, obtain the bay number information, obtain the Yardnum where the vehicle is currently located according to the initial pose, and determine whether the Yardnum is successfully obtained. If so, proceed to step S322;

[0055] S322: According to the bay number information, obtain the corresponding Yard value, determine the data required by the camera according to the Yard value, and then search for the corresponding Baynum in the high-precision map according to the Yardnum where the vehicle is currently located, and output the high-precision map Baynum;

[0056] S323: Convert to the vehicle body coordinate system based on the high-precision map Baynum, and output offset data information;

[0057] S324: Compensate the offset data information to the initial pose, and output the second vehicle pose.

[0058] Among them, the Yardnum and the Baynum have a mapping relationship.

[0059] In step S321, if not, optimize the initial pose, set the confidence level to 0, and output the optimized initial pose.

[0060] The vector feature information includes the position, direction, and steering angle information of the vehicle.

[0061] To achieve the above object and other related objects, the present invention also provides a visual feature matching and positioning system for a port autonomous container truck, including a computer device, which is programmed or configured to execute the steps of any one of the above-mentioned visual feature matching and positioning methods for a port autonomous container truck.

[0062] To achieve the above object and other related objects, the present invention also provides a computer-readable storage medium, on which a computer program is stored that is programmed or configured to execute any one of the above-mentioned visual feature matching and positioning methods for a port autonomous container truck.

[0063] Embodiment 2: A visual feature matching and positioning method for a port autonomous container truck, based on Embodiment 1, the lane line positioning principle is: Given a high-precision map with lane lines, use the initial pose to be corrected to find nearby lane lines in the high-precision map. After converting the found lane lines to the vehicle body coordinate system through the initial pose, compare them with the lane lines detected by the camera provided by the perception. The difference is the offset that needs to be corrected to the initial pose. There are many specific calculation methods for the offset. In this project, an optimization-based method is selected. Since the lane line is a curve on a 2D screen, the lane line matching algorithm can only provide corrections for the lateral (X-axis) and yaw angles.

[0064] To find lane lines in the high-precision map, the findBoarder interface in the map API needs to be used, providing the longitude and latitude, and outputting all lane lines within a certain radius of this position. The lane lines are stored in the form of global coordinate points in the high-precision map.

[0065] The lane lines detected by the camera are cubic curves in the vehicle body coordinate system as:

[0066] x = c0 + c1*y + c2*y 2 + c3*y 3 ;

[0067] Where x and y are the lateral and longitudinal coordinates in the vehicle body coordinate system, and c0 - c3 are four curve parameters. Currently, the perception module only outputs the first lane line on the left and the first lane line on the right. After obtaining the two lane lines, the following optimization problem can be established: The variable to be optimized is the pose correction amount, the model is the detected cubic curve, and the input is the high-precision map lane line points.

[0068] The calculation formula for converting the lane line position points to the vehicle body coordinates through the initial pose is:

[0069] , where T is the local compensation pose to be optimized, T0 is the initial pose, (x robot , y robot , 0) is the coordinate point of the lane line position point in the vehicle body coordinates, and (x world , y world , 0) is the lane line position point.

[0070] Assuming T is the compensation pose to be optimized, the following residual can be established:

[0071] ;

[0072] This project uses CERES to solve this optimization problem. It should be noted that here it is assumed that the lane line model is correct. If an incorrect model is input, the solver may converge but output completely wrong results. The abnormal results can be judged by comparing the deviation between the corrected pose and the input pose.

[0073] As Figure 3 shown, the difference between bay number positioning and lane line positioning is that the bay number has global invariance. Once the matching is successful, the bay number can provide very strong constraints. The bay number is organized in the form of a 2D pose (X, Y, YAW) in the high-precision map. This 2D pose naturally corresponds to the axial direction of the vehicle body pose. Therefore, we can directly calculate the deviation of the vehicle body relative to this absolute pose for compensation.

[0074] The high-precision map API provides a findBaynum interface, which can obtain all bay numbers at once. We choose to pre-cache all bay numbers before the program starts. The advantage of doing this is that it can avoid repeatedly calling the interface to obtain bay numbers each time, improving efficiency. And the bay number is unique in each Yard. We can establish a mapping from Yardnum to Baynum and store it using a hash table to further improve the query efficiency.

[0075] The vehicle is equipped with two cameras, one on the left and one on the right. We agree to use the left camera in odd-numbered Yards and the right camera in even-numbered Yards. Currently, the observed Bei position number in the perception output only has x and y in the vehicle body coordinate system, and there is no heading for the time being. Therefore, the Bei position number positioning module can only output the compensation of the global x and y.

[0076] As Figure 6 shown, in the scenes of the yard, under the quay crane, and the mixed vehicle lane, the lane line features are obvious. In this scenario, there may be a deviation in integrated navigation positioning. By matching the lane line information detected by the camera with the vector features in the high-precision map, the lateral deviation of the integrated navigation positioning can be corrected to ensure that the container truck runs on the center line of the lane. When the lane line features are obvious and the matching is good, the lateral accuracy of the lane line feature matching is less than 20 cm.

[0077] As Figure 5 shown, in the yard scene, the Bei position number features are obvious. In this scenario, there may be a deviation in integrated navigation positioning. By matching the Bei position information detected by the camera with the Bei position number features in the high-precision map, the lateral and longitudinal deviations of the integrated navigation positioning can be corrected to ensure that the container truck runs on the center line of the lane. When the Bei position features are obvious and the matching is good, the lateral and longitudinal accuracy of the lane line feature matching is less than 20 cm (1σ).

[0078] In summary, the present invention can achieve stable positioning of the driverless container truck in special scenarios such as the yard and the quay crane in the port environment, providing a reliable guarantee for the stable operation of the driverless container truck in the port.

[0079] The above has described the embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those of ordinary skill in the art in the relevant technical field, various changes can be made without departing from the gist of the present invention.

Claims

1. A vision feature matching and positioning method for driverless container trucks in ports, characterized in that It includes the following steps: S1. The vehicle travels on a preset road, obtains a current road condition picture based on an in-vehicle camera, performs image thresholding processing on the current road condition picture, and outputs target picture data information; S2. Based on the target picture data information, perform deep learning to obtain a lane line recognition data matrix, a road sign recognition data matrix, and a semantic segmentation data matrix, perform feature transformation on the semantic segmentation data matrix, and output the relative pose of the vehicle; S3. Based on the lane line recognition data matrix and the road sign recognition data matrix, couple the high-precision map data information, output the first vehicle pose according to the lane line matching positioning algorithm, and output the second vehicle pose according to the Beiwei number matching positioning algorithm; S4. According to the pre-collected vector map, obtain the vector feature information of the current position of the vehicle, couple the relative pose of the vehicle, the first vehicle pose, and the second vehicle pose, and output the corrected vehicle pose and confidence information; In step S3, the lane line matching positioning algorithm includes: S311. Judge whether the quality of the initial vehicle pose meets the requirements. If not, optimize the initial pose and set the confidence level to 0, and output the optimized initial pose. If so, proceed to step S312; S312. Based on the initial pose of the vehicle, obtain the lane line data information of the road surface at the current position of the vehicle, filter and retain the lane line data information of the left and right lanes of the vehicle, and then output the data matrix of the lane line position points according to the high-precision map; S313. Transform the data matrix of the lane line position points to the vehicle body coordinate system through the initial pose, and output the optimal offset of the vehicle based on the least squares optimization algorithm; S314. Compensate the optimal offset of the vehicle to the initial pose, and output the first vehicle pose; In step S313, the calculation formula for transforming the lane line position points to the vehicle body coordinates through the initial pose is: ， Among them, T is the local compensation pose to be optimized, T0 is the initial pose, (x robot , y robot , 0) is the coordinate point of the lane line position point in the vehicle body coordinate system, (x world , y world , 0) is the lane line position point; In step S3, the Beiwei number matching positioning algorithm includes: S321. Based on the road sign recognition data matrix and the high-precision map, obtain the Beiwei number information, obtain the Yardnum where the vehicle is currently located according to the initial pose, and judge whether the Yardnum is successfully obtained. If so, proceed to step S322; S322. According to the Beiwei number information, obtain the corresponding Yard value, judge the data required by the camera according to the Yard value, and then search for the corresponding Baynum in the high-precision map according to the Yardnum where the vehicle is currently located, and output the high-precision map Baynum; S323. Transform the high-precision map Baynum to the vehicle body coordinate system through the initial pose, and output the offset data information; S324. Compensate the offset data information to the initial pose and output the second vehicle pose.

2. The visual feature matching and positioning method for driverless port container trucks according to claim 1, wherein: In step S322, judging the data required by the camera according to the Yard value includes: obtaining the data of the left camera when obtaining an odd Yard, and obtaining the data of the right camera when obtaining an even Yard.

3. The visual feature matching and positioning method for driverless container trucks in a port according to claim 1, wherein: The Yardnum and the Baynum have a mapping relationship.

4. The visual feature matching and positioning method for driverless container trucks in ports according to claim 1, wherein: In step S321, if the answer is no, the initial pose is optimized and the confidence is set to 0, and the optimized initial pose is output.

5. The visual feature matching and positioning method for driverless container trucks in a port according to claim 1, wherein: The vector feature information includes the position, direction, and steering angle information of the vehicle.

6. A vision feature matching and positioning system for unmanned port container trucks, comprising a computer device, characterized in that, The computer device is programmed or configured to execute the steps of the visual feature matching and positioning method for the port autonomous container truck according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that is programmed or configured to execute the visual feature matching and positioning method for the port autonomous container truck according to any one of claims 1 to 5.

Images