A vehicle positioning method and system for port lock station

Abstract

The application discloses a vehicle positioning method and system for a port lock station, which switches positioning modes according to different situations and positions vehicles based on image processing and matching of markers in the lock station area, has the advantages of reducing redundant storage information, saving computing power, resisting interference caused by changes in environmental brightness and changes in the viewing angle of observed markers, improving the matching success rate, and having the advantages of high positioning accuracy in the lock station, no need for field end reconstruction, easy implementation, and low maintenance cost.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of port automatic driving, and particularly relates to a vehicle positioning method and system for port lock stations. BACKGROUND

[0002] In order to reduce port labor costs, reduce the rate of transportation accidents, and improve port operation efficiency, in recent years, automatic driving technology has been introduced into this scene more and more, and with the maturity of port automatic driving technology, AGVs using automatic driving technology are introduced into the port to replace traditional internal trucks as horizontal transportation equipment to perform horizontal transportation tasks.

[0003] The container unloading operation in the port scene mainly includes the following three links: 1. After the cargo ship arrives at the port, the shore-based container crane unloads the container from the cargo ship to the AGV; 2. The AGV horizontally transports the container to the yard; 3. The yard's field bridge stacking crane unloads the container from the AGV to the yard. Thus, the container loading and unloading operation is completed, and the container loading and unloading operation is the reverse process. In the above second link, during the process of AGV performing horizontal transportation tasks, it needs to pass through the lock station for disassembly and assembly lock operation. Since the passing area in the lock station is narrow, when passing through this scene, the AGV needs to have high lateral positioning accuracy, and the disassembly and assembly lock operation also has high requirements for longitudinal positioning accuracy, so providing a high-precision positioning scheme in the lock station area is a necessary condition for realizing port full-scene automatic driving.

[0004] In the port lock station, the mainstream AGV positioning scheme can be summarized as follows: 1. Based on magnetic nails and reflective studs for positioning; the positioning scheme based on magnetic nails needs to lay a large number of magnetic nails in the operation area, and its disadvantage is that there cannot be other magnetic materials in the navigation route. This scheme will be strongly disturbed when implemented in the lock station; the positioning scheme based on reflective studs also needs to lay reflective studs in the site, and both will face problems such as high operation and maintenance cost and difficult upgrading and reconstruction in the later period. 2. Based on using vehicle-mounted sensor laser radar to obtain surrounding environment point cloud and prior map point cloud for matching to realize positioning; this scheme will be disturbed by dynamic targets in the environment, and the sensor itself is high in cost. 3. Based on two-dimensional code label for positioning; this scheme has high requirements for the surface cleanliness and flatness of the two-dimensional code, and needs to modify the environment in the lock station area. 4. Based on UWB base station for positioning; this scheme needs at least three base stations to form a positioning network, and needs to modify the environment in the lock station area.

[0005] In summary of the above analysis, each method in the prior art has its own shortcomings, including being unable to meet the positioning accuracy requirements in the port lock station, being high in maintenance cost, and needing to modify the environment in the lock station area. SUMMARY

[0006] In view of the above problems, the main purpose of the present application is to design a vehicle positioning method and system for port lock station, which is used to solve the problem of accurate positioning in the lock station area.

[0007] In order to achieve the above purpose, the present application adopts the following technical scheme:

[0008] A vehicle positioning system for port lock station, a global positioning module is arranged outside the lock station, which is used to position the vehicle outside the lock station and provide the vehicle pose to the strategy management module;

[0009] A lock station area positioning module, which includes a vehicle positioning method, is used to position the vehicle entering the lock station area and provide the vehicle pose to the strategy management module;

[0010] A strategy management module, which is used to manage the positioning strategy and output the vehicle pose.

[0011] As a further description of the present application, when the vehicle is performing horizontal operation outside the lock station, the global positioning module is used for positioning.

[0012] As a further description of the present application, when the vehicle arrives at the lock station area, the strategy management module loads the lock station area map and detects the markers, if the markers in the lock station area are detected, the positioning strategy is adjusted to be based on the lock station area positioning module for positioning.

[0013] When the vehicle drives away from the lock station area, the strategy management module changes the positioning strategy and uses the global positioning module for positioning.

[0014] A vehicle positioning method for port lock station, which includes the following steps:

[0015] Step 1: vehicle pose initialization;

[0016] Step 2: load the lock station area map;

[0017] Step 3: obtain the image collected by the camera;

[0018] Step 4: image processing;

[0019] Step 5: same point matching;

[0020] Step 6: estimate the vehicle pose;

[0021] Step 7: positioning output.

[0022] As a further description of the present application, in step 1, the vehicle is positioned based on the global positioning module, and the strategy management module is used to detect whether the vehicle enters the lock station area, so that the current vehicle pose of the global positioning module is used as the initial value.

[0023] As a further description of the present application, in step 2, the pre-value vector element map in the lock station area is loaded by the strategy management module and the identification object is detected, which includes one, multiple or all of the lane boundary, the curb stone, the lock twist machine guide rail base, the gate machine lifting rod and the lock station inner wall;

[0024] It is judged whether the identification object in the lock station area is detected:

[0025] If the identification object cannot be detected, the vehicle is adjusted to re-detect;

[0026] If the identification object is detected, the strategy management module changes the positioning strategy to the positioning module in the lock station area.

[0027] As a further description of the present application, in step 3, the image around the vehicle is collected by the camera carried on the vehicle, and in step 4, the image processing is performed to obtain the detection and segmentation result of the identification object, and then the detection and segmentation result of the identification object is converted into the perception element image containing the point, line and surface elements thereof;

[0028] The image processing includes image stitching and repeated area elimination.

[0029] As a further description of the present application, in step 5, the same name point matching is performed between the perception element image and the map element image; and in step 6, it is judged whether the matching result is valid:

[0030] If the matching score is lower than the threshold value, the image collection range in step 3 is expanded, and the identification object is re-extracted and converted;

[0031] If the matching score is higher than the threshold value, the pose of the ego vehicle is corrected by the solved transformation matrix, and the positioning result is output to the strategy management module.

[0032] As a further description of the present application, in the same name point matching, the perception element image is the image containing the point, line and surface elements of the identification object in the lock station area, and the map element image is the vector element map image containing the point, line and surface elements of the identification object in the lock station area.

[0033] As a further description of the present application, in step 7, according to the position of the current vehicle and the set lock station area geographic fence, it is judged whether the vehicle drives out of the lock station area and the positioning result is output:

[0034] If the vehicle is in the lock station area, and the positioning is performed based on the positioning module in the lock station area, and the strategy management module outputs the positioning result of the positioning module in the lock station area to the downstream;

[0035] If the vehicle is outside the lock station area, or is in the lock station area but the initialization of the positioning module in the lock station area is not completed, the strategy management module outputs the positioning result of the global positioning module to the downstream.

[0036] The technical effect of the present application is as follows relative to the prior art:

[0037] The present application provides a vehicle positioning method and system for a port lock station, which switches positioning modes according to different situations and performs vehicle positioning based on image processing and matching of markers in the lock station area, has the advantages of reducing redundant stored information, saving computing power, resisting interference caused by changes in environmental brightness and changes in the viewing angle of observed markers, improving the matching success rate, and having the advantages of high positioning accuracy in the lock station, no need for field end modification, easy implementation, and low maintenance cost. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 It is a whole scheme implementation process flowchart of the positioning method of the present application.

[0039] Figure 2 It is a flowchart of the positioning method in the lock station area of the present application.

[0040] Figure 3 It is a schematic diagram of the positioning system of the present application. DETAILED DESCRIPTION

[0041] The present application will be described in detail below with reference to the accompanying drawings:

[0042] A vehicle positioning system for a port lock station, as shown in Figure 3 , includes a global positioning module, a lock station area positioning module, and a strategy management module.

[0043] Specifically, the present embodiment analyzes the corresponding modules in detail, and the analysis content is as follows:

[0044] 1. The global positioning module is arranged outside the lock station, and is responsible for positioning vehicles outside the port lock station and providing the pose of the vehicle to the strategy management module throughout the process.

[0045] The global positioning module includes one, multiple, or all of the three positioning modes of image, laser radar, and GPS for combined positioning.

[0046] 2. The lock station area positioning module includes the positioning method for vehicles in the lock station area disclosed in the present embodiment, is responsible for positioning vehicles entering the lock station area, and provides the pose of the vehicle to the strategy management module when the vehicle is in the lock station area.

[0047] In addition, the embodiment also includes other setting modes, for example: a positioning module in the lock station area, including an RFID tag-based positioning scheme, that is, a plurality of RFID electronic tags are laid in the lock station area, and relative position information is obtained through visual detection of the electronic tags, so as to estimate the self-vehicle pose and realize positioning; and a laser radar sensor-based positioning scheme, that is, point cloud information in the lock station area is obtained through scanning, and matching is performed with the point cloud in the lock station area, so that port automatic driving positioning can be realized.

[0048] 3. A policy management module, which is responsible for management of a positioning policy and output of a positioning result, that is, a vehicle pose, to a downstream;

[0049] The policy management module is implemented in the following manner:

[0050] According to a comparison between a current vehicle position and a set lock station area geographic fence, it is judged whether the vehicle is in the lock station area;

[0051] When entering the lock station area, a lock station area map is loaded. When a marker in the lock station area is detected, the positioning policy is changed, and positioning is performed by using the positioning module in the lock station area;

[0052] When the vehicle is positioned by using the positioning module in the lock station area, the policy management module outputs a positioning result of the positioning module in the lock station area to the downstream, and when the vehicle is positioned by using the global positioning module, the policy management module outputs a positioning result of the global positioning module to the downstream.

[0053] That is, when the vehicle performs horizontal work outside the lock station, positioning is performed based on the global positioning module; when the vehicle arrives at the lock station area and detects the marker in the lock station area, the policy management module loads the lock station area map and adjusts the positioning policy to perform positioning based on the positioning module in the lock station area; when the vehicle drives away from the lock station area, the policy management module changes the positioning policy and performs positioning based on the global positioning module.

[0054] Another embodiment of the application is a vehicle positioning method for a port lock station, including the following steps:

[0055] Step 1: vehicle pose initialization;

[0056] Step 2: loading of a lock station area map;

[0057] Step 3: obtaining of an image collected by a camera;

[0058] Step 4: image processing;

[0059] Step 5: same-name point matching;

[0060] Step 6: estimation of a self-vehicle pose;

[0061] Step 7: Positioning output.

[0062] The above disclosed steps are the complete process of the vehicle positioning outside the lock station area, then entering the lock station area, and finally leaving the lock station area outside the lock station area. Specifically, the embodiment analyzes each of the above steps in detail, and the analysis content is as follows:

[0063] In step 1, the vehicle is positioned based on the global positioning module, and whether the vehicle enters the lock station area is detected by the strategy management module. If yes, the current ego pose of the global positioning module is used as the initial value, and the following operation steps are executed.

[0064] More specifically, when the vehicle is performing horizontal transportation work outside the lock station area, positioning is performed based on the global positioning module. At this time, coarse positioning can be performed by relying on the vehicle's GPS information, UWB information, laser positioning, etc.

[0065] In addition, a geographic fence is set in the port lock station area, and whether the vehicle is in the geographic fence is determined based on the global coordinates of the ego vehicle obtained by the above operation:

[0066] a. If the vehicle is not in the geographic fence, positioning is performed based on the global positioning module;

[0067] b. If the vehicle is in the geographic fence, the following operations are performed.

[0068] It should be noted that in the embodiment, the geographic fence is usually determined according to the size of the lock station, for example, the lock station is 18 meters long and 6 meters wide, and the lock station area can be set as a quadrilateral with the lock station as the center, 35 meters long and 8 meters wide. The coordinates of the four corners of the region can be LLA (latitude and longitude coordinates) or navigation coordinates. After estimating the ego pose each time, it can be determined whether the vehicle is in the lock station area by determining whether the vehicle is in the quadrilateral.

[0069] In step 2, the strategy management module loads the pre-value vector element map in the lock station area, and detects and identifies the markers; the markers include one, multiple or all of the lane boundary, the road curb, the lock station machine guide rail base, the gate machine lifting rod, and the lock station inner wall.

[0070] It should be noted that the above-mentioned loaded vector element map can be obtained from a third-party map vendor;

[0071] It is determined whether the markers in the lock station area are detected:

[0072] If the markers cannot be detected, the vehicle yaw angle is adjusted and detection is performed again;

[0073] If the marker is detected, the policy management module changes the positioning strategy to positioning based on the lock-station area positioning module, and inputs the current ego pose as the initial value of the lock-station area positioning module to initialize the lock-station area positioning module.

[0074] In step 3, the images around the vehicle are collected by the cameras loaded on the vehicle, and image processing is performed; the image processing includes image stitching processing and repeated area elimination.

[0075] In step 4, the detection and segmentation results of the markers within a certain range in the image are obtained through image processing, and the obtained detection and segmentation results of the markers are converted into a perception element image containing point, line and surface elements.

[0076] The certain range in the image disclosed in the above content refers to the actual lock-station area on site, for example, 2 meters in the horizontal direction from the left and right sides of the vehicle, and 5 meters in the longitudinal direction from the front and rear of the vehicle.

[0077] The image processing is a bottom-layer data processing, which is then provided for the global positioning module and the lock-station area positioning module.

[0078] The specific image processing method is as follows:

[0079] 1. Time stamp alignment processing is performed on the 2D images collected by the cameras, for example, if the time stamp difference of each camera trigger is within 50 ms, it is considered that the images are collected at the same time, and the pictures with a time stamp difference greater than 50 ms are discarded (under normal circumstances, the time stamp difference is within 50 ms, and the abnormal value may be greater than 50 ms);

[0080] 2. The 2D images collected by the cameras are converted into BEV images (bird's eye view). The specific conversion method can be performed through LSS, BEVDepth, etc.

[0081] 3. The BEV pictures obtained from each camera are spliced to remove the repeated areas;

[0082] 4. Target detection is performed based on the BEV picture, which can be performed through RCNN, Fast-RCNN, YOLO, etc. for marker detection and semantic segmentation;

[0083] 5. The point cloud obtained through marker detection and semantic segmentation is saved in the form of an image;

[0084] 6. The converted perception element image is used for same-name point matching in the next step.

[0085] In step 5, the perception element image and the map element image are matched for same-name points.

[0086] The same name point matching method is to match the perception element image and the map element image, wherein the perception element image is an image containing the point, line and surface elements of the landmarks in the lock station area, and the map element image is a vector element map image containing the point, line and surface elements of the landmarks in the lock station area.

[0087] The specific process of the same name point matching method is that the matching score obtained by matching the points with the same semantics (such as lane lines, straight arrows, stop lines, etc.) in the perception element image and the map element image is the highest through the optimization methods such as the Gauss-Newton method and the Levenberg-Marquardt method.

[0088] The traditional visual positioning is based on the matching of the original image collected by the camera. The landmarks collected by the camera are disturbed by the changes of the environmental brightness and the viewing angle of the observed landmarks, which reduces the matching success rate. In the embodiment, the same name point matching is performed based on the perception element image and the map element image, that is, the real objects in the camera original image are abstracted into vector elements for matching, so as to avoid the influence of the above disturbances on the matching.

[0089] In step 6, it is judged whether the matching result is valid or not.

[0090] If the matching score is lower than the threshold value, the image processing range in step 4 is expanded, and the landmarks are re-extracted and converted.

[0091] If the matching score is higher than the threshold value, the pose of the ego vehicle is corrected through the solved transformation matrix, and the positioning result is output to the strategy management module.

[0092] The threshold value is usually set to 0.5. When the matching score is greater than 0.5, it means that 50% of the perception element image and the map element image are matched successfully, that is, the matching result is valid.

[0093] In step 7, it is judged whether the vehicle is in the lock station area and the positioning result is output according to the comparison between the position of the current vehicle and the set geographical fence of the lock station area.

[0094] If the vehicle is still in the lock station area, it returns to step 3 to continue positioning based on the in-lock station area positioning module.

[0095] If the vehicle drives out of the lock station area, the global map is loaded, and positioning is performed based on the global positioning module.

[0096] It should be further pointed out that if the vehicle is in the lock station area and positioning is performed based on the in-lock station area positioning module, the strategy management module outputs the positioning result of the in-lock station area positioning module to the downstream (usually referring to the path planning module and the vehicle information visualization module).

[0097] If the vehicle is outside the locking station area, or is within the locking station area but the initialization of the positioning module in the locking station area has not been completed, the policy management module outputs the positioning result of the global positioning module to the downstream.

[0098] In this embodiment, the markers in the lock station area are converted into perception element images containing their point, line and surface elements, and the perception element images are matched with the map element images for points of the same name, thereby improving the matching success rate. It has the advantages of high positioning accuracy in the lock station area, being able to resist interference caused by changes in ambient brightness and changes in the viewing angle of observing markers, and requiring no scene modification, being easy to implement and having low maintenance costs.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the scope of the present invention. Other modifications or equivalent substitutions made to the technical solutions of the present invention by ordinary technicians in this field should be included in the scope of the claims of the present invention as long as they do not depart from the spirit and scope of the technical solutions of the present invention.

Claims

1. A vehicle positioning system for port lock stations, characterized in that: include: The global positioning module is set outside the locking station to locate the vehicle outside the locking station and provide the vehicle's position and posture to the policy management module; The positioning module in the locking station area includes a method for implementing vehicle positioning, locating vehicles entering the locking station area, and providing the vehicle's position and posture to the policy management module; Strategy management module, positioning strategy management, output vehicle posture; When the vehicle is operating horizontally outside the locking station, positioning is performed based on the global positioning module; When the vehicle arrives at the lock station area, the strategy management module loads the lock station area map and detects and identifies objects. If a marker is detected in the lock station area, the positioning strategy is adjusted to positioning based on the positioning module in the lock station area; When the vehicle leaves the locking station area, the policy management module changes the positioning policy and performs positioning based on the global positioning module.

2. A vehicle positioning method for a port lock station based on the positioning system of claim 1, characterized in that: The steps include: Step 1: Vehicle pose initialization; Step 2: Load the lock station area map; Step 3: Get the image captured by the camera; Step 4: Image processing; Step 5: Matching of points with the same name; Step 6: Estimate the vehicle’s pose; Step 7: Positioning Output.

3. The vehicle positioning method for a port lock station according to claim 2, characterized in that: In step 1, the vehicle is positioned based on the global positioning module, and the policy management module detects whether the vehicle enters the lock station area. The current vehicle posture of the global positioning module in the lock station area is used as the initial value.

4. The vehicle positioning method for a port lock station according to claim 2, characterized in that: In step 2, the policy management module loads a pre-valued vector element map within the locking station area and detects and identifies objects, including lane edges, curbstones, locking machine guide rail bases, gate lifts, and locking station inner walls. Determine whether the lock station area contains any markers: If the marker cannot be detected, adjust the vehicle and retest; If the marker is detected, the policy management module changes the positioning policy to the positioning module within the locked station area.

5. The vehicle positioning method for a port lock station according to claim 2, characterized in that: In step 3, the camera mounted on the vehicle captures images of the vehicle's surroundings, and in step 4, image processing is performed to obtain the detection and segmentation results of the markers. The obtained detection and segmentation results of the markers are then converted into a perception element image containing their point, line, and surface elements. Image processing includes image stitching and removal of duplicate areas.

6. The vehicle positioning method for a port lock station according to claim 5, characterized in that: In step 5, the perception element image and the map element image are matched for points of the same name; and in step 6, whether the matching result is valid is determined: If the matching score is lower than the threshold, the image acquisition range in step 3 is expanded and the markers are re-extracted and converted; If the matching score is higher than the threshold, the vehicle's position is corrected using the solved transformation matrix, and the positioning result is output to the strategy management module.

7. The vehicle positioning method for a port lock station according to claim 6, characterized in that: In the matching of homonymous points, the perception element image is an image containing the point, line and surface elements of the markers in the lock station area, and the map element image is a vector element map image containing the point, line and surface elements of the markers in the lock station area.

8. The vehicle positioning method for a port lock station according to claim 2, characterized in that: In step 7, the current vehicle location is compared with the set lock station area geo-fence to determine whether the vehicle has left the lock station area and output the positioning result: If the vehicle is within the locking station area, it is positioned based on the positioning module within the locking station area, and the policy management module outputs the positioning results of the positioning module within the locking station area to the downstream; If the vehicle is outside the locking station area, or is within the locking station area but the initialization of the positioning module in the locking station area has not been completed, the policy management module outputs the positioning result of the global positioning module to the downstream.

Images