Vehicle, vehicle positioning method and apparatus, device, and computer-readable storage medium

By determining a lateral offset using lane boundary features in road maps, the method improves vehicle positioning accuracy and reduces computational complexity, ensuring reliable and timely vehicle operations even in GPS-inadequate environments.

JP2024540985AActive Publication Date: 2024-11-06YINWANG INTELLIGENT TECHNOLOGIES CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2024523921
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2021-10-22
Publication Date
2024-11-06
Estimated Expiration
2041-10-22

AI Technical Summary

Technical Problem

Current vehicle positioning systems using GPS and IMU experience reduced accuracy due to GPS signal loss, leading to high computational and time resource consumption, especially in environments where GPS is inadequate, affecting the reliability and real-time performance of unmanned or intelligent vehicle operations.

Method used

A vehicle positioning method that determines a lateral offset of the vehicle based on lane boundary features in a road feature map, combining this with a local map and sensor data to reduce computational complexity and improve accuracy, using a camera to capture road images and extract lane boundaries for precise positioning.

Benefits of technology

This method enhances vehicle positioning accuracy and reliability by narrowing the search range for image matching, reducing computational resource consumption and time, ensuring safe and timely operation in various driving conditions, including GPS signal loss.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024540985000001_ABST
    Figure 2024540985000001_ABST
Patent Text Reader

Abstract

This application relates to the field of intelligent driving, in particular to a vehicle, a vehicle positioning method and apparatus, a device and a computer-readable storage medium. In the embodiment of this application, a lateral offset is first determined by using a road feature map of a forward road image, and then a second position of the vehicle is determined based on the road feature map, a first pose, a local map corresponding to the first pose and the lateral offset. This can reduce the computational resource consumption and time consumption while improving the accuracy of vehicle positioning, thereby simultaneously improving the reliability and timeliness of vehicle positioning, and further enhancing driving safety.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present application relates to the field of intelligent driving, and in particular to a vehicle, a vehicle positioning method and apparatus, a device, and a computer-readable storage medium. [Background technology]

[0002] Vehicles depend heavily on positioning technology. Positioning is a prerequisite for other vehicle functions, and positioning accuracy directly affects the safety of the vehicle. Generally, the joint positioning of a global positioning system (GPS) and an inertial measuring unit (IMU) is used to achieve high-precision positioning of a vehicle. However, when the GPS has a poor signal or fails, drift will occur in the joint positioning of the GPS and the IMU, resulting in a reduced positioning accuracy.

[0003] Currently, in an environment where GPS fails or has insufficient signals, IMU is constrained to use laser and visual methods to correct the positioning results. Such methods require capturing and storing the feature map of the positioning environment as visual features, such as ORB (Oriented FAST and Rotated BRIEF) features or FAST (Features From Accelerated Segment Test) features, and performing feature matching in a wide range or even the entire map by using a point matching method such as the Iterative Closest Point (ICP) algorithm or Kd-Tree. This results in a large amount of data and high computational complexity, with serious waste in both computational and time resources. In this case, the hardware cost is high, while the work efficiency is low, and real-time vehicle positioning cannot be reliably performed, which brings safety risks to unmanned or intelligent driving of vehicles. Summary of the Invention

[0004] In order to solve the above technical problems, the embodiments of this application provide a vehicle, a vehicle positioning method and apparatus, a device, and a computer-readable storage medium for improving the accuracy of vehicle positioning while reducing computational resource consumption and time consumption.

[0005] A first aspect of the present application provides a vehicle positioning method, comprising: acquiring an image of the road ahead of the vehicle; obtaining a road feature map based on a forward road image; determining a lateral offset of the vehicle based on a road feature map; obtaining a second position of the vehicle based on a lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; Includes.

[0006] Since the lateral offset may indicate the lateral road distance of the vehicle, the lateral offset and the first pose may be combined to narrow the range of image matching, thereby effectively reducing the amount of calculation and the complexity of calculation while improving the accuracy of vehicle positioning.

[0007] In a possible implementation of the first aspect, the step of determining the lateral offset of the vehicle based on the road feature map includes the step of determining the lateral offset of the vehicle based on lane boundary line features in the road feature map. The lane boundary line features are reliable and complete. Therefore, by using the lane boundary line features as the basis for the lateral offset, an accurate and reliable lateral offset can be obtained.

[0008] In a possible implementation of the first aspect, the lane boundary line feature in the road feature map includes two lane boundary line features in a region of interest ROI in the road feature map, and the two lane boundary lines are located on the left and right sides of the first pose. The data amount of the ROI in the road feature map is small, and the lane boundary line features are complete and reliable. Therefore, the accuracy and reliability of the lateral offset of the vehicle can be further improved.

[0009] In a possible implementation manner of the first aspect, the step of determining the lateral offset of the vehicle based on the lane boundary line feature in the road feature map includes the steps of obtaining the lateral offset of the vehicle based on the lateral pixel offset and a preset pixel ratio, and determining the lateral pixel offset based on the lane boundary line feature in the top view of the ROI in the road feature map, and the lane boundary line feature in the top view of the ROI in the road feature map is obtained by using the features of two lane boundary lines in the ROI in the road feature map. The method of obtaining the lateral offset based on the lateral pixel offset and the pixel ratio is easy to implement. Therefore, the calculation complexity can be further reduced.

[0010] In the possible implementation of the first aspect, the lateral pixel offset is the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, the lane center point is the lane center point between the left and right lane boundary lines of the first pose, and the first camera is the camera that captures the forward road image. The position of the optical center of the first camera is easy to obtain, and the calculation amount of the coordinate transformation of the position is small. In this way, the calculation amount and the calculation complexity can be further reduced.

[0011] In a possible implementation of the first aspect, the vehicle positioning method further includes obtaining a second attitude of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map. In this way, an accurate attitude of the vehicle can be obtained in real time.

[0012] In a possible implementation of the first aspect, the second position or the second attitude or both of the vehicle are determined based on a plurality of candidate pose points in a local map, and the pose of each candidate pose point is determined based on the lateral offset of the vehicle, the first pose and the local map. In this way, the vehicle can be accurately positioned by using the candidate pose point selected by the lateral offset. The amount of calculation is small and the calculation complexity is low. This can effectively reduce the calculation resource consumption and the time consumption.

[0013] In a possible implementation of the first aspect, multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the correction position as the center, and the correction position is obtained based on the lateral offset of the vehicle and based on the lane center point position and road azimuth angle that correspond to the first pose and are in the local map. In this way, by using multiple candidate pose points in the same lane direction, the vehicle can be accurately located. The search range is smaller, which can further reduce the amount and complexity of calculation while improving the accuracy of vehicle positioning.

[0014] In a possible implementation of the first aspect, the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second pose is pose information of the candidate pose point selected by performing feature matching on the projection image and the road feature map, and the projection image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map. The degree of match between the projection image and the road feature map can indicate the degree of closeness between the corresponding candidate pose point and the actual vehicle pose, so that an accurate and reliable vehicle positioning result can be obtained.

[0015] In a possible implementation of the first aspect, the vehicle positioning method further includes: acquiring a third pose of the vehicle based on the second position, the first attitude of the vehicle and the sensor positioning data; or acquiring a third pose of the vehicle based on the second position, the second attitude of the vehicle and the sensor positioning data. In this way, through the fusion of multiple types of position data, a third attitude with higher accuracy and better reliability can be acquired, thereby further improving the accuracy and reliability of the vehicle positioning.

[0016] In a possible implementation of the first aspect, the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information and chassis information.

[0017] In a possible realization of the first aspect, the local map is from a vector map. The vector map information is complete and has high accuracy, which allows the accuracy of the vehicle positioning to be further improved.

[0018] A second aspect of the present application provides a vehicle positioning device, an image capture unit configured to capture an image of the road ahead of the vehicle; a feature acquisition unit configured to acquire a road feature map based on a forward road image; an offset determination unit configured to determine a lateral offset of the vehicle based on the road feature map; a position determining unit configured to obtain a second position of the vehicle based on a lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; Includes.

[0019] In a possible implementation of the second aspect, the feature acquisition unit is specifically configured to determine the lateral offset of the vehicle based on lane boundary features in the road feature map.

[0020] In a possible implementation of the second aspect, the lane boundary line features in the road feature map include two lane boundary line features in a region of interest ROI in the road feature map, the two lane boundary lines being located to the left and right of the first pose.

[0021] In a possible implementation manner of the second aspect, the feature acquisition unit is specifically configured to acquire a lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio, where the lateral pixel offset is determined based on lane boundary line features in a top view of the ROI in the road feature map, and the lane boundary line features in the top view of the ROI in the road feature map are acquired by using features of two lane boundary lines in the ROI in the road feature map.

[0022] In a possible implementation of the second aspect, the lateral pixel offset is the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, where the lane center point is the lane center point between the left and right lane boundary lines of the first pose, and the first camera is the camera that captures the forward road image.

[0023] In a possible implementation manner of the second aspect, the vehicle positioning device further includes an attitude determination unit configured to obtain a second attitude of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map.

[0024] In a possible implementation of the second aspect, a second position or a second pose, or both, of the vehicle is determined based on a plurality of candidate pose points in the local map, and a pose of each candidate pose point is determined based on a lateral offset of the vehicle, the first pose, and the local map.

[0025] In a possible implementation of the second aspect, multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the corrected position as the center, and the corrected position is obtained based on the lateral offset of the vehicle and based on the lane center point position and road azimuth angle that correspond to the first pose and are in the local map.

[0026] In a possible implementation manner of the second aspect, the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second posture is posture information of a candidate pose point selected by performing feature matching on the projection image and the road feature map, and the projection image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

[0027] In a possible implementation manner of the second aspect, the vehicle positioning device further includes a fusion unit configured to obtain a third pose of the vehicle based on the second position, the first attitude of the vehicle and the sensor positioning data, or to obtain the third pose of the vehicle based on the second position, the second attitude of the vehicle and the sensor positioning data.

[0028] In a possible implementation of the second aspect, the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information and chassis information.

[0029] In a possible realization of the second embodiment, the local map is from a vector map.

[0030] A third aspect of the present application provides a computing device including a processor and a memory, the memory storing program instructions which, when executed by the processor, enable the processor to perform the vehicle positioning method of the first aspect.

[0031] A fourth aspect of the present application provides a computer-readable storage medium storing program instructions that, when executed by a computer, enable the computer to perform the vehicle positioning method of the first aspect.

[0032] A fifth aspect of the present application provides a computer program product including a computer program which, when executed by a processor, enables the processor to perform the vehicle positioning method of the first aspect.

[0033] A sixth aspect of the present application provides a vehicle including a first camera configured to capture a forward road image and the vehicle positioning apparatus of the second aspect or the computing device of the third aspect.

[0034] In a possible implementation of the sixth aspect, the vehicle further includes one or more of the following: a GPS sensor, an IMU, a speed sensor, and an acceleration sensor.

[0035] In the embodiment of this application, the lateral offset is first determined, and then the second position of the vehicle is determined based on the lateral offset. This can effectively narrow the scope of complex processing such as image matching, and reduce the amount of calculation and the complexity of calculation. Thus, in the embodiment of this application, the accuracy of vehicle positioning can be improved while the calculation resource consumption and time cost can be reduced. When GPS is broken or has insufficient signal or other various cases, the accuracy, reliability and timeliness of vehicle positioning can be improved synchronously, thereby improving the driving safety of the vehicle in various driving modes such as unmanned driving and intelligent driving.

[0036] These and other aspects of the present application will become more apparent and easier to understand in the description of the embodiment(s) that follows. [Brief description of the drawings]

[0037] The features and relationships between features of this application are further described below with reference to the attached drawings. The attached drawings are all examples, and some features are not drawn to scale. Furthermore, in some of the attached drawings, common features in the field of this application that are not essential to this application may be omitted. Alternatively, additional features that are not essential to this application are shown. The combination of features shown in the attached drawings is not intended to limit this application. Furthermore, in this specification, the same reference numerals represent the same contents. Specific attached drawings are described as follows: [Figure 1] FIG. 2 is an exemplary diagram of an application scenario according to an embodiment of the present application. [Diagram 2] 1 is a schematic flowchart of a vehicle positioning method according to an embodiment of the present application. [Diagram 3] FIG. 2 is a schematic diagram of a road feature map in an example according to an embodiment of the present application. [Figure 4] FIG. 2 is a schematic diagram of a road feature map obtained by separating lane boundary features in an example according to an embodiment of the present application; [Diagram 5] 2 is a schematic diagram of an ROI in a camera coordinate system in an example according to an embodiment of the present application; [Figure 6] 2 is an exemplary diagram of an ROI in a road feature map in an example according to an embodiment of the present application. [Figure 7] FIG. 2 is a schematic diagram of a top view of an ROI in a road feature map in an example according to an embodiment of the present application. [Figure 8] 1 is a schematic diagram of a correction position, a first position, a lateral offset, and a relationship between the correction position, the first position, and the lateral offset in a reference coordinate system in an example according to an embodiment of the present application. FIG. [Figure 9] FIG. 2 is a schematic diagram of candidate pose points in an example according to an embodiment of the present application; [Figure 10] 1 is a schematic diagram of a specific implementation process of a vehicle positioning method according to an embodiment of this application; [Figure 11] 1 is a schematic diagram of the structure of a vehicle positioning device according to an embodiment of this application; [Figure 12] 1 is a schematic diagram of the structure of a computing device according to an embodiment of the present application. [Figure 13] 1 is a schematic diagram of a vehicle structure in an example according to an embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0038] In this specification and the claims, terms such as "first," "second," or similar terms such as unit A and unit B are used merely to distinguish between similar objects and do not indicate a particular order of the objects. It will be understood that the particular order or sequence may be interchanged where possible, thereby allowing the embodiments of this application described herein to be realized in orders other than those illustrated or described herein.

[0039] In the following description, the reference numerals indicating steps such as S210 and S220 do not necessarily indicate that the steps are performed in order, and the steps may be interchanged, if possible, or may be performed simultaneously.

[0040] The following describes important and related terms in the embodiments of this application.

[0041] World coordinate system: Also called the measurement coordinate system or objective coordinate system, it may be used as a reference for describing the three-dimensional position and three-dimensional posture of the camera and the measurement object. The world coordinate system is an absolute coordinate system of the objective three-dimensional world. Usually, the coordinate value Pw (Xw, Yw, Zw) on the three-dimensional coordinate axis indicates the three-dimensional position of the object, and the rotation angle (raw, pitch, yaw) of the object relative to the three-dimensional coordinate axis indicates the three-dimensional posture of the object. In the embodiment of this application, the world coordinate system may be used as a reference coordinate system of the global pose.

[0042] Camera coordinate system: The optical center of the camera is used as the coordinate origin. The Z axis and the optical axis are coincident and point to the front of the camera. The X axis points to the right of the camera. The Y axis points to the bottom of the camera. Usually, Pc(Xc,Yc,Zc) denotes the coordinate values ​​in the camera coordinate system.

[0043] External parameters of the camera: The relative positional relationship between the camera coordinate system and the world coordinate system may be determined. The parameters transformed from the world coordinate system to the camera coordinate system may include a rotation matrix R and a translation vector T. Pinhole imaging is used as an example. The external parameters of the camera, the world coordinates and the camera coordinates satisfy the relationship (1): Pc=RPw+T (1), where Pw is the coordinate value (Xw, Yw, Zw) in the world coordinate system, Pc is the coordinate value (Xc, Yc, Zc) in the camera coordinate system, T=(Tx, Ty, Tz) is the translation vector, and R=R(α, β, γ) is the rotation matrix, where the rotation angle around the Z axis of the camera coordinate system is γ, the rotation angle around the Y axis is β, and the rotation angle around the X axis is α. That is, the six parameters α, β, γ, Tx, Ty and Tz are the external parameters of the camera.

[0044] Intrinsic parameters of the camera: determine the projection relationship from the three-dimensional space to the two-dimensional image, and are related to the camera only. A small hole imaging model is used as an example. If image distortion is not considered, the intrinsic parameters may include the scale factor of the camera in two coordinate axes u and v in the pixel coordinate system, the principal point coordinate (x0, y0) relative to the image coordinate system, and the coordinate axis tilt parameter s. The scale factor in the u axis is the ratio of the physical length dx of each pixel in the x direction of the image coordinate system to the focal length f of the camera. The scale factor in the v axis is the ratio of the physical length dy of the pixel in the y direction of the image coordinate system to the focal length f of the camera. If image distortion is considered, the intrinsic parameters may include the scale factor of the camera in two coordinate axes u and v in the pixel coordinate system, the principal point coordinate relative to the imaging plane coordinate system, the coordinate axis tilt parameter, and the distortion parameter. The distortion parameters may include three radial distortion parameters and two tangential distortion parameters of the camera. The intrinsic and extrinsic parameters of the camera may be obtained through calibration.

[0045] The pixel coordinate system, also called the pixel coordinate system or the orthographic pixel coordinate system, is an image coordinate system in units of pixels. The upper left vertex of the image plane is used as the origin, the u axis points horizontally to the right, and the v axis points vertically down, and the u and v axes are parallel to the X and Y axes of the camera coordinate system, respectively. Usually, p(u,v) denotes a coordinate value in the pixel coordinate system. The pixel coordinate system indicates the location of a pixel in an image in units of pixels, while the image coordinate system indicates the location of a pixel in an image in physical units (e.g., millimeters).

[0046] Perspective projection: It is a perspective projection relationship from a 3D camera coordinate system to a 2D pixel coordinate system. Because the origin positions of the coordinate systems do not match and the scale sizes do not match, perspective projection involves stretching and translation transformations. A pinhole camera model is used as an example. The perspective projection relationship from a point (Xc, Yc, Zc) in the camera coordinate system to (u, v) in the pixel coordinate system satisfies the transformation formula shown in the following formula (2).

number

[0047] (u,v) are coordinate values ​​in the pixel coordinate system, (Xc,Yc,Zc) are coordinate values ​​in the camera coordinate system, and K is a matrix representation of the internal parameters of the camera.

[0048] If image distortion is not taken into consideration, equation (2) may be further expressed as equation (3).

number

[0049] f x is the ratio of the focal length f of the camera to the physical length dx of each pixel in the x direction of the image coordinate system, and indicates that the focal length in the x direction is described by using pixels. y is the focal length of the camera f relative to the physical length dy of each pixel in the y direction of the image coordinate system, indicating that the focal length in the y direction is described using pixels. (u0,v0) denotes the coordinates of the intersection between the optical axis of the camera and the image plane (i.e., the principal point) in the pixel coordinate system, where u0 is the ratio of the horizontal coordinate x0 of the principal point in the image coordinate system to dx, and v0 is the ratio of the vertical coordinate y0 of the principal point in the image coordinate system to dy.

[0050] The top view may also be referred to as an aerial view, and coordinate values ​​may be transformed from the orthographic pixel coordinate system to the top view pixel coordinate system through an affine transformation.

[0051] Affine transformation: An affine transformation is a transformation in which, under the three-point collinearity condition of the perspective center, image points and target points, the projection plane (i.e., the perspective plane) rotates around the trace line (i.e., the perspective axis) by a certain angle according to the perspective rotation law, destroying the original projection beam, but allowing the geometry of the projection on the projection plane to still remain unchanged.

[0052] The general formula for the affine transformation is the following formula (4).

number

[0053] (u,v) indicates the original coordinates, expressed in the form of an extended vector, and w = 1. In the affine transformation matrix M, a 11 , a 12 , a 21 and a 22 indicates a linear transformation (e.g., scaling, clipping, and rotation). 31 and a 32 is used for translation. 13 and a 23 is used to generate the perspective transformation.

[0054] Usually, the transformed coordinates (x, y) satisfy the following equation (5).

number

[0055] When the transformation matrix is ​​known, the following equations (6) and (7) may be obtained according to the transformation equations (5) and (4), and the transformed coordinates may be directly obtained by using the original coordinates according to the equations (6) and (7).

number

[0056] Here, the values ​​of the components of the affine transformation matrix M may be obtained through calculation according to the above formula by using the original coordinates and transformed coordinates of the three point pairs.

[0057] Global pose: Also called absolute pose, may include the position and attitude of an object in a reference coordinate system. A vehicle is used as an example. The position of an object may be indicated by using three-dimensional coordinate values ​​in the reference coordinate system (e.g., coordinate values ​​Pw(Xw,Yw,Zw) in the above world coordinate system). The attitude of an object may be indicated by using the pitch, yaw (also called yaw angle) and raw angle of the vehicle. The roll angle is the rotation angle around the X axis, the pitch angle is the rotation angle around the Y axis and the yaw angle is the rotation angle around the Z axis. Alternatively, the attitude of an object may be described by using a quaternion. In the embodiment of this application, the reference coordinate system of the global pose may be, but is not limited to, the above world coordinate system (including coordinate values ​​and attitude angles), a geodetic coordinate system, a Universal Transverse Mercator (UTM) Grid System (also called UTM coordinate system), etc.

[0058] Below, we first provide a brief analysis of possible implementation methods.

[0059] As mentioned above, when the GPS fails or has insufficient positioning effect, or in other cases, drift will occur in the joint positioning of the GPS and IMU, resulting in reduced positioning accuracy.

[0060] In a possible implementation, the positioning system includes a visual positioning device and an inertial navigation device. The inertial navigation device includes an inertial measurement unit. The inertial navigation device obtains the state information of the inertial measurement unit (including the specific force acceleration and Euler angles of the inertial measurement unit in the carrier coordinate system). The visual positioning device determines the linear acceleration of the inertial measurement unit based on the state information, and then determines the position information of the unmanned vehicle based on the visual information and the linear acceleration of the inertial measurement unit. This implementation can solve the problem of low positioning accuracy of the unmanned vehicle. However, in this implementation, there exists an accumulated error, the data volume of the visual information is large, and the calculation complexity is high. Therefore, the positioning accuracy in this implementation cannot reach the centimeter level, and the calculation resource consumption is large.

[0061] In the second possible implementation manner, the visual positioning method includes the steps of: acquiring an image of a marker in the environment where positioning needs to be performed in advance, and establishing a corresponding marker database; performing image matching between the recognition target image and the image in the marker database after the recognition target image of the marker in the environment where positioning needs to be performed is captured; and obtaining the current position through calculation based on the known geometric features in the matched image. This implementation manner can solve the problem of low positioning accuracy when GPS is broken or has insufficient positioning effect. However, positioning is mainly performed by using images, and a large amount of image matching processing needs to be performed. Therefore, this implementation manner also has the problem that the positioning accuracy cannot reach the centimeter level, and the calculation resource consumption is excessively large.

[0062] In view of this, the embodiments of this application provide a vehicle, a vehicle positioning method and apparatus, a device, and a computer-readable storage medium. First, a lateral offset is determined based on a road feature map of a forward road image, and then a second position of the vehicle is obtained based on the lateral offset, a first pose of the vehicle, and a local map. Since the lateral offset can indicate the lateral road distance of the vehicle, the lateral offset and the first pose are combined, so that the search range of the local map can be narrowed, that is, the range of image matching can be narrowed. This can effectively reduce the amount of calculation and the complexity of calculation while improving the accuracy of vehicle positioning. In this way, in the embodiments of this application, when the GPS is broken or has an insufficient signal, or in various other cases, the accuracy, reliability, and timeliness of vehicle positioning can be improved simultaneously.

[0063] The embodiments of the present application may be applicable to various scenarios where precise positioning is required. In particular, the embodiments of the present application are particularly applicable to the positioning of vehicles, ships, aircraft, unmanned aerial vehicles, and other means of transport, such as real-time precise positioning in the driving process of vehicles, positioning of unmanned guided vehicles in logistics scenarios, and real-time precise positioning of unmanned aerial vehicles in outdoor environments. Furthermore, the embodiments of the present invention may be applicable to various environments, such as indoors, outdoors, roads, and fields.

[0064] The "vehicle" in the embodiments of this application may be any type of transportation means. For example, the "vehicle" here may be, but is not limited to, a private car, a commercial vehicle, a bus, a passenger car, a high-speed rail, a subway, an unmanned vehicle, an unmanned aerial vehicle, a logistics transportation vehicle, an unmanned transportation vehicle, etc. The power type of the "vehicle" may be fuel-driven, pure electric drive, hydrogen fuel cell drive, hybrid electric drive, etc. Furthermore, the "vehicle" here may be a human-powered vehicle, an autonomous vehicle, an unmanned vehicle, or other types of vehicles. Those skilled in the art may understand that any transportation means that needs to be located in real time may be considered as a "vehicle" in the embodiments of this application.

[0065] Figure 1 is a schematic diagram of an exemplary application scenario according to an embodiment of this application. In the scenario of Figure 1, vehicle A, vehicle B, vehicle C and vehicle D on a road may all use the solution in the embodiment of this application to determine their respective positions in real time. For example, in scenarios where GPS is down or has insufficient signals, scenarios with low accuracy of vehicle sensors, scenarios in tunnels, underground garages, or various other scenarios, real-time accurate and reliable positioning of vehicles can be achieved by using the embodiment of this application.

[0066] For example, see Figure 1. The positioning location of the vehicle may be indicated by the location of the rear axle center of the vehicle (solid black dot in Figure 1).

[0067] Specific implementation methods of the embodiments of this application will be described in detail below.

[0068] 2 is a schematic flowchart of a vehicle positioning method according to an embodiment of this application. Please refer to FIG. 2. The vehicle positioning method provided in this embodiment of this application may include the following steps:

[0069] Step S210: Obtain an image of the road ahead of the vehicle.

[0070] The road ahead image may be captured by using a first camera mounted on the vehicle, and the road ahead image includes lane boundaries and markers of the road ahead of the vehicle, where the markers may include, but are not limited to, all traffic signs and road markings, such as traffic lights, underground parking poles, walls, toll booths, road gates, collision barriers, obstacles, parking positions, parking position lines, no parking areas, pedestrian crossings, deceleration belts, deceleration, stop, yield lines, and yields, road sign boards, utility poles, road edges, trees, shrubs, medians, guard rails, etc. In addition, the road ahead image may further include lane center lines, guide lines, guide arrows, etc. of the road.

[0071] The road ahead image is used to determine a second position and / or a second attitude of the vehicle based on the first pose of the vehicle, and the second pose formed by the second position and the second attitude is a pose subsequent to the first pose. In other words, the road ahead image is a road ahead image captured during a period from when the vehicle reaches the first pose until the vehicle has not yet reached or has just reached the second pose. For example, the capture time of the road ahead image may be the following times: when the vehicle reaches the first pose, when the vehicle reaches the second pose, or when the vehicle has not yet reached the second pose until the vehicle has reached the first pose.

[0072] The road ahead image may be, but is not limited to, a Time of Flight (TOF) image, a Red Green Blue (RGB) image, or other types of images. The first camera may be, but is not limited to, a TOF camera, an RGB camera, etc. The arrangement method, installation position, specific type and number of the first camera, and the specific type of the road ahead image, etc. are not limited in the embodiments of this application.

[0073] Step S220: Obtain a road feature map based on the road ahead image.

[0074] In some embodiments, the road feature map of the forward road image may be obtained by performing one or more operations on the forward road image, such as semantic feature extraction, post-fit processing (e.g., fitting incomplete lane boundary lines to long straight lines), and feature skeletonization (e.g., lane boundary line feature skeletonization, utility pole feature skeletonization, guide line contouring, and signboard contouring). The specific implementation manner of obtaining the road feature map is not limited in the embodiments of this application.

[0075] In some embodiments, the road feature map may be a grayscale map, and the pixel value range of the road feature map is [0, 255]. In a specific application, different types of road features may be indicated by using pixel values. For example, the pixel value of the lane boundary feature may be preset to 255, the pixel value of the utility pole feature may be set to 135, the pixel value of the guide arrow feature may be set to 95, and the pixel value of the signboard feature may be set to 45. In this way, in order to efficiently and accurately extract the required features from the road feature map, different types of road features in the road feature map may be distinguished by using pixel values. For example, when the lane boundary feature in the road feature map needs to be extracted, only the pixel whose pixel value in the road feature map is 255 needs to be extracted. If the pixel value of the pixel in the road feature map is 135, this indicates that the pixel is a feature point of the utility pole.

[0076] In some embodiments, the road feature map may include lane boundary line features and marker features of the road in the forward road image. Figure 3 is an exemplary diagram of a road feature map. In the example in Figure 3, the road feature map may include lane boundary line features 31, utility pole features 32, traffic light features 33, etc. From Figure 3, it can be seen that the road feature map includes two lane boundary line features located on the left and right sides of the first pose.

[0077] Step S230: Determine a lateral offset of the vehicle based on the road feature map.

[0078] In some embodiments, the lateral offset of the vehicle may indicate the road lateral distance from the vehicle (e.g., the rear axle center of the vehicle or the optical center of the first camera on the vehicle) to the lane centerline. The lateral offset may be, but is not limited to, an actual lateral offset or a lateral pixel offset. The actual lateral offset is the lateral offset in a reference coordinate system. The lateral pixel offset is the lateral offset in a top view of the road feature map.

[0079] In some embodiments, the lateral offset of the vehicle may be determined based on lane boundary line features in the road feature map. The lane boundary line features are complete and reliable. Therefore, by using the lane boundary line features as the basis for the lateral offset, a lateral offset with better accuracy and reliability can be obtained. In specific applications, other road features, such as lane centerlines, guide lines or guide arrows, may alternatively be used as the basis for determining the lateral offset.

[0080] In some embodiments, the lane boundary line features in the road feature map may include two lane boundary line features in the ROI in the road feature map, and the two lane boundary lines are located on the left and right sides of the first pose. In other words, the lateral offset of the vehicle may be determined based on the two lane boundary line features in the ROI in the road feature map. Because the data amount of the ROI in the road feature map is smaller and the features of the two lane boundary lines on the left and right sides of the first pose are reliable and complete, the lateral offset is obtained by using the two lane boundary line features in the ROI in the road feature map. This can further reduce the calculation complexity, the calculation resource consumption and the time consumption while improving the lateral offset accuracy. In a specific application, the lateral offset may alternatively be obtained by using the lane boundary line features of other features or other regions in the road feature map. The features used to determine the lateral offset of the vehicle are not limited in the embodiments of this application.

[0081] In some embodiments, the lateral offset of the vehicle may be obtained based on the lateral pixel offset and a preset pixel ratio. The lateral pixel offset is determined based on the lane boundary line feature in the top view of the ROI in the road feature map, and the lane boundary line feature in the top view of the ROI in the road feature map is obtained by using the features of two lane boundary lines in the ROI in the road feature map. Since the data amount of the ROI in the road feature map is smaller and the features of the two lane boundary lines included in the road feature map are reliable and complete, the top view of the ROI in the road feature map is equivalent to the aerial feature map of the road ahead of the vehicle, and the method of obtaining the lateral offset based on the lateral pixel offset and the pixel ratio is simple and easy to realize, and the lateral offset of the vehicle is obtained based on the lateral pixel offset. This can further reduce the calculation complexity, the calculation resource consumption and the time consumption while improving the lateral offset accuracy.

[0082] In some embodiments, the lateral pixel offset may be the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, where the lane center point is the lane center point between the left and right lane boundary lines of the first pose. Since the optical center of the first camera is the coordinate origin of the camera coordinate system and the pixel coordinate system of the first camera, and the ROI mapping point position of the optical center of the first camera is easy to obtain, the lateral pixel offset is obtained by using the ROI mapping point of the optical center of the first camera. This can further reduce the computational complexity.

[0083] In a specific application, the lateral pixel offset may alternatively be obtained by using the ROI mapping point position of another rigid position point on the vehicle (e.g., the rear axle center of the vehicle). The specific manner of obtaining the lateral pixel offset is not limited in the embodiments of this application.

[0084] In some embodiments, an exemplary specific implementation process for determining the lateral offset may include the following steps (1) to (4).

[0085] (1) Determine an ROI in a road feature map based on pre-set ROI size parameters, where the ROI in the road feature map includes left and right lane boundary features of the first pose.

[0086] (2) Determine an affine transformation matrix of the first camera based on the end point positions of the ROI in the road feature map and a preset pixel ratio, and obtain a top view of the ROI in the road feature map through the affine transformation of the first camera based on the affine transformation matrix.

[0087] (3) Determine the lane center point position in the top view of the ROI in the road feature map and the ROI mapping point position of the optical center of the first camera, and use the distance between the lane center point position and the ROI mapping point position as the lateral pixel offset.

[0088] (4) The product of the lateral pixel offset and the pixel ratio is used as the actual lateral offset, where the actual lateral offset is the final lateral offset.

[0089] In the above exemplary implementation, the lateral offset of the vehicle may be obtained through the perspective projection and affine transformation of the first camera based on the lane boundary line features. The amount of calculation is small and the computational complexity is low. Moreover, this can improve the accuracy of the lateral offset, and improve the accuracy of vehicle positioning while further reducing the computational resource consumption and time consumption.

[0090] Step S240: Obtain a second position of the vehicle based on the lateral offset of the vehicle, the first pose of the vehicle, the road feature map, and a local map corresponding to the first pose.

[0091] The first pose indicates an initial pose of the vehicle or a pose at a previous time point. The first pose may include a first position and a first attitude. The first position is an initial position of the vehicle or a position at a previous time point, and the first attitude is an initial attitude of the vehicle or an attitude at a previous time point.

[0092] The second pose is the next pose of the first pose, and the second pose may also be referred to as the visual positioning pose of the first pose. The second pose may include a second position and a second attitude. If the position of the vehicle does not change and only the attitude changes, the second position is equivalent to the first position. If the position of the vehicle changes, the second position may be obtained by performing the processes in steps S210 to S240 on the first pose. If the attitude of the vehicle does not change or if the change in the attitude of the vehicle is not of concern, the second attitude may be equivalent to the first attitude. If the attitude of the vehicle needs to be updated and the attitude of the vehicle changes, the second attitude may alternatively be obtained by performing the processes in steps S210 to S240 on the first pose.

[0093] The third pose is a pose (which may be called a fusion positioning pose) acquired by performing the process in steps S210 to S250 on the first pose. In other words, the third pose may be acquired by performing the process in step S250 on the second pose.

[0094] Both the second pose and the third pose may be used as the positioning pose or the current pose of the vehicle. In practical application, the second pose and / or the third pose may be selected as the final pose or the current pose of the vehicle based on requirements. Here, the first pose, the second pose and the subsequent third pose may be the above global pose.

[0095] In some embodiments, step S240 may further include acquiring a second pose of the vehicle based on the lateral offset, the first position, the first pose, the road feature map and the local map. In a specific application, the second pose may be acquired synchronously with the second position, or the second pose and the second position may be acquired in sequence, but the acquisition order is not limited.

[0096] In some embodiments, the embodiment of this application may further include a step of acquiring a local map corresponding to the first pose of the vehicle. In practical application, the data of the local map may be acquired from a map in the cloud, or may be extracted from a map stored locally, and may be acquired before the positioning method in the embodiment of this application starts, or may be acquired in real time during the execution process of the embodiment of this application (for example, in or before step S240).

[0097] The local map is a portion of the map that corresponds to an area around the first pose. In some embodiments, the local map may be a preset area centered on the corresponding location of the first pose in the map, and the preset area may be a circular area, a rectangular area, a triangular area, or an area of ​​other user-defined shape. A circular area is used as an example. The local map may be a circular area where the corresponding location of the first pose is used as the center of the circle and a preset length is used as the radius in the map. The preset length may be flexibly set or may be determined based on the vehicle speed. For example, the preset length may be 30 meters to 50 meters by default.

[0098] In order to improve the accuracy of vehicle positioning, the local map in the embodiments of this application may be from a vector map, since the data in the vector map is accurate and complete. In some embodiments, the vector map may be a high-definition vector map or a high-precision vector map. In a specific application, the vector map may be from the cloud or may be built in the storage device of the vehicle.

[0099] In some embodiments, the information recorded in the local map includes, but is not limited to, the position information of each point in each lane, the position information of lane center points, road azimuth angle information, marker information, etc. Road features (e.g., markers, lane boundary lines, etc.) in the local map may be indicated by using vector lines, curves or straight lines, and different types of road features may be distinguished by using pixel values. Road azimuth angle information of each lane in the local map may be associated with position information, and road azimuth angles corresponding to different positions of the same lane may be the same or different. In addition, the local map may also record information of various road signs, such as guide arrows and guide lines. The types, visual representation manners, specific recording contents, etc. of the local map are not limited in the embodiments of this application.

[0100] In some embodiments, the second position or the second pose or both may be determined based on multiple candidate pose points in a local map, and the pose of each candidate pose is determined based on the lateral offset, the first pose, and the local map. In this way, the vehicle can be accurately positioned by using the candidate pose points selected based on the lateral offset, and there is no need to perform large-scale global image matching for the area around the first pose. The amount of calculation is small and the computational complexity is low. This can effectively reduce the computational resource consumption and time consumption while improving the accuracy of vehicle positioning.

[0101] In some embodiments, multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the correction position as the center, and the correction position is obtained based on the lateral offset and based on the lane center point position and road azimuth angle that correspond to the first pose and are in the local map. In this way, by using multiple candidate pose points in the same lane direction, the vehicle can be accurately located. The search range is smaller and the image matching range is further narrowed. This can further reduce the computational resource consumption and time consumption while improving the accuracy of the vehicle positioning.

[0102] In some embodiments, the second position may be position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second pose may be pose information of the candidate pose point selected by performing feature matching on the projection image and the road feature map, and the projection image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map. Since the degree of match between the projection image and the road feature map may indicate the degree of closeness between the corresponding candidate pose point and the actual vehicle pose, an accurate and reliable vehicle positioning result can be obtained by performing feature matching between the projection image and the road feature map.

[0103] In some examples, the candidate pose point selected by performing feature matching on the projection image and the road feature map may be any candidate pose point among the multiple candidate pose points whose matching cost of the projection image and the road feature map is less than a preset threshold, or the candidate pose point having the minimum matching cost of the projection image and the road feature map. The minimum matching cost of the projection image and the road feature map indicates the highest matching degree between the projection image and the road feature map, and also indicates that the corresponding candidate pose point is closest to the actual pose of the vehicle. Therefore, the pose of the candidate pose point having a matching cost less than the preset threshold or having the minimum matching cost is closer to the actual pose of the vehicle. In this way, a more accurate and reliable vehicle positioning result can be obtained.

[0104] In some embodiments, the vehicle positioning method provided in the embodiments of this application may further include a step of acquiring a third pose of the vehicle based on the second position, the first attitude and the sensor positioning data, or a step S250 of acquiring a third pose of the vehicle based on the second position, the second attitude and the sensor positioning data. Here, the sensor positioning data may include one or more of the following: GPS information, IMU information, Inertial Navigation System (INS) vehicle attitude information and chassis information. In this way, the second attitude and the real-time positioning information of other sensors may be fused to acquire a third pose with higher accuracy and better reliability, and compensate for the influence of the poor performance of the first camera and external environmental conditions such as poor lighting on the accuracy of the second pose, and further improve the accuracy and reliability of the vehicle positioning.

[0105] In some embodiments, an Extended Kalman-Filter (EKF) may be used to perform multi-sensor information fusion to obtain the third pose of the vehicle through calculation, for example, fusion may be performed on the GPS information, the IMU information, the INS vehicle attitude information, the chassis information, and the second pose by using the covariance value of the GPS information, the IMU information, the INS vehicle attitude information, the chassis information, and the second pose as the weight of information validity to obtain the third pose of the vehicle. Here, the specific algorithm of the fusion calculation is not limited in the embodiments of this application.

[0106] In some embodiments, an exemplary implementation procedure of step S230 may include the following steps a1 to a5.

[0107] Step a1: Isolate lane boundary features.

[0108] In some embodiments, different markers in the road feature map may be classified by using pixel values. Assuming that the pixel value of the lane boundary line feature is agreed to be m, the pixel with pixel value m in the road feature map may be extracted to form a road feature map containing only lane boundary line features. Figure 4 is a road feature map obtained by separating the lane boundary line features in Figure 3.

[0109] This step is an optional step. The step of separating lane boundary features may be applied or omitted based on the actual application requirements. The road features are separated in advance. This can reduce the data volume of the subsequent processing process, reduce the calculation complexity, and filter out unnecessary feature data. If other features such as lane centerlines are offset laterally, other features such as lane centerlines may also be separated in a similar manner.

[0110] Step a2: Obtain an ROI in the road feature map.

[0111] The ROI may be selected as a fixed viewing area with high detection effect precision and accuracy. In practical application, nearby objects have larger representations in the image and are more likely to be detected. Distant objects have smaller representations in the image and are less likely to be detected. In addition, objects at the edge of the viewing field are more likely to be incomplete objects in the image and have a high probability of detection error. In the area close to and not at the edge of the viewing field, the image data is complete and accurate, and the detection accuracy is high. Therefore, a partial viewing area close to the camera and not at the edge is usually agreed upon as the ROI. In a specific application, the ROI may be preset as the actual size range of the fixed viewing area.

[0112] In some embodiments, a region of interest (ROI) may be selected in the camera coordinate system of the first camera, and size parameters of the ROI are preset. The size parameters may include, but are not limited to, the positions of the ROI corner points in the camera coordinate system, the length of the ROI, and the width of the ROI. A rectangular ROI is used as an example. The size parameters of the ROI may include, but are not limited to, the positions of the four ROI corner points (e.g., the coordinate values ​​of the four corner points in the camera coordinate system of the first camera), the length of the ROI, and the width of the ROI.

[0113] In this step, the ROI endpoints in the road feature map may be obtained by transforming the ROI endpoints into the pixel coordinate system of the first camera through the perspective projection of the first camera. The rectangular area defined by these endpoints is the ROI in the road feature map.

[0114] Fig. 5 is a schematic diagram of the ROI in the camera coordinate system of the first camera. In the example in Fig. 5, in the camera coordinate system XCY, in which the actual position C of the first camera (i.e., the position of the optical center of the first camera in the reference coordinate system) is used as the coordinate origin, a rectangular area bounded by four points P1, P2, P3, and P4 is selected on the ground as the ROI. The size parameters of the ROI are determined by the coordinate values ​​P1(x p1 ,y p1 ,z p1 ), P2(x p2 ,y p2 ,z p2 ), P3(x p3 ,y p3 ,z p3 ) and P4(x p4 ,y p4 ,z p4 ), the length m of the ROI (i.e., the actual distance in meters from P1 to P2), and the width n of the ROI (i.e., the actual distance in meters from P1 to P3).

[0115] In some embodiments, when the ROI in the road feature map is acquired, the ROI mapping point position of the optical center of the first camera may be further determined. Specifically, the ROI mapping point position of the optical center of the first camera in the camera coordinate system may be determined based on the positions of the ROI end points in the camera coordinate system, and the ROI mapping point position of the optical center of the first camera in the pixel coordinate system may be acquired through the perspective projection of the first camera based on the ROI mapping point position of the optical center of the first camera in the camera coordinate system.

[0116] See the example in FIG. 5. A perpendicular line is drawn from the actual position C of the first camera to the ROI, and the intersection of the perpendicular line (i.e., the Y-axis of the camera coordinate system of the first camera) with the side edge of the ROI closer to the camera is the ROI mapping point S of the optical center of the first camera. The positions of the ROI edge points in the camera coordinate system are known, and the position of the ROI mapping point S of the optical center of the first camera (S X, S Y ,S Z ) may be determined, i.e., S Y = 0, and S X = 0, and S Z =z P3 =z P4 It is.

[0117] The coordinates of the four points P1(x P1 ,y P1 ,z P1 ), P2(x P2 ,y P2 ,z P2 ), P3(x P3 ,y P3 ,z P3 ) and P4(x P4 ,y P4 ,z P4 ) and the position of the ROI mapping point S (S X, S Y ,S Z ) is known, and the positions P1'(u P1 ,v P1 ), P2'(u P2 ,vP2 ), P3'(u P3 ,v P3 ) and P4'(u P4 ,v P4 ), and the position S'(u S ,v S ) may be obtained through the perspective projection of Equation (2) and Equation (3). In this way, the ROI in the road feature map is obtained. Figure 6 is an example diagram of the ROI in the road feature map. In Figure 6, the dashed box indicates the boundary of the ROI, and the solid lines indicate the lane boundary lines.

[0118] Step a3: Determine the endpoint positions of the two left and right adjacent lane boundary lines within the ROI in the road feature map.

[0119] In some embodiments, two adjacent left and right lane boundaries L1 and L2 may be detected within the ROI in the road feature map by using the Hoff line detection method or the like, and the end points L of the lane boundary L1 may be detected. 1up (x l1 ,y l1 ) and L 1down (x l2 ,y l2 ) and the end point position L of the lane boundary line L2 2up (x l3 ,y l3 ) and L 2down (x l4 ,y l4 ) is determined. In this way, two left and right adjacent lane lines in the ROI may be selected based on the distribution of lane lines in the ROI in the road feature map as a basis for determining the lateral offset to obtain a more reliable lateral offset. FIG. 6 shows two left and right lane lines (thick black lines in FIG. 6) adjacent to a vehicle in the ROI in the road feature map, and the endpoints of the two lane lines.

[0120] Step a4: Through affine transformation, obtain the lane boundary line features in the top view of the ROI in the road feature map and the ROI mapping point position of the optical center of the first camera.

[0121] In some implementations, an exemplary implementation process of this step may include the following sub-steps (1) and (2).

[0122] (1) Construct a top view corresponding to a first camera, and determine an affine transformation matrix M between the ROI in the road feature map and the top view of the ROI in the road feature map.

[0123] Specifically, a top view with a pixel ratio of k (units are pixels per meter, i.e., the number of pixels per meter) may be pre-constructed, and B w Width and B h Based on the length m of the ROI and the width n of the ROI in the camera coordinate system of the first camera, B w and B. h may be obtained according to equation (8).

number

[0124] Specifically, the ROI end point P1'(u P1 ,v P1 ), P2'(u P2 ,v P2 ), P3'(u P3 ,v P3 ) and P4'(u P4 ,v P4 ) are known. The end point positions P1''(0,0) and P2''((B w -1),0), P3''((B w -1),(B h -1)) and P4''(0,(B h -1)) is B w and B. h In this way, the affine transformation matrix M between the ROI in the road feature map and the top view of the ROI in the road feature map may be obtained according to the above equations (6) and (7) of the affine transformation.

[0125] (2) Through affine transformation, obtain the lane boundary lines L3 and L4 of the top view of the ROI in the road feature map and the ROI mapping point S''.

[0126] First, the lane boundaries L1 and L2 in the ROI in the road feature map are transformed into the pixel coordinate system of the top view through affine transformation to obtain the corresponding lane boundaries L3 and L4 in the top view of the ROI in the road feature map. 3up (x l5 ,y l5 ), L 3down (x l6 ,y l6 ), L 4up (x l7 ,y l7 ) and L 2down (x l8 ,y l8 ) are the end points L of lane boundary lines L1 and L2 1up (x l1 ,y l1 ), L 1down (x l2 ,y l2 ), L 2up (x l3 ,y l3 ) and L 2down (x l4 ,y l4 ) into the affine transformation equations (6) and (7).

[0127] Then, based on the end point positions of the lane boundaries L3 and L4, equation (9) for lane boundary line L3 and equation (10) for lane boundary line L4 may be obtained.

number

[0128] k L3 indicates the slope of the lane boundary line L3, and b L3 indicates the y-axis intercept of the lane boundary line L3, and k L4 indicates the slope of the lane boundary line L4, and b L4 indicates the y-axis intercept of lane boundary line L4.

[0129] Furthermore, the ROI mapping point position S'(u S ,v S ) is substituted into equations (6) and (7) of the affine transformation to obtain the ROI mapping point position S″(x1,y1) of the optical center of the first camera in the top view of the ROI in the road feature map. FIG. 7 is an example diagram of the top view of the ROI in the road feature map. FIG. 7 also shows lanes L3 and L4 and the ROI mapping point S″ in the top view of the ROI in the road feature map. The dashed box indicates the top view of the ROI in the road feature map, the solid lines in the dashed box indicate the lane boundary lines, and the dashed line between the two solid lines indicates the lane cross-section lines.

[0130] Step a5: Calculate the horizontal offset.

[0131] Specifically, the position Q(x2,y2) of the lane center point in the top view of the ROI in the road feature map may be obtained based on equation (9) of the lane boundary line L3 and equation (10) of the lane boundary line L4, and the lateral pixel offset is obtained based on the position Q(x2,y2) of the lane center point and the ROI mapping point position S''(x1,y1), and then the actual lateral offset is obtained based on the lateral pixel offset and the preset pixel ratio.

[0132] See the example in Figure 7. In the pixel coordinate system of the top view, a perpendicular line is drawn through S''(x1,y1) to the lane boundary line L3 or the lane boundary line L4, and the intersection point Q1 of the perpendicular line with the lane boundary line L3 and the intersection point Q2 of the perpendicular line with the lane boundary line L4 are the two end points of the lane section line. The midpoint of the lane section line Q1Q2, i.e., the midpoint between the intersection points Q1 and Q2, is the position Q(x2,y2) of the lane center point.

[0133] The coordinates of the two end points of the lane section line Q1(l x1 ,l y1 ) and Q2(l x2 ,l y2) may be determined based on the lane boundary line L3 equation (9) and the lane boundary line L4 equation (10) and the ROI mapping point S″(s1,y1) in the top view of the ROI in the road feature map. Then, the position Q(x2,y2) of the lane center point may be obtained according to equation (11).

number

[0134] The lateral pixel offset is the road lateral offset in the pixel coordinate system of the top view of the ROI in the road feature map. From FIG. 7, it can be seen that the distance between the ROI mapping point S''(x1,y1) and the lane center point Q(x2,y2) in the pixel coordinate system of the top view of the ROI in the road feature map is the lateral pixel offset. In other words, the lateral pixel offset offset1 may be obtained according to Equation (12).

number

[0135] In some embodiments, the actual lateral offset is the road lateral offset in the reference coordinate system. Specifically, the actual lateral offset offset2 may be obtained according to Equation (13).

number

[0136] offset2 denotes the actual lateral offset, and k denotes the preset pixel ratio.

[0137] In some embodiments, an exemplary implementation procedure of step S240 may include the following steps b1 to b3.

[0138] Step b1: Correct the first position based on a lateral offset to obtain a corrected position corresponding to the first pose.

[0139] In some embodiments, the correction position H start (H x0 ,H y0 ,H z0 ) may be obtained according to equation (14).

number

[0140] 8 is a schematic diagram of the correction position, the first position, and the lateral offset in the reference coordinate system. In FIG. 8, the coordinate axis XOY is the XY plane of the reference coordinate system, O is the coordinate origin of the reference coordinate system, and E start indicates the first position, C start is the first position in the local map, E start , and offset2 indicates the actual lateral offset. From FIG. 8, the lateral offset components of each coordinate axis correspond to the road azimuth angle yaw l It can be seen that the corrected position H may be obtained by resolving the lateral offset with respect to the reference coordinate system based on start The coordinate values ​​of may be obtained by adding the lateral offset components of each coordinate axis to the corresponding coordinate values ​​of the lane center point.

[0141] Step b2: Candidate pose points are determined.

[0142] In the implementation, the candidate pose point is the corrected position H start (H x0 ,H y0 ,H z0 ) as the center, the pose of each candidate pose point is selected in the lane direction of the local map, and the pose of each candidate pose point is calculated by the corrected position H start (H x0 ,H y0 ,H z0 ) and road azimuth angle information in the local map.

[0143] Specifically, the correction position H start (H x0 ,H y0 ,H z0 ), the preset search range and the preset search step are known, and road direction data yaw i Based on the correction position H start (H x0 ,H y0 ,H z0 ) is the 0th candidate pose point H z0 The number of candidate pose points (2n+1) and the position information of each candidate pose point (H xi ,H yi ,H zi ) may be determined according to equations (15) to (18).

number

[0144] scope indicates the preset search scope, step indicates the preset search step, and H i (H xi ,H yi ,H zi ) indicates the position of the i-th candidate pose point, and yaw i (i=-n,-n+1,...,-1,0,1,...,n-1,n) indicates the road azimuth angle corresponding to the i-th candidate pose point. imay be obtained directly from the local map. From equation (18), it can be seen that the number of candidate pose points is 2n+1, where n is an integer equal to or greater than 1. From equations (15) to (17), H z-1 and H z1 is the 0th candidate pose point H z0 It can be seen that the position is the same as

[0145] 9 is an exemplary diagram of the candidate pose points. In FIG. 9, the white origin is the candidate pose point H i The black circle indicates the correction position H start L5 is the correction position H start 9, L0 indicates the lane direction passing through, L0 indicates the lane center line, XOY is the XY plane of the reference coordinate system, and O is the coordinate origin of the reference coordinate system. From equations (15) to (18) and FIG. 9, it can be seen that the candidate pose points are evenly distributed in the same lane direction of the local map, and the lane direction passes through the correction position. In this way, the range of feature matching can be reduced to the lanes in the grid of the map, and it is not necessary to cover all the points around the position of the vehicle in the map. The calculation amount of the projection imaging is greatly reduced, and the range of point matching in feature matching is greatly reduced. It can be seen that the calculation amount is greatly reduced, and the calculation complexity is also greatly reduced.

[0146] In some implementations, when the vehicle pose does not change essentially, or the change in the vehicle pose is not relevant, or the vehicle pose does not need to be determined synchronously, the pose information of each candidate pose point (raw i pitch i ,yaw i ) may be the first orientation. For example, if the first orientation is (raw start , pitch start ,yaw start ), the pose information of each candidate pose point is also the first pose (raw start pitch start ,yaw start In other words, the pose of the i-th candidate pose point is H i (H xi ,H yi ,H zi ,rawstart pitch start ,yaw start ) is also acceptable.

[0147] In some implementations, when the vehicle pose needs to be determined synchronously, the pose information of each candidate pose point is calculated based on the position information H i (H xi ,H yi ,H zi For example, the pose information of each candidate pose point may be determined in a manner similar to that of the first pose (raw start pitch start ,yaw start It should be noted that if the vehicle pose needs to be determined synchronously, the number of candidate pose points is not limited to the above "2n+1".

[0148] When the second pose of the vehicle is determined, the candidate pose points and the position and pose information of each candidate pose point may be selected in a similar manner, which can also significantly reduce the amount of calculations and the computational complexity.

[0149] Step b3: Projection imaging is performed.

[0150] In this step, for each candidate pose point, projection imaging may be performed by using the local map to obtain a projection image of each candidate pose point.

[0151] In some implementations, the projected imaging process of the single candidate pose point may include: adjusting the extrinsic parameters of the first camera by using the pose of the candidate pose point; projecting the pixels on the marker vector line of the local map into the pixel coordinate system of the first camera based on the extrinsic and intrinsic parameters of the first camera; synchronously establishing the linear correspondence between the pixels in the pixel coordinate system based on the linear correspondence between the pixels in the local map; and rendering the corresponding pixels based on the preset pixel values ​​of each marker to obtain a projected image of the candidate pose point. In this way, the projected image may be equivalent to a road feature map of the forward road image of the first camera with an optical center at the candidate pose point.

[0152] Here, markers include all traffic signs and road markings such as, but are not limited to, traffic lights, underground parking poles, walls, toll booths, road gates, collision avoidance zones, obstacles, storage positions, storage position lines, no parking areas, pedestrian crossings, speed belts, slow down, stop, yield lines and give way, road sign boards, utility poles, road edges, trees, shrubs, medians, guardrails, etc.

[0153] In some implementations, according to the above principles of equations (1) to (3), the process of projecting the pixels of the marker in the local map into the pixel coordinate system of the first camera may be realized according to the following equation (19):

number

[0154] (u,v) indicates the coordinate value of a pixel in the pixel coordinate system of the first camera, and (X W ,Y W ,Z W ) denotes the coordinate value of the pixel of the marker in the local map in the reference coordinate system, R is the rotation matrix of the first camera, T is the translation matrix of the first camera, and R is the pose information (raw i pitch i ,yaw i), and a rotation matrix in the initial extrinsic parameters of the first camera, and T may be determined based on the position information of the candidate pose points (H xi ,H yi ,H zi ), and the translation matrix among the initial extrinsic parameters of the first camera.

[0155] In some implementations, to facilitate feature matching, a preset pixel value of each marker in the projection image may match a preset pixel value of a corresponding marker in the road feature map.

[0156] Step b4: Obtain a second pose of the vehicle through feature matching.

[0157] In this step, feature matching is performed between the projection image of each candidate pose point and the road feature map to obtain a matching cost corresponding to the candidate pose point, and the pose of the candidate pose point having the minimum matching cost among all the candidate pose points is selected as the second pose of the vehicle, i.e., the position information of the candidate pose point having the minimum matching cost is used as the second position of the vehicle, and the posture information of the candidate pose point having the minimum matching cost is used as the second posture of the vehicle.

[0158] In some implementations, the feature matching between the projection image of each candidate pose point and the road feature map may include: For each pixel p1 in the road feature map, the complete projection image is traversed to find a pixel p2 in the projection image whose pixel value is closest to pixel p1, and the Euclidean distance D1 between pixel p2 in the projection image and pixel p1 in the road feature map is calculated. In this way, the Euclidean distance D1 between each pixel in the road feature map and the corresponding pixel in the projection image is obtained. The sum of the Euclidean distances D1 corresponding to all pixels in the road feature map is the matching cost between the projection image and the road feature map.

[0159] Figure 10 is an exemplary specific implementation procedure of the vehicle positioning method according to an embodiment of this application. Please refer to Figure 10. The exemplary specific implementation process of the vehicle positioning method provided in the embodiment of this application may include the following steps:

[0160] Step S1010: A first camera captures a road image ahead of the vehicle at a current time, and provides the road image ahead to a computing device.

[0161] Step S1020: The computing device obtains a local map corresponding to a first pose, where the first pose is a global pose of the vehicle at a previous time point.

[0162] Step S1030: The computing device determines a second pose of the vehicle based on the forward road image and a local map corresponding to the first pose.

[0163] Specifically, first, a road feature map of a forward road image is obtained by performing processes such as semantic feature extraction, semantic feature skeletonization, and post-fit processing; second, an ROI in the road feature map is extracted, and a lateral offset is determined based on the lane boundary features in the ROI in the road feature map and the first pose; third, a corrected position of the first position is obtained based on the lateral offset, and a longitudinal search is performed in the lane direction of the corrected position to obtain a second pose of the vehicle. Here, the longitudinal search in the lane direction of the corrected position is the above process of determining the second position of the vehicle. The process may include: selecting and determining a pose of a plurality of candidate pose points in the lane direction of the corrected position; performing projection imaging of a local map for each candidate pose point to obtain a projection image of the candidate pose point; performing feature matching on the projection image of each candidate pose point and the road feature map to find a candidate pose point with a minimum matching cost; and using the pose of the candidate pose point with the minimum matching cost as the second pose of the vehicle.

[0164] Step S1040: Using the EKF module, perform fusion calculation on the sensor positioning data at the current time and the second pose obtained in step S1030 to obtain a third pose of the vehicle, where the third pose is the pose of the vehicle at the current time. Here, the sensor positioning data includes GPS information, IMU information, INS vehicle attitude information, and / or chassis information. Through the fusion of multiple types of positioning information, the error range of the vehicle positioning may be controlled at a centimeter level, thereby further improving the accuracy of the vehicle positioning.

[0165] In some embodiments, the weight of the corresponding information may be adjusted in real time based on the state of each sensor in the fusion calculation process. For example, the GPS information carries flag bits used to describe the GPS state and accuracy (e.g., the GPS is working properly, is out of order, has an insufficient signal, or has other conditions), and whether to integrate the GPS information and the integration weight of the GPS information may be determined based on the flag bits in the GPS information. For example, when the GPS is out of order, the GPS sensor provides a flag bit indicating that the GPS information is unreliable or does not output the GPS information. In this case, the GPS information may not be included in the fusion in step S1040. In other words, the fusion calculation is performed on the IMU information, the INS vehicle attitude information, and the chassis information at the current time and the second pose obtained in step S1030 to obtain the third pose of the vehicle. When the GPS has a good signal, in step S1040, a fusion calculation may be performed on the current GPS information, IMU information, INS vehicle attitude information and chassis information and the second pose obtained in step S1030 to obtain a third pose of the vehicle.

[0166] In some embodiments, the GPS information includes specific location information of the vehicle, and may be described by using longitude and latitude and altitude information, or three-dimensional coordinate values ​​in a reference coordinate system (e.g., UTM coordinate system). The IMU information includes, but is not limited to, vehicle linear acceleration and vehicle angular acceleration. The INS vehicle attitude information includes vehicle attitude information obtained by analyzing the IMU information. The chassis information may include, but is not limited to, vehicle speed, vehicle acceleration, steering angle, steering angular velocity, and other information. The specific contents of the GPS information, IMU information, INS vehicle attitude information, and chassis information are not limited in the embodiments of this application.

[0167] In various cases where the GPS is adequate, faulty, has insufficient signal, or the like, according to the embodiments of this application, vehicle positioning with centimeter-level positioning error can be realized, and the computing resource consumption and time consumption can be reduced, thereby simultaneously improving the accuracy, reliability and timeliness of vehicle positioning, and ensuring the driving safety of the vehicle in driving modes such as unmanned driving or intelligent driving.

[0168] The specific implementation methods of the vehicle positioning apparatus, computing device, vehicle, etc. provided in the embodiments of this application are described in detail below.

[0169] FIG. 11 is a schematic diagram of the structure of a vehicle positioning device 1100 according to an embodiment of this application. Please refer to FIG. 11. The vehicle positioning device 1100 provided in this embodiment of this application includes: an image capture unit 1110 configured to capture an image of the road ahead of the vehicle; A feature acquisition unit 1120 configured to acquire a road feature map based on the forward road image; an offset determination unit 1130 configured to determine a lateral offset of the vehicle based on the road feature map; a position determining unit 1140 configured to obtain a second position of the vehicle based on a lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; may include:

[0170] In some embodiments, the feature acquisition unit 1120 may be specifically configured to determine the lateral offset of the vehicle based on lane boundary features in the road feature map.

[0171] In some embodiments, the lane boundary line features in the road feature map may include features of two lane boundary lines within a region of interest ROI in the road feature map, the two lane boundary lines being located to the left and right of the first pose.

[0172] In some embodiments, the feature acquisition unit 1120 may be specifically configured to acquire a lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio, where the lateral pixel offset is determined based on lane boundary line features in a top view of the ROI in the road feature map, and the lane boundary line features in the top view of the ROI in the road feature map are acquired by using features of two lane boundary lines in the ROI in the road feature map.

[0173] In some embodiments, the lateral pixel offset is the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, which is the lane center point between the left and right lane boundary lines of the first pose.

[0174] In some embodiments, the vehicle positioning device 1100 may further include an attitude determination unit 1150 configured to obtain a second attitude of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map.

[0175] In some embodiments, a second position or a second pose, or both, of the vehicle is determined based on a plurality of candidate pose points in the local map, and a pose of each candidate pose point is determined based on a lateral offset of the vehicle, the first pose, and the local map.

[0176] In some embodiments, the multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the corrected position as the center, and the corrected position is obtained based on the lateral offset and based on the lane center point position and road azimuth angle that correspond to the first pose and are in the local map.

[0177] In some embodiments, the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second pose is pose information of a candidate pose point selected by performing feature matching on the projection image and the road feature map, and the projection image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

[0178] In some embodiments, the vehicle positioning device 1100 may further include a fusion unit 1160 configured to obtain a third pose of the vehicle based on the second position, the first attitude of the vehicle and the sensor positioning data, or obtain a third pose of the vehicle based on the second position, the second attitude of the vehicle and the sensor positioning data, where the sensor positioning data may include one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

[0179] In some embodiments, the local map may be from a vector map.

[0180] The vehicle positioning device in the embodiment of this application may be realized by using software, hardware, or a combination thereof. For example, the vehicle positioning device 1100 may be realized as software in a computing device (the software has the above-mentioned functional modules), or may be directly realized as a computing device having the above-mentioned functional modules.

[0181] 12 is a schematic diagram of a structure of a computing device 1200 according to an embodiment of this application. The computing device 1200 includes one or more processors 1210 and one or more memories 1220.

[0182] Processor 1210 may be connected to memory 1220. Memory 1220 may be configured to store program codes and data. Thus, memory 1220 may be a storage unit within processor 1210, an external storage unit independent of processor 1210, or a component that includes a storage unit within processor 1210 and an external storage unit independent of processor 1210.

[0183] Optionally, the computing device 1200 may further include a communication interface 1230. It should be understood that the communication interface 1230 in the computing device 1200 shown in FIG. 12 may be used for communication with other devices.

[0184] Optionally, the computing device 1200 may further include a bus 1240. The memory 1220 and the communication interface 1230 may be connected to the processor 1210 through the bus 1240. The bus 1240 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus 1240 may be classified into an address bus, a data bus, a control bus, etc. For ease of representation, the bus is shown in FIG. 12 by using only one line. However, this does not indicate that there is only one bus or only one type of bus.

[0185] It should be understood that in this embodiment of the application, the processor 1210 may be a central processing unit (CPU). The processor may alternatively be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. Alternatively, the processor 1210 may be one or more integrated circuits, configured to execute associated programs to implement the technical solutions provided in the embodiments of this application.

[0186] The memory 1220 may include read-only memory and random access memory to provide instructions and data to the processor 1210. A portion of the processor 1210 may further include non-volatile random access memory. For example, the processor 1210 may further store device type information.

[0187] When the computing device 1200 operates, the processor 1210 executes computer-executable instructions in the memory 1220 to perform the operational steps of the vehicle positioning method described above.

[0188] It should be understood that the computing device 1200 according to this embodiment of the present application may correspond to a corresponding implementation of the method according to the embodiment of the present application, and the above and other operations and / or functions of the modules in the computing device 1200 are separately intended to realize the corresponding steps of the method in the embodiment. For the sake of brevity, the details will not be described again here.

[0189] In practical applications, the computing device 1200 may be implemented as a functional unit in a chip, an independent chip, a functional unit of an in-vehicle terminal device, or an independent in-vehicle terminal device. In some embodiments, the computing device 1200 may be a functional unit / module in an in-vehicle infotainment, a cockpit domain controller (CDC), or a mobile data center / multi-domain controller (MDC). The form and deployment manner of the computing device 1200 are not limited in the embodiments of this application.

[0190] An embodiment of this application further provides a vehicle. The vehicle includes a first camera (which may be referred to as a front view camera) configured to capture a road image ahead of the vehicle. The vehicle may further include a vehicle positioning apparatus 1100, a computing device 1200, a computer-readable storage medium described below, or a computer program product described below.

[0191] Figure 13 is an exemplary diagram of a vehicle according to an embodiment of this application. Please refer to Figure 13. A front-view camera 1310 (i.e., the first camera described above) is installed on the vehicle 1300. The front-view camera may capture a road image ahead of the vehicle in real time and transmit the road image ahead to the computing device 1200 (not shown) of the vehicle, so that the computing device 1200 can reliably position the vehicle in real time by using the road image ahead.

[0192] For example, a GPS sensor 1320, an IMU 1330, a vehicle speed sensor 1340, an acceleration sensor 1350, etc. are further installed in the vehicle, and these sensors are separately connected to the above-mentioned computing device. It should be noted that Fig. 10 is merely an example. In a specific application, the number, type, deployment position, installation method, etc. of the GPS sensor 1320, the IMU 1330, the vehicle speed sensor 1340, and the acceleration sensor 1350 are not limited in the embodiment of this application.

[0193] For example, the front view camera 1310, the GPS sensor 1320, the IMU 1330, the vehicle speed sensor 1340, and the acceleration sensor 1350 may be separately connected to a computing device, wirelessly or by wires, for example, the front view camera, the GPS, the IMU, the vehicle speed sensor, and the acceleration sensor may be connected to the computing device through Ethernet, Bluetooth, a wireless fidelity (Wi-Fi) network, a cellular network, a Controller Area Network (CAN) bus, a local interconnect network (LIN) bus, or by various other communication methods.

[0194] An embodiment of this application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the program is executed by a processor, the processor is enabled to execute the above vehicle positioning method. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0195] An embodiment of the present application further provides a computer program product including a computer program, which, when executed by a processor, enables the processor to execute the above vehicle positioning method, where the program design language of the computer program product may be one or more. The program design language may include, but is not limited to, an object-oriented program design language, such as Java or C++, and a conventional procedural program design language, such as the "C" language.

[0196] It should be noted that the above are merely exemplary embodiments of this application and the technical principles used. Those skilled in the art may understand that this application is not limited to the specific embodiments described herein, and those skilled in the art may make various obvious modifications, rearrangements and substitutions without departing from the scope of protection of this application. Therefore, this application is described in detail with reference to the above embodiments, but this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, all falling within the scope of protection of this application.

Claims

1. 1. A vehicle positioning method, comprising: acquiring an image of the road ahead of the vehicle; obtaining a road feature map based on the road ahead image; determining a lateral offset of the vehicle based on the road feature map; obtaining a second position of the vehicle based on the lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; The method includes:

2. The method of claim 1 , wherein determining a lateral offset of the vehicle based on the road feature map comprises determining the lateral offset of the vehicle based on lane boundary features in the road feature map.

3. 3. The method of claim 2, wherein the lane boundary line features in the road feature map include two lane boundary line features within a region of interest (ROI) in the road feature map, the two lane boundary lines located to the left and right of the first pose.

4. determining the lateral offset of the vehicle based on lane boundary line features in the road feature map includes obtaining the lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio; and determining the lateral pixel offset based on lane boundary line features in a top view of the ROI in the road feature map; The method of claim 3 , wherein the lane boundary line features in the top view of the ROI in the road feature map are obtained by using the features of the two lane boundary lines within the ROI in the road feature map.

5. 5. The method of claim 4, wherein the lateral pixel offset is the distance between an ROI mapping point of an optical center of a first camera in the top view and a lane center point, the lane center point being a lane center point between the lane boundary lines on the left and right sides of the first pose, and the first camera is a camera that captures the forward road image.

6. 6. The method of claim 1, further comprising obtaining a second pose of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map.

7. 7. The method of claim 1, wherein the second position or the second pose, or both, of the vehicle are determined based on a plurality of candidate pose points in the local map, and a pose of each candidate pose point is determined based on the lateral offset of the vehicle, the first pose, and the local map.

8. 8. The method of claim 7, wherein the multiple candidate pose points are evenly distributed in the same lane direction of the local map by using a corrected position as a center, and the corrected position is obtained based on the lateral offset of the vehicle and based on a lane center point position and road azimuth angle that correspond to the first pose and are in the local map.

9. The second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second posture is posture information of the candidate pose point selected by performing feature matching on the projection image and the road feature map, The method of claim 7 or 8, wherein the projection image is obtained based on poses of the candidate pose points, intrinsic and extrinsic parameters of a first camera, and the local map.

10. 10. The method of claim 1, further comprising: obtaining a third pose of the vehicle based on the second position, the first attitude of the vehicle and sensor positioning data; or obtaining a third pose of the vehicle based on the second position, the second attitude of the vehicle and sensor positioning data.

11. The method of claim 10 , wherein the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

12. The method of claim 1 , wherein the local map is from a vector map.

13. A vehicle positioning device, comprising: an image capture unit configured to capture an image of the road ahead of the vehicle; a feature acquisition unit configured to acquire a road feature map based on the road ahead image; an offset determination unit configured to determine a lateral offset of the vehicle based on the road feature map; a position determination unit configured to obtain a second position of the vehicle based on the lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; An apparatus comprising:

14. The apparatus of claim 13 , wherein the feature acquisition unit is specifically configured to determine the lateral offset of the vehicle based on lane boundary features in the road feature map.

15. 15. The apparatus of claim 14, wherein the lane boundary line features in the road feature map include two lane boundary line features within a region of interest (ROI) in the road feature map, the two lane boundary lines located to the left and right of the first pose.

16. 16. The apparatus of claim 15, wherein the feature acquisition unit is specifically configured to acquire the lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio, the lateral pixel offset being determined based on lane boundary line features in the top view of the ROI in the road feature map, and the lane boundary line features in the top view of the ROI in the road feature map are acquired by using the features of the two lane boundary lines in the ROI in the road feature map.

17. 17. The apparatus of claim 16, wherein the lateral pixel offset is a distance between an ROI mapping point of an optical center of a first camera in the top view and a lane center point, the lane center point being a lane center point between the lane boundary lines on the left and right sides of the first pose, and the first camera is a camera that captures the forward road image.

18. 18. The apparatus of claim 13, further comprising an attitude determination unit configured to obtain a second attitude of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map.

19. 19. The apparatus of claim 13, wherein the second position or the second pose, or both, of the vehicle are determined based on a plurality of candidate pose points in the local map, a pose of each candidate pose point being determined based on the lateral offset of the vehicle, the first pose, and the local map.

20. 20. The apparatus of claim 19, wherein the plurality of candidate pose points are evenly distributed in the same lane direction of the local map by using a corrected position as a center, and the corrected position is obtained based on the lateral offset of the vehicle and based on a lane center point position and road azimuth angle that correspond to the first pose and are in the local map.

21. The second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second posture is posture information of the candidate pose point selected by performing feature matching on the projection image and the road feature map, The apparatus of claim 19 or 20, wherein the projection image is obtained based on poses of the candidate pose points, intrinsic and extrinsic parameters of a first camera, and the local map.

22. 22. The apparatus according to claim 13, further comprising a fusion unit configured to obtain a third pose of the vehicle based on the second position, a first attitude of the vehicle and sensor positioning data, or to obtain a third pose of the vehicle based on the second position, the second attitude of the vehicle and sensor positioning data.

23. 23. The apparatus of claim 22, wherein the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

24. 24. Apparatus according to any one of claims 13 to 23, wherein the local map is from a vector map.

25. A computing device including a processor and a memory, A computing device, wherein the memory stores program instructions which, when executed by the processor, enable the processor to perform a method according to any one of claims 1 to 12.

26. 1. A computer-readable storage medium, comprising:

13. A computer readable storage medium storing program instructions which, when executed by a computer, enable the computer to carry out a method according to any one of claims 1 to 12.

27. A vehicle, a first camera configured to capture a road ahead image; A vehicle positioning device according to any one of claims 13 to 24 or a computing device according to claim 25; A vehicle including.

28. 28. The vehicle of claim 27, further comprising one or more of the following: a GPS sensor, an IMU, a vehicle speed sensor, and an acceleration sensor.

Citation Information

Patent Citations

  • Avoidance operation calculating device, avoidance controller, vehicle provided with each device, avoidance operation calculating method, and avoidance control method

    JP2007253745A

  • Apparatus and method for controlling vehicle

    JP2011195017A

  • Travelling location detection device and travelling location detection method

    JP2017161501A

  • Lane information output method and lane information output device

    JP2018200501A

  • Method and apparatus for estimating a localized position on a map

    US20200174487A1