Vehicle, vehicle positioning method and apparatus, device, and computer-readable storage medium

Abstract

This application relates to the field of intelligent driving, in particular to a vehicle, a vehicle positioning method and apparatus, a device and a computer-readable storage medium. In the embodiment of this application, a lateral offset is first determined by using a road feature map of a forward road image, and then a second position of the vehicle is determined based on the road feature map, a first pose, a local map corresponding to the first pose and the lateral offset. This can reduce the computational resource consumption and time consumption while improving the accuracy of vehicle positioning, thereby simultaneously improving the reliability and timeliness of vehicle positioning, and further enhancing driving safety.

Description

[Technical Field]

[0001] This application relates to the field of intelligent driving, and in particular to a vehicle, a vehicle positioning method and apparatus, a device, and a computer-readable storage medium. [Background technology]

[0002] Vehicles are highly dependent on positioning technology. Positioning is a prerequisite for other vehicle functions, and positioning accuracy directly affects vehicle safety. Generally, the combined positioning of a global positioning system (GPS) and an inertial measuring unit (IMU) is used to achieve high-precision vehicle positioning. However, when the GPS has a poor signal or fails, drift occurs in the combined positioning of the GPS and IMU, resulting in reduced positioning accuracy.

[0003] Currently, in environments where GPS fails or has insufficient signals, IMUs are limited to using laser and visual methods to correct positioning results. These methods involve capturing and storing a feature map of the positioning environment as visual features, such as ORB (Oriented Fast and Rotated Brief) features or FAST (Features From Accelerated Segment Test) features, and then performing feature matching over a wide area or even the entire map using point matching methods such as the Iterative Closest Point (ICP) algorithm or Kd-Tree. This results in large data volumes and high computational complexity, resulting in serious waste of both computational and time resources. This results in high hardware costs, low operational efficiency, and inability to reliably perform real-time vehicle positioning, which poses safety risks to unmanned or intelligent vehicle operation. Summary of the Invention

[0004] To solve the above technical problems, the embodiments of this application provide a vehicle, a vehicle positioning method and apparatus, a device, and a computer-readable storage medium for improving the accuracy of vehicle positioning while reducing computational resource consumption and time consumption.

[0005] A first aspect of the present application provides a vehicle positioning method, comprising: acquiring an image of the road ahead of the vehicle; obtaining a road feature map based on a forward road image; determining a lateral offset of the vehicle based on the road feature map; obtaining a second position of the vehicle based on a lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; Includes.

[0006] Since the lateral offset may indicate the lateral road distance of the vehicle, the lateral offset and the first pose may be combined to narrow the range of image matching, thereby effectively reducing the amount of calculation and complexity while improving the accuracy of vehicle positioning.

[0007] In a possible implementation of the first aspect, determining the lateral offset of the vehicle based on the road feature map includes determining the lateral offset of the vehicle based on lane boundary line features in the road feature map. The lane boundary line features are reliable and complete. Therefore, by using the lane boundary line features as a basis for the lateral offset, an accurate and reliable lateral offset can be obtained.

[0008] In a possible implementation of the first aspect, the lane boundary line features in the road feature map include two lane boundary line features in a region of interest (ROI) in the road feature map, where the two lane boundary lines are located on the left and right sides of the first pose. The data volume of the ROI in the road feature map is small, and the lane boundary line features are complete and reliable. Therefore, the accuracy and reliability of the lateral offset of the vehicle can be further improved.

[0009] In a possible implementation of the first aspect, the step of determining the lateral offset of the vehicle based on lane boundary line features in the road feature map includes the steps of obtaining the lateral offset of the vehicle based on lateral pixel offsets and a preset pixel ratio, and determining the lateral pixel offset based on lane boundary line features in a top view of the ROI in the road feature map, where the lane boundary line features in the top view of the ROI in the road feature map are obtained by using the features of two lane boundary lines in the ROI in the road feature map. The method of obtaining the lateral offset based on lateral pixel offsets and pixel ratios is easy to implement, and therefore the calculation complexity can be further reduced.

[0010] In a possible implementation of the first aspect, the lateral pixel offset is the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, where the lane center point is the lane center point between the left and right lane boundary lines of the first pose, and the first camera is a camera that captures a forward road image. The position of the optical center of the first camera is easy to obtain, and the calculation amount of the coordinate transformation of the position is small. In this way, the calculation amount and calculation complexity can be further reduced.

[0011] In a possible implementation of the first aspect, the vehicle positioning method further includes obtaining a second attitude of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map, so that an accurate attitude of the vehicle can be obtained in real time.

[0012] In a possible implementation of the first aspect, the second position or the second attitude, or both, of the vehicle are determined based on a plurality of candidate pose points in a local map, and the pose of each candidate pose point is determined based on the lateral offset of the vehicle, the first pose, and the local map. In this way, the vehicle can be accurately positioned by using the candidate pose point selected by the lateral offset. The amount of calculation is small and the calculation complexity is low, which can effectively reduce the calculation resource consumption and the time consumption.

[0013] In a possible implementation of the first aspect, multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the correction position as the center, and the correction position is obtained based on the lateral offset of the vehicle and based on the lane center point position and road azimuth angle that correspond to the first pose and are in the local map. In this way, by using multiple candidate pose points in the same lane direction, the vehicle can be accurately positioned. The search range is smaller, which can further reduce the amount and complexity of calculations while improving the accuracy of vehicle positioning.

[0014] In a possible implementation of the first aspect, the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second pose is pose information of the candidate pose point selected by performing feature matching on the projection image and the road feature map, and the projection image is acquired based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map. The degree of match between the projection image and the road feature map can indicate the degree of closeness between the corresponding candidate pose point and the actual vehicle pose, so that accurate and reliable vehicle positioning results can be obtained.

[0015] In a possible implementation of the first aspect, the vehicle positioning method further includes: acquiring a third pose of the vehicle based on the second position, the first attitude of the vehicle, and the sensor positioning data; or acquiring a third pose of the vehicle based on the second position, the second attitude of the vehicle, and the sensor positioning data. In this way, through the fusion of multiple types of position data, a third pose with higher accuracy and better reliability can be acquired, thereby further improving the accuracy and reliability of vehicle positioning.

[0016] In a possible implementation of the first aspect, the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

[0017] In a possible realization of the first aspect, the local map is from a vector map, where the vector map information is complete and has high accuracy, thereby allowing the accuracy of vehicle positioning to be further improved.

[0018] A second aspect of the present application provides a vehicle positioning device, an image capture unit configured to capture an image of the road ahead of the vehicle; a feature acquisition unit configured to acquire a road feature map based on a forward road image; an offset determination unit configured to determine a lateral offset of the vehicle based on the road feature map; a position determination unit configured to obtain a second position of the vehicle based on a lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; Includes.

[0019] In a possible implementation of the second aspect, the feature acquisition unit is specifically configured to determine the lateral offset of the vehicle based on lane boundary line features in the road feature map.

[0020] In a possible implementation of the second aspect, the lane boundary line features in the road feature map include two lane boundary line features within a region of interest (ROI) in the road feature map, the two lane boundary lines being located on the left and right sides of the first pose.

[0021] In a possible implementation manner of the second aspect, the feature acquisition unit is specifically configured to acquire a lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio, wherein the lateral pixel offset is determined based on lane boundary line features in a top view of the ROI in the road feature map, and the lane boundary line features in the top view of the ROI in the road feature map are acquired by using the features of two lane boundary lines in the ROI in the road feature map.

[0022] In a possible implementation of the second aspect, the lateral pixel offset is the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, which is the lane center point between the left and right lane boundary lines of the first pose, and the first camera is a camera that captures the forward road image.

[0023] In a possible implementation manner of the second aspect, the vehicle positioning device further includes an attitude determination unit configured to obtain a second attitude of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map.

[0024] In a possible implementation of the second aspect, a second position or a second pose, or both, of the vehicle is determined based on a plurality of candidate pose points in a local map, and the pose of each candidate pose point is determined based on the lateral offset of the vehicle, the first pose, and the local map.

[0025] In a possible implementation of the second aspect, multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the corrected position as the center, and the corrected position is obtained based on the lateral offset of the vehicle and based on the lane center point position and road azimuth angle that correspond to the first pose and are in the local map.

[0026] In a possible implementation of the second aspect, the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second posture is posture information of a candidate pose point selected by performing feature matching on the projection image and the road feature map, and the projection image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

[0027] In a possible implementation manner of the second aspect, the vehicle positioning device further includes a fusion unit configured to obtain a third pose of the vehicle based on the second position, the first attitude of the vehicle and the sensor positioning data, or to obtain the third pose of the vehicle based on the second position, the second attitude of the vehicle and the sensor positioning data.

[0028] In a possible implementation of the second aspect, the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

[0029] In a possible realization of the second embodiment, the local map is from a vector map.

[0030] A third aspect of the present application provides a computing device including a processor and a memory, the memory storing program instructions that, when executed by the processor, enable the processor to perform the vehicle positioning method of the first aspect.

[0031] A fourth aspect of the present application provides a computer-readable storage medium that stores program instructions that, when executed by a computer, enable the computer to perform the vehicle positioning method of the first aspect.

[0032] A fifth aspect of the present application provides a computer program product including a computer program that, when executed by a processor, enables the processor to perform the vehicle positioning method of the first aspect.

[0033] A sixth aspect of the present application provides a vehicle including a first camera configured to capture a road ahead image and the vehicle positioning device of the second aspect or the computing device of the third aspect.

[0034] In a possible implementation of the sixth aspect, the vehicle further comprises one or more of the following: a GPS sensor, an IMU, a vehicle speed sensor, and an acceleration sensor.

[0035] In an embodiment of the present application, a lateral offset is first determined, and then a second position of the vehicle is determined based on the lateral offset. This can effectively narrow the scope of complex processing such as image matching and reduce the amount and complexity of calculations. In this way, in an embodiment of the present application, the accuracy of vehicle positioning can be improved while reducing the consumption of calculation resources and time costs. When the GPS fails or has an insufficient signal, or in various other cases, the accuracy, reliability, and timeliness of vehicle positioning can be synchronously improved, thereby improving the driving safety of the vehicle in various driving modes such as unmanned driving and intelligent driving.

[0036] These and other aspects of the present application will become clearer and more easily understood in the description of the embodiment(s) that follows. [Brief explanation of the drawings]

[0037] The features and relationships between features of this application will be further described below with reference to the accompanying drawings. The accompanying drawings are all examples, and some features are not drawn to scale. Furthermore, in some of the accompanying drawings, common features that are not essential to the field of this application may be omitted. Alternatively, additional features that are not essential to the application may be shown. The combination of features shown in the accompanying drawings is not intended to limit this application. Furthermore, in this specification, the same reference numerals represent the same content. Specific accompanying drawings are described as follows: [Figure 1] FIG. 1 is an exemplary diagram of an application scenario according to an embodiment of the present application. [Figure 2] 1 is a schematic flowchart of a vehicle positioning method according to an embodiment of the present application; [Figure 3] FIG. 2 is a schematic diagram of a road feature map in an example according to an embodiment of the present application; [Figure 4] FIG. 2 is a schematic diagram of a road feature map obtained by separating lane boundary line features in an example according to an embodiment of the present application; [Figure 5] 1 is a schematic diagram of an ROI in a camera coordinate system in an example according to an embodiment of the present application; [Figure 6] 1 is an exemplary diagram of an ROI in a road feature map in an example according to an embodiment of the present application. [Figure 7] FIG. 2 is a schematic diagram of a top view of an ROI in a road feature map in an example according to an embodiment of the present application; [Figure 8] 1 is a schematic diagram of a correction position, a first position, a lateral offset, and a relationship between the correction position, the first position, and the lateral offset in a reference coordinate system in an example according to an embodiment of the present application. FIG. [Figure 9] FIG. 2 is a schematic diagram of candidate pose points in an example according to an embodiment of the present application; [Figure 10] 1 is a schematic diagram of a specific implementation process of a vehicle positioning method according to an embodiment of this application; [Figure 11] 1 is a schematic diagram of the structure of a vehicle positioning device according to an embodiment of this application; [Figure 12] 1 is a schematic diagram of the structure of a computing device according to an embodiment of the present application; [Figure 13] 1 is a schematic diagram of a vehicle structure in an example according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0038] In this specification and claims, terms such as "first," "second," or similar terms such as unit A and unit B are used merely to distinguish between similar objects and do not indicate a particular order of the objects. It will be understood that the particular order or sequence may be interchanged where possible, thereby allowing the embodiments of this application described herein to be realized in orders other than those illustrated or described herein.

[0039] In the following description, the reference numerals S210 and S220 indicating steps do not necessarily indicate that the steps are performed in order, and the steps may be interchanged if possible, or may be performed simultaneously.

[0040] The following describes important terms and related terms in the embodiments of this application.

[0041] World coordinate system: Also called a measurement coordinate system or an objective coordinate system, this may be used as a reference for describing the three-dimensional position and three-dimensional posture of a camera and a measurement object. The world coordinate system is an absolute coordinate system of the objective three-dimensional world. Typically, the coordinate value Pw (Xw, Yw, Zw) on the three-dimensional coordinate axes indicates the three-dimensional position of an object, and the rotation angle (raw, pitch, yaw) of the object relative to the three-dimensional coordinate axes indicates the three-dimensional posture of the object. In an embodiment of this application, the world coordinate system may be used as a reference coordinate system for a global pose.

[0042] Camera coordinate system: The optical center of the camera is used as the coordinate origin. The Z axis and the optical axis are coincident and point to the front of the camera. The X axis points to the right of the camera. The Y axis points to the bottom of the camera. Usually, Pc(Xc,Yc,Zc) denotes the coordinate values ​​in the camera coordinate system.

[0043] The camera's extrinsic parameters may determine the relative positional relationship between the camera coordinate system and the world coordinate system. Parameters transformed from the world coordinate system to the camera coordinate system may include a rotation matrix R and a translation vector T. Pinhole imaging is used as an example. The camera's extrinsic parameters, world coordinates, and camera coordinates satisfy the relationship (1): Pc = RPw + T (1), where Pw is the coordinate value (Xw, Yw, Zw) in the world coordinate system, Pc is the coordinate value (Xc, Yc, Zc) in the camera coordinate system, T = (Tx, Ty, Tz) is the translation vector, and R = R(α, β, γ) is the rotation matrix, where γ is the rotation angle around the Z axis of the camera coordinate system, β is the rotation angle around the Y axis, and α is the rotation angle around the X axis. That is, the six parameters α, β, γ, Tx, Ty, and Tz are the camera's extrinsic parameters.

[0044] Camera intrinsic parameters: determine the projection relationship from three-dimensional space to a two-dimensional image and relate only to the camera. A small-hole imaging model is used as an example. When image distortion is not considered, the intrinsic parameters may include the camera's scale factor in two coordinate axes u and v in the pixel coordinate system, the principal point coordinate (x0, y0) relative to the image coordinate system, and the coordinate axis tilt parameter s. The scale factor in the u axis is the ratio of the physical length dx of each pixel in the x direction of the image coordinate system to the camera's focal length f. The scale factor in the v axis is the ratio of the physical length dy of each pixel in the y direction of the image coordinate system to the camera's focal length f. When image distortion is considered, the intrinsic parameters may include the camera's scale factor in two coordinate axes u and v in the pixel coordinate system, the principal point coordinate relative to the imaging plane coordinate system, the coordinate axis tilt parameter, and the distortion parameters. The distortion parameters may include three radial distortion parameters and two tangential distortion parameters of the camera. The intrinsic and extrinsic parameters of the camera may be obtained through calibration.

[0045] The pixel coordinate system, also known as the pixel coordinate system or orthographic pixel coordinate system, is an image coordinate system in units of pixels. The upper left vertex of the image plane is used as the origin, the u axis points horizontally to the right, and the v axis points vertically downward, and the u axis and v axis are parallel to the X axis and Y axis of the camera coordinate system, respectively. Typically, p(u,v) denotes a coordinate value in the pixel coordinate system. The pixel coordinate system indicates the location of a pixel in an image in units of pixels, while the image coordinate system indicates the location of a pixel in an image in physical units (e.g., millimeters).

[0046] Perspective projection: It is a perspective projection relationship from the 3D camera coordinate system to the 2D pixel coordinate system. Because the origin positions of the coordinate systems do not match and the scale sizes do not match, perspective projection involves a stretching transformation and a translation transformation. A pinhole camera model is used as an example. The perspective projection relationship from a point (Xc, Yc, Zc) in the camera coordinate system to (u, v) in the pixel coordinate system satisfies the transformation formula shown in Equation (2) below.

number

[0047] (u,v) are coordinate values ​​in the pixel coordinate system, (Xc,Yc,Zc) are coordinate values ​​in the camera coordinate system, and K is a matrix representation of the internal parameters of the camera.

[0048] If image distortion is not taken into consideration, equation (2) may be further expressed as equation (3).

number

[0049] f x is the ratio of the focal length f of the camera to the physical length dx of each pixel in the x direction of the image coordinate system, and indicates that the focal length in the x direction is described using pixels. y is the focal length f of the camera relative to the physical length dy of each pixel in the y direction of the image coordinate system, indicating that the focal length in the y direction is described using pixels. (u0,v0) denotes the coordinates of the intersection between the optical axis of the camera and the image plane (i.e., the principal point) in the pixel coordinate system, where u0 is the ratio of the horizontal coordinate x0 of the principal point in the image coordinate system to dx, and v0 is the ratio of the vertical coordinate y0 of the principal point in the image coordinate system to dy.

[0050] The top view may also be referred to as an aerial view, and coordinate values ​​may be transformed from the orthographic pixel coordinate system to the top view pixel coordinate system through an affine transformation.

[0051] Affine transformation: An affine transformation is a transformation in which, under the three-point collinearity condition of the perspective center, image point, and target point, the projection plane (i.e., the perspective plane) rotates around the trace line (i.e., the perspective axis) by a certain angle according to the perspective rotation law, destroying the original projection beam, but the geometric shape of the projection on the projection plane can still be kept unchanged.

[0052] The general formula for the affine transformation is the following formula (4).

number

[0053] (u,v) indicates the original coordinates, expressed in the form of an extended vector, and w = 1. In the affine transformation matrix M, a 11 , a 12 , a 21 and a 22 indicates a linear transformation (e.g., scaling, clipping, and rotation). 31 and a 32 is used for translation. 13 and a 23 is used to generate the perspective transformation.

[0054] Usually, the transformed coordinates (x, y) satisfy the following equation (5).

number

[0055] If the transformation matrix is ​​known, the following equations (6) and (7) can be obtained according to the transformation equations (5) and (4), and the transformed coordinates can be directly obtained by using the original coordinates according to the equations (6) and (7).

number

[0056] Here, the values ​​of the components of the affine transformation matrix M may be obtained through calculation according to the above formula by using the original coordinates and transformed coordinates of the three point pairs.

[0057] Global pose: Also called absolute pose, global pose may include the position and orientation of an object in a reference coordinate system. A vehicle is used as an example. The position of an object may be represented by using three-dimensional coordinate values ​​in the reference coordinate system (e.g., coordinate values ​​Pw(Xw, Yw, Zw) in the world coordinate system described above). The orientation of the object may be represented by using the pitch angle, yaw angle (also called yaw angle), and roll angle of the vehicle. The roll angle is the angle of rotation around the X axis, the pitch angle is the angle of rotation around the Y axis, and the yaw angle is the angle of rotation around the Z axis. Alternatively, the orientation of the object may be described by using a quaternion. In the embodiment of this application, the reference coordinate system of the global pose may be, but is not limited to, the world coordinate system described above (including coordinate values ​​and orientation angles), a geodetic coordinate system, the Universal Transverse Mercator (UTM) Grid System (also called the UTM coordinate system), etc.

[0058] Below, we first briefly analyze possible implementation methods.

[0059] As mentioned above, when the GPS fails or has insufficient positioning effectiveness, or in other cases, drift occurs in the integrated positioning of the GPS and IMU, resulting in reduced positioning accuracy.

[0060] In a possible implementation, the positioning system includes a visual positioning device and an inertial navigation device. The inertial navigation device includes an inertial measurement unit (IMU). The IMU acquires status information of the IMU (including the specific force acceleration and Euler angles of the IMU in the carrier coordinate system). The visual positioning device determines the linear acceleration of the IMU based on the status information, and then determines the position information of the unmanned vehicle based on the visual information and the linear acceleration of the IMU. This implementation can solve the problem of low positioning accuracy of unmanned vehicles. However, this implementation has accumulated errors, a large amount of visual information data, and high computational complexity. Therefore, the positioning accuracy of this implementation cannot reach the centimeter level, and computational resource consumption is high.

[0061] In a second possible implementation, the visual positioning method includes the steps of: acquiring images of markers in the environment where positioning needs to be performed in advance and establishing a corresponding marker database; capturing a target image of the marker in the environment where positioning needs to be performed; performing image matching between the target image and the image in the marker database after capturing the target image; and obtaining the current position through calculation based on known geometric features in the matched image. This implementation can solve the problem of low positioning accuracy when GPS is broken or has insufficient positioning effectiveness. However, positioning is mainly performed using images, and a large amount of image matching processing is required. Therefore, this implementation also has the problems of positioning accuracy not reaching the centimeter level and excessively large computational resource consumption.

[0062] In consideration of this, embodiments of this application provide a vehicle, a vehicle positioning method and apparatus, a device, and a computer-readable storage medium. First, a lateral offset is determined based on a road feature map of a forward road image. Then, a second position of the vehicle is obtained based on the lateral offset, a first pose of the vehicle, and a local map. Since the lateral offset can indicate the lateral road distance of the vehicle, the lateral offset and the first pose are combined, thereby narrowing the search range of the local map, i.e., narrowing the range of image matching. This can effectively reduce the amount and complexity of calculations while improving the accuracy of vehicle positioning. In this way, embodiments of this application can simultaneously improve the accuracy, reliability, and timeliness of vehicle positioning when GPS fails or has insufficient signals, or in various other cases.

[0063] The embodiments of the present application may be applicable to various scenarios requiring precise positioning. In particular, the embodiments of the present application may be particularly applicable to positioning of transportation means such as vehicles, ships, aircraft, and unmanned aerial vehicles, for example, real-time precise positioning in vehicle driving processes, positioning of unmanned guided vehicles in logistics scenarios, and real-time precise positioning of unmanned aerial vehicles in outdoor environments. Furthermore, the embodiments of the present invention may be applicable to various environments such as indoors, outdoors, roads, and fields.

[0064] A "vehicle" in the embodiments of this application may be any type of transportation means. For example, the "vehicle" herein may be, but is not limited to, a private car, a commercial vehicle, a bus, a passenger car, a high-speed rail, a subway, an unmanned vehicle, an unmanned aerial vehicle, a logistics transportation vehicle, an unmanned transportation vehicle, etc. The power type of the "vehicle" may be fuel-driven, purely electric, hydrogen fuel cell-driven, hybrid electric, etc. Furthermore, the "vehicle" herein may be a human-powered vehicle, an autonomous vehicle, an unmanned vehicle, or any other type of vehicle. Those skilled in the art will understand that any transportation means that needs to be positioned in real time may be considered a "vehicle" in the embodiments of this application.

[0065] 1 is a schematic diagram of an exemplary application scenario according to an embodiment of this application. In the scenario of FIG. 1, vehicle A, vehicle B, vehicle C, and vehicle D on a road may all determine their respective positions in real time by using the solution in the embodiment of this application. For example, in scenarios where GPS is down or has insufficient signals, scenarios with low accuracy of vehicle sensors, tunnel scenarios, underground garages, or various other scenarios, real-time, accurate, and reliable positioning of vehicles can be achieved by using the embodiment of this application.

[0066] For example, see Figure 1. The vehicle's positioning location may be indicated by the location of the vehicle's rear axle center (the solid black dot in Figure 1).

[0067] Specific implementation methods of the embodiments of this application will be described in detail below.

[0068] 2 is a schematic flowchart of a vehicle positioning method according to an embodiment of this application. Please refer to FIG. 2. The vehicle positioning method provided in this embodiment of this application may include the following steps:

[0069] Step S210: An image of the road ahead of the vehicle is acquired.

[0070] The forward road image may be captured by using a first camera mounted on the vehicle, and the forward road image includes lane boundary lines and markers of the road ahead of the vehicle, where the markers may include all traffic signs and road markings such as, but not limited to, traffic lights, underground parking poles, walls, toll booths, road gates, collision prevention strips, obstacles, storage positions, storage position lines, no-parking zones, pedestrian crossings, speed belts, slow down, stop, yield lines, and yields, road sign boards, utility poles, road edges, trees, shrubs, medians, guardrails, etc. Furthermore, the forward road image may further include lane centerlines, guide lines, guide arrows, etc.

[0071] The forward road image is used to determine a second position and / or a second attitude of the vehicle based on the first pose of the vehicle, and the second pose formed by the second position and the second attitude is a pose subsequent to the first pose. In other words, the forward road image is a forward road image captured during a period from when the vehicle reaches the first pose until the vehicle has just reached or has not yet reached the second pose. For example, the capture time of the forward road image may be the following time: when the vehicle reaches the first pose, when the vehicle reaches the second pose, or when the vehicle has just reached the first pose until the vehicle has not yet reached the second pose.

[0072] The road ahead image may be, but is not limited to, a time-of-flight (TOF) image, a red-green-blue (RGB) image, or other types of image. The first camera may be, but is not limited to, a TOF camera, an RGB camera, etc. The arrangement method, installation position, specific type and number of the first camera, and specific type of road ahead image, etc. are not limited in the embodiments of this application.

[0073] Step S220: Obtain a road feature map based on the road ahead image.

[0074] In some embodiments, the road feature map of the forward road image may be obtained by performing one or more operations on the forward road image, such as semantic feature extraction, post-fit processing (e.g., fitting incomplete lane boundary lines to long straight lines), and feature skeletonization (e.g., lane boundary line feature skeletonization, utility pole feature skeletonization, guide line contouring, and signboard contouring). The specific implementation manner of obtaining the road feature map is not limited in the embodiments of this application.

[0075] In some embodiments, the road feature map may be a grayscale map, and the pixel values ​​of the road feature map may range from [0, 255]. In a specific application, different types of road features may be indicated by using pixel values. For example, the pixel value of a lane boundary feature may be preset to 255, the pixel value of a utility pole feature may be set to 135, the pixel value of a guide arrow feature may be set to 95, and the pixel value of a signboard feature may be set to 45. In this way, in order to efficiently and accurately extract required features from the road feature map, different types of road features in the road feature map may be distinguished by using pixel values. For example, if a lane boundary feature needs to be extracted in the road feature map, only pixels with a pixel value of 255 in the road feature map need to be extracted. If the pixel value of a pixel in the road feature map is 135, this indicates that the pixel is a utility pole feature point.

[0076] In some embodiments, the road feature map may include lane boundary line features and marker features of the road in the forward road image. Figure 3 is an exemplary diagram of a road feature map. In the example in Figure 3, the road feature map may include a lane boundary line feature 31, a utility pole feature 32, a traffic light feature 33, etc. From Figure 3, it can be seen that the road feature map includes two lane boundary line features located on the left and right sides of the first pose.

[0077] Step S230: Determine the lateral offset of the vehicle based on the road feature map.

[0078] In some embodiments, the lateral offset of the vehicle may indicate the road lateral distance from the vehicle (e.g., the rear axle center of the vehicle or the optical center of the first camera on the vehicle) to the lane centerline. The lateral offset may be, but is not limited to, an actual lateral offset or a lateral pixel offset. The actual lateral offset is the lateral offset in a reference coordinate system. The lateral pixel offset is the lateral offset in a top view of the road feature map.

[0079] In some embodiments, the lateral offset of the vehicle may be determined based on lane boundary line features in the road feature map. The lane boundary line features are complete and reliable. Therefore, by using the lane boundary line features as the basis for the lateral offset, a lateral offset with better accuracy and reliability can be obtained. In specific applications, other road features, such as lane centerlines, guide lines, or guide arrows, may alternatively be used as the basis for determining the lateral offset.

[0080] In some embodiments, the lane boundary line features in the road feature map may include two lane boundary line features within an ROI in the road feature map, where the two lane boundary lines are located on the left and right sides of the first pose. In other words, the lateral offset of the vehicle may be determined based on the two lane boundary line features within the ROI in the road feature map. Because the data volume of the ROI in the road feature map is smaller and the features of the two lane boundary lines on the left and right sides of the first pose are reliable and complete, the lateral offset is obtained by using the two lane boundary line features within the ROI in the road feature map. This can further reduce the computational complexity, computational resource consumption, and time consumption while improving the lateral offset accuracy. In specific applications, the lateral offset may alternatively be obtained by using lane boundary line features of other features or other regions in the road feature map. The features used to determine the lateral offset of the vehicle are not limited in the embodiments of this application.

[0081] In some embodiments, the lateral offset of the vehicle may be obtained based on the lateral pixel offset and a preset pixel ratio. The lateral pixel offset is determined based on lane boundary line features in the top view of the ROI in the road feature map, and the lane boundary line features in the top view of the ROI in the road feature map are obtained by using the features of two lane boundary lines in the ROI in the road feature map. Because the data volume of the ROI in the road feature map is smaller and the features of the two lane boundary lines included in the road feature map are reliable and complete, the top view of the ROI in the road feature map is equivalent to an aerial feature map of the road ahead of the vehicle. The method of obtaining the lateral offset based on the lateral pixel offset and the pixel ratio is simple and easy to implement, and the lateral offset of the vehicle is obtained based on the lateral pixel offset. This can further reduce calculation complexity, calculation resource consumption, and time consumption while improving the lateral offset accuracy.

[0082] In some embodiments, the lateral pixel offset may be the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, where the lane center point is the lane center point between the left and right lane boundary lines of the first pose. Since the optical center of the first camera is the coordinate origin of the camera coordinate system and the pixel coordinate system of the first camera, and the ROI mapping point position of the optical center of the first camera is easy to obtain, the lateral pixel offset is obtained by using the ROI mapping point of the optical center of the first camera. This can further reduce the computational complexity.

[0083] In a specific application, the lateral pixel offset may alternatively be obtained by using the ROI mapping point position of another rigid position point on the vehicle (e.g., the rear axle center of the vehicle). The specific manner of obtaining the lateral pixel offset is not limited in the embodiments of this application.

[0084] In some embodiments, an exemplary specific implementation process for determining the lateral offset may include the following steps (1) to (4).

[0085] (1) Determine an ROI in a road feature map based on a preset ROI size parameter, where the ROI in the road feature map includes the left and right lane boundary line features of the first pose.

[0086] (2) Determine an affine transformation matrix of the first camera based on the endpoint positions of the ROI in the road feature map and a preset pixel ratio, and obtain a top view of the ROI in the road feature map through the affine transformation of the first camera based on the affine transformation matrix.

[0087] (3) Determine the lane center point position in the top view of the ROI in the road feature map and the ROI mapping point position of the optical center of the first camera, and use the distance between the lane center point position and the ROI mapping point position as the lateral pixel offset.

[0088] (4) The product of the horizontal pixel offset and the pixel ratio is used as the actual horizontal offset, and the actual horizontal offset is the final horizontal offset.

[0089] In the above exemplary implementation, the lateral offset of the vehicle may be obtained through perspective projection and affine transformation of the first camera based on lane boundary line features. The amount of calculation is small and the computational complexity is low. Furthermore, this can improve the accuracy of the lateral offset and improve the accuracy of vehicle positioning, while further reducing the computational resource consumption and time consumption.

[0090] Step S240: Obtain a second position of the vehicle based on the lateral offset of the vehicle, the first pose of the vehicle, the road feature map, and the local map corresponding to the first pose.

[0091] The first pose indicates an initial pose of the vehicle or a pose at a previous point in time. The first pose may include a first position and a first attitude. The first position is the initial position of the vehicle or a position at a previous point in time, and the first attitude is the initial attitude of the vehicle or an attitude at a previous point in time.

[0092] The second pose is a pose subsequent to the first pose, and may also be referred to as a visual positioning pose of the first pose. The second pose may include a second position and a second attitude. If the position of the vehicle does not change and only the attitude changes, the second position is equivalent to the first position. If the position of the vehicle changes, the second position may be obtained by performing the processes in steps S210 to S240 on the first pose. If the attitude of the vehicle does not change or if changes in the attitude of the vehicle are not a concern, the second attitude may be equivalent to the first attitude. If the attitude of the vehicle needs to be updated and the attitude of the vehicle changes, the second attitude may alternatively be obtained by performing the processes in steps S210 to S240 on the first pose.

[0093] The third pose is a pose (which may be called a fusion positioning pose) acquired by performing the processes in steps S210 to S250 on the first pose. In other words, the third pose may be acquired by performing the process in step S250 on the second pose.

[0094] Both the second pose and the third pose may be used as the positioning pose or the current pose of the vehicle. In practical application, the second pose and / or the third pose may be selected as the final pose or the current pose of the vehicle based on requirements. Here, the first pose, the second pose and the subsequent third pose may be the above global pose.

[0095] In some embodiments, step S240 may further include acquiring a second pose of the vehicle based on the lateral offset, the first position, the first pose, the road feature map, and the local map. In a specific application, the second pose may be acquired synchronously with the second position, or the second pose and the second position may be acquired sequentially, but the acquisition order is not limited.

[0096] In some embodiments, the embodiment of the present application may further include a step of acquiring a local map corresponding to the first pose of the vehicle. In practical application, data of the local map may be acquired from a map in the cloud, or may be extracted from a locally stored map, or may be acquired before the positioning method in the embodiment of the present application starts, or may be acquired in real time during the execution process of the embodiment of the present application (e.g., in or before step S240).

[0097] The local map is a portion of the map that corresponds to the area around the first pose. In some embodiments, the local map may be a preset region centered on the corresponding position of the first pose in the map, and the preset region may be a circular region, a rectangular region, a triangular region, or a region of other user-defined shape. A circular region is used as an example. The local map may be a circular region where the corresponding position of the first pose is used as the center of the circle and a preset length is used as the radius in the map. The preset length may be flexibly set or may be determined based on the vehicle speed. For example, the preset length may be 30 meters to 50 meters by default.

[0098] Because the data in the vector map is accurate and complete, in order to improve the accuracy of vehicle positioning, the local map in the embodiments of this application may be from a vector map. In some embodiments, the vector map may be a high-definition vector map or a high-precision vector map. In specific applications, the vector map may be from the cloud or may be built on a storage device of the vehicle.

[0099] In some embodiments, the information recorded in the local map includes, but is not limited to, position information of each point in each lane, position information of lane center points, road azimuth angle information, marker information, etc. Road features (e.g., markers, lane boundary lines, etc.) in the local map may be represented by using vector lines, curves, or straight lines, and different types of road features may be distinguished by using pixel values. Road azimuth angle information for each lane in the local map may be associated with position information, and road azimuth angles corresponding to different positions on the same lane may be the same or different. Furthermore, the local map may also record information on various road signs, such as guide arrows and guide lines. The types, visual representations, specific recording contents, etc. of the local map are not limited in the embodiments of this application.

[0100] In some embodiments, the second position or the second pose, or both, may be determined based on multiple candidate pose points in a local map, and the pose of each candidate pose is determined based on the lateral offset, the first pose, and the local map. In this way, the vehicle can be accurately positioned by using the candidate pose points selected based on the lateral offset, without needing to perform large-scale global image matching on the area around the first pose. The amount of calculation is small and the computational complexity is low. This can effectively reduce computational resource consumption and time consumption while improving the accuracy of vehicle positioning.

[0101] In some embodiments, multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the corrected position as the center, and the corrected position is obtained based on the lateral offset and the lane center point position and road azimuth angle corresponding to the first pose and located in the local map. In this way, using multiple candidate pose points in the same lane direction can accurately position the vehicle. The search range is smaller, and the image matching range is further narrowed. This can further reduce the computational resource consumption and time consumption while improving the accuracy of vehicle positioning.

[0102] In some embodiments, the second position may be position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second pose may be pose information of a candidate pose point selected by performing feature matching on the projection image and the road feature map, where the projection image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map. Since the degree of match between the projection image and the road feature map may indicate the degree of closeness between the corresponding candidate pose point and the actual vehicle pose, accurate and reliable vehicle positioning results can be obtained by performing feature matching between the projection image and the road feature map.

[0103] In some examples, the candidate pose point selected by performing feature matching on the projection image and the road feature map may be any candidate pose point among multiple candidate pose points whose matching cost between the projection image and the road feature map is less than a preset threshold, or the candidate pose point having the minimum matching cost between the projection image and the road feature map. The minimum matching cost between the projection image and the road feature map indicates the highest matching degree between the projection image and the road feature map, and also indicates that the corresponding candidate pose point is closest to the actual pose of the vehicle. Therefore, the pose of the candidate pose point having a matching cost less than the preset threshold or the minimum matching cost is closer to the actual pose of the vehicle. In this way, more accurate and reliable vehicle positioning results can be obtained.

[0104] In some embodiments, the vehicle positioning method provided in the embodiments of this application may further include a step S250 of acquiring a third pose of the vehicle based on the second position, the first attitude, and the sensor positioning data, or a step S260 of acquiring a third pose of the vehicle based on the second position, the second attitude, and the sensor positioning data. Here, the sensor positioning data may include one or more of the following: GPS information, IMU information, Inertial Navigation System (INS) vehicle attitude information, and chassis information. In this way, the second attitude and real-time positioning information of other sensors may be fused to acquire a third pose with higher accuracy and better reliability, compensating for the effects of external environmental conditions such as poor performance of the first camera and poor lighting on the accuracy of the second pose, and further improving the accuracy and reliability of vehicle positioning.

[0105] In some embodiments, an Extended Kalman Filter (EKF) may be used to perform multi-sensor information fusion to obtain the third pose of the vehicle through calculation, for example, by using a covariance value of the GPS information, the IMU information, the INS vehicle attitude information, the chassis information, and the second pose as an information validity weight, fusion may be performed on the GPS information, the IMU information, the INS vehicle attitude information, the chassis information, and the second pose to obtain the third pose of the vehicle. Here, the specific algorithm of the fusion calculation is not limited in the embodiments of this application.

[0106] In some embodiments, an exemplary implementation procedure of step S230 may include the following steps a1 to a5.

[0107] Step a1: Isolate lane boundary features.

[0108] In some embodiments, different markers in the road feature map may be classified by using pixel values. Assuming that the pixel value of a lane boundary line feature is agreed upon as m, the pixel in the road feature map with pixel value m may be extracted to form a road feature map containing only lane boundary line features. Figure 4 is a road feature map obtained by separating the lane boundary line features in Figure 3.

[0109] This step is optional. The step of separating lane boundary line features may be applied or omitted based on actual application requirements. The road features are separated in advance, which can reduce the amount of data in the subsequent processing process, reduce the calculation complexity, and filter out unnecessary feature data. If other features such as lane centerlines are laterally offset, other features such as lane centerlines may also be separated in a similar manner.

[0110] Step a2: Obtain an ROI in the road feature map.

[0111] The ROI may be selected as a fixed field of view area with high detection accuracy and precision. In practical applications, nearby objects have larger representations in the image and are more likely to be detected. Distant objects have smaller representations in the image and are less likely to be detected. Furthermore, objects at the edge of the field of view are more likely to be incomplete objects in the image, resulting in a high probability of detection error. In areas close to but not at the edge of the field of view, the image data is complete and accurate, and the detection accuracy is high. Therefore, a partial field of view area close to the camera but not at the edge is usually agreed upon as the ROI. In specific applications, the ROI may be preset as the actual size range of the fixed field of view area.

[0112] In some embodiments, a region of interest (ROI) may be selected in the camera coordinate system of the first camera, and size parameters of the ROI are preset. The size parameters may include, but are not limited to, the positions of the ROI corner points in the camera coordinate system, the length of the ROI, and the width of the ROI. A rectangular ROI is used as an example. The size parameters of the ROI may include, but are not limited to, the positions of the four ROI corner points (e.g., the coordinate values ​​of the four corner points in the camera coordinate system of the first camera), the length of the ROI, and the width of the ROI.

[0113] In this step, the ROI endpoints in the road feature map may be obtained by transforming the ROI endpoints into the pixel coordinate system of the first camera through the perspective projection of the first camera, and the rectangular area defined by these endpoints is the ROI in the road feature map.

[0114] 5 is a schematic diagram of the ROI in the camera coordinate system of the first camera. In the example in FIG. 5, in the camera coordinate system XCY, where the actual position C of the first camera (i.e., the position of the optical center of the first camera in the reference coordinate system) is used as the coordinate origin, a rectangular area bounded by four points P1, P2, P3, and P4 is selected on the ground as the ROI. The size parameters of the ROI are determined by the coordinate values ​​P1(x p1 ,y p1 ,z p1 ), P2(x p2 ,y p2 ,z p2 ), P3(x p3 ,y p3 ,z p3 ) and P4(x p4 ,y p4 ,z p4 ), the length m of the ROI (i.e., the actual distance in meters from P1 to P2), and the width n of the ROI (i.e., the actual distance in meters from P1 to P3).

[0115] In some embodiments, when the ROI in the road feature map is acquired, an ROI mapping point position of the optical center of the first camera may be further determined. Specifically, the ROI mapping point position of the optical center of the first camera in the camera coordinate system may be determined based on the positions of the ROI end points in the camera coordinate system, and the ROI mapping point position of the optical center of the first camera in the pixel coordinate system may be obtained through perspective projection of the first camera based on the ROI mapping point position of the optical center of the first camera in the camera coordinate system.

[0116] See the example in Figure 5. A perpendicular line is drawn from the actual position C of the first camera to the ROI, and the intersection of the perpendicular line (i.e., the Y axis of the camera coordinate system of the first camera) with the side edge of the ROI that is closer to the camera is the ROI mapping point S of the optical center of the first camera. The positions of the ROI edge points in the camera coordinate system are known, and the position of the ROI mapping point S of the optical center of the first camera (S X, S Y ,S Z ) may be determined, i.e., S Y = 0, and S X = 0, and S Z =z P3 =z P4 is.

[0117] The coordinate values ​​of the four points P1(x P1 ,y P1 ,z P1 ), P2(x P2 ,y P2 ,z P2 ), P3(x P3 ,y P3 ,z P3 ) and P4(x P4 ,y P4 ,z P4 ) and the position of the ROI mapping point S (S X, S Y ,S Z ) is known, and the positions P1'(u P1 ,v P1 ), P2'(u P2 ,vP2 ), P3'(u P3 ,v P3 ) and P4'(u P4 ,v P4 ), and the position S'(u S ,v S ) may be obtained through perspective projection of Equation (2) and Equation (3). In this way, the ROI in the road feature map is obtained. Figure 6 is an exemplary diagram of the ROI in the road feature map. In Figure 6, the dashed box indicates the boundary of the ROI, and the solid lines indicate the lane boundary lines.

[0118] Step a3: Determine the endpoint positions of the two adjacent left and right lane boundary lines within the ROI in the road feature map.

[0119] In some embodiments, two adjacent left and right lane boundaries L1 and L2 may be detected within the ROI in the road feature map by using the Hoff line detection method or the like, and the end points L of the lane boundary L1 may be determined. 1up (x l1 ,y l1 ) and L 1down (x l2 ,y l2 ) and the end point position L of the lane boundary line L2 2up (x l3 ,y l3 ) and L 2down (x l4 ,y l4 ) is determined. In this way, two adjacent left and right lane lines within the ROI may be selected based on the distribution of lane lines within the ROI in the road feature map as a basis for determining the lateral offset to obtain a more reliable lateral offset. Figure 6 shows two adjacent left and right lane lines (thick black lines in Figure 6) to a vehicle within the ROI in the road feature map, and the endpoints of the two lane lines.

[0120] Step a4: Through affine transformation, obtain the lane boundary line features in the top view of the ROI in the road feature map and the ROI mapping point position of the optical center of the first camera.

[0121] In some implementations, an exemplary implementation process of this step may include the following sub-steps (1) and (2).

[0122] (1) Construct a top view corresponding to the first camera, and determine an affine transformation matrix M between the ROI in the road feature map and the top view of the ROI in the road feature map.

[0123] Specifically, a top view with a pixel ratio of k (units are pixels per meter, i.e., the number of pixels per meter) may be pre-constructed, and B w Width and B h Based on the length m of the ROI and the width n of the ROI in the camera coordinate system of the first camera, B w and B h may be obtained according to equation (8).

number

[0124] Specifically, the ROI end point P1'(u P1 ,v P1 ), P2'(u P2 ,v P2 ), P3'(u P3 ,v P3 ) and P4'(u P4 ,v P4 ) are known. The end point positions P1''(0,0) and P2''((B w -1),0), P3''((B w -1),(B h -1)) and P4''(0,(B h -1)) is B w and B h In this way, the affine transformation matrix M between the ROI in the road feature map and the top view of the ROI in the road feature map may be obtained according to the above equations (6) and (7) of the affine transformation.

[0125] (2) Through affine transformation, obtain the lane boundary lines L3 and L4 of the top view of the ROI in the road feature map and the ROI mapping point S''.

[0126] First, the lane boundary lines L1 and L2 in the ROI in the road feature map are transformed into the pixel coordinate system of the top view through affine transformation to obtain the corresponding lane boundary lines L3 and L4 in the top view of the ROI in the road feature map. Specifically, the end point positions L1 and L2 corresponding to the lane boundary lines L3 and L4 in the top view are calculated. 3up (x l5 ,y l5 ), L 3down (x l6 ,y l6 ), L 4up (x l7 ,y l7 ) and L 2down (x l8 ,y l8 ) are the end points L of lane boundary lines L1 and L2 1up (x l1 ,y l1 ), L 1down (x l2 ,y l2 ), L 2up (x l3 ,y l3 ) and L 2down (x l4 ,y l4 ) into the affine transformation equations (6) and (7).

[0127] Then, equation (9) for lane boundary line L3 and equation (10) for lane boundary line L4 may be obtained based on the end point positions of lane boundary lines L3 and L4.

number

[0128] k L3 indicates the slope of the lane boundary line L3, and b L3 indicates the y-axis intercept of the lane boundary line L3, and k L4 indicates the slope of the lane boundary line L4, and b L4 indicates the y-axis intercept of lane boundary line L4.

[0129] Furthermore, the ROI mapping point position S'(u S ,v S ) is substituted into equations (6) and (7) of the affine transformation to obtain the ROI mapping point position S"(x1, y1) of the optical center of the first camera in the top view of the ROI in the road feature map. FIG. 7 is an example diagram of the top view of the ROI in the road feature map. FIG. 7 also shows lanes L3 and L4 and the ROI mapping point S" in the top view of the ROI in the road feature map. The dashed box indicates the top view of the ROI in the road feature map, the solid line in the dashed box indicates the lane boundary line, and the dashed line between the two solid lines indicates the lane cross-section line.

[0130] Step a5: Calculate the horizontal offset.

[0131] Specifically, the position Q(x2, y2) of the lane center point in the top view of the ROI in the road feature map may be obtained based on equation (9) of the lane boundary line L3 and equation (10) of the lane boundary line L4, and the lateral pixel offset is obtained based on the position Q(x2, y2) of the lane center point and the ROI mapping point position S''(x1, y1), and then the actual lateral offset is obtained based on the lateral pixel offset and the preset pixel ratio.

[0132] See the example in Figure 7. In the pixel coordinate system of the top view, a perpendicular line is drawn through S''(x1,y1) to the lane boundary line L3 or the lane boundary line L4, and the intersection point Q1 between the perpendicular line and the lane boundary line L3 and the intersection point Q2 between the perpendicular line and the lane boundary line L4 are the two end points of the lane cross section line. The midpoint of the lane cross section line Q1Q2, i.e., the midpoint between the intersection point Q1 and the intersection point Q2, is the position Q(x2,y2) of the lane center point.

[0133] The coordinates of the two end points of the lane cross section line Q1(l x1 ,l y1 ) and Q2(l x2 ,l y2) may be determined based on equation (9) for lane boundary line L3 and equation (10) for lane boundary line L4, and the ROI mapping point S''(s1, y1) in the top view of the ROI in the road feature map. Then, the position Q(x2, y2) of the lane center point may be obtained according to equation (11).

number

[0134] The lateral pixel offset is the road lateral offset in the pixel coordinate system of the top view of the ROI in the road feature map. From FIG. 7, it can be seen that the distance between the ROI mapping point S''(x1, y1) and the lane center point Q(x2, y2) in the pixel coordinate system of the top view of the ROI in the road feature map is the lateral pixel offset. In other words, the lateral pixel offset offset1 may be obtained according to Equation (12).

number

[0135] In some embodiments, the actual lateral offset is the road lateral offset in the reference coordinate system. Specifically, the actual lateral offset, offset2, may be obtained according to Equation (13).

number

[0136] offset2 denotes the actual lateral offset, and k denotes the preset pixel ratio.

[0137] In some embodiments, an exemplary implementation procedure of step S240 may include the following steps b1 to b3.

[0138] Step b1: Correct the first position based on the lateral offset to obtain a corrected position corresponding to the first pose.

[0139] In some embodiments, the correction position H start (H x0 ,H y0 ,H z0 ) may be obtained according to equation (14).

number

[0140] 8 is a schematic diagram of the correction position, the first position, and the lateral offset in the reference coordinate system. In FIG. 8, the coordinate axis XOY is the XY plane of the reference coordinate system, O is the coordinate origin of the reference coordinate system, and E start indicates the first position, and C start is the first position E in the local map start , and offset2 indicates the actual lateral offset. From Figure 8, the lateral offset component of each coordinate axis is calculated based on the road azimuth angle yaw l It can be seen that the corrected position H start The coordinate values ​​of may be obtained by adding the lateral offset components of each coordinate axis to the corresponding coordinate values ​​of the lane center point.

[0141] Step b2: Determine candidate pose points.

[0142] In the implementation, the candidate pose point is the corrected position H start (H x0 ,H y0 ,H z0 ) as the center, the pose of each candidate pose point is selected in the lane direction of the local map, and the pose of each candidate pose point is calculated by the corrected position H start (H x0 ,H y0 ,H z0 ) and road azimuth information in the local map.

[0143] Specifically, the correction position H start (H x0 ,H y0 ,H z0 ), the preset search range and the preset search step are known, and road direction data yaw in the local map is i Based on this, the corrected position H start (H x0 ,H y0 ,H z0 ) is the 0th candidate pose point H z0 The number of candidate pose points (2n+1) and the position information of each candidate pose point (H xi ,H yi ,H zi ) may be determined according to equations (15) to (18).

number

[0144] scope indicates the preset search scope, step indicates the preset search step, and H i (H xi ,H yi ,H zi ) indicates the position of the i-th candidate pose point, and yaw i (i=-n,-n+1,...,-1,0,1,...,n-1,n) indicates the road azimuth angle corresponding to the i-th candidate pose point. imay be obtained directly from the local map. From equation (18), it can be seen that the number of candidate pose points is 2n+1, where n is an integer equal to or greater than 1. From equations (15) to (17), H z-1 and H z1 is the 0th candidate pose point H z0 It can be seen that the position is the same as

[0145] 9 is an exemplary diagram of candidate pose points. In FIG. 9, the white origin is the candidate pose point H i The black circle indicates the correction position H start L5 indicates the corrected position H start , L0 indicates the lane direction passing through, L0 indicates the lane centerline, XOY is the XY plane of the reference coordinate system, and O is the coordinate origin of the reference coordinate system. From Equations (15) to (18) and Figure 9, it can be seen that the candidate pose points are evenly distributed in the same lane direction in the local map, and the lane direction passes through the correction position. In this way, the range of feature matching can be reduced to the lanes in the map grid, and it is not necessary to cover all points around the vehicle's position in the map. The computational cost of projection imaging is greatly reduced, and the range of point matching in feature matching is greatly reduced. It can be seen that the computational cost is greatly reduced, and the computational complexity is also greatly reduced.

[0146] In some implementations, if the vehicle posture is essentially unchanged, or if the change in the vehicle posture is not relevant, or if the vehicle posture does not need to be determined synchronously, then the posture information (raw i , pitch i ,yaw i ) can be the first orientation. For example, if the first orientation is (raw start , pitch start ,yaw start ), the pose information of each candidate pose point is also the first pose (raw start , pitch start ,yaw start ) In other words, the pose of the i-th candidate pose point is H i (H xi ,H yi ,H zi ,rawstart , pitch start ,yaw start ) is also acceptable.

[0147] In some implementations, when the vehicle pose needs to be determined synchronously, the pose information of each candidate pose point is calculated based on the position information H i (H xi ,H yi ,H zi For example, the pose information of each candidate pose point may be determined in a manner similar to that of the first pose (raw start , pitch start ,yaw start It should be noted that if the vehicle pose needs to be determined synchronously, the number of candidate pose points is not limited to the above "2n+1".

[0148] When a second pose of the vehicle is determined, the candidate pose points and the position and pose information of each candidate pose point may be selected in a similar manner, which can also significantly reduce the amount of calculation and the computational complexity.

[0149] Step b3: Projection imaging is performed.

[0150] In this step, for each candidate pose point, projection imaging may be performed by using the local map to obtain a projection image of each candidate pose point.

[0151] In some implementations, the projection imaging process of a single candidate pose point may include: adjusting the extrinsic parameters of the first camera by using the pose of the candidate pose point; projecting pixels on the marker vector line of the local map into a pixel coordinate system of the first camera based on the extrinsic and intrinsic parameters of the first camera; synchronously establishing linear correspondences between pixels in the pixel coordinate system based on the linear correspondences between pixels in the local map; and rendering the corresponding pixels based on preset pixel values ​​of each marker to obtain a projection image of the candidate pose point. In this way, the projection image may be equivalent to a road feature map of a forward road image of the first camera having an optical center at the candidate pose point.

[0152] Here, markers include, but are not limited to, all traffic signs and road markings such as traffic lights, underground parking poles, walls, toll booths, road gates, collision avoidance zones, obstacles, storage locations, storage location lines, no-parking zones, pedestrian crossings, speed belts, slow downs, stops, yield lines and yields, road sign boards, utility poles, road edges, trees, shrubs, medians, guardrails, etc.

[0153] In some implementations, according to the above principles of equations (1) to (3), the process of projecting the pixels of the markers in the local map onto the pixel coordinate system of the first camera may be realized according to the following equation (19):

number

[0154] (u,v) indicates the coordinate value of a pixel in the pixel coordinate system of the first camera, and (X W ,Y W ,Z W ) indicates the coordinate value of the pixel of the marker in the local map in the reference coordinate system, R is the rotation matrix of the first camera, T is the translation matrix of the first camera, and R is the pose information (raw i , pitch i ,yaw i), and the rotation matrix in the initial extrinsic parameters of the first camera, and T is determined based on the position information of the candidate pose points (H xi ,H yi ,H zi ), and the translation matrix among the initial extrinsic parameters of the first camera.

[0155] In some implementations, to facilitate feature matching, the preset pixel value of each marker in the projection image may match the preset pixel value of the corresponding marker in the road feature map.

[0156] Step b4: Obtain a second pose of the vehicle through feature matching.

[0157] In this step, feature matching is performed between the projection image of each candidate pose point and the road feature map to obtain a matching cost corresponding to the candidate pose point, and the pose of the candidate pose point with the smallest matching cost among all the candidate pose points is selected as the second pose of the vehicle, i.e., the position information of the candidate pose point with the smallest matching cost is used as the second position of the vehicle, and the posture information of the candidate pose point with the smallest matching cost is used as the second posture of the vehicle.

[0158] In some implementations, feature matching between the projection image of each candidate pose point and the road feature map may include the following: For each pixel p1 in the road feature map, the complete projection image is traversed to find a pixel p2 in the projection image whose pixel value is closest to pixel p1, and the Euclidean distance D1 between pixel p2 in the projection image and pixel p1 in the road feature map is calculated. In this way, the Euclidean distance D1 between each pixel in the road feature map and the corresponding pixel in the projection image is obtained. The sum of the Euclidean distances D1 corresponding to all pixels in the road feature map is the matching cost between the projection image and the road feature map.

[0159] 10 is an exemplary specific implementation procedure of the vehicle positioning method according to an embodiment of this application. Refer to FIG. 10. The exemplary specific implementation process of the vehicle positioning method provided in the embodiment of this application may include the following steps:

[0160] Step S1010: A first camera captures a road image ahead of the vehicle at the current time, and provides the road image ahead to a computing device.

[0161] Step S1020: The computing device obtains a local map corresponding to a first pose, where the first pose is a global pose of the vehicle at a previous time point.

[0162] Step S1030: The computing device determines a second pose of the vehicle based on the forward road image and a local map corresponding to the first pose.

[0163] Specifically, first, a road feature map of the forward road image is obtained by performing processes such as semantic feature extraction, semantic feature skeletonization, and post-fit processing; second, an ROI is extracted in the road feature map; a lateral offset is determined based on lane boundary line features in the ROI in the road feature map and the first pose; third, a corrected position of the first position is obtained based on the lateral offset; and a longitudinal search is performed in the lane direction of the corrected position to obtain a second pose of the vehicle. Here, the longitudinal search in the lane direction of the corrected position is the above process of determining the second position of the vehicle. The process may include selecting and determining poses of multiple candidate pose points in the lane direction of the corrected position; performing projection imaging of a local map for each candidate pose point to obtain a projection image of the candidate pose point; performing feature matching on the projection image of each candidate pose point and the road feature map to find a candidate pose point with a minimum matching cost; and using the pose of the candidate pose point with the minimum matching cost as the second pose of the vehicle.

[0164] Step S1040: Using the EKF module, perform fusion calculation on the current sensor positioning data and the second pose obtained in step S1030 to obtain a third pose of the vehicle, where the third pose is the pose of the vehicle at the current time. Here, the sensor positioning data includes GPS information, IMU information, INS vehicle attitude information, and / or chassis information. Through the fusion of multiple types of positioning information, the error range of the vehicle positioning can be controlled at the centimeter level, thereby further improving the accuracy of the vehicle positioning.

[0165] In some embodiments, the weight of corresponding information may be adjusted in real time based on the status of each sensor in the fusion calculation process. For example, GPS information carries flag bits used to describe the GPS status and accuracy (e.g., whether the GPS is working properly, has failed, has an insufficient signal, or has other conditions), and whether to integrate the GPS information and the integration weight of the GPS information may be determined based on the flag bits in the GPS information. For example, when the GPS is failed, the GPS sensor provides a flag bit indicating that the GPS information is unreliable or does not output the GPS information. In this case, the GPS information may not be included in the fusion in step S1040. In other words, a fusion calculation is performed on the current IMU information, INS vehicle attitude information, and chassis information and the second pose obtained in step S1030 to obtain a third pose of the vehicle. When the GPS has a good signal, in step S1040, a fusion calculation may be performed on the current GPS information, IMU information, INS vehicle attitude information and chassis information and the second pose obtained in step S1030 to obtain a third pose of the vehicle.

[0166] In some embodiments, the GPS information includes specific location information of the vehicle, which may be described using longitude, latitude, and altitude information, or three-dimensional coordinate values ​​in a reference coordinate system (e.g., the UTM coordinate system). The IMU information includes, but is not limited to, vehicle linear acceleration and vehicle angular acceleration. The INS vehicle attitude information includes vehicle attitude information obtained by analyzing the IMU information. The chassis information may include, but is not limited to, vehicle speed, vehicle acceleration, steering angle, steering angular velocity, and other information. The specific contents of the GPS information, IMU information, INS vehicle attitude information, and chassis information are not limited in the embodiments of this application.

[0167] In various cases where the GPS is adequate, faulty, has insufficient signal, or the like, according to the embodiments of this application, vehicle positioning with centimeter-level positioning error can be realized, and the computational resource consumption and time consumption can be reduced, thereby simultaneously improving the accuracy, reliability, and timeliness of vehicle positioning, and ensuring the driving safety of the vehicle in driving modes such as unmanned driving or intelligent driving.

[0168] The specific implementation methods of the vehicle positioning device, computing device, vehicle, etc. provided in the embodiments of this application will be described in detail below.

[0169] 11 is a schematic diagram of the structure of a vehicle positioning device 1100 according to an embodiment of this application. Please refer to FIG. 11. The vehicle positioning device 1100 provided in this embodiment of this application includes: an image capture unit 1110 configured to capture an image of the road ahead of the vehicle; a feature acquisition unit 1120 configured to acquire a road feature map based on the forward road image; an offset determination unit 1130 configured to determine a lateral offset of the vehicle based on the road feature map; a position determining unit 1140 configured to obtain a second position of the vehicle based on a lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; may include:

[0170] In some embodiments, the feature acquisition unit 1120 may be specifically configured to determine the lateral offset of the vehicle based on lane boundary features in the road feature map.

[0171] In some embodiments, the lane boundary line features in the road feature map may include two lane boundary line features within a region of interest (ROI) in the road feature map, the two lane boundary lines being located to the left and right of the first pose.

[0172] In some embodiments, the feature acquisition unit 1120 may be specifically configured to acquire a lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio, where the lateral pixel offset is determined based on lane boundary line features in a top view of the ROI in the road feature map, and the lane boundary line features in the top view of the ROI in the road feature map are acquired by using features of two lane boundary lines in the ROI in the road feature map.

[0173] In some embodiments, the lateral pixel offset is the distance between the ROI mapping point of the optical center of the first camera in the top view and the lane center point, which is the lane center point between the left and right lane boundary lines of the first pose.

[0174] In some embodiments, the vehicle positioning device 1100 may further include an attitude determination unit 1150 configured to obtain a second attitude of the vehicle based on the lateral offset of the vehicle, the first pose, the road feature map and the local map.

[0175] In some embodiments, a second position or a second pose, or both, of the vehicle is determined based on a plurality of candidate pose points in the local map, and a pose of each candidate pose point is determined based on the lateral offset of the vehicle, the first pose, and the local map.

[0176] In some embodiments, the multiple candidate pose points are evenly distributed in the same lane direction of the local map by using the corrected position as the center, and the corrected position is obtained based on the lateral offset and based on the lane center point position and road azimuth angle that correspond to the first pose and are in the local map.

[0177] In some embodiments, the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second pose is pose information of a candidate pose point selected by performing feature matching on the projection image and the road feature map, and the projection image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

[0178] In some embodiments, the vehicle positioning device 1100 may further include a fusion unit 1160 configured to obtain a third pose of the vehicle based on the second position, the first attitude of the vehicle, and the sensor positioning data, or to obtain a third pose of the vehicle based on the second position, the second attitude of the vehicle, and the sensor positioning data, where the sensor positioning data may include one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

[0179] In some embodiments, the local map may be from a vector map.

[0180] The vehicle positioning device in the embodiments of this application may be realized by using software, hardware, or a combination thereof. For example, the vehicle positioning device 1100 may be realized as software in a computing device (the software has the above-mentioned functional modules), or may be directly realized as a computing device having the above-mentioned functional modules.

[0181] 12 is a schematic diagram of the structure of a computing device 1200 according to an embodiment of this application. The computing device 1200 includes one or more processors 1210 and one or more memories 1220.

[0182] Processor 1210 may be connected to memory 1220. Memory 1220 may be configured to store program codes and data. Thus, memory 1220 may be a storage unit within processor 1210, an external storage unit separate from processor 1210, or a component that includes a storage unit within processor 1210 and an external storage unit separate from processor 1210.

[0183] Optionally, computing device 1200 may further include a communication interface 1230. It should be understood that communication interface 1230 in computing device 1200 shown in FIG. 12 may be used for communication with other devices.

[0184] Optionally, computing device 1200 may further include a bus 1240. Memory 1220 and communication interface 1230 may be connected to processor 1210 through bus 1240. Bus 1240 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. Bus 1240 may be categorized into an address bus, a data bus, a control bus, and the like. For ease of representation, the bus is shown in FIG. 12 by using only one line. However, this does not indicate that only one bus or only one type of bus is present.

[0185] It should be understood that in this embodiment of the application, the processor 1210 may be a central processing unit (CPU). The processor may alternatively be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. Alternatively, the processor 1210 may be one or more integrated circuits, configured to execute associated programs to implement the technical solutions provided in the embodiments of the application.

[0186] Memory 1220 may include read-only memory and random access memory and may provide instructions and data to processor 1210. Portions of processor 1210 may also include non-volatile random access memory. For example, processor 1210 may further store device type information.

[0187] When computing device 1200 operates, processor 1210 executes computer-executable instructions in memory 1220 to perform the operational steps of the vehicle positioning method described above.

[0188] It should be understood that the computing device 1200 according to this embodiment of the present application may correspond to a corresponding execution entity of the method according to the embodiment of the present application, and the above and other operations and / or functions of the modules in the computing device 1200 are separately intended to realize corresponding steps of the method in the embodiment. For the sake of brevity, the details will not be described again here.

[0189] In practical applications, the computing device 1200 may be implemented as a functional unit within a chip, an independent chip, a functional unit of an in-vehicle terminal device, or an independent in-vehicle terminal device. In some embodiments, the computing device 1200 may be a functional unit / module in an in-vehicle infotainment system, a cockpit domain controller (CDC), or a mobile data center / multi-domain controller (MDC). The form and deployment manner of the computing device 1200 are not limited to the embodiments of this application.

[0190] An embodiment of the present application further provides a vehicle. The vehicle includes a first camera (which may also be referred to as a front-view camera) configured to capture a road image ahead of the vehicle. The vehicle may further include a vehicle positioning device 1100, a computing device 1200, a computer-readable storage medium described below, or a computer program product described below.

[0191] 13 is an exemplary diagram of a vehicle according to an embodiment of the present application. Please refer to FIG. 13. A front-view camera 1310 (i.e., the first camera described above) is installed on the vehicle 1300. The front-view camera may capture a road image ahead of the vehicle in real time and transmit the road image ahead to the vehicle's computing device 1200 (not shown), so that the computing device 1200 can reliably position the vehicle in real time by using the road image ahead.

[0192] For example, a GPS sensor 1320, an IMU 1330, a vehicle speed sensor 1340, an acceleration sensor 1350, etc. may be further installed in the vehicle, and these sensors may be separately connected to the computing device. It should be noted that Fig. 10 is merely an example. In specific applications, the number, type, deployment location, installation method, etc. of the GPS sensor 1320, the IMU 1330, the vehicle speed sensor 1340, and the acceleration sensor 1350 are not limited in the embodiments of this application.

[0193] For example, the front-view camera 1310, the GPS sensor 1320, the IMU 1330, the vehicle speed sensor 1340, and the acceleration sensor 1350 may be separately connected to a computing device wirelessly or by wires. For example, the front-view camera, the GPS, the IMU, the vehicle speed sensor, and the acceleration sensor may be connected to the computing device via Ethernet, Bluetooth, a wireless fidelity (Wi-Fi) network, a cellular network, a Controller Area Network (CAN) bus, a local interconnect network (LIN) bus, or by various other communication methods.

[0194] An embodiment of this application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the program is executed by a processor, the processor is enabled to perform the above-described vehicle positioning method. The computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium may be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0195] An embodiment of the present application further provides a computer program product including a computer program, which, when executed by a processor, enables the processor to perform the above vehicle positioning method. Here, the computer program product may be written in one or more program design languages. The program design languages ​​may include, but are not limited to, object-oriented program design languages, such as Java or C++, and conventional procedural program design languages, such as the "C" language.

[0196] It should be noted that the above are merely exemplary embodiments and technical principles used in this application. Those skilled in the art may understand that this application is not limited to the specific embodiments described herein, and those skilled in the art may make various obvious modifications, rearrangements, and substitutions without departing from the scope of protection of this application. Therefore, although this application is described in detail with reference to the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, and all fall within the scope of protection of this application.

Claims

1. 1. A vehicle positioning method, comprising: acquiring an image of the road ahead of the vehicle; obtaining a road feature map based on the forward road image; determining a lateral offset of the vehicle based on the road feature map; obtaining a second pose of the vehicle based on the lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; Including, the first pose includes a first position and a first posture, the second pose includes a second position and a second posture, and the second pose is a pose subsequent to the first pose; the second position or the second pose, or both, of the vehicle are determined based on a plurality of candidate pose points in the local map, a pose of each candidate pose point being determined based on the lateral offset of the vehicle, the first pose, and the local map; the plurality of candidate pose points are evenly distributed in the same lane direction of the local map by using a corrected position as a center, and the corrected position is obtained by correcting the first position based on the lateral offset of the vehicle and based on a lane center point position and a road azimuth angle that correspond to the first pose and are in the local map.

2. The method of claim 1 , wherein determining a lateral offset of the vehicle based on the road feature map comprises determining the lateral offset of the vehicle based on lane boundary features in the road feature map.

3. 3. The method of claim 2, wherein the lane boundary line features in the road feature map include features of two lane boundary lines within a region of interest (ROI) in the road feature map, the two lane boundary lines being located to the left and right of the first pose.

4. determining the lateral offset of the vehicle based on lane boundary line features in the road feature map includes: obtaining the lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio, wherein the lateral pixel offset is a distance between an ROI mapping point of an optical center of a first camera and a lane center point in a top view of the ROI in the road feature map, the lane center point being a lane center point between the lane boundary lines on the left and right sides of the first pose, the first camera being a camera that captures the forward road image, and the preset pixel ratio is a number of pixels per unit of length; and determining the lateral pixel offset based on lane boundary line features in the top view of the ROI in the road feature map; The method of claim 3 , wherein the lane boundary line features in the top view of the ROI in the road feature map are obtained by using the features of the two lane boundary lines within the ROI in the road feature map.

5. the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second posture is posture information of the candidate pose point selected by performing feature matching on the projection image and the road feature map; 5. The method of claim 1, wherein the projection image is an image in a 2D pixel coordinate system projected from the road ahead image in a 3D camera coordinate system, and is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

6. 6. The method of claim 1, further comprising: acquiring a third pose of the vehicle based on the second position, a first attitude of the vehicle, and sensor positioning data; or acquiring a third pose of the vehicle based on the second position, the second attitude of the vehicle, and sensor positioning data, wherein the third pose comprises a third position and a third attitude, and the accuracy of the third position is higher than the accuracy of the second position.

7. The method of claim 6 , wherein the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

8. The method of claim 1 , wherein the local map is from a vector map.

9. A vehicle positioning device, an image capture unit configured to capture an image of the road ahead of the vehicle; a feature acquisition unit configured to acquire a road feature map based on the forward road image; an offset determination unit configured to determine a lateral offset of the vehicle based on the road feature map; a position determination unit configured to obtain a second pose of the vehicle based on the lateral offset of the vehicle, a first pose of the vehicle, the road feature map, and a local map corresponding to the first pose; Including, the first pose includes a first position and a first posture, the second pose includes a second position and a second posture, and the second pose is a pose subsequent to the first pose; the second position or the second pose, or both, of the vehicle are determined based on a plurality of candidate pose points in the local map, a pose of each candidate pose point being determined based on the lateral offset of the vehicle, the first pose, and the local map; the plurality of candidate pose points are evenly distributed in the same lane direction of the local map by using a corrected position as a center, and the corrected position is obtained by correcting the first position based on the lateral offset of the vehicle and based on a lane center point position and a road azimuth angle that correspond to the first pose and are in the local map.

10. The apparatus of claim 9 , wherein the feature acquisition unit is specifically configured to determine the lateral offset of the vehicle based on lane boundary features in the road feature map.

11. 11. The apparatus of claim 10, wherein the lane boundary line features in the road feature map include features of two lane boundary lines within a region of interest (ROI) in the road feature map, the two lane boundary lines being located to the left and right of the first pose.

12. 12. The device of claim 11, wherein the feature acquisition unit is specifically configured to acquire the lateral offset of the vehicle based on a lateral pixel offset and a preset pixel ratio, wherein the lateral pixel offset is a distance between an ROI mapping point of an optical center of a first camera and a lane center point in a top view of the ROI in the road feature map, the lane center point being a lane center point between the lane boundary lines on the left and right sides of the first pose, the first camera is a camera that captures the forward road image, the preset pixel ratio is a number of pixels per unit of length, the lateral pixel offset is determined based on lane boundary line features in the top view of the ROI in the road feature map, and the lane boundary line features in the top view of the ROI in the road feature map are acquired by using the features of the two lane boundary lines within the ROI in the road feature map.

13. the second position is position information of one candidate pose point selected by performing feature matching on the projection image and the road feature map, and / or the second posture is posture information of the candidate pose point selected by performing feature matching on the projection image and the road feature map; 13. The apparatus of claim 9, wherein the projection image is an image in a 2D pixel coordinate system projected from the road ahead image in a 3D camera coordinate system, and is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

14. 14. The apparatus of claim 9, further comprising a fusion unit configured to: obtain a third pose of the vehicle based on the second position, a first attitude of the vehicle, and sensor positioning data; or obtain a third pose of the vehicle based on the second position, the second attitude of the vehicle, and sensor positioning data, wherein the third pose comprises a third position and a third attitude, and wherein an accuracy of the third position is higher than an accuracy of the second position.

15. The apparatus of claim 14 , wherein the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

16. 16. The apparatus of claim 9, wherein the local map is from a vector map.

17. A computing device including a processor and a memory, A computing device, wherein the memory stores program instructions that, when executed by the processor, enable the processor to perform a method according to any one of claims 1 to 8.

18. 1. A computer-readable storage medium, comprising:

9. A computer-readable storage medium storing program instructions that, when executed by a computer, enable the computer to perform the method of any one of claims 1 to 8.

19. A vehicle, a first camera configured to capture a road ahead image; A vehicle positioning device according to any one of claims 9 to 16 or a computing device according to claim 17; Vehicles including:

20. 20. The vehicle of claim 19, wherein the vehicle further comprises one or more of the following: a GPS sensor, an IMU, a vehicle speed sensor, and an acceleration sensor.

Images