Vehicle and positioning method, device and equipment thereof, computer readable storage medium

Abstract

A vehicle positioning method, device (1100), computing equipment (1200), computer readable storage medium and vehicle (1300). The positioning method comprises: acquiring a front road image of the vehicle (S210), obtaining a road feature map according to the front road image (S220), determining a lateral deviation of the vehicle according to the road feature map (S230), and obtaining a second position of the vehicle according to the lateral deviation of the vehicle, a first pose of the vehicle, the road feature map and a local map corresponding to the first pose (S240). Thereby, the vehicle positioning accuracy can be improved, the computing resource consumption can be reduced and the time can be saved, the reliability and timeliness of vehicle positioning can be improved synchronously, and the driving safety can be enhanced.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and more particularly to a vehicle and its positioning method, device, equipment, and computer-readable storage medium. Background Technology

[0002] Vehicles heavily rely on positioning technology; positioning is a prerequisite for other vehicle functions, and positioning accuracy directly affects driving safety. Typically, a combination of Global Positioning System (GPS) and Inertial Measurement Unit (IMU) is used to achieve high-precision vehicle positioning. However, in cases of poor GPS signal or GPS failure, the combined GPS and IMU positioning will drift, leading to reduced positioning accuracy.

[0003] Currently, in environments where GPS fails or the GPS signal is poor, laser and vision methods are mainly used to constrain the IMU to correct the positioning results. When using laser and vision methods to constrain the IMU, it is necessary to specifically collect feature maps of the positioning environment and store them as visual features, such as ORB (Oriented Fast and Rotated BRIEF) features and FAST (Features From Accelerated Segment Test) features. Furthermore, point matching methods such as Iterative Closest Point (ICP) and Kd-Tree are required to perform feature matching over a large area or even the entire map. This results in a large amount of data and high computational complexity, consuming significant computing resources, being time-consuming, costly in hardware, and inefficient. It also fails to provide real-time and reliable vehicle positioning, posing safety hazards to autonomous or intelligent driving systems. Summary of the Invention

[0004] To address the aforementioned technical problems, embodiments of this application provide a vehicle and its positioning method, apparatus, device, and computer-readable storage medium, which can improve vehicle positioning accuracy while reducing computational resource consumption and time consumption.

[0005] The first aspect of this application provides a vehicle positioning method, including:

[0006] Acquire an image of the road ahead of the vehicle;

[0007] Obtain a road feature map based on the image of the road ahead;

[0008] Determine the lateral deviation of the vehicle based on the road feature map;

[0009] The vehicle's second position is obtained based on the vehicle's lateral deviation, the vehicle's first pose, the road feature map, and the local map corresponding to the first pose.

[0010] Since lateral deviation can indicate the lateral distance of a vehicle on the road, the range of image matching can be narrowed by combining lateral deviation and first pose, thereby improving vehicle positioning accuracy while effectively reducing computational load and computational complexity.

[0011] As one possible implementation of the first aspect, determining the lateral deviation of a vehicle based on a road feature map includes: determining the lateral deviation of the vehicle based on lane line features in the road feature map. Lane line features are relatively reliable and complete; therefore, using lane line features as the basis for lateral deviation can yield a more accurate and reliable lateral deviation.

[0012] As one possible implementation of the first aspect, the lane line features in the road feature map include the features of two lane lines in the region of interest (ROI) of the road feature map, with the two lane lines located on the left and right sides of the first pose. The ROI data volume of the road feature map is relatively small, and the lane line features within it are complete and reliable, thus further improving the accuracy and reliability of vehicle lateral deviation.

[0013] As one possible implementation of the first aspect, determining the vehicle's lateral deviation based on lane line features in the road feature map includes: obtaining the vehicle's lateral deviation based on pixel lateral deviation and a pre-set pixel ratio; and determining the pixel lateral deviation based on lane line features in the top view of the road feature map ROI, where the lane line features are obtained from the features of two lane lines in the road feature map ROI. The method of obtaining the lateral deviation based on pixel lateral deviation and pixel ratio is relatively easy to implement, thus further reducing computational complexity.

[0014] As one possible implementation of the first aspect, the pixel lateral deviation is the distance between the ROI mapping point of the optical center of the first camera and the lane center point in the top view. The lane center point is the lane center point between the lane lines on the left and right sides of the first pose. The first camera is a camera that acquires images of the road ahead. The position of the optical center of the first camera is easy to obtain and its coordinate transformation computation is relatively small, thereby further reducing the amount of computation and lowering the computational complexity.

[0015] As one possible implementation of the first aspect, the vehicle localization method further includes obtaining the vehicle's second pose based on the vehicle's lateral deviation, first pose, road feature map, and local map. This allows for the real-time acquisition of the vehicle's accurate pose.

[0016] As one possible implementation of the first aspect, the second position and / or the vehicle's second pose is determined based on multiple candidate pose points in the local map. The pose of each candidate pose point is determined based on the vehicle's lateral deviation, the first pose, and the local map. Therefore, the vehicle can be accurately located using candidate pose points selected through the lateral deviation, with relatively low computational load and computational complexity, effectively reducing computational resource consumption and saving time.

[0017] As one possible implementation of the first aspect, multiple candidate pose points are evenly distributed along the same lane direction on the local map, centered on the corrected position. The corrected position is obtained based on the vehicle's lateral deviation and the position of the lane center point corresponding to the first pose in the local map, along with the road orientation angle. Therefore, the vehicle can be accurately located using multiple candidate pose points along the same lane direction, with a smaller search range. This improves vehicle positioning accuracy while further reducing computational load and computational complexity.

[0018] As one possible implementation of the first aspect, the second position is the position information of a candidate pose point selected by feature matching between the projected image and the road feature map; and / or, the second pose is the pose information of a candidate pose point selected by feature matching between the projected image and the road feature map; wherein the projected image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map. Since the degree of matching between the projected image and the road feature map indicates the closeness of the corresponding candidate pose point to the vehicle's true pose, a more accurate and reliable vehicle localization result can be obtained.

[0019] As one possible implementation of the first aspect, the vehicle localization method further includes: obtaining a third pose of the vehicle based on a second position, a first pose of the vehicle, and sensor localization data; or, obtaining a third pose of the vehicle based on a second position, a second pose of the vehicle, and sensor localization data. Thus, a more accurate and reliable third pose can be obtained through the fusion of multiple position data, further improving the accuracy and reliability of vehicle localization.

[0020] As one possible implementation of the first aspect, the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

[0021] As one possible implementation of the first aspect, the local map is derived from a vector map. Vector maps offer complete and high-precision information, which can further improve the accuracy of vehicle positioning.

[0022] A second aspect of this application provides a vehicle positioning device, comprising:

[0023] Image acquisition unit, used to acquire images of the road ahead of the vehicle;

[0024] The feature acquisition unit is used to obtain a road feature map based on the image of the road ahead;

[0025] The deviation determination unit is used to determine the lateral deviation of the vehicle based on the road feature map;

[0026] The position determination unit is used to obtain the second position of the vehicle based on the vehicle's lateral deviation, the vehicle's first pose, the road feature map, and the local map corresponding to the first pose.

[0027] As one possible implementation of the second aspect, the feature acquisition unit is specifically used to determine the lateral deviation of the vehicle based on the lane line features in the road feature map.

[0028] As one possible implementation of the second aspect, the lane line features in the road feature map include the features of two lane lines in the region of interest (ROI) of the road feature map, with the two lane lines located on the left and right sides of the first pose.

[0029] As one possible implementation of the second aspect, the feature acquisition unit is specifically used to obtain the lateral deviation of the vehicle based on the pixel lateral deviation and a pre-set pixel ratio. The pixel lateral deviation is determined based on the lane line features in the top view of the road feature map ROI. The lane line features in the top view of the road feature map ROI are obtained through the features of two lane lines in the road feature map ROI.

[0030] As one possible implementation of the second aspect, the pixel lateral deviation is the distance between the ROI mapping point of the optical center of the first camera and the lane center point in the top view. The lane center point is the lane center point between the lane lines on the left and right sides of the first pose. The first camera is a camera that captures images of the road ahead.

[0031] As a possible implementation of the second aspect, the vehicle positioning device further includes: an attitude determination unit for obtaining the second attitude of the vehicle based on the vehicle's lateral deviation, first attitude, road feature map, and local map.

[0032] As one possible implementation of the second aspect, the second position and / or the second pose of the vehicle is determined based on multiple candidate pose points in the local map, and the pose of each candidate pose point is determined based on the vehicle's lateral deviation, the first pose, and the local map.

[0033] As one possible implementation of the second aspect, multiple candidate pose points are evenly distributed in the same lane direction of the local map with the correction position as the center. The correction position is obtained based on the lateral deviation of the vehicle and the position of the lane center point corresponding to the first pose in the local map and the road orientation angle.

[0034] As one possible implementation of the second aspect, the second position is the position information of a candidate pose point selected by feature matching of the projected image and the road feature map; and / or, the second pose is the pose information of a candidate pose point selected by feature matching of the projected image and the road feature map; wherein the projected image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

[0035] As a possible implementation of the second aspect, the vehicle positioning device further includes: a fusion unit, configured to obtain a third pose of the vehicle based on the second position, the first pose of the vehicle, and sensor positioning data; or, to obtain a third pose of the vehicle based on the second position, the second pose of the vehicle, and sensor positioning data.

[0036] As one possible implementation of the second aspect, the sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

[0037] As one possible implementation of the second aspect, the local map is derived from the vector map.

[0038] A third aspect of this application provides a computing device, including a processor and a memory, wherein the memory stores program instructions that, when executed by the processor, cause the processor to perform the vehicle positioning method of the first aspect.

[0039] The fourth aspect of this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a computer, cause the computer to perform the vehicle positioning method of the first aspect.

[0040] The fifth aspect of this application provides a computer program product, including a computer program that, when run by a processor, causes the processor to execute the vehicle positioning method of the first aspect.

[0041] The sixth aspect of this application provides a vehicle, including: a first camera for acquiring images of the road ahead, and a vehicle positioning device (as described in the second aspect) or a computing device (as described in the third aspect).

[0042] As a possible implementation of the sixth aspect, the vehicle may also include one or more of the following: GPS sensor, IMU, vehicle speed sensor, and acceleration sensor.

[0043] In this embodiment, by first determining the lateral deviation and then combining it to determine the vehicle's second position, the scope of complex processing such as image matching can be effectively narrowed, reducing the amount of computation and computational complexity. Therefore, this embodiment can improve vehicle positioning accuracy while reducing the consumption of computing resources and time costs. It can simultaneously improve the accuracy, reliability, and timeliness of vehicle positioning under various conditions such as GPS failure or poor GPS signal, thereby enhancing the safety of vehicle operation in various driving modes such as autonomous driving and intelligent driving.

[0044] These and other aspects of this application will become more apparent in the description of the following embodiments(s). Attached Figure Description

[0045] The following description, with reference to the accompanying drawings, further illustrates the various features of this application and the relationships between them. The drawings are exemplary; some features are not shown to scale, and some drawings may omit conventional features in the field of this application that are not essential to it, or additional features that are not essential to this application may be shown. The combination of features shown in the drawings is not intended to limit this application. Furthermore, throughout this specification, the same reference numerals refer to the same things. Specific descriptions of the drawings are as follows:

[0046] Figure 1 This is an example diagram illustrating an application scenario of an embodiment of this application.

[0047] Figure 2 This is a flowchart illustrating the vehicle positioning method according to an embodiment of this application.

[0048] Figure 3 This is a schematic diagram of a road feature map in an example of Embodiment 1 of this application.

[0049] Figure 4 This is a schematic diagram of the road feature map after separating lane line features in an example of Embodiment 1 of this application.

[0050] Figure 5 This is a schematic diagram of the ROI in the camera coordinate system in an example of Embodiment 1 of this application.

[0051] Figure 6 This is an example diagram of the Road Feature Map ROI in Embodiment 1 of this application.

[0052] Figure 7 This is a schematic diagram of a top view of the road feature map ROI in an example of Embodiment 1 of this application.

[0053] Figure 8 This is a schematic diagram of the correction position, first position, lateral deviation and their relationship in the reference coordinate system in an example of Embodiment 1 of this application.

[0054] Figure 9 This is a schematic diagram of candidate pose points in an example of Embodiment 1 of this application.

[0055] Figure 10 This is a schematic diagram illustrating the specific implementation process of the vehicle positioning method in this application embodiment.

[0056] Figure 11 This is a schematic diagram of the vehicle positioning device according to an embodiment of this application.

[0057] Figure 12 This is a schematic diagram of the structure of the computing device according to an embodiment of this application.

[0058] Figure 13 This is a schematic diagram of the vehicle structure in an example of Embodiment 1 of this application. Detailed Implementation

[0059] The terms "first," "second," etc., or similar terms such as Unit A and Unit B, used in the specification and claims are only used to distinguish similar objects and do not represent a specific ordering of the objects. It is understood that, where permissible, a specific order or sequence may be interchanged so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0060] In the following description, the labels of the steps, such as S210, S220, etc., do not indicate that the steps will necessarily be executed. The order of the steps can be interchanged or executed simultaneously, where permissible.

[0061] The important terms and related terminology used in the embodiments of this application are explained as follows:

[0062] World coordinate system: also known as measurement coordinate system or objective coordinate system, it is used as a reference to describe the three-dimensional position and three-dimensional pose of the camera and the object under test. It is the absolute coordinate system of the objective three-dimensional world. The three-dimensional position of the object is usually represented by the coordinate values ​​Pw(Xw,Yw,Zw) of the three-dimensional coordinate axes, and the three-dimensional pose of the object is represented by the rotation angle (raw,pitch,yaw) of the object relative to the three-dimensional coordinate axes. In the embodiments of this application, the world coordinate system can be used as the reference coordinate system for global pose.

[0063] Camera coordinate system: The origin of the coordinate system is the optical center of the camera. The Z-axis coincides with the optical axis and points in front of the camera, the X-axis points to the right of the camera, and the Y-axis points to the bottom of the camera. The coordinate values ​​are usually represented by Pc(Xc,Yc,Zc).

[0064] The camera's extrinsic parameters determine the relative positional relationship between the camera coordinate system and the world coordinate system. The parameters for transforming from the world coordinate system to the camera coordinate system can include the rotation matrix R and the translation vector T. Taking pinhole imaging as an example, the camera's extrinsic parameters, world coordinates, and camera coordinates satisfy the relationship (1): Pc = RPw + T (1); where Pw is the coordinate value (Xw, Yw, Zw) of the world coordinate system, Pc is the coordinate value (Xc, Yc, Zc) of the camera coordinate system, T = (Tx, Ty, Tz) is the translation vector, and R = R(α, β, γ) is the rotation matrix, which are the rotation angles γ, β, and α around the Z-axis of the camera coordinate system, respectively. In other words, the six parameters α, β, γ, Tx, Ty, and Tz constitute the camera's extrinsic parameters.

[0065] Camera intrinsic parameters determine the projection relationship from 3D space to a 2D image and are solely related to the camera. Taking a pinhole camera model as an example, without considering image distortion, the intrinsic parameters can include the camera's scale factors along the u and v axes of the pixel coordinate system, the principal point coordinates (x0, y0) relative to the image coordinate system, and the coordinate axis tilt parameter s. The u-axis scale factor is the ratio of the physical length dx of each pixel in the x-direction of the image coordinate system to the camera's focal length f, and the v-axis scale factor is the ratio of the physical length dy of the pixel in the y-direction of the image coordinate system to the camera's focal length. If image distortion is considered, the intrinsic parameters can include the camera's scale factors along the u and v axes of the pixel coordinate system, the principal point coordinates relative to the imaging plane coordinate system, the coordinate axis tilt parameter, and distortion parameters. The distortion parameters can include the camera's three radial distortion parameters and two tangential distortion parameters. The camera's intrinsic and extrinsic parameters can be obtained through calibration.

[0066] The pixel coordinate system, also known as the pixel coordinate system or the view-by-view pixel coordinate system, is an image coordinate system that uses pixels as its unit. The origin is the top left corner of the image plane, the u-axis is horizontal to the right, and the v-axis is vertical to the bottom. The u-axis and v-axis are parallel to the X-axis and Y-axis of the camera coordinate system, respectively. Its coordinate value is usually represented by p(u,v). The pixel coordinate system represents the position of a pixel in the image in units of pixels, while the image coordinate system represents the position of a pixel in the image in physical units (e.g., millimeters).

[0067] Perspective projection: Transforming a 3D camera coordinate system into a 2D pixel coordinate system is called perspective projection. Because the origins and scales of the coordinate systems are inconsistent, perspective projection involves scaling and translation transformations. Taking a pinhole camera model as an example, a point (X...) in the camera coordinate system... C ,Y C Z C The perspective projection relationship when transformed to the pixel coordinate system (u, v) satisfies the transformation equation shown in equation (2) below:

[0068]

[0069] Where (u, v) represent the coordinate values ​​in the pixel coordinate system, (Xc, Yc, Zc) represent the coordinate values ​​in the camera coordinate system, and K is the matrix representation of the camera intrinsic parameters.

[0070] Without considering image distortion, equation (2) can also be written as equation (3):

[0071]

[0072] Among them, f x f is the ratio of the camera focal length f to the physical length dx of each pixel in the x-direction of the image coordinate system, representing the length of the focal length in the x-axis direction described by pixels. y The ratio of the camera focal length f to the physical length dy of each pixel in the y-direction of the image coordinate system represents the length of the focal length in the y-axis direction described by pixels. (u0, v0) represents the pixel coordinates of the intersection point (i.e., the principal point) of the camera optical axis and the image plane. u0 is the ratio of the horizontal coordinate x0 to dx of the principal point in the image coordinate system, and v0 is the ratio of the vertical coordinate y0 to dy of the principal point in the image coordinate system.

[0073] A top-down view, also known as a bird's-eye view, can be transformed from the pixel coordinate system of a horizontal view to the pixel coordinate system of a top-down view through affine transformation.

[0074] Affine transformation: Affine transformation refers to the transformation that takes advantage of the collinearity of the perspective center, image point, and target point, and rotates the projection plane (i.e., the perspective plane) around the trace line (i.e., the perspective axis) by a certain angle according to the law of perspective rotation, thereby destroying the original projection ray beam, while still maintaining the geometric shape of the projection on the projection plane.

[0075] The general formula for affine transformation is as follows (4):

[0076]

[0077] Where (u, v) represent the original coordinates; written in augmented vector form, w = 1; in the affine transformation matrix M, a 11 ,a 12 ,a 21 ,a 22 Represents linear transformations (e.g., scaling, shearing, rotation); a 31 ,a 32 Used for translation, a 13 ,a 23 Used to generate perspective transformations.

[0078] Typically, the transformed coordinates (x, y) satisfy the following equation (5):

[0079]

[0080] Given the transformation matrix, equations (6) to (7) can be obtained from equation (5) and transformation formula (4). The transformed coordinates can be directly obtained from the original coordinates using equations (6) to (7).

[0081]

[0082]

[0083] Here, the original coordinates and transformed coordinates of three pairs of points can be used to calculate the values ​​of each component in the affine transformation matrix M using the formula above.

[0084] Global pose: also known as absolute pose, includes the position and attitude of an object in a reference coordinate system. Taking a vehicle as an example, the position of the object can be represented by three-dimensional coordinate values ​​in the reference coordinate system (e.g., the coordinate values ​​Pw(Xw,Yw,Zw) in the world coordinate system mentioned above); the attitude of the object can be represented by the vehicle's pitch angle, yaw angle (also called yaw angle), and roll angle (raw angle). The roll angle is the angle of rotation around the X-axis, the pitch angle is the angle of rotation around the Y-axis, and the yaw angle is the angle of rotation around the Z-axis, or the attitude of the object can be described by quaternions. In the embodiments of this application, the reference coordinate system of the global pose can be, but is not limited to, the world coordinate system (including coordinate values ​​and attitude angles), the geodetic coordinate system, the Universal Transverse Mercator (UTM) grid system (also known as the UTM coordinate system), etc.

[0085] The following is a brief analysis of the possible implementation methods.

[0086] As mentioned earlier, in situations such as GPS failure or poor GPS positioning, the combined positioning of GPS and IMU will drift, resulting in reduced positioning accuracy.

[0087] In one possible implementation, the positioning system includes a visual positioning device and an inertial navigation device. The inertial navigation device includes an inertial measurement unit (IMU). The IMU acquires the state information of the IMU (including the specific force acceleration and Euler angles of the IMU in the vehicle coordinate system). The visual positioning device determines the linear acceleration of the IMU based on this state information, and then determines the position information of the autonomous vehicle based on the linear acceleration of the IMU and the visual information. Although this implementation can solve the problem of low positioning accuracy of autonomous vehicles, it suffers from cumulative errors, and the large amount of visual information and high computational complexity prevent it from achieving centimeter-level positioning accuracy and consumes a large amount of computational resources.

[0088] In the second possible implementation, the visual positioning method includes: pre-acquiring images of landmarks in the environment to be located and establishing a corresponding landmark database; after acquiring the images to be identified of the landmarks in the environment to be located, performing image matching between the images to be identified and the images in the landmark database; and calculating the current position based on the known geometric features in the matched images. Although this implementation can solve the problem of low positioning accuracy when GPS fails or positioning is poor, because it mainly relies on images for positioning and requires a large amount of image matching processing, it also suffers from problems such as positioning accuracy not reaching the centimeter level and excessive consumption of computational resources.

[0089] In view of this, embodiments of this application provide a vehicle and its positioning method, apparatus, device, and computer-readable storage medium. First, the lateral deviation is determined based on the road feature map of the road ahead. Then, the second position of the vehicle is obtained based on the lateral deviation, the vehicle's first pose, and a local map. Since the lateral deviation indicates the vehicle's lateral distance on the road, the combination of the lateral deviation and the first pose can narrow the search range of the local map, that is, narrow the range of image matching. This can effectively reduce the amount of computation and lower the computational complexity, while also improving the vehicle positioning accuracy. Therefore, embodiments of this application can simultaneously improve the accuracy, reliability, and timeliness of vehicle positioning under various conditions such as GPS failure and poor GPS signal.

[0090] The embodiments of this application are applicable to various scenarios requiring precise positioning. In particular, the embodiments of this application are especially suitable for positioning transportation vehicles such as vehicles, ships, aircraft, and drones. For example, real-time precise positioning of vehicles during operation, positioning of unmanned guided vehicles in logistics scenarios, and real-time precise positioning of drones in outdoor environments. Furthermore, the embodiments of this application are applicable to various environments such as indoors, outdoors, roads, and the wilderness.

[0091] In this application's embodiments, "vehicle" can be any type of transportation. For example, "vehicle" here can be, but is not limited to, private cars, commercial vehicles, buses, passenger vehicles, high-speed trains, subways, driverless cars, drones, logistics vehicles, unmanned transport vehicles, etc. The power type of the "vehicle" can be fuel-powered, pure electric, hydrogen fuel cell-powered, hybrid, etc. Furthermore, "vehicle" here can be a manually driven vehicle, an autonomous vehicle, a driverless vehicle, or other types of vehicles. Those skilled in the art will understand that any transportation tool requiring real-time location tracking can be considered a "vehicle" in this application's embodiments.

[0092] Figure 1 A schematic diagram of an exemplary application scenario of Embodiment 1 of this application is shown. Figure 1In this scenario, vehicles A, B, C, and D on the road can all locate their positions in real time using the solution provided in this application. For example, in various scenarios such as poor GPS signal, low vehicle sensor accuracy, tunnels, underground parking garages, and GPS failure, this application can achieve real-time, accurate, and reliable vehicle positioning.

[0093] For example, see Figure 1 As shown, the vehicle's positioning can be determined by the center of the vehicle's rear axle ( Figure 1 The position is indicated by the solid black dot in the middle.

[0094] The specific implementation methods of the embodiments of this application will be described in detail below.

[0095] Figure 2 A flowchart illustrating the vehicle positioning method according to an embodiment of this application is shown. See also... Figure 2 As shown, the vehicle positioning method provided in this application embodiment may include:

[0096] Step S210: Obtain an image of the road ahead of the vehicle;

[0097] The image of the road ahead can be captured by a first camera mounted on the vehicle. This image may include lane markings and signs on the road ahead. These signs may include, but are not limited to, traffic lights, parking garage pillars, walls, toll booths, turnstiles, crash barriers, obstacles, parking spaces and their lines, no-stopping zones, pedestrian crossings, speed bumps, speed bumps, stop signs, yield lines, and all other traffic signs and road markings, road signs, utility poles, road edges, trees, shrubs, medians, and guardrails. Furthermore, the image may also include lane center lines, guide lines, and guide arrows.

[0098] The forward road image is used to determine the vehicle's second position and / or second pose by combining it with the vehicle's first pose. The second pose formed by the second position and the second pose is the next pose after the first pose. In other words, the forward road image is an image of the road ahead acquired during the period when the vehicle has reached the first pose but has not yet reached or has just reached the second pose. For example, the acquisition time of the forward road image can be the following: the moment the vehicle reaches the first pose, the moment the vehicle reaches the second pose, or the period when the vehicle has reached the first pose but has not yet reached the second pose.

[0099] The image of the road ahead can be, but is not limited to, a time-of-flight (TOF) image, a color (Red, Green, Blue, RGB) image, or other types of images. The first camera can be, but is not limited to, a TOF camera, an RGB camera, etc. The embodiments of this application do not impose any restrictions on the deployment method, installation location, specific type, number of deployments of the first camera, or the specific type of the image of the road ahead.

[0100] Step S220: Obtain a road feature map based on the image of the road ahead;

[0101] In some embodiments, a road feature map of the road ahead image can be obtained by performing one or more of the following operations: semantic feature extraction, post-fitting processing (e.g., fitting incomplete lane lines into long straight lines), and feature skeletonization (e.g., lane line feature skeletonization, utility pole feature skeletonization, guide line contouring, sign contouring, etc.). This application does not limit the specific implementation method for obtaining the road feature map.

[0102] In some embodiments, the road feature map can be a grayscale image with pixel values ​​ranging from [0, 255]. In specific applications, pixel values ​​can indicate different types of road features. For example, the pixel value of lane line features can be preset to 255, the pixel value of utility pole features can be set to 135, the pixel value of guide arrow features can be set to 95, and the pixel value of sign features can be set to 45. This allows for the differentiation of different types of road features in the road feature map through pixel values, facilitating efficient and accurate extraction of the desired features. For instance, to extract lane line features from the road feature map, only pixels with a pixel value of 255 need to be extracted. If a pixel in the road feature map has a pixel value of 135, it indicates that the pixel is a utility pole feature.

[0103] In some embodiments, the road feature map may include lane line features and landmark features of the road in the preceding road image. Figure 3 An example diagram of a road feature map is shown. Figure 3 In the example, the road feature map may include lane line feature 31, utility pole feature 32, traffic light feature 33, etc. Through Figure 3 As can be seen, the road feature map includes the features of the two lane lines located on the left and right sides of the first pose.

[0104] Step S230: Determine the lateral deviation of the vehicle based on the road feature map;

[0105] In some embodiments, the lateral deviation of the vehicle may represent the lateral distance of the vehicle (e.g., the rear axle center or the optical center of the first camera on the vehicle) from the lane centerline. The lateral deviation may be, but is not limited to, actual lateral deviation and pixel lateral deviation. Actual lateral deviation is the lateral deviation in a reference coordinate system, while pixel lateral deviation is the lateral deviation in a top view of the road feature map.

[0106] In some embodiments, the lateral deviation of a vehicle can be determined based on lane line features in a road feature map. Because lane line features are relatively complete and reliable, using them as the basis for lateral deviation yields lateral deviations with both good accuracy and reliability. In specific applications, other road features can also be used as the basis for determining lateral deviation, such as lane centerlines, guide lines, and guide arrows.

[0107] In some embodiments, the lane line features in the road feature map may include the features of two lane lines in the ROI of the road feature map, which are located on the left and right sides of the first pose. That is, the lateral deviation of the vehicle can be determined based on the features of the two lane lines in the ROI of the road feature map. Because the ROI of the road feature map has a smaller data volume and the features of the two lane lines on the left and right sides of the first pose are relatively reliable and complete, obtaining the lateral deviation through the features of the two lane lines in the ROI of the road feature map can improve the accuracy of the lateral deviation while further reducing the computational complexity, reducing the consumption of computing resources and time. In specific applications, the lateral deviation can also be obtained through other features in the road feature map or lane line features in other areas. This application embodiment does not limit the features used to determine the lateral deviation of the vehicle.

[0108] In some embodiments, the lateral deviation of the vehicle can be obtained based on the pixel lateral deviation and a preset pixel ratio. The pixel lateral deviation is determined based on the lane line features in the top view of the road feature map ROI. The lane line features in the top view of the road feature map ROI are obtained through the features of two lane lines in the road feature map ROI. Because the road feature map ROI data volume is smaller and the lane line features it contains are more reliable and complete, the top view of the road feature map ROI is equivalent to a bird's-eye view of the road in front of the vehicle. Furthermore, obtaining the lateral deviation through pixel lateral deviation and pixel ratio is relatively simple and easy to implement. Therefore, obtaining the vehicle's lateral deviation through pixel lateral deviation can improve the accuracy of the lateral deviation while further reducing computational complexity, computational resource consumption, and time consumption.

[0109] In some embodiments, the pixel lateral deviation can be the distance between the ROI mapping point of the first camera's optical center and the lane center point in the top view, where the lane center point is the lane center point between the lane lines on the left and right sides of the first pose. Since the first camera's optical center is the origin of both the camera coordinate system and the pixel coordinate system of the first camera, the position of the ROI mapping point of the first camera's optical center is easy to obtain. Therefore, obtaining the pixel lateral deviation through the ROI mapping point of the first camera's optical center can further reduce computational complexity.

[0110] In specific applications, the lateral deviation of pixels can also be obtained by the ROI mapping point position of other rigid position points on the vehicle (e.g., the center of the rear axle of the vehicle). This application does not limit the specific method of obtaining the lateral deviation of pixels.

[0111] In some embodiments, an exemplary specific implementation process for determining the lateral deviation may include the following steps 1) to 4):

[0112] 1) Determine the road feature map ROI based on the pre-set ROI size parameters. The road feature map ROI includes the features of the lane lines on the left and right sides of the first pose.

[0113] 2) Determine the affine transformation matrix of the first camera based on the endpoint positions of the ROI in the road feature map and the preset pixel ratio, and obtain the top view of the ROI in the road feature map through the affine transformation of the first camera based on the affine transformation matrix.

[0114] 3) Determine the position of the lane center point in the top view of the road feature map ROI and the position of the ROI mapping point of the optical center of the first camera, and use the distance between the lane center point position and the ROI mapping point position as the pixel lateral deviation.

[0115] 4) The product of the pixel lateral deviation and the pixel ratio is used as the actual lateral deviation, which is the final lateral deviation.

[0116] The above-described exemplary implementation method can obtain the lateral deviation of the vehicle by means of the lane line features through perspective projection and affine transformation of the first camera. It not only has low computational load and low computational complexity, but also improves the accuracy of lateral deviation. It can further reduce the consumption of computing resources and time while improving the vehicle positioning accuracy.

[0117] Step S240: Obtain the second position of the vehicle based on the lateral deviation of the vehicle, the first pose of the vehicle, the road feature map, and the local map corresponding to the first pose.

[0118] The first pose represents the vehicle's initial pose or the pose at the previous moment. The first pose can include a first position and a first attitude, where the first position is the vehicle's initial position or the position at the previous moment, and the first attitude is the vehicle's initial attitude or the attitude at the previous moment.

[0119] The second pose is the next pose after the first pose, and can also be called the visual positioning pose of the first pose. The second pose can include a second position and a second attitude. If the vehicle's position remains unchanged but its attitude changes, the second position is equal to the first position; if the vehicle's position changes, the second position can be obtained by performing steps S210 to S240 on the first pose. If the vehicle's attitude remains unchanged or changes in vehicle attitude are not a concern, the second attitude can be equal to the first attitude. If it is necessary to update the vehicle's attitude simultaneously and the vehicle's attitude has changed, the second attitude can also be obtained by performing steps S210 to S240 on the first pose.

[0120] The third pose is the pose obtained by performing steps S210 to S250 on the first pose (also known as the fused positioning pose). In other words, the third pose can be obtained by performing step S250 on the second pose.

[0121] Both the second and third poses can be used as the vehicle's positioning pose or the pose at the current moment. In practical applications, the second and / or third poses can be selected as the vehicle's final pose or the pose at the current moment, depending on the needs. Here, the first pose, the second pose, and the third pose (described below) can all be the global poses mentioned earlier.

[0122] In some embodiments, step S240 may further include: obtaining the vehicle's second pose based on the lateral deviation, the first position, the first attitude, the road feature map, and the local map. In specific applications, the second pose can be obtained synchronously with the second position, or the second pose and the second position can be obtained sequentially, but the order in which they are obtained is not limited.

[0123] In some embodiments, this application may further include: obtaining a local map of the first pose of the corresponding vehicle. In practical applications, the local map data can be obtained from a map in the cloud or extracted from a locally stored map. It can be obtained before the positioning method of this application begins, or it can be obtained in real time during the execution of this application (e.g., in or before step S240).

[0124] A local map is a portion of the map surrounding the corresponding first pose. In some embodiments, the local map can be a predetermined area centered on the corresponding position of the first pose on the map. This predetermined area can be a circular area, a rectangular area, a triangular area, or an area of ​​other custom shapes. Taking a circular area as an example, the local map can be a circular area on the map with the corresponding position of the first pose as the center and a predetermined length as the radius. This predetermined length can be flexibly set or determined according to the vehicle speed. For example, the predetermined length can be 30 meters to 50 meters by default.

[0125] Because vector maps contain relatively accurate and complete data, the local map in this embodiment may be derived from a vector map to improve vehicle positioning accuracy. In some embodiments, the vector map may be a high-definition vector map or a high-precision vector map. In specific applications, the vector map may come from the cloud or be built into the vehicle's storage device.

[0126] In some embodiments, the information recorded in the local map includes, but is not limited to, the location information of each point in each lane, the location information of the lane center point, the road orientation angle information, and landmark information. Road features in the local map (e.g., landmarks, lane lines, etc.) can be represented by vector lines, curves, or straight lines, and different types of road features can be distinguished by pixel values. The road orientation angle information of each lane in the local map can be associated with the location information; the road orientation angles corresponding to different locations within the same lane can be the same or different. Furthermore, the local map can also record information on various road signs such as guide arrows and guide lines. This application embodiment does not limit the type of local map, its visualization method, or the specific content recorded.

[0127] In some embodiments, the second position and / or second pose can be determined based on multiple candidate pose points in the local map, and the pose of each candidate pose point is determined based on the lateral deviation, the first pose, and the local map. Therefore, the vehicle can be accurately located using the candidate pose points selected by the lateral deviation, without the need for large-scale global image matching of the area surrounding the first pose. This reduces computational load and computational complexity, effectively reducing computational resource consumption and saving time while improving vehicle positioning accuracy.

[0128] In some embodiments, multiple candidate pose points are evenly distributed along the same lane direction in the local map, centered on the corrected position. The corrected position is obtained based on the lateral deviation and the position of the lane center point corresponding to the first pose in the local map, along with the road orientation angle. Therefore, the vehicle can be accurately located using multiple candidate pose points along the same lane direction, resulting in a smaller search range and further narrowing the image matching scope. This improves vehicle positioning accuracy while reducing computational resource consumption and saving time.

[0129] In some embodiments, the second position can be the position information of a candidate pose point selected by feature matching of the projected image and the road feature map; and / or, the second pose can be the pose information of a candidate pose point selected by feature matching of the projected image and the road feature map; wherein the projected image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map. Since the degree of matching between the projected image and the road feature map indicates the closeness of the corresponding candidate pose point to the vehicle's true pose, feature matching of the projected image and the road feature map can obtain a relatively accurate and reliable vehicle positioning result.

[0130] In some examples, a candidate pose point selected by feature matching between the projected image and the road feature map can be any candidate pose point among multiple candidate pose points whose matching cost between the projected image and the road feature map is less than a predetermined threshold, or the candidate pose point with the smallest matching cost between the projected image and the road feature map. Since the smallest matching cost between the projected image and the road feature map indicates the highest degree of matching, it means that the corresponding candidate pose point is closest to the vehicle's true pose. Therefore, the pose of the candidate pose point with a matching cost less than the predetermined threshold or the smallest matching cost is closer to the vehicle's true pose, thus obtaining a more accurate and reliable vehicle localization result.

[0131] In some embodiments, the vehicle positioning method provided in this application may further include: step S250, obtaining a third pose of the vehicle based on the second position, the first attitude, and sensor positioning data; or, obtaining the third pose of the vehicle based on the second position, the second attitude, and sensor positioning data. Here, the sensor positioning data may include one or more of the following: GPS information, IMU information, inertial navigation system (INS) vehicle attitude information, and chassis information. In this way, the second attitude can be fused with real-time positioning information from other sensors to obtain a more accurate and reliable third pose, which can compensate for the influence of external environmental conditions such as poor performance of the first camera and poor lighting on the accuracy of the second pose, further improving the accuracy and reliability of vehicle positioning.

[0132] In some embodiments, an Extended Kalman Filter (EKF) can be used to perform multi-sensor information fusion calculation to obtain the vehicle's third pose. For example, GPS information, IMU information, INS vehicle attitude information, chassis information, and the aforementioned second pose can be fused using their covariance as weights for information validity to obtain the vehicle's third pose. The specific algorithm for fusion calculation is not limited in the embodiments of this application.

[0133] In some embodiments, an exemplary implementation of step S230 may include the following steps a1 to a5:

[0134] Step a1: Separate lane line features;

[0135] In some embodiments, different landmarks in the road feature map can be classified by pixel values. Assuming the pixel value of the lane line feature is defined as m, extracting pixels with a pixel value of m from the road feature map will form a road feature map containing only lane line features. Figure 4 It shows how to separate Figure 3 The road feature map is obtained after the lane line features are defined.

[0136] This step is optional; the separation of lane line features can be flexibly chosen based on actual application requirements. Pre-separating road features can reduce the amount of data in subsequent processing, lower computational complexity, and filter out unnecessary feature data. If other features, such as lane centerlines, are lateral deviations, similar methods can be used to separate them.

[0137] Step a2: Obtain the Road Feature Map ROI;

[0138] The Region of Interest (ROI) can be selected as a fixed field of view area with high detection accuracy and precision. In practical applications, objects closer to the camera appear larger in the image and are easier to detect; objects farther away appear smaller and are harder to detect. Furthermore, objects at the edge of the field of view are likely to be incomplete in the image, increasing the probability of false detections. Regions that are close to the camera and not at the edge of the field of view contain more complete and accurate image data, resulting in higher detection accuracy. Therefore, the portion of the field of view closest to the camera and not at the edge is typically defined as the ROI. In specific applications, the ROI can be pre-set to the actual size range of a fixed field of view area.

[0139] In some embodiments, a region of interest (ROI) can be selected in the camera coordinate system of the first camera, and the size parameters of the ROI can be preset. These size parameters may include, but are not limited to, the positions of the ROI endpoints in the camera coordinate system, the length of the ROI, and the width of the ROI. Taking a rectangular ROI as an example, the size parameters of the ROI may include, but are not limited to: the positions of the four ROI endpoints (e.g., the coordinate values ​​of the four endpoints in the camera coordinate system of the first camera), the length of the ROI, and the width of the ROI.

[0140] In this step, the endpoints of the ROI are transformed to the pixel coordinate system of the first camera by perspective projection of the first camera to obtain the ROI endpoints in the road feature map. The rectangular area defined by these endpoints is the ROI of the road feature map.

[0141] Figure 5 A schematic diagram of the ROI in the camera coordinate system of the first camera is shown. Figure 5 In the example, in the camera coordinate system XCY with the actual position C of the first camera (i.e., the position of the optical center of the first camera in the reference coordinate system) as the origin, a rectangular area bounded by four points P1, P2, P3, and P4 is selected on the ground as the ROI. The size parameter of the ROI may include the coordinate values ​​P1(x, y, y) of the four points in the camera coordinate system. P1 y P1 , z P1 P2(x) P2 y P2 , z P2 P3(x) P3 y P3 , z P3 P4(x) P4 y P4 , z P4 ), ROI length m (i.e., the actual distance from P1 to P2, in meters) and ROI width n (i.e., the actual distance from P1 to P3, in meters).

[0142] In some embodiments, while obtaining the Road Feature Map ROI, the position of the ROI mapping point of the optical center of the first camera can be further determined. Specifically, the position of the ROI mapping point of the optical center of the first camera in the camera coordinate system can be determined based on the position of the ROI endpoint in the camera coordinate system. Based on the position of the ROI mapping point of the optical center of the first camera in the camera coordinate system, the position of the ROI mapping point of the optical center of the first camera in the pixel coordinate system can be obtained through perspective projection of the first camera.

[0143] See also Figure 5 For example, draw a perpendicular line from the actual position C of the first camera to the ROI. The intersection of this perpendicular line (i.e., the Y-axis of the camera coordinate system of the first camera) and the ROI region near the camera's side edge is the ROI mapping point S of the optical center of the first camera. Knowing the position of the ROI endpoint in the camera coordinate system, the position of the ROI mapping point S of the optical center of the first camera can be determined (S...). X S Y S Z ), that is, S Y =0, S X =0, S Z =z P3 =z P4 .

[0144] Given the coordinates of four points P1(x) in the camera coordinate system P1 y P1 , z P1 P2(x)P2 y P2 , z P2 P3(x) P3 y P3 , z P3 P4(x) P4 y P4 , z P4 ) and the location of the ROI mapping point S (S X S Y S Z By using perspective projection from equations (2) to (3), the positions P1′(u) of the ROI endpoints P1, P2, P3, and P4 in the pixel coordinate system of the first camera can be obtained. P1 v P1 ), P2′(u P2 v P2 ), P3′(u P3 v P3 ), P4′(u P4 v P4 The position S′(u) of the ROI mapping point S in the pixel coordinate system of the first camera. S v S Thus, the Road Feature Map (ROI) is obtained. Figure 6 An example diagram of the Road Feature Map ROI is shown. Figure 6 The dashed boxes in the diagram represent the boundaries of the ROI, and the solid lines represent lane lines.

[0145] Step a3: Determine the endpoint positions of two adjacent lane lines in the ROI of the road feature map;

[0146] In some embodiments, methods such as the Hough line detection method can be used to detect two adjacent lane lines L1 and L2 in the road feature map ROI, and determine the endpoint position of lane line L1. and End point of lane line L2 and Therefore, by combining the distribution of lane lines in the ROI of the road feature map, two adjacent lanes in the left and right sides of the ROI can be selected as the basis for determining the lateral deviation, so as to obtain a more reliable lateral deviation. Figure 6 This shows two lane lines adjacent to the vehicle on the left and right sides in the ROI of the road feature map. Figure 6 (The bold black line in the text) and its endpoints.

[0147] Step a4: Obtain the lane line features in the top view of the road feature map ROI and the ROI mapping point position of the optical center of the first camera through affine transformation.

[0148] In some implementations, the exemplary implementation process of this step may include the following sub-steps 1) to 2):

[0149] 1) Construct a top view corresponding to the first camera and determine the affine transformation matrix M between the road feature map ROI and its top view.

[0150] Specifically, a top view with a pixel ratio of k (unit: pixels / meter, i.e., the number of pixels per meter) and a width of B can be pre-constructed. w B is high h Based on the ROI length m and ROI width n in the camera coordinate system of the first camera, B can be obtained based on equation (8). w and B h .

[0151]

[0152] Specifically, given the ROI endpoint P1′(u P1 v P1 ), P2′(u P2 v P2 ), P3′(u P3 v P3 ), P4′(u P4 v P4 According to B w and B h The midpoint positions P1″(0,0) and P2″((B) in the pixel coordinate system of the top view can be obtained. w -1), 0), P3″((B w -1), (B h -1)), P4″(0,(B h -1)), and thus the affine transformation matrix M between the road feature map ROI and its top view can be obtained by using the formulas (6) to (7) of the affine transformation mentioned above.

[0153] 2) Obtain the lane lines L3 and L4 and the ROI mapping point S″ of the top view of the road feature map ROI through affine transformation.

[0154] First, lane lines L1 and L2 in the road feature map ROI are transformed into the pixel coordinate system of the top view using an affine transformation, resulting in the corresponding lane lines L3 and L4 in the top view of the road feature map ROI. Specifically, the endpoints of lane lines L1 and L2 are... Substituting into the affine transformation formulas (6) to (7), we can obtain the endpoint positions of the corresponding lane lines L3 and L4 in the top view.

[0155] Then, based on the endpoint positions of lane lines L3 and L4, we can obtain the expression for lane line L3 (9) and the expression for lane line L4 (10).

[0156]

[0157]

[0158] in, This represents the slope of lane line L3. This represents the y-intercept of lane line L3. This represents the slope of lane line L4. This represents the y-intercept of lane line L4.

[0159] Simultaneously, the location S′(u) of the ROI mapping point of the first camera's optical center in the road feature map ROI is mapped. S v S Substituting into the affine transformation formulas (6) to (7), we can obtain the ROI mapping point position S″(x1, y1) of the optical center of the first camera in the top view of the road feature map ROI. Figure 7 An example diagram showing a top view of a road feature map ROI is provided. Figure 7 The diagram also shows lane lines L3 and L4 and ROI mapping point S″ in the top view of the road feature map ROI. The dashed box represents the top view of the road feature map ROI, the solid lines inside the dashed box represent lane lines, and the dashed line between the two solid lines represents the lane cross section.

[0160] Step a5: Calculate the lateral deviation.

[0161] Specifically, the position Q(x2, y2) of the lane center point in the top view of the road feature map ROI can be obtained based on the lane line L3 expression (9) and the lane line L4 expression (10). The pixel lateral deviation is obtained by using the position Q(x2, y2) of the lane center point and the position S″(x1, y1) of the ROI mapping point. The actual lateral deviation is then obtained by using the pixel lateral deviation and the preset pixel ratio.

[0162] See Figure 7 For example, in the pixel coordinate system of the top view, a perpendicular line is drawn from S″(x1, y1) to lane line L3 or lane line L4. The intersection point Q1 of this perpendicular line with lane line L3 and the intersection point Q2 with lane line L4 are the two endpoints of the lane cross section. The midpoint of the lane cross section Q1Q2, that is, the midpoint between intersection point Q1 and intersection point Q2, is the position Q(x2, y2) of the lane center point.

[0163] Based on the expressions (9) for lane line L3 and (10) for lane line L4 in the top view of the ROI feature map, and the ROI mapping point S″(x1, y1), the coordinates Q1(l) of the two endpoints of the lane cross section can be determined. x1 , l y1 ) and Q2(l x2 , l y2 Then, the position Q(x2, y2) of the lane center point can be obtained according to equation (11).

[0164]

[0165] Pixel lateral offset is the lateral offset of the road in the pixel coordinate system of the top view of the road feature map ROI. (By...) Figure 7 As can be seen, the distance between the ROI mapping point S″(x1, y1) and the lane center point Q(x2, y2) in the pixel coordinate system of the top view of the road feature map ROI is the pixel lateral offset. In other words, the pixel lateral offset offset1 can be obtained by equation (12).

[0166]

[0167] In some embodiments, the actual lateral deviation is the lateral offset of the road in the reference coordinate system. Specifically, the actual lateral deviation offset2 can be obtained by equation (13).

[0168] offset2 = k * offset1 (13)

[0169] Here, offset2 represents the actual lateral deviation, and k represents the preset pixel ratio.

[0170] In some embodiments, an exemplary implementation of step S240 may include the following steps b1 to b3:

[0171] Step b1: Correct the first position based on the lateral deviation to obtain the corrected position of the corresponding first pose.

[0172] In some embodiments, the correction position H start (H x0 H y0 H z0 It can be obtained through equation (14):

[0173]

[0174] Among them, C start (C x C y C z ) represents the first location E in the local map. start (Ex E y E z The corresponding lane center point C start The position, offset2 represents the actual lateral deviation, yaw l This represents the first location E on the local map. start (E x E y E z The road facing angle is typically C. start (C x C y C z ) and yaw l Recorded in the local map, it can be accessed via the first location E. start (E x E y E z Read directly from the local map.

[0175] Figure 8 A schematic diagram showing the correction position, first position, and lateral deviation in the reference coordinate system is shown. Figure 8 In the diagram, the XOY axes represent the XY plane of the reference coordinate system, O is the origin of the reference coordinate system, and E... start Indicates the first position, C start This represents the first location E on the local map. start The lane center point, offset2 represents the actual lateral deviation, through Figure 8 It can be seen that, combined with the road orientation angle yaw l By decomposing the lateral deviation into a reference coordinate system, the lateral deviation components of each coordinate axis can be obtained. Adding the lateral deviation components of each coordinate axis to the corresponding coordinate values ​​of the lane center point yields the correction position H. start The coordinate values.

[0176] Step b2: Determine candidate pose points;

[0177] In one implementation, the position H is corrected. start (H x0 H y0 H z0 Centered on the lane direction of the local map, candidate pose points are selected, and the corrected position H is used as the reference point. start (H x0 H y0 H z0 The pose of each candidate pose point can be determined by the road orientation angle information in the local map and the map itself.

[0178] Specifically, the known correction position H start (H x0 Hy0 H z0 ), a pre-set search range and search step size, combined with road orientation data from the local map (yaw) i To correct position H start (H x0 H y0 H z0 H is the 0th candidate pose point. z0 The position can be determined by equations (15) to (18), which can be used to determine the number (2n+1) of candidate pose points and the position information (H) of each candidate pose point. xi H yi H zi ).

[0179]

[0180]

[0181] H zi =H z0 ,i=-n,-n+1,…,-1,0,1,…,n-1,n(17)

[0182]

[0183] Where scope represents the pre-defined search range, step represents the pre-defined search step size, and H i (H xi H yi H zi ) represents the position of the i-th candidate pose point, yaw i (i = -n, -n+1, ​​..., -1, 0, 1, ..., n1, n) represents the road orientation angle corresponding to the i-th candidate pose point, yaw i It can be obtained directly from the local map. Equation (18) shows that the number of candidate pose points is 2n+1, where n is an integer greater than or equal to 1. Equations (15) to (17) show that H... z-1 and H z1 With the 0th candidate pose point H z0 They are in the same position.

[0184] Figure 9 An example diagram of candidate pose points is shown. Figure 9 In the diagram, the hollow origin represents the candidate pose point H. i Solid dots indicate the correction position H. start L5 indicates the corrected position H start The lane direction is defined by L0, which represents the lane centerline. XOY is the XY plane of the reference coordinate system, and O is the origin of the reference coordinate system. Using equations (15) to (18) and... Figure 9 As can be seen, the candidate pose points are evenly distributed in the same lane direction on the local map, and the lane direction has been corrected. Therefore, the traversal range of feature matching can be reduced to a single line on the map grid, without having to traverse all points around the vehicle's position on the map. The computational cost of projection imaging is significantly reduced, and the range of point matching during feature matching is significantly reduced. Thus, the computational cost is significantly reduced, and the computational complexity is also significantly lowered.

[0185] In some implementations, when the vehicle's posture remains essentially unchanged, changes in the vehicle's posture are not a concern, or there is no need to synchronously determine the vehicle's posture, the posture information (raw) of each candidate pose point is... i pitch i ,yaw i The first pose can be taken. For example, the first pose is (raw). start pitch start ,yaw start When this is the case, the pose information of each candidate pose point is also the first pose (raw). start pitch start ,yaw start In other words, the pose of the i-th candidate pose point can be H. i (H xi H yi H zi raw start pitch start ,yaw start ).

[0186] In some implementations, when the vehicle's attitude needs to be determined synchronously, it can be determined using location information H. i (H xi H yi H zi The pose information of each candidate pose point can be determined in a similar manner to that of the first pose (raw). start pitch start ,yaw start The attitude information of each candidate pose point can be determined by combining the pre-set angle search step size. It should be noted that when the vehicle attitude needs to be determined simultaneously, the number of candidate pose points will not be limited to the above "2n+1".

[0187] When determining the vehicle's second pose, candidate pose points and their position and pose information can be selected in a similar manner. Similarly, this significantly reduces computational load and computational complexity.

[0188] Step b3, projection imaging;

[0189] In this step, for each candidate pose point, projection imaging can be performed using a local map to obtain a projected image of each candidate pose point.

[0190] In some implementations, the projection imaging process for a single candidate pose may include: adjusting the extrinsic parameters of the first camera based on the pose of the candidate pose point; projecting each pixel on the landmark vector line in the local map onto the pixel coordinate system of the first camera based on the extrinsic and intrinsic parameters of the first camera; synchronously establishing the linear correlation between these pixels in the pixel coordinate system according to the linear correlation between pixels in the local map; and rendering the corresponding pixels according to the preset pixel values ​​of each landmark, thereby obtaining the projected image of the candidate pose point. In this way, the projected image can be equivalent to a road feature map of the road ahead when the optical center of the first camera is located at the candidate pose point.

[0191] Here, signs include, but are not limited to, traffic lights, underground parking pillars, walls, toll booths, gate machines, crash barriers, obstacles, parking spaces and parking space lines, no-parking zones, pedestrian crossings, speed bumps, speed reduction signs, stop signs, yield lines, yield signs, and all other traffic signs and road markings, road signs, utility poles, roadside edges, trees, shrubs, medians, and guardrails.

[0192] In some implementations, combining the principles of equations (1) to (3) above, the process of projecting the pixels of landmarks in the local map onto the pixel coordinate system of the first camera can be achieved by the following equation (19):

[0193]

[0194] Where (u, v) represents the coordinates of a pixel in the pixel coordinate system of the first camera, (X... W Y W Z W This represents the coordinates of the pixels of the landmarks in the local map in the reference coordinate system. R is the rotation matrix of the first camera, and T is the translation matrix of the first camera. R can be determined based on the pose information of the candidate pose points (raw). i pitch i ,yaw i T is determined by the rotation matrix in the initial extrinsic parameters of the first camera and can be based on the position information (H) of the candidate pose points. xi H yi H zi The translation matrix in the initial extrinsic parameters of the first camera is determined.

[0195] In some implementations, to facilitate feature matching, the preset pixel values ​​of each marker in the projected image can be consistent with the preset pixel values ​​of the corresponding markers in the road feature image.

[0196] Step b4: Obtain the second pose of the vehicle through feature matching.

[0197] In this step, the projected image of each candidate pose point is matched with the road feature map to obtain the matching cost value corresponding to each candidate pose point. The pose of the candidate pose point with the smallest matching cost value among all candidate pose points is selected as the second pose of the vehicle. That is, the position information of the candidate pose point with the smallest matching cost value is used as the second position of the vehicle, and the pose information of the candidate pose point with the smallest matching cost value is used as the second pose of the vehicle.

[0198] In some implementations, feature matching between the projected image of each candidate pose point and the road feature map may include: for each pixel p1 in the road feature map, traversing the entire projected image to find the pixel p2 in the projected image whose pixel value is closest to that of pixel p1, calculating the Euclidean distance D1 between pixel p2 in the projected image and pixel p1 in the road feature map, thus obtaining the Euclidean distance D1 between each pixel in the road feature map and its corresponding pixel in the projected image, and the sum of the Euclidean distances D1 corresponding to all pixels in the road feature map is the matching cost between the projected image and the road feature map.

[0199] Figure 10 An exemplary implementation flow of the vehicle positioning method according to an embodiment of this application is shown. See also Figure 10 As shown, an exemplary implementation of the vehicle positioning method in this application may include the following steps:

[0200] Step S1010: The first camera captures an image of the road ahead of the vehicle at the current moment and provides the image of the road ahead to the computing device;

[0201] Step S1020: The computing device obtains the local map corresponding to the first pose, where the first pose is the global pose of the vehicle at the previous moment.

[0202] In step S1030, the computing device uses the image of the road ahead and the local map corresponding to the first pose to determine the second pose of the vehicle.

[0203] Specifically, firstly, a road feature map of the road ahead is obtained through semantic feature extraction, skeletonized semantic features, and post-fitting processing. Secondly, the Region of Interest (ROI) of the road feature map is extracted, and the lateral deviation is determined based on the lane line features and the first pose in the ROI. Thirdly, the corrected position of the first position is obtained based on the lateral deviation, and a longitudinal search is performed along the lane direction of the corrected position to obtain the vehicle's second pose. Here, the longitudinal search along the lane direction of the corrected position is the process of determining the vehicle's second position described above. This process may include: selecting and determining the poses of multiple candidate pose points along the lane direction of the corrected position; performing local map projection imaging on each candidate pose point to obtain its projected image; performing feature matching between the projected images of each candidate pose point and the road feature map to find the candidate pose point with the minimum matching cost; and using the pose of the candidate pose point with the minimum matching cost as the vehicle's second pose.

[0204] In step S1040, the EKF module is used to fuse the current sensor positioning data and the second pose obtained in step S1030 to obtain the vehicle's third pose, which is the vehicle's pose at the current moment. Here, the sensor positioning data includes GPS information, IMU information, INS vehicle attitude information, and / or chassis information. By fusing multiple positioning information, the vehicle positioning error range can be controlled within the centimeter level, further improving the vehicle positioning accuracy.

[0205] In some embodiments, the weights of relevant information can be adjusted in real time based on the status of each sensor during the fusion calculation. For example, GPS information carries flags describing GPS status and accuracy (e.g., GPS working normally, GPS malfunctioning, or poor GPS signal). The inclusion of GPS information and its weight can be determined based on these flags. For instance, when GPS malfunctions, the GPS sensor may provide a flag indicating unreliable GPS information or not output GPS information at all. In this case, GPS information may not be included in the fusion calculation of step S1040; instead, the current IMU information, INS vehicle attitude information, chassis information, and the second pose obtained in step S1030 can be fused to obtain the vehicle's third pose. When the GPS signal is good, step S1040 can fuse the current GPS information, IMU information, INS vehicle attitude information, chassis information, and the second pose obtained in step S1030 to obtain the vehicle's third pose.

[0206] In some embodiments, GPS information includes the vehicle's specific location information, which can be described using latitude, longitude, and altitude information or three-dimensional coordinate values ​​in a reference coordinate system (e.g., the UTM coordinate system). IMU information may include, but is not limited to, vehicle linear acceleration and vehicle angular acceleration. INS vehicle attitude information includes the vehicle's attitude information obtained by resolving the IMU information. Chassis information may include, but is not limited to, vehicle speed, vehicle acceleration, steering wheel angle, and steering wheel angular velocity. The specific content of GPS information, IMU information, INS vehicle attitude information, and chassis information is not limited in the embodiments of this application.

[0207] Under various conditions, such as normal GPS, GPS failure, or poor GPS signal, the embodiments of this application can achieve vehicle positioning with centimeter-level positioning error. At the same time, it can reduce computing resource consumption and save time, and simultaneously improve the accuracy, reliability, and timeliness of vehicle positioning, thus providing a guarantee for the driving safety of vehicles in driving modes such as unmanned driving or intelligent driving.

[0208] The specific implementation methods of the vehicle positioning device, computing device, vehicle, etc. provided in the embodiments of this application will be described in detail below.

[0209] Figure 11 A schematic diagram of the vehicle positioning device 1100 provided in an embodiment of this application is shown. See also... Figure 11 As shown, the vehicle positioning device 1100 provided in this application embodiment may include:

[0210] Image acquisition unit 1110 is used to acquire images of the road ahead of the vehicle;

[0211] The feature acquisition unit 1120 is used to obtain a road feature map based on the image of the road ahead;

[0212] Deviation determination unit 1130 is used to determine the lateral deviation of the vehicle based on the road feature map;

[0213] The position determination unit 1140 is used to obtain the second position of the vehicle based on the lateral deviation of the vehicle, the first pose of the vehicle, the road feature map, and the local map corresponding to the first pose.

[0214] In some embodiments, the feature acquisition unit 1120 may be specifically used to determine the lateral deviation of a vehicle based on lane line features in a road feature map.

[0215] In some embodiments, the lane line features in the road feature map may include: features of two lane lines in the region of interest (ROI) of the road feature map, the two lane lines being located on the left and right sides of the first pose.

[0216] In some embodiments, the feature acquisition unit 1120 can be specifically used to obtain the lateral deviation of the vehicle based on the pixel lateral deviation and a preset pixel ratio. The pixel lateral deviation is determined based on the lane line features in the top view of the road feature map ROI. The lane line features in the top view of the road feature map ROI are obtained through the features of two lane lines in the road feature map ROI.

[0217] In some embodiments, the pixel lateral deviation is the distance between the ROI mapping point of the optical center of the first camera and the lane center point in the top view, and the lane center point is the lane center point between the lane lines on the left and right sides of the first pose.

[0218] In some embodiments, the vehicle positioning device 1100 may further include: an attitude determination unit 1150, used to obtain the second attitude of the vehicle based on the lateral deviation of the vehicle, the first attitude, the road feature map and the local map.

[0219] In some embodiments, the second position and / or the second pose of the vehicle is determined based on multiple candidate pose points in a local map, and the pose of each candidate pose point is determined based on the vehicle's lateral deviation, the first pose, and the local map.

[0220] In some embodiments, multiple candidate pose points are evenly distributed in the same lane direction of the local map with the correction position as the center. The correction position is obtained based on the lateral deviation and the position of the lane center point corresponding to the first pose in the local map and the road orientation angle.

[0221] In some embodiments, the second position is the position information of a candidate pose point selected by feature matching of the projected image and the road feature map; and / or, the second pose is the pose information of a candidate pose point selected by feature matching of the projected image and the road feature map; wherein the projected image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

[0222] In some embodiments, the vehicle positioning device 1100 may further include a fusion unit 1160, configured to obtain a third pose of the vehicle based on the second position, the first pose of the vehicle, and sensor positioning data; or, to obtain a third pose of the vehicle based on the second position, the second pose of the vehicle, and sensor positioning data. Here, the sensor positioning data may include one or more of the following: GPS information, IMU information, INS vehicle pose information, and chassis information.

[0223] In some embodiments, the local map may be derived from a vector map.

[0224] The vehicle positioning device of this application embodiment can be implemented by software, hardware, or a combination of both. For example, the vehicle positioning device 1100 can be implemented as software in a computing device (the software has the above-mentioned functional modules), or directly implemented as a computing device having the above-mentioned functional modules.

[0225] Figure 12 This is a schematic structural diagram of a computing device 1200 provided in an embodiment of this application. The computing device 1200 includes one or more processors 1210 and one or more memories 1220.

[0226] The processor 1210 can be connected to the memory 1220. The memory 1220 can be used to store the program code and data. Therefore, the memory 1220 can be a storage unit inside the processor 1210, an external storage unit independent of the processor 1210, or a component that includes both the storage unit inside the processor 1210 and the external storage unit independent of the processor 1210.

[0227] Optionally, the computing device 1200 may also include a communication interface 1230. It should be understood that... Figure 12 The communication interface 1230 in the computing device 1200 shown can be used to communicate with other devices.

[0228] Optionally, the computing device 1200 may further include a bus 1240. The memory 1220 and communication interface 1230 can be connected to the processor 1210 via the bus 1240. The bus 1240 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus 1240 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 12 The symbol is represented by only one line, but this does not mean that there is only one bus or one type of bus.

[0229] It should be understood that in the embodiments of this application, the processor 1210 may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor. Alternatively, the processor 1210 may employ one or more integrated circuits to execute relevant programs to implement the technical solutions provided in the embodiments of this application.

[0230] The memory 1220 may include read-only memory and random access memory, and provides instructions and data to the processor 1210. A portion of the processor 1210 may also include non-volatile random access memory. For example, the processor 1210 may also store device type information.

[0231] When the computing device 1200 is running, the processor 1210 executes the computer execution instructions in the memory 1220 to perform the operation steps of the above-described vehicle positioning method.

[0232] It should be understood that the computing device 1200 according to the embodiments of this application can correspond to the corresponding subject in executing the methods according to the various embodiments of this application, and the above and other operations and / or functions of each module in the computing device 1200 are respectively for implementing the corresponding processes of the methods of this embodiment. For the sake of brevity, they will not be described in detail here.

[0233] In practical applications, the computing device 1200 can be implemented as a functional unit within a chip, an independent chip, a functional unit within an in-vehicle terminal device, or an independent in-vehicle terminal device. In some embodiments, the computing device 1200 can be a functional unit / module within an in-vehicle infotainment system, a cockpit domain controller (CDC), or a mobile data center / multi-domain controller (MDC). This application does not limit the form or deployment method of the computing device 1200.

[0234] This application embodiment also provides a vehicle that includes a first camera (also referred to as a forward-facing camera) for acquiring images of the road ahead of the vehicle. The vehicle may also include: a vehicle positioning device 1100, a computing device 1200, a computer-readable storage medium as described below, or a computer program product as described below.

[0235] Figure 13 An example diagram of a vehicle according to an embodiment of this application is shown. See also Figure 13 As shown, a forward-facing camera 1310 (i.e., the first camera mentioned above) is installed in the vehicle 1300. The forward-facing camera can capture images of the road ahead of the vehicle in real time and transmit them to the vehicle's computing device 1200 (not shown in the figure) so that the computing device 1200 can use the images of the road ahead to reliably locate the vehicle in real time.

[0236] For example, the vehicle is also equipped with a GPS sensor 1320, an IMU 1330, a vehicle speed sensor 1340, and an acceleration sensor 1350, which are respectively connected to the aforementioned computing device. It should be noted that... Figure 10 This is for illustrative purposes only. In specific applications, this application does not limit the number, type, deployment location, or installation method of the GPS sensor 1320, IMU 1330, vehicle speed sensor 1340, and acceleration sensor 1350.

[0237] For example, the forward-facing camera 1310, GPS sensor 1320, IMU 1330, vehicle speed sensor 1340, and accelerometer 1350 can be connected to the computing device wirelessly or via wired means. For instance, the forward-facing camera, GPS, IMU, vehicle speed sensor, and accelerometer can be connected to the computing device via various communication methods such as Ethernet, Bluetooth, Wi-Fi, cellular networks, Controller Area Network (CAN) bus, and Local Interconnect Network (LIN) bus.

[0238] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, causes the processor to perform the vehicle positioning method described above. Here, the computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0239] This application also provides a computer program product, which includes a computer program that, when run by a processor, causes the processor to execute the vehicle positioning method described above. Here, the computer program product may use one or more programming languages, including but not limited to object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C.

[0240] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments. Many other equivalent embodiments may be included without departing from the concept of this application, all of which fall within the scope of protection of this application.

Claims

1. A vehicle positioning method, characterized in that, include: Acquire an image of the road ahead of the vehicle; A road feature map is obtained based on the image of the road ahead; The lateral deviation of the vehicle is determined based on the road feature map; The second position of the vehicle is obtained based on the lateral deviation of the vehicle, the first pose of the vehicle, the road feature map, and the local map corresponding to the first pose. The second position and / or the second pose of the vehicle are determined based on a plurality of candidate pose points in the local map, and the pose of each candidate pose point is determined based on the lateral deviation of the vehicle, the first pose, and the local map. The multiple candidate pose points are evenly distributed in the same lane direction of the local map with the correction position as the center. The correction position is obtained based on the lateral deviation of the vehicle and the position of the lane center point corresponding to the first pose in the local map and the road orientation angle.

2. The method according to claim 1, characterized in that, Determining the lateral deviation of the vehicle based on the road feature map includes: determining the lateral deviation of the vehicle based on the lane line features in the road feature map.

3. The method according to claim 2, characterized in that, The lane line features in the road feature map include the features of two lane lines in the region of interest (ROI) of the road feature map, and the two lane lines are located on the left and right sides of the first pose.

4. The method according to claim 3, characterized in that, The step of determining the lateral deviation of the vehicle based on the lane line features in the road feature map includes: obtaining the lateral deviation of the vehicle based on the pixel lateral deviation and a preset pixel ratio; determining the pixel lateral deviation based on the lane line features in the top view of the road feature map ROI; and obtaining the lane line features in the top view of the road feature map ROI through the features of two lane lines in the road feature map ROI.

5. The method according to claim 4, characterized in that, The pixel lateral deviation is the distance between the ROI mapping point of the optical center of the first camera and the lane center point in the top view. The lane center point is the lane center point between the lane lines on the left and right sides of the first pose. The first camera is a camera that captures the image of the road ahead.

6. The method according to any one of claims 1-5, characterized in that, The method further includes obtaining a second pose of the vehicle based on the lateral deviation of the vehicle, the first pose, the road feature map, and the local map.

7. The method according to claim 1, characterized in that, The second position is the position information of a candidate pose point selected by feature matching between the projected image and the road feature map; and / or, the second pose is the pose information of a candidate pose point selected by feature matching between the projected image and the road feature map; The projected image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

8. The method according to any one of claims 1-5 and 7, characterized in that, The method further includes: The third pose of the vehicle is obtained based on the second position, the first pose of the vehicle, and the sensor positioning data; or, the third pose of the vehicle is obtained based on the second position, the second pose of the vehicle, and the sensor positioning data.

9. The method according to claim 8, characterized in that, The sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

10. The method according to any one of claims 1-5, 7 and 9, characterized in that, The local map is derived from a vector map.

11. A vehicle positioning device, characterized in that, include: Image acquisition unit, used to acquire images of the road ahead of the vehicle; The feature acquisition unit is used to obtain a road feature map based on the image of the road ahead; A deviation determination unit is used to determine the lateral deviation of the vehicle based on the road feature map; The position determination unit is used to obtain the second position of the vehicle based on the lateral deviation of the vehicle, the first pose of the vehicle, the road feature map, and the local map corresponding to the first pose. The second position and / or the second pose of the vehicle are determined based on a plurality of candidate pose points in the local map, and the pose of each candidate pose point is determined based on the lateral deviation of the vehicle, the first pose, and the local map. The multiple candidate pose points are evenly distributed in the same lane direction of the local map with the correction position as the center. The correction position is obtained based on the lateral deviation of the vehicle and the position of the lane center point corresponding to the first pose in the local map and the road orientation angle.

12. The apparatus according to claim 11, characterized in that, The feature acquisition unit is specifically used to determine the lateral deviation of the vehicle based on the lane line features in the road feature map.

13. The apparatus according to claim 12, characterized in that, The lane line features in the road feature map include the features of two lane lines in the region of interest (ROI) of the road feature map, and the two lane lines are located on the left and right sides of the first pose.

14. The apparatus according to claim 13, characterized in that, The feature acquisition unit is specifically used to obtain the lateral deviation of the vehicle based on the pixel lateral deviation and a preset pixel ratio. The pixel lateral deviation is determined based on the lane line features in the top view of the road feature map ROI. The lane line features in the top view of the road feature map ROI are obtained through the features of two lane lines in the road feature map ROI.

15. The apparatus according to claim 14, characterized in that, The pixel lateral deviation is the distance between the ROI mapping point of the optical center of the first camera and the lane center point in the top view. The lane center point is the lane center point between the lane lines on the left and right sides of the first pose. The first camera is a camera that captures the image of the road ahead.

16. The apparatus according to any one of claims 11-15, characterized in that, The device further includes an attitude determination unit, used to obtain a second attitude of the vehicle based on the lateral deviation of the vehicle, the first pose, the road feature map, and the local map.

17. The apparatus according to claim 11, characterized in that, The second position is the position information of a candidate pose point selected by feature matching between the projected image and the road feature map; and / or, the second pose is the pose information of a candidate pose point selected by feature matching between the projected image and the road feature map; The projected image is obtained based on the pose of the candidate pose point, the intrinsic and extrinsic parameters of the first camera, and the local map.

18. The apparatus according to any one of claims 11-15, 17, characterized in that, The device further includes: The fusion unit is used to obtain a third pose of the vehicle based on the second position, the first pose of the vehicle, and sensor positioning data; or, to obtain a third pose of the vehicle based on the second position, the second pose of the vehicle, and sensor positioning data.

19. The apparatus according to claim 18, characterized in that, The sensor positioning data includes one or more of the following: GPS information, IMU information, INS vehicle attitude information, and chassis information.

20. The apparatus according to any one of claims 11-15, 17 and 19, characterized in that, The local map is derived from a vector map.

21. A computing device, characterized in that, include: A processor and a memory, the memory storing program instructions that, when executed by the processor, cause the processor to perform the method as described in any one of claims 1-10.

22. A computer-readable storage medium having program instructions stored thereon, characterized in that, When executed by a computer, the program instructions cause the computer to perform the method as described in any one of claims 1-10.

23. A vehicle, characterized in that, include: The first camera used to capture images of the road ahead; The vehicle positioning device as described in any one of claims 11-20 or the computing device as described in claim 21.

24. The vehicle according to claim 23, characterized in that, The vehicle also includes one or more of the following: GPS sensor, IMU, vehicle speed sensor, and acceleration sensor.

Images