Positioning method and device of vehicle, electronic equipment and readable storage medium

Abstract

Embodiments of the present application provide a positioning method and device for a vehicle, an electronic device and a readable storage medium. The traffic light image captured by a camera in the vehicle and the lane information corresponding to the vehicle are obtained. The first global pose of the traffic light is determined according to the traffic light image and the parameters corresponding to the camera. The second global pose of the vehicle is determined according to the traffic light image, the parameters corresponding to the camera and the first global pose. The relative angle of the traffic light relative to the vehicle is determined according to the type point information of the traffic light, the parameters corresponding to the camera, the first global pose and the second global pose. The positioning information used to represent the lane in which the vehicle is located is determined according to the second global pose, the relative angle and the lane information. Embodiments of the present application use the traffic light as a natural landmark, and combine the lane information to achieve a high-precision lane-level positioning which does not depend on GPS signals.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a vehicle positioning method, device, electronic device, and readable storage medium. Background Technology

[0002] Currently, autonomous driving is in a phase of rapid development. With technological advancements and increasing demands for safer and more convenient transportation, technologies related to autonomous driving are constantly emerging, driving a transformation across the entire transportation industry. Autonomous vehicles require high-precision lane positioning information to achieve stable autonomous driving functions. Traditional autonomous driving technologies primarily rely on GPS (Global Positioning System) signals for vehicle positioning. However, during vehicle operation, GPS signals are easily affected by factors such as building obstructions, weather changes, and electromagnetic interference, severely impacting the accuracy of vehicle positioning. Therefore, current positioning technologies relying on GPS signals struggle to achieve high-precision vehicle positioning. Summary of the Invention

[0003] This application provides a vehicle positioning method, apparatus, electronic device, and readable storage medium, aiming to improve the problem that current positioning technologies relying on GPS signals are difficult to achieve high-precision vehicle positioning.

[0004] This application discloses a vehicle positioning method, the method comprising:

[0005] The system acquires traffic light images captured by a camera in the vehicle and lane information corresponding to the vehicle. The traffic light images contain traffic lights, each with corresponding point information. The camera has corresponding parameters, and the lane information is used to represent the information of each lane on the road in which the vehicle is traveling.

[0006] The first global pose of the traffic light is determined based on the traffic light image and the parameters corresponding to the camera.

[0007] Based on the traffic light image, the parameters corresponding to the camera, and the first global pose, determine the second global pose corresponding to the vehicle;

[0008] The relative angle of the traffic light to the vehicle is determined based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose.

[0009] Based on the second global pose, the relative angle, and the lane information, positioning information is determined to characterize the lane in which the vehicle is located on the road.

[0010] Optionally, the traffic light image includes at least a first image and a second image with a time difference, and the parameters corresponding to the camera include intrinsic parameters and a first extrinsic parameter, wherein the first extrinsic parameter is used to characterize the rotation and translation of the camera in the world coordinate system; the step of determining the first global pose corresponding to the traffic light based on the traffic light image and the parameters corresponding to the camera includes:

[0011] A first feature point is extracted from the first image and a second feature point is extracted from the second image; both the first feature point and the second feature point correspond to the target point of the traffic light.

[0012] Determine the first coordinates of the first feature point in the first image and the second coordinates of the second feature point in the second image;

[0013] The measurement coordinates of the target point in the world coordinate system are determined based on the first coordinate, the second coordinate, the intrinsic parameter, and the first extrinsic parameter.

[0014] The measured coordinates are determined as the first global pose corresponding to the traffic light.

[0015] Optionally, the parameters corresponding to the camera further include a second extrinsic parameter, which characterizes the rotation and translation of the camera relative to the rear axle of the vehicle; determining the second global pose of the vehicle based on the traffic light image, the parameters corresponding to the camera, and the first global pose includes:

[0016] The intermediate global pose of the camera is determined based on the first coordinates, the intrinsic parameters, and the first global pose; or, the intermediate global pose of the camera is determined based on the second coordinates, the intrinsic parameters, and the first global pose.

[0017] The second global pose corresponding to the vehicle is determined based on the second extrinsic parameter and the intermediate global pose.

[0018] Optionally, the shape point information includes the geographic coordinates of the target point provided by the navigation system in the vehicle in the world coordinate system. Determining the relative angle of the traffic light relative to the vehicle based on the shape point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose includes:

[0019] The perception angle of the traffic light relative to the vehicle is determined based on the second extrinsic parameter and the first global pose.

[0020] The geographical angle of the traffic light relative to the vehicle is determined based on the geographical coordinates and the second global pose.

[0021] The perceived angle and the geographical angle are determined as the relative angle.

[0022] Optionally, determining the positioning information for characterizing the lane in which the vehicle is located on the road based on the second global pose, the relative angle, and the lane information includes:

[0023] The second global pose is adjusted according to the relative angle to obtain the target global pose corresponding to the vehicle.

[0024] Based on the target global pose and the lane information, positioning information is determined to characterize the lane in which the vehicle is located on the road.

[0025] Optionally, adjusting the second global pose according to the relative angle to obtain the target global pose corresponding to the vehicle includes:

[0026] Calculate the angle difference between the perceived angle and the geographic angle;

[0027] A factor graph is constructed based on the angle difference and the second global pose; wherein the factor nodes of the factor graph are residual functions constructed based on the angle difference, and the variable nodes of the factor graph are the second global pose;

[0028] Based on the sum of the residual functions in the factor graph, update the second global pose in the factor graph to obtain the intermediate factor graph;

[0029] Define the sliding window;

[0030] Based on the data within the sliding window, the intermediate factor map is locally optimized to obtain the target factor map;

[0031] The optimized second global pose is extracted from the target factor map and used as the target global pose corresponding to the vehicle.

[0032] Optionally, the lane information includes the coordinates of the centerlines of each lane in the world coordinate system, and the step of determining the positioning information representing the lane in which the vehicle is located on the road based on the target global pose and the lane information includes:

[0033] Based on the target global pose and the coordinates corresponding to the centerline, determine the horizontal distance between the vehicle and the centerline of each lane;

[0034] The target lane in which the vehicle is located on the road is determined based on the horizontal distance;

[0035] Generate positioning information to characterize the vehicle's location in the target lane.

[0036] This application also discloses a vehicle positioning device, characterized in that it includes:

[0037] The acquisition module is used to acquire traffic light images captured by a camera in the vehicle and lane information corresponding to the vehicle; the traffic light images contain traffic lights, the traffic lights have corresponding point information, the camera has corresponding parameters, and the lane information is used to represent the information of each lane on the road in which the vehicle is traveling;

[0038] The first global pose module is used to determine the first global pose of the traffic light based on the traffic light image and the parameters corresponding to the camera.

[0039] The second global pose module is used to determine the second global pose of the vehicle based on the traffic light image, the parameters corresponding to the camera, and the first global pose.

[0040] The relative angle determination module is used to determine the relative angle of the traffic light relative to the vehicle based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose.

[0041] The positioning module is used to determine positioning information representing the lane in which the vehicle is located on the road, based on the second global pose, the relative angle, and the lane information.

[0042] This application also discloses an electronic device, including a processor and a memory, wherein the memory is used to store computer programs; the processor is used to execute the programs stored in the memory to implement the vehicle positioning method as described in any one of the embodiments of this application.

[0043] This application also discloses a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the vehicle positioning method as described in any one of the embodiments of this application.

[0044] The embodiments of this application have the following advantages:

[0045] In this embodiment, traffic light images captured by a camera in the vehicle and lane information corresponding to the vehicle are acquired. The traffic light images contain traffic lights with corresponding point information, the camera has corresponding parameters, and the lane information is used to represent the information of each lane on the road in which the vehicle is traveling. A first global pose corresponding to the traffic light is determined based on the traffic light images and the parameters corresponding to the camera. A second global pose corresponding to the vehicle is determined based on the traffic light images, the parameters corresponding to the camera, and the first global pose. The relative angle of the traffic light to the vehicle is determined based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose. The positioning information used to represent the lane in which the vehicle is located on the road is determined based on the second global pose, the relative angle, and the lane information. This application embodiment utilizes traffic light images and lane information captured by a camera, combined with camera parameters, to determine the vehicle's lane through multi-step pose calculation and relative angle analysis. This overcomes the limitations of traditional GPS positioning, which is susceptible to environmental interference. By combining visual features (traffic lights) with high-precision map data (lane information and point information), autonomous positioning based on the spatial location of traffic lights and the geometric relationship between lanes is achieved. This avoids the risk of inaccurate positioning due to GPS signal loss. By using traffic lights as natural landmarks and combining them with lane information, a high-precision lane-level positioning method that does not rely on GPS signals is realized. Attached Figure Description

[0046] Figure 1 This is a flowchart of a vehicle positioning method provided in an embodiment of this application;

[0047] Figure 2 This is a flowchart of a vehicle positioning method provided in another embodiment of this application;

[0048] Figure 3 This is a structural diagram of the vehicle positioning device provided in the embodiments of this application;

[0049] Figure 4 This is a structural diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0050] To make the technical problems, technical solutions, and beneficial effects solved by this application clearer, the following detailed description is provided in conjunction with embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0051] With the continuous development of autonomous driving technology, ADAS (Advanced Driving Assistance System), as an important component of autonomous driving technology, is also constantly evolving and expanding. ADAS requires a series of sensors and algorithms to assist drivers and improve driving safety. Currently, various ADAS products have emerged on the market, such as lane departure warning systems and adaptive cruise control systems.

[0052] Autonomous vehicles and ADAS systems typically use multiple sensors (such as cameras, radar, and lidar) to acquire information about the vehicle's surroundings. However, effectively fusing data from different sensors to improve positioning accuracy and reliability remains a technical challenge.

[0053] Secondly, map matching is often a crucial step in vehicle navigation and location services. However, due to the complexity and variability of roads, map matching often results in errors, leading to inaccurate vehicle positioning.

[0054] Furthermore, to improve the accuracy and stability of positioning, the algorithm needs to be continuously optimized. However, algorithm optimization often involves complex mathematical models and calculation processes, which require a significant amount of time and resources.

[0055] In addition, traditional vehicle positioning methods often become unstable when faced with changes in the road environment, such as wear, obstruction, or changes in lane markings. Weather conditions (such as rain or fog) may also affect the stability of positioning.

[0056] Clearly, traditional lane positioning methods are highly dependent on external environmental conditions, such as the strength of GPS signals, the performance of radar sensors, weather conditions, and road conditions. In adverse or restricted external environments, the positioning effectiveness of these methods will be severely affected.

[0057] In existing technologies, sensor switching can be performed by determining whether the GNSS (Global Navigation Satellite System) is functioning correctly. When GNSS is functioning correctly, positioning uses GNSS information; when GNSS is malfunctioning but the lidar is functioning correctly, positioning uses lidar information. However, this method does not take into account the position jump that occurs when switching from GNSS to lidar, which can lead to changes in positioning results and severely affect the accuracy of vehicle positioning.

[0058] Therefore, this application provides a vehicle positioning method that uses traffic lights installed beside the road as known reference points. By capturing images of the traffic lights with a camera, the global positioning of the traffic lights is obtained, and lane positioning is performed accordingly. This method can accurately determine the three-dimensional position of the vehicle in the current lane, providing stable and high-precision lane positioning information for autonomous vehicles. This helps the vehicle maintain the correct driving trajectory, avoid deviating from the lane, and thus improve the safety and reliability of autonomous driving.

[0059] In this application, the point information refers to the precise three-dimensional coordinate information (such as height, position, and orientation) of the traffic light in the physical world, which usually comes from high-precision maps or navigation systems. The point information can be used as a known "spatial anchor point" to compare with the traffic light position sensed by the camera, thereby achieving the fusion of visual positioning and map data.

[0060] In this application, global pose (first global pose / second global pose / intermediate global pose / target global pose) refers to a complete description of the position (translation) and orientation (rotation) of an object (such as a traffic light, vehicle, or camera) in the world coordinate system.

[0061] In this application, the extrinsic parameters (first extrinsic parameter / second extrinsic parameter) refer to the geometric relationship parameters between the camera and an external coordinate system (such as a vehicle or world coordinate system), including rotation matrix and translation vector.

[0062] In this application, the Factor Graph refers to a probabilistic graphical model consisting of "variable nodes" (parameters to be optimized, such as vehicle pose) and "factor nodes" (observation constraints, such as angle differences), used for mathematical optimization of multi-source data fusion.

[0063] In this application, the sliding window refers to a dynamic data processing strategy that retains only the observation data from the most recent few moments for calculation and discards historical data to control computational complexity.

[0064] In this application, the residual function refers to a mathematical function (such as the square of the angle difference) representing the difference between observed and predicted values, used to measure the magnitude of error in optimization problems. The physical constraints (such as the angle difference) are quantified into optimizable mathematical indicators in the factor graph, driving pose correction.

[0065] In this application, the perception angle refers to the azimuth angle (such as yaw angle) of the traffic light relative to the vehicle, which is calculated directly from the camera image.

[0066] In this application, the geographical angle refers to the theoretical azimuth angle calculated through geometric relationships based on the geographical coordinates of the traffic lights provided on the map.

[0067] In this application, the target point refers to a point on the traffic light with clear geometric features (such as the center point or border corner points), used for image matching and 3D reconstruction. The target point serves as a reference point for feature extraction and spatial localization, ensuring consistency between multiple frames of images.

[0068] In this application, the coordinates corresponding to the centerline refer to the continuous coordinate sequence of the lane centerline in a high-precision map, describing the geometry of the lane. By mapping the vehicle pose to the lane topology, the lane location can be determined through distance calculation.

[0069] This application provides a vehicle positioning method, which includes: acquiring a traffic light image captured by a camera in the vehicle and lane information corresponding to the vehicle; the traffic light image contains traffic lights, the traffic lights have corresponding point information, the camera has corresponding parameters, and the lane information is used to characterize the information of each lane on the road in which the vehicle is traveling;

[0070] The first global pose of the traffic light is determined based on the traffic light image and the parameters corresponding to the camera.

[0071] Based on the traffic light image, the parameters corresponding to the camera, and the first global pose, determine the second global pose corresponding to the vehicle;

[0072] The relative angle of the traffic light to the vehicle is determined based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose.

[0073] Based on the second global pose, the relative angle, and the lane information, positioning information is determined to characterize the lane in which the vehicle is located on the road.

[0074] This application embodiment utilizes traffic light images and lane information captured by a camera, combined with camera parameters, to determine the vehicle's lane through multi-step pose calculation and relative angle analysis. This overcomes the limitations of traditional GPS positioning, which is susceptible to environmental interference. By combining visual features (traffic lights) with high-precision map data (lane information and point information), autonomous positioning based on the spatial location of traffic lights and the geometric relationship between lanes is achieved. This avoids the risk of inaccurate positioning due to GPS signal loss. By using traffic lights as natural landmarks and combining them with lane information, a high-precision lane-level positioning method that does not rely on GPS signals is realized.

[0075] Example 1

[0076] This application provides a vehicle positioning method, please refer to... Figure 1 This includes the following steps:

[0077] S110. Obtain traffic light images captured by a camera in the vehicle and lane information corresponding to the vehicle; the traffic light images contain traffic lights, the traffic lights have corresponding point information, the camera has corresponding parameters, and the lane information is used to characterize the information of each lane on the road in which the vehicle travels;

[0078] S120. Determine the first global pose of the traffic light based on the traffic light image and the parameters corresponding to the camera.

[0079] S130. Determine the second global pose of the vehicle based on the traffic light image, the parameters corresponding to the camera, and the first global pose.

[0080] S140. Determine the relative angle of the traffic light to the vehicle based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose.

[0081] S150. Based on the second global pose, the relative angle, and the lane information, determine positioning information to characterize the lane in which the vehicle is located on the road.

[0082] In step S110, an image containing traffic lights is captured using an onboard camera. Lane information (such as the number of lanes, centerline coordinates, etc.) of the current road is extracted from a high-precision map or navigation system, and the camera's calibration parameters are obtained. Specifically, when capturing two consecutive frames of images containing traffic lights using the camera, it is necessary to ensure that there is sufficient parallax between the two frames, meaning that the position of the traffic lights changes significantly between the two images.

[0083] As an example, when a vehicle is in motion, two frames of images are quickly captured by a camera in front of the vehicle. Theoretically, as long as the vehicle is moving, there will inevitably be parallax between the two frames within a certain time.

[0084] To ensure sufficient parallax between two frames, feature point matching and displacement quantization can be used to verify whether there is sufficient parallax between the two frames. If not, the images need to be reshot to ensure effective parallax between the images, thereby supporting subsequent high-precision positioning.

[0085] In this embodiment, traffic light images are used as input for visual positioning, providing fixed landmarks (traffic lights) in a dynamic environment. By acquiring lane information, the positioning range of the vehicle can be constrained. Through camera parameters, a conversion relationship between image pixel coordinates and real-world coordinates can be established, realizing the conversion from the pixel coordinates of traffic lights in the image to the world coordinates of the vehicle in the real world.

[0086] In step S120, based on the traffic light image and the parameters corresponding to the camera, the global three-dimensional coordinates of the traffic light, i.e. the first global pose, can be calculated using the triangulation principle.

[0087] Specifically, the triangulation principle is based on observing the same point on the same object with two cameras or at two different times. By calculating the position of this point in the coordinate systems of the two cameras, the position of the object in three-dimensional space can be obtained, that is, the three-dimensional coordinates of the object in the world coordinate system can be obtained.

[0088] In the real world, traffic lights, as road infrastructure, have relatively fixed locations, and the locations provided by high-precision maps or navigation systems are accurate. Using traffic lights as reference points for triangulation can significantly improve positioning accuracy.

[0089] Therefore, in this embodiment, the two-dimensional image coordinates in the traffic light image can be converted into three-dimensional coordinates by the parameters corresponding to the camera, which can be used as a reference anchor point for subsequent vehicle positioning. This eliminates the need to rely on GPS signals and allows for direct positioning of the vehicle in space through visual perception.

[0090] In step S130, the vehicle's position and orientation in the world coordinate system, i.e., the second global pose, are inferred from the first global pose of the traffic light and the camera parameters. This embodiment converts the absolute pose information of the traffic light into the vehicle's own positioning result, achieving real-time positioning based on a single landmark (traffic light). This eliminates the need for multi-sensor fusion, avoiding the problem of low positioning accuracy due to the low precision of multi-sensor fusion.

[0091] In step S140, the relative angle between the traffic light and the vehicle is calculated by combining the traffic light's point information (known geographic coordinates), camera parameters, and vehicle pose. This is so that the consistency of the pose calculation can be verified based on the geometric relationship between the traffic light and the vehicle, error correction can be performed, and the reliability of the positioning can be further improved.

[0092] In step S150, the vehicle's current lane can be obtained through spatial geometric calculation based on the vehicle's second global pose, relative angle, and lane information. This embodiment maps the abstract global pose to a specific lane topology, outputting lane-level positioning results that can be directly used by the autonomous driving decision-making system, such as positioning information like "the vehicle is located in the second lane from the left."

[0093] This application embodiment utilizes traffic light images and lane information captured by a camera, combined with camera parameters, to determine the vehicle's lane through multi-step pose calculation and relative angle analysis. This overcomes the limitations of traditional GPS positioning, which is susceptible to environmental interference. By combining visual features (traffic lights) with high-precision map data (lane information and point information), autonomous positioning based on the spatial location of traffic lights and the geometric relationship between lanes is achieved. This avoids the risk of inaccurate positioning due to GPS signal loss. By using traffic lights as natural landmarks and combining them with lane information, a high-precision lane-level positioning method that does not rely on GPS signals is realized.

[0094] In one embodiment of this application, the traffic light image includes at least a first image and a second image with a time difference, and the parameters corresponding to the camera include intrinsic parameters and a first extrinsic parameter, wherein the first extrinsic parameter is used to characterize the rotation and translation of the camera in the world coordinate system; step S120, determining the first global pose corresponding to the traffic light based on the traffic light image and the parameters corresponding to the camera, includes:

[0095] A first feature point is extracted from the first image and a second feature point is extracted from the second image; both the first feature point and the second feature point correspond to the target point of the traffic light.

[0096] Determine the first coordinates of the first feature point in the first image and the second coordinates of the second feature point in the second image;

[0097] The measurement coordinates of the target point in the world coordinate system are determined based on the first coordinate, the second coordinate, the intrinsic parameter, and the first extrinsic parameter.

[0098] The measured coordinates are determined as the first global pose corresponding to the traffic light.

[0099] In step S110, two frames of images containing traffic lights are acquired, namely the first image and the second image, and there is sufficient parallax between the first image and the second image, that is, the first image and the second image have a time difference.

[0100] This application embodiment uses feature point extraction, feature point matching, and triangulation to convert traffic lights in the first and second images into traffic lights in real space, thereby obtaining the first global pose corresponding to the traffic lights.

[0101] Specifically, firstly, deep learning methods can be used to identify the overall traffic light in two frames (the first image and the second image) and extract its feature points, such as the corner points of the traffic light's border, the center point of the traffic light, or the vertices of the arrow markers. These feature points will be used for subsequent feature point matching and triangulation. As an example, in this embodiment, feature points of the traffic light can be extracted using traditional visual algorithms (such as Canny edge detection and Harris corner localization) or deep learning models (such as attention-based keypoint detection networks).

[0102] After extracting the feature points of the traffic lights from the two image frames, image processing and computer vision algorithms (such as SIFT, SURF, and other feature matching algorithms) are needed to determine which feature points represent the same traffic light portion in both images. As an example, when the target point refers to the upper right corner of the traffic light frame, then the upper right corner of the traffic light frame in the first image is the first feature point, and the upper right corner of the traffic light frame in the second image is the second feature point. Both the first and second feature points correspond to the upper right corner of the traffic light frame.

[0103] By calculating the position of the target point in the two image coordinate systems (the position of the first feature point in the first image and the position of the second feature point in the second image), the position of the traffic light in three-dimensional space can be obtained.

[0104] It should be noted that, from the perspective of pure mathematical triangulation principles, as long as a corresponding point on the same object can be found in two image frames, and the relative pose and other parameters (intrinsic and extrinsic parameters) of the camera that captured the images are known, the position of that point in three-dimensional space can be determined by calculating its position in the coordinate systems of the two images. For example, under ideal calibration and imaging conditions, even a single feature point can be used to construct the geometric relationship for triangulation calculations, and thus obtain its corresponding three-dimensional coordinates.

[0105] However, in practical applications, multiple points are typically needed to improve the accuracy and stability of 3D reconstruction. This is because a single point may contain measurement errors or matching mistakes, leading to inaccurate 3D coordinate calculations. Therefore, multiple points can be used for data redundancy and averaging to reduce the impact of errors. For example, in robot navigation and autonomous driving, a large amount of environmental point information is required to accurately perceive the 3D structure of the surrounding environment.

[0106] Therefore, in this embodiment, feature points corresponding to multiple target points can be obtained through feature point matching. This embodiment describes the process of determining the first global pose of a traffic light using the feature points corresponding to a target point.

[0107] A camera (camera) is typically modeled as a pinhole camera, and its projection equation is expressed as follows:

[0108] λp=PX

[0109] Where λ is the depth value, p is the pixel coordinate on the image plane, P is the projection matrix, which contains the intrinsic and extrinsic parameters (first extrinsic parameter) of the camera, and X is the point coordinate of the traffic light in three-dimensional space, i.e., the first global pose.

[0110] The projection matrix P can be represented as follows:

[0111] P = K[R|t]

[0112] Where K is the camera's intrinsic parameter matrix, which includes information such as focal length and principal point coordinates; R is the camera's rotation matrix, representing the camera's rotation relative to the world coordinate system; and t is the camera's translation vector, representing the camera's translation relative to the world coordinate system.

[0113] Assuming the traffic lights appear in two separate images, we can obtain the pixel coordinates p1 (first coordinates) of the first feature point in the first image and the pixel coordinates p2 (second coordinates) of the second feature point in the second image, as well as the corresponding projection matrices P1 and P2. Based on the projection equation (i.e., formula (1)), we can obtain the following system of equations containing two equations:

[0114]

[0115] Combining the two equations above yields a system of linear equations about X. However, since λ1 and λ2 are depth values ​​of the position, this system of equations is homogeneous and cannot be solved directly.

[0116] To solve for X, we use the Singular Value Decomposition (SVD) method. Specifically, we first need to convert the system of equations into a matrix form, as shown below:

[0117] AX = 0

[0118] Here, A is a matrix consisting of projection matrix P(P1 / P2) and pixel coordinates (p1 / p2). Then, SVD decomposition is performed on A to obtain: A=U∑VT, where U and V are orthogonal matrices, and ∑ is a diagonal matrix containing the singular values ​​of A.

[0119] Finally, take the column vector corresponding to the minimum singular value in the orthogonal matrix V (i.e., the last column of the orthogonal matrix V) and normalize it. The resulting X is the three-dimensional coordinate (measurement coordinate) of the target point in the world coordinate system, which is also the first global pose corresponding to the traffic light.

[0120] This application embodiment uses road infrastructure (traffic lights) as reference points for triangulation. Only one sensor (camera) is needed to obtain the accurate location of the traffic lights, eliminating the need for multiple sensors. This avoids the problem of low accuracy in traffic light positioning caused by the fusion of multiple sensors, thus improving the accuracy of traffic light positioning. This allows for better vehicle positioning based on the traffic light positioning results.

[0121] In one embodiment of this application, the parameters corresponding to the camera further include a second extrinsic parameter, which is used to characterize the rotation and translation of the camera relative to the rear axle of the vehicle; step S130, determining the second global pose corresponding to the vehicle based on the traffic light image, the parameters corresponding to the camera, and the first global pose, includes:

[0122] The intermediate global pose of the camera is determined based on the first coordinates, the intrinsic parameters, and the first global pose; or, the intermediate global pose of the camera is determined based on the second coordinates, the intrinsic parameters, and the first global pose.

[0123] The second global pose corresponding to the vehicle is determined based on the second extrinsic parameter and the intermediate global pose.

[0124] In this embodiment, after obtaining the first global pose corresponding to the traffic light, the intermediate global pose corresponding to the camera can be calculated using the camera imaging principle. Specifically, the calculation essentially uses the Perspective-n-Point (PnP) algorithm to deduce the position and orientation of an object in three-dimensional space from points in a two-dimensional image. This can be understood as follows: given a point in three-dimensional space (the first global pose) and its corresponding two-dimensional image projection point (the first coordinate or the second coordinate), the PnP method can solve for the camera's orientation, i.e., the camera's rotation matrix and translation vector, which is also the intermediate global pose in this embodiment.

[0125] It should be noted that the second extrinsic parameter refers to the rotation and translation of the camera relative to the rear axle of the vehicle. That is, taking the rear axle of the vehicle as the origin of the vehicle coordinate system, the second extrinsic parameter can be understood as the position of the camera in the vehicle coordinate system.

[0126] In this embodiment, the transformation between the camera coordinate system, the world coordinate system, and the vehicle coordinate system can be achieved based on the intermediate global pose and the second extrinsic parameter. Specifically, the transformation from the camera coordinate system to the vehicle coordinate system can be achieved through the second extrinsic parameter, and the transformation from the camera coordinate system to the world coordinate system can be achieved through the intermediate global pose. Therefore, by combining the intermediate global pose and the second extrinsic parameter, the second global pose corresponding to the vehicle can be obtained.

[0127] As an example, this application can multiply the intermediate global pose and the second extrinsic parameter to obtain the second global pose.

[0128] This application embodiment achieves high-precision and high-reliability mapping from visual perception (traffic light image) to vehicle body pose (second global pose) through the transformation between different coordinate systems, thereby realizing high-precision vehicle positioning.

[0129] In one embodiment of this application, the shape point information includes the geographic coordinates of the target point provided by the navigation in the vehicle in the world coordinate system. Step S140, determining the relative angle of the traffic light relative to the vehicle based on the shape point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose, includes:

[0130] The perception angle of the traffic light relative to the vehicle is determined based on the second extrinsic parameter and the first global pose.

[0131] The geographical angle of the traffic light relative to the vehicle is determined based on the geographical coordinates and the second global pose.

[0132] The perceived angle and the geographical angle are determined as the relative angle.

[0133] In this embodiment, although the vehicle's position in real-world space (second global pose) has been determined in step S130, there will be some error if lane-level positioning is performed directly based on the second global pose. Therefore, this embodiment also calculates the relative angle between the traffic light and the vehicle so that the positioning result can be optimized based on the relative angle to improve the accuracy of the positioning result.

[0134] It should be noted that the point information includes the precise geographic coordinates of the traffic lights in the world coordinate system provided by the navigation system.

[0135] As an example, the perception angle specifically reflects the orientation of the traffic light from the camera's perspective, and is affected by the camera's installation position (second extrinsic parameter). The first global pose is determined based on the image captured by the camera. Therefore, based on the combination of the first global pose and the second extrinsic parameter, the angle of the traffic light relative to the camera can be calculated, and this angle is the perception angle zij in the embodiments of this application.

[0136] As an example, the geographic angle specifically reflects the absolute position of the traffic light relative to the vehicle in the map coordinate system. Relying on the accuracy of the high-precision map, it is necessary to calculate the relative angle between the second global pose and the geographic coordinates based on the geographic coordinates in the point information provided by the navigation, that is, the geographic angle rij(x) of the traffic light relative to the vehicle, where x is the second global pose corresponding to the vehicle.

[0137] This application embodiment establishes a geometric constraint and data fusion mechanism by fusing visual perception angles (based on camera parameters and image positioning) and geographic angles (based on navigation coordinates from high-precision maps). This effectively corrects errors from a single positioning source, improves the accuracy of the vehicle's global pose, and provides a more reliable spatial relationship basis for subsequent lane-level positioning.

[0138] In one embodiment of this application, step S150, determining positioning information for characterizing the lane in which the vehicle is located on the road based on the second global pose, the relative angle, and the lane information, includes:

[0139] The second global pose is adjusted according to the relative angle to obtain the target global pose corresponding to the vehicle.

[0140] Based on the target global pose and the lane information, positioning information is determined to characterize the lane in which the vehicle is located on the road.

[0141] In this embodiment of the application, lane-level vehicle three-dimensional positioning is achieved based on relative angles. Specifically, the second global pose needs to be continuously adjusted based on relative angles until it meets the requirements. At this time, the second global pose is the target global pose.

[0142] The target global pose is the final position of the vehicle in the real world space calculated in this embodiment of the application. Based on this, combined with the lane information of the current road on which the vehicle is traveling, the positioning information of which lane the vehicle is located in can be determined.

[0143] This application embodiment dynamically optimizes the vehicle's global pose (second global pose) by relative angles, and combines it with lane geometry information (such as centerline coordinates) to refine the positioning results to the lane level, thereby achieving high-precision three-dimensional positioning and meeting the real-time and accuracy requirements of autonomous driving for lane-level position judgment.

[0144] In one embodiment of this application, adjusting the second global pose according to the relative angle to obtain the target global pose corresponding to the vehicle includes:

[0145] Calculate the angle difference between the perceived angle and the geographic angle;

[0146] A factor graph is constructed based on the angle difference and the second global pose; wherein the factor nodes of the factor graph are residual functions constructed based on the angle difference, and the variable nodes of the factor graph are the second global pose;

[0147] Based on the sum of the residual functions in the factor graph, update the second global pose in the factor graph to obtain the intermediate factor graph;

[0148] Define the sliding window;

[0149] Based on the data within the sliding window, the intermediate factor map is locally optimized to obtain the target factor map;

[0150] The optimized second global pose is extracted from the target factor map and used as the target global pose corresponding to the vehicle.

[0151] In this embodiment, it is necessary to calculate the angle difference between the perceived angle and the geographic angle, and construct a factor graph based on this angle difference and the second global pose. The factor graph is a probabilistic graphical model composed of variable nodes (parameters to be optimized) and factor nodes (observation constraints). In this embodiment, the second global pose is the parameter to be optimized, and the residual function constructed from the angle difference is the observation constraint. Therefore, the factor nodes in the factor graph of this embodiment are residual functions, and the variable nodes are the second global pose. Specifically, the residual function eij(x) = zij - rij(x).

[0152] As an example, the global localization problem of a vehicle in this application embodiment can be transformed into minimizing the sum or weighted sum of the residuals generated by all factors: min x∑ ij ρ(eij(x)), where ρ is a robust loss function used to handle noise and outliers. Subsequently, a nonlinear optimization algorithm (such as Levenberg-Marquardt) can be used to iteratively update the variable nodes (second global pose) in the factor graph to obtain an intermediate factor graph. The intermediate factor graph contains the pre-optimized second global pose, but has not yet undergone sliding window local optimization.

[0153] In this embodiment, a sliding window needs to be set according to computing resources and real-time requirements to dynamically update the optimization variables and avoid the accumulation of historical errors. Specifically, if the window is too large, it will lead to high computational overhead and poor real-time performance; if the window is too small, it will lead to unstable optimization results and susceptibility to noise.

[0154] It should be noted that in this embodiment, only the data within the sliding window is locally optimized, updating the variable nodes (the second global pose after preliminary optimization). An iterative optimization algorithm using gradient descent is employed, updating only the variable and factor nodes that change within the window, while retaining historical data outside the window as prior constraints to avoid pose jumps, thus obtaining the final target factor map. The target factor map contains the final optimized second global pose, which is the target global pose in this application. The target global pose in this application is a high-precision pose that has undergone angle difference verification and sliding window optimization, and can be directly used for lane-level localization.

[0155] In this embodiment, a residual function is constructed using angle differences to achieve iterative correction of vehicle pose. The optimization results are dynamically updated through a sliding window mechanism to suppress cumulative errors and instantaneous noise. Furthermore, local optimization can retain historical data as prior constraints to avoid pose jumps and improve system stability. In addition, angle difference verification achieves closed-loop fusion of visual positioning and map data to ensure the long-term accuracy of vehicle positioning.

[0156] In one embodiment of this application, the lane information includes the coordinates of the centerlines of each lane in the world coordinate system. The step of determining positioning information representing the lane in which the vehicle is located on the road, based on the target global pose and the lane information, includes:

[0157] Based on the target global pose and the coordinates corresponding to the centerline, determine the horizontal distance between the vehicle and the centerline of each lane;

[0158] The target lane in which the vehicle is located on the road is determined based on the horizontal distance;

[0159] Generate positioning information to characterize the vehicle's location in the target lane.

[0160] It should be noted that lane information includes not only the lane centerline and its corresponding coordinates in the world coordinate system, but also lane-related information such as the number of lanes and lane width. The target global pose is the vehicle's coordinates in the world coordinate system.

[0161] As an example, the target's global pose is projected onto the road plane, and then a point closest to the vehicle is found on the centerline of each lane. The distance between the vehicle and that point can be calculated using Euclidean distance, which is the horizontal distance between the vehicle and the centerline of each lane.

[0162] Specifically, among multiple horizontal distances, the lane corresponding to the smallest horizontal distance can be identified as the target lane where the vehicle is currently located. Therefore, positioning information that represents the vehicle's location in the target lane can be generated.

[0163] As an example, suppose there are five lanes in the current road: lane A, lane B, lane C, lane D, and lane E. Calculate the target's global pose and the horizontal distances between the five lanes, which are d1, d2, d3, d4, and d5, respectively. Sort the horizontal distances by size, and we get d3 < d2 < d4 < d1 < d5. Therefore, lane C, which corresponds to d3, can be identified as the target lane. That is, the vehicle is currently located in lane C of the current road.

[0164] This application embodiment achieves centimeter-level lane positioning based on the optimized geometric relationship between the target global pose and the lane centerline coordinates. Furthermore, it has low horizontal distance calculation complexity and short calculation time per operation, enabling rapid matching of the vehicle's lane. This not only improves the accuracy but also the efficiency of vehicle positioning, meeting the real-time requirements of autonomous driving and providing reliable lane-level position information for autonomous driving decisions.

[0165] It should be noted that the embodiments of this application can be used as part of ADAS, providing drivers with real-time lane departure warnings, lane keeping assist, and other functions. Through accurate lane positioning information, ADAS can monitor whether the vehicle deviates from its lane in real time and, when necessary, issue warnings to the driver or automatically adjust the vehicle's trajectory.

[0166] This application's embodiments achieve high-precision three-dimensional positioning of vehicles relative to lanes by combining traffic light triangulation and feature extraction techniques. Traffic lights, as road infrastructure, have relatively fixed and accurate positions, making them useful as reference points for triangulation, which significantly improves positioning accuracy. Simultaneously, by utilizing the reciprocity of traffic light features to calculate angular residuals, multiple verifications and error corrections can be achieved, further enhancing positioning reliability. This high-precision lane positioning is crucial for applications such as autonomous driving and vehicle navigation, ensuring vehicles maintain the correct driving path in complex and ever-changing road environments.

[0167] Traditional lane positioning methods are often highly dependent on the external environment. For example, GPS signals are easily affected by obstructions, and the performance of sensors such as radar or lidar may degrade under adverse weather conditions. However, this application's embodiments reduce the dependence of vehicle positioning on the external environment by utilizing traffic lights as road infrastructure. Even in complex and changing road environments, such as those with tall buildings or tunnels, reliable lane positioning can be achieved. This significantly reduces the impact of environmental factors on positioning accuracy and improves the stability and reliability of positioning.

[0168] Furthermore, high-precision lane positioning is fundamental to achieving safe autonomous driving and vehicle navigation. This application's embodiments, by improving positioning accuracy and stability, can provide autonomous vehicles with more accurate driving paths and obstacle information, thereby effectively avoiding the risk of vehicles deviating from their lanes or colliding with other vehicles or pedestrians. Simultaneously, for vehicle navigation systems, high-precision lane positioning also provides more accurate navigation information, helping drivers better plan their routes and reducing misjudgments and erroneous operations during driving.

[0169] Example 2

[0170] This application also provides a vehicle positioning method, please refer to... Figure 2 This includes the following steps:

[0171] S210. Obtain traffic light images captured by cameras in the vehicle and lane information corresponding to the vehicle; the traffic light images contain traffic lights and include at least a first image and a second image;

[0172] S220. Preprocess the first image and the second image to confirm whether the first image and the second image meet the requirements;

[0173] In step S220, it is determined whether there is a parallax between the first image and the second image. If the requirement is not met, the process returns to step S210 to reacquire the traffic light image; if the requirement is met, step S230 can then be performed.

[0174] S230. Extract and match feature points in the first and second images, perform triangulation calculation based on the feature points, and obtain the first global pose corresponding to the traffic light.

[0175] S240. Based on the camera's intrinsic and extrinsic parameters, obtain the vehicle's second global pose.

[0176] S250. Calculate the perception angle and geographic angle based on the point information corresponding to the traffic lights provided on the map.

[0177] S260. Construct a factor map corresponding to the global positioning of the vehicle through perception angle and geographic angle;

[0178] S270. Iteratively optimize the factor map to obtain the optimized target factor map, and then determine the target global pose of the vehicle from the target factor map.

[0179] In this embodiment, the method descriptions of steps S210 to S270 are the same as in Embodiment 1. For a detailed description, please refer to Embodiment 1, which will not be repeated here.

[0180] This application also provides a vehicle positioning device 30, please refer to... Figure 3It includes the following modules:

[0181] The acquisition module 310 is used to acquire traffic light images captured by a camera in the vehicle and lane information corresponding to the vehicle; the traffic light images contain traffic lights, the traffic lights have corresponding point information, the camera has corresponding parameters, and the lane information is used to represent the information of each lane on the road in which the vehicle is traveling;

[0182] The first global pose module 320 is used to determine the first global pose of the traffic light based on the traffic light image and the parameters corresponding to the camera.

[0183] The second global pose module 330 is used to determine the second global pose of the vehicle based on the traffic light image, the parameters corresponding to the camera, and the first global pose.

[0184] The relative angle determination module 340 is used to determine the relative angle of the traffic light relative to the vehicle based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose.

[0185] The positioning module 350 is used to perform a process of determining positioning information that characterizes the lane in which the vehicle is located on the road, based on the second global pose, the relative angle, and the lane information.

[0186] This application also provides an electronic device 40, please refer to... Figure 4 It includes a processor 410 and a memory 420, wherein the memory 410 is used to store computer programs; the processor 420 is used to execute the programs stored in the memory 410 to implement the vehicle positioning method described in any embodiment of this application.

[0187] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the vehicle positioning method described in any embodiment of this application.

[0188] In this application, "multiple" refers to two or more.

[0189] In this application, unless otherwise expressly defined, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0190] The terms “first,” “second,” “third,” “fourth,” etc., in this application (if present) are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence.

[0191] In this application, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, in this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0192] Unless otherwise specified, all steps in this application may be performed sequentially or randomly. For example, if the method includes steps A and B, it means that the method may include steps A and B performed sequentially, or it may include steps B and A performed sequentially. For example, if the method may also include step C, it means that step C may be added to the method in any order. For example, the method may include steps A, B, and C, or it may include steps A, C, and B, or it may include steps C, A, and B, etc.

[0193] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A method for locating a vehicle, characterized in that, include: Acquire traffic light images captured by cameras inside the vehicle and lane information corresponding to the vehicle; The traffic light image contains traffic lights, each with corresponding point information; the camera has corresponding parameters; and the lane information is used to characterize the information of each lane on the road in which the vehicle is traveling. The first global pose of the traffic light is determined based on the traffic light image and the parameters corresponding to the camera. Based on the traffic light image, the parameters corresponding to the camera, and the first global pose, determine the second global pose corresponding to the vehicle; The relative angle of the traffic light to the vehicle is determined based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose. Based on the second global pose, the relative angle, and the lane information, positioning information is determined to characterize the lane in which the vehicle is located on the road.

2. The method according to claim 1, characterized in that, The traffic light image includes at least a first image and a second image with a time difference. The parameters corresponding to the camera include intrinsic parameters and a first extrinsic parameter. The first extrinsic parameter is used to characterize the rotation and translation of the camera in the world coordinate system. The step of determining the first global pose corresponding to the traffic light based on the traffic light image and the parameters corresponding to the camera includes: A first feature point is extracted from the first image and a second feature point is extracted from the second image; both the first feature point and the second feature point correspond to the target point of the traffic light. Determine the first coordinates of the first feature point in the first image and the second coordinates of the second feature point in the second image; The measurement coordinates of the target point in the world coordinate system are determined based on the first coordinate, the second coordinate, the intrinsic parameter, and the first extrinsic parameter. The measured coordinates are determined as the first global pose corresponding to the traffic light.

3. The method according to claim 2, characterized in that, The parameters corresponding to the camera also include a second extrinsic parameter, which is used to characterize the rotation and translation of the camera relative to the rear axle of the vehicle. Determining the second global pose of the vehicle based on the traffic light image, the parameters corresponding to the camera, and the first global pose includes: The intermediate global pose of the camera is determined based on the first coordinates, the intrinsic parameters, and the first global pose; or, the intermediate global pose of the camera is determined based on the second coordinates, the intrinsic parameters, and the first global pose. The second global pose corresponding to the vehicle is determined based on the second extrinsic parameter and the intermediate global pose.

4. The method according to claim 3, characterized in that, The point information includes the geographic coordinates of the target point provided by the navigation system in the vehicle in the world coordinate system. Determining the relative angle of the traffic light with respect to the vehicle based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose includes: The perception angle of the traffic light relative to the vehicle is determined based on the second extrinsic parameter and the first global pose. The geographical angle of the traffic light relative to the vehicle is determined based on the geographic coordinates and the second global pose. The perceived angle and the geographical angle are determined as the relative angle.

5. The method according to claim 4, characterized in that, The step of determining the positioning information representing the lane in which the vehicle is located on the road based on the second global pose, the relative angle, and the lane information includes: The second global pose is adjusted according to the relative angle to obtain the target global pose corresponding to the vehicle. Based on the target global pose and the lane information, positioning information is determined to characterize the lane in which the vehicle is located on the road.

6. The method according to claim 5, characterized in that, The step of adjusting the second global pose according to the relative angle to obtain the target global pose corresponding to the vehicle includes: Calculate the angle difference between the perceived angle and the geographic angle; A factor graph is constructed based on the angle difference and the second global pose; wherein the factor nodes of the factor graph are residual functions constructed based on the angle difference, and the variable nodes of the factor graph are the second global pose; Based on the sum of the residual functions in the factor graph, update the second global pose in the factor graph to obtain the intermediate factor graph; Define the sliding window; Based on the data within the sliding window, the intermediate factor map is locally optimized to obtain the target factor map; The optimized second global pose is extracted from the target factor map and used as the target global pose corresponding to the vehicle.

7. The method according to claim 5, characterized in that, The lane information includes the coordinates of the centerlines of each lane in the world coordinate system. The step of determining the positioning information representing the lane in which the vehicle is located on the road, based on the target global pose and the lane information, includes: Based on the target global pose and the coordinates corresponding to the centerline, determine the horizontal distance between the vehicle and the centerline of each lane; The target lane in which the vehicle is located on the road is determined based on the horizontal distance; Generate positioning information to characterize the vehicle's location in the target lane.

8. A vehicle positioning device, characterized in that, include: The acquisition module is used to acquire traffic light images captured by cameras in the vehicle and lane information corresponding to the vehicle; The traffic light image contains traffic lights, each with corresponding point information; the camera has corresponding parameters; and the lane information is used to characterize the information of each lane on the road in which the vehicle is traveling. The first global pose module is used to determine the first global pose of the traffic light based on the traffic light image and the parameters corresponding to the camera. The second global pose module is used to determine the second global pose of the vehicle based on the traffic light image, the parameters corresponding to the camera, and the first global pose. The relative angle determination module is used to determine the relative angle of the traffic light relative to the vehicle based on the point information corresponding to the traffic light, the parameters corresponding to the camera, the first global pose, and the second global pose. The positioning module is used to determine positioning information representing the lane in which the vehicle is located on the road, based on the second global pose, the relative angle, and the lane information.

9. An electronic device, characterized in that, Including processor and memory, among which Memory, used to store computer programs; A processor for executing a program stored in memory to implement the method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-7.

Images