Vehicle positioning method and system based on intelligent network connection, vehicle and medium

Abstract

The invention provides a vehicle positioning method and system based on intelligent network connection, a vehicle and a medium, and belongs to the technical field of vehicle positioning, and the method comprises the steps: obtaining the related positioning data of the vehicle; obtaining a semantic landmark and environment point cloud data; constructing a factor graph containing state nodes and constraint factors; performing optimization solution on the factor graph to obtain a fusion positioning result of the vehicle and a screened semantic landmark detection set; broadcasting the fusion positioning result and the semantic landmark detection set to an adjacent vehicle; receiving the collaboration data broadcasted by the adjacent vehicles, and comparing and retaining the collaboration data with consistent semantics; a target semantic landmark overlapped with the self-vehicle semantic landmark detection set is screened out through coordinate conversion and feature comparison, and the observation deviation of the target semantic landmark is calculated; and when the observation deviation exceeds a preset threshold value, correcting the position of the vehicle in the local coordinate system through a least square method. According to the invention, the accuracy, reliability and robustness of positioning in extreme weather are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of vehicle positioning technology, specifically to a vehicle positioning method, system, vehicle, and medium based on intelligent network connectivity. Background Technology

[0002] With the rapid development of autonomous driving technology, the demand for high-precision and high-reliability positioning for vehicles is becoming increasingly urgent. Global Navigation Satellite Systems (GNSS) can provide good absolute positioning information in open environments, but their signals are easily affected by obstruction and multipath interference from environments such as urban canyons, tunnels, and tree-lined roads, leading to a decrease in positioning accuracy or even failure, and thus failing to meet the continuous positioning requirements in complex scenarios.

[0003] To address the inherent limitations of GNSS, the industry has widely adopted positioning schemes based on the fusion of multiple sensors, including inertial measurement units (IMUs), cameras, lidar, and millimeter-wave radar. These schemes use algorithms such as Kalman filtering and factor graph optimization to fuse data from various sensors, providing continuous pose estimation even when GNSS signals fail. For example, existing technologies include schemes that switch between different positioning algorithms based on satellite signal stability, and fusion positioning systems that combine visual road sign recognition with inertial navigation systems.

[0004] However, the aforementioned existing technical solutions mainly rely on "vehicle-centric intelligence," which has inherent limitations: First, their performance is highly dependent on the configuration of the vehicle's own sensors and the accuracy of the environmental perception algorithm. In extreme weather (such as rain, snow, and fog) or when the sensors themselves malfunction, positioning reliability will plummet. Second, the perception range of a single vehicle is limited, making it unable to acquire environmental information beyond the line of sight of its own sensors. This poses significant safety hazards in complex traffic scenarios such as large vehicles obstructing traffic lights at intersections or unexpected situations behind curves. Furthermore, such solutions only utilize the vehicle's own information and cannot correct inherent biases in its own perception system. The lack of external verification mechanisms leads to the continuous accumulation of positioning errors in scenarios where GNSS is frequently unavailable, making it difficult to solve the fundamental positioning drift problem. Summary of the Invention

[0005] In view of this, it is necessary to provide a vehicle positioning method, system, vehicle and medium based on intelligent network connectivity to solve the technical problems of sensor fusion that rely on single-vehicle intelligence in the existing technology, which has limited sensing range and low reliability in extreme environments.

[0006] To address the aforementioned technical problems, in a first aspect, the present invention provides a vehicle positioning method based on intelligent connected vehicles, comprising: Obtain relevant vehicle location data; Environmental point cloud data is obtained from the raw laser data, and semantic landmarks are obtained from the collected environmental images; Based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data, a factor graph containing state nodes and constraint factors is constructed. The factor graph is solved to obtain the vehicle's fused localization result and a filtered semantic landmark detection set; The fused positioning results and the semantic landmark detection set are broadcast to nearby vehicles using vehicle-to-vehicle communication technology. After receiving the collaborative data broadcast by the nearby vehicles, the vehicle performs time synchronization verification and semantic consistency comparison on the collaborative data between itself and the nearby vehicles, and retains the semantically consistent collaborative data; the collaborative data includes the fusion positioning results and semantic landmark detection set of the nearby vehicles; From the semantically consistent collaborative data, target semantic landmarks that overlap with the vehicle semantic landmark detection set are filtered out through coordinate transformation and feature comparison, and the observation deviation of the vehicle and the neighboring vehicles on the target semantic landmarks is calculated. When the observation deviation exceeds a preset threshold, the position of the vehicle in the local coordinate system is corrected by least squares method based on the semantic landmark coordinates with the highest confidence.

[0007] In one possible implementation, obtaining environmental point cloud data based on raw laser data and obtaining semantic landmarks based on acquired environmental images includes: The environmental image is processed using the YOLOv8 model to output the semantic landmarks; the semantic landmarks include category labels, pixel coordinates, and confidence scores. Using camera calibration parameters, the pixel coordinates are transformed into three-dimensional coordinates in the vehicle's local coordinate system; Based on the transformed 3D coordinates, category labels, and confidence scores, a structured semantic landmark detection set is generated; Acquire raw laser data generated by radar, the raw laser data containing distance and angle information of obstacles in the environment; The original laser data was processed using the DBSCAN clustering algorithm to distinguish between dynamic and static obstacles; Extract the geometric contour features of the point cloud clusters corresponding to the static obstacles, and generate structured environmental point cloud data based on the geometric contour features.

[0008] In one possible implementation, the relevant positioning data includes Global Navigation Satellite System data, inertial measurement data, and wheel odometer data. The step of constructing a factor graph containing state nodes and constraint factors based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data includes: The vehicle's pose in a discrete time series is defined as a state node; Establish multiple constraint factors connecting the state nodes; the multiple constraint factors include GNSS factors obtained from the Global Navigation Satellite System data, IMU factors obtained from the inertial measurement data, odometer factors obtained from the wheel odometer data, semantic landmark factors obtained from the semantic landmarks, and radar point cloud factors obtained from the environmental point cloud data.

[0009] In one possible implementation, solving the factor graph to obtain the vehicle's fused localization result and a filtered semantic landmark detection set includes: Construct the first optimization function with the objective of minimizing the weighted sum of squared residuals for all constraint factors: ; in, Let X be the first optimization function, M be the set of all state nodes, ri be the residual vector of the i-th constraint factor, and Wi be the weight matrix of the i-th constraint factor. The first optimization function is solved using the Gauss-Newton iterative algorithm to obtain the optimal vehicle pose sequence as the fusion localization result, and a set of semantic landmark detections with a confidence level higher than the target threshold after filtering is output.

[0010] In one possible implementation, the step of filtering out target semantic landmarks overlapping with the vehicle semantic landmark detection set through coordinate transformation and feature comparison includes: Based on the fusion positioning results of the vehicle and the neighboring vehicles, the relative poses of the two vehicles are calculated, and the corresponding coordinate transformation matrix is ​​constructed. Using the coordinate transformation matrix, the coordinates of the semantic landmarks in the neighboring vehicle semantic landmark detection set are transformed to the vehicle's local coordinate system to obtain the transformed landmark coordinates; Calculate the Euclidean distance between the transformed landmark coordinates and the vehicle's landmark coordinates; Calculate the radar point cloud feature error and visual texture similarity between the vehicle and the neighboring vehicles for the same candidate landmark pair; When the Euclidean distance is less than a first preset threshold, the radar point cloud feature error is less than a second preset threshold, and the visual texture similarity is greater than a third preset threshold, the candidate landmark pair is determined to be the target semantic landmark.

[0011] In one possible implementation, correcting the vehicle's position in the local coordinate system using the least squares method includes: Construct a second optimization function with the objective of minimizing the weighted positional bias: ; in, For the second optimization function, The number of overlapping landmarks selected; The confidence level of the m-th landmark for the baseline vehicle; It is a correction amount, including and Quantity; Let m be the coordinates of the m-th semantic landmark observed by the vehicle in its local coordinate system; The m-th semantic landmark coordinates after conversion for adjacent vehicles; The position of the vehicle in the local coordinate system is updated based on the optimal position correction obtained by solving the second optimization function.

[0012] In one possible implementation, after correcting the vehicle's position in the local coordinate system using the least squares method, the method further includes: The updated vehicle positions are used as the new initial state to update the factor graph; Using vehicle-to-vehicle communication technology, the corrected positioning results based on the factor graph are broadcast along with the updated semantic landmark detection set.

[0013] Secondly, the present invention also provides a vehicle positioning system based on intelligent network connectivity, comprising: The data acquisition module is used to acquire relevant vehicle location data; The environment perception module is used to obtain environmental point cloud data based on the raw laser data and semantic landmarks based on the collected environmental images. The factor graph construction module is used to construct a factor graph containing state nodes and constraint factors based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data. The data fusion module is used to solve the factor graph to obtain the vehicle's fused positioning result and a filtered semantic landmark detection set; The vehicle-to-vehicle communication module is used to broadcast the fused positioning results and the semantic landmark detection set to nearby vehicles via vehicle-to-vehicle communication technology; The verification and comparison module is used to receive the collaborative data broadcast by the nearby vehicles, and then perform time synchronization verification and semantic consistency comparison on the collaborative data between the vehicle and the nearby vehicles, retaining the semantically consistent collaborative data; the collaborative data includes the fusion positioning results and semantic landmark detection set of the nearby vehicles. The filtering and calculation module is used to filter out target semantic landmarks that overlap with the vehicle semantic landmark detection set from the semantically consistent collaborative data through coordinate transformation and feature comparison, and to calculate the observation deviation of the vehicle and the neighboring vehicles on the target semantic landmarks. The collaborative correction module is used to correct the position of the vehicle in the local coordinate system by using the least squares method, based on the semantic landmark coordinates with the highest confidence, when the observation deviation exceeds a preset threshold.

[0014] Thirdly, the present invention also provides an intelligent vehicle, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the vehicle positioning method based on intelligent connectivity described in any of the above implementations.

[0015] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instructions, which, when executed by a processor, can implement the steps in the vehicle positioning method based on intelligent network connectivity described in any of the above implementations.

[0016] The beneficial effects of this invention are as follows: The vehicle positioning method based on intelligent connected vehicles provided by this invention firstly integrates positioning-related data through deep fusion of factor graphs, semantic landmarks obtained from visual sensors, and environmental point cloud data acquired by radar. In scenarios where a single sensor fails, such as weak positioning signals or visual interference, other sensors can instantly fill in, ensuring the continuity and accuracy of positioning and demonstrating strong environmental robustness. Furthermore, with the help of V2X communication, vehicles can acquire environmental information beyond their own sensor line of sight (such as traffic lights obscured by large vehicles ahead, road conditions after curves), which not only improves the reliability of positioning but also greatly enhances the global perception capability of intelligent connected vehicles in complex traffic scenarios, providing more comprehensive information support for high-level autonomous driving decisions. Furthermore, through a triple filtering mechanism of "time verification-semantic comparison-feature matching," the quality of collaborative data is ensured. Then, least squares correction is performed based on high-confidence observations, which can effectively discover and correct inherent biases and cumulative drift that are difficult to avoid in single-vehicle sensor fusion, fundamentally improving the long-term stability of the system. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of an embodiment of the vehicle positioning method based on intelligent network connectivity provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S102; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S103; Figure 4 For the present invention Figure 1 A schematic diagram of an embodiment of S104; Figure 5 For the present invention Figure 1 A schematic diagram of an embodiment of S107; Figure 6 For the present invention Figure 1 A schematic diagram of an embodiment of S108; Figure 7 A schematic diagram of an embodiment of the intelligent connected vehicle positioning system provided by the present invention; Figure 8 A schematic diagram of an embodiment of the intelligent vehicle provided by the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0020] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0021] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0022] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0023] Before demonstrating the embodiments, the following terms will be explained.

[0024] V2V (Vehicle-to-Vehicle) allows vehicles to exchange data wirelessly directly without the need for intermediate facilities such as base stations.

[0025] This invention provides a vehicle positioning method, system, vehicle, and medium based on intelligent network connectivity, which will be described below.

[0026] Figure 1 This is a schematic flowchart of an embodiment of the vehicle positioning method based on intelligent connected vehicles provided by the present invention, as shown below. Figure 1 As shown, vehicle positioning methods based on intelligent connected vehicles include: S101. Obtain relevant vehicle location data.

[0027] It should be noted that the relevant positioning data includes Global Navigation Satellite System data (e.g., GNSS), Inertial Measurement Unit (IMU) data, and wheel odometer data, as described in the embodiments below. Global satellite positioning data (latitude, longitude, altitude, speed, PDOP value, etc.) is acquired through an onboard GNSS receiver, and its confidence level is calculated (based on the number of satellites, signal-to-noise ratio, and multipath index). IMU data is obtained by collecting the vehicle's angular velocity (three-axis gyroscope) and acceleration (three-axis accelerometer). Wheel speed sensors mounted on the wheels measure the number of wheel rotations and instantaneous rotational speed. Combined with the known tire circumference, the vehicle's cumulative travel distance and real-time speed are calculated, providing a continuous planar motion reference. By fusing these three complementary data types—GNSS (absolute positioning), IMU (high-frequency relative motion), and wheel odometer (planar motion accumulation)—a stable, multi-dimensional initial state estimate is provided for subsequent fusion positioning, effectively reducing the risk of failure or abrupt changes in a single data source.

[0028] S102. Obtain environmental point cloud data based on the original laser data, and obtain semantic landmarks based on the collected environmental images.

[0029] It should be noted that two environmental perception tasks are executed in parallel: visual semantic landmark recognition and environmental point cloud data acquisition. The visual semantic landmark recognition task involves acquiring environmental images using a forward-looking camera (monocular or binocular), identifying static landmarks with spatial stability and semantic uniqueness. Static landmarks include traffic signs (speed limit signs, stop signs, directional arrows), road markings (lane lines, stop lines, zebra crossings), streetlights, traffic signal poles, utility poles, building corners, house numbers, billboards, etc. The Perspective-n-Point (PnP) algorithm, combined with in-camera ground priors (e.g., landmarks located on the road surface), is used to solve for the 3D coordinates of the static landmarks in the camera coordinate system. Alternatively, feature points can be extracted and triangulated using monocular SLAM (e.g., ORB-SLAM3) to generate a sparse 3D map, which is then associated with semantic labels. Each semantic landmark is assigned a semantic category label (e.g., "stop_sign_01", "pole_03") and a confidence score (determined by detection probability and geometric consistency).

[0030] The environmental point cloud data acquisition task includes using LiDAR (such as 16-line, 32-line or solid-state radar) to acquire 360° environmental point clouds, removing dynamic objects (such as vehicles and pedestrians) based on RANSAC ground segmentation and DBSCAN clustering, and extracting the geometric contour features (such as width and height) of static environmental structures (curb, wall, column, traffic sign reflector, etc.) by combining velocity filtering to generate structured environmental point cloud data.

[0031] S103. Based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data, construct a factor graph containing state nodes and constraint factors.

[0032] It's important to note that factor graphs, using a unified and visual framework, integrate multi-source heterogeneous sensor data (related positioning data, semantic information, and geometric information) and their uncertainties, transforming the complex fusion positioning problem into a clear graphical model optimization problem. A factor graph is a bipartite graph containing state nodes and constraint factors. State nodes represent the variables to be optimized, i.e., the vehicle's pose in the discrete time series t0, t1, ..., tn, typically including its position (X, Y, Z) in the global coordinate system, its attitude (represented by quaternions), and its linear velocity. Constraint factors represent the constraints imposed on the state nodes by different sensor observation data. A multimodal factor graph is constructed based on related positioning data, semantic landmarks, and environmental point cloud data to jointly optimize the vehicle trajectory and landmark positions. With each new frame of data, a new pose node is added; if a semantic landmark is detected, a corresponding landmark node (for the first observation) or observation factor (for repeated observations) is added.

[0033] S104. Solve the factor graph to obtain the vehicle fusion localization result and the filtered semantic landmark detection set.

[0034] It should be noted that a nonlinear least squares-based optimization algorithm (such as the Gauss-Newton method or the Levenberg-Marquardt method) is used to solve the constructed factor graph. The optimization objective is to find a set of optimal state nodes that minimizes the weighted sum of squared residuals of all constraint factors. A factor graph is constructed based on relevant localization data, semantic landmarks, and environmental point cloud data. A factor graph-based optimization algorithm is then used for fusion to obtain the fused localization result and the semantic landmark detection set. The semantic landmark detection set in step S102 is the initial set containing category labels and confidence levels. The semantic landmark detection set in S103 is a filtered set based on the semantic landmark detection set in S102, retaining only semantic landmarks with confidence levels greater than the target threshold; that is, filtering out some semantic landmarks with low confidence levels. Factor graphs are bipartite graphs that, through bidirectional connections between "state nodes" and "constraint factors," structurally and visually represent the "state to be solved" and the "constraint relationships of multi-source observation data" in vehicle localization. Essentially, they transform the complex multi-source fusion problem into a "graph constraint optimization problem." By converting relevant localization data, semantic landmarks, and radar point clouds into state nodes and various constraint factors, factor graphs not only visualize and trace the constraint relationships of multi-source data but also allow for the complementary use of different factors. Through iterative optimization using factor graph-based algorithms, factor weights can be dynamically adjusted to eliminate outliers, correct historical accumulated errors, and avoid the divergence problems of traditional filtering, laying the foundation for subsequent V2V collaborative correction.

[0035] S105. Using vehicle-to-vehicle communication technology, the fused positioning results and the semantic landmark detection set are broadcast to nearby vehicles.

[0036] It should be noted that the vehicle utilizes its C-V2X communication module to periodically or event-triggeredly broadcast its own fused positioning results (such as global coordinates, speed, and heading) obtained in S104, along with a filtered semantic landmark detection set (including landmark category, vehicle coordinate system coordinates, and confidence level), according to a specific message format and communication protocol (such as SAE J2735). The V2V communication in this application is based on C-V2X cellular vehicle-to-everything (V2X) technology, with core advantages in low latency, wide coverage, and high reliability. Vehicle-to-vehicle communication via V2V overcomes the limitations of single-vehicle perception. On one hand, the fused positioning results allow nearby vehicles to quickly obtain references to their own motion state, while the semantic landmark detection set provides environmental anchors with category and confidence levels, creating conditions for cross-vehicle feature matching. On the other hand, compared to transmitting only single-location data, the combination of both allows nearby vehicles to more comprehensively assess data reliability, avoiding collaborative failures caused by single-vehicle positioning deviations. The data volume is small and will not consume too much C-V2X communication bandwidth. It can be transmitted stably in scenarios with high traffic volume, providing key data support for subsequent timestamp verification, semantic consistency comparison and positioning deviation correction.

[0037] S106. After receiving the collaborative data broadcast by the neighboring vehicles, perform time synchronization verification and semantic consistency comparison on the collaborative data between the vehicle and the neighboring vehicles, and retain the semantically consistent collaborative data; the collaborative data includes the fusion positioning results and semantic landmark detection set of the neighboring vehicles.

[0038] It should be noted that after receiving collaborative data broadcast by neighboring vehicles, the vehicle first performs time synchronization verification, comparing the C-V2X timestamp in the data packet with the vehicle's system time to ensure that the data from both vehicles are basically aligned in time, avoiding feature matching errors due to communication delays. Next, semantic consistency comparison is performed, checking whether the category labels of landmarks in the semantic landmark detection sets of the vehicle and neighboring vehicles are consistent. For example, if the vehicle observes a "stop line" landmark and a neighboring vehicle also reports a "stop line" landmark, then the two are considered semantically consistent, and this data is retained as valid collaborative data; otherwise, it is discarded. Specifically, after receiving broadcast information, the vehicle first verifies the C-V2X communication timestamps of the two vehicles, compares the semantic landmark category recognition results of the two vehicles for the same area, retains semantically consistent collaborative data, and from the collaborative data, filters out semantic landmarks that overlap with the vehicle's own semantic landmark detection set through coordinate transformation and feature comparison, determining the observation deviation of the two vehicles for that landmark location. Determining the observation bias between the two vehicles regarding the landmark location involves: the semantic landmark location of vehicle A is (x1, y1), and the semantic landmark location of vehicle B is (x2, y2). Transforming the semantic landmark location of vehicle B (x2, y2) into the local coordinate system of vehicle A yields (x2', y2'). Comparing the landmark coordinates observed by vehicle A itself with the transformed landmark coordinates (x2', y2') of vehicle B, the observation bias is determined to be equal to... If observation bias exists, the coordinates of vehicles with high landmark confidence are used as a benchmark, and the vehicle's position in the local coordinate system is corrected using the least squares method. Timestamp verification ensures spatiotemporal alignment of data, avoiding matching bias caused by time delays; semantic landmark categories are compared to retain consistent data and eliminate erroneous coordination sources caused by misidentification. Overlapping landmarks are filtered to accurately locate observation biases. Using high-confidence landmarks as a benchmark and correcting with the least squares method avoids the propagation of errors from low-confidence data and reduces the impact of noise from individual landmarks through mathematical optimization. By filtering interference layer by layer and focusing on reliable data, the positioning accuracy of the vehicle's local coordinate system can be effectively improved.

[0039] S107. From the semantically consistent collaborative data, target semantic landmarks that overlap with the vehicle semantic landmark detection set are selected through coordinate transformation and feature comparison, and the observation deviation of the vehicle and the neighboring vehicles on the target semantic landmarks is calculated.

[0040] It should be noted that the target semantic landmark is the semantic landmark jointly identified by the vehicle and neighboring vehicles based on their respective perception data, after spatiotemporal alignment and feature verification, representing the same physical entity in the environment. Specifically, the target semantic landmark satisfies semantic consistency (both the vehicle and neighboring vehicles recognize the landmark and assign it the same semantic category label), spatial overlap (after coordinate transformation, both the vehicle and neighboring vehicles observe the same spatial location), and feature identity (identical geometric contours and visual textures; that is, through radar point cloud data comparison, it is confirmed that the landmarks observed by the two vehicles have the same or extremely similar geometric shapes and sizes, and through camera data comparison, it is confirmed that the landmark image blocks captured by the two vehicles are highly similar in visual features such as texture and color). Based on semantically consistent collaborative data, the landmarks of neighboring vehicles are first unified to the vehicle's coordinate system through coordinate transformation. Then, spatial distance and feature similarity are combined for matching to identify overlapping target semantic landmarks. Finally, the observation deviation of the target semantic landmark between the vehicle and neighboring vehicles is quantitatively calculated.

[0041] S108. When the observation deviation exceeds a preset threshold, the position of the vehicle in the local coordinate system is corrected by least squares method based on the semantic landmark coordinates with the highest confidence.

[0042] It should be noted that correction is initiated when the calculated observation bias indicates a systematic shift in the positioning of the vehicle and neighboring vehicles. The confidence levels of the vehicle and neighboring vehicles regarding the target semantic landmarks are compared. The landmark coordinates with the higher confidence level are used as the baseline ground truth, ensuring that the correction process relies on more reliable observations. All selected target semantic landmarks and their confidence levels are included in the calculation. A least-squares optimization problem is constructed, aiming to find an optimal position correction amount that minimizes the weighted sum of squared distances between the vehicle's observed coordinates of all target semantic landmarks and the baseline coordinates after correction. The weights used in calculating the weighted sum of squared distances are the confidence levels of each target semantic landmark. The calculated optimal correction amount is applied to the vehicle's current local coordinate system position estimate to complete the correction.

[0043] In summary, the vehicle positioning method based on intelligent connected vehicles provided in this invention firstly utilizes factor graph deep fusion of positioning-related data, semantic landmarks obtained from visual sensors, and environmental point cloud data acquired by radar. In scenarios where a single sensor fails, such as weak positioning signals or visual interference, other sensors can instantly fill the gap, ensuring the continuity and accuracy of positioning and demonstrating strong environmental robustness. Furthermore, leveraging V2X communication, the vehicle can acquire environmental information beyond its own sensor line of sight (such as traffic lights obscured by large vehicles ahead, road conditions after curves), not only improving positioning reliability but also significantly enhancing the intelligent connected vehicle's global perception capability in complex traffic scenarios, providing more comprehensive information support for high-level autonomous driving decisions. Moreover, through a triple filtering mechanism of "time verification - semantic comparison - feature matching," the quality of collaborative data is ensured. Then, least squares correction is performed based on high-confidence observations, effectively identifying and correcting inherent biases and cumulative drift that are difficult to avoid in single-vehicle sensor fusion, fundamentally improving the long-term stability of the system.

[0044] In some embodiments of the present invention, such as Figure 2 As shown, step S102 includes: S201. The environmental image is processed using the YOLOv8 model to output the semantic landmark; the semantic landmark includes category label, pixel coordinates and confidence score.

[0045] It should be noted that after the vehicle-mounted camera captures the raw environmental image (such as an RGB image), it is input into a pre-trained YOLOv8 deep learning neural network model. The core of the YOLOv8 model consists of a feature extraction network (such as CSPDarknet) and a detection head. The feature extraction network performs multiple downsampling and convolution operations on the input image to extract multi-level, deep feature maps. The detection head performs calculations on these feature maps, predicting the category label of the object contained within each preset anchor box (e.g., "traffic light", "stop sign", "curb"), the pixel coordinates of the object's bounding box (usually the center point x, y, width w, height h), and a confidence score (between 0 and 1). The confidence score comprehensively reflects the model's confidence that the target exists within the bounding box and is correctly classified. By recognizing semantic landmarks and transforming coordinates through the YOLOv8 model, and extracting static obstacle features through millimeter-wave radar combined with DBSCAN clustering, high-quality and complementary environmental perception data is provided for localization. YOLOv8 can accurately output semantic landmark categories, confidence levels, and vehicle coordinates, clearly defining landmark attributes, avoiding misjudgments based on similar features, and providing semantic anchor information for localization.

[0046] S202. Using camera calibration parameters, the pixel coordinates are transformed into three-dimensional coordinates in the vehicle local coordinate system.

[0047] It's important to note that mapping the 2D pixel positions in an image to the 3D physical space surrounding the vehicle requires first obtaining the camera calibration parameters, which include intrinsic and extrinsic matrices. The intrinsic matrix describes the camera's internal geometry and optical characteristics, including the focal length (fx, fy) and principal point coordinates (cx, cy). It's used to transform the pixel coordinate system to the camera coordinate system. The extrinsic matrix describes the relative relationship between the camera and vehicle coordinate systems, including a rotation matrix and a translation vector.

[0048] The transformation process typically employs the principle of inverse perspective projection transformation. For the pixel coordinates (u, v) of a semantic landmark, its normalized coordinates in the camera coordinate system are first calculated using the intrinsic parameter matrix. Then, combined with the estimated depth information of the semantic landmark (obtained through monocular scale estimation, prior landmark size, or fusion with radar), and using the extrinsic parameter matrix, it is transformed into the vehicle's local coordinate system, resulting in a three-dimensional coordinate (x, y, z), where x typically points forward, y points to the left, and z points upward.

[0049] S203. Based on the transformed 3D coordinates, category labels, and confidence scores, a structured semantic landmark detection set is generated.

[0050] It should be noted that the outputs of S201 and S202 are integrated to create a standardized, structured data entry for each successfully identified and converted semantic landmark. Each data entry typically contains the following fields: a unique identifier, a category label (e.g., "speed_limit60"), three-dimensional coordinates (x, y, z) in the vehicle coordinate system, and a confidence score. The collection of all these entries constitutes a structured semantic landmark detection set. This semantic landmark detection set can be directly used in subsequent factor graph optimization processes.

[0051] S204. Obtain the raw laser data generated by the radar, wherein the raw laser data includes distance and angle information of obstacles in the environment.

[0052] It should be noted that millimeter-wave radar provides direct and accurate distance measurement, is unaffected by lighting conditions, and can penetrate certain rain, fog, and dust, providing reliable environmental geometry information. Millimeter-wave radar emits electromagnetic waves and receives reflected signals, generating raw laser data containing obstacle distances (straight-line distance between the radar and the target) and angles (horizontal angles of the target relative to the radar's central axis). Since the raw point cloud contains both dynamic targets (such as other vehicles and pedestrians) and static targets (such as streetlights and guardrails), clustering is performed using the DBSCAN algorithm. Based on Euclidean distance, spatially adjacent points are grouped into the same cluster. Dynamic clusters whose positions change over time are filtered out by considering the vehicle's own motion state (such as speed and heading), retaining the static clusters with stable positions. Geometric contour extraction is performed on the static clusters to obtain feature data of static obstacles. Finally, combined with the raw distance and angle information, structured environmental point cloud data is formed.

[0053] S205. The original laser data is processed using the DBSCAN clustering algorithm to distinguish between dynamic and static obstacles.

[0054] It should be noted that DBSCAN is a density-based spatial clustering algorithm that does not require pre-specifying the number of clusters, can discover clusters of arbitrary shapes, and can identify noise points (outliers). First, key parameters are set (including the neighborhood radius eps and the minimum number of points minPts). The algorithm traverses every point in the original laser data, searching for all points within the neighborhood radius eps. If a point's neighborhood contains at least minPts points, it is considered a core point, and a cluster is formed starting from this core point. All reachable core points and boundary points are grouped into the same cluster using density-connected clusters. After clustering, points belonging to large, dense clusters typically correspond to static obstacles (such as guardrails, walls, and lampposts), while sparse, small clusters, or noise points that cannot be classified into any cluster, correspond to dynamic objects (such as vehicles and pedestrians). Further verification can be performed using Doppler velocity information provided by radar points. Clusters corresponding to points with significantly non-zero velocities are identified as dynamic obstacles and filtered out, thus completing the distinction between dynamic and static obstacles.

[0055] S206. Extract the geometric contour features of the point cloud clusters corresponding to the static obstacles, and generate structured environmental point cloud data based on the geometric contour features.

[0056] It should be noted that for each static obstacle separated in S205, geometric features are extracted from the corresponding point cloud cluster. This involves calculating the 3D bounding box (minimum enclosing cuboid) of the point cloud cluster corresponding to the static obstacle, obtaining its geometric contour features (including but not limited to center point coordinates and bounding box parameters, such as width, height, average distance, and angle information). Alternatively, planar or surface fitting is performed on the points within the point cloud cluster corresponding to the static obstacle to obtain its main geometric contour features. Finally, these extracted geometric contour features are integrated into structured data entries to form structured environmental point cloud data for localization.

[0057] In this embodiment, rich and interpretable semantic landmarks are provided through vision (YOLOv8), while accurate and interference-resistant geometric contours are provided through radar (DBSCAN clustering). This dual perception of "semantics + geometry" solves the problems of insufficient accuracy and susceptibility to environmental interference in geometric measurements by pure vision methods, and also compensates for the shortcomings of pure radar methods in lacking semantic information, difficulty in high-level data association, and mismatch discrimination. This achieves deep fusion and complementary advantages of semantic and geometric information. Furthermore, through confidence output and DBSCAN dynamic and static filtering, the system can automatically filter out high-reliability environmental features. In factor graph optimization, high-confidence landmarks and static geometric features can be given higher weights, while low-quality or dynamic interference data is effectively suppressed or eliminated, thus laying a high-quality data foundation for subsequent fusion positioning and significantly improving the quality and reliability of environmental perception data. Furthermore, in well-lit and visually clear scenarios, high-precision semantic landmark factors play a dominant role, while in scenarios such as nighttime, rain, and fog where visual degradation occurs but radar is unaffected, reliable radar point cloud geometric factors become the main constraint for positioning. The collaboration between these two systems ensures that the system can obtain stable and continuous environmental feature inputs under various complex and extreme environments, thereby guaranteeing the high robustness of the positioning system. Furthermore, the process of generating structured semantic landmark detection sets and structured environmental point cloud data from raw data forms a standardized and modular perception data stream. This also allows the visual perception module and radar perception module to be developed and optimized independently, and are easily integrated into larger positioning systems, enhancing the system's engineering feasibility and scalability.

[0058] In some embodiments of the present invention, such as Figure 3 As shown, the relevant positioning data includes global navigation satellite system data, inertial measurement data, and wheel odometer data. Step S103 includes: S301. Define the vehicle's pose in a discrete time series as a state node.

[0059] It is important to note that acquiring relevant vehicle positioning data is the core foundation for ensuring the positioning accuracy and stability of intelligent connected vehicles. On the one hand, it provides a benchmark for subsequent multi-source fusion; on the other hand, the introduction of relevant positioning data reduces reliance on a single sensor. For example, when visual semantic landmark recognition is affected by lighting interference or millimeter-wave radar point cloud resolution is insufficient, the stability of relevant positioning data can prevent significant jumps in positioning results, ensuring the initial accuracy and robustness of multi-source data fusion, and laying a crucial foundation for final positioning. Relevant vehicle positioning data includes GNSS data, IMU data, and wheel odometer data. GNSS data consists of pseudorange, carrier phase, and preliminary positioning results acquired through GPS or BeiDou receivers; IMU data consists of angular velocity and acceleration data acquired through inertial measurement units; and wheel odometer data consists of travel distance and wheel speed data acquired through wheel speed sensors. Specifically, GNSS data relies on GPS or BeiDou satellite navigation systems, capturing satellite signals through receivers. It not only outputs raw observation information such as pseudorange and carrier phase but also calculates preliminary positioning results (latitude, longitude, and altitude), providing global spatial coordinate reference for the vehicle. IMU data is collected by the gyroscopes and accelerometers in the inertial measurement unit, outputting in real time the vehicle's angular velocity (rotational speed around the X, Y, and Z axes, reflecting steering and tilting) and acceleration (linear acceleration along the X, Y, and Z axes, reflecting acceleration, deceleration, and bumps) in three-dimensional space, enabling high-frequency capture of instantaneous vehicle motion changes. Wheel odometer data, obtained through wheel speed sensors mounted on the wheels, calculates the distance traveled based on the number of wheel rotations and tire circumference, while directly acquiring wheel speed (the speed corresponding to the number of wheel rotations per unit time), accurately reflecting the vehicle's cumulative longitudinal mileage and real-time speed. Acquiring GNSS, IMU, and wheel odometer data provides multi-dimensional and highly reliable underlying support for intelligent connected vehicle positioning, effectively compensating for the limitations of single data sources. GNSS data can provide initial position anchors in open environments, IMU data can fill positioning gaps when GNSS signals are interrupted (such as in tunnels or urban canyons), and wheel odometer data can assist in correcting IMU cumulative drift. The three work together to provide continuous, high-frequency motion state support for positioning. GNSS provides global absolute coordinates, IMU provides high-frequency relative motion, and wheel odometer provides planar motion reference. When the GNSS signal is interrupted, positioning is maintained by the IMU and odometer. GNSS corrects the accumulated error between the IMU and odometer, and the odometer constrains the vertical error of the IMU.

[0060] Factor graph optimization is a probabilistic graphical model used to estimate the state of a system. The state node to be estimated is the vehicle's pose changing over time. Therefore, the entire timeline is discretized into a series of key moments t0, t1, t2, ..., tn. At each moment tk, a state node Xk is created. Each state node Xk contains a complete vehicle state vector, typically including at least: the vehicle's position pk, attitude qk, and linear velocity Vk. Position represents the vehicle's three-dimensional coordinates in a global coordinate system (e.g., WGS-84 or UTM coordinate system). Attitude describes the vehicle's rotation (i.e., orientation) relative to the global coordinate system, and is typically represented by a quaternion [q_w, q_x, q_y, q_z], which avoids gimbal lock problems compared to Euler angles. Linear velocity represents the vehicle's three-dimensional velocity vector in the global coordinate system. Alternatively, the zero biases b_a and b_g of the IMU's accelerometer and gyroscope can be added as part of the state node Xk to correct IMU errors.

[0061] S302. Establish multiple constraint factors connecting the state nodes; the multiple constraint factors include GNSS factors obtained from the Global Navigation Satellite System data, IMU factors obtained from the inertial measurement data, odometer factors obtained from the wheel odometer data, semantic landmark factors obtained from the semantic landmarks, and radar point cloud factors obtained from the environmental point cloud data.

[0062] It's important to note that constraint factors are the essence of factor graphs. Constraint factors transform sensor observation data into mathematical constraints on state nodes. This example uses GNSS factors based on Global Navigation Satellite System (GNSS) data as observations. When a vehicle receives available GNSS signals, the GNSS receiver outputs a preliminary global position calculation result. At this point, a GNSS factor is added to the state node Xk corresponding to the current time tk in the factor graph. Mathematically, the GNSS factor generates a residual term. r gnss : ; p k For state variables, p gnss This is the result of the global position calculation. This residual term... r gnss The estimated position in the state node was measured. p k Locations directly observed by GNSS p gnssThe difference between them. The optimizer's goal is to minimize this residual term. The GNSS factor provides a global absolute position constraint, acting like an "anchor point" to pull the entire trajectory toward the true absolute coordinates on Earth, preventing the trajectory from drifting globally due to accumulated errors in the inertial navigation system.

[0063] IMU factors are generated through pre-integration of inertial measurement data. The acquisition frequency of IMU data (angular velocity ω and acceleration a) is extremely high (typically 100-500Hz). Creating a state node for each IMU data point would be computationally prohibitive. Pre-integration solves this problem. Between two adjacent state nodes Xi and Xj, all IMU measurements within this time period are integrated in the local coordinate system to obtain relative motion increments ΔR_{ij}, Δv_{ij}, and Δp_{ij} (representing relative rotation, velocity change, and position change, respectively). Then, based on this pre-integrated quantity, an IMU factor connecting Xi and Xj is constructed. The IMU factor describes the difference between Xj predicted from Xi and Xj generated through IMU pre-integration. The IMU factor provides high-frequency, continuous kinematic constraints, which are crucial during GNSS signal interruptions (such as in tunnels), enabling the inference of pose changes based on the vehicle's own motion and ensuring positioning continuity. Simultaneously, it forces the entire motion trajectory to conform to physical dynamics, making it smoother.

[0064] An odometer factor is constructed based on wheel odometer data. The wheel odometer provides displacement information of the vehicle in a two-dimensional plane. The relative translation Δs and rotation Δθ (heading angle change) of the vehicle within a time interval Δt can be calculated using wheel speed pulses. Based on this, an odometer factor connecting adjacent state nodes Xi and Xj is constructed. The odometer factor describes the difference between the planar motion of Xj predicted from Xi and the planar motion observed by the odometer. The odometer factor provides accurate planar motion constraints and is particularly effective in constraining the two-dimensional motion of the vehicle on the road surface. It can very accurately estimate longitudinal displacement and effectively suppress the cumulative drift of the IMU in the horizontal direction (especially the heading error caused by Z-axis gyroscope drift), forming a strong complement to the IMU.

[0065] The semantic landmark factor is defined as the coordinates of the semantic landmark in the vehicle coordinate system and its prior position in the global coordinate system as the observation value. Specifically, when the vehicle observes a semantic landmark at time tk, its coordinates pl in the vehicle coordinate system are obtained through transformation S202. vehicle Meanwhile, assume there exists a prior global high-precision map that stores the true coordinates (pl) of this landmark (e.g., a specific "stop line" or "traffic sign") in the global coordinate system. globalAt this point, a semantic landmark factor can be constructed, connecting the state node Xk at the moment the landmark is observed. Its residual function is: ; Here, T(Xk) is a transformation function that transforms a point from the vehicle coordinate system to the global coordinate system. This transformation is determined by the position pk and pose qk corresponding to the state node Xk. Semantic landmark factors provide sparse but high-precision absolute pose correction, aligning the vehicle's pose with stable and unique semantic features in the environment. When GNSS fails, semantic landmark factors can replace GNSS as absolute anchors.

[0066] Radar point cloud factors are calculated using the geometric contours of static obstacles in environmental point cloud data as observed values. Radar point cloud factors utilize the geometric contour features of point cloud clusters corresponding to static obstacles generated by S206. For example, if a vehicle observes a wall at time tk, the plane equation fitted by its radar point cloud is n·x+d=0 (in the vehicle coordinate system). Simultaneously, the global coordinate system equation of this wall is also known in the high-precision map. The residual of the radar point cloud factor can be defined as the error (e.g., the distance from a point to a surface) between the locally observed plane (estimated by state node Xk) and the prior plane in the map after transforming it to the global coordinate system. Radar point cloud factors do not rely on specific, identifiable semantic landmarks but utilize ubiquitous macroscopic geometric structures in the environment (such as walls, guardrails, and curbs). Due to the radar's immunity to illumination and weather conditions, radar point cloud factors are particularly powerful in visually degraded scenarios, providing robust and interference-resistant geometric support for the positioning system.

[0067] In summary, state nodes represent the vehicle state to be solved, and constraint factors represent the observation constraints on the state from different data sources. State nodes define the vehicle's state in a discrete time series, including its position in the global coordinate system, its attitude represented by quaternions, and its linear velocity. GNSS factors use preliminary GNSS positioning results as observations to construct absolute position constraints and anchor the global coordinate system. IMU factors construct kinematic constraints on adjacent time-series states through pre-integration of IMU angular velocity and acceleration, reflecting the smoothness of continuous vehicle motion. Semantic landmark factors use landmark coordinates in the vehicle coordinate system as observations, combined with the prior positions of landmarks in the global coordinate system, to construct constraints. Radar point cloud factors construct geometric constraints based on the static obstacle geometric contours obtained from DBSCAN clustering, leveraging radar's anti-interference characteristics to strengthen constraints on the static environment in visual failure scenarios. A factor graph is constructed, using state nodes to clarify the core states to be solved, such as the vehicle's global position, attitude, and linear velocity in discrete time, providing a unified focus target for multi-source data fusion. By employing four types of constraint factors to form complementary support and multi-dimensional constraints working synergistically, the fusion results achieve both high accuracy and strong adaptability, significantly improving the robustness of positioning in complex scenarios. The construction of the factor graph involves transforming the "vehicle state to be solved" and "observation constraints from various data sources" into "state nodes" and "constraint factor nodes" of the factor graph, and then connecting these nodes to form a complete constraint relationship network. State nodes represent the vehicle state to be solved (position, vehicle attitude, linear velocity), which is also the target variable for subsequent optimization. The GNSS factor uses the preliminary GNSS positioning results as observations to construct absolute position constraints, anchoring the global coordinates. The IMU factor constructs kinematic constraints for adjacent time-series states through pre-integration of IMU angular velocity and acceleration, reflecting the smoothness of continuous vehicle motion. The semantic landmark factor uses the landmark coordinates in the vehicle coordinate system as observations, combined with the prior position of the landmark in the global coordinate system, to construct constraints. The radar point cloud factor constructs geometric constraints based on the static obstacle geometric contours obtained from DBSCAN clustering, leveraging the radar's anti-interference characteristics to strengthen the constraints on the static environment in visual failure scenarios. The aforementioned state nodes and constraint factor nodes are connected according to time and constraint logic (each state node is connected to a GNSS factor, a semantic landmark factor, and a radar point cloud factor, and adjacent state nodes are connected to an IMU factor) to form the final factor graph.

[0068] In this embodiment, by constructing a factor graph including GNSS factors, IMU factors, odometer factors, semantic landmark factors, and radar point cloud factors, five sensor data with different characteristics and complementary advantages and disadvantages (the absoluteness of GNSS, the continuity of IMU, the planar accuracy of odometer, the semantic nature of vision, and the geometric robustness of radar) can be unified under a mathematical framework. Driven by the optimization objective, they work together to form a three-dimensional constraint network, a multi-layered and highly robust spatial constraint network. This ensures that the vehicle's pose is firmly locked on the real trajectory from multiple dimensions and in multiple ways. The temporary failure or performance degradation of any single sensor will not cause the positioning system to collapse, and the system has extremely high robustness.

[0069] In some embodiments of the present invention, such as Figure 4 As shown, step S104 includes: S401. Construct the first optimization function with the objective of minimizing the weighted sum of squared residuals of all constraint factors: ; in, Let X be the first optimization function, M be the set of all state nodes, ri be the residual vector of the i-th constraint factor, and Wi be the weight matrix of the i-th constraint factor. It's important to note that for each constraint factor (GNSS factor, IMU factor, odometry factor, semantic landmark factor, and radar point cloud factor), a mathematical function needs to be defined to calculate its residual vector *ri*. The residual vector essentially represents the difference between the predicted and observed values. For all four types of constraint factors, both observed and predicted values ​​need to be defined; the residual vector is the difference between these two. The goal of the optimization algorithm is to adjust the state *X* to minimize the weighted sum of squared residuals for all constraint factors. Taking the GNSS factor as an example, its residual vector... The residual vectors of other constraint factors are also defined in the same way. For example, the IMU factor is the difference between the relative motion predicted by the state node and the IMU pre-integrated observation, and the semantic landmark factor is the difference between the predicted position of the landmark in the global coordinate system and its prior map position.

[0070] The weight matrix Wi is crucial for measuring the reliability or uncertainty of an observation. The lower the noise (the more reliable) an observation is, the larger the inverse Wi of its covariance matrix Σi, meaning the greater the weight of that constraint factor in the optimization. For example, in open areas with high-quality GNSS signals, the weight matrix will be assigned a larger value; while in urban canyons with high GNSS noise, the weight matrix value will be correspondingly smaller. Similarly, a semantic landmark with high confidence (e.g., 0.95) will have a much higher weight matrix value than a semantic landmark with low confidence (e.g., 0.6).

[0071] The objective function J(X) is the weighted sum of squares of the residuals of all constraint factors. The goal of the optimization problem is to find a set of optimal state nodes that minimizes the objective function J(X).

[0072] S402. The first optimization function is solved using the Gauss-Newton iterative algorithm to obtain the optimal vehicle pose sequence as the fusion localization result, and a set of semantic landmark detections with a confidence level higher than the target threshold after filtering is output.

[0073] It should be noted that the objective function is gradually reduced by iteratively updating the state vector using the Gauss-Newton method until convergence, thus obtaining the optimal fused positioning result. The residual vector represents the difference between the predicted and observed values, while the state vector represents all the unknown variables to be solved in the factor graph optimization problem. The state vector serves as both the input and output of the optimization algorithm. The optimization process involves continuously fine-tuning each value in the state vector to find an optimal state vector X that minimizes the weighted sum of squared residuals of all constraint factors. Near the current state estimate, a linear function is used to approximate the objective function J(X), and the minimum point of this linear function is found, serving as the starting point for the next iteration. This process is repeated until convergence. This application aims to minimize the weighted sum of errors of all constraint factors, enabling precise quantification of the constraints from GNSS, IMU, semantic landmarks, and environmental point cloud data. High-reliability data is given greater influence through weighting, reducing interference from low-quality data. The iterative solution using the Gauss-Newton method can efficiently converge to the optimal pose sequence, ensuring better positioning accuracy and stability compared to other algorithms, and avoiding the drift problem that easily occurs in traditional fusion methods.

[0074] The Gauss-Newton method is an iterative algorithm for solving nonlinear least squares problems. It begins with an initial state estimate (e.g., the first GNSS observation or the optimization result from the previous time step). Linearization is performed at the current state estimate point Xop, by performing a first-order Taylor expansion on each residual function ri(X), approximating it as a linear function. This process requires calculating the derivative of the residual function with respect to the state variables, i.e., the Jacobian matrix Ji. Through linearization, the original nonlinear problem is transformed into a linear least squares problem with respect to the state increment δX, resulting in a least squares linear equation. Solving this least squares linear equation yields the state increment δX, and the state estimate is then updated. This process of linearization, solving, and updating is repeated until the norm of the state increment δX is less than a preset convergence threshold, or the maximum number of iterations is reached. The resulting state sequence is the optimal vehicle pose sequence, which is the fused localization result for the vehicle. After the optimization process is completed, the system will traverse all semantic landmarks that participated in the optimization. Semantic landmarks with low confidence are considered unreliable observations and are removed. Only those semantic landmarks with confidence above the preset target threshold (e.g., 0.8) are retained to form the final output of a high-quality semantic landmark detection set.

[0075] In this embodiment, by minimizing the weighted residual sum of all sensor constraints, a vehicle trajectory that is most compatible with all observation data in a maximum likelihood sense is found. This dynamically corrects the accumulated errors of the IMU and odometer, smooths GNSS jumps, and uses environmental features to suppress drift, ultimately outputting a fused positioning result with higher accuracy in the global coordinate system and smoother, more reliable local trajectory. Furthermore, intelligent data fusion is achieved through weight matrices and confidence thresholds, allowing more reliable data (high weight, high confidence) to have greater weight in decision-making, while automatically weakening or eliminating the influence of unreliable data (low weight, low confidence). This makes the system highly tolerant to sensor noise, temporary interference, and misidentification, significantly improving overall robustness. Furthermore, the output fused localization result serves as the baseline for calculating collaborative deviations by the vehicle. Its accuracy directly determines the upper limit of collaborative correction. The filtered high-confidence semantic landmark detection set provides reliable evidence for feature matching with neighboring vehicles. This step ensures that the vehicle provides the most essential and reliable information at the start of collaboration, significantly reducing the risk of mismatches or correction failures due to vehicle data quality issues during subsequent vehicle-to-vehicle collaboration, thus guaranteeing the reliability of the entire collaborative localization chain. Moreover, compared to Kalman filter-based sequence processing methods, factor graph optimization, as a batch processing method, can utilize all information (historical and future) across the entire time series to correct current and historical poses, more effectively correcting historical accumulated errors and improving localization accuracy and reliability.

[0076] In some embodiments of the present invention, such as Figure 5 As shown, step S107 includes: S501. Based on the fusion positioning results of the vehicle and the neighboring vehicles, calculate the relative pose of the two vehicles and construct the corresponding coordinate transformation matrix.

[0077] It should be noted that both the vehicle and neighboring vehicles have obtained high-precision fused localization results in a global coordinate system (e.g., UTM coordinate system) via S104. The fused localization results include the vehicle's global position (Xego, Yego, Zego) and attitude (heading angle θego, usually derived from quaternions or rotation matrices). Similarly, the fused localization results for neighboring vehicles are (Xadj, Yadj, Zadj, θadj). The relative positions between the vehicle and neighboring vehicles are then calculated. , And calculate the relative heading angle between the vehicle and neighboring vehicles. Relative pose includes relative position ΔX, ΔY and relative heading angle Δθ.

[0078] To combine the points in the coordinate system of nearby vehicles P adj Transform to vehicle coordinate system P ego This requires constructing a two-dimensional or three-dimensional coordinate transformation matrix. Under the assumption of a two-dimensional plane, the coordinate transformation matrix... It can be represented as: ; Its inverse matrix is ​​the matrix required to transform the coordinates of neighboring vehicles to the vehicle's coordinate system.

[0079] S502. Using the coordinate transformation matrix, the coordinates of the semantic landmarks in the neighboring vehicle semantic landmark detection set are transformed to the vehicle's local coordinate system to obtain the transformed landmark coordinates.

[0080] It should be noted that for each landmark in the neighboring vehicle semantic landmark detection set, its coordinates in the local vehicle coordinate system are taken. P adj =[Xadj, Yadj, 1] T (Homogeneous coordinate form). The coordinate transformation is performed using the following formula: ; The coordinates of the semantic landmark in the vehicle coordinate system were calculated. P adj =[Xadj, Yadj, 1] T Perform this operation on all neighboring vehicle landmarks to generate a transformed set of neighboring vehicle landmarks.

[0081] S503. Calculate the Euclidean distance between the transformed landmark coordinates and the vehicle landmark coordinates.

[0082] It should be noted that for each semantic landmark observed by the vehicle, semantic landmarks with the same semantic category in the transformed set of neighboring vehicle landmarks are considered as a candidate landmark pair. For each such candidate landmark pair, the Euclidean distance between them in the vehicle's coordinate system is calculated based on the transformed landmark coordinates and the vehicle's landmark coordinates. ; Determine if the Euclidean distance is less than a first preset threshold (e.g., 5 meters). The first preset threshold comprehensively considers the vehicle's positioning error, coordinate transformation error, and sensor observation noise. Candidate landmark pairs that meet the condition of an Euclidean distance less than the first preset threshold are considered to be physically close and pass the initial spatial screening.

[0083] S504. Calculate the radar point cloud feature error and visual texture similarity between the vehicle and the neighboring vehicles for the same candidate landmark pair.

[0084] It should be noted that for candidate landmark pairs that pass the initial spatial screening, a more refined feature-level comparison is performed. First, the radar point cloud feature error is calculated, which involves extracting the local radar point cloud clusters associated with the vehicle and neighboring vehicles for that semantic landmark (i.e., the original environmental point cloud data generated in S206 to generate the geometric contour features of the landmark). Using the coordinate transformation matrix in S501, the point clouds of neighboring vehicles are transformed to the vehicle's coordinate system. Variations of the iterative nearest-point algorithm or direct calculation of the Hausdorff distance or Chamfer distance between the point clouds can be used to quantify the geometric differences between the two local radar point cloud clusters. The calculated average distance or matching error is the radar point cloud feature error.

[0085] Additionally, visual texture similarity needs to be calculated, which involves cropping image patches containing the semantic landmark from the original images of the vehicle and neighboring vehicles. These image patches are then preprocessed (e.g., scaled, converted to grayscale). Feature extraction algorithms (such as traditional SIFT, ORB, or more advanced deep learning feature extractors) are used to extract feature descriptors for the image patches. The similarity between two feature descriptors (e.g., cosine similarity, the reciprocal of the Euclidean distance) is the visual texture similarity; a higher similarity value indicates greater similarity between the two image patches.

[0086] S505. When the Euclidean distance is less than a first preset threshold, the radar point cloud feature error is less than a second preset threshold, and the visual texture similarity is greater than a third preset threshold, the candidate landmark pair is determined to be the target semantic landmark.

[0087] It should be noted that the three thresholds are: a first preset threshold (spatial distance threshold, such as 5 meters), a second preset threshold (radar point cloud error threshold, such as 0.3 meters), and a third preset threshold (visual texture similarity threshold, such as 0.8). Only when a candidate landmark pair meets all three conditions is it finally recognized as a valid target semantic landmark (i.e., overlapping semantic landmark) and enters the subsequent deviation calculation process. If any condition is not met, it is considered a mismatch and discarded.

[0088] In summary, overlapping semantic landmarks in the semantic landmark detection set are screened using spatial distance and feature similarity criteria. The spatial distance criterion involves obtaining the relative poses of the two vehicles, constructing a transformation matrix to convert the landmark coordinates of adjacent vehicles to their own local coordinate systems, and calculating the spatial distance to the landmark coordinates of the vehicle using Euclidean distance. If the spatial distance is less than a target threshold, the spatial distance criterion is met. The feature similarity criterion first assesses the consistency of the landmark category labels. If they are consistent, and the radar point cloud error is less than a first target threshold while the visual texture similarity is greater than a second target threshold, the feature similarity criterion is met. If both the spatial distance and similarity criteria are met, the landmarks are determined to be overlapping semantic landmarks. Radar point cloud error: The environmental point cloud data related to the semantic landmarks of vehicles A and B are transformed to the same coordinate system. For each point in the point cloud of vehicle A, the nearest point in vehicle B is found, and the Euclidean distance between them is calculated. The average distance of all pairs of points is the radar point cloud error. Visual texture similarity: Compare image patches containing the landmark taken by two vehicles. After cropping and feature extraction of the image patches, similarity is calculated using a similarity algorithm.

[0089] The process involves acquiring the relative poses of two vehicles via V2V communication, constructing a transformation matrix that includes a rotation matrix and a translation vector. First, the rotation matrix aligns the poses of the two vehicles, then the translation vector compensates for positional differences, transforming the landmark coordinates of adjacent vehicles to their own local coordinate systems. Overlapping semantic landmarks are identified through a dual-condition approach: spatial distance ensures that selected landmarks are physically close, avoiding spatial misalignment and misjudgment; feature similarity excludes incompatible features. This dual-condition approach guarantees both the spatial correlation of overlapping landmarks and the feature matching degree, laying the foundation for accurate calculation of observation bias and reliable position correction. This significantly reduces the risk of mismatches in collaborative localization and improves positioning accuracy.

[0090] In this embodiment, a triple matching mechanism combining spatial location, geometric contour, and visual texture significantly improves the accuracy of vehicle-to-vehicle data association, effectively distinguishing between similar semantic landmarks and identical semantic landmarks. This solves the most challenging data association problem in collaborative perception, avoiding collaborative localization failures or performance degradation caused by mismatches. Furthermore, the triple matching mechanism, through multi-dimensional cross-validation, makes the system highly robust to various interferences and anomalies, ensuring the data quality of the input correction stage. Moreover, the triple matching mechanism can produce a near-error-free set of target semantic landmarks, allowing subsequent least-squares correction to find the optimal solution on this near-error-free set, thereby maximizing the potential of collaborative correction and ensuring that the final improvement in positioning accuracy is genuine, reliable, and accurate.

[0091] In some embodiments of the present invention, such as Figure 6 As shown, step S108 includes: S601. Construct a second optimization function with the objective of minimizing the weighted positional bias: ; in, For the second optimization function, The number of overlapping landmarks selected; The confidence level of the m-th landmark for the baseline vehicle; It is a correction amount, including and Quantity; Let m be the coordinates of the m-th semantic landmark observed by the vehicle in its local coordinate system; The coordinates of the m-th semantic landmark after conversion for adjacent vehicles.

[0092] It should be noted that for each pair of overlapping semantic landmarks, the observation confidence of the target semantic landmark is compared between the vehicle and its neighboring vehicles. The coordinates of the semantic landmark with the higher confidence are determined as the benchmark for the target semantic landmark. That is, in the optimization, the coordinates of the semantic landmark with higher confidence are regarded as reference values ​​that are closer to the true value.

[0093] For each target semantic landmark, its location deviation value is The position deviation value indicates whether there is a correction amount for the vehicle's position. Then the coordinates of all target semantic landmarks observed by the vehicle will be adjusted according to the correction amount. After inverse correction, the coordinates should be aligned with those of the target semantic landmarks observed by neighboring vehicles. The second objective function J is the square of the magnitude of all these residual vectors, expressed as a baseline confidence level w. m Sum the weights. Confidence level w mThe higher the value, the greater the weight, and the greater the impact on the final correction amount.

[0094] S602. Update the position of the vehicle in the local coordinate system according to the optimal position correction obtained by solving the second optimization function.

[0095] It should be noted that the second optimization function J mentioned above is about the correction amount. The quadratic function can be solved directly by taking its derivative and setting it to zero. Expanding the second optimization function J and taking its partial derivatives with respect to δx and δy, respectively, and setting these partial derivatives to zero, yields a system of linear equations for δx and δy. Solving this system of linear equations provides the optimal position correction. ; This refers to the correction value that best eliminates the systematic bias in the observation system between the vehicle and neighboring vehicles under the weighted least squares approach. Assume that before correction, the estimated position of the vehicle in the local coordinate system is... The optimal correction value obtained by solving the problem. If applied to the vehicle's location, then the updated vehicle location... In other words, , The result after the update It is a more precise local position of the vehicle after vehicle-to-vehicle collaborative correction.

[0096] In summary, for the selected overlapping semantic landmarks, the observation deviation between the two vehicles is calculated using the transformed coordinates; the landmark coordinates of the vehicle with high landmark confidence are used as the benchmark, and the objective function is constructed using the least squares method to obtain the optimal correction amount; the position of the vehicle in the local coordinate system is updated according to the optimal correction amount.

[0097] In this embodiment, by establishing a second optimization function and solving for the optimal position correction, the systematic deviation in vehicle positioning can be accurately calculated and eliminated, achieving precise and quantitative elimination of cooperative positioning deviations, resulting in a significant and reliable improvement in positioning accuracy. Furthermore, by increasing the confidence level w... mIntroducing a second optimization function as a weight ensures that the correction process always favors the more reliable perception, effectively avoiding the risk of the vehicle's positioning being skewed by accidental misidentification or low-quality observations from nearby vehicles, and greatly improving the overall robustness and reliability of the cooperative system. Furthermore, by integrating observation information from multiple (K) overlapping semantic landmarks to jointly solve for a correction value, the least squares method itself has the effect of suppressing random noise. Even if the observation of a single landmark contains noise, the statistical effect of multiple landmarks can effectively average out these random errors, thus solving for a more stable and reliable correction value. This is much more robust than relying on a single landmark for correction, leveraging the data fusion advantages of multi-vehicle observations, suppressing noise interference, and constructing a closed loop of "single-vehicle perception, single-vehicle fusion, vehicle-to-vehicle collaboration, and collaborative correction." This allows intelligent connected vehicles to truly evolve from simple information sharing to a collaborative intelligent agent capable of mutual verification and joint improvement, effectively improving the accuracy, reliability, and robustness of positioning under extreme weather conditions.

[0098] In some embodiments of the present invention, after correcting the position of the vehicle in the local coordinate system using the least squares method, the method further includes: S109. Update the factor graph using the updated vehicle positions as the new initial state; S110. Using vehicle-to-vehicle communication technology, the corrected positioning results based on the factor graph and the updated semantic landmark detection set are broadcast.

[0099] It should be noted that: the corrected vehicle position, i.e., the updated vehicle position, is obtained through the least squares method. Afterwards, the system will update the vehicle's location. The feedback is then incorporated into the factor graph optimization framework. Specifically, the initial state of the factor graph optimization process at the current and subsequent time steps is set to the updated vehicle position. This allows subsequent factor graph optimization to be based on this more accurate updated vehicle position. The calculation starts from the baseline, rather than the position where there might be deviations before correction. Alternatively, a strongly constrained prior factor can be added to the current state node in the factor graph; the observed value of this prior factor is the updated vehicle position. This is done by assigning a higher weight matrix (weights) to the vehicle, thereby pulling the optimization result toward this more reliable, collaboratively validated position. Where computational resources allow, it's even possible to base the optimization on the updated vehicle position. The state nodes within a short historical trajectory are locally re-optimized to eliminate the cumulative drift caused by initial errors during that period.

[0100] The updated factor graph optimization results from S109, i.e., the higher-precision fusion localization results, and the corresponding updated semantic landmark detection set, which has been further filtered, are packaged together. The semantic landmark detection set here has been updated based on the vehicle location. The system has been updated, resulting in more accurate coordinate estimations in the global coordinate system. The vehicle, through its C-V2X communication module, broadcasts this factor-map-based corrected localization result along with the updated semantic landmark detection set, following standard V2X message formats (such as MAP or SPAT message extensions in SAE J2735). Nearby vehicles equipped with the same system receive this updated broadcast information and use it as high-quality collaborative data for recalculating deviations and correcting positions in steps S106 to S108. Similarly, the vehicle continuously receives updated data from neighboring vehicles for ongoing verification.

[0101] In this embodiment, the updated vehicle positions are used to update the factor map again, thereby further correcting and updating the positioning results and semantic landmark detection set. This constructs a complete technical closed loop of "perception-fusion-cooperation-correction-feedback," transforming single, static collaborative correction into continuous, dynamic collaborative evolution. This means that the connected vehicle group is no longer isolated in a single data exchange, but forms a collaborative perception network that mutually verifies and reinforces each other. Improvements in the accuracy of each vehicle benefit its neighboring vehicles through broadcasting, generating a network effect that ultimately improves the collective positioning accuracy and reliability of the entire vehicle group. Each collaborative correction is no longer the end point, but a new starting point for improving the accuracy of the next single-vehicle fusion. This closed-loop architecture enables the positioning system to have self-iterative and continuous optimization capabilities, fundamentally changing the situation where individual vehicles and collaborative modules are independent in traditional solutions. By feeding back the corrected state to the factor map, the long-term accumulation and divergence of errors can be effectively suppressed. Even after prolonged GNSS denial or severe weather, due to regular or continuous collaborative corrections for calibration and updates, the drift of the entire positioning trajectory can be controlled at an extremely low level, ensuring the stability and reliability of the system over its long lifespan. Furthermore, by broadcasting updated high-precision data (corrected positioning results and updated semantic landmark detection sets), a positive feedback loop is formed among vehicles. An improved positioning accuracy in one vehicle helps improve the accuracy of its neighboring vehicles, and these improved neighboring vehicles, in turn, become better cooperative partners, enabling the collective evolution of the positioning capabilities of the entire intelligent connected vehicle group. Moreover, under extreme conditions (such as torrential rain or dense fog), the performance of individual vehicle sensors may collectively decline. In such cases, through this closed-loop cooperative mechanism, vehicles can rely more heavily on high-quality cooperative data provided by neighboring vehicles in areas with relatively better perception conditions, serving as an effective supplement and verification of their own perception systems. This significantly enhances the survivability and performance maintenance capabilities of the entire system in extreme scenarios.

[0102] Example 1: During the morning rush hour, vehicle A (equipped with the positioning system described in this solution) travels to a densely packed intersection in the city center. Due to the obstruction of tall buildings, the GNSS signal is weak, and light rain causes glare interference to the visual sensors. Furthermore, a large truck ahead obstructs the traffic lights at the intersection. Vehicle A simultaneously collects three types of relevant positioning data: the GNSS receiver (supporting BeiDou + GPS dual-mode) acquires pseudorange, carrier phase, and preliminary positioning results (latitude and longitude E116.4°, N39.9°, altitude 45m, error approximately 4m); the IMU (MEMS type) outputs angular velocity (0.5° / s around the Z-axis, slight steering correction) and acceleration (0.3m / s² along the X-axis, slow acceleration); the wheel odometer calculates, through wheel speed sensors, that the vehicle has traveled a cumulative distance of 15m within 10 seconds, with the wheel speed stabilizing at 15km / h. Vehicle A's front-facing camera captures environmental images, which are then recognized by a pre-trained YOLOv8 model. The model successfully detects the "Speed ​​Limit 50" road sign on the right (category label: speed limit sign, confidence 0.89) and the corner of a tall building in the distance (category label: fixed building outline, confidence 0.85). Combining the camera's intrinsic parameters (focal length 1200 pixels, principal point coordinates (960, 540)) and extrinsic parameters (relative to the vehicle coordinate system X=0.6m, Y=0, Z=1.4m), the pixel coordinates of the two are converted to the vehicle's local coordinate system coordinates (road sign: X=35m, Y=4.2m, Z=2.5m; tall building corner: X=80m, Y=-2.1m, Z=15m), forming a semantic landmark detection set. Simultaneously, forward-facing millimeter-wave radar generates point cloud data (road sign distance 35.3m, angle +6.8°; high-rise corner distance 80.5m, angle -1.5°). The DBSCAN clustering algorithm (neighborhood radius ε=0.5m, minimum number of points minPts=5) filters out dynamic vehicles, retaining static target clusters and extracting their geometric contours (road sign width 1.2m, height 0.8m; high-rise corner width 0.5m, height 3m), outputting environmental point cloud data. A factor graph is constructed: using vehicle poses (Xt, Yt, θt) every 0.1s within 20s as state nodes, four types of constraint factors are set: GNSS factor (weight 0.7), IMU factor (weight 0.8), semantic landmark factor (road sign weight 0.89, high-rise corner weight 0.85), and radar point cloud factor (weight 0.8). With the goal of minimizing the weighted sum of squared residuals, the Gauss-Newton method was iterated 12 times (convergence threshold 1e-6) to obtain the global coordinates of vehicle A at t15s (E116.4002°, N39.9001°), reducing the planar error to 1.2m. Semantic landmarks with a confidence level >0.8 were also selected to form the fused localization result. Vehicle A broadcasts the fused localization result and the semantic landmark detection set to its surroundings via C-V2X technology. Vehicle B, located 100m behind, receives this data and transmits its own data (including the same road sign; vehicle B's local coordinates X=65m, Y=4.2m, confidence level 0.91).After verifying that the timestamps and semantic categories of the two vehicles are consistent, vehicle A constructs a transformation matrix based on the relative poses of the two vehicles (ΔX=33.3m, ΔY=0) to convert the road sign coordinates of vehicle B into its own local coordinate system (X=31.7m, Y=4.2m). Feature comparison shows an Euclidean distance of 3.3m (<5m), a radar error of 0.2m (<0.5m), and a visual texture similarity of 0.88 (>0.8), indicating an overlapping landmark. Because the road sign of vehicle B has higher confidence, it is used as a benchmark. A correction of δx=3.3m is calculated using the least squares method. After correction, the positioning error of vehicle A is reduced to 0.8m. Simultaneously, information about the blocked traffic light is obtained using data from vehicle B, overcoming perceptual limitations.

[0103] Example 2: On a mountainous highway tunnel section, vehicle C needs to pass through three short tunnels consecutively. GNSS signals are completely ineffective inside the tunnels, and the reflection from the tunnel walls makes the visual sensors prone to misidentification. Vehicle C collects relevant positioning data: Before entering the first tunnel, the GNSS receiver acquires pseudorange, carrier phase, and preliminary positioning results (latitude and longitude E110.5°, N30.2°, altitude 850m, error 2m); the IMU (MEMS type, angular velocity range ±2000° / s) outputs data in real time. When the vehicle is traveling at a constant speed (80km / h), the angular velocity around the Z-axis is 0°, and the acceleration along the X-axis is 0m / s²; the wheel odometer calculates the travel distance through the wheel speed sensor, accumulating 222m every 10s, with the wheel speed stable at 80km / h. Vehicle C's front-facing camera captures images inside the tunnel. Using the YOLOv8 model, it detects an "Emergency Exit" sign on the tunnel wall (category label: safety sign, confidence level 0.87) and a reflective contour post at the tunnel entrance (category label: road boundary sign, confidence level 0.83). Combining the camera's intrinsic parameters (focal length 1100 pixels, principal point coordinates (960, 540)) and extrinsic parameters (relative to the vehicle coordinate system: X=0.5m, Y=0, Z=1.5m), the sign pixel coordinates are converted to the vehicle's local coordinate system coordinates (Emergency Exit: X=40m, Y=3.8m, Z=2.2m; Reflective Contour Post: X=20m, Y=-3.5m, Z=1.8m), forming a semantic landmark detection set. Meanwhile, forward-facing millimeter-wave radar (detection range 0.1-150m, angle range ±60°) generates point cloud data inside the tunnel. The distance to the emergency exit is 40.2m and the angle is +5.5°, while the distance to the reflective contour pile is 20.1m and the angle is -5.2°. Using the DBSCAN clustering algorithm (neighborhood radius ε=0.4m, minimum number of points minPts=4), construction carts (dynamic targets) that occasionally appear in the tunnel are filtered out, and the static clusters corresponding to the emergency exit and reflective contour piles are retained. The geometric contours of the static clusters are extracted (emergency exit width 1.5m, height 2.0m; reflective contour pile diameter 0.3m, height 1.2m), and the environmental point cloud data is output. Constructing a factor graph: Using the vehicle pose (Xt, Yt, θt, θ is the heading angle) every 0.1s within 15s after entering the tunnel as state nodes, four types of constraint factors are set: GNSS factor (only applies to the initial moment at the tunnel entrance, weight 0.9, anchored to global coordinates), IMU factor (kinematic constraints between adjacent moments, weight 0.85, high-frequency data ensures the continuity of motion within the tunnel), semantic landmark factor (emergency exit weight 0.87, reflective contour stake weight 0.83, assigned based on confidence), and radar point cloud factor (geometric constraints, weight 0.8, resists tunnel reflective interference).With the objective of minimizing the weighted sum of squared residuals of all constraint factors, the Gauss-Newton method was iterated 10 times to obtain the global coordinates of vehicle C in the tunnel at t10s (E110.5001°, N30.2000°, altitude 852m). The planar error was reduced to 1.3m. Semantic landmarks with a confidence level >0.8 were selected to form the fused positioning result. Vehicle C broadcast the fused positioning result and semantic landmark detection set to vehicle D (using the same system, just entering the tunnel) 200m behind via C-V2X technology. After receiving the data, vehicle D transmitted its own data back (including the same emergency exit marker, vehicle D's local coordinates X=60m, Y=3.7m, confidence level 0.89). Vehicle C first verified the C-V2X communication timestamps of the two vehicles to ensure spatiotemporal alignment; then compared the semantic landmark categories, both of which were "emergency exit" markers, and the collaborative data was retained. Based on the relative poses of the two vehicles (ΔX=200m, ΔY=0.1m), a transformation matrix was constructed to convert the emergency exit coordinates of vehicle D into its own local coordinate system (X=40.1m, Y=3.8m). Feature comparison revealed: Euclidean distance 0.1m (<3m, dense targets in the tunnel, threshold lowered), radar point cloud error 0.1m (<0.4m), and visual texture similarity 0.90 (>0.8), indicating overlapping landmarks. Since the confidence score of vehicle D's emergency exit (0.89) was higher than that of vehicle C (0.87), using vehicle D's coordinates as a reference, a target function was constructed using the least squares method, yielding correction values ​​δx=0.1m and δy=0. After correction, vehicle C's positioning error decreased to 0.7m, solving the positioning problem caused by GNSS failure and visual interference in the tunnel.

[0104] This application uses high-confidence landmark coordinates as a benchmark to ensure the accuracy of the correction reference and avoid error propagation caused by low-confidence data. It constructs an objective function with landmark confidence weights using the least squares method, so that high-confidence landmarks have a higher proportion in the correction. At the same time, it integrates multiple overlapping landmark data to solve for the optimal correction amount, effectively reducing the influence of noise from a single landmark, eliminating positioning bias, and improving the stability of the correction results. This application proposes a positioning method for intelligent connected vehicles. By integrating vehicle-related positioning data, visual semantic landmarks, and millimeter-wave environmental point cloud data, and relying on a factor graph optimization algorithm to achieve deep coupling of multi-source data, it not only compensates for the deficiencies of single sensors but also enhances the robustness of fusion with semantic information, significantly improving the positioning accuracy of single vehicles and effectively compensating for the shortcomings of weak GNSS signals, visual interference from the environment, and low radar resolution. Simultaneously, by leveraging V2V communication to fuse positioning results with a semantic landmark detection set, and through screening processes such as timestamp verification, semantic category comparison, coordinate transformation, and feature comparison, the reliability of the collaborative data is ensured. Then, using vehicle observations with high landmark confidence as a benchmark, the local coordinate system position is corrected using the least squares method. This avoids error propagation caused by low-confidence data and solves the problems of coarse benchmark selection and insufficient data verification in traditional vehicle-to-vehicle collaboration. Ultimately, in complex scenarios such as urban canyons and tunnels, it achieves improved positioning accuracy and significantly enhanced robustness, providing strong support for the safe and reliable operation of intelligent connected vehicles. Thus, it solves the problems of low positioning reliability and limited sensing range in extreme weather conditions found in existing technologies.

[0105] This application acquires relevant vehicle positioning data; identifies semantic landmarks through visual sensors and acquires environmental point cloud data through millimeter-wave radar; constructs a factor graph based on the relevant positioning data, semantic landmarks, and environmental point cloud data, and uses a factor graph-based optimization algorithm for fusion to obtain a fused positioning result and a semantic landmark detection set; the vehicle communicates with nearby vehicles via V2V, broadcasting its own fused positioning result and semantic landmark detection set; after receiving the broadcast information, the vehicle first verifies the C-V2X communication timestamps of the two vehicles, compares the semantic landmark category recognition results of the two vehicles for the same area, retains semantically consistent collaborative data, and filters out semantic landmarks that overlap with the vehicle's own semantic landmark detection set from the collaborative data through coordinate transformation and feature comparison, determines the observation deviation of the two vehicles for the landmark position, and if there is an observation deviation, uses the landmark coordinates of the vehicle with high landmark confidence as a benchmark, and corrects the vehicle's position in the local coordinate system using the least squares method. This approach integrates five types of data simultaneously: GNSS, IMU, wheel odometer, visual semantic landmarks, and millimeter-wave radar point clouds. Furthermore, it utilizes YOLOv8 semantic recognition and DBSCAN clustering to achieve dual environmental constraints of "semantic + geometric," resulting in stronger anti-interference capabilities. This application constructs multiple constraint relationships through factor graphs and combines semantic landmark confidence screening (removing low-confidence data) for more precise optimization targets and higher robustness of the fusion results. This application uses a dual matching mechanism of semantic consistency comparison and spatial features to screen reliable collaborative data, avoiding error propagation. Simultaneously, it uses high-confidence landmarks as a benchmark and least squares correction to address the issue of coarse collaborative benchmarks. This application first achieves single-vehicle multi-source data fusion through factor graphs, and then achieves external verification and correction through vehicle-to-vehicle collaboration, forming a closed loop of single-vehicle optimization and collaborative correction. Compared to existing single optimization mechanisms, this significantly improves positioning accuracy and stability. This application integrates vehicle-related positioning data, visual semantic landmarks, and millimeter-wave environmental point cloud data. Utilizing a factor graph optimization algorithm, it achieves deep coupling of multi-source data, compensating for the limitations of single sensors and enhancing fusion robustness through semantic information. This significantly improves single-vehicle positioning accuracy and effectively overcomes the deficiencies of weak GNSS signals, visual interference from the environment, and low radar resolution. Simultaneously, by leveraging V2V communication to fuse positioning results with a semantic landmark detection set, and employing filtering processes such as timestamp verification, semantic category comparison, coordinate transformation, and feature comparison, the reliability of the collaborative data is ensured. Using vehicle observations with high landmark confidence as a benchmark, the local coordinate system position is corrected using the least squares method. This avoids error propagation caused by low-confidence data and solves the problems of coarse benchmark selection and insufficient data verification in traditional vehicle-to-vehicle collaboration. Ultimately, in complex scenarios such as urban canyons and tunnels, it achieves improved positioning accuracy and significantly enhanced robustness, providing strong support for the safe and reliable operation of intelligent connected vehicles. Thus, it solves the problems of low positioning reliability and limited sensing range in extreme weather conditions found in existing technologies.

[0106] To better implement the vehicle positioning method based on intelligent connectivity in the embodiments of the present invention, based on the vehicle positioning method based on intelligent connectivity, correspondingly, as follows: Figure 7 As shown, this embodiment of the invention also provides a vehicle positioning system 700 based on intelligent connectivity, which includes: Data acquisition module 701 is used to acquire relevant vehicle positioning data; The environmental perception module 702 is used to obtain environmental point cloud data based on the raw laser data and to obtain semantic landmarks based on the collected environmental images; The factor graph construction module 703 is used to construct a factor graph containing state nodes and constraint factors based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data. The data fusion module 704 is used to solve the factor graph to obtain the vehicle's fused positioning result and a filtered semantic landmark detection set; The vehicle-to-vehicle communication module 705 is used to broadcast the fused positioning results and the semantic landmark detection set to nearby vehicles via vehicle-to-vehicle communication technology; The verification and comparison module 706 is used to receive the collaborative data broadcast by the neighboring vehicles, and then perform time synchronization verification and semantic consistency comparison on the collaborative data between the vehicle and the neighboring vehicles, retaining the semantically consistent collaborative data; the collaborative data includes the fusion positioning results and semantic landmark detection set of the neighboring vehicles. The filtering calculation module 707 is used to filter out target semantic landmarks that overlap with the vehicle semantic landmark detection set from the semantically consistent collaborative data through coordinate transformation and feature comparison, and to calculate the observation deviation of the vehicle and the neighboring vehicles on the target semantic landmarks. The collaborative correction module 708 is used to correct the position of the vehicle in the local coordinate system by using the least squares method, based on the semantic landmark coordinates with the highest confidence, when the observation deviation exceeds a preset threshold.

[0107] The intelligent connected vehicle positioning system 700 provided in the above embodiments can realize the technical solutions described in the above embodiments of the intelligent connected vehicle positioning method. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the intelligent connected vehicle positioning method, which will not be repeated here.

[0108] like Figure 8 As shown, the present invention also provides an intelligent vehicle 800. The intelligent vehicle 800 includes a processor 801, a memory 802, and a display 803. Figure 8 Only some of the components of the intelligent vehicle 800 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0109] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as the vehicle positioning method based on intelligent network connectivity in this invention.

[0110] In some embodiments, processor 801 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 801 may be local or remote. In some embodiments, processor 801 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0111] In some embodiments, memory 802 may be an internal storage unit of the intelligent vehicle 800, such as a hard disk or memory of the intelligent vehicle 800. In other embodiments, memory 802 may also be an external storage device of the intelligent vehicle 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the intelligent vehicle 800.

[0112] Furthermore, the memory 802 may include both internal storage units of the intelligent vehicle 800 and external storage devices. The memory 802 is used to store application software and various types of data installed in the intelligent vehicle 800.

[0113] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display information from the intelligent vehicle 800 and to display a visual user interface. Components 801-803 of the intelligent vehicle 800 communicate with each other via a system bus.

[0114] In one embodiment, when the processor 801 executes the vehicle positioning program based on intelligent network connectivity in the memory 802, the following steps can be implemented: Obtain relevant vehicle location data; Environmental point cloud data is obtained from the raw laser data, and semantic landmarks are obtained from the collected environmental images; Based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data, a factor graph containing state nodes and constraint factors is constructed. The factor graph is solved to obtain the vehicle's fused localization result and a filtered semantic landmark detection set; The fused positioning results and the semantic landmark detection set are broadcast to nearby vehicles using vehicle-to-vehicle communication technology. After receiving the collaborative data broadcast by the nearby vehicles, the vehicle performs time synchronization verification and semantic consistency comparison on the collaborative data between itself and the nearby vehicles, and retains the semantically consistent collaborative data; the collaborative data includes the fusion positioning results and semantic landmark detection set of the nearby vehicles; From the semantically consistent collaborative data, target semantic landmarks that overlap with the vehicle semantic landmark detection set are filtered out through coordinate transformation and feature comparison, and the observation deviation of the vehicle and the neighboring vehicles on the target semantic landmarks is calculated. When the observation deviation exceeds a preset threshold, the position of the vehicle in the local coordinate system is corrected by least squares method based on the semantic landmark coordinates with the highest confidence.

[0115] It should be understood that when the processor 801 executes the vehicle positioning program based on intelligent network in the memory 802, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0116] Furthermore, this embodiment of the invention does not specifically limit the type of intelligent vehicle 800 mentioned. Intelligent vehicle 800 can be a portable intelligent vehicle such as a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, or laptop computer. Exemplary embodiments of portable intelligent vehicles include, but are not limited to, portable intelligent vehicles running iOS, Android, Microsoft, or other operating systems. The aforementioned portable intelligent vehicle can also be other portable intelligent vehicles, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, intelligent vehicle 800 may not be a portable intelligent vehicle, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0117] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the vehicle positioning method based on intelligent network connectivity provided in the above-described method embodiments.

[0118] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0119] The above provides a detailed description of the vehicle positioning method, device, equipment, and medium based on intelligent network connectivity provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A vehicle positioning method based on intelligent connected vehicles, characterized in that, include: Obtain relevant vehicle location data; Environmental point cloud data is obtained from the raw laser data, and semantic landmarks are obtained from the collected environmental images; Based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data, a factor graph containing state nodes and constraint factors is constructed. The factor graph is solved to obtain the vehicle's fused localization result and a filtered semantic landmark detection set; The fused positioning results and the semantic landmark detection set are broadcast to nearby vehicles using vehicle-to-vehicle communication technology. After receiving the collaborative data broadcast by the nearby vehicles, the vehicle performs time synchronization verification and semantic consistency comparison on the collaborative data between itself and the nearby vehicles, and retains the semantically consistent collaborative data; the collaborative data includes the fusion positioning results and semantic landmark detection set of the nearby vehicles; From the semantically consistent collaborative data, target semantic landmarks that overlap with the vehicle semantic landmark detection set are filtered out through coordinate transformation and feature comparison, and the observation deviation of the vehicle and the neighboring vehicles on the target semantic landmarks is calculated. When the observation deviation exceeds a preset threshold, the position of the vehicle in the local coordinate system is corrected by least squares method based on the semantic landmark coordinates with the highest confidence.

2. The vehicle positioning method based on intelligent network connectivity according to claim 1, characterized in that, The process of obtaining environmental point cloud data based on raw laser data and semantic landmarks based on acquired environmental images includes: The environmental image is processed using the YOLOv8 model to output the semantic landmarks; the semantic landmarks include category labels, pixel coordinates, and confidence scores. Using camera calibration parameters, the pixel coordinates are transformed into three-dimensional coordinates in the vehicle's local coordinate system; Based on the transformed 3D coordinates, category labels, and confidence scores, a structured semantic landmark detection set is generated; Acquire raw laser data generated by radar, the raw laser data containing distance and angle information of obstacles in the environment; The original laser data was processed using the DBSCAN clustering algorithm to distinguish between dynamic and static obstacles; Extract the geometric contour features of the point cloud clusters corresponding to the static obstacles, and generate structured environmental point cloud data based on the geometric contour features.

3. The vehicle positioning method based on intelligent network connectivity according to claim 2, characterized in that, The relevant positioning data includes Global Navigation Satellite System data, inertial measurement data, and wheel odometer data. The step of constructing a factor graph containing state nodes and constraint factors based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data includes: The vehicle's pose in a discrete time series is defined as a state node; Establish multiple constraint factors connecting the state nodes; the multiple constraint factors include GNSS factors obtained from the Global Navigation Satellite System data, IMU factors obtained from the inertial measurement data, odometer factors obtained from the wheel odometer data, semantic landmark factors obtained from the semantic landmarks, and radar point cloud factors obtained from the environmental point cloud data.

4. The vehicle positioning method based on intelligent network connectivity according to claim 3, characterized in that, Solving the factor graph to obtain the vehicle's fused localization result and a filtered semantic landmark detection set includes: Construct the first optimization function with the objective of minimizing the weighted sum of squared residuals for all constraint factors: ； in, Let X be the first optimization function, M be the set of all state nodes, ri be the residual vector of the i-th constraint factor, and Wi be the weight matrix of the i-th constraint factor. The first optimization function is solved using the Gauss-Newton iterative algorithm to obtain the optimal vehicle pose sequence as the fusion localization result, and a set of semantic landmark detections with a confidence level higher than the target threshold after filtering is output.

5. The vehicle positioning method based on intelligent network connectivity according to claim 1, characterized in that, The process of filtering out target semantic landmarks that overlap with the vehicle semantic landmark detection set through coordinate transformation and feature comparison includes: Based on the fusion positioning results of the vehicle and the neighboring vehicles, the relative poses of the two vehicles are calculated, and the corresponding coordinate transformation matrix is ​​constructed. Using the coordinate transformation matrix, the coordinates of the semantic landmarks in the neighboring vehicle semantic landmark detection set are transformed to the vehicle's local coordinate system to obtain the transformed landmark coordinates; Calculate the Euclidean distance between the transformed landmark coordinates and the vehicle's landmark coordinates; Calculate the radar point cloud feature error and visual texture similarity between the vehicle and the neighboring vehicles for the same candidate landmark pair; When the Euclidean distance is less than a first preset threshold, the radar point cloud feature error is less than a second preset threshold, and the visual texture similarity is greater than a third preset threshold, the candidate landmark pair is determined to be the target semantic landmark.

6. The vehicle positioning method based on intelligent network connectivity according to claim 5, characterized in that, The step of correcting the vehicle's position in the local coordinate system using the least squares method includes: Construct a second optimization function with the objective of minimizing the weighted positional bias: ； in, For the second optimization function, The number of overlapping landmarks selected; The confidence level of the m-th landmark for the baseline vehicle; It is a correction amount, including and Quantity; Let m be the coordinates of the m-th semantic landmark observed by the vehicle in its own local coordinate system; The m-th semantic landmark coordinates after transformation for adjacent vehicles; The position of the vehicle in the local coordinate system is updated based on the optimal position correction obtained by solving the second optimization function.

7. The vehicle positioning method based on intelligent network connectivity according to claim 1, characterized in that, After correcting the position of the vehicle in the local coordinate system using the least squares method, the method further includes: The updated vehicle positions are used as the new initial state to update the factor graph; Using vehicle-to-vehicle communication technology, the corrected positioning results based on the factor graph are broadcast along with the updated semantic landmark detection set.

8. A vehicle positioning system based on intelligent network connectivity, characterized in that, include: The data acquisition module is used to acquire relevant vehicle location data; The environment perception module is used to obtain environmental point cloud data based on the raw laser data and semantic landmarks based on the collected environmental images. The factor graph construction module is used to construct a factor graph containing state nodes and constraint factors based on the relevant positioning data, the semantic landmarks, and the environmental point cloud data. The data fusion module is used to solve the factor graph to obtain the vehicle's fused positioning result and a filtered semantic landmark detection set; The vehicle-to-vehicle communication module is used to broadcast the fused positioning results and the semantic landmark detection set to nearby vehicles via vehicle-to-vehicle communication technology; The verification and comparison module is used to receive the collaborative data broadcast by the neighboring vehicles, perform time synchronization verification and semantic consistency comparison on the collaborative data between the vehicle and the neighboring vehicles, and retain the semantically consistent collaborative data. The collaborative data includes the fused localization results of the nearby vehicles and the semantic landmark detection set; The filtering and calculation module is used to filter out target semantic landmarks that overlap with the vehicle semantic landmark detection set from the semantically consistent collaborative data through coordinate transformation and feature comparison, and to calculate the observation deviation of the vehicle and the neighboring vehicles on the target semantic landmarks. The collaborative correction module is used to correct the position of the vehicle in the local coordinate system by using the least squares method, based on the semantic landmark coordinates with the highest confidence, when the observation deviation exceeds a preset threshold.

9. An intelligent vehicle, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the vehicle positioning method based on intelligent connectivity as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, Used to store computer-readable programs or instructions, which, when executed by a processor, can implement the steps in the vehicle positioning method based on intelligent connectivity as described in any one of claims 1 to 7.

Images