Positioning method and system based on multi-source data fusion and vehicle

Abstract

The application relates to the technical field of positioning and navigation, and discloses a multi-source data fusion positioning method and system and a vehicle. The method comprises the following steps: collecting and preprocessing inertial measurement, satellite navigation, cellular network, geomagnetic vector and visual image data in real time; independently solving each data to construct a corresponding positioning source; identifying a current scene type based on visual image; determining an enabled positioning source combination and a working mode according to the scene type; based on a factor graph optimization framework, the inertial positioning source is incorporated into the factor graph in the form of a motion constraint, and the remaining positioning sources are incorporated in the form of an observation constraint, weighted fusion solving is carried out, and position, attitude and speed information of a target is output. Through scene perception driven positioning source dynamic selection and weight self-adaptive distribution, the technical problems that the existing scheme lacks scene adaptive capability, fusion weight is fixed and lacks an abnormal processing mechanism are solved.

Description

Technical Field

[0001] This invention relates to the field of positioning and navigation technology, specifically to a positioning method, system, and vehicle based on multi-source data fusion. Background Technology

[0002] With the rapid development of intelligent driving and autonomous navigation technologies, high-precision and high-reliability positioning capabilities have become a core fundamental requirement. Currently, commonly used positioning technologies include satellite navigation positioning, inertial navigation positioning, cellular network positioning, geomagnetic positioning, and visual positioning.

[0003] However, all of the aforementioned positioning technologies have inherent limitations. Satellite navigation positioning offers high accuracy in open outdoor environments, but in obstructed or enclosed environments such as urban canyons, tunnels, and underground parking garages, signal attenuation is severe or even completely lost, leading to positioning failure. Inertial navigation positioning has the advantage of strong short-term autonomy, but its errors accumulate over time, resulting in a sharp decline in positioning accuracy after prolonged operation. Cellular network positioning offers wide coverage, but its positioning accuracy is limited and cannot meet high-precision requirements. Geomagnetic positioning is unaffected by lighting and satellite signals, making it suitable for indoor and underground scenarios, but its reliability is significantly reduced in environments with strong magnetic interference. Visual positioning performs well in environments with rich textures, but feature extraction is difficult in low-light, strong-light, or monochromatic scenarios, making it difficult to guarantee positioning accuracy.

[0004] To overcome the limitations of a single positioning source, multi-source fusion positioning schemes have emerged in existing technologies. However, existing schemes still have the following shortcomings:

[0005] First, most solutions use a fixed combination of positioning sources and fixed fusion weights, lacking the ability to dynamically adjust according to the actual scenario. When environmental conditions change, the fixed configuration cannot adapt, leading to a decrease in fusion accuracy or even divergence.

[0006] Second, existing solutions typically integrate all location sources equally without fully considering the differences in availability of each location source in different scenarios, and lack a linkage mechanism between scenario awareness and location source selection.

[0007] Third, when a positioning source produces abnormal output due to sudden environmental changes or sensor failure, the existing solution lacks an effective anomaly detection and adaptive processing mechanism. Abnormal data may contaminate the fusion results and affect the overall positioning reliability.

[0008] Therefore, there is an urgent need for a multi-source data fusion positioning scheme that can adaptively select the combination of positioning sources according to the scenario, dynamically adjust the fusion weights, and has the ability to handle anomalies. Summary of the Invention

[0009] In view of the shortcomings of the prior art, the purpose of this application is to provide a positioning method, system and vehicle for multi-source data fusion, which aims to solve the technical problems of existing multi-source fusion positioning schemes lacking scene adaptation capability, fixed fusion weights and lacking anomaly handling mechanism.

[0010] In a first aspect, embodiments of this application provide a positioning method based on multi-source data fusion, comprising:

[0011] Real-time acquisition of target inertial measurement data, satellite navigation positioning data, cellular network positioning data, geomagnetic vector data, and visual image data, followed by preprocessing.

[0012] Each preprocessed data point is solved independently to construct corresponding inertial positioning sources, satellite navigation positioning sources, cellular network positioning sources, geomagnetic positioning sources, and visual positioning sources.

[0013] Based on the preprocessed visual image data, identify the current scene type;

[0014] Based on the scenario type, determine the currently enabled combination of positioning sources and the working mode of each positioning source from the existing positioning sources;

[0015] Based on the factor graph optimization framework, the inertial positioning source is incorporated into the factor graph in the form of motion constraints, and the solution results of the other enabled positioning sources are used as observation constraints for weighted fusion solution to output the target's position, attitude and velocity information.

[0016] In the above technical solution, scene recognition is achieved through visual perception, and the combination and working mode of positioning sources are dynamically determined accordingly. This enables the positioning strategy to adapt to changes in the environment and avoids the performance degradation of fixed configurations when switching scenes. At the same time, based on the factor graph optimization framework, inertial positioning sources are incorporated in the form of motion constraints, and other positioning sources are incorporated in the form of observation constraints. This fully utilizes the complementary characteristics of each positioning source in terms of temporal continuity and spatial absoluteness, effectively improving the accuracy and robustness of fusion positioning.

[0017] In one embodiment, the scene types include outdoor open scenes, semi-obscured scenes, indoor blind spot scenes, low-light scenes, and strong magnetic interference scenes; the step of identifying the current scene type based on preprocessed visual image data specifically includes: extracting environmental features from the visual image through semantic segmentation, and determining the current scene type based on the environmental features.

[0018] In the above technical solution, the scenarios are divided into five typical types, covering the main positioning failure scenarios such as satellite signal degradation, insufficient lighting and magnetic field interference, providing a clear decision basis for subsequent positioning source selection; the scene determination is performed by extracting environmental features through semantic segmentation, which has a more intuitive and accurate scene perception capability compared with the indirect judgment method based on signal strength.

[0019] In one embodiment, determining the currently enabled combination of location sources and the operating mode of each location source from the established location sources according to the scenario type specifically includes:

[0020] In open outdoor scenarios, satellite navigation positioning sources and cellular network positioning sources are the main positioning sources, while geomagnetic positioning sources, inertial positioning sources and visual positioning sources are the auxiliary positioning sources.

[0021] In partially obscured scenarios, geomagnetic positioning sources, visual positioning sources, and inertial positioning sources are used as the main positioning sources, while cellular network positioning sources are used as auxiliary positioning sources.

[0022] In indoor blind zone scenarios, geomagnetic positioning sources, visual positioning sources, and inertial positioning sources are the main positioning sources;

[0023] In low-light scenarios, visual positioning sources are disabled, and satellite navigation positioning sources, cellular network positioning sources, geomagnetic positioning sources, and inertial positioning sources are used as the main positioning sources.

[0024] In scenarios with strong magnetic interference, geomagnetic positioning sources are disabled, and satellite navigation positioning sources, cellular network positioning sources, visual positioning sources, and inertial positioning sources are used as the main positioning sources.

[0025] In the above technical solution, a differentiated positioning source configuration strategy is formulated for each scenario type. The positioning source with the highest reliability in the current scenario is given priority as the main positioning source, and the positioning source with the next highest reliability is used as an auxiliary positioning source to supplement the constraints. Positioning sources that are obviously unusable in the current scenario are disabled, thereby ensuring positioning accuracy while avoiding the contamination of the fusion results by unreliable data.

[0026] In one embodiment, the step of independently solving each preprocessed data to construct a corresponding location source specifically includes:

[0027] Pre-integration calculations are performed on the inertial measurement data to construct an inertial positioning source;

[0028] Pseudorange and carrier phase calculations are performed on satellite navigation and positioning data to construct a satellite navigation and positioning source;

[0029] Calculate the time difference of arrival or angle of arrival for cellular network positioning data to construct a cellular network positioning source;

[0030] Geomagnetic feature matching calculations are performed on geomagnetic vector data to construct a geomagnetic positioning source;

[0031] Feature extraction and visual odometry calculation are performed on visual image data to construct a visual positioning source.

[0032] In the above technical solution, each positioning source uses a solution algorithm that matches its data characteristics to perform independent solution, which ensures the solution accuracy of each positioning source output. At the same time, the independent solution method makes the positioning sources decoupled from each other, and the anomaly of any positioning source will not be propagated to other positioning sources through the solution process, which provides a basis for anomaly isolation in the subsequent fusion stage.

[0033] In one embodiment, the weighted fusion solution based on the factor graph optimization framework specifically includes: in the factor graph, the inertial positioning source is represented in the form of motion constraint factors to connect state nodes at adjacent times; the remaining enabled positioning sources are represented in the form of observation constraint factors to constrain the state estimation at the current time; the fusion weight of each observation constraint factor is dynamically allocated according to the scene type and the real-time quality assessment results of the positioning source corresponding to each observation constraint factor; the factor graph is iteratively optimized and solved to output the optimal state estimate.

[0034] In the above technical solution, the inertial positioning source is modeled as a motion constraint factor and the other positioning sources are modeled as observation constraint factors, which conforms to the standard modeling paradigm of motion model and observation model in the factor graph framework, making the factor graph structure clear and the physical meaning explicit; the fusion weight is dynamically allocated according to the scene type and real-time quality assessment results, so that high-quality positioning sources can obtain greater constraint contributions and the negative impact of low-quality positioning sources is effectively suppressed, thereby achieving near-optimal fusion accuracy in different scenarios.

[0035] In one embodiment, the system further includes anomaly adaptive processing: real-time monitoring of the working status and output quality of each positioning source; when an anomaly is detected in any positioning source, reducing the constraint factor fusion weight corresponding to that positioning source and correspondingly increasing the constraint factor fusion weight of the remaining available positioning sources; and restoring the fusion weight configuration to its state before the anomaly is eliminated.

[0036] In the above technical solution, soft isolation of abnormal location sources is achieved through real-time monitoring and adaptive weight adjustment mechanism, avoiding the contamination of fusion results by abnormal data; at the same time, the weight configuration is automatically restored after the anomaly is eliminated, without manual intervention, ensuring the continuous autonomous operation capability of the positioning system under complex dynamic conditions.

[0037] In one embodiment, the preprocessing specifically includes:

[0038] Denoising and zero-bias compensation are performed on the inertial measurement data;

[0039] Perform carrier-to-noise ratio filtering and multipath suppression on satellite navigation and positioning data;

[0040] Residual removal is performed on cellular network location data;

[0041] Low-pass filtering and hard magnetic compensation are performed on the geomagnetic vector data;

[0042] Distortion correction and exposure normalization are performed on visual image data.

[0043] In the above technical solution, corresponding preprocessing strategies were designed for the typical error sources of various sensor data, which improved the quality of the input data of each positioning source from the data source source, and provided a basic guarantee for the accuracy of subsequent independent calculation and fusion calculation.

[0044] Secondly, embodiments of this application provide a positioning system based on multi-source data fusion, comprising:

[0045] The multi-source sensing module includes an inertial measurement unit, a satellite navigation receiving unit, a cellular network positioning unit, a geomagnetic acquisition unit, and a visual sensing unit, which are used to collect corresponding sensor data and perform preprocessing.

[0046] The location source construction module is used to independently solve each preprocessed data and construct the corresponding location source.

[0047] The scene recognition module is used to identify the current scene type based on the image data collected by the visual perception unit;

[0048] The fusion solution module is used to determine the currently enabled combination of positioning sources and the working mode of each positioning source from the constructed positioning sources according to the scene type, perform weighted fusion solution based on the factor graph optimization framework, and output the position, attitude and velocity information of the target.

[0049] In the above technical solution, the multi-source data acquisition, independent calculation, scene recognition and fusion calculation are decoupled into independent functional modules through a modular architecture. Each module has a clear responsibility and a clear interface, which facilitates the development, debugging and subsequent functional expansion of the system.

[0050] In one embodiment, the fusion solution module includes:

[0051] The quality assessment submodule is used to assess the data quality of each location source in real time and output confidence index.

[0052] The weight allocation submodule is used to dynamically allocate the fusion weights of each positioning source according to the scene type and the confidence index.

[0053] The optimization solution submodule is used to construct the factor graph and perform iterative optimization to output the optimal state estimate.

[0054] In the above technical solution, the fusion solution process is subdivided into three sub-modules: quality assessment, weight allocation, and optimization solution, which realizes the refined control of the fusion strategy. Quality assessment provides a quantitative basis for weight allocation, and weight allocation provides a constraint configuration for optimization solution. The three form a closed loop to ensure that the fusion result approaches the optimum under the current working conditions.

[0055] Thirdly, embodiments of this application provide a vehicle including the aforementioned multi-source data fusion positioning system.

[0056] In the above technical solution, the vehicle is equipped with the aforementioned multi-source data fusion positioning system, which can continuously provide high-precision and high-reliability positioning services under various complex working conditions such as urban roads, tunnels, underground garages, low-light environments, and areas with strong magnetic interference, thus meeting the stringent requirements of intelligent driving and autonomous navigation for positioning capabilities. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in the embodiments of this application or the background art, the accompanying drawings used in the embodiments of this application will be described below.

[0058] Figure 1 This is a flowchart illustrating the multi-source data fusion localization method disclosed in an embodiment of this application;

[0059] Figure 2 This is a schematic diagram of one embodiment of the multi-source data fusion positioning system disclosed in this application.

[0060] Figure 3 This is a schematic diagram of another implementation of the multi-source data fusion positioning system disclosed in the embodiments of this application.

[0061] Explanation of reference numerals in the attached figures:

[0062] 100 - Multi-source perception module, 200 - Localization source construction module, 300 - Scene recognition module, 400 - Fusion calculation module, 410 - Quality assessment sub-module, 420 - Weight allocation sub-module, 430 - Optimization solution sub-module. Detailed Implementation

[0063] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0064] In one embodiment, please refer to Figure 1 , Figure 1This is a flowchart illustrating the multi-source data fusion localization method disclosed in an embodiment of this application. This application provides a multi-source data fusion localization method, which includes the following steps:

[0065] Step S101: Real-time acquisition of target inertial measurement data, satellite navigation positioning data, cellular network positioning data, geomagnetic vector data, and visual image data, and preprocessing them respectively.

[0066] Specifically, the inertial measurement unit (IMU) acquires the target's three-axis acceleration and three-axis angular velocity data in real time to obtain inertial measurement data; the satellite navigation receiver receives Global Navigation Satellite System (GNSS) signals and demodulates them to obtain satellite navigation positioning data; the cellular network positioning unit receives 5G base station signals to obtain cellular network positioning data; the geomagnetic acquisition unit measures the three-dimensional geomagnetic vector of the target's location in real time to obtain geomagnetic vector data; and the visual perception unit acquires images of the target's surrounding environment in real time to obtain visual image data.

[0067] In one embodiment, the preprocessing specifically includes:

[0068] The inertial measurement data is denoised and zero-bias compensated. Specifically, a wavelet threshold denoising algorithm is used to filter out high-frequency random noise in the inertial measurement data, and the zero-bias error of the accelerometer and gyroscope is compensated based on the zero-bias parameters obtained during the static calibration stage, thereby eliminating the systematic deviation of the inertial measurement unit caused by temperature drift and manufacturing deviation.

[0069] Satellite navigation and positioning data undergoes C / N0 (Carrier-to-Noise Density Ratio) filtering and multipath suppression. Specifically, a C / N0 threshold is set, and satellite observation data with a C / N0 below the threshold are removed. Simultaneously, the RAIM (Receiver Autonomous Integrity Monitoring) algorithm is used to identify and suppress pseudorange jumps caused by multipath effects, retaining satellite observation data of acceptable quality.

[0070] Residual removal is performed on cellular network positioning data. Specifically, residuals are calculated from multi-base station measurement data, and abnormal measurement values ​​with residuals exceeding a preset threshold are removed to avoid gross errors caused by NLOS (Non-Line-of-Sight) propagation contaminating the positioning solution.

[0071] Low-pass filtering and hard magnetic compensation are performed on the geomagnetic vector data. Specifically, a low-pass filter is used to remove high-frequency electromagnetic interference components from the geomagnetic vector data; at the same time, based on pre-calibrated hard magnetic interference compensation parameters, the deviation of the fixed magnetic field caused by the carrier's own ferromagnetic material is compensated to restore the true geomagnetic vector.

[0072] Distortion correction and exposure normalization are performed on the visual image data. Specifically, distortion correction is performed on the original image based on the intrinsic parameter matrix and distortion coefficients of the visual perception unit to eliminate the influence of lens distortion on feature point localization; at the same time, exposure normalization is performed on the image to reduce the impact of illumination changes on subsequent feature extraction and matching consistency.

[0073] Step S102: Perform independent calculations on each of the preprocessed data to construct the corresponding inertial positioning source, satellite navigation positioning source, cellular network positioning source, geomagnetic positioning source, and visual positioning source.

[0074] In one embodiment, the independent solution specifically includes:

[0075] Inertial measurement data is pre-integrated to construct an inertial positioning source. Specifically, acceleration and angular velocity are pre-integrated between adjacent keyframes to obtain relative displacement increment, relative velocity increment, and relative attitude increment, which are used as the output of the inertial positioning source. The pre-integration method compresses high-frequency inertial data into low-frequency relative quantities, reducing the computational dimensionality of subsequent fusion optimization.

[0076] Pseudorange and carrier phase calculations are performed on satellite navigation and positioning data to construct a satellite navigation and positioning source. Specifically, least squares positioning calculations are performed based on the pseudorange observation equation to obtain the target's position estimate; when carrier phase observation conditions are met, RTK (Real-Time Kinematic) positioning calculations are further performed to obtain a position estimate with centimeter-level accuracy. The output of the satellite navigation and positioning source includes the target's three-dimensional position in the geodetic coordinate system and the corresponding DOP (Dilution of Precision).

[0077] Cellular network positioning data is analyzed using TDOA (Time Difference of Arrival) or AOA (Angle of Arrival) calculations to construct a cellular network positioning source. Specifically, when the target is within the coverage area of ​​multiple 5G base stations, hyperbolic positioning is performed based on the TDOA of each base station's signal, or triangulation is performed based on the AOA of the signal to obtain a two-dimensional or three-dimensional position estimate of the target. The output of the cellular network positioning source includes the target position estimate and the corresponding positioning accuracy index.

[0078] Geomagnetic feature matching is performed on geomagnetic vector data to construct a geomagnetic positioning source. Specifically, the real-time acquired 3D geomagnetic vectors are matched with a pre-constructed geomagnetic reference map. The optimal matching position is searched using a particle filtering algorithm or a correlation matching algorithm to obtain the target's location estimate. The output of the geomagnetic positioning source includes the matching position and the matching confidence level.

[0079] Feature extraction and visual odometry calculation are performed on visual image data to construct a visual localization source. Specifically, the ORB (Oriented Fast and Rotated BRIEF) algorithm is used to extract sparse feature points in the image. Camera motion is recovered through inter-frame feature matching and multi-view geometric constraints, thereby calculating the relative displacement and pose change of the target. The output of the visual localization source includes inter-frame relative pose transformation and the corresponding reprojection error.

[0080] Each location source is solved independently, so that the location sources are not coupled with each other. An anomaly in any location source will not be propagated to other location sources through the solution process, which provides a basis for anomaly isolation in the subsequent fusion stage.

[0081] Step S103: Identify the current scene type based on the preprocessed visual image data.

[0082] In one embodiment, the scene types include outdoor open scenes, semi-obscured scenes, indoor blind spot scenes, low-light scenes, and strong magnetic interference scenes.

[0083] The process of identifying the current scene type based on preprocessed visual image data specifically includes: performing pixel-level semantic segmentation on the preprocessed visual image using a semantic segmentation network to extract environmental features from the visual image, including the proportion of the visible sky area, the degree of building occlusion, the distribution of light intensity, and the richness of environmental texture; and determining the current scene type based on the environmental features using a classifier.

[0084] Specifically, when the proportion of the visible sky area is greater than a preset threshold and the lighting is sufficient, it is determined to be an open outdoor scene; when the proportion of the visible sky area is between the upper and lower thresholds or there is a partially obstructing structure, it is determined to be a partially obstructed scene; when the sky is not visible and the environment is an enclosed space, it is determined to be an indoor blind spot scene; when the light intensity is lower than a preset threshold, it is determined to be a low light scene; when a large area of ​​metal structure or strong electromagnetic equipment is detected in the environment, it is determined to be a strong magnetic interference scene.

[0085] Step S104: Based on the scenario type, determine the currently enabled combination of positioning sources and the working mode of each positioning source from the constructed positioning sources.

[0086] In one embodiment, it specifically includes:

[0087] In open outdoor scenarios, satellite navigation and cellular network positioning sources are the primary positioning sources, while geomagnetic, inertial, and visual positioning sources serve as auxiliary sources. In this scenario, GNSS signal quality is good, allowing satellite navigation to provide high-precision absolute position constraints; cellular network positioning serves as a redundancy supplement, providing a backup position when satellite signals are briefly obstructed.

[0088] In partially obscured scenarios, geomagnetic, visual, and inertial positioning sources are used as the primary positioning sources, with cellular network positioning sources serving as auxiliary sources. In this scenario, GNSS signals degrade due to partial obstruction, reducing the accuracy of satellite navigation positioning sources; therefore, their weight is reduced. Geomagnetic and visual positioning sources are unaffected by satellite signals and can provide stable position and attitude constraints.

[0089] In indoor blind zone scenarios, geomagnetic positioning, visual positioning, and inertial positioning are the primary positioning sources. GNSS signals are completely unavailable in this scenario, and cellular network signals may also be severely attenuated. Therefore, fusion positioning relies solely on geomagnetic positioning, visual positioning, and inertial positioning, which are independent of external wireless signals.

[0090] In low-light scenarios, visual positioning sources are disabled, and satellite navigation positioning sources, cellular network positioning sources, geomagnetic positioning sources, and inertial positioning sources are used as the main positioning sources. Visual image quality is severely degraded in this scenario, and feature extraction and matching reliability are insufficient. Therefore, visual positioning sources are removed from the fusion system to avoid their low-quality output contaminating the fusion results.

[0091] In scenarios with strong magnetic interference, geomagnetic positioning sources are disabled, and satellite navigation positioning sources, cellular network positioning sources, visual positioning sources, and inertial positioning sources are used as the main positioning sources. In this scenario, the geomagnetic vector is severely distorted by external interference, making geomagnetic matching results unreliable; therefore, geomagnetic positioning sources are removed from the fusion system.

[0092] Step S105: Based on the factor graph optimization framework, the inertial positioning source is incorporated into the factor graph in the form of motion constraints, and the solution results of the other enabled positioning sources are used as observation constraints to perform weighted fusion solution, and the position, attitude and velocity information of the target are output.

[0093] In one embodiment, the weighted fusion solution based on the factor graph optimization framework specifically includes:

[0094] In the factor graph, the target state at each time step is used as a state node, which contains the target's position, attitude, and velocity information. Inertial positioning sources are represented as motion constraint factors, used to connect state nodes at adjacent time steps, representing the kinematic constraint relationship between the target at adjacent time steps. The remaining enabled positioning sources are represented as observation constraint factors, used to constrain the current state estimate, providing absolute or relative position and attitude observation information.

[0095] Based on the scene type and the real-time quality assessment results of the corresponding positioning sources for each observation constraint factor, the fusion weights of each observation constraint factor are dynamically allocated. Specifically, for each enabled positioning source, a data quality index is calculated in real time. The data quality index includes the DOP and the number of visible satellites for satellite positioning sources, the base station GDOP (Geometric Dilution of Precision) and signal strength for cellular network positioning sources, the matching confidence for geomagnetic positioning sources, and the number of feature points and reprojection error for visual positioning sources. Based on the data quality index, the confidence of each positioning source is calculated, and the confidence is mapped to the fusion weight of the corresponding observation constraint factor.

[0096] The factor graph is iteratively optimized to output the optimal state estimate. Specifically, the GN (Gauss-Newton) iterative method or the LM (Levenberg-Marquardt) iterative method is used to perform nonlinear optimization on the factor graph. By minimizing the weighted sum of squared residuals of all constraint factors, the optimal estimate of each state node is obtained, and the target's position, attitude, and velocity information are output.

[0097] In one embodiment, anomaly adaptive processing is also included:

[0098] The system monitors the working status and output quality of each positioning source in real time. Specifically, it performs a consistency check on the output of each positioning source, compares the output of each positioning source with the fused prediction value, and determines that the positioning source is abnormal when the deviation exceeds a preset threshold.

[0099] When any location source is detected to be abnormal, the constraint factor fusion weight corresponding to that location source is reduced, and the constraint factor fusion weight of the remaining available location sources is increased accordingly. Specifically, the fusion weight of the abnormal location source is gradually reduced to near zero according to a preset attenuation coefficient to achieve soft isolation of the abnormal location source; at the same time, the released weight is redistributed to the remaining available location sources proportionally to ensure the sufficiency of fusion constraints.

[0100] Once the anomaly is resolved, the fusion weight configuration is restored to its pre-anomaly state. Specifically, after the output of the abnormal location source passes the consistency check again and continues to meet the preset time limit, its fusion weight is gradually restored to its pre-anomaly value, achieving a smooth reconnection of the location source.

[0101] In one embodiment, after completing the fusion calculation, the vehicle positioning system timestamps and fuses the following multi-module data information to form a structured data packet:

[0102] Real-time image data: Images of the vehicle's surrounding environment captured by the vehicle-mounted camera, including views of the road ahead and the scene in which the vehicle is located. The image data is compressed using H.264 / H.265 encoding to form a video stream with a frame rate of at least 15fps and a resolution of at least 720P. In low-light conditions, the image data also includes images enhanced with infrared illumination or image enhancement.

[0103] Fusion positioning results: High-precision location information output after factor map optimization in step S105, including latitude and longitude coordinates (WGS-84 coordinate system), heading angle, current speed, and current scene type identifier (outdoor open / semi-obstructed / indoor blind spot / low light / strong magnetic interference).

[0104] System status information: the current working status (primary positioning source / auxiliary positioning source / disabled) of each positioning source (inertial positioning source, satellite navigation positioning source, cellular network positioning source, geomagnetic positioning source, visual positioning source), the current fusion weight allocation ratio, and the data quality assessment results of each positioning source.

[0105] The fields in the structured data packet are synchronized and aligned using a unified timestamp with a timestamp accuracy of no less than milliseconds, ensuring consistency between the image and location information in the time dimension.

[0106] The structured data packets are transmitted in real time to the user's mobile terminal (mobile app) via the vehicle-mounted communication module. The vehicle-mounted communication module supports the following transmission methods:

[0107] Cellular network transmission: A long-term connection is established between the vehicle and the cloud server via a 4G / 5G cellular network. Structured data packets are uploaded to the cloud server, which then pushes them to the user's mobile app. This method is suitable for scenarios where the vehicle and the user are far apart, and the transmission delay is generally no more than 2 seconds.

[0108] LAN Direct Connection Transmission: A LAN direct connection channel is established between the vehicle and the user's mobile phone via an in-vehicle Wi-Fi hotspot or Bluetooth Low Energy (BLE) module, directly transmitting structured data packets to the mobile app. This method is suitable for scenarios where the user is near the vehicle (such as looking for their car in a parking lot), and the transmission latency can be controlled within 500 milliseconds.

[0109] During the transmission process, location information and system status information are serialized and encoded using lightweight JSON or Protocol Buffers format, and the video stream is streamed using RTP / RTSP protocol to balance transmission efficiency and real-time performance.

[0110] After receiving the structured data packet, the user's mobile app visualizes it using a combination of map overlay and real-time video, specifically including:

[0111] Map location display: The vehicle's location, as shown in the current fusion location result, is marked with a vehicle icon on an electronic map (such as Amap or Baidu Map) and refreshed in real time. The map interface also displays the current scene type identifier and location accuracy indicator (such as the estimated location error value).

[0112] Real-time video display: The real-time video stream transmitted from the vehicle's camera is simultaneously displayed in the split-screen area or floating window of the map interface, allowing users to intuitively view the environment around the vehicle.

[0113] System Status Panel: Displays the working status of each positioning source, fusion weight allocation, and data quality assessment results in the form of a dashboard or list, enabling users to understand the current operating status of the positioning system.

[0114] Historical trajectory playback: The app caches historical location data locally or pulls it from the cloud, allowing users to replay the vehicle's driving trajectory and image records for the corresponding time period.

[0115] Through the aforementioned real-time image transmission and remote visualization solution, users can view real-time images and precise locations of their vehicles in complex environments such as large parking lots and underground garages via an app, enabling rapid vehicle location. It also supports remote viewing of real-time images around the vehicle, facilitating vehicle anti-theft monitoring and abnormal status alerts. Furthermore, the integrated location results are displayed synchronously with real-time images, allowing users to intuitively verify the accuracy and reasonableness of the location results, enhancing system credibility. This data interaction architecture can also be further extended to fleet management scenarios, supporting centralized monitoring and dispatching of multiple vehicle locations and statuses.

[0116] In one embodiment, please refer to Figure 2 , Figure 2 This is a schematic diagram illustrating one embodiment of the multi-source data fusion positioning system disclosed in this application. This application provides a multi-source data fusion positioning system, including a multi-source sensing module 100, a positioning source construction module 200, a scene recognition module 300, and a fusion calculation module 400.

[0117] The multi-source sensing module 100 includes an inertial measurement unit, a satellite navigation receiving unit, a cellular network positioning unit, a geomagnetic acquisition unit, and a visual perception unit, which are used to acquire corresponding sensor data and perform preprocessing. Specifically, the inertial measurement unit acquires the target's three-axis acceleration and three-axis angular velocity data and performs noise reduction and zero-bias compensation; the satellite navigation receiving unit receives signals from the Global Navigation Satellite System and performs carrier-to-noise ratio filtering and multipath suppression; the cellular network positioning unit receives signals from 5G base stations and performs residual elimination; the geomagnetic acquisition unit measures the three-dimensional geomagnetic vector and performs low-pass filtering and hard magnetic compensation; and the visual perception unit acquires environmental images and performs distortion correction and exposure normalization.

[0118] The positioning source construction module 200 is used to independently solve each preprocessed data to construct the corresponding positioning source. Specifically, the positioning source construction module 200 performs pre-integration calculation on inertial measurement data to construct an inertial positioning source; performs pseudorange and carrier phase calculation on satellite navigation positioning data to construct a satellite navigation positioning source; performs time difference of arrival or angle of arrival calculation on cellular network positioning data to construct a cellular network positioning source; performs geomagnetic feature matching calculation on geomagnetic vector data to construct a geomagnetic positioning source; and performs feature extraction and visual odometry calculation on visual image data to construct a visual positioning source.

[0119] The scene recognition module 300 is used to identify the current scene type based on image data collected by the visual perception unit. Specifically, the scene recognition module 300 extracts environmental features from the visual image through a semantic segmentation network, and determines the current scene type as an outdoor open scene, a semi-obscured scene, an indoor blind spot scene, a low-light scene, or a strong magnetic interference scene based on the environmental features.

[0120] The fusion solution module 400 is used to determine the currently enabled combination of positioning sources and the working mode of each positioning source from the constructed positioning sources according to the scene type, perform weighted fusion solution based on the factor graph optimization framework, and output the position, attitude and velocity information of the target.

[0121] Please see Figure 3 , Figure 3 This is a schematic diagram illustrating another implementation of the multi-source data fusion positioning system disclosed in this application. In one embodiment, the fusion solution module 400 includes a quality assessment submodule 410, a weight allocation submodule 420, and an optimization solution submodule 430.

[0122] The quality assessment submodule 410 is used to assess the data quality of each positioning source in real time and output a confidence index. Specifically, the quality assessment submodule 410 performs real-time quality monitoring on the output of each positioning source, calculates the data quality index, and maps the data quality index to a normalized confidence index.

[0123] The weight allocation submodule 420 is used to dynamically allocate the fusion weights of each positioning source according to the scene type and the confidence index. Specifically, the weight allocation submodule 420 determines the baseline weight of each positioning source according to the scene type output by the scene recognition module 300, and then dynamically corrects the baseline weights in combination with the confidence index output by the quality assessment submodule 410 to obtain the final fusion weights of each positioning source.

[0124] The optimization solution submodule 430 is used to construct a factor graph and perform iterative optimization to output the optimal state estimate. Specifically, the optimization solution submodule 430 constructs a weighted factor graph based on the fused weights output by the weight allocation submodule 420, and uses a nonlinear optimization algorithm to perform iterative solution to output the optimal position, attitude, and velocity estimates of the target.

[0125] This application also provides a vehicle equipped with the aforementioned multi-source data fusion positioning system. The vehicle can be, but is not limited to, a pure electric vehicle, a hybrid vehicle, a range-extended electric vehicle, a plug-in hybrid vehicle, a new energy vehicle, or a gasoline vehicle. After being equipped with the aforementioned multi-source data fusion positioning system, the vehicle can continuously provide high-precision and high-reliability positioning services under various complex conditions such as urban roads, tunnels, underground parking garages, low-light environments, and areas with strong magnetic interference, meeting the stringent requirements of intelligent driving and autonomous navigation for positioning capabilities.

[0126] It should be noted that the vehicle can be, but is not limited to, a pure electric vehicle (PEV / BEV), a hybrid electric vehicle (HEV), a range-extended electric vehicle (REEV), a plug-in hybrid electric vehicle (PHEV), a new energy vehicle, or a gasoline vehicle.

[0127] Those skilled in the art will understand that the order of steps in the above embodiments is for ease of description only and does not constitute a limitation on the execution order. Where there is no conflict, the technical features in the various embodiments can be combined with each other.

[0128] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A positioning method based on multi-source data fusion, characterized in that, include: Real-time acquisition of target inertial measurement data, satellite navigation positioning data, cellular network positioning data, geomagnetic vector data, and visual image data, followed by preprocessing. Each preprocessed data point is solved independently to construct corresponding inertial positioning sources, satellite navigation positioning sources, cellular network positioning sources, geomagnetic positioning sources, and visual positioning sources. Based on the preprocessed visual image data, identify the current scene type; Based on the scenario type, determine the currently enabled combination of positioning sources and the working mode of each positioning source from the existing positioning sources; Based on the factor graph optimization framework, the inertial positioning source is incorporated into the factor graph in the form of motion constraints, and the solution results of the other enabled positioning sources are used as observation constraints for weighted fusion solution to output the target's position, attitude and velocity information.

2. The positioning method based on multi-source data fusion according to claim 1, characterized in that: The scene types include outdoor open scenes, semi-obstructed scenes, indoor blind spot scenes, low light scenes, and strong magnetic interference scenes. The process of identifying the current scene type based on the preprocessed visual image data specifically includes: extracting environmental features from the visual image through semantic segmentation, and determining the current scene type based on the environmental features.

3. The positioning method based on multi-source data fusion according to claim 2, characterized in that: The step of determining the currently activated combination of positioning sources and the working mode of each positioning source from the existing positioning sources according to the scenario type specifically includes: In open outdoor scenarios, satellite navigation positioning sources and cellular network positioning sources are the main positioning sources, while geomagnetic positioning sources, inertial positioning sources and visual positioning sources are the auxiliary positioning sources. In partially obscured scenarios, geomagnetic positioning sources, visual positioning sources, and inertial positioning sources are used as the main positioning sources, while cellular network positioning sources are used as auxiliary positioning sources. In indoor blind zone scenarios, geomagnetic positioning sources, visual positioning sources, and inertial positioning sources are the main positioning sources; In low-light scenarios, visual positioning sources are disabled, and satellite navigation positioning sources, cellular network positioning sources, geomagnetic positioning sources, and inertial positioning sources are used as the main positioning sources. In scenarios with strong magnetic interference, geomagnetic positioning sources are disabled, and satellite navigation positioning sources, cellular network positioning sources, visual positioning sources, and inertial positioning sources are used as the main positioning sources.

4. The positioning method based on multi-source data fusion according to claim 1, characterized in that, The step of independently solving each preprocessed data point to construct a corresponding location source specifically includes: Pre-integration calculations are performed on the inertial measurement data to construct an inertial positioning source; Pseudorange and carrier phase calculations are performed on satellite navigation and positioning data to construct a satellite navigation and positioning source; Calculate the time difference of arrival or angle of arrival for cellular network positioning data to construct a cellular network positioning source; Geomagnetic feature matching calculations are performed on geomagnetic vector data to construct a geomagnetic positioning source; Feature extraction and visual odometry calculation are performed on visual image data to construct a visual positioning source.

5. The positioning method based on multi-source data fusion according to claim 1, characterized in that, The weighted fusion solution based on the factor graph optimization framework specifically includes: In the factor graph, the inertial positioning source is represented as a motion constraint factor, which is used to connect state nodes at adjacent times; the other enabled positioning sources are represented as observation constraint factors, which are used to constrain the state estimation at the current time. Based on the scene type and the real-time quality assessment results of the localization source corresponding to each observation constraint factor, the fusion weight of each observation constraint factor is dynamically allocated. The factor graph is iteratively optimized to solve for the optimal state estimate.

6. The positioning method based on multi-source data fusion according to claim 5, characterized in that, It also includes adaptive exception handling: Real-time monitoring of the working status and output quality of each positioning source; When any location source is detected to be abnormal, the constraint factor fusion weight corresponding to that location source is reduced, and the constraint factor fusion weight of the other available location sources is increased accordingly. Once the anomaly is resolved, the fusion weight configuration will be restored to its state before the anomaly.

7. The positioning method based on multi-source data fusion according to claim 1, characterized in that, The preprocessing specifically includes: Denoising and zero-bias compensation are performed on the inertial measurement data; Perform carrier-to-noise ratio filtering and multipath suppression on satellite navigation and positioning data; Residual removal is performed on cellular network location data; Low-pass filtering and hard magnetic compensation are performed on the geomagnetic vector data; Distortion correction and exposure normalization are performed on visual image data.

8. A positioning system based on multi-source data fusion, characterized in that, include: The multi-source sensing module includes an inertial measurement unit, a satellite navigation receiving unit, a cellular network positioning unit, a geomagnetic acquisition unit, and a visual sensing unit, which are used to collect corresponding sensor data and perform preprocessing. The location source construction module is used to independently solve each preprocessed data and construct the corresponding location source. The scene recognition module is used to identify the current scene type based on the image data collected by the visual perception unit; The fusion solution module is used to determine the currently enabled combination of positioning sources and the working mode of each positioning source from the constructed positioning sources according to the scene type, perform weighted fusion solution based on the factor graph optimization framework, and output the position, attitude and velocity information of the target.

9. The positioning system based on multi-source data fusion according to claim 8, characterized in that, The fusion solution module includes: The quality assessment submodule is used to assess the data quality of each location source in real time and output confidence index. The weight allocation submodule is used to dynamically allocate the fusion weights of each positioning source according to the scene type and the confidence index. The optimization solution submodule is used to construct the factor graph and perform iterative optimization to output the optimal state estimate.

10. A vehicle, characterized in that: This includes a positioning system that integrates multi-source data as described in claim 8 or 9.

Images