Tunnel positioning method and device based on Beidou and visual target fusion and medium

Abstract

The invention relates to a tunnel positioning method and device based on Beidou and visual target fusion and a medium, and belongs to the technical field of high-precision positioning and navigation.According to the tunnel positioning method based on Beidou and visual target fusion, a unified tunnel coordinate system is established, and high-precision visual targets are arranged in a tunnel in the driving direction; an initial positioning result of a vehicle in a tunnel under a tunnel coordinate system is determined by identifying a target number and a pixel coordinate in the tunnel, a vehicle initial state vector of the vehicle under the tunnel coordinate system is determined according to the initial positioning result and a reference coordinate, IMU prediction and visual target prediction are carried out respectively, an IMU observation value and a visual observation value are obtained, and the vehicle initial state vector of the vehicle under the tunnel coordinate system is determined. IMU dead reckoning and visual observation are fused based on a weighted least square algorithm, centimeter-level continuous positioning is achieved, and continuous and high-precision positioning of the vehicle in the tunnel environment is achieved.

Description

Technical Field

[0001] This invention relates to the field of high-precision positioning and navigation technology, and in particular to a tunnel positioning method, device and medium based on the fusion of Beidou and visual targets. Background Technology

[0002] In smart highways and intelligent transportation systems, highway tunnels are a crucial component of the transportation network. However, the tunnel environment, characterized by its enclosed space, complex internal structure, and insufficient lighting, presents significant challenges to high-precision vehicle positioning. For ordinary vehicles, traditional positioning methods rely on the Global Navigation Satellite System (GNSS) after entering a tunnel. However, because tunnels are GNSS signal-denied environments, GNSS positioning methods become ineffective within tunnels, making it impossible for vehicles to rely on GNSS to calculate continuous and reliable vehicle position information.

[0003] In the absence of GNSS positioning services inside tunnels, vehicles typically rely on the following methods to maintain positioning: (1) Inertial Measurement Unit (IMU) dead reckoning: The relative position can be calculated in a short time by measuring the vehicle's motion state using onboard accelerometers and gyroscopes. However, due to accumulated errors, significant drift occurs in medium- to long-distance tunnels, and positioning accuracy drops rapidly. (2) Positioning based on prior maps: Matching high-precision maps with vehicle trajectories can improve positioning performance to some extent. However, prior maps require high-precision construction and real-time updates, and matching failures may occur when the vehicle trajectory does not perfectly match the map. (3) Simultaneous Localization and Mapping (SLAM): Real-time mapping and positioning are performed using tunnel structural feature points. However, the tunnel environment is sparse and dimly lit, resulting in insufficient effective features extracted by visual or lidar systems. When the number of feature points available for matching is lower than the threshold of the SLAM localization algorithm, the system will be unable to establish a stable correspondence between consecutive frames, resulting in a "tracking loss" phenomenon. Consequently, the vehicle's position cannot be calculated, and the vehicle cannot continuously obtain high-precision self-localization.

[0004] Therefore, in tunnel environments, vehicle positioning schemes relying on GNSS, IMU, prior maps, and visual / laser SLAM have the following shortcomings: (1) They cannot achieve continuous and high-precision positioning in tunnels; (2) IMU cumulative errors are large in long-distance tunnels, making it difficult to guarantee trajectory accuracy; (3) Map matching is limited by map quality and construction update cycle, making it difficult to meet real-time requirements.

[0005] Therefore, there is an urgent need to propose a technology that can provide high-precision positioning services for vehicle users in tunnels, in order to assist users in calculating the high-precision location information of vehicles in tunnel scenarios. Summary of the Invention

[0006] In view of this, it is necessary to provide a tunnel positioning method, device and medium based on the fusion of Beidou and visual targets, so as to achieve the purpose of calculating high-precision vehicle position information in tunnel scenarios.

[0007] To achieve the above objectives, in a first aspect, the present invention provides a tunnel positioning method based on the fusion of BeiDou and visual targets, comprising: Obtain the three-dimensional coordinates of GNSS control points outside the tunnel, and transform the three-dimensional coordinates of the GNSS control points to the inside of the tunnel to construct a tunnel coordinate system; Based on the parameter information of visual targets inside the tunnel, the initial positioning result of the vehicle inside the tunnel in the tunnel coordinate system is determined; the visual targets are staggered along the vehicle's direction of travel on both sides of the tunnel; the parameter information includes the target number, the target pixel coordinates, and the target three-dimensional coordinates. Based on the initial positioning results and reference coordinates, the initial state vector of the vehicle in the tunnel coordinate system is determined; the reference coordinates are determined based on BeiDou PPP positioning at the entrance. IMU prediction is performed based on the initial state vector to obtain IMU observations; Visual target prediction is performed based on the initial state vector to obtain visual observation values; The IMU observations and the visual observations are fused using a weighted least squares algorithm to obtain continuous positioning results for the vehicle in the tunnel.

[0008] In one possible implementation, determining the initial positioning result of the vehicle in the tunnel coordinate system based on the parameter information of the visual target inside the tunnel includes: The visual target is identified by a camera installed on the vehicle, and the target number is obtained. Based on the target number, extract the target's three-dimensional coordinates and the target's pixel coordinates; Based on the target's three-dimensional coordinates, the target's pixel coordinates, and camera intrinsic parameters, the three-dimensional relative position information between the vehicle and the visual target is determined; the three-dimensional relative position information includes horizontal distance, lateral offset, altitude difference, heading angle, and pitch angle; The three-dimensional relative position information is converted into vehicle three-dimensional coordinate information in the tunnel coordinate system based on the camera extrinsic parameters to obtain the initial positioning result.

[0009] In one possible implementation, the step of performing IMU prediction based on the initial state vector to obtain IMU observations includes: Based on the initial state vector and IMU measurement data, a kinematic prediction model for the vehicle is determined; the kinematic prediction model is used to determine the IMU prediction results; the IMU prediction results include the vehicle's prior state vector and covariance matrix. The IMU prediction results are converted into an equivalent pseudo-observation form to obtain the IMU observations; the IMU observations include the IMU observation residuals and the weight matrix of the IMU observations.

[0010] In one possible implementation, the visual observations include visual observation residuals and a weight matrix of visual observations; The step of predicting the visual target based on the initial state vector to obtain visual observations includes: Based on the transformation relationship between the vehicle coordinate system, camera coordinate system, and tunnel coordinate system, the imaging model corresponding to the initial state vector is determined; the imaging model is used to project the three-dimensional points corresponding to the initial state vector onto the pixel plane. Based on the target pixel coordinates and the imaging model, determine the pixel residual corresponding to each visual target; The pixel residuals corresponding to all identified visual targets are stitched together to obtain the visual observation residual; The measurement weight matrices corresponding to all identified visual targets are spliced ​​together to obtain the weight matrix of the visual observation.

[0011] In one possible implementation, the fusion of the IMU observations and the visual observations based on the weighted least squares algorithm to obtain continuous vehicle positioning results in the tunnel includes: Based on the IMU observations and the visual observations, a joint observation equation is constructed; The initial state vector is linearized to the first order to obtain the observation equation; Based on the weighted least squares principle, the joint observation equation, and the observation equation, an objective function is constructed. The objective function is solved to obtain the optimal increment of the current vehicle state; The continuous positioning result is obtained by updating the vehicle's state vector based on the optimal incremental update.

[0012] In one possible implementation, the objective function is expressed as follows:

[0013] in, Indicates the state increment. Represents the residuals of IMU observations. Represents the Jacobian matrix of IMU observations. The weight matrix representing IMU observations, Represents the residual of visual observations. The Jacobian matrix represents visual observation. The weight matrix represents visual observation.

[0014] In one possible implementation, the GNSS control points are located outside the exits at both ends of the tunnel; The GNSS control points are statically observed using a dual-frequency GNSS receiver, and their three-dimensional coordinates are determined by combining BeiDou PPP technology and CORS reference data.

[0015] In one possible implementation, a wireless broadcasting device is deployed inside the tunnel; The wireless broadcasting device is used to periodically broadcast data messages to the vehicle; the data messages include the target number, the target's three-dimensional coordinates, and the coordinate data update timestamp.

[0016] Secondly, the present invention also provides a tunnel positioning device based on the fusion of BeiDou and visual targets, comprising: A construction unit is used to acquire the three-dimensional coordinates of GNSS control points outside the tunnel and transform the three-dimensional coordinates of the GNSS control points into the tunnel interior to construct a tunnel coordinate system. An initial positioning unit is used to determine the initial positioning result of a vehicle in the tunnel coordinate system based on the parameter information of visual targets inside the tunnel. The visual targets are staggered on both sides of the tunnel along the vehicle's driving direction. The parameter information includes the target number, the target pixel coordinates, and the target three-dimensional coordinates. The determining unit is used to determine the vehicle's initial state vector in the tunnel coordinate system based on the initial positioning result and the reference coordinates; the reference coordinates are determined based on BeiDou PPP positioning at the entrance. The first prediction unit is used to perform IMU prediction based on the initial state vector to obtain IMU observations. The second prediction unit is used to predict the visual target based on the initial state vector to obtain the visual observation value. A continuous positioning unit is used to fuse the IMU observations and the visual observations based on a weighted least squares algorithm to obtain the continuous positioning results of the vehicle in the tunnel.

[0017] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the tunnel positioning method based on BeiDou and visual target fusion as described in any of the above implementations.

[0018] Fourthly, the present invention also provides a computer-readable storage medium for storing a computer-readable program or instruction, which, when executed by a processor, can implement the steps in the tunnel positioning method based on the fusion of BeiDou and visual targets described in any of the above implementations.

[0019] The beneficial effects of this invention are as follows: The tunnel positioning method, device, and medium based on the fusion of BeiDou and visual targets provided by this invention establish a unified tunnel coordinate system and deploy high-precision visual targets along the driving direction inside the tunnel. By identifying the target number and pixel coordinates inside the tunnel, the initial positioning result of the vehicle in the tunnel coordinate system is determined. Based on the initial positioning result and the reference coordinates, the initial state vector of the vehicle in the tunnel coordinate system is determined. IMU prediction and visual target prediction are performed respectively to obtain IMU observation values ​​and visual observation values. Then, based on the weighted least squares algorithm, IMU dead reckoning and visual observation are fused to achieve centimeter-level continuous positioning. This realizes continuous and high-precision positioning of vehicles in tunnel environments, and is especially suitable for high-precision vehicle navigation and positioning applications in scenarios such as highway tunnels, urban underground passages, and mountain tunnels. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic flowchart of one embodiment of the tunnel positioning method based on the fusion of BeiDou and visual targets provided by the present invention; Figure 2 This is a flowchart illustrating the vehicle visual target recognition and relative pose calculation method provided by the present invention. Figure 3 This is a flowchart illustrating the weighted least squares fusion localization method for IMU and visual observation provided by the present invention. Figure 4 This is a second schematic flowchart of an embodiment of the tunnel positioning method based on the fusion of BeiDou and visual targets provided by the present invention. Figure 5 A schematic diagram of an embodiment of the tunnel positioning device based on the fusion of BeiDou and visual targets provided by the present invention; Figure 6A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0023] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0024] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0025] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0026] This invention provides a tunnel positioning method, device, and medium based on the fusion of BeiDou and visual targets, which will be described below.

[0027] Figure 1 This is one of the flowcharts illustrating an embodiment of the tunnel positioning method based on BeiDou and visual target fusion provided by the present invention, as shown below. Figure 1 As shown, the tunnel positioning method based on the fusion of BeiDou and visual targets includes: S101. Obtain the three-dimensional coordinates of the GNSS control points outside the tunnel, and transform the three-dimensional coordinates of the GNSS control points to the inside of the tunnel to construct the tunnel coordinate system; S102. Based on the parameter information of the visual targets inside the tunnel, determine the initial positioning result of the vehicle inside the tunnel in the tunnel coordinate system; the visual targets are staggered on both sides of the tunnel along the vehicle's driving direction; the parameter information includes the target number, the target pixel coordinates, and the target three-dimensional coordinates. S103. Based on the initial positioning results and reference coordinates, determine the vehicle's initial state vector in the tunnel coordinate system; the reference coordinates are determined based on BeiDou PPP positioning at the entrance. S104. Based on the initial state vector, perform IMU prediction to obtain IMU observations; S105. Based on the initial state vector, perform visual target prediction to obtain visual observation values; S106. The IMU observations and the visual observations are fused based on the weighted least squares algorithm to obtain the continuous positioning results of the vehicle in the tunnel.

[0028] In S01, multiple BeiDou / GNSS control points are deployed outside the tunnel. PPP-RTK calculations are performed using BeiDou Precise Point Positioning (PPP) technology in conjunction with CORS reference stations to quickly obtain high-precision three-dimensional coordinates outside the tunnel. A total station is then used to import these external coordinates into the tunnel interior, accurately establishing a unified high-precision BeiDou spatial coordinate reference frame for the tunnel. The established local spatiotemporal coordinate system provides a unified coordinate benchmark for calculating the vehicle's position within the tunnel.

[0029] In tunnel S102, a pair of visual targets are deployed every 20 meters along the vehicle's direction of travel. The targets employ a high-contrast nine-square grid coding design, uniquely identifying their positions within a tunnel. High-precision three-dimensional coordinates of each target's center are determined using a surveying robot or total station, and the information for each target (including its number, pattern, and coordinates) is entered into the tunnel control center system, providing usable prior target position information for subsequent vehicle positioning.

[0030] Deploy wireless broadcasting devices, such as Wi-Fi or Bluetooth modules, inside the tunnel to package the information of each target (including its number, pattern, and coordinates) into a data message and periodically broadcast it to vehicles inside the tunnel via wireless signals. The broadcasting frequency can be set from 1Hz to 10Hz depending on the actual scenario, ensuring that vehicles can receive the target coordinate information as soon as they enter the tunnel, guaranteeing real-time performance and availability even in the absence of GNSS signals.

[0031] The vehicle is equipped with binocular cameras, an IMU, and a wireless communication module, which receive target information broadcast in the tunnel in real time during operation. The onboard camera identifies the visual target ahead, extracts the pixel coordinates of the target in the camera coordinate system, and combines them with the high-precision three-dimensional coordinates of the target stored in the tunnel control system. The relative positional relationship between the vehicle and the target is calculated using the principle of spatial intersection, thereby obtaining the initial positioning result of the vehicle in the tunnel.

[0032] In S103, when a vehicle enters the tunnel entrance, the reference coordinates obtained by BeiDou PPP positioning at the entrance are used, combined with the IMU attitude calculation results and camera extrinsic information, to determine the vehicle's initial state vector in the tunnel's unified coordinate system, including the position vector, velocity vector, and vehicle attitude vector.

[0033] In S104, within the tunnel, the distance between adjacent visual targets is typically 10–30 meters. To maintain continuous positioning, the vehicle primarily relies on IMUs for short-term dead reckoning between adjacent targets.

[0034] Based on the initial state vector and IMU measurement data, a kinematic prediction model for the vehicle is determined. This kinematic prediction model can determine the IMU prediction results, the vehicle's prior state vector, and the covariance matrix (IMU). The IMU prediction results are then converted into an equivalent pseudo-observation form to obtain the IMU observations, including the IMU observation residuals and the IMU observation weight matrix.

[0035] In S105, the imaging model corresponding to the initial state vector is determined based on the transformation relationship between the vehicle coordinate system, the camera coordinate system, and the tunnel coordinate system. The imaging model is used to project the three-dimensional points corresponding to the initial state vector onto the pixel plane.

[0036] The pixel residual corresponding to each visual target is determined based on the pixel coordinates of the target image and the imaging model. The pixel residuals corresponding to all identified visual targets are stitched together to obtain the visual observation residual. The measurement weight matrices corresponding to all identified visual targets are stitched together to obtain the visual observation weight matrix.

[0037] In S106, an IMU-visual observation fusion algorithm based on weighted least squares (WLS) is used. By utilizing both IMU dead reckoning results and visual target recognition results, the optimal vehicle pose estimation is achieved through optimized calculation. In the tunnel environment, the IMU provides high-frequency acceleration and angle observations of the vehicle, while the visual target observations provide periodic absolute position measurements to correct for IMU drift.

[0038] By integrating the absolute coordinate reference provided by BeiDou PPP, the relative position observations obtained through visual recognition, and the short-term high-frequency motion data of the IMU, and through information fusion processing from multiple sensors, it is possible to significantly suppress the cumulative drift of the IMU, while correcting noise or errors that may occur during the visual recognition process, ultimately achieving continuous high-precision positioning at the centimeter level.

[0039] In summary, the tunnel positioning method based on the fusion of BeiDou and visual targets provided in this embodiment of the invention establishes a unified tunnel coordinate system and deploys high-precision visual targets along the driving direction inside the tunnel. By identifying the target number and pixel coordinates inside the tunnel, the initial positioning result of the vehicle in the tunnel coordinate system is determined. Based on the initial positioning result and the reference coordinates, the initial state vector of the vehicle in the tunnel coordinate system is determined. IMU prediction and visual target prediction are performed respectively to obtain IMU observation values ​​and visual observation values. Then, based on the weighted least squares algorithm, IMU dead reckoning and visual observation are fused to achieve centimeter-level continuous positioning. This method realizes continuous and high-precision positioning of vehicles in tunnel environments, and is especially suitable for high-precision vehicle navigation and positioning applications in scenarios such as highway tunnels, urban underground passages, and mountain tunnels.

[0040] In some embodiments of the present invention, the GNSS control points are located outside the exits at both ends of the tunnel; The GNSS control points are statically observed using a dual-frequency GNSS receiver, and their three-dimensional coordinates are determined by combining BeiDou PPP technology and CORS reference data.

[0041] The tunnel control point data mainly includes: control point number, three-dimensional coordinates, GNSS status, and other information. Since GNSS signals cannot be effectively observed inside the tunnel, in order to achieve high-precision, continuous, and usable vehicle positioning within the tunnel, it is necessary to establish high-precision coordinate control points outside the tunnel and transfer the unified external coordinate reference to the tunnel interior, thereby providing a locally consistent high-precision coordinate system for target deployment and vehicle positioning.

[0042] For example, the establishment of tunnel control points and the establishment of the BeiDou spatiotemporal reference frame specifically include the following steps: (1) Control point layout and data collection.

[0043] Three stable, unobstructed control points were selected at each of the tunnel's two exits. Static observations were conducted using a dual-frequency GNSS receiver, and the three-dimensional coordinates of the control points in the CGCS2000 coordinate system were calculated using BeiDou PPP technology and CORS (Continuously Operating Reference Station) reference data, achieving a positioning accuracy better than ±3cm.

[0044] (2) High-precision coordinates are introduced into the tunnel.

[0045] Using a total station, the coordinates of control points outside the tunnel are gradually introduced into the tunnel through traverse surveying, angle observation, and distance transfer to establish a unified tunnel coordinate system. This ensures that the coordinates of the visual target inside the tunnel are consistent with the high-precision PPP coordinate reference outside the tunnel.

[0046] (3) Coordinate data entry and verification.

[0047] The coordinate data of external control points and tunnel entry points are entered into the tunnel control system for coordinate closure error and accuracy verification to ensure the reliability of the tunnel's internal reference coordinates. The coordinate accuracy of all control points and entry points is better than ±3cm, providing a high-precision spatiotemporal coordinate reference for subsequent target deployment and vehicle positioning.

[0048] In some embodiments of the present invention, a wireless broadcasting device is installed inside the tunnel; The wireless broadcasting device is used to periodically broadcast data messages to the vehicle; the data messages include the target number, the target's three-dimensional coordinates, and the coordinate data update timestamp.

[0049] Visual target data mainly includes target number, coding information, installation location, and three-dimensional coordinates. Due to the lack of satellite signals and stable feature points inside the tunnel, vehicles cannot achieve high-precision, continuous positioning solely based on IMU dead reckoning and prior maps.

[0050] To address this issue, the present invention deploys high-precision visual targets inside the tunnel along the vehicle's direction of travel. By identifying the targets using an onboard camera and combining their three-dimensional coordinates, periodic position information correction is achieved, ensuring the vehicle's continuous high-precision positioning capability within the tunnel.

[0051] For example, the design and placement of visual targets specifically includes the following steps: (1) Target design.

[0052] The visual target adopts a high-contrast matrix encoding scheme. The encoding mode can be a unique identifier encoding of a nine-square matrix or other machine vision-recognizable and robust encoding methods.

[0053] Each target has a unique number, ensuring that the target numbers identified by the vehicle at different locations are not repeated and are unique; the coding scheme has a high fault tolerance rate and can maintain stable recognition in complex tunnel environments such as insufficient lighting, partial occlusion, and low resolution; the target supports direct calculation of relative pose, and the vehicle position can be quickly calculated by combining the pixel coordinates collected by the vehicle camera with the three-dimensional coordinates of the target; with a conventional vehicle camera configuration, the effective target recognition distance can reach more than 30 meters, and the recognition angle range can cover horizontal ±45° and vertical ±30° to suit vehicle models with different heights and viewing angles.

[0054] (2) Target deployment.

[0055] Inside the tunnel, a pair of visual targets are placed every 20 meters along the direction of vehicle travel, staggered on both tunnel walls. The target height is set between 2.0 and 2.5 meters to ensure that the vehicle's camera can correctly identify the visual targets regardless of its height. The total number of targets is determined based on the tunnel length; for example, in a 2500-meter-long tunnel, approximately 125 pairs of targets are deployed to ensure that any vehicle can identify at least one target within a 30-meter radius in front of or behind it.

[0056] (3) Target coordinate measurement.

[0057] A total station or surveying robot is used to perform high-precision three-dimensional coordinate measurements on each target. The coordinates are unified under the local coordinate system of the tunnel, and the measurement accuracy of each target position is better than ±2cm.

[0058] After the measurement is completed, the target's status information (including number, installation location and three-dimensional coordinates) is entered into the tunnel control system database to provide prior coordinate data support for vehicle positioning and fusion calculation.

[0059] After a vehicle identifies a visual target ID within a tunnel, it needs to obtain the high-precision three-dimensional coordinates of each target in order to complete the vehicle's position intersection calculation. To solve this problem, this invention deploys a wireless broadcasting device within the tunnel, which broadcasts the target number and three-dimensional coordinates to the vehicle via low-latency communication, ensuring that the vehicle can obtain the prior coordinate information of the targets in real time.

[0060] For example, broadcasting tunnel target coordinate information specifically includes the following steps: (1) Deployment of broadcasting equipment.

[0061] Wireless broadcasting devices, such as Wi-Fi access points, Bluetooth beacons, or UWB base stations, are installed near the target inside the tunnel. The broadcasting devices are fixed at a height of approximately 2.5 meters above the ground, close to the visual target, to ensure that the vehicle can stably receive the corresponding coordinate information while recognizing the target.

[0062] (2) Data encapsulation.

[0063] Each wireless broadcasting device periodically broadcasts a data message containing the following information: the target's unique ID; the target's three-dimensional coordinates (X, Y, Z); and a timestamp indicating when the coordinate data was updated. The data is encapsulated in binary code, and each data packet is less than 128 bytes long.

[0064] (3) Broadcasting protocol and frequency.

[0065] Depending on the specific application scenario, this invention supports multiple communication methods, including Wi-Fi (2.4GHz / 5GHz), Bluetooth (BlueTeeth), or Ultra Wideband (UWB) communication. Wi-Fi is suitable for centralized architectures with large data volumes, while Bluetooth (BLE) offers low power consumption, wide coverage, and low cost, making it suitable for long tunnels. The broadcast frequency can be set from 1Hz to 10Hz depending on the tunnel scenario, with 5Hz recommended to ensure that the delay in receiving coordinate data when a vehicle enters the target recognition range does not exceed 200ms.

[0066] (4) Multi-node redundancy mechanism.

[0067] Real-time performance and reliability assurance: In order to ensure that the vehicle can receive target coordinates in real time while traveling at high speed, this invention designs a multi-node redundancy mechanism: there is cross-backup of coordinate information between adjacent target broadcasting devices. When a single broadcasting node fails, the vehicle can still obtain data through adjacent nodes to ensure that the positioning service is not interrupted.

[0068] (5) Data management and updates.

[0069] All target coordinate data are stored uniformly in the tunnel control center database. If tunnel maintenance or construction causes changes in the target position, the database can be updated in the background and synchronized to all broadcast nodes, achieving dynamic maintainability of the high-precision positioning service system throughout the tunnel.

[0070] In some embodiments of the present invention, determining the initial positioning result of a vehicle in the tunnel coordinate system based on the parameter information of a visual target inside the tunnel includes: The visual target is identified by a camera installed on the vehicle, and the target number is obtained. Based on the target number, extract the target's three-dimensional coordinates and the target's pixel coordinates; Based on the target's three-dimensional coordinates, the target's pixel coordinates, and camera intrinsic parameters, the three-dimensional relative position information between the vehicle and the visual target is determined; the three-dimensional relative position information includes horizontal distance, lateral offset, altitude difference, heading angle, and pitch angle; The three-dimensional relative position information is converted into vehicle three-dimensional coordinate information in the tunnel coordinate system based on the camera extrinsic parameters to obtain the initial positioning result.

[0071] The data for vehicle visual recognition and relative positioning mainly includes: the unique ID of the target observed at the current moment, its three-dimensional coordinates, the image pixel coordinates of the camera, camera parameters, and raw data from the vehicle IMU.

[0072] Due to the lack of GNSS signals inside the tunnel, vehicles must rely on onboard cameras to identify visual targets and combine the three-dimensional coordinates of the targets with the camera imaging model to calculate the precise position of the vehicle relative to the targets in real time, thereby obtaining the vehicle's position in the tunnel's unified coordinate system.

[0073] For example, vehicle visual recognition and relative positioning specifically include the following steps: (1) Configuration of vehicle-mounted sensing and communication equipment.

[0074] The vehicle is equipped with binocular or high-performance monocular cameras, an IMU sensor, a wireless communication module, and an onboard computing unit. The visual camera has a resolution of ≥1280×720, a frame rate of ≥30fps, and supports low-light imaging; the IMU includes an accelerometer and a gyroscope, with an output frequency of ≥100Hz; the wireless module supports Wi-Fi or Bluetooth communication and receives target information data in real time; the computing unit runs visual recognition and positioning algorithms and processes the visual images and IMU data acquired by the sensors in real time.

[0075] (2) Visual target detection and recognition.

[0076] After observing the visual targets, a visual recognition algorithm is used to detect the target area and extract the target's internal coding information. Upon successful target identification, the unique target ID is immediately read, and the high-precision 3D coordinates of the identified target are obtained from the tunnel information broadcasting system. During target installation in the tunnel, it is ensured that the vehicle-mounted camera can always observe at least four visual targets.

[0077] (3) Pixel coordinate extraction and spatial intersection.

[0078] After the vehicle identifies the target, it extracts the pixel coordinates of the target in the image. Combined with the calibrated camera intrinsic parameters (focal length, principal point coordinates, distortion coefficients, etc.), it uses the PnP (Perspective-n-Point) algorithm or spatial intersection method to calculate the three-dimensional relative positional relationship between the vehicle's camera and the target, including: horizontal distance (X direction); lateral offset (Y direction); height difference (Z direction); attitude information such as heading angle and pitch angle.

[0079] (4) Calculation of vehicle position.

[0080] Based on the camera's installation extrinsic parameters (arm parameters) relative to the vehicle, the relative position information between the vehicle and the target is converted into the vehicle's three-dimensional coordinates in the tunnel's unified coordinate system, forming the initial positioning solution for the vehicle.

[0081] In some embodiments of the present invention, the step of performing IMU prediction based on the initial state vector to obtain IMU observations includes: Based on the initial state vector and IMU measurement data, a kinematic prediction model for the vehicle is determined; the kinematic prediction model is used to determine the IMU prediction results; the IMU prediction results include the vehicle's prior state vector and covariance matrix. The IMU prediction results are converted into an equivalent pseudo-observation form to obtain the IMU observations; the IMU observations include the IMU observation residuals and the weight matrix of the IMU observations.

[0082] Inside the tunnel, where GNSS signals cannot be received, real-time vehicle positioning employs a weighted least squares (WLS) fusion filtering framework combining IMU prediction and visual target observation. During vehicle operation, continuous and stable centimeter-level high-precision position, velocity, and attitude information is acquired through recursive calculations by the IMU's inertial navigation orchestration system and periodic corrections by visual observations.

[0083] This invention ensures that the vehicle can simultaneously observe no fewer than four targets at any location by deploying high-density targets, so the IMU only undertakes the short-term prediction function between targets.

[0084] For example, IMU and visual multi-source fusion localization specifically includes the following steps: (1) Initial state establishment.

[0085] When a vehicle enters the tunnel entrance, the initial state vector of the vehicle in the tunnel's unified coordinate system is determined using the reference coordinates obtained from the BeiDou PPP positioning at the entrance, the IMU attitude calculation results, and the camera extrinsic parameter information. :

[0086] in, For position vectors, For velocity vectors, This is the vehicle attitude vector.

[0087] The vehicle's initial state vector This provides initial state conditions for subsequent fusion and solution.

[0088] (2) IMU short-cycle range estimation.

[0089] Inside the tunnel, the distance between adjacent visual targets is typically 10–30 meters. To maintain continuous positioning, the vehicle relies primarily on IMUs for short-term dead reckoning between adjacent targets. By integrating the specific force and angular velocity data collected by the IMUs, the predicted values ​​of the vehicle's position, velocity, and attitude in the tunnel coordinate system can be obtained.

[0090] Vehicles in the era The state vector is The IMU measurement during this period was The kinematic prediction model of the vehicle can be obtained as follows:

[0091]

[0092]

[0093] in, Let be the rotation matrix from the vehicle body coordinate system to the tunnel local coordinate system. Let be the gravitational acceleration vector, and ⊕ be the attitude update operator, using quaternion multiplication.

[0094] The prior state of the vehicle is obtained based on this model. And propagate the covariance matrix:

[0095] in, Here is the state transition matrix. Let be the noise covariance matrix of the discrete process.

[0096] To suppress drift in subsequent fusion, IMU predictions can be converted into equivalent pseudo-observations to obtain IMU observations. and ),as follows:

[0097]

[0098] The above formula can be used as a constraint term in the weighted least squares solution, so that the vehicle can obtain continuous and smooth positioning results during the visual observation interval.

[0099] In some embodiments of the present invention, the visual observations include visual observation residuals and a weight matrix of visual observations; The step of predicting the visual target based on the initial state vector to obtain visual observations includes: Based on the transformation relationship between the vehicle coordinate system, camera coordinate system, and tunnel coordinate system, the imaging model corresponding to the initial state vector is determined; the imaging model is used to project the three-dimensional points corresponding to the initial state vector onto the pixel plane. Based on the target pixel coordinates and the imaging model, determine the pixel residual corresponding to each visual target; The pixel residuals corresponding to all identified visual targets are stitched together to obtain the visual observation residual; The measurement weight matrices corresponding to all identified visual targets are spliced ​​together to obtain the weight matrix of the visual observation.

[0100] Inside the tunnel, the vehicle can obtain the three-dimensional coordinates of each visual target in the tunnel coordinate system by decoding the messages received in the tunnel. The vehicle can identify visual targets in the tunnel using YOLO (Yarn Origin and Out) detection in a single frame image. The onboard vision sensor detects the corner features of the target through an image recognition module and performs three-dimensional geometric inversion based on camera intrinsic and extrinsic parameters.

[0101] Figure 2 This is a flowchart illustrating the vehicle visual target recognition and relative pose calculation method provided by the present invention, as shown below. Figure 2 As shown, the method for vehicle visual target recognition and relative pose calculation is as follows.

[0102] The system defines a vehicle volume coordinate system (b-frame), a camera coordinate system (c-frame), and a tunnel global coordinate system (w-frame), and their relationships are as follows:

[0103] in, This is the rotation matrix corresponding to the vehicle's attitude. This is the rotation matrix from the camera to the vehicle. This represents the position of the origin of the vehicle body coordinate system within the tunnel coordinate system. This is the vehicle position vector.

[0104] The imaging model can be represented as: ,in, For the camera intrinsic parameter matrix, This is a perspective projection function that projects 3D points onto a pixel plane.

[0105] The actual target image obtained by detection has the following pixel count: Then the pixel residual is defined as:

[0106] When the vehicle is detected at the same time When there are multiple targets, all residuals can be concatenated into a joint observation vector (i.e., visual observations):

[0107]

[0108] in, For target The measurement weight matrix can be adaptively determined based on the observation pixel accuracy, target distance, and viewing angle.

[0109] The vehicle camera pose can be estimated by minimizing the sum of squared pixel residuals.

[0110] When acquiring a single frame of observation, the least squares solution method based on the perspective n-point (PnP) model can be used to solve for the attitude and position of the vehicle (or camera) relative to the tunnel coordinate system.

[0111] The PnP problem can be represented as:

[0112] The rotation matrix can be solved using least squares optimization. With translation vector This allows for the acquisition of the relative pose estimation of the vehicle-mounted vision sensor camera in the local coordinate system of the tunnel.

[0113] To improve robustness, this invention introduces a RANSAC filtering strategy in the PnP solution to eliminate abnormal matching points caused by false detections and occlusions in image recognition. When multiple targets are identified simultaneously, joint solutions can be performed, and the error amplification effect of a single target can be suppressed through multi-observation redundancy.

[0114] After calculating the camera pose, the three-dimensional position and orientation of the vehicle's coordinate system origin are calculated based on the extrinsic parameters of the vision sensor installation. This yields the vehicle's absolute pose in the tunnel's unified coordinate system, as detailed below:

[0115]

[0116] in, This indicates the conversion from rotation matrices to Euler angles or quaternions.

[0117] Therefore, in the frame where the vehicle recognizes the visual target, the absolute pose coordinates can be directly output; while in the interval between two observations of a new target, the vehicle state is maintained by short-cycle calculations by the IMU.

[0118] In some embodiments of the present invention, the fusion of the IMU observations and the visual observations based on the weighted least squares algorithm to obtain continuous positioning results of the vehicle in the tunnel includes: Based on the IMU observations and the visual observations, a joint observation equation is constructed; The initial state vector is linearized to the first order to obtain the observation equation; Based on the weighted least squares principle, the joint observation equation, and the observation equation, an objective function is constructed. The objective function is solved to obtain the optimal increment of the current vehicle state; The continuous positioning result is obtained by updating the vehicle's state vector based on the optimal incremental update.

[0119] In some embodiments of the present invention, the expression of the objective function is as follows:

[0120] in, Indicates the state increment. Represents the residuals of IMU observations. Represents the Jacobian matrix of IMU observations. The weight matrix representing IMU observations, Represents the residuals of visual observations. The Jacobian matrix represents visual observation. The weight matrix represents visual observation.

[0121] To achieve continuous high-precision vehicle positioning in tunnel environments, this invention proposes an IMU-visual observation fusion algorithm based on Weighted Least Squares (WLS). This algorithm simultaneously utilizes IMU dead reckoning results and visual target recognition results at each epoch, optimizing the calculation to achieve the optimal estimation of the vehicle's pose.

[0122] Figure 3 The flowchart of the weighted least squares fusion localization method for IMU and visual observation provided by this invention is shown below. Figure 3 As shown, the weighted least squares fusion localization method of IMU and visual observation is described below.

[0123] In tunnel environments, IMUs provide high-frequency acceleration and angle observations of vehicles, but these accumulate over time; visual target observations, on the other hand, provide periodic absolute position measurements to correct for IMU drift.

[0124] By modeling the two types of information in a unified coordinate system, a joint observation equation can be formed:

[0125]

[0126] in, For IMU observation residuals, For visual observation residuals, and These are the corresponding weight matrices, used to reflect the confidence levels of the IMU and visual sensor data sources.

[0127] For vehicle state vector After performing first-order linearization, the observation equation is obtained:

[0128] in, For the joint Jacobian matrix, For state increment, This is the noise term.

[0129] Based on the principle of weighted least squares, define the objective function as follows:

[0130] in, Indicates the state increment. Represents the residuals of IMU observations. Represents the Jacobian matrix of IMU observations. The weight matrix representing IMU observations, Represents the residuals of visual observations. The Jacobian matrix represents visual observation. The weight matrix represents visual observation.

[0131] Taking the partial derivative of the objective function and setting it to zero, we obtain the normal equation:

[0132] in, , , .

[0133] By solving this system of equations, the optimal increment of the vehicle state at the current epoch can be obtained in real time. .

[0134] Solving for the results Then, the vehicle state vector and covariance matrix can be updated:

[0135]

[0136] in, It is a nonlinear state update operator used for incremental correction of attitude parameters; This is the posterior covariance matrix, used to represent the solution accuracy.

[0137] To mitigate the impact of abnormal observations on the fusion results, this invention introduces a robust weight adjustment mechanism into the WLS algorithm. When a target observation suffers from problems such as insufficient illumination, occlusion, or identification errors, its residual is too large, and the system automatically reduces its weight.

[0138] The weight update rule can be expressed as:

[0139] in, The Huber robust loss function (or Cauchy function) can adaptively reduce the impact of outlier observations.

[0140] When the vehicle loses visual observation for a short period of time (such as when the target is temporarily invisible), the system automatically enters the IMU inertial prediction mode to maintain continuous positioning; when a new target is re-identified, the fusion positioning calculation is immediately triggered to correct the cumulative drift in the previous period and achieve seamless trajectory connection.

[0141] By using weighted least squares fusion of IMU measurements and visual observations, this invention enables centimeter-level continuous positioning in tunnel environments without GNSS signals. The IMU provides high-frequency dynamic response, while visual observations provide absolute position constraints. The complementary information from both effectively suppresses accumulated errors and ensures the stability and continuity of the positioning results.

[0142] This invention achieves centimeter-level continuous positioning in tunnel environments without GNSS signals through five steps: BeiDou PPP benchmark establishment, target visual recognition, IMU dead reckoning, and weighted least squares fusion. It provides a feasible technical solution for high-precision vehicle navigation in complex environments.

[0143] Figure 4 This is a second schematic diagram of an embodiment of the tunnel positioning method based on BeiDou and visual target fusion provided by the present invention, as shown below. Figure 4 As shown, this invention provides a high-precision tunnel positioning method based on BeiDou and target visual recognition. This method involves deploying visual targets within the tunnel and obtaining their high-precision coordinates. A unified coordinate system for the tunnel is established using BeiDou PPP technology, and the target information is transmitted to the vehicle via a wireless broadcasting device. The vehicle identifies the target ID through its onboard camera, and, combined with the onboard IMU and BeiDou PPP data, uses a multi-source fusion algorithm to calculate the vehicle's precise position in the tunnel. This achieves centimeter-level continuous positioning even in the absence of GNSS signals in the tunnel, providing stable and reliable high-precision positioning services for vehicles in highway tunnels.

[0144] The specific steps are as follows: S401, Tunnel control points and BeiDou PPP coordinates established.

[0145] GNSS control points were deployed outside the tunnel, and high-precision three-dimensional coordinates of the tunnel exterior were obtained by combining BeiDou PPP technology with CORS reference stations. These high-precision coordinates were then transferred into the tunnel interior using a total station to establish a unified coordinate system for the tunnel.

[0146] S402, Visual Target Design and Deployment.

[0147] Inside the tunnel, a pair of high-precision visual targets are deployed every 20 meters along the direction of vehicle travel. The targets use a high-contrast nine-square grid coding design and have unique numbers. The coordinates of the targets are accurately measured using a total station and entered into the tunnel control system.

[0148] S403, broadcasting of tunnel target coordinate information.

[0149] Deploy wireless broadcasting devices, such as Wi-Fi or Bluetooth modules, inside the tunnel. The system encapsulates each target's number and high-precision coordinates into a data message and broadcasts it periodically via wireless signal. The broadcasting frequency can be set from 1Hz to 10Hz depending on the actual scenario, ensuring that vehicles can obtain target information in real time.

[0150] S404, Vehicle visual recognition and relative positioning.

[0151] The vehicle is equipped with a vision camera, IMU, and wireless communication module, and receives target coordinate information in real time after entering the tunnel. By identifying the target ID and obtaining its pixel coordinates through the camera, and combining this with the known three-dimensional coordinates of the target, the vehicle's position and attitude information can be obtained through intersection calculation, thus acquiring the vehicle's initial state information in the local coordinate system of the tunnel.

[0152] S405, IMU and Beidou PPP integrated positioning.

[0153] Based on the extended Kalman filter (EKF), the system integrates the high-precision coordinate reference of BeiDou PPP, the relative position observation of visual recognition, and the observation data of IMU to achieve multi-source information complementarity and output continuous high-precision positioning results of vehicles in tunnels.

[0154] This invention significantly improves the high continuity, high reliability, and high precision positioning capabilities of vehicles in tunnel environments by introducing the BeiDou PPP coordinate reference, visual target recognition, and IMU multi-source fusion. At the same time, the system server architecture is simple and flexible in deployment, and can stably output centimeter-level high-precision positioning results under conditions without GNSS signals, which has high practicality and promotion value.

[0155] This invention provides a high-precision tunnel positioning method based on the fusion of BeiDou satellite navigation system and target visual recognition. This method addresses the problem of vehicle positioning interruption caused by GNSS signal obstruction in tunnels. It establishes a unified coordinate system using PPP technology and periodically deploys uniquely numbered high-precision visual targets along the driving direction inside the tunnel. The system uses a total station to measure the three-dimensional coordinates of the targets and records them in a database. The target coordinate information is periodically broadcast to the vehicle via a wireless broadcasting device (Wi-Fi, Bluetooth, or UWB). The vehicle is equipped with a camera, IMU, and communication module. Inside the tunnel, it visually recognizes the target ID and pixel coordinates, combining this with prior three-dimensional coordinates to calculate the vehicle's relative pose. Then, based on a weighted least squares algorithm, it fuses IMU dead reckoning with visual observation to achieve centimeter-level continuous positioning. When visual observation is lost, the IMU maintains short-period predictions, and automatically corrects drift upon visual recovery. This invention achieves continuous, high-precision, and recoverable vehicle positioning in tunnel environments. It has the advantages of simple deployment, strong robustness, and high computational efficiency, and is especially suitable for high-precision vehicle navigation and positioning applications in scenarios such as highway tunnels, urban underground passages, and mountain tunnels.

[0156] The present invention proposes the following improvements: (1) Tunnel absolute coordinate reference construction technology. By using the BeiDou PPP and total station joint transmission method, a unified high-precision coordinate system can be established inside the tunnel, so that the coordinate system of the visual target in this invention, the vehicle-mounted visual sensor, and the IMU can be unified under the same local coordinate system.

[0157] With independently encoded visual targets as reliable features in feature-deficient environments (tunnels), the problem of SLAM failure caused by low texture and missing feature points and lines in tunnels is solved, and a controllable, stable and repeatable source of visual observation is achieved.

[0158] (2) Wireless active broadcasting mechanism of target three-dimensional coordinates: the vehicle does not need to store the map locally or build a point cloud. It can obtain the precise position of the observable visual target by receiving broadcast decoding for positioning.

[0159] (3) IMU-vision is based on a weighted least squares fusion framework to ensure that the system can rely on IMU to solve the problem without diverging in the short term when vision fails. At the same time, it suppresses the observation anomalies of the vision sensor caused by occlusion and abnormal lighting through robust weights.

[0160] (4) Automatic backtracking correction mechanism after visual recovery: When the target is observed again, the system automatically corrects the IMU drift to maintain the continuity and stability of the trajectory.

[0161] The present invention has the following advantages: (1) Strong continuity: When the target is not visible for a short time, the IMU dead reckoning can maintain continuous trajectory output to ensure that the positioning is not interrupted; (2) High robustness: The estimation strategy based on residual adaptive weighting effectively suppresses abnormal observations caused by changes in illumination, occlusion and identification errors; (3) Good recoverability: After visual observation is recovered, it can automatically backtrack and correct IMU drift, maintaining the stability and consistency of the trajectory; (4) Positioning accuracy does not diverge: Through sliding window smoothing and dynamic weight management, it is ensured that the positioning error in the tunnel does not increase with the accumulation of vehicle travel distance, and the overall accuracy can be stably maintained at the centimeter level; (5) Excellent real-time performance: The weighted least squares algorithm has low computational complexity and can run in real time in vehicle embedded systems, meeting the latency requirements of high-speed driving scenarios.

[0162] In summary, this invention achieves high-precision continuous positioning in tunnels without GNSS signals. It has the advantages of simple engineering implementation, low computational cost, and strong scalability, and is suitable for environments without GNSS signal observation, such as highway tunnels, urban underground passages, and mountain tunnels.

[0163] To better implement the tunnel positioning method based on BeiDou and visual target fusion in the embodiments of the present invention, based on the tunnel positioning method based on BeiDou and visual target fusion, correspondingly, as follows: Figure 5 As shown, this embodiment of the invention also provides a tunnel positioning device based on the fusion of BeiDou and visual targets. The tunnel positioning device 500 based on the fusion of BeiDou and visual targets includes: The construction unit 501 is used to obtain the three-dimensional coordinates of GNSS control points outside the tunnel and transform the three-dimensional coordinates of the GNSS control points into the tunnel interior to construct the tunnel coordinate system. The initial positioning unit 502 is used to determine the initial positioning result of the vehicle in the tunnel coordinate system based on the parameter information of the visual targets inside the tunnel; the visual targets are staggered on both sides of the tunnel along the vehicle's driving direction; the parameter information includes the target number, the target pixel coordinates, and the target three-dimensional coordinates. The determining unit 503 is used to determine the vehicle's initial state vector in the tunnel coordinate system based on the initial positioning result and the reference coordinates; the reference coordinates are determined based on the BeiDou PPP positioning at the entrance. The first prediction unit 504 is used to perform IMU prediction based on the initial state vector to obtain IMU observations. The second prediction unit 505 is used to predict the visual target based on the initial state vector to obtain the visual observation value. The continuous positioning unit 506 is used to fuse the IMU observations and the visual observations based on the weighted least squares algorithm to obtain the continuous positioning results of the vehicle in the tunnel.

[0164] The tunnel positioning device 500 based on the fusion of BeiDou and visual targets provided in the above embodiments can realize the technical solutions described in the above embodiments of the tunnel positioning method based on the fusion of BeiDou and visual targets. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the tunnel positioning method based on the fusion of BeiDou and visual targets, which will not be repeated here.

[0165] like Figure 6 As shown, the present invention also provides an electronic device 600. The electronic device 600 includes a processor 601, a memory 602, and a display 603. Figure 6 Only some components of the electronic device 600 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0166] In some embodiments, processor 601 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 602 or process data, such as the tunnel positioning method based on BeiDou and visual target fusion in this invention.

[0167] In some embodiments, processor 601 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 601 may be local or remote. In some embodiments, processor 601 may be implemented on a cloud platform. In some embodiments, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, internal cloud, multi-cloud, or any combination thereof.

[0168] In some embodiments, memory 602 may be an internal storage unit of electronic device 600, such as a hard disk or memory of electronic device 600. In other embodiments, memory 602 may also be an external storage device of electronic device 600, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 600.

[0169] Furthermore, the memory 602 may include both internal storage units of the electronic device 600 and external storage devices. The memory 602 is used to store application software and various types of data installed on the electronic device 600.

[0170] In some embodiments, display 603 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an organic light-emitting diode (OLED) touchscreen. Display 603 is used to display information from electronic device 600 and to display a visual user interface. Components 601-603 of electronic device 600 communicate with each other via a system bus.

[0171] In one embodiment, when the processor 601 executes the tunnel positioning program based on the fusion of BeiDou and visual targets stored in the memory 602, the following steps can be implemented: Obtain the three-dimensional coordinates of GNSS control points outside the tunnel, and transform the three-dimensional coordinates of the GNSS control points to the inside of the tunnel to construct a tunnel coordinate system; Based on the parameter information of visual targets inside the tunnel, the initial positioning result of the vehicle inside the tunnel in the tunnel coordinate system is determined; the visual targets are staggered along the vehicle's direction of travel on both sides of the tunnel; the parameter information includes the target number, the target pixel coordinates, and the target three-dimensional coordinates. Based on the initial positioning results and reference coordinates, the initial state vector of the vehicle in the tunnel coordinate system is determined; the reference coordinates are determined based on BeiDou PPP positioning at the entrance. IMU prediction is performed based on the initial state vector to obtain IMU observations; Visual target prediction is performed based on the initial state vector to obtain visual observation values; The IMU observations and the visual observations are fused using a weighted least squares algorithm to obtain continuous positioning results for the vehicle in the tunnel.

[0172] It should be understood that when the processor 601 executes the tunnel positioning program based on the fusion of Beidou and visual targets in the memory 602, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0173] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 600 mentioned. Electronic device 600 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 600 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0174] Accordingly, embodiments of the present invention also provide a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the tunnel positioning method based on BeiDou and visual target fusion provided in the above-described method embodiments.

[0175] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0176] The tunnel positioning method, device, and medium based on the fusion of BeiDou and visual targets provided by this invention have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this invention. Therefore, the content of this specification should not be construed as a limitation of this invention.

Claims

1. A tunnel positioning method based on the fusion of BeiDou and visual targets, characterized in that, include: Obtain the three-dimensional coordinates of GNSS control points outside the tunnel, and transform the three-dimensional coordinates of the GNSS control points to the inside of the tunnel to construct a tunnel coordinate system; Based on the parameter information of visual targets inside the tunnel, the initial positioning result of the vehicle inside the tunnel in the tunnel coordinate system is determined; the visual targets are staggered along the vehicle's direction of travel on both sides of the tunnel; the parameter information includes the target number, the target pixel coordinates, and the target three-dimensional coordinates. Based on the initial positioning results and reference coordinates, the initial state vector of the vehicle in the tunnel coordinate system is determined; the reference coordinates are determined based on BeiDou PPP positioning at the entrance. IMU prediction is performed based on the initial state vector to obtain IMU observations; Visual target prediction is performed based on the initial state vector to obtain visual observation values; The IMU observations and the visual observations are fused using a weighted least squares algorithm to obtain continuous positioning results for the vehicle in the tunnel.

2. The tunnel positioning method based on BeiDou and visual target fusion according to claim 1, characterized in that, The determination of the initial positioning result of the vehicle in the tunnel coordinate system based on the parameter information of the visual target inside the tunnel includes: The visual target is identified by a camera installed on the vehicle, and the target number is obtained. Based on the target number, extract the target's three-dimensional coordinates and the target's pixel coordinates; Based on the target's three-dimensional coordinates, the target's pixel coordinates, and camera intrinsic parameters, the three-dimensional relative position information between the vehicle and the visual target is determined; the three-dimensional relative position information includes horizontal distance, lateral offset, altitude difference, heading angle, and pitch angle; The three-dimensional relative position information is converted into vehicle three-dimensional coordinate information in the tunnel coordinate system based on the camera extrinsic parameters to obtain the initial positioning result.

3. The tunnel positioning method based on BeiDou and visual target fusion according to claim 1, characterized in that, The step of performing IMU prediction based on the initial state vector to obtain IMU observations includes: Based on the initial state vector and IMU measurement data, a kinematic prediction model for the vehicle is determined; the kinematic prediction model is used to determine the IMU prediction results; the IMU prediction results include the vehicle's prior state vector and covariance matrix. The IMU prediction results are converted into an equivalent pseudo-observation form to obtain the IMU observations; the IMU observations include the IMU observation residuals and the weight matrix of the IMU observations.

4. The tunnel positioning method based on BeiDou and visual target fusion according to claim 1, characterized in that, The visual observations include visual observation residuals and visual observation weight matrices; The step of predicting the visual target based on the initial state vector to obtain visual observations includes: Based on the transformation relationship between the vehicle coordinate system, camera coordinate system, and tunnel coordinate system, the imaging model corresponding to the initial state vector is determined; the imaging model is used to project the three-dimensional points corresponding to the initial state vector onto the pixel plane. Based on the target pixel coordinates and the imaging model, determine the pixel residual corresponding to each visual target; The pixel residuals corresponding to all identified visual targets are stitched together to obtain the visual observation residual; The measurement weight matrices corresponding to all identified visual targets are spliced ​​together to obtain the weight matrix of the visual observation.

5. The tunnel positioning method based on BeiDou and visual target fusion according to claim 1, characterized in that, The method of fusing the IMU observations and the visual observations based on the weighted least squares algorithm to obtain the continuous positioning results of the vehicle in the tunnel includes: Based on the IMU observations and the visual observations, a joint observation equation is constructed; The initial state vector is linearized to the first order to obtain the observation equation; Based on the weighted least squares principle, the joint observation equation, and the observation equation, an objective function is constructed. The objective function is solved to obtain the optimal increment of the current vehicle state; The continuous positioning result is obtained by updating the vehicle's state vector based on the optimal incremental update.

6. The method according to claim 5, characterized in that, The expression for the objective function is as follows: in, Indicates the state increment. Represents the residuals of IMU observations. Represents the Jacobian matrix of IMU observations. The weight matrix representing IMU observations, Represents the residuals of visual observations. The Jacobian matrix represents visual observation. The weight matrix represents visual observation.

7. The tunnel positioning method based on BeiDou and visual target fusion according to any one of claims 1 to 6, characterized in that, The GNSS control points are located outside the exits at both ends of the tunnel; The GNSS control points are statically observed using a dual-frequency GNSS receiver, and their three-dimensional coordinates are determined by combining BeiDou PPP technology and CORS reference data.

8. The tunnel positioning method based on BeiDou and visual target fusion according to any one of claims 1 to 6, characterized in that, A wireless broadcasting device is installed inside the tunnel. The wireless broadcasting device is used to periodically broadcast data messages to the vehicle; the data messages include the target number, the target's three-dimensional coordinates, and the coordinate data update timestamp.

9. A tunnel positioning device based on the fusion of BeiDou and visual targets, characterized in that, include: A construction unit is used to acquire the three-dimensional coordinates of GNSS control points outside the tunnel and transform the three-dimensional coordinates of the GNSS control points into the tunnel interior to construct a tunnel coordinate system. An initial positioning unit is used to determine the initial positioning result of a vehicle in the tunnel coordinate system based on the parameter information of visual targets inside the tunnel. The visual targets are staggered on both sides of the tunnel along the vehicle's driving direction. The parameter information includes the target number, the target pixel coordinates, and the target three-dimensional coordinates. The determining unit is used to determine the vehicle's initial state vector in the tunnel coordinate system based on the initial positioning result and the reference coordinates; the reference coordinates are determined based on BeiDou PPP positioning at the entrance. The first prediction unit is used to perform IMU prediction based on the initial state vector to obtain IMU observations. The second prediction unit is used to predict the visual target based on the initial state vector to obtain the visual observation value. A continuous positioning unit is used to fuse the IMU observations and the visual observations based on a weighted least squares algorithm to obtain the continuous positioning results of the vehicle in the tunnel.

10. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the tunnel positioning method based on BeiDou and visual target fusion as described in any one of claims 1 to 8.

Images