Wireless-based visual positioning system and method for automotive vehicles

By combining radio receiver and image acquisition features, identifying signal reflectors, and using LOS-NLOS correlation and Bayesian filtering, the problem of inaccurate vehicle positioning in complex environments is solved, achieving accurate pose estimation and supporting autonomous driving and traffic perception.

CN116558531BActive Publication Date: 2026-04-17GM GLOBAL TECHNOLOGY OPERATIONS LLC
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
GM GLOBAL TECHNOLOGY OPERATIONS LLC
Filing Date
2022-10-14
Publication Date
2026-04-17

AI Technical Summary

Technical Problem

Existing vehicle positioning systems lack positioning accuracy in complex environments, especially where buildings and reflective surfaces are present. Multipath effects lead to signal interference and inaccurate positioning.

Method used

By combining radio receivers, image acquisition features, and sensor fusion technology, and by identifying signal reflectors and visual features, using LOS-NLOS correlation and Bayesian filters, GPS, IMU sensor, and cloud computing location data are fused to generate accurate vehicle pose.

Benefits of technology

It improves positioning accuracy in complex environments, solves the signal interference problem caused by multipath effects, provides accurate pose in environments that reject global navigation satellite systems, and supports autonomous driving and traffic perception systems.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN116558531B_ABST
    Figure CN116558531B_ABST
Patent Text Reader

Abstract

A wireless-based visual positioning system for an automotive vehicle includes an automotive vehicle having a radio receiver. A map contains candidate locations of access points (APs) and media access control (MAC) IDs corresponding to the access points. A wireless distance sensor determines different distances to each detected AP visible to the automotive vehicle. An image acquisition feature recognition identifies image data visible to the automotive vehicle. A real-time feature matching element matches features identified by the image acquisition feature recognition to data from the map. A filter receives output from the real-time feature matching element to generate a pose of the automotive vehicle.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] introduction

[0002] This disclosure relates to a vehicle positioning system using wireless technology. Background Technology

[0003] Wireless signals and visual features are used separately for positioning and mapping. Positioning using wireless signals typically requires accurate mapping of the wireless infrastructure before use. Global Positioning System (GPS) operation using wireless signals for vehicles, including automobiles, provides the wireless infrastructure, but this can be negatively affected by environmental conditions, including buildings, structures, reflective surfaces, etc. If the vehicle environment contains negative environmental conditions that reduce the accurate use of wireless signals, the precise position or pose of the vehicle is necessary.

[0004] Multipath propagation is also known to degrade the performance of wireless-based positioning systems. In wireless and radio communications, multipath propagation is a phenomenon that causes a signal to reach the receiving antenna via two or more paths. Causes of multipath include atmospheric waveguides, ionospheric reflection and refraction, and reflection from bodies of water and land objects such as mountains and buildings. When the same signal is received via more than one path, multipath reception can produce interference and phase shift in the received signal, and therefore, using the received signal can generate inaccurate vehicle locations. Destructive interference causes attenuation, which can cause the radio signal to become too weak to be adequately received in some areas.

[0005] Therefore, while current vehicle positioning systems have achieved their intended purpose, a new and improved vehicle positioning system is needed. Summary of the Invention

[0006] According to several aspects, a visual wireless-based positioning system for an automotive vehicle includes an automotive vehicle equipped with a radio receiver. A map contains candidate locations of access points (APs) and corresponding Media Access Control (MAC) IDs for the APs, and the map identifies signal reflectors. A wireless range sensor determines different distances to various detected APs visible to the automotive vehicle. Image acquisition features identify image data visible to the automotive vehicle. A real-time feature matching element matches features identified by the image acquisition features with data from the map. A filter receives the output from the real-time feature matching element to generate the automotive vehicle pose.

[0007] In another aspect of this disclosure, MAC association is performed on each distance among the different distances; and the sensor fusion module fuses positioning inputs from the GPS receiver, IMU sensor and cloud computing location to produce a final output of precise location that defines the vehicle's pose.

[0008] In another aspect of this disclosure, LOS-NLOS correlation is performed on the output from MAC correlation to resolve ambiguity between multiple AP locations, thereby determining which identified AP is the "real" AP.

[0009] In another aspect of this disclosure, the data acquisition features include operational and location data from at least one of a vehicle inertial measurement unit (IMU), a vehicle wheel speed sensor (WSS), and a global positioning system (GPS) device.

[0010] In another aspect of this disclosure, the image feature extraction function receives image data to determine whether the database content can be trusted to obtain credible and reliable information.

[0011] In another aspect of this disclosure, the image acquisition features include at least one of a camera and a LiDAR component located in the vehicle.

[0012] In another aspect of this disclosure, the map includes: the location of a signal reflector that defines a surface from which a wireless signal can be reflected; and semantic data that identifies roads, buildings, and intersections.

[0013] In another aspect of this disclosure, the radio receiver provides distance measurements as line-of-sight (LOS) or non-line-of-sight (NLOS) measurements to different APs.

[0014] In another aspect of this disclosure, the map further includes image features and their coordinates developed from systems including, for example, Scale Invariant Feature Transform (SIFT) and Speed-Up Robust Feature (SURF).

[0015] In another aspect of this disclosure, the signal reflector includes at least one of a first surface defining the surface of a sign, a second surface defining the wall of a building, and a third surface defining a visual feature.

[0016] According to several aspects, a wireless-based vision positioning system for an automotive vehicle includes an automotive vehicle having a radio receiver and a transmitter. A map containing candidate locations of access points (APs) also includes reflectors that identify signals. The vehicle's onboard positioning system acquires wireless positioning measurements from detected APs. The transmitter operates to transmit the wireless positioning measurements to the cloud-edge in real time. A sensor fusion module fuses positioning inputs from sources including a GPS receiver, an IMU sensor, and cloud-based location data returned from the cloud-edge to produce a final, precise location output defining the vehicle's pose.

[0017] In another aspect of this disclosure, the MAC address is associated with the detected AP in the AP, and the detected AP in the AP is associated with the candidate location of the access point AP.

[0018] In another aspect of this disclosure, the predicted vehicle pose and the position of signal reflectors from the map are used to correlate non-line-of-sight (NLOS) and line-of-sight (LOS) measurements.

[0019] In another aspect of this disclosure, if the signal reflector falls between the predicted vehicle pose and the mapped AP location, the longer distance measurement is associated with the mapped AP location such that the longer distance measurement of the mapped AP defines the reflection.

[0020] In another aspect of this disclosure, if the signal reflector does not fall between the predicted vehicle pose and the AP location marked on the map, the shorter distance measurement is associated with the AP marked on the map, and the AP marked on the map defines the LOS feature.

[0021] In another aspect of this disclosure, the cloud-based localization group includes a Bayesian filter that obtains data from a sample point cloud, a set of global virtual APs, and a set of planar surface models.

[0022] In another aspect of this disclosure, a single measurement of the wireless positioning signal data is passed to a Bayesian filter, wherein the output of the Bayesian filter defines an initial position estimate of the vehicle.

[0023] According to several aspects, a method for determining the location of a vehicle with a radio receiver includes: identifying candidate locations of access points (APs) and corresponding Media Access Control (MAC) IDs on a map; downloading the identities of signal reflectors from the map; using a wireless distance sensor to determine different distances of various detected APs visible to the vehicle; using image acquisition features to acquire image data visible to the vehicle; matching features identified by the image acquisition features with data from the map; and sending the output from a real-time feature matching element to a filter to generate the vehicle pose.

[0024] In another aspect of this disclosure, the method further includes: performing an activation operation immediately after the vehicle is activated; sampling particles from previous operations of the vehicle to establish an initial map reference; loading planar reflectors previously identified during sampling from the map reference; and obtaining the path length and power of the access point (AP) from the map reference.

[0025] In another aspect of this disclosure, the method further includes: inputting the LOS and NLOS AP locations onto a map reference; associating the AP path length and power based on the AP's MACID; and updating the AP's particle weights, which are selected from a plurality of particle weights having the highest probability of being a non-reflector.

[0026] This invention includes at least the following solutions:

[0027] Solution 1. A wireless-based visual positioning system for automobiles, the wireless-based visual positioning system for automobiles comprising:

[0028] A car vehicle having a radio receiver;

[0029] The map includes candidate locations of access points (APs) and the media access control (MAC) IDs corresponding to the access points, and the map also identifies signal reflectors;

[0030] A wireless distance sensor that determines different distances to various detected APs visible to the vehicle.

[0031] Image acquisition features, wherein the image acquisition features identify image data visible to the vehicle;

[0032] A real-time feature matching element that matches features identified by the image acquisition features with data from the map; and

[0033] A filter that receives output from the real-time feature matching element to generate the vehicle pose.

[0034] Option 2. The wireless-based visual positioning system for automobiles according to Option 1, the wireless-based visual positioning system further includes: MAC association for each of the different distances; and a sensor fusion module, the sensor fusion module being used to fuse positioning inputs from a GPS receiver, an IMU sensor, and a cloud computing location, the sensor fusion module generating a final output of a precise position that defines the pose of the automobile.

[0035] Option 3. The wireless-based visual positioning system for automobiles according to Option 2, the wireless-based visual positioning system further includes performing LOS-NLOS association on the output from the MAC association to resolve ambiguity among multiple AP locations, thereby determining which identified AP is the "real" AP.

[0036] Option 4. The wireless-based visual positioning system for automobiles according to Option 3, wherein the wireless-based visual positioning system further includes data acquisition features, wherein the data acquired by the data acquisition features includes operational and position data from at least one of a vehicle inertial measurement unit (IMU), a vehicle wheel speed sensor (WSS), and a global positioning system (GPS) device.

[0037] Option 5. The wireless-based visual positioning system for automobiles according to Option 1, wherein the wireless-based visual positioning system further includes an image feature extraction function, wherein the image feature extraction function receives image data to determine whether the database content can be trusted to obtain credible and reliable information.

[0038] Option 6. The wireless-based visual positioning system for an automobile vehicle according to Option 1, wherein the image acquisition features are defined as at least one of a camera and a LIDAR component located in the automobile vehicle.

[0039] Option 7. The wireless-based visual positioning system for automobiles according to Option 1, wherein the map includes: the location of a signal reflector defining a surface from which wireless signals can be reflected; and semantic data that identifies roads and intersections.

[0040] Option 8. The wireless-based visual positioning system for automobiles according to Option 1, wherein the radio receiver provides distance measurements as line-of-sight (LOS) or non-line-of-sight (NLOS) measurements to different access points (APs).

[0041] Option 9. A wireless-based visual positioning system for automobiles according to Option 1, wherein the map further comprises image features and their coordinates developed from a system including Scale Invariant Feature Transform (SIFT) and Speed-Up Robust Features (SURF).

[0042] Option 10. The wireless-based visual positioning system for an automobile vehicle according to Option 1, wherein the signal reflector includes at least one of the following: a first surface defining a sign, a second surface defining a building wall, and a third surface defining a visual feature.

[0043] Solution 11. A wireless-based visual positioning system for automobiles, the wireless-based visual positioning system comprising:

[0044] A car vehicle having a radio receiver and a transmitter;

[0045] A map containing multiple candidate locations of access points (APs), and the map also identifying signal reflectors;

[0046] The vehicle-mounted positioning system of the automobile collects wireless positioning measurement values ​​from the detected AP among the plurality of APs;

[0047] The transmitter operates to transmit the wireless positioning measurements to the cloud-edge in real time; and

[0048] A sensor fusion module is used to fuse positioning inputs from sources to generate a final, precise location output that defines the vehicle's pose. The sources include a GPS receiver, an IMU sensor, and a cloud computing location returned from the cloud-edge.

[0049] Solution 12. The wireless-based visual positioning system for automobiles according to Solution 11, the wireless-based visual positioning system further includes a MAC address associated with a detected AP among the plurality of APs, and associates the detected AP among the plurality of APs with the candidate locations of the plurality of APs.

[0050] Option 13. The wireless-based visual positioning system for an automobile vehicle according to Option 12, the wireless-based visual positioning system further includes using a predicted vehicle pose and the position of the signal reflector from the map to correlate non-line-of-sight (NLOS) and line-of-sight (LOS) measurements.

[0051] Solution 14. The wireless-based visual positioning system for a vehicle according to Solution 13, wherein if a signal reflector falls between the predicted vehicle pose and the location of a first AP marked on a map, a longer distance measurement is associated with the first AP marked on the map, such that the first AP marked on the map limits the reflection.

[0052] Option 15. The wireless-based visual positioning system for a vehicle according to Option 14, wherein if the signal reflector does not fall between the predicted vehicle pose and the second AP marked on the map, the shorter distance measurement among the distance measurements is associated with the second AP marked on the map, and the second AP marked on the map defines a LOS feature.

[0053] Option 16. The wireless-based visual positioning system for automobiles according to Option 11, wherein the wireless-based visual positioning system further includes a cloud-based positioning group, the cloud-based positioning group including a Bayesian filter, the Bayesian filter obtaining data from a sample point cloud, a set of global virtual APs and a set of planar surface models.

[0054] Option 17. The wireless-based visual positioning system for an automobile according to Option 16, the wireless-based visual positioning system further includes a single measurement of wireless positioning signal data being passed to the Bayesian filter, wherein the output of the Bayesian filter defines an initial position estimate of the automobile.

[0055] Option 18. A method for determining the location of a vehicle having a radio receiver, the method comprising:

[0056] Identify candidate locations of multiple access points (APs) and their corresponding Media Access Control (MAC) IDs on a map; and

[0057] Download the identity of the signal reflector from the map;

[0058] A wireless distance sensor is used to determine the different distances of each detected AP among the plurality of APs visible to the vehicle.

[0059] Image acquisition features are used to obtain image data visible to the vehicle.

[0060] The features identified by the image acquisition features are matched with data from the map; and

[0061] The output from the real-time feature matching element is sent to the filter to generate the vehicle pose.

[0062] Option 19. The method according to Option 18, wherein the method further comprises:

[0063] The start-up operation is performed immediately after the vehicle is connected.

[0064] Particles from previous operations of the vehicle are sampled to establish an initial map reference;

[0065] Load the planar reflectors previously identified during sampling from the map reference; and

[0066] The AP path length and power of the multiple APs are obtained from the map reference.

[0067] Option 20. The method according to Option 19, wherein the method further comprises:

[0068] Input the LOS and NLOS AP locations into the map reference;

[0069] The AP's path length and power are associated based on its MAC ID; and

[0070] Update the particle weights of the plurality of APs, wherein the particle weights are selected from the plurality of particle weights that have the highest probability of being non-reflective.

[0071] Further areas of applicability will become apparent from the description provided herein. It should be understood that the descriptions and specific examples are intended for illustrative purposes only and not to limit the scope of this disclosure. Attached Figure Description

[0072] The accompanying drawings described herein are for illustrative purposes only and are not intended to limit the scope of this disclosure in any way.

[0073] Figure 1 This is an illustration of a wireless-based visual positioning system based on an exemplary aspect;

[0074] Figure 2 It is presented for determining Figure 1 A flowchart of the process steps for determining the pose of a vehicle in a system;

[0075] Figure 3 It is presented for use Figure 1 A diagram illustrating the steps involved in using cloud-based data from the system to perform a real-time location process;

[0076] Figure 4 From Figure 3 The modified diagram is used to illustrate the real-time location process with cloud data and filtering located in the vehicle.

[0077] Figure 5It is a graph showing the particles output from the particle filter, which identifies... Figure 1 The reflected surface on the system;

[0078] Figure 6 It is a presentation Figure 5 The diagram showing the weighted effect of particles presented in the image; and

[0079] Figure 7 It is a flowchart showing the process steps used to determine the vehicle's location. Detailed Implementation

[0080] The following description is exemplary in nature and is not intended to limit this disclosure, application, or use.

[0081] refer to Figure 1 According to several aspects, a wireless-based visual positioning system 10 is used to determine the position or pose of a vehicle 12 on a map 14. The vehicle 12 is equipped with a radio receiver 16, such as, but not limited to, WiFi Functional Tone Management (FTM), 5G, etc. The environment in which the vehicle 12 operates may impede the performance of the Global Positioning System (GPS). The map 14 contains candidate locations of access points (APs) and their corresponding Media Access Control (MAC) IDs. The locations of potential signal reflectors are identified, defining surfaces from which wireless signals can be reflected, such as a first surface 18 defining the surface of a sign 20, a second surface 22 such as the wall of a building 24, objects 26 (such as trees), etc. The map 14 may further include image features and their coordinates developed from systems such as Scale Invariant Feature Transform (SIFT). The map 14 further includes other relevant semantic data that identifies, for example, roads, signs, intersections, etc. The radio receiver 16 can also provide distance measurements to different APs; however, due to the aforementioned signal reflectors, multiple distances can be reported, and the measurements can be provided as line-of-sight (LOS) or non-line-of-sight (NLOS) measurements.

[0082] refer to Figure 2 And refer again Figure 1Flowchart 28 provides the process steps for operating the wireless-based vision positioning system 10. In a first process 30, a wireless distance sensor 32, including an AoA sensor 32, is used to determine different distances to various detected APs visible to the vehicle 12. In a second or MAC association process 34, MAC association is performed for each of the different distances. In a third process 36, a LOS-NLOS association 38 is then performed on the output from the MAC association process 34. The LOS-NLOS association 38 resolves ambiguities between multiple AP locations to determine which identified AP is the “real” AP, which defines an AP that excludes data from reflections or data returned from reflectors. Example ambiguities may include reflectors located between the determined vehicle location and the map-recognized AP. Such reflectors may be ignored as reflections rather than real or true vehicle APs. The output from the LOS-NLOS association 38 is fed into a filter 40.

[0083] Parallel to the first process 30, map 14 is accessed to identify the AP location and ID of the item output from the first process 30 for use in the MAC association process 34. Map 14 is also accessed to identify one or more reflectors input into the LOS-NLOS association 38.

[0084] Parallel to the first process 30, an image detection process 42 identifies image data from image acquisition features, including the vehicle 12's camera 44, LiDAR (Light Detection and Ranging) component 46, etc. This image data is then forwarded to an image feature extraction process 48, which may use, for example, a Scale Invariant Feature Transform (SIFT) system algorithm 50 to extract visual features from the image. The output from the image feature extraction process 48 is forwarded to a feature matching process 52, which matches the features identified by the camera 44, LiDAR component 46, etc., with data from the map 14 in real time. The output from the feature matching process 52 is then forwarded to a filter 40.

[0085] Further vehicle data is acquired in data acquisition feature or process 54 to assist in identifying the pose of vehicle 12. This data includes operational and positional data, such as from vehicle inertial measurement unit (IMU) 56, vehicle wheel speed sensor (WSS) 58, or global positioning system (GPS) device 60. This data is forwarded directly to filter 40. The output of filter 40 defines vehicle pose 62. Vehicle pose 62 can also be returned to third process 36 and LOS-NLOS association process 38 for further iteration.

[0086] The detected wireless access point (AP) is associated with the AP marked on the map using its MAC address. The vehicle pose 62 of vehicle 12 and data from the reference... Figure 1 The location of the reflective surface in the described map 14 is used to associate NLOS / LOS measurements. If the reflector falls between the predicted vehicle pose and the AP location marked on the map, the longer distance measurement is associated with the AP, such that the longer distance measurement of the AP defines the reflection. If the reflector does not fall between the predicted vehicle pose and the AP location marked on the map, the shorter of the two distance measurements is associated with the AP, and the AP defines the LOS feature.

[0087] refer to Figure 3 And refer again Figure 1 and Figure 2 The vehicle positioning system 64 of the vehicle 12 uses a nearby WiFi AP to collect wireless positioning measurements 66 and uses a transmitter to transmit the measurements to the cloud / edge 68 in real time. Then, the vehicle positioning system 64 uses a sensor fusion module 70 to fuse positioning inputs from multiple sources (including a GPS receiver 72, an inertial measurement unit (IMU) sensor 74, cloud computing location 76, etc.) to produce a final output of a precise position with a defined pose 62.

[0088] A cloud-based positioning group 80 is used to assist the vehicle's onboard positioning system 64. Wireless positioning measurements 66 are grouped and transferred as measurement groups 78 for real-time transfer to the cloud / edge 68. The cloud-based positioning group 80 includes a Bayesian filter 82 that takes data from a sample point cloud 84, a set of global virtual access points 86, and a set of planar surface models 88. Individual measurements of wireless positioning signal data 90 are also passed to the Bayesian filter 82. The output of the Bayesian filter 82 defines an initial position estimate 92 for the vehicle 12. After multiple iterations of the wireless positioning signal via the cloud-based positioning group 80, the precise position 94 of the vehicle 12 is returned from the cloud-based positioning group 80 to the vehicle's onboard positioning system 64 to assist in determining the pose 62.

[0089] refer to Figure 4 And refer again Figures 1 to 3 References can also be used. Figure 4 The cloud-based positioning group 96, modified relative to cloud-based positioning group 80, performs real-time vehicle localization. Bayesian filter 82 is removed from cloud-based positioning group 96 and localized in vehicle-mounted positioning system 98 along with a download module including sample point cloud 84a, a set of global virtual APs 86a, and a set of planar surface models 88a. Vehicle-mounted positioning system 98 with Bayesian filter 82 then functions similarly to vehicle-mounted positioning system 64 in other respects.

[0090] refer to Figure 5 And refer again Figures 1 to 4 On the cloud side, such as in cloud positioning group 80, the real-time positioning process receives data from vehicle 12 (in...). Figure 1 , Figure 3 and Figure 4 The wireless positioning measurements (shown in the image) are used to estimate the position of the vehicle 12 (not shown in this view), and a particle filter (such as a Bayesian filter 82) is employed. Figure 5 The diagram further illustrates a road 100 having multiple reflectors positioned along it, including a first reflective surface 102, a first structure 104 with a second reflective surface 106, a third reflective surface 108, and a fourth reflective surface, such as a designated parking sign. Along the vehicle's direction of travel 112, a first set of hollow dots 114 represents particles from a particle filter (such as a Bayesian filter 82), and a second set of solid dots 116 represents samples from a sample location database 118. As shown in Equation 1, the particle weights uj can be calculated as follows:

[0091] Equation 1:

[0092] .

[0093] For Equation 1, the function d() is a function that returns the distance between si and uj; the function Parallel Distributed Processing (PDP) PDP_diff() returns the difference between si and uj.

[0094] refer to Figure 6 And refer again Figure 5 Figure 120 shows the power 122 in dBm relative to the delay time 124 in microseconds. The first curve 126 represents the PDP distribution of si in Equation 1, and the second curve 128 represents the PDP collected in real time from the vehicle 12. The function PDP_diff() in Equation 1 returns the cumulative difference between the first curve 126 and the second curve 128.

[0095] Continue to refer to Figure 5 and Figure 6 If particle uj is very close to sample si, and uj and si have very similar power delay profiles, then si will contribute a higher weight to uj.

[0096] refer to Figure 7 And refer again Figures 1 to 6 The process flowchart 130 illustrates the operational steps for subsequent operations of the wireless-based visual positioning system 10. For example, in reference... Figure 1Following the activation operation of the described vehicle 12, a startup operation 132 is performed. In sampling operation 134, particles from previous operations of the system are sampled to establish an initial map reference. In the first loading operation 136, planar reflectors previously identified in sampling operation 134 are loaded from the map. In acquisition operation 138, the path length and power of the identified APs are acquired. In the second loading operation 140, the LOS and NLOS AP locations are loaded. In association operation 142, the AP measurements loaded in the previous acquisition operation 138 are associated with the map information based on the MAC ID. In update operation 144, particle weights are updated using the particle weights with the highest probability of being non-reflectors. In query operation 146, a query is performed if particle evaluation from update operation 144 is complete. If the response 148 from query operation 146 is negative (NO), the program continues with resampling operation 150, where particles are resampled and propagated. After resampling operation 150, the program returns to the first loading operation 136. If the response 152 from query operation 146 is YES, the program terminates at termination operation 154.

[0097] According to the first aspect, the autonomous driving system (ADS) includes a cloud-based or local map that provides relevant features that can be matched in real time to determine the pose of the vehicle 12 in an environment where a Global Navigation Satellite System (GNSS) is not available.

[0098] According to the second aspect, the precise pose of the vehicle 12 provided in a GNSS-rejected environment using a vehicle-to-vehicle (V2V) system provides the level of accuracy required for V2V applications.

[0099] According to the third aspect, for traffic perception systems, precise pose 62 can be used to learn traffic conditions on specific roads in GPS-free environments. This data can be shared with a cloud-based interface and used to determine new vehicle routes to avoid congested areas.

[0100] The wireless-based visual positioning system disclosed herein offers several advantages. These advantages include a system in which algorithms provide positioning in an outdoor environment using a combination of visual features and wireless signals. Multipath effects are taken into account by identifying and modeling reflection sources using a map (such as map 14). Accurate positioning or pose 62 is modeled using distance measurements while taking into account multipath effects and reflections. Visual features are utilized to identify reflections and moving objects in the environment during map marking. Visual features are used in conjunction with wireless measurements to help create a consistent map. A set of positioning algorithms utilizes a planar model in a joint visual / wireless map to estimate multipath effects and improve positioning accuracy.

[0101] The description in this disclosure is exemplary in nature only, and variations thereof without departing from the spirit and scope of this disclosure are intended to fall within its scope. Such variations shall not be considered as departing from the spirit and scope of this disclosure.

Claims

1. A wireless-based visual positioning system for an automobile, the wireless-based visual positioning system for the automobile comprising: A car vehicle having a radio receiver; The map contains candidate locations of APs and the MAC IDs corresponding to the APs, and the map also identifies signal reflectors; A wireless distance sensor that determines different distances to various detected APs visible to the vehicle. Image acquisition features, wherein the image acquisition features identify image data visible to the vehicle; A real-time feature matching element that matches features identified by the image acquisition features with data from the map; and A filter that receives output from the real-time feature matching element to generate the vehicle pose.

2. The wireless-based visual positioning system for automotive vehicles of claim 1, further comprising: MAC association performed for each of the different distances; And a sensor fusion module, which fuses positioning inputs from a GPS receiver, an IMU sensor, and a cloud computing location, and generates a final, precise position output that defines the vehicle's pose.

3. The wireless-based vision positioning system for an automotive vehicle according to claim 2, the wireless-based vision positioning system further comprising performing line-of-sight (LOS)-non-line-of-sight (NLOS) correlation on the output from the MAC association to resolve ambiguity among multiple AP locations, thereby determining which identified AP is the "real" AP.

4. The wireless-based visual positioning system for automotive vehicles of claim 3, further comprising a data acquisition feature, wherein, The data acquired by the data acquisition features includes operational and location data from at least one of the vehicle inertial measurement unit (IMU), vehicle wheel speed sensor (WSS), and global positioning system (GPS) devices.

5. The wireless-based visual positioning system for automobiles according to claim 1, wherein the wireless-based visual positioning system further includes an image feature extraction function, wherein the image feature extraction function receives image data to determine whether the database content can be trusted to obtain credible and reliable information.

6. The wireless-based visual positioning system for automotive vehicles of claim 1, wherein, The image acquisition feature is defined as at least one of a camera and a LiDAR component located in the vehicle.

7. The wireless-based visual positioning system for automotive vehicles of claim 1, wherein, The map includes: the location of signal reflectors defining surfaces from which wireless signals can be reflected; and semantic data that identifies roads and intersections.

8. The wireless-based visual positioning system for automotive vehicles of claim 1, wherein, The radio receiver provides distance measurements as either line-of-sight (LOS) or non-line-of-sight (NLOS) measurements to different APs.

9. The wireless-based visual positioning system for automotive vehicles of claim 1, wherein, The map further includes image features and their coordinates developed from a system that includes Scale Invariant Feature Transform (SIFT) and Speed-Up Robust Feature Transform (SURF).

10. The wireless-based visual positioning system for automotive vehicles of claim 1, wherein, The signal reflector includes at least one of the following: a first surface defining the surface of a sign, a second surface defining the wall of a building, and a third surface defining a visual feature.

11. A wireless-based visual positioning system for an automobile, the wireless-based visual positioning system comprising: A car vehicle having a radio receiver and a transmitter; A map containing multiple candidate locations of APs, and the map also identifying signal reflectors; The vehicle-mounted positioning system of the automobile collects wireless positioning measurement values ​​from the detected AP among the plurality of APs; The transmitter operates to transmit the wireless positioning measurements to the cloud-edge in real time; as well as A sensor fusion module is used to fuse positioning inputs from sources to generate a final, precise location output that defines the vehicle's pose. The sources include a GPS receiver, an IMU sensor, and a cloud computing location returned from the cloud-edge.

12. The wireless-based visual positioning system for an automobile vehicle according to claim 11, the wireless-based visual positioning system further includes a MAC address associated with a detected AP among the plurality of APs, and associates the detected AP among the plurality of APs with the candidate locations of the plurality of APs.

13. The wireless-based visual positioning system for an automobile vehicle according to claim 12, the wireless-based visual positioning system further comprising using a predicted vehicle pose and the position of the signal reflector from the map to correlate non-line-of-sight (NLOS) and line-of-sight (LOS) measurements.

14. The wireless-based visual positioning system for automotive vehicles of claim 13, wherein, If the signal reflector falls between the predicted vehicle pose and the location of the first AP marked on the map, the longer distance measurement is associated with the first AP marked on the map, such that the first AP marked on the map limits the reflection.

15. The wireless-based visual positioning system for automotive vehicles of claim 14, wherein, If the signal reflector does not fall between the predicted vehicle pose and the second AP marked on the map, the shorter distance measurement is associated with the second AP marked on the map, and the second AP marked on the map defines the line-of-sight (LOS) feature.

16. The wireless-based visual positioning system for automobiles according to claim 11, the wireless-based visual positioning system further comprising a cloud-based positioning group, the cloud-based positioning group comprising a Bayesian filter, the Bayesian filter acquiring data from a sample point cloud, a set of global virtual APs and a set of planar surface models.

17. The wireless-based visual positioning system of an automotive vehicle of claim 16, further comprising a single measurement of wireless positioning signal data passed to the Bayesian filter, wherein, The output of the Bayesian filter defines the initial position estimate of the vehicle.

18. A method for determining the location of a vehicle having a radio receiver, the method comprising: Identify candidate locations of multiple access points (APs) and their corresponding MACIDs on the map; as well as Download the identity of the signal reflector from the map; A wireless distance sensor is used to determine the different distances of each detected AP among the plurality of APs visible to the vehicle. Image acquisition features are used to obtain image data visible to the vehicle. The features identified by the image acquisition features are matched with data from the map; and The output from the real-time feature matching element is sent to the filter to generate the vehicle pose.

19. The method of claim 18, further comprising: The start-up operation is performed immediately after the vehicle is connected. Particles from previous operations of the vehicle are sampled to establish an initial map reference; Load the planar reflectors previously identified during sampling from the map reference; as well as The AP path length and power of the multiple APs are obtained from the map reference.

20. The method of claim 19, further comprising: Input the line-of-sight (LOS) and non-line-of-sight (NLOS) AP locations into the map reference; The AP's path length and power are associated based on the AP's MAC ID; as well as Update the particle weights of the plurality of APs, wherein the particle weights are selected from the plurality of particle weights that have the highest probability of being non-reflective.

Citation Information

Patent Citations

  • Location and mobile-adaptation of wireless access-points using map-based navigation

    CN102202257A

  • Vehicle remote driving system established by primary and secondary wireless devices by means of internet of things connection

    WO2020151468A1