Improved positional accuracy using sensor data

Abstract

Techniques are provided for determining a location of a mobile device based on visual positioning resolution (VPS). An exemplary method for determining a position estimate of a mobile device includes acquiring sensor information, detecting one or more identifiable features in the sensor information, determining a range to at least one of the one or more identifiable features, acquiring coarse map information, determining a location of the at least one of the one or more identifiable features based on the coarse map information, and determining a position estimate of the mobile device based at least in part on the range to the at least one of the one or more identifiable features.

Description

Technical Field

[0001] Cross - reference to Related Applications This application claims the benefit of U.S. Patent Application No. 17 / 198,560, filed on March 11, 2021, entitled "IMPROVED POSITION ACCURACY USING SENSOR DATA", which has been assigned to the assignee of this application and the entire contents of which are incorporated herein by reference for all purposes.

Background Art

[0002] The Global Positioning System (GPS) is an example of a GNSS navigation system that determines its position by accurately measuring the arrival times of signaling events received from multiple satellites by a receiver. Each satellite transmits a navigation message that includes the exact time the message was sent and ephemeris information. Each sub - frame of the navigation message starts with a telemetry word (TLM) and the number of the sub - frame. The start of the sub - frame can be detected by a preamble sequence within the TLM. Each sub - frame also includes a handover word (HOW) that gives the exact time of the week (TOW) at which the satellite will transmit the next sub - frame, according to a local version of the GPS time held by the satellite's clock. The ephemeris information includes details about the satellite's orbit and corrections to the satellite's own clock compared to GPS time. The ephemeris parameters and clock correction parameters can be collectively known as ephemeris information.

[0003] GNSS accuracy can be significantly degraded under weak signal conditions, such as when the line-of-sight (LOS) to satellite vehicles is obstructed by natural or artificial objects. For example, in natural or urban canyons, limitations on visible satellites and other multipath effects can induce absolute position errors of several tens of meters (e.g., around 50 meters) and relative position errors of several meters. In addition, accuracy can be further degraded by the limited availability of acceptable GNSS measurements. For example, GNSS measurements in urban areas (e.g., urban canyons) can be degraded by multipath effects of signals transmitted from satellite vehicles (SV). [Overview of the project] [Means for solving the problem]

[0004] An exemplary method for determining a mobile device location estimate according to this disclosure includes the steps of: acquiring sensor information; detecting one or more identifiable features in the sensor information; determining a range of at least one of the one or more identifiable features; acquiring rough map information; determining the location of at least one of the one or more identifiable features based on the rough map information; and determining a mobile device location estimate based at least partially on a range of at least one of the one or more identifiable features.

[0005] Implementations of such methods may include one or more of the following features: The step of determining the range of at least one of one or more identifiable features may be based on the output of a remote sensor. The remote sensor is a LiDAR device or a radar device. At least one of the one or more identifiable features may include at least one of a crosswalk, intersection, traffic signal, and road sign. The step of obtaining rough map information may include providing a rough location to a remote server and receiving rough map information from the remote server. The rough location may be based on a location calculated by a satellite positioning system. The rough location of a mobile device may be based on ground navigation techniques. The sensor information may be an image, and the step of detecting one or more identifiable features in the sensor information may include performing an optical character recognition process on the image. The step of detecting one or more identifiable features in the sensor information may include determining a street name or business name.

[0006] An exemplary method for determining the location estimate of a mobile device according to this disclosure includes the steps of: acquiring rough map information; acquiring sensor information; performing an optical character recognition process on the sensor information; determining one or more identifiable features based on the optical character recognition process and the rough map information; and determining the location estimate of the mobile device at least in part based on the rough map information and one or more identifiable features.

[0007] An implementation of such a method may include one or more of the following features: The method may further include the steps of determining the distance to one or more identifiable features and determining a location estimate of a mobile device based at least in part on the distance to one or more identifiable features. The step of determining the distance to one or more identifiable features may be based on the output of a range sensor. The range sensor may be a lidar device or a radar device. The step of determining one or more identifiable features may include determining a street name and intersection location. The step of determining one or more identifiable features may include determining a company name and an address associated with the company name. The step of obtaining rough map information may include providing a rough location to a remote server and receiving rough map information from the remote server. The rough location of the mobile device may be based on a location obtained using a global navigation satellite system. The rough location of the mobile device may be based on a location obtained using ground navigation techniques.

[0008] An exemplary method for determining a location estimate of a mobile device according to this disclosure includes the steps of: determining the approximate location of the mobile device; obtaining rough map information based on the approximate location; obtaining sensor information; determining one or more road curvatures based on the sensor information; determining that a comparison value satisfies a threshold, wherein the comparison value is based on a comparison between a road curvature feature in the rough map information and one or more road curvatures detected in the sensor information; determining the heading of the mobile device at least in part on the road curvature feature in the rough map information; and determining a location estimate of the mobile device at least in part on the heading.

[0009] An implementation of such a method may include one or more of the following features: Sensor information may be an image. Sensor information may be a point cloud acquired using one or more radar sensors or lidar sensors. Sensor information may be acquired using one or more remote sensors. One or more road curvature features may include one or more lane markers. One or more road curvature features may include one or more guardrails. The step of determining that a comparison value satisfies a threshold may include the step of performing a convolution between road curvature features detected in the sensor information and roads in the rough map information. The rough location of the mobile device may be based on a location acquired using a global navigation satellite system. The rough location of the mobile device may be based on a location acquired using ground navigation techniques. The rough map information may include the steps of providing the rough location to a remote server and receiving the rough map information from the remote server. The step of determining the position estimate may include providing heading and global navigation satellite signal information to a recursive algorithm. The recursive algorithm may be a Kalman filter.

[0010] An exemplary apparatus according to the present disclosure includes a memory, at least one transceiver, and at least one processor communicatively coupled to the memory and the at least one transceiver, wherein the at least one processor is configured to acquire sensor information, detect one or more identifiable features in the sensor information, determine the range of up to one of the one or more identifiable features, acquire rough map information, determine the location of at least one of the one or more identifiable features based on the rough map information, and determine a location estimate based at least partially on the range of up to one of the one or more identifiable features.

[0011] An exemplary apparatus according to the present disclosure includes a memory, at least one transceiver, and at least one processor communically coupled to the memory and the at least one transceiver, wherein the at least one processor is configured to acquire rough map information, acquire sensor information, perform an optical character recognition process on the sensor information, determine one or more identifiable features based on the optical character recognition process and the rough map information, and determine a position estimate at least in part based on the rough map information and the one or more identifiable features.

[0012] An exemplary apparatus according to the present disclosure includes a memory, at least one transceiver, and at least one processor communically coupled to the memory and the at least one transceiver, the at least one processor configured to determine a rough location, acquire rough map information based on the rough location, acquire sensor information, determine one or more road curvature features based on the sensor information, determine that a comparison value satisfies a threshold, wherein the comparison value is based on a comparison between road curvature features in the rough map information and one or more road curvature features detected in the sensor information, determine a heading at least partially based on the road curvature features in the rough map information, and determine a location estimate at least partially based on the heading.

[0013] The sections and / or techniques described herein may provide one or more of the following capabilities and / or other capabilities not mentioned herein: A mobile device may calculate a rough location based on satellite and / or ground positioning techniques. Rough map information may be obtained based on the rough location. Visually identifiable features such as street names, intersection locations, company names, and addresses may be included in the rough map information. An image sensor on a mobile device may obtain images of the environment. In some examples, the images may be analyzed based on visually identifiable information in the rough map information. In some examples, the images may be analyzed for road curvature information. The location of a mobile device may be estimated based on a comparison of visually identifiable information and / or road curvature information in the images and rough map information. Location estimation does not rely on a dedicated high-resolution image mapping database. Other capabilities may be provided, and not all implementations provided herein must provide any, much less, of the capabilities discussed.

[0014] Refer to the following diagrams to illustrate non-exclusive and non-exclusive aspects, and unless otherwise specified, the same reference numbers refer to the same parts throughout the various diagrams. [Brief explanation of the drawing]

[0015] [Figure 1A] This is a block diagram of a mobile device that may be used to implement the techniques discussed herein. [Figure 1B] This diagram shows an overall view of a mobile device equipped with a front-facing camera and a rear-facing camera. [Figure 2] Figures 1A and 1B show block diagrams of exemplary network architectures configured to communicate with mobile devices. [Figure 3] This figure shows an example coordinate system that can be adapted to mobile devices. [Figure 4A] This is a block diagram of an exemplary navigation system equipped with visual sensors. [Figure 4B]This is a block diagram of an exemplary distributed system for sensor-assisted navigation. [Figure 5] This figure shows an exemplary image with identifiable features acquired by a visual sensor. [Figure 6] This figure shows an exemplary rough map associated with the image in Figure 5, containing identifiable feature information. [Figure 7] This figure shows an example of an image transformation using an image sensor. [Figure 8] This figure shows an example of a heading measurement based on the image in Figure 7. [Figure 9] This is a flowchart of a method for determining the estimated location of a mobile device based on identifiable features within an image. [Figure 10] This is a flowchart of a method for determining the location estimate of a mobile device, at least partially based on the heading input. [Figure 11] This is a flowchart of a method for determining the location estimate of a mobile device, based at least partially on the range of features. [Modes for carrying out the invention]

[0016] Techniques for determining the location of a mobile device based on a visual positioning solution (VPS) are described herein. The positioning resolution in an urban environment may result in higher errors during positioning due to the multipath effect on signals transmitted from SVs and ground stations. There is a need for higher accuracy and more reliable positioning resolution for various mobile computing applications and use cases. For example, an Augmented Reality (AR) device, a VPS device, and a vehicle navigation device may include one or more image sensors, such as one or more cameras disposed on the device or vehicle and oriented in the direction of movement. In one example, the image sensor may be configured to generate a video stream that can be integrated with a navigation system (e.g., an Inertial Measurement Unit (IMU), GNSS, etc.) to improve the positioning resolution generated on the device. Previous camera-integrated navigation resolutions rely on a dedicated pre-built high-definition and feature mapping infrastructure. Such a resolution is thus limited to areas where existing mapping images are generated, available, and / or compatible with the mobile device. The proposed resolution can identify visual features within widely available rough mapping information and utilize the visual features to improve positioning calculations and corresponding positioning estimates. Further, the proposed solution can utilize inputs from different sensors and / or different devices that may be disposed at different locations. These techniques and configurations are examples, and other techniques and configurations may be used.

[0017] Referring to Figure 1A, a block diagram of a mobile device 100 that may be used to implement the enhanced RTA and consistency detection techniques is shown. The mobile device 100 may include or implement the functions of various mobile communication and / or computing devices, including, but not limited to, in-vehicle navigation systems, VPS, wearable navigation devices, smartphones, watches, helmets, virtual reality (VR) goggles, smart glasses, cameras, etc., whether currently existing or to be developed in the future. The mobile device 100 includes a processor 111 (or processor core), one or more digital signal processors (DSPs) 120, and a memory unit 160. The processor 111 may be a central processing unit (CPU), a multi-core CPU, a graphics processing unit (GPU), a multi-core GPU, a video engine, or any combination thereof. The processor core may be an application processor. The navigation processor 115 and the optical flow processor 125 are shown in a mobile device 100, not as an example, but as an example. The navigation processor 115 and the optical flow processor 125 may be contained within a memory unit 160 and utilize processor 111. The navigation processor 115 and / or optical flow processor 125 may be a system on a chip (SoC) in the mobile device (for example, it may be dedicated hardware, or a discrete chipset or part of a discrete chipset (for example, on a discrete application processor)), or it may be contained within one or more auxiliary systems (for example, remote from the mobile device 100).In one embodiment, the mobile device includes one or more cameras 105 (e.g., forward-facing and / or backward-facing), such as a complementary metal-oxide-semiconductor (CMOS) image sensor with a suitable lens configuration. Other imaging techniques may be used, such as charge-coupled devices (CCDs) and back-illuminated CMOSs. Camera 105 is configured to acquire image information and provide it to an optical flow processor 125. In another embodiment, one or more external cameras may be used. In one example, camera 105 may consist of depth sensors, such as an infrared depth sensor for determining the range to an object.

[0018] The mobile device 100 may also include a wireless transceiver 130 configured to transmit and receive wireless signals 134 via a wireless antenna 132 over a wireless network. The wireless transceiver 130 is connected to a bus 101. Here, the mobile device 100 is shown as having a single wireless transceiver 130. However, the mobile device 100 may, alternatively, have multiple wireless transceivers 130 and wireless antennas 132 to support multiple communication standards, such as Wi-Fi®, CDMA, Wideband CDMA (WCDMA®), Long Term Evolution (LTE), Fifth Generation (5G), New Radio (NR), and Bluetooth® short-range wireless communication technologies.

[0019] Wireless transceiver 130 may support operation on multiple carriers (waveform signals of different frequencies). A multi-carrier transmitter can simultaneously transmit modulated signals on multiple carriers. Each modulated signal may be a Code Division Multiple Access (CDMA) signal, a Time Division Multiple Access (TDMA) signal, an Orthogonal Frequency Division Multiple Access (OFDMA) signal, a Single-Carrier Frequency Division Multiple Access (SC-FDMA) signal, etc. Each modulated signal may be transmitted on a different carrier and may carry pilot, overhead information, data, etc.

[0020] The mobile device 100 also includes a Global Navigation Satellite System (GNSS) receiver 170 that receives SPS signals 174 (for example, from SPS satellites) via a satellite positioning system (SPS) antenna 172. The GNSS receiver 170 may communicate with a single Global Navigation Satellite System (GNSS) or multiple such systems. GNSS may include, but is not limited to, the Global Positioning System (GPS), Galileo, Glonass, Beidou (Compass), etc. SPS satellites are also called satellites, space vehicles (SV), etc. The GNSS receiver 170 may process the SPS signals 174 whole or partially and use these SPS signals 174 to determine the location of the mobile device 100. Processor 111, DSP 120, and memory 160, and / or specialized processors (not shown) may be used to process the SPS signal 174 in whole or in part, and in conjunction with the GNSS receiver 170, to calculate the location of the mobile device 100. Storing information from the SPS signal 174 or other location signals is performed using memory units 160 or registers (not shown). The navigation processor 115 may have instructions configured to calculate position information based on the GNSS signal and / or dead reckoning position information based on information received from microelectromechanical systems (MEMS) such as an accelerometer 140, a gyroscope 145, and / or other sensors 150 (e.g., pressure sensors, magnetometers, microphones). In one example, the other sensors 150 may include a depth sensor and / or a lidar for measuring range from the mobile device 100. In one example, the other sensors 150 may include an always-on motion device configured to send an interruption to processor 111 when motion is detected.For example, STMicroelectronics part number LSM6DSL is an example of a continuously operating 3D accelerometer and 3D gyroscope that can function as an accelerometer 140, a gyroscope 145, and / or other sensors 150. One processor 111, a DSP 120, and a memory unit 160 are shown in Figure 1A, but two or more, pairs, or all of any of these components may be used by the mobile device 100.

[0021] The memory unit 160 may include a non-temporary machine-readable storage medium that stores functions as one or more instructions or codes. The media that may constitute the memory unit 160 include, but are not limited to, RAM, ROM, FLASH®, disk drives, etc. Generally, the functions stored by the memory unit 160 are executed by the processor 111, DSP 120, or other dedicated processor. Thus, the memory unit 160 is processor-readable memory and / or computer-readable memory that stores software (programmable code, instructions, machine code, etc.) configured to cause the processor 111 to perform the functions described. Alternatively, one or more functions of the mobile device 100 may be executed entirely or partially in hardware. The memory unit 160 may be communicatively coupled to the processor 111 via the bus 101. The term communicatively coupled describes the ability of components within the mobile device 100, or other systems, to exchange and process electronic signals.

[0022] The mobile device 100 may estimate its current location within associated systems using various techniques based on other communication entities in the view and / or information available to the mobile device 100. For example, the mobile device 100 may estimate its location using information obtained from map constraint data acquired from one or more wireless local area networks (LANs) utilizing short-range wireless communication technologies such as Wi-Fi, Bluetooth®, or ZigBee®, access points associated with personal area networks (PANs), SPS satellites, and / or map servers or other location servers. The mobile device 100 may also estimate its current location based on dead reckoning techniques using inertial sensors such as an accelerometer 140, a gyroscope 145, and other sensors 150 (e.g., a magnetometer, pressure sensor, solid-state compass). In one example, the mobile device 100 may determine its current location at least in part based on an image acquired by the camera 105 (e.g., by comparing the current image with a previously stored image). In general, inertial sensors are used to measure the displacement of the mobile device 100. For example, the mobile device 100 can enter an INS state such that the navigation processor 115 is configured to receive signals from one or more inertial sensors (e.g., accelerometer, gyroscope, solid-state compass) and calculate a dead reckoning position. The dead reckoning position may be calculated on a periodic basis based on context changes and / or when course and velocity information changes. In some examples, the dead reckoning position may be adjusted when another position is determined (e.g., GNSS, trilateration, user input). For pedestrian applications, the accelerometer 140 may include a triaxial accelerometer to drive a pedometer to determine the step count or step rate.

[0023] In one embodiment, one or more of the components within the mobile device 100 may be performed by peripheral devices configured to provide data to the mobile device. For example, the GNSS receiver 170 may be a peripheral GNSS receiver configured to provide navigation data via a wireless transceiver 130 (e.g., Bluetooth). Other sensors, such as radar, lidar, and optical sensors such as camera 105, may be located remotely from the mobile device 100 and may be configured to provide information via a wired or wireless interface. For example, a camera or lidar sensor installed at different locations on a vehicle and configured to connect to the mobile device 100 via a wireless link. In another example, the mobile device 100 may have wireless connectivity with a wearable action camera, AR goggles, smart glasses (with a camera), or other wearable sensors, such that the wearable sensors are configured to provide sensor information to improve positional accuracy using the techniques provided herein.

[0024] Figure 1B is an example of a mobile device 100 capable of performing the functions described herein. Figure 1B may represent a smartphone using one or more components of the mobile device of Figure 1A. However, the functions described herein are not limited to the use of a smartphone, and any device similar to Figure 1A, with capabilities suitable for performing such functions, may be used. These devices may include mobile devices, digital cameras, camcorders, tablets, PDAs, smart glasses, VR goggles, or any other similar devices. Figure 1B shows the front 180 and rear 190 of the mobile device 100. The front 180 includes a display 182 and a first camera 105a. The first camera 105a, coupled to the front side of the mobile device 100, is also called the front camera. The rear 190 of the mobile device 100 includes a second camera 105b, also called the rear camera in this specification. The mobile device 100 may be held or mounted such that the front camera 105a faces the user of the mobile device 100 and the rear camera 105b faces away from the user of the device. Alternatively, the reverse may be appropriate depending on how the mobile device 100 is held by the user or how it is mounted in a holder (e.g., an armband) or cradle. Both the front camera 105a and the rear camera 105 may be one implementation of camera 105 and may be configured to provide image information to the optical flow processor 125, as discussed with reference to Figure 1A. In one example, the first camera 105a and the second camera 105b may include a range-detection device, such as an infrared transmitter and receiver configured as depth sensors to determine the range to an object and enable cameras 105a-b to electrically and / or mechanically focus on that object.

[0025] Referring to Figure 2, an exemplary network architecture 200 configured to communicate with the mobile device in Figure 1A is shown. The mobile device 100 may transmit radio signals to and receive radio signals from the wireless communication network. In one example, the mobile device 100 may communicate with the cellular communication network by transmitting radio signals to or receiving radio signals from a cellular transceiver 220, which may include a wireless base transceiver subsystem (BTS), Node B, evolved Node B (eNodeB), and / or next generation Node B (gNodeB) on a wireless communication link 222. Similarly, the mobile device 100 may transmit radio signals to or receive radio signals from a local wireless transceiver 230 on a wireless communication link 232. The local transceiver 230 may include an access point (AP), femtocell, home base station, small cell base station, home node B (HNB), or home eNode B (HeNB), and may provide access to a wireless local area network (WLAN, e.g., an IEEE 802.11 network), a wireless personal area network (WPAN, e.g., a Bluetooth® network), or a cellular network (e.g., a 5G NR and / or LTE network or other wireless wide area network). Of course, these are merely examples of networks that can communicate with mobile devices over a wireless link, and the subject matter claimed is not limited to this.

[0026] Examples of network technologies that may support wireless communication link 222 include Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Broadband CDMA (WCDMA), Long-Term Evolution (LTE), 5G NR, and High Rate Packet Data (HRPD). GSM, WCDMA, LTE, and 5G NR are technologies defined by 3GPP®. CDMA and HRPD are technologies defined by 3rd Generation Partnership Project 2 (3GPP2). WCDMA is also part of the Universal Mobile Telecommunications System (UMTS) and may be supported by HNB. Cellular transceiver 220 may include the deployment of equipment that provides subscriber access to the wireless telecommunications network for service (for example, under a service contract). Here, the cellular transceiver 220 may perform the functions of a cellular base station when serving subscriber devices in a cell determined at least partially on the extent to which the cellular transceiver 220 can provide access services. Examples of wireless technologies that may support the wireless communication link 222 are IEEE 802.11, Bluetooth®, LTE, and 5G NR.

[0027] In a particular implementation, the cellular transceiver 220 and the local transceiver 230 may communicate with one or more servers 240 on the network 225, where the network 225 may comprise any combination of wired and / or wireless links and may include the cellular transceiver 220 and / or the local transceiver 230 and / or the server 240. In a particular implementation, the network 225 may comprise the Internet Protocol (IP) or other infrastructure that facilitates communication between the mobile device 100 and the server 240 via the local transceiver 230 or the cellular transceiver 220. In one implementation, the network 225 may comprise cellular communication network infrastructure, such as a base station controller or a packet-based or circuit-based switching center (not shown), to facilitate mobile cellular communication with the mobile device 100. In certain implementations, network 225 may comprise local area network (LAN) elements such as WLAN APs, routers, and bridges, and in such cases, may include, or have, links to gateway elements that provide access to a wide area network such as the Internet. In other implementations, network 225 may comprise a LAN and may or may not have access to a wide area network, and may not provide any such access to the mobile device 100 (if supported). In some implementations, network 225 may comprise multiple networks (e.g., one or more wireless networks and / or the Internet). In one implementation, network 225 may be an NG-RAN including one or more serving gateways (e.g., eNBs) or packet data network gateways.In addition, one or more of the 240 servers, Access and Mobility Management Function (AMF), Session Management Function (SMF), Location Management Function (LMF), and Gateway Mobile Location Center (GMLC).

[0028] In certain implementations, as discussed below, the mobile device 100 may have circuitry and processing resources that can acquire location-related measurements (for example, signals received from GPS or other satellite positioning system (SPS) satellites 210, cellular transceivers 220, or local transceivers 230) and, optionally, calculate the fixed or estimated location of the mobile device 100 based on these location-related measurements. In some implementations, the location-related measurements acquired by the mobile device 100 may be transferred to a location server, such as a location management function (LMF) (for example, one of one or more servers 240), which may then estimate or determine the location of the mobile device 100 based on the measurements. In the example currently illustrated, location-related measurements obtained by the mobile device 100 may include measurements of SPS signals 174 received from satellites belonging to SPS or a Global Navigation Satellite System (GNSS) such as GPS, GLONASS, Galileo, or Beidou, and / or measurements of signals (222 and / or 232, etc.) received from a ground transmitter fixed at a known location (e.g., a cellular transceiver 220, etc.).The mobile device 100 or a separate location server may then obtain a location estimate of the mobile device 100 based on these location-related measurements, using one of several location methods, such as GNSS, Assisted GNSS (A-GNSS), Advanced Forward Link Trilateration (AFLT), Observed Time Difference Of Arrival (OTDOA), Round Trip Time (RTT), Received Signal Strength Indication (RSSI), Angle of Arrival (AoA), or Enhanced Cell ID (E-CID), or a combination thereof. In some of these techniques (e.g., A-GNSS, AFLT, and OTDOA), a pseudorange or timing difference can be measured in the mobile device 100 relative to three or more ground transmitters fixed at known locations, or relative to four or more SVs whose orbital data is precisely known, or in combination thereof, based at least in part on pilots, positioning reference signals (PRS), or other positioning-related signals transmitted by a transmitter or satellite and received by the mobile device 100. Doppler measurements may be performed on various signal sources, such as cellular transceivers 220, local transceivers 230, and satellites 210, and various combinations thereof. One or more servers 240 may provide the mobile device 100 with positioning support data, including, for example, information about the signal to be measured (e.g., signal timing), location and identification information of ground transmitters, and / or signal, timing, and orbital information about GNSS satellites to facilitate positioning techniques such as A-GNSS, AFLT, OTDOA, RTT, RSSI, AoA, and E-CID.For example, one or more servers 240 may include an almanac showing location and identification information of cellular transceivers and / or local transceivers in a specific area or area such as a specific venue, and may provide information representing signals transmitted by cellular base stations or APs, such as transmit power and signal timing. In the case of E-CID, the mobile device 100 may obtain signal strength measurements for signals received from cellular transceivers 220 and / or local transceivers 230, and / or obtain round-trip signal propagation time between the mobile device 100 and the cellular transceivers 220 or local transceivers 230. The mobile device 100 may use these measurements together with supporting data received from one or more servers 240 (e.g., ground almanac data or GNSS satellite data such as GNSS almanac and / or GNSS ephemeris information) to determine the location of the mobile device 100, or it may forward the measurements to one or more servers 240 to perform the same determination. In one embodiment, the server 240 may include an optical flow processor configured to receive image information from a mobile device and perform feature analysis (e.g., optical character recognition (OCR), distance and heading measurement, etc.) as described herein.

[0029] A mobile device (for example, mobile device 100 in Figure 1A) may be referred to as a device, mobile device, wireless device, mobile terminal, terminal, mobile station (MS), user equipment (UE), SUPL Enabled Terminal (SET), or several other names, and may correspond to a cell phone, smartphone, wristwatch, in-vehicle navigation system, tablet, PDA, tracking device, smart glasses, VR goggles, or several other portable or mobile devices. Generally, but not necessarily, a mobile device may support wireless communications such as GSM, WCDMA, LTE, 5G NR, CDMA, HRPD, Wi-Fi®, BT, WiMAX, etc. A mobile device may also support wireless communications using, for example, wireless LAN (WLAN), DSL, or packet cable. A mobile device may include a single entity or multiple entities, such as a personal area network in which the user may employ audio, video, and / or data I / O devices, as well as / or body sensors and separate wired or wireless modems. An estimate of the location of a mobile device (e.g., mobile device 100) may be called location, location estimate, location fix, fix, position, location estimate, or location fix, and may be geographical and therefore provide location coordinates (e.g., latitude and longitude) for the mobile device, which may or may not include an elevation component (e.g., elevation, ground, floor, or height or depth from underground). Alternatively, the location of a mobile device may be expressed as an urban location (e.g., as the address or designation of a point or narrow area somewhere in a building, such as a specific room or floor). The location of a mobile device may be expressed as an area or volume (defined either geographically or by the shape of a city) in which the mobile device is expected to be located with some probability or confidence level (e.g., 67% or 95%).The location of a mobile device may also be a relative location, including distance and direction or relative X, Y (and Z) coordinates defined with respect to an origin in a known location, where this origin may be defined, for example, geographically, or in urban terms, or based on a point, area, or volume shown on a map, floor plan, or building plan. In the descriptions contained herein, the use of the term location may include any of these variations unless otherwise indicated.

[0030] Referring further to Figures 1A and 1B, and then to Figure 3, an exemplary coordinate system 300 is shown that may be used, in whole or in part, to facilitate or support measurements acquired via the inertial sensors of the mobile device 100. Inertial sensor measurements may be acquired, for example, based at least in part on output signals generated by associated accelerometers 140 or gyroscopes 145. The exemplary coordinate system 300 may include, for example, a three-dimensional Cartesian coordinate system. For example, the displacement of the mobile device 100 representing acceleration oscillations may be detected or measured at least in part by a suitable accelerometer, such as a three-dimensional (3D) accelerometer, which references three linear dimensions or axes X, Y, and Z relative to the origin of the exemplary coordinate system 300. It should be noted that the exemplary coordinate system 300 may or may not be aligned with the body of the mobile device 100. In one implementation, a non-Cartesian coordinate system may be used so that the coordinate system can define mutually orthogonal dimensions.

[0031] Sometimes, the rotational motion of the mobile device 100, such as a change in orientation due to gravity, can be detected or measured at least partially by a suitable accelerometer while referring to one or more dimensions. For example, in some cases, the rotational motion of the mobile device 100 can be measured using coordinates (

[0032]

number

[0033] (phi), which can be converted to τ(tau) and detected or measured, where phi

[0034]

number

[0035] τ represents roll or rotation on the X axis, as generally indicated by the arrow in 306, and tau(τ) represents pitch or rotation on the Y axis, as generally indicated in 308. As will be discussed below, the rotational motion of the mobile device 100 can also be detected or measured by a suitable gyroscope, for example, with respect to the X, Y, and Z orthogonal axes. Thus, the 3D accelerometer can at least partially detect or measure changes in the level of acceleration oscillations and gravity, for example, with respect to the roll dimension or pitch dimension, thereby enabling observability in five dimensions.

[0036]

number

[0037] This provides, of course, just examples of motions that can be detected or measured at least partially with reference to the exemplary coordinate system 300, and the claimed subject matter is not limited to any particular motion or coordinate system.

[0038] In some cases, the rotational motion of the mobile device 100 can be detected or measured at least partially by a suitable gyroscope 145 to provide a sufficient or suitable degree of observability. The gyroscope 145 can detect or measure the rotational motion of the mobile device 100 while referring to one, two, or three dimensions. Thus, in some cases, the gyroscope rotation can be, for example, coordinate

[0039]

number

[0040] It can be at least partially detected or measured by converting it to phi, where phi

[0041]

number

[0042] θ represents roll or rotation 306 with respect to the X axis, tau(τ) represents pitch or rotation 308 with respect to the Y axis, and psi(ψ) represents yaw or rotation with respect to the Z axis, as generally referred to in 310. A gyroscope may, though not necessarily, generally provide measurements converted to angular acceleration (e.g., change in angle per unit of time squared), angular velocity (e.g., change in angle per unit of time), etc. Similarly, the details relating to motion that can be detected or measured at least partially by the gyroscope with respect to the exemplary coordinate system 300 are merely examples, and the subject matter claimed is not limited in that way. The gyroscope 145 may have a bias value, and the accuracy of the output may drift over time. In one embodiment, the RTA and matching techniques described herein may be used to determine the bias value and enable the processor to compensate for the gyroscope bias.

[0043] Referring to Figure 4A, a block diagram 400 of an exemplary navigation system with a visual sensor is shown. System 400 may include system-on-chip components such as a modem, or it may be implemented in separate devices and / or different systems configured to communicate with each other. System 400 is an example, not an limitation, as other architectures and components may be used. A distributed system may include an auxiliary processor (AP) 402 and a main processor (MP) 404. AP 402 may be operably connected to MP 404 and configured to respond to instructions for events received from MP 404. Generally, AP 402 is configured to provide optical flow analysis information, such as pixel displacement information, based on image frames received from the visual sensor 405. Optical flow analysis may include feature detection and OCR methods for the image frames. The visual sensor 405 may also include one or more range sensors 406, such as a depth sensor (e.g., an IR ranging component associated with a camera), a lidar device (e.g., a laser and coherent light-based one), and / or a radar device (e.g., a millimeter-wave radar component). The visual sensor 405 may be a CMOS imaging technique such as a camera 105, and the range sensors 406 may be other sensors 150 in Figure 1A. AP402 includes an optical flow processor (OF) 425 operably connected to the visual sensor 405 and configured to activate the visual sensor 405 and receive images and / or depth information acquired by the range sensors 406. MP404 includes a navigation processor (NAV) 415 operably coupled to a GNSS processor 470. The inertial sensor 450 is configured to receive commands from the navigation processor 415 and return inertial data, such as outputs from an accelerometer and a gyroscope.For example, the gyroscope 145 is configured to provide angular velocity signals, and the accelerometer 140 is configured to provide specific force signals. The GNSS processor 470 is configured to receive satellite positioning signals 474 and provide position and time information to the navigation processor 415.

[0044] In one example, the navigation processor 415 consists of a recursive algorithm, such as a Kalman filter, to estimate the state of the distributed system 400. The Kalman filter may receive measurements from the inertial sensor 450, GNSS 470, and optical flow processor 425 and apply a mathematical model to determine the estimated orientation and position. Other positioning methods may be used to provide location information. For example, a mobile device 100 or a separate server 240 may obtain a location estimate of the mobile device 100 based on assisted GNSS (A-GNSS), advanced forward link trilateration (AFLT), observation time difference (OTDOA), or extended cell ID (E-CID). In one embodiment, the AP 402, MP 404, vision sensor 405, range sensor 406, and inertial sensor 450 may be contained within a single device, such as the mobile device 100. For example, AP402 is an example of an optical flow processor 125, MP404 is an example of a navigation processor 115, the vision sensor 405 is an example of a camera 105, and the inertial sensor 450 may include an accelerometer 140, a gyroscope 145, and other sensors 150. In another embodiment, as shown in Figure 4A, AP402, MP404, vision sensor 405, range sensor 406, and inertial sensor 450 may be contained within separate devices and configured to exchange information via wired or wireless communication paths. In one example, the vision sensor 405 and range sensor 406 may be mounted in fixed positions relative to the vehicle frame.

[0045] Referring to Figure 4B, a block diagram 450 of an exemplary distributed system for sensor-assisted navigation is shown. Figure 450 includes a mobile device 452 and one or more remote sensors 454. The mobile device 452 may include some or all of the components of mobile device 100, and mobile device 100 may be an example of mobile device 452. The remote sensors 454 may be one or more sensors configured to acquire measurements of the environment in proximity to the mobile device 452. For example, the remote sensors 454 may include one or more of the following: remote cameras and other vision sensors, radio frequency (RF) sensing devices, radar, lidar (e.g., laser or coherent light-based), inertial sensors, barometers, magnetometers, and other measuring devices. In one embodiment, the remote sensors 454 may be communicably coupled to the mobile device via one or more wired or wireless connections. For example, in a vehicle application, the remote sensor 454 may be a camera, radar, lidar, and other sensor mounted on the vehicle and configured to provide images and other data to a mobile device 452 via wireless technologies such as WiFi and / or Bluetooth. Other techniques may be used. In some examples, the remote sensor 454 may be located within another device. For example, an image sensor in AR goggles may be the remote sensor 454 and may be configured to provide image information to a mobile device. Other remote devices, such as action cameras, WiFi radar sensors, and range detection devices, may also be configured to provide sensor information to the mobile device 452. During operation, the mobile device 452 may be configured to communicate with an edge server 462 via a communication network including a base station 460 and a first communication link 456. For example, the base station 460 may be a cellular network, and the first communication link 456 may be a radio access technology such as LTE or 5G NR.In one example, base station 460 may be an access point within a wide area network (WAN), and the first communication link 456 may be based on WiFi, Bluetooth, or wireless access technology. Edge server 462 may be a third-party mapping application or other location-based service provider configured to receive and / or provide positioning information.

[0046] In one embodiment, the remote sensor 454 may be configured to provide sensor data to the edge server 462 and / or mobile device 452 via a second communication link 458 and a base station 460. The second communication link 458 and base station 460 may be cellular-based, WAN-based, or other radio access technology. In one example, the edge server 462 may be configured to calculate the location of the mobile device 452 based on the method described herein, using information provided by the mobile device 452 (e.g., rough location information) and information provided by the remote sensor 454 (e.g., images, radar / lidar cloud points). For example, the mobile device 452 may be a smartphone, smartwatch, etc., and may provide rough location information to a third-party application such as Google Maps (i.e., the edge server 462). The remote sensor 454 may be smart glasses, AR goggles, etc., associated with the mobile device 452 and configured to acquire images. A third-party application may receive both rough location information and images via the first communication link 456 (for example, from a mobile device 452) or via a combination of the first and second communication links 456 and 458 (for example, from the mobile device 452 and the remote sensor 454, respectively). The third-party application may combine branched data sources to improve the accuracy of the location estimate of the mobile device 452.

[0047] Referring to Figure 5, an exemplary image 500 is shown with identifiable features acquired by a visual sensor. Image 500 is a street view acquired by a visual sensor 405 positioned on the front of a vehicle and oriented in the direction of travel. A sub-image 502 within Image 500 includes two identifiable features, including a street sign 504 (i.e., "Mission") and a business sign 506 (e.g., "Soma Park Inn"). Identifiable features 504, 506 are examples and generally represent text information available in public, proprietary mapping applications such as Google Maps, Apple Maps, Waze®, and other mapping service providers. During operation, the mobile device 100 may be configured to acquire a rough location based on GNSS location estimates. In an urban environment such as shown in Image 500, the GNSS location estimates may have areas of uncertainty amplified by obstructed SV signals (e.g., caused by buildings, nearby vehicles, etc.) and other multipath effects. Mobile device 100 is configured to retrieve map information from a service provider based on rough GNSS location estimates (e.g., calculated latitude and longitude), and the map information may include geolocated points of interest (POI) information and associated identifiable features such as street names, intersection names, bus stop numbers, company names, and addresses. OF425 may be configured to use computer vision techniques to extract identifiable features (e.g., street names, company names, addresses, street corners / intersections) from image 500 (e.g., OCR techniques) based on a corpus of text within the rough map information. In one example, text associated with features and POIs within the rough map information may be used to constrain the OCR resolution. The size of the rough map information may be based on a constituent range from the GNSS location estimates (e.g., 100, 200, 500, 1000 yards (91, 183, 457, 914 meters), etc.). Other factors, such as the current speed and capabilities of the mobile device 100, may be used to determine the extent of the rough map information obtained.In one embodiment, the outputs of the range sensor 406 and / or the vision sensor 405 can be used to determine distance estimates 508 to identifiable features such as crosswalks, streetlights, street signs, traffic signals, and intersections. The range sensor 406 may be used in conjunction with inertial measuring devices (e.g., gyroscope 145, accelerometer 140) to determine the bearing and altitude of an object based on the coordinate system 300 and the orientation of the range sensor 406 (e.g., boresight). The accuracy of the GNSS position estimate can be improved using a predetermined location of an identifiable feature and / or the distance to the identifiable feature.

[0048] In one embodiment, the mobile device 100 may consist of one or more computer vision algorithms known in the art to identify features such as crosswalks, building corners, sidewalk bends, stop signal marker lines, and signal text. These algorithms may perform preprocessing or other filtering steps on the image, such as applying color thresholds, S-channel gradient thresholds, and region of interest filters. Viewpoint transformations may be used to transform the image from a forward view to a bird's-eye view. Transformation matrices may be calculated, and the image may be warped based on these transformation matrices. Histogram peaks may be used to determine markings, such as crosswalks, stop signal markings, or other identifiable features on the road. Sliding window techniques may be used to extract active pixels and perform a polyfit process to construct lines based on the active pixels. Relative distances between lane markings may be used to distinguish lane markers from crosswalks. Precautioned camera parameters or cameras may be used in combination with accelerometer and gyroscope inputs to determine the relative distance from the camera to identifiable features, such as crosswalks. Other algorithms and image processing techniques may be used.

[0049] Referring to Figure 6, an exemplary rough map 600 associated with image 500 in Figure 5 is shown. The rough map 600 includes location and text information associated with identifiable features, such as street names 602 (i.e., "Mission") and company names 604 (i.e., "Soma Park Inn"). In one example, the mobile device 100 may obtain an initial GNSS location estimate 606 indicating its current location on 9th Street between Minna and Natoma Street (shown in Figure 6). Simultaneously with obtaining the initial GNSS location estimate 606, the mobile device also obtains image 500 in Figure 5. The OF processor 425 may be configured to determine identifiable features based on the text information associated with the rough map 600. In one example, the OF processor 425 may determine street names (e.g., Natoma, Minna, Mission, 9) in the rough map information. th Based on St), OCR techniques can be utilized. In one example, the OF processor 425 can utilize OCR techniques based on text associated with street names and other POIs (e.g., company names and building names) in rough map information (e.g., Moya, Soma Park Inn, Coffee Cultures, BCC Bar, etc.). Constraining the OCR resolution based on rough map information can enable more robust and faster text recognition. In one embodiment, the OF processor 425 may be configured to obtain OCR results based only on image 500 (i.e., without constraints).

[0050] Referring to the street sign 504 in Figure 5, the navigation processor 415 may update the GNSS location estimate using identifiable features and rough map data detected by the OF processor 425. Using the identification information and OCR of the street sign 504 (i.e., "Mission Street") that is close to the mobile device 100, an updated GNSS location estimate 608 may be generated indicating that the current location is northwest of the initial GNSS location estimate 606 (i.e., closer to Mission St.). Other identifiable features, such as a company name 604, may be used to update the location estimate. A distance estimate 508 may be used to update the location estimate. For example, the distance estimate 508 indicates the estimated distance to the intersection of Mission St and 9th St shown on the rough map 600. Comparing identifiable features (e.g., a readable street sign) to rough map feature data may help to remove out-of-range location estimates and achieve improved positioning accuracy. Distance and / or heading measurements may be used as measurement updates for the Kalman filtering algorithm used to determine the position.

[0051] Referring to Figure 7, an exemplary transformation 700 of an image 702 acquired using a visual sensor 405 is shown. In one use case, the visual sensor 405 may be mounted on a vehicle and oriented to acquire an image in the direction of the vehicle. The street image 702 may include natural and artificial topographic features, such as lane markers, jersey barriers, guardrails, shoulder lanes, and other features based on the viewpoint of the visual sensor 405. The optical flow processor 425 may be configured to transform the street image 702 into a bird's-eye view warp image 704 using viewpoint transformation techniques. Other image resolution and adjustment techniques (e.g., contrast adjustment, binarization, sharpening, etc.) may be used. For example, the warp image 704 may highlight high-speed lane markings, including a series of dashed right-hand lane marks 704a and left-hand lane stripes 704b. Other features may also be identifiable.

[0052] Referring to Figure 8, an example of heading measurement based on warp image 704 is shown. Image 800 is a negative image of warp image 704 and is provided to facilitate the description of heading measurement. Image 800 is an example and not limiting. The OF processor 425 may be configured to determine the contours of image features such as dashed right lane mark 704a and left lane stripe 704b. For example, the OF processor 425 may utilize image processing techniques to obtain the right lane boundary 804a based on the dashed right lane mark 704a and the left lane boundary 804b based on the left lane stripe 704b. In one embodiment, the mobile device 100 may be configured to obtain rough map information based on initial GNSS position estimates, and the navigation processor 415 may be configured to determine the vehicle's current heading 806 by aligning the angles and / or curvatures of the right and left lane boundaries 804a~b with the road in the rough map data. Heading information derived from an image can be used as input to a recursive algorithm, such as a Kalman filter, configured to generate a position estimate.

[0053] In one embodiment, the vision sensor 405 may be configured to acquire images periodically and / or based on predetermined trigger conditions. In one example, a camera duty cycle may be implemented to activate the integration of position estimates with the vision sensor 405. The duty cycle may be determined using relative turning trajectories and / or user dynamics. For example, a turning vehicle may utilize a higher duty cycle compared to a vehicle traveling in a straight line. It is also possible to determine the duty cycle using rough map information. For example, proximity to intersections may be used to increase the duty cycle compared to public roads with fewer intersections. Other states of the mobile device 100 may be used to trigger conditions for activating the vision sensor 405 for navigation applications.

[0054] Referring to Figures 1A-8 and then to Figure 9, a method 900 for determining the location estimate of a mobile device based on identifiable features in an image is shown. However, method 900 is an example and not limiting. Method 900 can be modified, for example, by adding, removing, rearranging, combining, performing simultaneously, and / or dividing a single stage into multiple stages. Method 900 can be performed in an integrated device within a mobile device, in a plurality of discrete components operably coupled to a mobile device, in a plurality of communicatively coupled devices, and / or remotely via a network connection having different stages performed by different sensors and different processors.

[0055] In step 902, the method includes the step of obtaining rough map information. A wireless transceiver 130 may be a means for obtaining rough map information. In one embodiment, the mobile device 100 may be configured to provide location coordinates (e.g., latitude / longitude) via the network 225 to a third-party mapping service provider (e.g., Google Maps, Apple Maps, etc.) in order to obtain rough map information for an area adjacent to the rough location of the mobile device. The location coordinates may be based on SV signals received by the mobile device 100, or other ground navigation techniques based on signals received via the wireless transceiver 130. The rough location may be based on positioning techniques such as A-GNSS, AFLT, OTDOA, RTT, RSSI, AoA, and E-CID. The rough map information may be based on existing map datasets and does not rely on constructing and utilizing image-specific mapping data. The nearby area included in the rough map information may, based on configuration options or other application criteria (e.g., location uncertainty values), encompass 100, 200, 500, and 1000 yards (91, 183, 457, and 924 meters) around the mobile device's location. The rough map information may include georeferenced points of interest (POIs), such as street names and intersection locations, business names and addresses, or other labels corresponding to identifiable features and their corresponding locations. In one example, the mobile device 100 may store the rough map information in local memory 160. In another example, the mobile device 100 may parse the rough map information to determine street names and intersection locations near the mobile device. The mobile device 100 may be configured to parse other visually identifiable features, such as business signs and addresses, bus stop signs, highway signs, elevated line signs, and streetlights, so that each of these visually identifiable features is associated with a known location in the rough map information.

[0056] In step 904, the method includes the step of acquiring sensor information. A visual sensor 405 or a remote sensor 454 may be a means for acquiring sensor information. In one example, the visual sensor 405 or the remote sensor 454 may be a camera such as a second camera 105b, or another camera such as a vehicle-mounted dashboard camera, AR goggles, or other sensors configured to acquire images close to a mobile device. For example, the visual sensor 405 or the remote sensor 454 may acquire a street view of an urban environment, such as image 500 in Figure 5. In one embodiment, sensor information is obtained based on data received from one or more remote sensors 454. For example, a radar system or a lidar system may acquire range and orientation to one or more points (e.g., a point cloud). Other sensors may be configured to provide other information. Sensor information may be acquired on demand or periodically, such as based on a sensor duty cycle.

[0057] In step 906, the method includes the step of performing an optical character recognition process on sensor information. OF425 or processor 111 may be means for performing the OCR process. In one embodiment, the visual sensor 405 and / or remote sensor 454 may acquire images of an area adjacent to the mobile device 100. OF425 or processor 111 may be configured to extract identifiable features (e.g., street names, company names, addresses, street corners / intersections) from the sensor information acquired in step 904 by utilizing OCR and other computer vision techniques on the sensor information.

[0058] In step 908, the method may include the step of determining one or more identifiable features based on an optical character recognition process and rough map information. In one embodiment, a computer vision technique may identify a location in rough map information by utilizing information (e.g., text) associated with one or more visually identifiable features in sensor information. For example, in step 906, the results of the OCR process can be compared with the text associated with the features in the rough map information and the POI. In one embodiment, the rough map information may be used to constrain the OCR resolution. Other constraint-based techniques may be used to improve the efficiency of the OCR process. Other identifiable features associated with the sensor input may be included in the rough map information. For example, geolocated objects such as road signs or other reflectors may be configured to provide a detectable return signature. Location and corresponding identification information may be included in the rough map information.

[0059] In step 910, the method includes the step of determining a location estimate for the mobile device based at least in part on rough map information and one or more identifiable features. A navigation processor 415 or processor 111 is a means for determining the location estimate. The navigation processor 415 may be configured to determine the location of identifiable features based on the location of features in the rough map information. For example, referring to Figure 6, using the identification information of “Mission Street” which is close to the mobile device 100, an updated GNSS location estimate 608 may be generated, indicating that the current location is northwest of the initial GNSS location estimate 606 (i.e., the updated GNSS location estimate 608 is closer to Mission St.). In one embodiment, the mobile device 100 may be configured to further refine the location estimate by utilizing the output of one or more range sensors 406 or remote sensors 454. For example, a range sensor 406, a remote sensor 454, and / or a vision sensor 405 can be used to determine an estimated distance 508 to an identifiable feature such as a crosswalk, streetlights, street signs, or intersection.

[0060] Referring to Figure 10, with further reference to Figures 1A–8, a method 1000 for determining a mobile device's position estimate based at least partially on a heading input is shown. However, method 1000 is an example and not limiting. Method 1000 can be modified, for example, by adding, removing, rearranging, combining, performing simultaneously, and / or dividing a single step into multiple steps.

[0061] In step 1002, the method includes the step of determining the approximate location of the mobile device. A GNSS receiver 170 or a wireless transceiver 130 may be the means for determining the approximate location. The approximate location may be based on the SV signal received by the mobile device 100, or on other ground navigation techniques based on the signal received via the wireless transceiver 130. The approximate location may be based on positioning techniques such as A-GNSS, AFLT, OTDOA, RTT, RSSI, AoA, and E-CID.

[0062] In step 1004, the method includes the step of obtaining rough map information based on a rough location. A wireless transceiver 130 may be a means for determining the rough location. In one embodiment, a mobile device 100 may be configured to provide location coordinates (e.g., latitude / longitude) via the network 225 to a third-party mapping service provider (e.g., Google Maps, Apple Maps, etc.) in order to obtain rough map information for an area adjacent to the rough location. The rough map information may be existing map information (e.g., publicly available) and does not rely on constructing and utilizing image-specific mapping data. The adjacent area may be based on configuration options or other application criteria (e.g., location uncertainty values) and may encompass 100, 200, 500, and 1000 yards (91, 183, 457, and 914 meters) around the rough location measurement. The rough map information may include georeferenced points of interest (POIs), such as street names and intersection locations, company names and addresses, or other labels corresponding to identifiable features and their corresponding locations. In one example, the mobile device 100 may store rough map information in its local memory 160.

[0063] In step 1006, the method includes the step of acquiring sensor information. A visual sensor 405 or a remote sensor may be a means for acquiring sensor information. In one example, the visual sensor 405 or the remote sensor 454 may be a camera such as a second camera 105b, or another camera such as a vehicle-mounted dashboard camera, AR goggles, or other sensors configured to acquire images close to a mobile device. In one embodiment, the VPS navigation device may be mounted on the dashboard of a vehicle or autonomous delivery system and may include a camera pointing in the direction of motion. The visual sensor 405 and / or the remote sensor 454 may be configured, for example, to acquire street images 702 of the road the vehicle is currently crossing. In one embodiment, the images may be acquired based on the visual sensor duty cycle. In one example, the images may be acquired based on input from other sensors such as radar or lidar (e.g., lidar point cloud, radar point data). Other sensors may be configured to provide other information. Sensor information may be acquired on demand or periodically, such as based on the sensor duty cycle.

[0064] In step 1008, the method includes the step of determining one or more road curvature features based on sensor information. The OF processor 425 or processor 111 may be the means for determining one or more road curvature features. In one example, the sensor information may be an image such as a street image 702, which may include natural and artificial terrain features such as lane markers, jersey barriers, guardrails, shoulder lanes, and other features based on the viewpoint of the visual sensor 405. The OF processor 425 or processor 111 may be configured to use visual transformation information techniques to transform the street image 702 into a bird's-eye view warp image 704. In one embodiment referring to Figure 8, the OF processor 425 or processor 111 may utilize image processing techniques to obtain a right lane boundary 804a based on a dashed right lane mark 704a and a left lane boundary 804b based on a left lane stripe 704b. The right and left boundaries 804a and 804b may be used to determine the curvature of the road. In one embodiment, one or more road curvature features are obtained based on radar and / or lidar data acquired by a remote sensor 454. Other sensor processing techniques may be used to determine the curvature.

[0065] In step 1010, the method includes the step of determining whether a comparison value satisfies a threshold, wherein the comparison value is based on a comparison between road curvature features in rough map information and one or more road curvature features detected in sensor information. The navigation processor 415 or processor 111 may be means for determining whether the comparison value satisfies a threshold. In one example, the mobile device 100 may, in step 1004, acquire rough map information, and the navigation processor 415 or processor 111 may be configured to determine the vehicle's current heading 806 by comparing the angles and / or curvature of the right and left lane boundaries 804a-b with the roads in the rough map data. For example, the roads may be identified based on rough location information (e.g., US 5 Northbound), and the comparison algorithm may be based on pixel analysis of the identified roads and the detected curvature. An exemplary comparison algorithm may include convolution (i.e., a kernel) to transpose the curvature information along the identified roads in the forward and reverse directions to find the resolution. Thresholds can be established based on probability coefficients associated with the comparison algorithm. For example, a higher threshold may indicate an increased likelihood of actual matching between road curvature features and rough map information. Other matching algorithms may be used.

[0066] In step 1012, the method includes determining the heading of the mobile device based at least in part on road curvature features in the rough map information. The navigation processor 415 or processor 111 may be the means for determining the heading of the mobile device. In one embodiment, the navigation processor 415 or processor 111 may be configured to calculate the heading based on georeferenced rough map information. For example, coordinate information associated with the rough map (e.g., north) may be used as a criterion for determining the heading (e.g., angle) of a location on a matching road segment. The resulting heading information may be used as input to a recursive algorithm used to determine a location estimate.

[0067] In step 1014, the method includes the step of determining a position estimate of the mobile device, at least in part, based on the heading. A navigation processor 415 or processor 111 may be the means for determining the position estimate. In one embodiment, the navigation processor 415 may consist of a recursive algorithm (e.g., a Kalman filter) for estimating the state of the mobile device 100. The recursive algorithm may receive measurements from the inertial sensor 450 and GNSS 470, in addition to the heading information based on the OF processor 425, and apply a mathematical model to determine the estimated orientation and position. The heading information determined in step 1012 may be used to improve the accuracy of the GNSS position estimate.

[0068] Referring to Figure 11, with further reference to Figures 1A to 8, a method 1100 for determining a mobile device's location estimate based on the range to features is shown. However, method 1100 is an example and not limiting. Method 1100 can be modified, for example, by adding, removing, rearranging, combining, performing simultaneously, and / or dividing a single stage into multiple stages.

[0069] In step 1102, the method includes the step of acquiring sensor information. The visual sensor 405 and the remote sensor 454 may be means for acquiring sensor information. In one example, the visual sensor 405 may be a camera such as a second camera 105b, or another camera such as a vehicle-mounted dashboard camera, AR goggles, or a camera in another device configured to acquire images close to a mobile device. For example, the visual sensor 405 may acquire a street view of an urban environment, such as image 500 in Figure 5. In one embodiment, the sensor information is obtained based on data received from one or more remote sensors 454. For example, a radar system or lidar system may acquire range and orientation to one or more points (e.g., a point cloud). Other sensors may be configured to provide other information. The sensor information may be acquired on demand or periodically, such as based on a sensor duty cycle.

[0070] In step 1104, the method includes the step of detecting one or more identifiable features in the sensor information. OF425 and processor 111 may be means for detecting one or more identifiable features. In one example, the sensor information may be an image, and OF425 may consist of one or more computer vision algorithms for identifying features such as crosswalks, building corners, sidewalk bends, stop signal marker lines, intersections, traffic signals, streetlights, and other features. These algorithms may perform preprocessing or other filtering steps on the image, such as applying color thresholds, S-channel gradient thresholds, and region of interest filters. Viewpoint transformations may be used to transform the image from a forward-looking view to a bird's-eye view. Transformation matrices may be calculated, and the image may be warped based on these transformation matrices. Histogram peaks may be used to determine markings, such as crosswalks, stop signal markings, or other identifiable features on the road. Processor 111 may be configured to identify radar and / or lidar return information based on the signal intensity, Doppler shift, or phase of the return signal. For example, a pre-configured reflector or other object (e.g., a traffic signal, a street sign, etc.) may be provided with an identifiable radar or lidar return signal. Memory 160 may include a data structure for correlating the return signal with the object.

[0071] In step 1106, the method includes the step of determining a range to one or more identifiable features. OF425 or processor 111 may be means for determining the range to the identifiable features. In one example, the sensor information may be an image, and a camera may be used in combination with pre-calibrated camera parameters or an accelerometer and gyroscope to determine the relative distance from the camera to the identifiable features detected in step 1104. In one embodiment, a range sensor 406 and / or remote sensors 454 may be used to determine the range to the identifiable features. The range sensor 406 may be used in combination with an inertial measuring device (e.g., gyroscope 145, accelerometer 140) to determine the bearing and altitude of an object based on the coordinate system 300 and the orientation of the range sensor 406 (e.g., boresight). One or more remote sensors 454 may be remote devices configured to range information independently (e.g., without input from an inertial measuring device). In some cases, known distances based on traffic and / or civilian standards (e.g., standard distances between lane markings) may be used to distinguish between a feature and the estimated range to that feature.

[0072] In step 1108, the method includes the step of acquiring rough map information. A wireless transceiver 130 and a transceiver 111 may be means for acquiring rough map information. In one embodiment, a mobile device 100 may be configured to provide location coordinates (e.g., latitude / longitude) via the network 225 to a third-party mapping service provider (e.g., Google Maps, Apple Maps, etc.) in order to acquire rough map information for an area adjacent to the mobile device. The location coordinates may be based on SV signals received by the mobile device 100, or other ground navigation techniques based on signals received via the wireless transceiver 130. The rough location may be based on positioning techniques such as A-GNSS, AFLT, OTDOA, RTT, RSSI, AoA, and E-CID. The rough map information may be based on existing map datasets and does not rely on constructing and utilizing image-specific mapping data. The nearby area included in the rough map information may be based on configuration options or other application criteria (e.g., location uncertainty value) and may encompass areas of 100, 200, 500, 1000 yards, etc. (91, 183, 457, 924 meters) around the mobile device's location.

[0073] In step 1110, the method includes the step of determining the location of at least one of one or more identifiable features based on rough map information. The navigation processor 415 or processor 111 may be means for determining the location of the identifiable feature. In one embodiment, the rough map information may include georeferenced POIs, such as street names and intersection locations, business names and addresses, or other labels corresponding to identifiable features and their corresponding locations. The navigation processor 415, processor 111, or other processor on the mobile device 100 may be configured to parse the rough map information to determine street names, intersection locations, business signs and addresses, bus stop signs, highway signs, elevated line signs, streetlights, etc., so that each of the visually identifiable features is associated with a known location in the rough map information.

[0074] In step 1112, the method includes the step of determining a location estimate for a mobile device based at least partially on a range of at least one of one or more identifiable features. A navigation processor 415 or processor 111 may be a means for determining the location estimate. The navigation processor 415 and / or processor 111 may be configured to determine the location of a visually identifiable feature based on the location of the feature in rough map information. For example, referring to Figures 5 and 6, the identification of “Mission Street” in rough map information may be used in conjunction with a distance estimate 508 (i.e., the range determined in step 1106) to determine the location estimate. Other range, sensor, and orientation information may be used to determine the location estimate. In one embodiment, a remote server, such as an edge server 462, may be configured to determine the location estimate based on the rough location estimate and sensor information. For example, a mobile device 452 and / or a remote sensor 454 may be configured to provide sensor information, and the edge server 462 may be configured to perform method 1100.

[0075] Throughout this specification, any reference to “an example,” “a certain example,” “some examples,” or “exemplary embodiments” means that any particular feature, structure, or characteristic described with respect to the features and / or examples may be included in at least one feature and / or example of the claimed subject matter. Therefore, occurrences of phrases such as “in an example,” “a certain example,” “in a particular example,” or “in some embodiments” or other similar phrases in various places throughout this specification do not necessarily all refer to the same features, examples, and / or limitations. Furthermore, any particular feature, structure, or characteristic may be combined in one or more examples and / or features.

[0076] Some portions of the detailed descriptions contained herein are presented with respect to algorithms or symbolic representations of operations for binary digital signals stored in the memory of a particular apparatus or dedicated computing device or platform. In the context of this particular specification, the term "particular apparatus, etc." includes general-purpose computers that, once programmed, perform specific operations according to instructions from program software. An algorithmic description or symbolic representation is an example of a technique used by those skilled in the art to communicate the nature of their work to others skilled in the art. In this specification, an algorithm is also generally considered to be a self-consistent set of operations or similar signal operations that produce a desired result. In this context, an operation or operation involves the physical manipulation of a physical quantity. While not always the case, such quantities generally take the form of electrical or magnetic signals that can be stored, transferred, combined, compared, or otherwise manipulated. It has been found that it is sometimes convenient to refer to such signals as bits, data, values, elements, symbols, characters, terms, digits, numerical values, etc., mainly because they are common usages. However, it should be understood that all of these terms or similar terms should be associated with the appropriate physical quantities and are merely convenient designations. Unless otherwise specified, as will be apparent from the discussion herein, discussions throughout this specification using terms such as “process,” “calculate,” “compute,” and “determine” should be understood to refer to actions or processes of specific devices, such as dedicated computers, dedicated computing devices, or similar dedicated electronic computing devices. Therefore, in the context of this specification, dedicated computers or similar dedicated electronic computing devices are generally capable of manipulating or converting signals that are represented as physical electronic or magnetic quantities within the memory, registers, or other information storage devices, transmitting devices, or display devices of the dedicated computer or similar dedicated electronic computing device.

[0077] The wireless communication techniques described herein may be connected to various wireless communication networks, such as wireless wide area networks ("WWAN"), wireless local area networks ("WLAN"), and wireless personal area networks (WPAN). The terms "network" and "system" may be used interchangeably herein. A WWAN may be a code division multiple access (CDMA) network, a time division multiple access (TDMA) network, a frequency division multiple access (FDMA) network, an orthogonal frequency division multiple access (OFDMA) network, a single-carrier frequency division multiple access (SC-FDMA) network, or any combination of the above networks. A CDMA network may implement one or more radio access technologies (RATs), such as cdma2000 and wideband-CDMA (W-CDMA), which are just a few examples of radio technologies. Here, cdma2000 may include technologies implemented in accordance with the IS-95, IS-2000, and IS-856 standards. The TDMA network may implement the Global System for Mobile Communications (GSM), Digital Advanced Mobile Phone Systems (D-AMPS), or any other RAT. GSM and W-CDMA are described in documents from an organization called the "Third Generation Partnership Project" ("3GPP"). 5G NR and LTE communication networks may also be implemented according to the claimed subject matter in one embodiment. For example, the WLAN may comprise an IEEE 802.11x network, and the WPAN may comprise a Bluetooth® network, IEEE 802.15x. The wireless communication implementations described herein may also be used in any combination of WWAN, WLAN, or WPAN.

[0078] The techniques described herein may be used with SPS, including one of several GNSSs and / or a combination of GNSSs. Furthermore, such techniques may be used with a ground transmitter acting as a “pseudosatellite,” or a positioning system that utilizes a combination of an SV and such a ground transmitter. A ground transmitter may include, for example, a ground-based transmitter that broadcasts a PN code or other ranging code (similar to, for example, GPS or CDMA cellular signals). Such a transmitter may be assigned a unique PN code to enable identification by a remote receiver. Ground transmitters may be useful in supplementing SPS in situations where SPS signals from orbiting SVs are unavailable, such as in tunnels, mines, buildings, urban canyons, or other enclosed areas. Another implementation of a pseudosatellite is known as a radio beacon. The term “SV” herein is intended to include ground transmitters acting as pseudosatellites, pseudosatellite equivalents, and possibly other. The terms “SPS signal” and / or “SV signal” are intended herein to include SPS-like signals from ground transmitters, including ground transmitters acting as pseudo-satellites or pseudo-satellite equivalents.

[0079] The detailed description above includes numerous specific details to give a complete understanding of the claimed subject matter. However, it will be understood by those skilled in the art that the claimed subject matter can be put into practice without these specific details. In other cases, methods and apparatus that would be known to those skilled in the art are not described in detail so as not to obscure the claimed subject matter.

[0080] As used herein, the terms “and,” “or,” and “and / or” may have a variety of meanings, which are also expected to depend at least in part on the context in which such terms are used. In general, when “or” is used to relate an enumeration such as A, B, or C, it is intended to mean A, B, and C as used here in an inclusive sense, as well as A, B, or C as used here in an exclusive sense. In addition, as used herein, the term “one or more” may be used to describe any singular feature, structure, or characteristic, or to describe multiple features, structures, or characteristics, or any other combination of features, structures, or characteristics. However, it should be noted that these are merely illustrative examples, and the claimed subject matter is not limited to these examples.

[0081] While exemplary features and those currently considered to be exemplary are illustrated and described, it will be understood by those skilled in the art that various other modifications may be made and equivalents may be substituted without departing from the claimed subject matter. In addition, many modifications may be made to adapt specific situations to the teachings of the claimed subject matter without departing from the central concepts described herein.

[0082] Therefore, it is intended that the claimed subject matter is not limited to the specific examples disclosed, but may also include all embodiments and their equivalents that fall within the scope of the attached claims.

[0083] In implementations involving firmware and / or software, the method may be implemented using modules (e.g., processors, memory, procedures, functions, etc.) that perform the functions described herein. Any machine-readable medium that tangibly embodies instructions may be used when performing the method described herein. For example, software code may be stored in memory and executed by a processor unit. Memory may be implemented within or outside the processor unit. As used herein, the term “memory” refers to any type of long-term memory, short-term memory, volatile memory, non-volatile memory, or other memory, and should not be limited to any particular type of memory or any particular number of memories, nor should it be limited to any particular type of medium in which memory is stored.

[0084] When implemented in firmware and / or software, the functionality may be stored as one or more instructions or codes on a non-temporary machine-readable storage medium. Examples include computer-readable media encoded using data structures and computer-readable media encoded using computer programs. Machine- or computer-readable media include physical computer storage media (e.g., non-temporary machine-readable media). The storage medium may be any available medium that can be accessed by a computer. Such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage, semiconductor storage, or any other medium that can be accessed by a computer and used to store desired program code in the form of instructions or data structures. The above combinations should also be included within the scope of computer-readable media.

[0085] Instructions and / or data may be provided as signals on a transmitting medium included in a communication device, in addition to being stored in a computer-readable storage medium. For example, a communication device may include a transceiver having signals representing instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the claims. That is, the communication device includes a transmitting medium having signals indicating information for performing the disclosed functions. Initially, the transmitting medium included in the communication device may include a first portion of the information for performing the disclosed functions, and a second portion may include a second portion of the information for performing the disclosed functions.

[0086] Implementation examples are described in the following numbered sections.

[0087] Clause 1. A method for determining a location estimate of a mobile device, comprising the steps of: acquiring sensor information; detecting one or more identifiable features in the sensor information; determining the range of at least one of the one or more identifiable features; acquiring rough map information; determining the location of at least one of the one or more identifiable features based on the rough map information; and determining a location estimate of the mobile device based at least partially on the range of at least one of the one or more identifiable features.

[0088] Clause 2. The method of Clause 1, wherein the step of determining the range of at least one of one or more identifiable features is based on the output of a remote sensor.

[0089] Clause 3. The method of Clause 2, wherein the remote sensor is a LiDAR device or a radar device.

[0090] Clause 4. The method of Clause 1, wherein at least one of one or more identifiable features includes at least one of a pedestrian crossing, an intersection, a traffic signal, and a road sign.

[0091] Clause 5. The method of Clause 1, wherein the step of obtaining rough map information includes the steps of providing a rough location to a remote server and receiving rough map information from the remote server.

[0092] Clause 6. The method of Clause 5, based on a location whose approximate position is calculated by a satellite positioning system.

[0093] Clause 7. The approximate location of a mobile device is determined by the method of Clause 5, based on ground navigation techniques.

[0094] Clause 8. The method of Clause 1, wherein the sensor information is an image, and the step of detecting one or more identifiable features in the sensor information includes the step of performing an optical character recognition process on the image.

[0095] Clause 9. The method of Clause 8, wherein the step of detecting one or more identifiable features in sensor information includes the step of determining a street name or business name.

[0096] Clause 10. A method for determining a location estimate of a mobile device, comprising the steps of: acquiring rough map information; acquiring sensor information; performing an optical character recognition process on the sensor information; determining one or more identifiable features based on the optical character recognition process and the rough map information; and determining a location estimate of the mobile device at least in part based on the rough map information and one or more identifiable features.

[0097] Clause 11. The method of Clause 10, further comprising the steps of determining the distance to one or more identifiable features and determining a location estimate of a mobile device based at least in part on the distance to one or more identifiable features.

[0098] Clause 12. The method of Clause 11, wherein the step of determining the distance to one or more identifiable features is based on the output of a range sensor.

[0099] Clause 13. The method of Clause 12, wherein the range sensor is a lidar device or a radar device.

[0100] Clause 14. The method of Clause 10, wherein the step of determining one or more identifiable features includes the step of determining a street name and an intersection location.

[0101] Clause 15. The method of Clause 10, wherein the step of determining one or more identifiable features includes the step of determining a company name and an address associated with the company name.

[0102] Clause 16. The method of Clause 10, wherein the step of obtaining rough map information includes the steps of providing a rough location to a remote server and receiving rough map information from the remote server.

[0103] Clause 17. The method of Clause 16, wherein the approximate location of a mobile device is based on a location obtained using a global navigation satellite system.

[0104] Clause 18. The method of Clause 16, wherein the approximate location of a mobile device is based on a location obtained using ground navigation techniques.

[0105] Clause 19. A method for determining a location estimate of a mobile device, comprising the steps of: determining the approximate location of the mobile device; obtaining rough map information based on the approximate location; obtaining sensor information; determining one or more road curvature features based on the sensor information; determining that a comparison value satisfies a threshold, wherein the comparison value is based on a comparison between a road curvature feature in the rough map information and one or more road curvature features detected in the sensor information; determining the heading of the mobile device at least partially based on the road curvature features in the rough map information; and determining a location estimate of the mobile device at least partially based on the heading.

[0106] Clause 20. The method of Clause 19, wherein the sensor information is an image.

[0107] Clause 21. The method of Clause 19, wherein the sensor information is a point cloud acquired using one or more radar sensors or lidar sensors.

[0108] Clause 22. The method of Clause 19, wherein sensor information is obtained using one or more remote sensors.

[0109] Clause 23. The method of Clause 19, wherein one or more road curvature features include one or more lane markers.

[0110] Clause 24. The method of Clause 19, wherein one or more road curvature features include one or more guardrails.

[0111] Clause 25. The method of Clause 19, wherein the step of determining that a comparison value meets a threshold includes the step of performing a convolution between road curvature features detected in sensor information and roads in rough map information.

[0112] Clause 26. The method of Clause 19, where the approximate location of a mobile device is based on a location obtained using a global navigation satellite system.

[0113] Clause 27. The approximate location of a mobile device is determined by the method of Clause 19, based on a location obtained using ground navigation techniques.

[0114] Clause 28. The method of Clause 19, wherein the step of obtaining rough map information includes the steps of providing a rough location to a remote server and receiving rough map information from the remote server.

[0115] Clause 29. The method of Clause 19, wherein the step of determining a position estimate includes the step of providing heading and global navigation satellite signal information to a recursive algorithm.

[0116] Clause 30. The method of Clause 29, wherein the recursive algorithm is a Kalman filter.

[0117] Clause 31. An apparatus comprising a memory, at least one transceiver, and at least one processor communicatively coupled to the memory and the at least one transceiver, wherein the at least one processor is configured to acquire sensor information, detect one or more identifiable features in the sensor information, determine the range of up to one of the one or more identifiable features, acquire rough map information, determine the location of at least one of the one or more identifiable features based on the rough map information, and determine a position estimate based at least partially on the range of at least one of the one or more identifiable features.

[0118] Clause 32. The apparatus of Clause 31, further configured to determine a range of up to one of one or more identifiable features based on the output of a remote sensor.

[0119] Clause 33. The apparatus of Clause 32, wherein the remote sensor is a lidar device or radar device.

[0120] Clause 34. A device of Clause 31 in which at least one of one or more identifiable features includes at least one of a pedestrian crossing, an intersection, a traffic signal, and a road sign.

[0121] Clause 35. The apparatus of Clause 31, further configured with at least one processor to send a rough location to a remote server and to receive rough map information from the remote server.

[0122] Clause 36. The device of Clause 35, whose approximate location is based on a location calculated by a satellite positioning system.

[0123] Clause 37. The apparatus of Clause 35, further configured with at least one processor to determine a rough position based on ground navigation techniques.

[0124] Clause 38. The apparatus of Clause 31, wherein the sensor information is an image, and at least one processor is further configured to perform an optical character recognition process on the image.

[0125] Clause 39. The apparatus of Clause 38, further configured to determine a street name or business name based on an optical character recognition process for an image, wherein at least one processor is configured to determine a street name or business name.

[0126] Clause 40. An apparatus comprising a memory, at least one transceiver, and at least one processor communicatively coupled to the memory and the at least one transceiver, wherein the at least one processor is configured to acquire rough map information, acquire sensor information, perform an optical character recognition process on the sensor information, determine one or more identifiable features based on the optical character recognition process and the rough map information, and determine a position estimate at least in part based on the rough map information and the one or more identifiable features.

[0127] Clause 41. The apparatus of Clause 40, further configured to determine distances to one or more identifiable features and to determine a position estimate based at least in part on the distances to one or more identifiable features.

[0128] Clause 42. The apparatus of Clause 41, wherein at least one processor is configured to determine the distance to one or more identifiable features based on the output of a range sensor.

[0129] Clause 43. The apparatus of Clause 42, wherein the range sensor is a lidar device or radar device.

[0130] Clause 44. The apparatus of Clause 40, further configured with at least one processor to determine street names and intersection locations.

[0131] Clause 45. The apparatus of Clause 40, further configured to determine a company name and an address associated with the company name, with at least one processor.

[0132] Clause 46. The apparatus of Clause 40, further configured with at least one processor to send a rough location to a remote server and to receive rough map information from the remote server.

[0133] Clause 47. An apparatus of Clause 46 whose approximate location is based on a location obtained using a global navigation satellite system.

[0134] Clause 48. The apparatus of Clause 46, further configured with at least one processor to determine a rough position based on ground navigation techniques.

[0135] Clause 49. An apparatus comprising memory, at least one transceiver, and at least one processor communicatively coupled to the memory and the at least one transceiver, wherein the at least one processor is configured to determine a rough location, acquire rough map information based on the rough location, acquire sensor information, determine one or more road curvature features based on the sensor information, determine that a comparison value satisfies a threshold, wherein the comparison value is based on a comparison between road curvature features in the rough map information and one or more road curvature features detected in the sensor information, determine a heading at least partially based on the road curvature features in the rough map information, and determine a position estimate at least partially based on the heading.

[0136] Clause 50. The apparatus of Clause 49, wherein the sensor information is an image.

[0137] Clause 51. The device of Clause 49, wherein the sensor information is a point cloud acquired using one or more radar sensors or lidar sensors.

[0138] Clause 52. The apparatus of Clause 49, further configured to acquire sensor information from one or more remote sensors, with at least one processor.

[0139] Article 53. The apparatus of Article 49, wherein one or more road curvature features include one or more lane markers.

[0140] Article 54. Devices of Article 49, in which one or more road curvature features include one or more guardrails.

[0141] Clause 55. The apparatus of Clause 49, further configured to perform a convolution between road curvature features detected in sensor information and roads in rough map information, wherein at least one processor is also configured.

[0142] Clause 56. An instrument of Clause 49 whose approximate location is based on a location obtained using a global navigation satellite system.

[0143] Clause 57. The apparatus of Clause 49, further configured with at least one processor to determine a rough position based on ground navigation techniques.

[0144] Clause 58. The apparatus of Clause 49, further configured with at least one processor to transmit a rough location to a remote server and to receive rough map information from the remote server.

[0145] Clause 59. The apparatus of Clause 49, further configured to utilize heading and global navigation satellite signal information in a recursive algorithm, with at least one processor.

[0146] Clause 60. The apparatus of Clause 59, where the recursive algorithm is a Kalman filter.

[0147] Clause 61. An apparatus for determining the location estimate of a mobile device, comprising: means for acquiring sensor information; means for detecting one or more identifiable features in the sensor information; means for determining the range of at least one of the one or more identifiable features; means for acquiring rough map information; means for determining the location of at least one of the one or more identifiable features based on the rough map information; and means for determining the location estimate of the mobile device based at least partially on the range of at least one of the one or more identifiable features.

[0148] Clause 62. An apparatus for determining the location estimate of a mobile device, comprising means for acquiring rough map information, means for acquiring sensor information, means for performing an optical character recognition process on the sensor information, means for determining one or more identifiable features based on the optical character recognition process and the rough map information, and means for determining the location estimate of the mobile device at least in part based on the rough map information and one or more identifiable features.

[0149] Article 63. An apparatus for determining the location estimate of a mobile device, comprising: means for determining the approximate location of the mobile device; means for acquiring approximate map information based on the approximate location; means for acquiring sensor information; means for determining one or more road curvature features based on the sensor information; means for determining whether a comparison value satisfies a threshold, wherein the comparison value is based on a comparison between a road curvature feature in the approximate map information and one or more road curvature features detected in the sensor information; means for determining the heading of the mobile device at least partially based on the road curvature features in the approximate map information; and means for determining the location estimate of the mobile device at least partially based on the heading.

[0150] Clause 64. A non-temporary processor-readable storage medium comprising processor-readable instructions configured to cause one or more processors to determine a location estimate of a mobile device, wherein the processor-readable instructions comprise: a code for acquiring sensor information; a code for detecting one or more identifiable features in the sensor information; a code for determining the range of at least one of the one or more identifiable features; a code for acquiring rough map information; a code for determining the location of at least one of the one or more identifiable features based on the rough map information; and a code for determining a location estimate of a mobile device based at least partially on the range of at least one of the one or more identifiable features.

[0151] Clause 65. A non-temporary processor-readable storage medium comprising processor-readable instructions configured to cause one or more processors to determine a location estimate of a mobile device, wherein the processor-readable instructions comprise: a code for acquiring rough map information; a code for acquiring sensor information; a code for performing an optical character recognition process on the sensor information; a code for determining one or more identifiable features based on the optical character recognition process and the rough map information; and a code for determining a location estimate of a mobile device at least partially based on the rough map information and the one or more identifiable features.

[0152] Clause 66. A non-temporary processor-readable storage medium comprising processor-readable instructions configured to cause one or more processors to determine a location estimate of a mobile device, wherein the processor-readable instructions comprise: a code for determining the approximate location of a mobile device; a code for obtaining rough map information based on the approximate location; a code for obtaining sensor information; a code for determining one or more road curvature features based on the sensor information; a code for determining that a comparison value satisfies a threshold, wherein the comparison value is based on a comparison between a road curvature feature in the rough map information and one or more road curvature features detected in the sensor information; a code for determining the heading of a mobile device at least partially based on the road curvature features in the rough map information; and a code for determining a location estimate of a mobile device at least partially based on the heading. [Explanation of symbols]

[0153] 100 mobile devices 101 Bus 105 Camera 105a First camera, front camera, camera 105b Second camera, rear camera, camera 111 processors 115 Navigation Processor 120 Digital Signal Processors (DSPs) 125 Optical Flow Processors 130 Wire Restaurant Seaba 132 Wireless Antenna 134 Wireless Signals 140 Accelerometer 145 Gyroscope 150 sensors 160 memory units 170 Global Navigation Satellite System (GNSS) receiver 172 Satellite Positioning System (SPS) Antennas 174 SPS signals 180 front 182 displays 190 Back 200 Network Architectures 210 Satellite Positioning System (SPS) satellites, satellites 220 Cellular Transceiver 222 Wireless communication links, wireless communication, signals 225 Network 230 Local Wire Restaurant Seeba, Local Transceiver 232 Wireless communication link, signal 240 servers 300 coordinate system 306 rotations 308 rotations 400 Block diagrams of navigation systems, systems, and distributed systems 402 (Auxiliary Processor AP) 404 Main Processor (MP) 405 Vision Sensor 406 Range Sensor 415 Navigation Processor (NAV) 425 Optical Flow Processor (OF) 450 Inertial Sensor, Block Diagram of Distributed System 452 mobile devices 454 Remote Sensors 456 First communication link 458 Second communication link 460 base station 462 Edge Servers 470 GNSS processors, GNSS 474 Satellite positioning signals 500 images 502 Sub-images 504 Street signs, features 506 Company logos and features 508 Distance Estimates 600 rough map 602 Street names 604 Company name 606 Initial GNSS position estimate 608 Updated GNSS position estimates 700 conversion 702 images, street images 704 Warp Image 704a Right lane mark 704b Left lane stripe 800 images 804a Right lane boundary, right boundary 804b Left lane boundary, left boundary 806 Heading 900 ways 1000 ways 1100 methods

Claims

1. A method performed by a mobile device to determine the location estimate of the mobile device, A step of acquiring sensor information that includes an image, A step of detecting one or more identifiable features in the sensor information, wherein the step of detecting one or more identifiable features includes a step of performing optical character recognition processing on the image, A step of determining the range up to at least one of the one or more identifiable features, A step of obtaining rough map information, the step of obtaining rough map information includes the steps of providing a rough location to a remote server and receiving the rough map information from the remote server, A step of determining the location of at least one of the one or more identifiable features based on the rough map information, A step of determining the estimated position of the mobile device based at least partially on the range up to at least one of the one or more identifiable features; Methods that include...

2. The method according to claim 1, wherein the step of determining the range up to at least one of the one or more identifiable features is based on the output of a remote sensor.

3. The method according to claim 2, wherein the remote sensor is a lidar device or a radar device.

4. The method according to claim 1, wherein at least one of the one or more identifiable features includes at least one of a pedestrian crossing, an intersection, a traffic signal, and a road sign.

5. The method according to claim 1, wherein the aforementioned approximate position is based on a location calculated by a satellite positioning system.

6. The method according to claim 1, wherein the approximate location of the mobile device is based on ground navigation techniques.

7. The method according to claim 1, wherein the step of detecting one or more identifiable features in the sensor information includes the step of determining a street name or company name.

8. It is a device, Memory and At least one transceiver, The system comprises the memory and at least one processor communicatively coupled to the at least one transceiver, wherein the at least one processor Acquiring sensor information that includes images, The detection of one or more identifiable features in the sensor information, wherein the detection of one or more identifiable features includes performing optical character recognition processing on the image. Determine the range up to at least one of the one or more identifiable features mentioned above. The acquisition of rough map information includes providing a rough location to a remote server and receiving the rough map information from the remote server. Based on the aforementioned rough map information, the location of at least one of the one or more identifiable features is determined, Determining a position estimate based at least partially on the range up to at least one of the one or more identifiable features. A device configured to perform the following actions.

9. The apparatus according to claim 8, wherein the at least one processor is further configured to determine the range up to at least one of the one or more identifiable features based on the output of the remote sensor.

10. The remote sensor is a LiDAR device or a radar device. The apparatus according to claim 9.

11. The at least one of the one or more identifiable features includes at least one of a pedestrian crossing, an intersection, a traffic signal, and a road sign. The apparatus according to claim 8.

12. The aforementioned approximate location is based on the location calculated by the satellite positioning system. The apparatus according to claim 8.

13. The at least one processor is further configured to determine the rough position based on ground navigation techniques. The apparatus according to claim 8.

14. The at least one processor is further configured to determine a street name or company name based on the optical character recognition processing on the image. The apparatus according to claim 8.

15. A non-temporary processor-readable storage medium comprising processor-readable instructions configured to cause one or more processors to execute the method according to any one of claims 1 to 7.

Images