Cellular radio frequency (RF) sensing in automotive navigation

Abstract

An example method for obtaining data for vehicle navigation at a vehicle based on cellular radio frequency (RF) sensing includes obtaining a camera image of an environment around the vehicle with a camera at the vehicle. The method also includes determining a fault condition of the camera that obstructs object detection of the camera image, wherein the fault condition is caused by severe weather, a blockage on a lens of the camera, an electrical fault of the camera, or a combination thereof. The method also includes, in response to determining the fault condition of the camera, configuring a cellular communication interface of the vehicle to obtain cellular RF sensing data of the environment around the vehicle. The method also includes performing the vehicle navigation support based at least in part on cellular RF sensing data of the environment around the vehicle.

Description

[0001] Related Applications

[0002] This application claims the benefit of U.S. application No. 17 / 820,068, filed on August 16, 2022, entitled “CELLULAR RADIO FREQUENCY (RF) SENSING IN AUTOMOBILE NAVIGATION,” which is assigned to the assignee of this application and is incorporated herein by reference in its entirety. Background Art

[0003] Vehicle systems such as autonomous driving and advanced driver assistance systems (ADAS) often use camera-based navigation systems to operate properly. However, camera-based systems are susceptible to weather conditions that block vision, such as snow, rain, and fog. To reduce the impact of bad weather, some vehicles implement additional non-vision-based detection systems such as radio detection and ranging (radar) or other radio frequency (RF)-based systems as back-up in addition to camera-based detection systems. However, those additional systems are expensive and may not be available for some vehicles for cost-saving reasons. Summary of the invention

[0004] The technology provided herein relates to obtaining data for automobile navigation at a vehicle based on cellular communication (e.g., 5th generation (5G) and / or 5G new radio (NR) radio frequency (RF) sensing). An embodiment includes a vehicle determining a fault condition of a camera at the vehicle that prevents object detection from being performed on a camera image captured by the camera, the fault condition being caused by inclement weather, a blockage on a lens of the camera, an electrical fault of the camera, or a combination thereof. The vehicle can then configure a cellular communication interface of the vehicle (e.g., a 5G communication interface) to detect cellular RF sensing data of a geographic location of the vehicle and perform automobile navigation based at least in part on the 5G RF sensing data of the geographic location of the vehicle.

[0005] An example method for obtaining data for automobile navigation at a vehicle based on cellular radio frequency (RF) sensing includes obtaining a camera image of an environment surrounding the vehicle using a camera at the vehicle. The method also includes determining a fault condition of the camera that prevents object detection by the camera image, wherein the fault condition is caused by inclement weather, an obstruction on a lens of the camera, an electrical fault of the camera, or a combination thereof. The method also includes configuring a cellular communication interface of the vehicle to obtain cellular RF sensing data of the environment surrounding the vehicle in response to determining the fault condition of the camera. The method also includes performing automobile navigation support based at least in part on the cellular RF sensing data of the environment surrounding the vehicle.

[0006] An example method for updating a cellular radio frequency (RF) sensing map based on cellular RF sensing includes obtaining a first set of cellular RF sensing data of an environment around the first vehicle from a first vehicle. The method also includes determining a unified cellular RF sensing map based on unifying the first set of cellular RF sensing data with a second set of cellular RF sensing data, wherein the second set of cellular RF sensing data is obtained from a second vehicle. The method also includes updating the RF sensing map based on the unified cellular RF sensing map in response to determining that a match between the unified cellular RF sensing map and an RF sensing map corresponding to the environment around the first vehicle is below a predetermined threshold.

[0007] An example method for using an RF sensing map to assist cellular RF sensing for automobile navigation includes obtaining an RF sensing map corresponding to a trip from a server prior to the trip. The method also includes obtaining cellular RF sensing data of an environment surrounding a vehicle through a cellular communication interface of the vehicle, wherein obtaining the cellular RF sensing data is responsive to a determination of a fault condition of a camera of the vehicle that prevents object detection from a camera image obtained by the camera, wherein the condition is caused by inclement weather, an obstruction on a lens of the camera, an electrical fault of the camera, or a combination thereof. The method also includes raising an alert level in response to determining that a match between the cellular RF sensing data and the RF sensing map is below a predetermined threshold, wherein raising the alert level includes at least one of reducing a driving speed of the vehicle or reverting to a manual driving mode.

[0008] An example method for automobile navigation using a cellular RF sensing map includes receiving a request from a vehicle for a cellular RF sensing map corresponding to an environment surrounding the vehicle, wherein the cellular RF sensing map is generated based on data obtained from a cellular RF communication interface. The method also includes determining the cellular RF sensing map based on specifications of the vehicle. The method also includes sending the cellular RF sensing map to the vehicle.

[0009] This summary is neither intended to identify key features or essential features of the claimed subject matter nor is it intended to be used alone to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of the disclosure, any or all of the drawings, and each claim. The foregoing and other features and examples will be described in more detail in the following description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 The accompanying drawing is a perspective view of a vehicle.

[0011] Figure 2 is a block diagram of a position estimation system according to an embodiment.

[0012] Figure 3is a block diagram of a method of obtaining radar and camera data for radar and camera map layers of an HD map according to an embodiment.

[0013] Figure 4 is a flow chart of a method for a vehicle to perform cellular RF sensing to support car navigation according to an embodiment.

[0014] Figure 5 is a flow chart of a method for performing automobile navigation based on cellular RF sensing according to an embodiment.

[0015] Figure 6 is a flow chart of a method for a server to send an RF map to a requesting vehicle to perform cellular RF sensing according to an embodiment.

[0016] Figure 7 is a flow chart of a method for obtaining data at a vehicle to update an RF layer of an HD map (eg, an RF map), according to some embodiments.

[0017] Figure 8 is a block diagram of an implementation of a mobile computing system.

[0018] Fig. 9 is a block diagram of an embodiment of a computer system.

[0019] Similar reference symbols in the various drawings indicate similar elements according to certain example implementations. In addition, multiple instances of an element may be indicated by adding a letter or hyphen and a second digit after the first digit of the element. For example, multiple instances of element 110 may be indicated as 110-1, 110-2, 110-3, etc. or 110a, 110b, 110c, etc. When only the first digit is used to refer to such an element, any instance of the element should be understood (for example, element 110 in the previous example will refer to elements 110-1, 110-2, and 110-3 or to elements 110a, 110b, and 110c). DETAILED DESCRIPTION

[0020] Several exemplary embodiments will now be described with reference to the accompanying drawings which form a part thereof. The following description provides only embodiments and is not intended to limit the scope, applicability or configuration of the present disclosure. On the contrary, the following description of the embodiments will provide a description for enabling implementation of the embodiments to those skilled in the art. It should be understood that various changes may be made to the functions and arrangements of the elements without departing from the scope of the present disclosure.

[0021] The described implementations may be implemented in any device, system, or network capable of transmitting and receiving radio frequency (RF) signals according to any communication standard, such as any of the Institute of Electrical and Electronics Engineers (IEEE) IEEE 802.11 standards (including those identified as Wi-Fi® technology standards), Bluetooth ® Standard, Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM), GSM / General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Terrestrial Trunked Radio (TETRA), Wideband CDMA (W-CDMA), Evolution-Data Optimized (EV-DO), 1xEV-DO, EV-DO Revision A, EV-DO Revision B, High Rate Packet Data (HRPD), High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolved High Speed ​​Packet Access (HSPA+), Long Term Evolution (LTE), Advanced Mobile Phone System (AMPS), or other known signals for communicating within a wireless, cellular, or Internet of Things (IoT) network, such as a system utilizing 3G, 4G, 5G, 6G or further implementations thereof.

[0022] It should be understood that although some embodiments described herein are directed to 5th generation (5G) and / or 5G new radio (NR) wireless communications, the techniques provided herein are not limited to the disclosed embodiments. Any other suitable cellular communication technology as a system utilizing 3G, 4G, 5G, 6G or further implementations thereof may be used to provide additional support to the automotive navigation as disclosed herein.

[0023] As used herein, "RF signals" include electromagnetic waves that transmit information through the space between a transmitter (or transmitting device) and a receiver (or receiving device). As used herein, a transmitter may transmit a single "RF signal" or multiple "RF signals" to a receiver. However, due to the propagation characteristics of each RF signal through multiple channels or paths, a receiver may receive multiple "RF signals" corresponding to each transmitted RF signal.

[0024] As used herein, the term "autonomous driving" for a vehicle may refer to providing any level of vehicle autonomy from level 1 to level 5 driving assistance (e.g., any vehicle autonomy). This may mean, for example, providing driver assistance (e.g., steering or acceleration / cruise control), partial driving automation (e.g., steering and acceleration / cruise control, such as an advanced driver assistance system (ADAS)), conditional driving automation (e.g., including environmental sensing in addition to partial driving automation), high driving automation, and / or full driving automation.

[0025] As used herein, the term "position estimate" of a vehicle is an estimate of the location of the vehicle within a reference frame. This may mean, for example, an estimate of the location of the vehicle in a 2D coordinate frame (e.g., latitude and longitude on a 2D map, etc.) or in a 3D coordinate frame (e.g., latitude, longitude, and altitude (LLA) on a 3D map), and may optionally include orientation information, such as heading. In some embodiments, the position estimate may include an estimate of six degrees of freedom (6-DOF) (also called "pose"), which includes translation (latitude, longitude, and altitude) and orientation (pitch, roll, and yaw) information.

[0026] As used herein, the terms "map," "map data," and their derivatives may refer to an electronic representation of a physical location or geographic area. As noted, the map data may include different "layers" of different data types (such as radar, camera, RF sensing, etc.). As will be appreciated by one of ordinary skill in the art, this electronic representation may be stored in a database or other data structure (in any of a variety of storage media) as one or more electronic files, data objects, etc.

[0027] It may be noted that the embodiments described below relate to determining the position / object detection of a vehicle, but the embodiments are not limited thereto. For example, alternative embodiments may relate to other mobile devices and / or applications in which position / object detection determinations are made. Those of ordinary skill in the art will recognize many variations of the embodiments described herein.

[0028] Autonomous driving systems and advanced driver assistance systems (ADAS) using sensors (e.g., Global Navigation Satellite System (GNSS) receivers and cameras) can provide centimeter-level accuracy. As previously mentioned, such solutions may fail in certain scenarios (tunnels, poor lighting conditions, rainy or snowy days, urban canyon scenarios, camera failures, etc.), thereby inhibiting the seamless and ubiquitous use of autonomous driving technology. On the other hand, using additional complementary systems such as radio detection and ranging (radar) or other radio frequency (RF)-based systems as backup systems may be expensive and therefore may not be available for some vehicle models that are branded to save costs, such as economy vehicles.

[0029] According to the embodiments disclosed herein, a 5G communication interface (e.g., a 5G transceiver) equipped on a vehicle and primarily configured to communicate 5G RF signals (e.g., electromagnetic waves comprising information transmitted through space between a transmitter (or transmitting device) and a receiver (or receiving device)) can be configured to provide additional detection and / or positioning in situations where a camera is unable to function (e.g., caused by severe weather, obstruction on the camera's lens, electrical failure of the camera, or a combination thereof).

[0030] When activated (e.g., when a fault condition of the camera that prevents object detection of the camera image is determined), the 5G communication interface can obtain 5G RF sensing data of the environment around the vehicle (e.g., using 5G RF signals) to provide additional object detection and / or car navigation support. For example, the 5G RF sensing data can be used to detect objects / obstacles directly (e.g., determined based on reflected 5G RF signals) or indirectly. For example, when an object / obstacle is detected indirectly, the 5G RF sensing data can be compared with an RF sensing map (e.g., received from a server) corresponding to the environment around the vehicle (e.g., the geographic area where the vehicle is located) to determine whether a change in the environment has occurred (e.g., the match between the RF sensing map and the 5G RF sensing data is below a predetermined threshold). In some embodiments, the comparison between the 5G RF sensing data and the RF sensing map can also be used to support GNSS / complementary vehicle positioning with GNSS (e.g., when the GNSS signal is poor or inoperative). In some embodiments, the RF sensing map (also referred to as an "RF map" or "5G RF sensing map") can be a layer of a crowdsourced HD map.

[0031] In some embodiments, an RF map may be obtained (e.g., downloaded from a server) in response to activation of 5G RF sensing. For example, upon detecting a fault condition of a camera that prevents object detection in a camera image, the vehicle may send a request to a server for an RF map of the environment surrounding the vehicle. The server may send the RF map in response to receiving the request. For another example, the RF map may be obtained in advance (e.g., before a trip and / or in an area where the network is good for data transmission). In some embodiments, before sending the RF map to the vehicle, the server may pre-process the RF map (e.g., adjust a unified RF map according to the specifications of the requesting vehicle and / or send an RF map corresponding to the specifications of the requesting vehicle) according to the specifications of the requesting vehicle (e.g., the model and / or arrangement of the 5G RF antenna on the vehicle).

[0032] In some embodiments, because the 5G communication system can provide approximate / rough detection of objects / obstacles that may interfere with driving and / or approximate / rough positioning of the vehicle, the alert level for the car navigation can be increased when those objects / obstacles are detected directly (e.g., based on the returned 5G RF signal) or indirectly (e.g., based on comparison with an RF map of the environment surrounding the vehicle). For example, the driving speed of the vehicle can be reduced and / or the car navigation can fall back to manual driving mode. Therefore, the 5G communication system can be implemented as a cost-effective support / supplement to camera-based detection systems for object detection and / or positioning in challenging scenarios (e.g., nighttime, bad weather, camera malfunction, underground or tunnel scenarios).

[0033] In addition, in some embodiments, upon determining that the environment has changed, the vehicle may publish 5G RF sensing data (e.g., to a server) for updating the RF map (e.g., in a crowdsourced mapping manner). For example, the server may determine whether to update the RF map. If the server determines to update the RF map, the RF map may be updated based on the published 5G RF sensing data according to the specifications of the publishing vehicle. In some embodiments, before updating the RF sensing map, the server may unify the 5G RF sensing data according to the specifications of the publishing vehicle (e.g., classify the 5G RF sensing data according to the specifications of the publishing vehicle and / or adjust the 5G RF sensing data to a unified format according to the specifications of the publishing vehicle).

[0034] Relative to Figure 1 and Figure 2 Additional details are provided regarding object detection / localization of the vehicle. Note that localization and object detection are described somewhat interchangeably herein. This is because localization can be based in part on object detection, and vice versa. For example, a localization function can include performing an object detection function (e.g., including using a sensor to detect an environment, and using the detected data to localize the vehicle), and a detection function can also include performing a localization function (e.g., including comparing the detected data to a corresponding map based on the determined position of the vehicle). Thus, although Figure 1 and Figure 2 The components and functionality of a positioning system are described, but it should be noted that these components can also be used for object / obstacle detection in automotive navigation.

[0035] Figure 1 is a perspective view of a vehicle 110 according to an embodiment, illustrating how the positioning of the vehicle 110 may generally be performed. In some embodiments, the vehicle 110 may also perform crowdsourced mapping (e.g., generating / updating an HD map including an RF map layer based on crowdsourcing) based on the positioning of the vehicle 110. Here, the vehicle 110 (also referred to as the "ego vehicle") may first determine its position. In some embodiments, the vehicle 110 may also use its determined position in conjunction with sensors such as cameras, radars, or RF interfaces (e.g., RF transceivers) to collect information of one or more layers of the HD map under certain conditions (e.g., determining that a change in a geographic area has occurred). In some embodiments, as described above, the vehicle's position and crowdsourced maps may then also be used in object / obstacle detection for car navigation support / supplementation (e.g., comparing 5G RF sensing data with corresponding RF maps).

[0036] Positioning may be performed using a GNSS receiver at the vehicle 110 to receive radio frequency (RF) signals transmitted by the GNSS satellites 120. (Of course, although for visual simplicity, Figure 1The satellite 120 in FIG. 1 is illustrated as being relatively close to the vehicle 110, but it should be understood that the satellite 120 will be in orbit around the earth. In addition, the satellite 120 may be part of a large satellite constellation of a GNSS system. Figure 1 Additional satellites of such a constellation are not shown.) Additionally or alternatively, ground positioning may be performed using RF signals from ground beacons and transceivers (such as base stations from cellular communication networks). Vehicle sensors and HD maps may also be used to help determine the exact location of vehicle 110. (For Figure 2 Additional details are provided on how these various components can be used for positioning.)

[0037] The location of the vehicle 110 may be used for purposes such as vehicle maneuvering, navigation, car navigation, and mapping. For example, as described above, when detecting objects / obstacles by comparing sensor data (e.g., 5G RF sensing data) with a map (e.g., an RF map obtained from a server), the determined location of the vehicle may help determine the portion of the map corresponding to the geographic area of ​​the vehicle (e.g., the 5G RF sensing data may be compared to the corresponding portion of the map to determine whether the geographic area represented by the map has changed). In some embodiments, positioning may also be used when updating a map (e.g., using sensor data to determine the corresponding portion of the map to be updated).

[0038] Figure 2 is a block diagram of a position estimation system 200 according to an embodiment. The position estimation system 200 collects data from a variety of different sources and outputs a position estimate of the vehicle. The position estimate can be used by an autonomous driving system, an ADAS system, and / or other automotive navigation systems on the vehicle, as well as systems remote from the vehicle (e.g., a traffic monitoring system). In addition, as described, the vehicle's mapping system can use the vehicle's position estimate when performing the techniques for mapping described below. The position estimation system 200 includes sensors 205, which include one or more cameras 210, an inertial measurement unit (IMU) 220, and a GNSS unit 230. In some embodiments, the position estimation system 200 may also include a radar 235. It is contemplated that, although a radar 235 is described herein, the techniques disclosed herein may also be implemented in a system without a radar 235.

[0039] The position estimation system 200 further includes a perception unit 240, a map database 250, and a positioning unit 260, which includes a sensor positioning unit 270 and a map fusion unit 280. In an alternative embodiment, Figure 2The illustrated components may be combined, separated, omitted, rearranged, and / or otherwise modified depending on the desired functionality. Furthermore, in alternative embodiments, additional or alternative data and / or data sources may be used to determine the position estimate. For example, sensor 205 may include one or more additional or alternative sensors (e.g., lidar, sonar, etc.). One or more components of position estimation system 200 may be implemented in hardware and / or software, such as Figure 8 One or more hardware and / or software components of the mobile computing system 800 illustrated in and described in more detail below. For example, the positioning unit 260 can be implemented by one or more processing units. The various hardware and / or software components that implement the position estimation system 200 can be distributed at various different locations on the vehicle depending on the desired functionality.

[0040] The wireless transceiver 225 may include one or more RF transceivers (e.g., Wi-Fi transceivers, wireless wide area network (WWAN) or cellular transceivers, Bluetooth transceivers, etc.) for receiving positioning data from various ground positioning data sources. These ground positioning data sources may include, for example, Wi-Fi access points (APs) (Wi-Fi signals including dedicated source range communication (DSRC) signals), cellular base stations (e.g., cellular-based signals such as positioning reference signals (PRS) or signals communicated via vehicle-to-everything (V2X), cellular V2X (CV2X), or long-term evolution (LTE) direct protocols, etc.), and / or other positioning sources such as roadside units (RSUs), etc. The wireless transceiver 225 may also be used for wireless communication (e.g., via Wi-Fi, cellular, etc.), in which case the wireless transceiver 225 may be incorporated into the wireless communication interface of the vehicle.

[0041] In some embodiments, the wireless transceiver 225 may also include a 5G RF transceiver (also referred to as a 5G communication interface), which is primarily configured to communicate 5G RF signals (e.g., send and receive 5G RF signals). As will be described in detail below, under certain conditions (e.g., when a fault condition of the camera that prevents object detection of the camera image is determined), the 5G RF transceiver may be configured to provide additional detection and / or positioning as a supplement to the other components described in the position estimation system 200. In some embodiments, the 5G RF transceiver (e.g., a 5G RF antenna) may be positioned forward, upward, backward, downward, and / or otherwise on the top of the vehicle to communicate 5G RF signals and collect information about the environment around the vehicle when activated. In some embodiments, the 5G RF transceiver may include an antenna array capable of sending and / or receiving RF signals using one or more directional beams. These beams may be fixed relative to the vehicle (e.g., facing forward, facing upward, etc.) or may be dynamic (e.g., "steering" in different directions).

[0042] The GNSS unit 230 may include a GNSS receiver and a GNSS processing circuit configured to receive signals from GNSS satellites (e.g., satellites 120) and GNSS-based positioning data. The positioning data output by the GNSS unit 230 may vary depending on the desired functionality. In some embodiments, the GNSS unit 230 may provide, among other things, a three-degree-of-freedom (3-DOF) position determination (e.g., latitude, longitude, and altitude). Additionally or alternatively, the GNSS unit 230 may output basic satellite measurements for performing 3-DOF position determination. Additionally or alternatively, the GNSS unit may output raw measurements, such as pseudoranges and carrier phase measurements.

[0043] The camera 210 may include one or more cameras disposed on or in the vehicle that are configured to capture images from the perspective of the vehicle to help track the movement of the vehicle. The camera 210 may face forward, upward, backward, downward, and / or be otherwise positioned on the vehicle. Other aspects of the camera 210, such as resolution, light band (e.g., visible light, infrared (IR), etc.), frame rate (e.g., 30 frames per second (FPS)), etc., may be determined based on the desired functionality. Various image processing techniques may be used to track the movement of the vehicle 110 from the images captured by the camera 210 to determine motion blur, object tracking, etc. The raw images and / or information generated therefrom may be passed to the sensor positioning unit 270, which may perform visual inertial odometry (VIO) using data from both the camera 210 and the IMU 220.

[0044] The IMU 220 may include one or more accelerometers, gyroscopes, and / or (optionally) other sensors (such as a magnetometer) to provide inertial measurements. Similar to the camera 210, the output of the IMU 220 to the sensor positioning unit 270 may vary depending on the desired functionality. In some embodiments, the output of the IMU 220 may include information indicating the 3-DOF position or 6-DOF attitude of the vehicle 110 and / or the 6-DOF linear velocity and angular velocity of the vehicle 110, and may be provided periodically based on a schedule and / or in response to a triggering event. The position information may be relative to an initial position or a reference position. Alternatively, the IMU 220 may provide raw sensor measurements.

[0045] In some embodiments, radar 235 may include one or more radar sensors disposed in or on the vehicle. Similar to camera 210, the radar may be positioned forward, upward, backward, downward, and / or otherwise positioned on the vehicle to collect information about the vehicle's surroundings. According to some embodiments, the radar may scan an area or volume near the vehicle at a rate of one or more times per second or several times per second (e.g., 5, 10, 20, 50, or 100 times per second), and the scanning rate may be dynamic, depending on sensor capabilities, processing capabilities, traffic conditions, etc. Radar scans may also be referred to as "frames" herein.

[0046] Note that radar can complement cameras to help provide robust autonomous features, for example, achieving truly autonomous driving may require a robust solution for positioning in all types of weather and environmental conditions, so that the vehicle knows its posture within a few centimeters. Just like the human eye, lidar and cameras may not be able to observe at night or when there is too much fog in the surrounding environment. Global positioning sensors (such as GNSS) may not be available in underground or tunnel scenarios, and may also face challenges in urban canyon scenarios. In some embodiments, radar sensors can utilize lower frequencies, such as using millimeter wave (mmWave) radar (e.g., with a frequency in the range of 30GHz to 300GHz) to achieve sub-meter accuracy positioning in such challenging scenarios. However, as mentioned above, radar may not always be available for many vehicles due to reasons such as cost savings. Therefore, the 5G RF communication interface can be used to provide some degree of support to car navigation in such challenging scenarios (e.g., determining whether to increase the autonomous driving alert level).

[0047] The sensor localization unit 270 may include a module (implemented in software and / or hardware) configured to fuse data from the sensor 205 to determine the location of the vehicle. As described, the sensor localization unit 270 may perform VIO by combining data received from the camera 210 and the IMU 220. In addition to or in lieu of VIO data, the sensor localization unit 270 may utilize data from the GNSS unit 230, the radar 235, and / or the wireless transceiver 225 to determine the location of the vehicle and / or modify the determined location of the vehicle. In some embodiments, data from different sensors may be given different weights based on input type, confidence metrics (or other indications of the reliability of the input), etc. In general, the sensor localization unit 270 may output an estimated location of the vehicle based on the received input. Depending on the accuracy of the received input (e.g., the accuracy of the data from the sensor 205), the output of the sensor localization unit 270 may include a highly accurate vehicle position estimate in the global frame (or other reference frame) of the map fusion unit 280.

[0048] The map fusion unit 280 operates to provide a vehicle position estimate within a map framework based on the position estimate from the sensor localization unit 270 and information from the map database 250 and the perception unit 240. The map database 250 may provide a map (e.g., a high definition (HD) map in the form of one or more electronic files, data objects, etc.) of the area in which the vehicle 110 is located. As will be disclosed in detail below, the map of the area / trip may be obtained / downloaded to the map database 250 in advance of the trip, in real time, or in response to certain conditions (e.g., determining a fault condition of a camera that prevents object detection in the camera image).

[0049] The perception unit 240 can observe lane markings, traffic signs, and / or other visual features in the vehicle's surroundings. To this end, the perception unit 240 can include a feature extraction engine that performs image processing and computer vision on images received from the camera 210. In some embodiments, the perception unit 240 can also operate using input from the radar 235 and / or other sensors (e.g., lidar).

[0050] As previously described, the position estimate provided by the map fusion unit 280 (i.e., the output of the positioning unit 260) can serve any of a variety of functions depending on the desired functionality. For example, it can be provided to the autonomous driving, ADAS, and / or other systems of the vehicle 110 (and can be transmitted via a controller area network (CAN) bus), communicated to a device separate from the vehicle 110 (including other vehicles; servers maintained by government agencies, service providers, etc.; etc.), displayed on a display of the vehicle (e.g., displayed to the driver or other users for navigation or other purposes), etc. According to embodiments herein, the vehicle can additionally use the vehicle's position to perform crowdsourced mapping.

[0051] Reference again Figure 1 , when the vehicle 110 is traveling within a geographic area corresponding to an area of ​​the map, the vehicle 110 may use different sensors (e.g., sensor 205) to collect information of different corresponding map layers of the HD map. For example, one or more cameras (e.g., camera 210) may be used to collect information of a camera map layer. Optionally, a radar (e.g., radar 235) may be used to collect information of a radar map layer, a lidar may be used to collect information of a lidar map layer, and so on, and an RF transceiver (e.g., a 5G RF communication interface) may be used to collect information of an RF map layer. Map layer information may be collected not only about the road 125 on which the vehicle 110 is traveling (e.g., lane boundary information, road curvature, road hazards, etc.), but also about static objects 130 in or near the road 125. (Although Figure 1The static objects in the map are illustrated as trees, but other static objects may include traffic signs, sidewalks, traffic lights, mile markers, etc.). As described in further detail below, the map layer information may also filter out dynamic objects 140. Dynamic objects may generally include moving objects, such as other vehicles, pedestrians, cyclists, etc.

[0052] As previously mentioned, radar can be complementary to other sensors and / or map layers and can be used for autonomous driving and other features in all types of weather and environmental conditions, allowing the vehicle to know its attitude within a few centimeters. However, as mentioned above, radar may not always be available for many vehicles for reasons such as cost savings. Therefore, the 5G RF communication interface can be used to provide some level of safety / support to the car navigation in such challenging scenarios (e.g., determining whether to increase the autonomous driving alert level).

[0053] Figure 3 is a high-level conceptual diagram illustrating how a vehicle may perform 5G RF sensing to support automotive navigation according to some embodiments. The functionality illustrated in box 305 may be performed by the vehicle, and the functionality illustrated in box 310 may be performed by a server (e.g., a cloud / edge server). Alternative embodiments may distribute the functionality in different manners between the vehicle and the server. In addition, while both camera and RF sensing map layers are described, some embodiments may include one or the other (e.g., 5G RF sensing or cameras) but not both. Furthermore, to enable the generation and maintenance of a crowdsourced map of a geographic area, many vehicles may each perform the functionality at box 305 simultaneously and / or at different times while in the geographic area. The functionality illustrated in box 305 may be performed by a computing system of the vehicle, such as Figure 8 The mobile computing system 800 illustrated in and described below. Figure 3 The car navigation functionality illustrated in the example can be combined with Figure 2 In addition, the functionality in block 310 may be performed by one or more computer systems (e.g., Figure 8 The communication between the vehicle 315 and the server may be facilitated via a wireless communication network (eg, a cellular / mobile communication network and / or a similar wireless wide area network (WWAN)).

[0054] Beginning at block 320, the vehicle 315 may perform car navigation (e.g., any level of autonomous driving) based on a camera-based sensor that collects camera data (e.g., camera images). In some embodiments, the vehicle 315 may include a camera (e.g., Figure 2210 shown in ). For example, the camera may include a stereo camera configured to collect data about depth perception of the surrounding area and provide information about aspects including the location, distance, speed, etc. of an object. The camera may capture the same scene from two different viewpoints, simulating a pair of eyes. Using triangulation and based on the arrangement of pixels of the camera images, software (e.g., run by a processor) may compare the camera images and determine the depth information required for a 3D image.

[0055] In some embodiments, the camera may include a time-of-flight (ToF) camera that measures the distance of an object. In some embodiments, the detected camera image may include at least two types of information about each pixel of the camera image: an intensity value given as a grayscale value and the distance of the object from the camera (e.g., depth of field). In some embodiments, the ToF camera may be equipped with an image chip with thousands of receiving elements. This allows the scene to be completely captured with a high level of detail in a single shot. It should be understood that the type of camera is not limited to the embodiments disclosed herein, and any other suitable type of camera (e.g., a combination camera) may be used to provide driving assistance.

[0056] At block 325, a camera fault condition may be determined / detected. In some embodiments, a camera fault condition may be determined if an object may not be detected in the camera image and / or an object beyond a predetermined distance may not be detected in the camera image. For example, as described above, Figure 2 The perception unit 240 shown in the figure can observe lane markings, traffic signs and / or other visual features in the surrounding environment of the vehicle by performing image processing and computer vision on the camera image detected by the camera. In some cases where the perception unit 240 may not detect an object in the camera image and / or may not detect an object beyond a predetermined distance in the camera image, the camera may be determined to be in a fault condition. In some embodiments, a camera map corresponding to the determined location of the vehicle 315 (e.g., included in an HD map) can be received from a server. If the matching level between the camera image and the corresponding camera map is below a predetermined threshold, a fault condition of the camera can be determined.

[0057] In some embodiments, the camera's fault condition may be caused by bad weather (e.g., fog, rain, snow, storm), an obstruction on the camera's lens (e.g., dust blocking a portion of the lens or the entire aperture), low light, an electrical failure of the camera (e.g., malfunction), or a combination thereof. Thus, when a camera's fault condition is detected, the performance of the camera-based car navigation may be negatively impacted. Therefore, failing to take prompt action in response to this determination would be dangerous.

[0058] At block 330, in response to determining that a fault condition of the camera has occurred, the 5G communication interface (e.g., Figure 2 The 5G RF transceiver in the wireless transceiver 225 shown in the figure) is used to perform 5G RF sensing of the environment around the vehicle 315 to provide additional support for car navigation. For example, the 5G communication interface can be configured to send a 5G signal to the surrounding environment of the vehicle 315 (e.g., scan the surrounding environment of the vehicle 315). For example, the 5G communication interface can transmit a 5G RF sensing signal in a predetermined direction (e.g., the front, back, and / or different sides), and can detect a returned 5G RF sensing signal (e.g., 5G RF sensing data) reflected by an object / obstacle. The presence and / or position of the object / obstacle can be determined based on the time difference between the transmission of the 5G RF sensing signal and the reception of the returned 5G RF sensing signal reflected by the object / obstacle, as well as the transmission direction of the 5G RF sensing signal.

[0059] In some embodiments, in direct detection, the presence and / or location of any object / obstacle that may interfere with the driving of the vehicle 315 / cause a safety problem to the driving of the vehicle (e.g., a fallen tree / branch or rock located in the middle of the road or in the path of the vehicle 315) may be directly determined based on the 5G RF sensing data. For example, upon determining the presence of an object that may interfere with the driving trajectory of the vehicle 315, the alert level of the car navigation may be increased accordingly (e.g., the driving speed may be reduced, and / or the car navigation mode may be exited and returned to the manual mode).

[0060] In some embodiments, in indirect detection, as illustrated in arrow 335, an RF map corresponding to a geographic area of ​​vehicle 315 (e.g., determined based on the positioning of vehicle 315 as described above) may be received / downloaded from a server. For example, in some embodiments, before sending the RF map to vehicle 315, the server may pre-process the RF map according to the specifications of the requesting vehicle (e.g., vehicle 315) (e.g., the model and / or arrangement of the 5G RF antennas on the vehicle). For example, the server may adjust a unified RF map (e.g., a layer of an HD map) according to the specifications of the requesting vehicle. In some embodiments, the server may adjust the unified RF map according to the vehicle model, 5G RF antenna arrangement (e.g., height, distribution, transmission / detection angles and / or scanning patterns) of the requesting vehicle, etc., so that the RF map sent to the requesting vehicle may be compatible with the specifications of the requesting vehicle. In some embodiments, the RF map may be obtained / downloaded to the memory of the requesting vehicle (e.g., in real time) before the trip, or in response to activation of 5G RF sensing. Figure 2 ).

[0061] At block 340, the 5G RF sensing data may be compared to the RF map received from the server to determine whether the match between the 5G RF sensing data and the RF map is below a predetermined threshold. For example, if the match level is below a predetermined threshold, a change in the environment may be determined. Thus, similar to direct detection, at the end of block 340, if the match level is below a predetermined threshold, the alert level of the car navigation may be increased accordingly.

[0062] Because 5G RF signals have better penetration of inclement weather than visible light, 5G RF sensing can complement camera-based automotive navigation (e.g., with better detection capabilities) in those challenging scenarios and / or when camera failure conditions occur.

[0063] In some embodiments, in response to determining at box 340 that the match level is below a predetermined threshold, as shown by arrow 345, the vehicle 315 may further upload / publish 5G RF sensing data to the server for updating the RF map stored in the server (e.g., crowdsourcing mapping). In some embodiments, the 5G RF sensing data may be updated together with data indicating the match level to the server. In some embodiments, the 5G RF sensing data may be uploaded in real time (e.g., streaming), at predetermined intervals, or at the end of a trip. In some embodiments, for the purpose of RF map updating, before comparing the 5G RF sensing data to the RF map, dynamic objects detected in the 5G RF sensing data, such as moving vehicles or pedestrians, may be filtered out, for example, on a frame-by-frame basis (e.g., similar to a method for filtering out unwanted targets in radar data). This can reduce the amount of data to be published by removing unwanted information from the raw data (e.g., 5G RF sensing data).

[0064] For example, frame-by-frame filtering may include a Doppler check, a self-speed check, and an optional metadata check. It may be noted that some embodiments may not perform the functionality of all functions disclosed herein and / or may not necessarily perform the operations in the order described. In order to identify and remove moving targets, Doppler checks and self-speed checks may be performed on a per-frame basis, which may filter out unwanted or noisy points. As will be appreciated by one of ordinary skill in the art, the Doppler check may perform better filtering of moving objects at higher speeds, while the self-speed check may perform better filtering of moving objects at lower speeds. The optional metadata check may include checks of certain metrics (e.g., signal-to-noise ratio (SNR), specific indicators on multipath targets, etc.) to determine the reliability of the 5G RF sensing data and filter out data that fails to meet a minimum threshold.

[0065] The Doppler check can be performed, for example, by estimating the radial velocity of the objects detected in the environment and comparing the detected velocity of the point object with the theoretical velocity of a stationary object (given the vehicle's velocity). For example, if the ego velocity of a 5G RF interface mounted on a vehicle is And the 5G RF interface is installed at a yaw angle (relative to the forward direction) equal to If the target is on a vehicle with an azimuth The expected Doppler velocity at is Since typically the vehicle velocity will not be measured at the 5G RF interface itself, but at some other fixed frame on the vehicle, it may be necessary to transform the vehicle's measured velocity to the 5G RF sensing frame using the vehicle's linear and angular velocities measured at the fixed frame. Doppler checking can be done by filtering out the reported Doppler velocities at All targets outside the m / s range are achieved. The typical value can be less than 1m / s.

[0066] In some embodiments, after the frame-by-frame filtering, the batch filtering may include performing batch filtering based on a clustering algorithm on the 5G RF sensing data to further reduce the amount of raw data. It may be noted that some embodiments may not necessarily perform the filtering operations in the order described herein. That is, batch filtering may not necessarily be performed after the frame-by-frame filtering operation is completed.

[0067] Once the server receives the published data, the server may perform the operations illustrated in box 310. These operations may include performing the functionality illustrated at box 370, where the server unifies information received from multiple vehicles. As described, the server may be communicatively coupled to many vehicles in a particular geographic area. Using the functionality illustrated in box 305, each of these vehicles may publish information to the server including the specifications of the vehicle. However, the sensor types (e.g., arrangement, layout, parameters) and other specifications of the publishing vehicles may differ between the publishing vehicles, resulting in sensor data having different coordinate information, scales, formats, etc. Therefore, the unification performed at box 370 may include unifying the data to a common coordinate system, scale, format, etc. according to the specifications of the vehicle published with the 5G RF sensing data (e.g., including specifications of the 5G RF interface).

[0068] At box 373, the server may then aggregate the data (e.g., 5G RF sensing data), thereby storing data published by multiple vehicles within the time window. The server may continue to aggregate the data until an update trigger is received, as shown by arrow 375. Depending on the desired functionality, the trigger may cause the server to aggregate data across periodic time windows and / or before a particular event occurs. Thus, map update triggers may include periodic triggers (e.g., once every six hours, once every 12 hours, once a day, etc.) and / or event-based triggers (e.g., based on determining that the quality of the HD map layer is below a threshold and needs to be updated, based on input received from one or more vehicles, etc.).

[0069] In response to the map update trigger activating switch 377 (e.g., a logical switch), the server may process the aggregated data to determine an offline trajectory and optionally a map optimization, as indicated at block 380. This may generate newly optimized RF sensing layer information, which may be used to update the RF map layer, as shown at block 385. More specifically, because the newly optimized RF map layer information generated at block 380 may be for a portion (e.g., a block in a city) of a larger / global HD map (e.g., a city, state, country, etc.), the update performed at block 385 may include an update of the larger / global HD map layer using the newly optimized RF map layer.

[0070] Figure 4 is a flow chart of a method 400 for a vehicle to perform cellular RF sensing to support vehicle navigation according to an embodiment. Figure 4 Means of functionality illustrated in one or more of the shown blocks may be performed by hardware and / or software components of a vehicle or, more generally, a mobile computing system (which may be incorporated into a vehicle). Figure 8 Example components of a mobile computing system are illustrated in FIG. 4 and described in more detail below.

[0071] At block 410, the functionality includes obtaining a camera image of the environment surrounding the vehicle using a camera at the vehicle. As indicated in the previously described embodiments, the vehicle (e.g., Figure 3 The vehicle 315 shown in FIG. 315 may include a stereo camera, a ToF camera, a combination camera, or any other camera suitable for obtaining camera images of the environment around the vehicle for autonomous driving. The components for performing the functionality at block 410 may include a bus 805, a processor 810, a digital signal processor (DSP) 820, a wireless communication interface, a sensor 840 (including a radar 841), a memory 860, and / or other components of the mobile computing system 800, such as Figure 8 exemplified in .

[0072] At block 420, the functionality includes determining a fault condition of the camera that prevents object detection in the camera image. As indicated in the previously described embodiments, if: 1. Figure 2 1. The perception unit 240 shown in ) may not detect an object (e.g., lane markings, traffic signs, and / or other visual features in the vehicle's surroundings); 2. No objects beyond a predetermined distance may be detected in the camera image; or a combination thereof, it may be determined that the camera is in a fault condition. In some embodiments, a camera map corresponding to the determined location of the vehicle may be received from a server (e.g., included in an HD map). If the matching level between the camera image and the corresponding camera map is below a predetermined threshold, a fault condition of the camera may be determined. In some embodiments, the fault condition of the camera may be caused by bad weather (e.g., fog, rain, snow, storm), obstruction on the lens of the camera (e.g., dust blocks part or the entire aperture of the lens), low light, electrical failure of the camera (e.g., malfunction), or a combination thereof. Components for performing the functionality at box 410 may include a bus 805, a processor 810, a digital signal processor (DSP) 820, a wireless communication interface, a sensor 840 (including a radar 841), a memory 860, and / or other components of the mobile computing system 800, such as Figure 8 exemplified in .

[0073] At block 430, the functionality includes performing vehicle navigation based at least in part on cellular RF sensing data of the environment surrounding the vehicle. As indicated in the previously described embodiments, in response to determining that a fault condition of the camera occurs, the cellular communication interface (e.g., Figure 2 A cellular RF transceiver in the wireless transceiver 225 shown in FIG. 1 is used to perform cellular RF sensing of the environment around the vehicle to provide additional support for car navigation.

[0074] For example, Figure 5 It is used to execute according to the implementation plan Figure 4 430 is a flowchart of a method 500 for executing Figure 5 Means of functionality illustrated in one or more of the shown blocks may be performed by hardware and / or software components of a vehicle or, more generally, a mobile computing system (which may be incorporated into a vehicle). Figure 8 Example components of a mobile computing system are illustrated in FIG. 4 and described in more detail below.

[0075] At block 532, the functionality includes configuring the cellular communication interface to transmit a cellular signal to the surroundings of the vehicle (e.g., to scan the surroundings of the vehicle). For example, the cellular communication interface may transmit a cellular RF sensing signal in a predetermined direction (e.g., the front, back, and / or different sides), and may detect a returned cellular RF sensing signal (e.g., cellular RF sensing data) reflected by an object / obstacle. The presence and / or location of the object / obstacle may be determined based on a time difference between the transmission of the cellular RF sensing signal and the reception of the returned cellular RF sensing signal reflected by the object / obstacle, and the transmission direction of the cellular RF sensing signal.

[0076] At box 534, the functionality includes determining whether a direct interference object is detected (e.g., a fallen tree / branch or rock located in the middle of the road or in the trajectory of the vehicle 315). In some embodiments, if a direct interference object is detected (the output at box 534 is "yes"), then at box 536, the functionality includes performing car navigation support based at least in part on cellular RF sensing data of the environment surrounding the vehicle. For example, in direct detection, the presence and / or location of any object / obstacle that may interfere with the driving of the vehicle / pose a safety issue to the driving of the vehicle can be directly determined based on the cellular RF sensing data. As indicated in the previously described embodiments, performing car navigation support can include increasing the alert level of the car navigation accordingly (e.g., reducing the driving speed, and / or exiting the car navigation mode and returning to the manual mode).

[0077] In some embodiments, if no direct interfering object is detected (the output at box 534 is "No"), then in indirect detection, at box 538, the functionality includes obtaining an RF map corresponding to the geographic area of ​​the vehicle (e.g., determined based on the positioning of the vehicle as described above) from the server. For example, in some embodiments, before sending the RF map to the vehicle, the server may pre-process the RF map according to the specifications of the requesting vehicle (e.g., the vehicle) (e.g., the model and / or arrangement of the cellular RF antennas on the vehicle). For example, the server may adjust the unified RF map (e.g., the layer of the HD map) according to the specifications of the requesting vehicle. In some embodiments, the server may adjust the unified RF map according to the vehicle model, cellular RF antenna arrangement (e.g., height, distribution, transmission / detection angles and / or scanning patterns) of the requesting vehicle, etc., so that the RF map sent to the requesting vehicle is compatible with the specifications of the requesting vehicle. In some embodiments, the RF map may be obtained / downloaded to the memory of the requesting vehicle (e.g., in real time) before the trip, or in response to the activation of cellular RF sensing. Figure 2 ).

[0078] In some embodiments, at box 539, the functionality includes determining whether the level of match between the cellular RF sensing data and the RF map received from the server is below a predetermined threshold. For example, if the level of match is below a predetermined threshold, a change in the environment can be determined. Therefore, method 500 proceeds to box 536, where the functionality includes performing car navigation support based at least in part on the cellular RF sensing data of the environment surrounding the vehicle. As indicated in the previously described embodiments, performing car navigation support may include increasing the alert level of the car navigation accordingly (e.g., reducing the driving speed, and / or exiting the car navigation mode and returning to the manual mode).

[0079] The means for performing the functionality at blocks 532 , 534 , 536 , 538 , and 539 may include a bus 805 , a processor 810 , a digital signal processor (DSP) 820 , a wireless communication interface, sensors 840 (including radar 841 ), a memory 860 , and / or other components of the mobile computing system 800 , such as Figure 8 exemplified in .

[0080] Re-reference Figure 4 , in response to determining that the matching level is below a predetermined threshold ( Figure 5 ), method 400 proceeds to block 440, where functionality includes sending the cellular RF sensing data to a server for updating an RF map. As indicated in the previously described embodiments, the vehicle 315 may also upload / publish cellular RF sensing data to a server for updating an RF map stored in the server (e.g., crowdsourced mapping). In some embodiments, the cellular RF sensing data may be updated along with data indicating a level of match to the server. In some embodiments, the cellular RF sensing data may be uploaded in real time (e.g., streaming), at predetermined intervals, or at the end of a trip. In some embodiments, for RF map update purposes, dynamic objects detected in the cellular RF sensing data, such as moving vehicles or pedestrians, may be filtered out, for example, on a frame-by-frame basis (e.g., similar to methods for filtering out unwanted targets in radar data) before comparing the cellular RF sensing data to the RF map. This may reduce the amount of data to be published by removing unwanted information from the raw data (e.g., cellular RF sensing data).

[0081] For example, frame-by-frame filtering may include a Doppler check, a self-velocity check, and an optional metadata check. It may be noted that some embodiments may not perform the functionality of all functions disclosed herein and / or may not necessarily perform the operations in the order described. In order to identify and remove moving targets, a Doppler check and a self-velocity check may be performed on a per-frame basis, which may filter out unwanted or noisy points. As one of ordinary skill in the art will appreciate, the Doppler check may perform better filtering of moving objects at higher speeds, while the self-velocity check may perform better filtering of moving objects at lower speeds. The optional metadata check may include checks of certain metrics (e.g., signal-to-noise ratio (SNR), specific indicators on multipath targets, etc.) to determine the reliability of the cellular RF sensing data and filter out data that fails to meet a minimum threshold.

[0082] The Doppler check can be performed, for example, by estimating the radial velocity of objects detected in the environment and comparing the detected velocity of the point object with the theoretical velocity of a stationary object (given the vehicle's velocity). For example, if the ego velocity of a cellular RF interface mounted on a vehicle is and the cellular RF interface is mounted at a yaw angle (relative to the forward direction) equal to If the target is on a vehicle with an azimuth The expected Doppler velocity at is Since typically the vehicle velocity will not be measured at the cellular RF interface itself, but rather at some other fixed frame on the vehicle, it may be necessary to transform the vehicle's measured velocity to the cellular RF sensing frame using the vehicle's linear and angular velocities measured at the fixed frame. Doppler checking can be done by filtering out the reported Doppler velocities at All targets outside the m / s range are achieved. The typical value can be less than 1m / s.

[0083] In some embodiments, after the frame-by-frame filtering, the batch filtering may include performing batch filtering based on a clustering algorithm on the cellular RF sensing data to further reduce the amount of raw data. It may be noted that some embodiments may not necessarily perform the filtering operations in the order described herein. That is, the batch filtering may not necessarily be performed after the frame-by-frame filtering operation is completed. The components for performing the functionality at block 440 may include a bus 805, a processor 810, a digital signal processor (DSP) 820, a wireless communication interface, a sensor 840 (including a radar 841), a memory 860, and / or other components of the mobile computing system 800, such as Figure 8 exemplified in .

[0084] Figure 6is a flow chart of a method 600 for a server to send an RF map to a requesting vehicle to perform cellular RF sensing according to an embodiment. Figure 6 The functional components illustrated in one or more of the illustrated blocks may be performed by hardware and / or software components of a vehicle or more generally a mobile computing system (which may be incorporated into a vehicle). Fig. 9 Example components of a computing system are illustrated in FIG. 4 and described in more detail below.

[0085] At block 610, the functionality includes receiving a request from the vehicle for a cellular RF sensing map corresponding to an environment surrounding the vehicle. As indicated in previously described embodiments, the cellular RF sensing map may be a layer of an HD map or a map generated based on data obtained from cellular RF communication interfaces of different service vehicles (e.g., a crowdsourced cellular RF sensing map). In some embodiments, the corresponding environment surrounding the requesting vehicle may be determined based on the location of the requesting vehicle. Components for performing the functionality at block 610 may include the bus 905, the processor 910, the input device 915, the communication subsystem 930 (wireless communication interface 933), the memory 935, and / or other components of the computer system 900, such as Fig. 9 exemplified in .

[0086] At box 620, the functionality includes determining a cellular RF sensing map based on the specifications of the vehicle. As indicated in the previously described embodiments, the cellular RF sensing map stored at the server can be unified (e.g., cellular RF sensing data classified according to the specifications of the issuing vehicle and / or cellular RF sensing data adjusted according to a unified format based on the specifications of the issuing vehicle). For example, the server can pre-process the cellular RF sensing map according to the specifications of the requesting vehicle (e.g., vehicle 315) (e.g., the model and / or arrangement of the cellular RF antennas on the vehicle). For example, the server can adjust the unified cellular RF sensing map (e.g., a layer of an HD map) according to the specifications of the requesting vehicle. In some embodiments, the server can adjust the unified cellular RF sensing map according to the vehicle model, cellular RF antenna arrangement (e.g., height, distribution, transmission / detection angles and / or scanning patterns) of the requesting vehicle, etc., so that the cellular RF sensing map sent to the requesting vehicle can be compatible with the specifications of the requesting vehicle. Means for performing the functionality at block 620 may include bus 905, processor 910, input device 915, communication subsystem 930 (wireless communication interface 933), memory 935, and / or other components of computer system 900, such as Fig. 9 exemplified in .

[0087] At block 630, the functionality includes sending the cellular RF sensing map to the vehicle. As indicated in the previously described embodiments, the cellular RF sensing map is sent in response to receiving a request, or the cellular RF sensing map is obtained in advance (e.g., before the trip and / or in an area with a good network for data transmission). The components for performing the functionality at block 630 may include the bus 905, the processor 910, the input device 915, the communication subsystem 930 (wireless communication interface 933), the memory 935, and / or other components of the computer system 900, such as Fig. 9 exemplified in .

[0088] Figure 7 is a flow chart of a method 700 for obtaining data of an RF layer (eg, RF map) of an HD map at a vehicle according to an embodiment. Figure 7 The functional components illustrated in one or more of the illustrated blocks may be performed by hardware and / or software components of a server or more generally a computer system. Fig. 9 Example components of computer system 900 are illustrated in FIG. 9 and described in more detail below.

[0089] At box 710, the functionality includes receiving cellular RF sensing data from the vehicle. As indicated in the previously described embodiments, receiving the cellular RF sensing data from the vehicle may occur after the computer server initially provides the vehicle with an RF map within a geographic area corresponding to at least a portion of the HD map. Providing the RF map may be in response to a request from the vehicle. In addition, the request may indicate that the vehicle can ingest an RF map layer of the HD map. As described, the vehicle may send the cellular RF sensing data in response to determining that the level of match between the cellular RF sensing data and the RF map is below a threshold. In addition, the cellular RF sensing data may include cellular RF sensing data obtained by a cellular RF communication interface of the vehicle, the cellular RF sensing data being based at least in part on a frame-by-frame basis on a Doppler velocity measured by the cellular RF communication interface and the linear velocity and angular velocity of the vehicle when the cellular RF sensing data is obtained. The cellular RF sensing data may be obtained within a geographic area corresponding to a sub-area / tile of the HD map. The cellular RF sensing data may be used, in particular, for position determination of the vehicle and / or positioning of the vehicle in the HD map.

[0090] The content of the received cellular RF sensing data (and / or the aggregated cellular RF sensing data described below with respect to box 720) and / or the manner in which the cellular RF sensing data is received may vary depending on the desired functionality. The cellular RF sensing data may be filtered, as previously described, and / or may include other types of cellular RF sensing data. In certain implementations or road scenarios, for example, the cellular RF sensing data may include dynamic and / or static obstacle data detected by radar (in addition to or in lieu of the filtered cellular RF sensing data). In some implementations, the cellular RF sensing data received at a server generating an HD map layer may not come directly from the vehicle that collected the cellular RF sensing data, but may come via a backhaul network. For example, this may occur when the vehicle uploads the data to the nearest infrastructure radio head, which then sends the data to the server that maintains the HD map via wireless and / or wired network components. Such data on the backhaul network may be in a different format than, for example, the cellular RF sensing data received directly from the vehicle (e.g., as previously described with respect to Figure 4 described).

[0091] Means for performing the functionality at block 710 may include bus 905, processor 910, input device 915, communication subsystem 930 (wireless communication interface 933), memory 935, and / or other components of computer system 900, such as Fig. 9 exemplified in .

[0092] At block 720, the functionality includes combining the cellular RF sensing data with the aggregated cellular RF sensing data. As previously indicated, combining the cellular RF sensing data from the vehicle with the aggregated cellular RF sensing data (e.g., from one or more additional vehicles and / or previously provided by the vehicle) may include unifying the cellular RF sensing data. This may include, for example, conforming the received cellular RF sensing data to a particular coordinate system, scale, format, etc., or a combination thereof. The cellular RF sensing data may be stored relative to the rear axle frame of the ego vehicle along with an associated ego pose, or it may be stored directly in a fixed reference frame (such as ECEF), in which case the ego pose is used to map points from the rear axle frame to the ECEF.

[0093] Means for performing the functionality at block 720 may include bus 905, processor 910, input device 915, communication subsystem 930 (wireless communication interface 933), memory 935, and / or other components of computer system 900, such as Fig. 9 exemplified in .

[0094] The functionality at block 730 includes determining that an RF map update trigger condition is satisfied, which may be optional (e.g. Figure 7). As previously indicated, the aggregated data can be used to optimize different map layers (including the RF map layer) and the trajectories of the vehicles that reported the data. Therefore, according to some embodiments, if the RF map update trigger condition is met, the computer system can optimize the RF map and posture trajectory and update the RF map layer of the HD map. As previously described, the RF map update trigger condition can be periodic or event-based. Determining an event-based trigger can include, for example, determining that a certain number of reports from vehicles have arrived in a particular area, or that multiple vehicles (e.g., a threshold number of vehicles) have indicated changes in the environment as perceived by the uploaded cellular RF sensing data and / or camera map layer data (e.g., lane closures, construction signs added or removed, significant changes in surrounding parked cars, etc.).

[0095] Means for performing the functionality at block 730 may include bus 905, processor 910, input device 915, communication subsystem 930 (wireless communication interface 933), memory 935, and / or other components of computer system 900, such as Fig. 9 exemplified in .

[0096] The functionality at block 740 includes updating the RF map layer of the HD map. According to some embodiments, this may include replacing an existing corresponding tile in the HD map with the newly constructed radar map tile. According to some embodiments, if a list of candidate tiles has been listed (and optionally, if the list contains at least a threshold number of candidate tiles), the server may select a replacement tile from the list and replace the existing corresponding tile in the HD map with the replacement tile. The selection of the replacement tile may involve selecting a replacement tile as previously described with respect to the HD map. Figure 8 The selection process described in box 880.

[0097] Means for performing the functionality at block 740 may include bus 905, processor 910, input device 915, communication subsystem 930 (wireless communication interface 933), memory 935, and / or other components of computer system 900, such as Fig. 9 exemplified in .

[0098] Figure 8 is a block diagram of an embodiment of a mobile computing system 800 that may be incorporated into a vehicle and described above (e.g., with Figures 1 to 7 ) as described in connection with the present invention to enable the vehicle to perform the functions of the embodiments described herein. For example, the mobile computing system 800 may execute Figure 4 and Figure 5 It should be noted that Figure 8 It is intended only to provide a generalized illustration of various components, any or all of which may be utilized as appropriate. It may be noted that in some cases, Figure 8 The illustrated components may be localized as a single physical device and / or distributed among various networked devices that may be located at different physical locations. In addition, as previously mentioned, the functionality of the UE discussed in the previously described embodiments may be provided by Figure 8 The invention may be executed by one or more of the illustrated hardware and / or software components.

[0099] Mobile computing system 800 is shown as including hardware elements that may be electrically coupled (or may otherwise communicate as appropriate) via bus 805, which may include a vehicle-specific bus, such as a controller area network (CAN) bus. The hardware elements may include a processor 810, which may include, but is not limited to, one or more general-purpose processors (e.g., application processors), one or more special-purpose processors (such as DSP chips, graphics acceleration processors, application-specific integrated circuits (ASICs), etc.), and / or other processing structures or components. Processor 810 may include one or more processing units, which may be housed in a single integrated circuit (IC) or multiple ICs. Figure 8 As shown, some embodiments may have a separate DSP 820, depending on the desired functionality. Position determination and / or other determinations based on wireless communications (discussed below) may be provided in the processor 810 and / or the wireless communication interface 830. The mobile computing system 800 may also include one or more input devices 870, which may include but are not limited to one or more keyboards, touch screens, touch pads, microphones, buttons, dials, switches, etc.; and one or more output devices 815, which may include but are not limited to one or more displays (e.g., touch screens), light emitting diodes (LEDs), speakers, etc.

[0100] The mobile computing system 800 may also include a wireless communication interface 830, which may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device and / or a chipset (such as Bluetooth ®Devices, IEEE 802.11 devices, IEEE 802.15.4 devices, Wi-Fi devices, WiMAX devices, WAN devices and / or various cellular devices, etc.), etc., which can enable the mobile computing system 800 to communicate with other devices as described in the above embodiments. The wireless communication interface 830 can allow the transmission / reception point (TRP) of the network to be communicated (e.g., send and receive) data and signaling, for example, via a cellular base station (e.g., eNB, gNB, ng-eNB, etc.), an access point and / or other access node types, and / or other network components, a computer system, and / or any other electronic device communicatively coupled to the TRP as described herein. Communication can be performed via one or more wireless communication antennas 832 that transmit and / or receive wireless signals 834. According to some embodiments, the wireless communication antenna 832 may include multiple discrete antennas, antenna arrays, or any combination thereof. The antenna 832 may be capable of using beams (e.g., Tx beams and Rx beams) to send and receive wireless signals. Beamforming may be performed using digital and / or analog beamforming techniques with corresponding digital and / or analog circuitry. Wireless communication interface 830 may include such circuitry.

[0101] Depending on the desired functionality, the wireless communication interface 830 may include separate receivers and transmitters, or any combination of transceivers, transmitters, and / or receivers to communicate with base stations (e.g., ng-eNBs and gNBs) and other terrestrial transceivers (such as wireless devices and access points). The mobile computing system 800 may communicate with different data networks, which may include various network types. For example, a wireless wide area network (WWAN) may be a code division multiple access (CDMA) network, a time division multiple access (TDMA) network, a frequency division multiple access (FDMA) network, an orthogonal frequency division multiple access (OFDMA) network, a single carrier frequency division multiple access (SC-FDMA) network, a WiMAX (IEEE 802.16) network, etc. A CDMA network may implement one or more RATs, such as CDMA2000. ® , WCDMA, etc. CDMA2000 ® Includes IS-95, IS-2000, and / or IS-856 standards. A TDMA network may implement Global System for Mobile Communications (GSM), Digital Advanced Mobile Phone System (D-AMPS), or some other radio access technology (RAT). An OFDMA network may employ Long Term Evolution (LTE), Advanced LTE, Fifth Generation (5G) New Radio (NR), and the like. 5G NR, LTE, Advanced LTE, GSM, and WCDMA are described in documents from 3GPP. CDMA2000 ®Described in documents from an organization named "3rd Generation Partnership Project 2" (3GPP2). 3GPP and 3GPP2 documents are publicly available. A wireless local area network (WLAN) may also be an IEEE 802.11x network, while a wireless personal area network (WPAN) may be a Bluetooth network, IEEE 802.15x, or some other type of network. The techniques described herein may also be used for any combination of WWAN, WLAN, and / or WPAN.

[0102] The mobile computing system 800 may also include a sensor 840. The sensor 840 may correspond to Figure 2 Sensors 205 of the vehicle and thus may include radar 841, camera 842, and other sensors. As illustrated, sensors 840 may also include lidar 843, IMU 844, etc. Sensors 840 may also include accelerometers, gyroscopes, magnetometers, altimeters, microphones, proximity sensors, light sensors, barometers, sonars, and / or sensors from vehicle systems (e.g., wheel sensors, speedometers, etc.). As described herein, sensors may be used to obtain information about location and / or movement.

[0103] Embodiments of the mobile computing system 800 may also include a GNSS receiver 880 (eg, corresponding to Figure 2 830 ) is capable of receiving signals 884 from one or more GNSS satellites using antenna 882 (which may be the same as antenna 832 ). Positioning based on GNSS signal measurements may be used to supplement and / or incorporate the techniques described herein. GNSS receiver 880 may extract the location of mobile computing system 800 from GNSS satellites of GNSS systems such as Global Positioning System (GPS), Galileo, GLONASS, Quasi-Zenith Satellite System (QZSS) over Japan, IRNSS over India, BeiDou Navigation Satellite System (BDS) over China, etc. using conventional techniques. In addition, GNSS receiver 880 may be used with various augmentation systems (e.g., satellite-based augmentation systems (SBAS)) that may be associated with or otherwise enabled for use with one or more global and / or regional navigation satellite systems, such as, for example, Wide Area Augmentation System (WAAS), European Geostationary Navigation Overlay Service (EGNOS), Multi-Function Satellite Augmentation System (MSAS), and Geographic Augmentation Navigation System (GAGAN), etc.

[0104] It may be pointed out that although Figure 8880 is illustrated as a different component, but the embodiments are not limited thereto. As used herein, the term "GNSS receiver" may include hardware and / or software components configured to obtain GNSS measurements (measurements from GNSS satellites). Therefore, in some embodiments, the GNSS receiver may include (as software) a measurement engine executed by one or more processors, such as processor 810, DSP 820, and / or a processor within wireless communication interface 830 (e.g., in a modem). The GNSS receiver may also optionally include a positioning engine that may use the GNSS measurements from the measurement engine to determine the positioning of the GNSS receiver using an extended Kalman filter (EKF), weighted least squares (WLS), a hatch filter, a particle filter, etc. The positioning engine may also be executed by one or more processors such as processor 810 or DSP 820.

[0105] The mobile computing system 800 may also include and / or be in communication with memory 860. The memory 860 may include, but is not limited to, local and / or network accessible storage, disk drives, drive arrays, optical storage devices, solid-state storage devices such as random access memory (RAM) and / or read-only memory (ROM), which may be programmable, flash-updatable, etc. Such storage devices may be configured to implement any suitable data storage, including, but not limited to, various file systems, database structures, etc.

[0106] The memory 860 of the mobile computing system 800 may also include software elements ( Figure 8 ), these software elements include operating systems, device drivers, executable libraries, and / or other code (such as one or more applications), which may include computer programs provided by various embodiments, and / or may be designed to implement methods provided by other embodiments, and / or configure systems provided by other embodiments, as described herein. By way of example only, one or more procedures described with respect to the methods discussed above may be implemented as code and / or instructions in memory 860 that are executable by mobile computing system 800 (and / or processor 810 or DSP 820 within mobile computing system 800). Then, in some embodiments, such code and / or instructions may be used to configure and / or adapt a general-purpose computer (or other device) to perform one or more operations according to the described methods.

[0107] Fig. 9 is a block diagram of an embodiment of a computer system 900 that may be used, in whole or in part, to provide a server or other computing device as described in the embodiments herein (e.g., as described with respect to Figures 3 to 7For example, the computing system 900 may execute Figure 6 and Figure 7 One or more of the functions of the method shown in . It should be noted that Fig. 9 It is intended only to provide a generalized illustration of the various components, any or all of which may be utilized as appropriate. Fig. 9 The broad illustrations show how various system elements can be implemented in a relatively separate or relatively more integrated manner. Fig. 9 The illustrated components may be localized to a single device and / or distributed among various networked devices that may be located at different geographical locations.

[0108] The computer system 900 is shown as including hardware elements that can be electrically coupled via a bus 905 (or can communicate in other ways as appropriate). The hardware elements may include a processor 910, which may include but is not limited to one or more general-purpose processors, one or more special-purpose processors (such as digital signal processing chips, graphics acceleration processors, etc.) and / or other processing structures, which can be configured to perform one or more of the methods described herein. The computer system 900 may also include one or more input devices 915, which may include but are not limited to a mouse, a keyboard, a camera, a microphone, etc.; and one or more output devices 920, which may include but are not limited to a display device, a printer, etc.

[0109] The computer system 900 may also include (and / or communicate with) one or more non-transitory storage devices 925, which may include, but are not limited to, local and / or network accessible storage devices, and / or may include, but are not limited to, disk drives, drive arrays, optical storage devices, solid-state storage devices such as RAM and / or ROM, which may be programmable, flash-updatable, and the like. Such storage devices may be configured to implement any suitable data storage, including, but not limited to, various file systems, database structures, and the like. Such data storage may include databases and / or other data structures for storing and managing messages and / or other information to be transmitted to one or more devices via the hub, as described herein.

[0110] The computer system 900 may also include a communication subsystem 930, which may include wireless communication technologies managed and controlled by a wireless communication interface 933, as well as wired technologies (such as Ethernet, coaxial communication, universal serial bus (USB), etc.). The wireless communication interface 933 may include one or more wireless transceivers, which may transmit and receive wireless signals 955 (e.g., signals according to 5G NR or LTE) via a wireless antenna 950. Therefore, the communication subsystem 930 may include a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset, etc., which may enable the computer system 900 to communicate with any device on the corresponding network (including user equipment (UE), base station and / or other TRP, and / or any other electronic device described herein) on any or all of the communication networks described herein. Therefore, the communication subsystem 930 can be used to receive and transmit data, as described in the embodiments of this document.

[0111] In many embodiments, the computer system 900 will also include a working memory 935, which may include a RAM or ROM device, as described above. The software elements shown as being located within the working memory 935 may include an operating system 940, device drivers, executable libraries, and / or other code such as one or more applications 945, which may include computer programs provided by various embodiments, and / or may be designed to implement methods provided by other embodiments and / or configure systems provided by other embodiments, as described herein. By way of example only, one or more of the procedures described with respect to the methods discussed above may be implemented as code and / or instructions that can be executed by a computer (and / or a processor within a computer); then, in one aspect, such code and / or instructions may be used to configure and / or adapt a general purpose computer (or other device) to perform one or more operations according to the described methods.

[0112] The set of these instructions and / or codes may be stored on a non-transitory computer-readable storage medium (such as the storage device 925 described above). In some cases, the storage medium may be incorporated into a computer system such as the computer system 900. In other embodiments, the storage medium may be separate from the computer system (e.g., a removable medium such as an optical disk), and / or may be provided in an installation package so that the storage medium may be used to program, configure, and / or adapt a general-purpose computer having the instructions / codes stored therein. These instructions may take the form of an executable code that can be executed by the computer system 900, and / or may take the form of source code and / or installable code that, when compiled and / or installed on the computer system 900 (e.g., using any of a variety of commonly available compilers, installers, compression / decompression utilities, etc.), takes the form of executable code.

[0113] It will be apparent to those skilled in the art that substantial modifications may be made according to specific requirements. For example, customized hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software, such as applets, etc.), or both. In addition, connections to other computing devices such as network input / output devices may be employed.

[0114] With reference to the accompanying drawings, components that may include memory may include non-transitory machine-readable media. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any storage medium that participates in providing data that causes a machine to operate in a particular manner. In the embodiments provided above, various machine-readable media may be involved when providing instructions / codes to a processor and / or other devices for execution. Additionally or alternatively, machine-readable media may be used to store and / or carry such instructions / codes. In many specific implementations, computer-readable media are physical and / or tangible storage media. Such media may take many forms, including but not limited to non-volatile media and volatile media. Common forms of computer-readable media include, for example: magnetic and / or optical media, any other physical media with hole patterns, RAM, programmable ROM (PROM), erasable PROM (EPROM), FLASH-EPROM, any other memory chip or memory box, or any other medium from which a computer can read instructions and / or codes.

[0115] The methods, systems and devices discussed herein are examples. Various embodiments may omit, replace or add various procedures or components as appropriate. For example, the features described in certain embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments may be combined in a similar manner. The various components of the drawings provided herein may be embodied in hardware and / or software. In addition, technology may evolve, and therefore many elements are examples, which do not limit the scope of the present disclosure to those specific examples.

[0116] It has proven convenient at times, primarily for reasons of common usage, to refer to such signals as bits, information, values, elements, symbols, characters, variables, terms, numbers, digital symbols, and the like. However, it should be understood that all of these or similar terms are to be associated with the appropriate physical quantities and are merely convenient labels. Unless otherwise specifically stated, as will be apparent from the above discussion, it should be understood that throughout this specification, discussions utilizing terms such as "processing," "computing," "calculating," "determining," "ascertaining," "identifying," "correlating," "measuring," "performing," and the like refer to the actions or processes of a particular device such as a special-purpose computer or similar special-purpose electronic computing device. Thus, in the context of this specification, a special-purpose computer or similar special-purpose electronic computing device is capable of manipulating or transforming signals, typically represented as physical, electronic, electrical, or magnetic quantities in a memory, register, or other information storage device, a transmitting device, or a display device of the special-purpose computer or similar special-purpose electronic computing device.

[0117] As used herein, the terms "and" and "or" may include multiple meanings that are also intended to depend at least in part on the context in which such terms are used. Generally, "or", if used in association with a list, such as A, B, or C, is intended to represent A, B, and C (inclusive meanings are used here) as well as A, B, or C (exclusive meanings are used here). In addition, as used herein, the term "one or more" may be used to describe any feature, structure, or characteristic in the singular form, or may be used to describe a certain combination of features, structures, or characteristics. However, it should be noted that this is merely an illustrative example, and the subject matter claimed is not limited to this example. In addition, the term "at least one of...", if used in association with a list, such as A, B, or C, may be interpreted to represent any combination of A, B, and / or C, such as A, AB, AA, AAB, AABBCCC, etc.

[0118] Several embodiments have been described, and various modifications, alternative configurations, and equivalents may be used without departing from the scope of the present disclosure. For example, the above elements may be components of a larger system, wherein other rules may take precedence over the application of the various embodiments or otherwise modify the application of the various embodiments. In addition, multiple steps may be performed before, during, or after considering the above elements. Accordingly, the above description does not limit the scope of the present disclosure.

[0119] In view of this description, various embodiments may include different combinations of features. Specific implementation examples are described in the following numbered clauses:

[0120] Clause 1. An example method for obtaining data for automotive navigation at a vehicle based on cellular radio frequency (RF) sensing, the method comprising obtaining a camera image of an environment surrounding the vehicle using a camera at the vehicle. The method also includes determining a fault condition of the camera that prevents object detection by the camera image, wherein the fault condition is caused by inclement weather, an obstruction on a lens of the camera, an electrical fault of the camera, or a combination thereof. The method also includes configuring a cellular communication interface of the vehicle to obtain cellular RF sensing data of the environment surrounding the vehicle in response to determining the fault condition of the camera. The method also includes performing the automotive navigation support based at least in part on the cellular RF sensing data of the environment surrounding the vehicle.

[0121] Clause 2. The method according to clause 1, further comprising: obtaining an RF sensing map corresponding to the environment around the vehicle from a server; and increasing an alarm level in response to determining that a match between the cellular RF sensing data and the RF sensing map is below a predetermined threshold, wherein increasing the alarm level comprises at least one of reducing the driving speed of the vehicle or returning to a manual driving mode.

[0122] Clause 3. The method of clauses 1 and 2, wherein the RF sensing map comprises a crowd-sourced RF sensing map modified based on specifications of the vehicle.

[0123] Clause 4. The method of clauses 1 to 3, further comprising filtering the cellular RF sensing data on a frame-by-frame basis to remove cellular RF sensing data generated by moving objects.

[0124] Clause 5. The method of clauses 1 to 4, further comprising: determining a level of match between the cellular RF sensing data and the RF sensing map; and sending data indicative of the match to the server for updating the RF sensing map.

[0125] Clause 6. The method of clauses 1 to 5, wherein the RF sensing map is obtained in response to the determination of the fault condition of the camera.

[0126] Clause 7. The method of clauses 1 to 6, wherein the RF sensing map corresponding to the environment around the vehicle is obtained at a predetermined point in time.

[0127] Clause 8. A method according to clauses 1 to 7, wherein the fault condition of the camera is determined based on: determining that no object is detected in the camera image; determining that no object is detected in the camera image beyond a predetermined distance; or a combination thereof.

[0128] Clause 9. A method according to clauses 1 to 8, wherein determining the fault condition of the camera further comprises: obtaining a camera map corresponding to the environment around the vehicle from a server, and wherein the fault condition of the camera is determined based on: determining that a matching level between the camera image and the camera map is lower than a predetermined threshold.

[0129] Clause 10. A device comprising: a transceiver; a memory; and one or more processing units, the one or more processing units being communicatively coupled to the transceiver and the memory, the one or more processing units being configured to perform a method according to any one of clauses 1 to 10.

[0130] Clause 11. An example method for updating a cellular radio frequency (RF) sensing map based on cellular RF sensing, the method comprising obtaining a first set of cellular RF sensing data of an environment around a first vehicle from a first vehicle. The method further comprises determining a unified cellular RF sensing map based on unifying the first set of cellular RF sensing data with a second set of cellular RF sensing data, wherein the second set of cellular RF sensing data is obtained from a second vehicle. The method further comprises updating the RF sensing map based on the unified cellular RF sensing map in response to determining that a match between the unified cellular RF sensing map and an RF sensing map corresponding to the environment around the first vehicle is below a predetermined threshold.

[0131] Clause 12. The method of clause 11, wherein the first set of cellular RF sensing data is determined in response to determining a fault condition of a camera of the first vehicle.

[0132] Clause 13. A method according to any one of clauses 11 and 12, wherein the fault condition of the camera is determined based on: determining that no object is detected in a camera image obtained by the camera; determining that no object is detected in the camera image beyond a predetermined distance; or a combination thereof.

[0133] Clause 14. The method of any one of clauses 11 to 13, wherein the first set of cellular RF sensing data is filtered on a frame-by-frame basis to remove cellular RF sensing data generated by moving objects.

[0134] Clause 15. A method according to any one of clauses 11 to 14, the method further comprising: receiving a request for an updated RF sensing map corresponding to the environment around the third vehicle from a third vehicle; adjusting the updated RF sensing map based on the specifications of the third vehicle; and sending the adjusted RF sensing map to the third vehicle.

[0135] Clause 16. The method of any one of clauses 11 to 15, wherein the request is sent in response to a determination of a fault condition of a camera of the third vehicle.

[0136] Clause 17. A method according to any one of clauses 11 to 16, wherein the fault condition of the camera is determined based on: determining that no object is detected in a camera image obtained by the camera; determining that no object is detected in the camera image beyond a predetermined distance; or a combination thereof.

[0137] Clause 18. A device comprising: a transceiver; a memory; and one or more processing units, the one or more processing units being communicatively coupled to the transceiver and the memory, the one or more processing units being configured to perform a method according to any one of clauses 11 to 17.

[0138] Clause 19. An example method of using an RF sensing map to assist cellular RF sensing for automobile navigation, the method comprising obtaining the RF sensing map corresponding to a trip from a server prior to the trip. The method also includes obtaining cellular RF sensing data of an environment surrounding the vehicle through a cellular communication interface of the vehicle, wherein obtaining the cellular RF sensing data is responsive to a determination of a fault condition of a camera of the vehicle that prevents object detection in a camera image obtained by the camera, wherein the condition is caused by inclement weather, an obstruction on a lens of the camera, an electrical fault of the camera, or a combination thereof. The method also includes raising an alert level in response to determining that a match between the cellular RF sensing data and the RF sensing map is below a predetermined threshold, wherein raising the alert level includes at least one of reducing a driving speed of the vehicle or reverting to a manual driving mode.

[0139] Clause 20. The method of clause 19, wherein the RF sensing map comprises a crowd-sourced RF sensing map modified based on specifications of the vehicle.

[0140] Clause 21. The method of any one of clauses 19 and 20, further comprising filtering the cellular RF sensing data on a frame-by-frame basis to remove cellular RF sensing data generated by mobile objects.

[0141] Clause 22. A method according to any one of clauses 19 to 21, the method further comprising: determining a level of match between the cellular RF sensing data and the RF sensing map; and sending data indicative of the match to the server for updating the RF sensing map.

[0142] Clause 23. A method according to any one of clauses 19 to 22, wherein the fault condition of the camera is determined based on: determining that no object is detected in the camera image; determining that no object is detected in the camera image beyond a predetermined distance; or a combination thereof.

[0143] Clause 24. A method according to any one of clauses 19 to 23, wherein determining the fault condition of the camera includes: obtaining a camera map corresponding to the environment around the vehicle from a server, and wherein the fault condition of the camera is determined based on the following situation: determining that the matching level between the camera image and the camera map is lower than a predetermined threshold.

[0144] Clause 25. A device comprising: a transceiver; a memory; and one or more processing units, the one or more processing units being communicatively coupled to the transceiver and the memory, the one or more processing units being configured to perform a method according to any one of clauses 19 to 24.

[0145] Clause 26. An example method for automobile navigation using a cellular RF sensing map, the method comprising: receiving a request from a vehicle for a cellular RF sensing map corresponding to an environment surrounding the vehicle, wherein the cellular RF sensing map is generated based on data obtained from a cellular RF communication interface. The method further comprises determining the cellular RF sensing map based on specifications of the vehicle. The method further comprises transmitting the cellular RF sensing map to the vehicle.

[0146] Clause 27. The method of clause 26, wherein the request is sent in response to a determination of a fault condition of a camera of the vehicle.

[0147] Clause 28. A method according to any one of clauses 26 and 27, wherein the fault condition of the camera is determined based on: determining that no object is detected in a camera image obtained by the camera; determining that no object is detected in the camera image beyond a predetermined distance; or a combination thereof.

[0148] Clause 29. The method of any one of clauses 26 to 28, wherein the cellular RF sensing map is determined based on unifying a first set of cellular RF sensing data received from a first vehicle with a second set of cellular RF sensing data from a second vehicle.

[0149] Clause 30. The method of any one of clauses 26 to 29, wherein the first set of cellular RF sensing data is determined in response to determining a fault condition of a camera of the first vehicle.

[0150] Clause 31. The method of clauses 26 to 30, wherein the first set of cellular RF sensing data is filtered on a frame-by-frame basis to remove cellular RF sensing data generated by moving objects.

[0151] Clause 32. A device comprising: a transceiver; a memory; and one or more processing units, the one or more processing units being communicatively coupled to the transceiver and the memory, the one or more processing units being configured to perform a method according to any one of clauses 26 to 31.

Claims

1. A method for obtaining data for automobile navigation at a vehicle based on cellular radio frequency (RF) sensing, the method comprising: obtaining a camera image of an environment around the vehicle using a camera at the vehicle; determining a fault condition of the camera that prevents object detection of the camera image, wherein the fault condition is caused by inclement weather, an obstruction on a lens of the camera, an electrical fault of the camera, or a combination thereof; responsive to determining the fault condition of the camera, configuring a cellular communication interface of the vehicle to obtain cellular RF sensing data of the environment surrounding the vehicle; as well as The automotive navigation support is performed based at least in part on cellular RF sensing data of the environment around the vehicle.

2. The method according to claim 1, further comprising: obtaining, from a server, an RF sensing map corresponding to the environment around the vehicle; as well as An alert level is raised in response to determining that a match between the cellular RF sensing data and the RF sensing map is below a predetermined threshold, wherein raising the alert level includes at least one of reducing a driving speed of the vehicle or reverting to a manual driving mode. 3 . The method of claim 2 , wherein the RF sensing map comprises a crowd-sourced RF sensing map modified based on specifications of the vehicle.

4. The method according to claim 3, further comprising: The cellular RF sensing data is filtered on a frame-by-frame basis to remove cellular RF sensing data generated by moving objects.

5. The method according to claim 2, further comprising: determining a level of match between the cellular RF sensing data and the RF sensing map; as well as Data indicative of the match is sent to the server for use in updating the RF sensing map. The method of claim 1 , wherein the RF sensing map is obtained in response to the determination of the fault condition of the camera. 7 . The method of claim 1 , wherein the RF sensing map corresponding to the environment around the vehicle is obtained at a predetermined point in time.

8. The method of claim 1 , wherein the fault condition of the camera is determined based on: determining that no object is detected in the camera image; determining that no object beyond a predetermined distance is detected in the camera image; or A combination of them.

9. The method of claim 1 , wherein determining the fault condition of the camera further comprises: A camera map corresponding to the environment around the vehicle is obtained from a server, and wherein the fault condition of the camera is determined based on: It is determined that a level of matching between the camera image and the camera map is below a predetermined threshold.

10. A method for updating a cellular radio frequency (RF) sensing map based on RF sensing, the method comprising: obtaining, from a first vehicle, a first set of cellular RF sensing data of an environment surrounding the first vehicle; determining a unified cellular RF sensing map based on unifying the first set of cellular RF sensing data with a second set of cellular RF sensing data, wherein the second set of cellular RF sensing data is obtained from a second vehicle; as well as In response to determining that a match between the unified cellular RF sensing map and an RF sensing map corresponding to the environment around the first vehicle is below a predetermined threshold, updating the RF sensing map based on the unified cellular RF sensing map. 11 . The method of claim 10 , wherein the first set of cellular RF sensing data is determined in response to determining a fault condition of a camera of the first vehicle.

12. The method of claim 11, wherein the fault condition of the camera is determined based on: determining that no object is detected in a camera image obtained by the camera; determining that no object beyond a predetermined distance is detected in the camera image; or A combination of them.

13. The method of claim 10, wherein the first set of cellular RF sensing data is filtered on a frame-by-frame basis to remove cellular RF sensing data generated by moving objects.

14. The method according to claim 10, further comprising: receiving, from a third vehicle, a request for an updated RF sensing map corresponding to an environment surrounding the third vehicle; adjusting the updated RF sensing map based on specifications of the third vehicle; as well as The adjusted RF sensing map is transmitted to the third vehicle.

15. The method of claim 14, wherein the request is sent in response to a determination of a fault condition of a camera of the third vehicle.

16. The method of claim 15, wherein the fault condition of the camera is determined based on: determining that no object is detected in a camera image obtained by the camera; determining that no object beyond a predetermined distance is detected in the camera image; or A combination of them.

17. A method of using RF sensing maps to assist cellular RF sensing for automotive navigation, the method comprising: Before the trip, the RF sensing map corresponding to the trip is obtained from a server; obtaining, via a cellular communication interface of the vehicle, cellular RF sensing data of an environment surrounding the vehicle, wherein obtaining the cellular RF sensing data is responsive to determining a fault condition of a camera of the vehicle that prevents object detection in a camera image obtained by the camera, wherein the condition is caused by inclement weather, an obstruction on a lens of the camera, an electrical fault in the camera, or a combination thereof; as well as An alert level is raised in response to determining that a match between the cellular RF sensing data and the RF sensing map is below a predetermined threshold, wherein raising the alert level includes at least one of reducing a driving speed of the vehicle or reverting to a manual driving mode.

18. The method of claim 17, wherein the RF sensing map comprises a crowd-sourced RF sensing map modified based on specifications of the vehicle.

19. The method according to claim 18, further comprising: The cellular RF sensing data is filtered on a frame-by-frame basis to remove cellular RF sensing data generated by moving objects.

20. The method according to claim 18, further comprising: determining a level of match between the cellular RF sensing data and the RF sensing map; as well as Data indicative of the match is sent to the server for use in updating the RF sensing map.

21. The method of claim 17, wherein the fault condition of the camera is determined based on: determining that no object is detected in the camera image; determining that no object beyond a predetermined distance is detected in the camera image; or A combination of them.

22. The method of claim 17, wherein determining the fault condition of the camera comprises: A camera map corresponding to the environment around the vehicle is obtained from a server, and wherein the fault condition of the camera is determined based on: It is determined that a level of matching between the camera image and the camera map is below a predetermined threshold.

23. A method for automobile navigation using cellular RF sensing maps, the method comprising: receiving, from a vehicle, a request for a cellular RF sensing map corresponding to an environment surrounding the vehicle, wherein the cellular RF sensing map is generated based on data obtained from a cellular RF communication interface; determining the cellular RF sensing map based on specifications of the vehicle; as well as The cellular RF sensing map is transmitted to the vehicle.

24. The method of claim 23, wherein the request is sent in response to a determination of a fault condition of a camera of the vehicle.

25. The method of claim 24, wherein the fault condition of the camera is determined based on: determining that no object is detected in a camera image obtained by the camera; determining that no object beyond a predetermined distance is detected in the camera image; or A combination of them.

26. The method of claim 23, wherein the cellular RF sensing map is determined based on unifying a first set of cellular RF sensing data received from a first vehicle with a second set of cellular RF sensing data from a second vehicle.

27. The method of claim 26, wherein the first set of cellular RF sensing data is determined in response to determining a fault condition of a camera of the first vehicle.

28. The method of claim 27, wherein the first set of cellular RF sensing data is filtered on a frame-by-frame basis to remove cellular RF sensing data generated by moving objects.

Images