Anti-spoofing in camera assisted positioning and perception
By integrating the camera-assisted positioning component in the user equipment, and using multiple methods to verify image authenticity, the problem of spoofed feature interference in camera-assisted positioning is solved, and more reliable and safe positioning results are achieved.
Patent Information
- Application Number
- CN202480011344.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-02-14
- Filing Date
- 2024-01-11
- Publication Date
- 2025-09-05
AI Technical Summary
In existing wireless communication systems, camera-assisted positioning technology is susceptible to interference from spoofing features, resulting in inaccurate positioning and reduced security.
By integrating camera-assisted positioning components in the user equipment (UE), detecting spoofed features in the image, using methods such as high-precision map comparison, changing camera sampling rate, cross-checking images and geometric inspection using additional sensors, the authenticity of the image is verified, and positioning after determining that the image is not spoofed.
Improves the safety and reliability of camera-assisted positioning to ensure the accuracy and credibility of positioning results.
Smart Images

Figure CN120604263A_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims the benefit of U.S. non-provisional patent application serial number 18 / 168,977, entitled “ANTI-SPOOFING IN CAMERA-AIDED LOCATION AND PERCEPTION,” filed on February 14, 2023, which is expressly incorporated herein by reference in its entirety. Technical Field
[0003] The present disclosure relates generally to communication systems and, more particularly, to wireless communications with respect to positioning. Background Art
[0004] Wireless communication systems are widely deployed to provide a variety of telecommunication services, such as telephony, video, data, messaging, and broadcasts. Typical wireless communication systems employ multiple-access technologies capable of supporting communication with multiple users by sharing available system resources. Examples of such multiple-access technologies include code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal frequency division multiple access (OFDMA), single-carrier frequency division multiple access (SC-FDMA), and time division synchronous code division multiple access (TD-SCDMA).
[0005] These multiple access technologies have been adopted in various telecommunication standards to provide a common protocol that enables different wireless devices to communicate at the city, national, regional, and even global levels. An example telecommunication standard is 5G New Radio (NR). 5G NR is part of the continued evolution of mobile broadband, promulgated by the 3rd Generation Partnership Project (3GPP) to meet new requirements associated with latency, reliability, security, scalability (e.g., for the Internet of Things (IoT)), and other requirements. 5G NR includes services associated with enhanced mobile broadband (eMBB), massive machine-type communications (mMTC), and ultra-reliable low-latency communications (URLLC). Certain aspects of 5G NR may be based on the 4G Long Term Evolution (LTE) standard. Further improvements to 5G NR technology are needed. In addition, these improvements may also be applicable to other multiple access technologies and telecommunication standards that adopt these technologies. Summary of the Invention
[0006] The following presents a simplified summary of one or more aspects in order to provide a basic understanding of these aspects. This summary is not an extensive overview of all contemplated aspects. This summary does not identify key or critical elements of all aspects, nor does it delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that will be presented later.
[0007] In one aspect of the present disclosure, a method, computer-readable medium, and apparatus are provided. The apparatus obtains a set of images associated with a vision-assisted localization session, wherein the set of images is captured using at least one first camera. The apparatus detects the presence of at least one spoofing feature in the set of images during the vision-assisted localization session. The apparatus stores or outputs an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images.
[0008] To achieve the foregoing and related ends, one or more aspects may include the features fully described below and particularly pointed out in the claims. The following description and the accompanying drawings set forth in detail some illustrative features of one or more aspects. However, these features are indicative of only some of the various ways in which the principles of the various aspects may be employed. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Figure 1 is a diagram illustrating an example of a wireless communication system and an access network.
[0010] Figure 2A is a diagram illustrating an example of a first frame according to various aspects of the present disclosure.
[0011] Figure 2B is a diagram illustrating an example of downlink (DL) channels within a subframe according to various aspects of the present disclosure.
[0012] Figure 2C is a diagram illustrating an example of a second frame according to various aspects of the present disclosure.
[0013] Figure 2D is a diagram illustrating an example of uplink (UL) channels within a subframe according to various aspects of the present disclosure.
[0014] Figure 3 is a diagram illustrating an example of a base station and a user equipment (UE) in an access network.
[0015] Figure 4 is a diagram illustrating an example of UE positioning based on reference signal measurement.
[0016] Figure 5 is a diagram illustrating an example of camera-assisted positioning according to various aspects of the present disclosure.
[0017] Figure 6 is a diagram illustrating example unintentional deception according to various aspects of the present disclosure.
[0018] Figure 7 is a diagram illustrating example intentional deception according to various aspects of the present disclosure.
[0019] Figure 8 is a diagram illustrating example intentional deception according to various aspects of the present disclosure.
[0020] Figure 9A is a diagram illustrating example images from a map database according to various aspects of the present disclosure.
[0021] Figure 9B is a diagram illustrating an example spoofing image according to various aspects of the present disclosure.
[0022] Figure 10A is a diagram illustrating example images captured by a camera from a real environment according to various aspects of the present disclosure.
[0023] Figure 10B is a diagram illustrating example images captured from a video displayed at a display rate that is lower than the sampling rate of a camera, according to various aspects of the present disclosure.
[0024] Figure 11 is a flow chart of a method of wireless communication.
[0025] Figure 12 is a flow chart of a method of wireless communication.
[0026] Figure 13 are diagrams illustrating examples of hardware implementations for example apparatuses and / or network entities. DETAILED DESCRIPTION
[0027] Various aspects presented herein provide various anti-spoofing mechanisms / solutions that can improve the security and reliability of camera-based positioning and / or camera-assisted positioning. The various aspects presented herein can enable a UE to identify whether an image captured by the UE's camera is spoofed (e.g., whether it is a real image or a fake / manipulated / virtual image, etc.). In one aspect of the present disclosure, the UE can verify whether an image captured by its camera is spoofed by comparing the captured image to a map (e.g., a high-definition (HD) map). If the captured image has components that differ from those in the map, the captured image is likely spoofed. In another aspect of the present disclosure, the UE can verify whether an image captured by its camera is spoofed by changing the camera's sampling rate. If the captured image exhibits certain patterns or observations (e.g., blank gaps) that may occur when the camera operates at a higher sampling rate that is faster than the display rate of the video, the UE can determine that the captured image is spoofed. In another aspect of the present disclosure, the UE can verify whether an image captured by its camera is spoofed by cross-checking the captured image with images captured by other cameras and / or by changing the focal length of its camera. In another aspect of the present disclosure, the UE may verify whether the image captured by its camera is spoofed based on performing a geometric check on one or more objects in the captured image using additional sensors. If the UE determines that the image it captured is spoofed, the UE may perform camera-based positioning and / or camera-assisted positioning without the spoofed image / feature. However, if the image is not spoofed, the UE may continue to perform camera-based positioning and / or camera-assisted positioning using its camera.
[0028] The detailed description set forth below in conjunction with the accompanying drawings is a description of various configurations and does not represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific details to provide a thorough understanding of the various concepts. However, these concepts may be practiced without these specific details. In some cases, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.
[0029] Several aspects of telecommunications systems are presented with reference to various apparatus and methods. These apparatus and methods are described in the following detailed description and illustrated in the accompanying drawings by various blocks, components, circuits, processes, algorithms, etc. (collectively referred to as "elements"). These elements can be implemented using electronic hardware, computer software, or any combination thereof. Whether these elements are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system.
[0030] As an example, an element, or any part of an element, or any combination of elements can be implemented as a "processing system" that includes one or more processors. Examples of processors include microprocessors, microcontrollers, graphics processing units (GPUs), central processing units (CPUs), application processors, digital signal processors (DSPs), reduced instruction set computing (RISC) processors, systems on a chip (SoCs), baseband processors, field programmable gate arrays (FPGAs), programmable logic devices (PLDs), state machines, gating logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionalities described throughout this disclosure. One or more processors in a processing system can execute software. Whether referred to as software, firmware, middleware, microcode, hardware description language, or other terms, software should be broadly interpreted to mean instructions, instruction sets, codes, code segments, program codes, programs, subroutines, software components, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, processes, functions, or any combination thereof.
[0031] Thus, in one or more example aspects, implementations, and / or use cases, the functionality described may be implemented in hardware, software, or any combination thereof. If implemented in software, the functionality may be stored or encoded as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media. A storage medium can be any available medium that can be accessed by a computer. By way of example, such computer-readable media may include random access memory (RAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), optical disk storage, magnetic disk storage, other magnetic storage devices, combinations of these types of computer-readable media, or any other medium that can be used to store computer-executable code in the form of instructions or data structures that can be accessed by a computer.
[0032] While various aspects, implementations, and / or use cases are described herein through the lens of a few examples, additional or different aspects, implementations, and / or use cases may arise in many different arrangements and scenarios. The various aspects, implementations, and / or use cases described herein can be implemented across many different platform types, devices, systems, shapes, sizes, and packaging arrangements. For example, the various aspects, implementations, and / or use cases may arise via integrated chip implementations and other non-module component-based devices (e.g., end-user devices, vehicles, communications devices, computing devices, industrial equipment, retail / purchase equipment, medical devices, artificial intelligence (AI)-enabled devices, etc.). While some examples may or may not be specifically targeted at use cases or applications, the examples described may have broad applicability. The various aspects, implementations, and / or use cases may range from chip-level or modular components to non-modular, non-chip-level implementations, and further to aggregated, distributed, or original equipment manufacturer (OEM) devices or systems incorporating one or more of the techniques described herein. In some practical settings, devices incorporating the various described aspects and features may also include additional components and features for implementing and practicing the claimed and described aspects. For example, the transmission and reception of wireless signals necessarily involve multiple components for both analog and digital purposes (e.g., hardware components including antennas, RF chains, power amplifiers, modulators, buffers, processors, interleavers, adders / summers, etc.). The techniques described herein can be practiced in a wide variety of devices of various sizes, shapes, and configurations, including chip-level components, systems, distributed arrangements, aggregated or disaggregated components, end-user devices, and the like.
[0033] The deployment of a communication system, such as a 5G NR system, can be arranged in a variety of ways using various components or elements. In a 5G NR system or network, a network node, a network entity, a mobility element of a network, a radio access network (RAN) node, a core network node, a network element, or network equipment (such as a base station (BS)), or one or more units (or one or more components) performing base station functions can be implemented in a converged or disaggregated architecture. For example, a base station (such as a node B (NB), an evolved NB (eNB), a NR base station, a 5G NB, an access point (AP), a transmit / receive point (TRP), or a cell) can be implemented as a converged base station (also known as a standalone base station or a monolithic base station) or a disaggregated base station.
[0034] A converged base station may be configured to utilize a radio protocol stack that is physically or logically integrated within a single RAN node. A disaggregated base station may be configured to utilize a protocol stack that is physically or logically distributed across two or more units, such as one or more central or centralized units (CUs), one or more distributed units (DUs), or one or more radio units (RUs). In some aspects, a CU may be implemented within a RAN node, and one or more DUs may be co-located with the CU, or alternatively, may be geographically or virtually distributed across one or more other RAN nodes. A DU may be implemented to communicate with one or more RUs. Each of the CU, DU, and RU may be implemented as a virtual unit, namely a virtual central unit (VCU), a virtual distributed unit (VDU), or a virtual radio unit (VRU).
[0035] Base station operation or network design may take into account the aggregated nature of base station functionality. For example, a disaggregated base station may be utilized in an integrated access backhaul (IAB) network, an open radio access network (O-RAN (such as the network configuration initiated by the O-RAN Alliance)), or a virtualized radio access network (vRAN, also known as a cloud radio access network (C-RAN)). Disaggregation may include distributing functionality across two or more units at various physical locations, as well as virtually distributing the functionality of at least one unit, which enables flexibility in network design. The various units of a disaggregated base station or disaggregated RAN architecture may be configured for wired or wireless communication with at least one other unit.
[0036] Figure 1 Figure 100 illustrates an example of a wireless communication system and access network. The illustrated wireless communication system includes a disaggregated base station architecture. The disaggregated base station architecture may include one or more CUs 110, which may communicate directly with a core network 120 via a backhaul link or indirectly with the core network 120 through one or more disaggregated base station elements, such as a near real-time (near-RT) RAN intelligent controller (RIC) 125 via an E2 link, a non-real-time (non-RT) RIC 115 associated with a service management and orchestration (SMO) framework 105, or both. CUs 110 may communicate with one or more DUs 130 via corresponding midhaul links, such as an F1 interface. DUs 130 may communicate with one or more RUs 140 via corresponding fronthaul links. RUs 140 may communicate with corresponding UEs 104 via one or more radio frequency (RF) access links. In some implementations, a UE 104 may be served simultaneously by multiple RUs 140.
[0037] Each of the units (i.e., CU 110, DU 130, RU 140, as well as near-RT RIC 125, non-RT RIC 115, and SMO framework 105) may include or be coupled to one or more interfaces configured to receive or transmit signals, data, or information (collectively, signals) via a wired or wireless transmission medium. Each of these units, or an associated processor or controller providing instructions to the communication interface of these units, may be configured to communicate with one or more of the other units via the transmission medium. For example, these units may include a wired interface configured to receive signals or transmit signals to one or more of the other units via a wired transmission medium. Additionally, these units may include a wireless interface, which may include a receiver, transmitter, or transceiver (such as an RF transceiver) configured to receive and / or transmit signals to one or more of the other units via a wireless transmission medium.
[0038] In some aspects, the CU 110 may host one or more higher-layer control functions. Such control functions may include radio resource control (RRC), packet data convergence protocol (PDCP), service data adaptation protocol (SDAP), etc. Each control function may be implemented using an interface configured to communicate signals with other control functions hosted by the CU 110. The CU 110 may be configured to handle user plane functionality (i.e., central unit-user plane (CU-UP)), control plane functionality (i.e., central unit-control plane (CU-CP)), or a combination thereof. In some implementations, the CU 110 may be logically split into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP units may communicate bidirectionally with the CU-CP units via an interface (such as an E1 interface). As needed, the CU 110 may be implemented to communicate with the DU 130 for network control and signaling.
[0039] The DU 130 may correspond to a logical unit that includes one or more base station functions for controlling the operation of one or more RUs 140. In some aspects, the DU 130 may host one or more of a radio link control (RLC) layer, a medium access control (MAC) layer, and one or more higher physical (PHY) layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation and demodulation, etc.) based at least in part on a functional split (such as those defined by 3GPP). In some aspects, the DU 130 may also host one or more lower PHY layers. Each layer (or module) may be implemented using an interface configured for signal communication with other layers (and modules) hosted by the DU 130 or with control functions hosted by the CU 110.
[0040] Lower layer functionality may be implemented by one or more RUs 140. In some deployments, a RU 140 controlled by a DU 130 may correspond to a logical node that hosts RF processing functionality or low PHY layer functionality (such as performing Fast Fourier Transforms (FFTs), Inverse FFTs (iFFTs), digital beamforming, Physical Random Access Channel (PRACH) extraction and filtering, etc.), or both, based at least in part on a functional split (such as a lower layer functional split). In such an architecture, the RU 140 may be implemented to handle over-the-air (OTA) communications with one or more UEs 104. In some implementations, both real-time and non-real-time aspects of control and user plane communications with the RU 140 may be controlled by the corresponding DU 130. In some scenarios, this configuration may enable the implementation of the DU 130 and CU 110 in a cloud-based RAN architecture, such as a vRAN architecture.
[0041] The SMO framework 105 can be configured to support RAN deployment and provisioning of both non-virtualized and virtualized network elements. For non-virtualized network elements, the SMO framework 105 can be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which can be managed via an operations and maintenance interface (such as the O1 interface). For virtualized network elements, the SMO framework 105 can be configured to interact with a cloud computing platform (such as Open Cloud (O-Cloud) 190) to perform network element lifecycle management (such as instantiating virtualized network elements) via cloud computing platform interfaces (such as the O2 interface). Such virtualized network elements may include, but are not limited to, the CU 110, DU 130, RU 140, and near-RT RIC 125. In some implementations, the SMO framework 105 can communicate with hardware aspects of the 4G RAN (such as the Open eNB (O-eNB) 111) via the O1 interface. Additionally, in some implementations, the SMO framework 105 can communicate directly with one or more RUs 140 via the O1 interface. The SMO framework 105 may also include a non-RT RIC 115 configured to support the functionality of the SMO framework 105 .
[0042] The non-RT RIC 115 can be configured to include logic that enables non-real-time control and optimization of RAN elements and resources, artificial intelligence (AI) / machine learning (ML) (AI / ML) workflows including model training and updating, or policy-based guidance of applications / features in the near-RT RIC 125. The non-RT RIC 115 can be coupled to or in communication with the near-RT RIC 125 (e.g., via an A1 interface). The near-RT RIC 125 can be configured to include logic that enables near-real-time control and optimization of RAN elements and resources through data collection and actions via an interface (e.g., via an E2 interface) that connects one or more CUs 110, one or more DUs 130, or both, and the O-eNB with the near-RT RIC 125.
[0043] In some implementations, to generate AI / ML models to be deployed in the near-RT RIC 125, the non-RT RIC 115 may receive parameters or external enrichment information from an external server. Such information may be utilized by the near-RT RIC 125 and may be received from non-network data sources or from network functions at the SMO framework 105 or the non-RT RIC 115. In some examples, the non-RT RIC 115 or the near-RT RIC 125 may be configured to tune RAN behavior or performance. For example, the non-RT RIC 115 may monitor long-term trends and patterns in performance and employ AI / ML models to execute corrective actions through the SMO framework 105 (such as via reconfiguration of O1) or through the creation of RAN management policies (such as A1 policies).
[0044] At least one of the CU 110, DU 130, and RU 140 may be referred to as a base station 102. Thus, base station 102 may include one or more of CU 110, DU 130, and RU 140 (each component is indicated by a dashed line to indicate that each component may or may not be included in base station 102). Base station 102 provides a UE 104 with access to core network 120. Base station 102 may include a macro cell (a high-power cellular base station) and / or a small cell (a low-power cellular base station). Small cells include femto cells, pico cells, and micro cells. A network that includes both small cells and macro cells may be referred to as a heterogeneous network. Heterogeneous networks may also include a home evolved Node B (eNB) (HeNB), which may provide services to a restricted group known as a closed subscriber group (CSG). The communication link between RU 140 and UE 104 may include uplink (UL) (also known as reverse link) transmissions from UE 104 to RU 140 and / or downlink (DL) (also known as forward link) transmissions from RU 140 to UE 104. The communication link may utilize multiple-input, multiple-output (MIMO) antenna technology, including spatial multiplexing, beamforming, and / or transmit diversity. The communication link may be over one or more carriers. Base station 102 / UE 104 may utilize spectrum with a bandwidth of up to Y MHz (e.g., 5 MHz, 10 MHz, 15 MHz, 20 MHz, 100 MHz, 400 MHz, etc.) for each carrier allocated in a carrier aggregation for transmission in each direction, totaling up to Yx MHz (x component carriers). These carriers may or may not be adjacent to each other. Carrier allocation may be asymmetric for DL and UL (e.g., more or fewer carriers may be allocated for DL compared to UL). Component carriers may include a primary component carrier and one or more secondary component carriers. The primary component carrier may be referred to as a primary cell (PCell) and the secondary component carrier may be referred to as a secondary cell (SCell).
[0045] Certain UEs 104 may communicate with each other using device-to-device (D2D) communication links 158. D2D communication links 158 may utilize DL / UL wireless wide area network (WWAN) spectrum. D2D communication links 158 may utilize one or more sidelink channels, such as a physical sidelink broadcast channel (PSBCH), a physical sidelink discovery channel (PSDCH), a physical sidelink shared channel (PSSCH), and a physical sidelink control channel (PSCCH). D2D communication may be accomplished via various wireless D2D communication systems, such as, for example, Bluetooth, Wi-Fi based on the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard, LTE, or NR.
[0046] The wireless communication system may also include a Wi-Fi AP 150 that communicates with a UE 104 (also referred to as a Wi-Fi station (STA)) via a communication link 154, for example, in the 5 GHz unlicensed spectrum. When communicating in the unlicensed spectrum, the UE 104 / AP 150 may perform a clear channel assessment (CCA) to determine whether the channel is available before communicating.
[0047] The electromagnetic spectrum is typically subdivided into various categories, bands, channels, and so on, based on frequency / wavelength. In 5G NR, two initial operating bands have been identified as frequency ranges designated FR1 (410 MHz-7.125 GHz) and FR2 (24.25 GHz-52.6 GHz). Although a portion of FR1 extends beyond 6 GHz, FR1 is often (interchangeably) referred to as the "sub-6 GHz" band in various documents and articles. A similar naming issue sometimes arises with FR2, which is often (interchangeably) referred to as the "millimeter wave" band in documents and articles, despite being distinct from the extremely high frequency (EHF) band (30 GHz-300 GHz), which is designated as a "millimeter wave" band by the International Telecommunication Union (ITU).
[0048] Frequencies between FR1 and FR2 are often referred to as mid-band frequencies. Recent 5G NR research has identified the operating band for these mid-band frequencies as frequency range designation FR3 (7.125 GHz-24.25 GHz). Frequency bands falling within FR3 can inherit FR1 characteristics and / or FR2 characteristics, effectively extending the features of FR1 and / or FR2 to mid-band frequencies. Furthermore, higher frequency bands are currently being explored to extend 5G NR operation beyond 52.6 GHz. For example, three higher operating bands have been identified as frequency range designations FR2-2 (52.6 GHz-71 GHz), FR4 (71 GHz-114.25 GHz), and FR5 (114.25 GHz-300 GHz). Each of these higher frequency bands falls within the EHF band.
[0049] With the above in mind, unless otherwise specified, if the term "sub-6 GHz" or the like is used herein, it may broadly refer to frequencies that may be less than 6 GHz, may be within FR1, or may include mid-band frequencies. Furthermore, unless otherwise specified, if the term "millimeter wave" or the like is used herein, it may broadly refer to frequencies that may include mid-band frequencies, may be within FR2, FR4, FR2-2, and / or FR5, or may be within the EHF band.
[0050] Base station 102 and UE 104 may each include multiple antennas (such as antenna elements, antenna panels, and / or antenna arrays) to facilitate beamforming. Base station 102 may transmit beamformed signals 182 to UE 104 in one or more transmit directions. UE 104 may receive beamformed signals from base station 102 in one or more receive directions. UE 104 may also transmit beamformed signals 184 to base station 102 in one or more transmit directions. Base station 102 may receive beamformed signals from UE 104 in one or more receive directions. Base station 102 / UE 104 may perform beam training to determine the optimal receive and transmit directions for each of base station 102 / UE 104. The transmit and receive directions of base station 102 may or may not be the same. The transmit and receive directions of UE 104 may or may not be the same.
[0051] The base station 102 may include and / or be referred to as a gNB, Node B, eNB, access point, base transceiver station, radio base station, radio transceiver, transceiver function, basic service set (BSS), extended service set (ESS), TRP, network node, network entity, network equipment, or some other suitable terminology. The base station 102 may be implemented as an integrated access and backhaul (IAB) node, a relay node, a sidelink node, a converged (monolithic) base station having a baseband unit (BBU) (including a CU and a DU) and a RU, or as a disaggregated base station including one or more of a CU, a DU, and / or a RU. A collection of base stations that may include disaggregated base stations and / or converged base stations may be referred to as a next generation (NG) RAN (NG-RAN).
[0052] The core network 120 may include an access and mobility management function (AMF) 161, a session management function (SMF) 162, a user plane function (UPF) 163, a unified data management (UDM) 164, one or more location servers 168, and other functional entities. The AMF 161 is a control node that handles signaling between the UE 104 and the core network 120. The AMF 161 supports registration management, connection management, mobility management, and other functions. The SMF 162 supports session management and other functions. The UPF 163 supports packet routing, packet forwarding, and other functions. The UDM 164 supports the generation of authentication and key agreement (AKA) credentials, user identity handling, access authorization, and subscription management. The one or more location servers 168 are exemplified as including a gateway mobile location center (GMLC) 165 and a location management function (LMF) 166. However, in general, the one or more location servers 168 may include one or more location / positioning servers, which may include one or more of the GMLC 165, LMF 166, Position Determination Entity (PDE), Serving Mobile Location Center (SMLC), Mobile Positioning Center (MPC), etc. The GMLC 165 and LMF 166 support UE location services. The GMLC 165 provides an interface for clients / applications (e.g., emergency services) to access UE positioning information. The LMF 166 receives measurements and assistance information from the NG-RAN and UE 104 via the AMF 161 to calculate the location of the UE 104. The NG-RAN may utilize one or more positioning methods to determine the location of the UE 104. Locating the UE 104 may involve signal measurements, position estimation, and optional velocity calculation based on these measurements. Signal measurements may be performed by the UE 104 and / or the base station 102 serving the UE 104. The measured signals may be based on one or more of a satellite positioning system (SPS) 170 (e.g., one or more of a global navigation satellite system (GNSS), a global positioning system (GPS), a non-terrestrial network (NTN), or other satellite positioning / location systems), LTE signals, wireless local area network (WLAN) signals, Bluetooth signals, a terrestrial beacon system (TBS), sensor-based information (e.g., an atmospheric pressure sensor, a motion sensor), NR enhanced cell ID (NR E-CID) methods, NR signals (e.g., multi-round trip time (multi-RTT), DL angle of departure (DL-AoD), DL time difference of arrival (DL-TDOA), UL time difference of arrival (UL-TDOA), and UL angle of arrival (UL-AoA) positioning), and / or other systems / signals / sensors.
[0053] Examples of UE 104 include a cellular phone, a smartphone, a Session Initiation Protocol (SIP) phone, a laptop, a personal digital assistant (PDA), a satellite radio, a global positioning system, a multimedia device, a video device, a digital audio player (e.g., an MP3 player), a camera, a game console, a tablet, a smart device, a wearable device, a vehicle, an electric meter, a gas pump, a large or small kitchen appliance, a healthcare device, an implant, a sensor / actuator, a display, or any other similarly functional device. Some of UE 104 may be referred to as IoT devices (e.g., a parking meter, a gas pump, a toaster, a vehicle, a heart rate monitor, etc.). UE 104 may also be referred to as a station, a mobile station, a subscriber station, a mobile unit, a subscriber unit, a wireless unit, a remote unit, a mobile device, a wireless device, a wireless communication device, a remote device, a mobile subscriber station, an access terminal, a mobile terminal, a wireless terminal, a remote terminal, a handset, a user agent, a mobile client, a client, or some other suitable terminology. In some scenarios, the term UE may also apply to one or more companion devices, such as in a device constellation arrangement. One or more of these devices may access the network collectively and / or individually.
[0054] Reference again Figure 1 In certain aspects, the UE 104 may include a camera-assisted positioning component 198 that may be configured to obtain a set of images associated with a visually assisted positioning session, wherein the set of images is captured using at least one first camera; detect the presence of at least one spoofing feature in the set of images during the visually assisted positioning session; and store or output an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images. In certain aspects, the base station 102 may have a positioning configuration component 199 that may be configured to configure camera-assisted / camera-based positioning parameters for the UE.
[0055] Figure 2A FIG200 is a diagram illustrating an example of a first subframe within a 5G NR frame structure. Figure 2B FIG230 is a diagram illustrating an example of DL channels within a 5G NR subframe. Figure 2C FIG250 is a diagram illustrating an example of a second subframe within a 5G NR frame structure. Figure 2D FIG280 is a diagram illustrating an example of UL channels within a 5G NR subframe. The 5G NR frame structure may be frequency division duplex (FDD), where for a particular set of subcarriers (carrier system bandwidth), subframes within that subcarrier set are dedicated to either DL or UL, or time division duplex (TDD), where for a particular set of subcarriers (carrier system bandwidth), subframes within that subcarrier set are dedicated to both DL and UL. Figure 2A 、 Figure 2CIn the example provided, the 5G NR frame structure is assumed to be TDD, with subframe 4 configured with slot format 28 (mostly DL), where D stands for DL, U stands for UL, and F stands for flexible use between DL / UL, and subframe 3 configured with slot format 1 (all UL). While subframes 3 and 4 are shown with slot formats 1 and 28, respectively, any particular subframe can be configured with any of the various available slot formats 0-61. Slot formats 0 and 1 are all DL and all UL, respectively. The other slot formats 2-61 include a mix of DL, UL, and flexible symbols. The UE is configured with the slot format via a received slot format indicator (SFI), either dynamically via DL control information (DCI) or semi-statically / statically via radio resource control (RRC) signaling. Note that the following description also applies to the 5G NR frame structure as TDD.
[0056] Figures 2A to 2D This example illustrates a frame structure, and various aspects of this disclosure are applicable to other wireless communication technologies that may have different frame structures and / or different channels. A frame (10 ms) can be divided into 10 equally sized subframes (1 ms). Each subframe may include one or more slots. A subframe may also include mini-slots, which may include 7, 4, or 2 symbols. Each slot may include 14 or 12 symbols, depending on whether the cyclic prefix (CP) is normal or extended. For a normal CP, each slot may include 14 symbols, and for an extended CP, each slot may include 12 symbols. Downlink symbols may be CP-orthogonal frequency division multiplexing (OFDM) symbols. Uplink symbols may be CP-OFDM symbols (for high-throughput scenarios) or discrete Fourier transform (DFT)-spread OFDM (DFT-s-OFDM) symbols (for power-limited scenarios; limited to single-stream transmission). The number of slots within a subframe depends on the CP and the parameter set. The parameter set defines the subcarrier spacing (SCS) (see Table 1). Symbol length / duration can be scaled with 1 / SCS.
[0057]
[0058] Table 1: Parameter set, SCS and CP
[0059] For normal CP (14 symbols / slot), different parameter sets µ 0 to 4 allow 1, 2, 4, 8, and 16 slots per subframe, respectively. For extended CP, parameter set 2 allows 4 slots per subframe. Therefore, for normal CP and parameter set µ, there are 14 symbols / slot and 2 µ time slots / subframe. The subcarrier spacing can be equal to ,in For parameter sets 0 to 4. Therefore, the subcarrier spacing for parameter set µ=0 is 15 kHz, and the subcarrier spacing for parameter set µ=4 is 240 kHz. The symbol length / duration is inversely related to the subcarrier spacing. Figures 2A to 2D An example is provided for a normal CP with 14 symbols per slot and a parameter set µ=2 with 4 slots per subframe. The slot duration is 0.25ms, the subcarrier spacing is 60kHz, and the symbol duration is approximately 16.67µs. Within a frame set, there may be one or more different bandwidth parts (BWPs) frequency-division multiplexed (see Figure 2B ). Each BWP may have a specific parameter set and CP (normal or extended).
[0060] A resource grid can be used to represent the frame structure. Each slot consists of a resource block (RB) (also known as a physical RB (PRB)) that extends over 12 consecutive subcarriers. The resource grid is divided into multiple resource elements (REs). The number of bits carried by each RE depends on the modulation scheme.
[0061] like Figure 2A As illustrated, some of the REs carry reference (pilot) signals (RS) for the UE. The RSs may include a demodulation RS (DM-RS) (indicated as R for one specific configuration, but other DM-RS configurations are possible) and a channel state information reference signal (CSI-RS) used for channel estimation at the UE. The RSs may also include a beamforming RS (BRS), a beam refinement RS (BRRS), and a phase tracking RS (PT-RS).
[0062] Figure 2BExamples of various downlink channels within a subframe of a frame are illustrated. The physical downlink control channel (PDCCH) carries DCI within one or more control channel elements (CCEs) (e.g., 1, 2, 4, 8, or 16 CCEs). Each CCE consists of six resource element groups (REGs), with each REG comprising 12 contiguous REs within an OFDM symbol of a RB. The PDCCH within a BWP is referred to as a control resource set (CORESET). During PDCCH monitoring opportunities within a CORESET, a UE is configured to monitor PDCCH search spaces (e.g., common search space, UE-specific search space) for PDCCH candidates with different DCI formats and aggregation levels. Additional BWPs may be located at higher and / or lower frequencies across the channel bandwidth. The primary synchronization signal (PSS) may be within symbol 2 of specific subframes of a frame. The PSS is used by UE 104 to determine subframe / symbol timing and physical layer identification. The secondary synchronization signal (SSS) may be within symbol 4 of specific subframes of a frame. The SSS is used by the UE to determine the physical layer cell identity group number and radio frame timing. Based on the physical layer identity and the physical layer cell identity group number, the UE can determine the physical cell identifier (PCI). Based on the PCI, the UE can determine the location of the DM-RS. The physical broadcast channel (PBCH), which carries the master information block (MIB), can be logically grouped with the PSS and SSS to form a synchronization signal (SS) / PBCH block (also known as an SS block (SSB)). The MIB provides the system frame number (SFN) and the number of RBs in the system bandwidth. The physical downlink shared channel (PDSCH) carries user data, broadcast system information not sent via the PBCH (such as the system information block (SIB)), and paging messages.
[0063] like Figure 2C As illustrated, some of the REs carry DM-RSs (indicated as R for a specific configuration, but other DM-RS configurations are possible) for channel estimation at the base station. The UE can transmit DM-RSs for the physical uplink control channel (PUCCH) and DM-RSs for the physical uplink shared channel (PUSCH). The PUSCH DM-RS can be transmitted in the first or first two symbols of the PUSCH. The PUCCH DM-RS can be transmitted in different configurations depending on whether a short or long PUCCH is transmitted and the specific PUCCH format used. The UE can transmit a sounding reference signal (SRS). The SRS can be transmitted in the last symbol of the subframe. The SRS can have a comb structure, and the UE can transmit the SRS on one of the comb structures within the comb structure. The SRS can be used by the base station for channel quality estimation to achieve frequency-dependent scheduling of the UL.
[0064] Figure 2DExamples of various UL channels within a subframe of a frame are illustrated. The PUCCH may be located as indicated in one configuration. The PUCCH carries uplink control information (UCI), such as scheduling requests, channel quality indicators (CQI), precoding matrix indicators (PMI), rank indicators (RI), and hybrid automatic repeat request (HARQ) acknowledgement (ACK) (HARQ-ACK) feedback (i.e., one or more HARQ ACK bits indicating one or more ACKs and / or negative ACKs (NACKs)). The PUSCH carries data and may additionally be used to carry buffer status reports (BSRs), power headroom reports (PHRs), and / or UCI.
[0065] Figure 3 Figure 3 is a block diagram of a base station 310 communicating with a UE 350 in an access network. In the DL, Internet Protocol (IP) packets may be provided to a controller / processor 375. The controller / processor 375 implements Layer 3 and Layer 2 functionality. Layer 3 includes the Radio Resource Control (RRC) layer, and Layer 2 includes the Service Data Adaptation Protocol (SDAP) layer, the Packet Data Convergence Protocol (PDCP) layer, the Radio Link Control (RLC) layer, and the Medium Access Control (MAC) layer. The controller / processor 375 provides RRC layer functionality associated with broadcasting of system information (e.g., MIB, SIB), RRC connection control (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), inter-radio access technology (RAT) mobility, and measurement configuration for UE measurement reporting; PDCP layer functionality associated with header compression / decompression, security (ciphering, deciphering, integrity protection, integrity verification), and handover support functions; RLC layer functionality associated with delivery of upper layer packet data units (PDUs), error correction through ARQ, concatenation, segmentation and reassembly of RLC service data units (SDUs), resegmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, multiplexing of MAC SDUs onto transport blocks (TBs), demultiplexing of MAC SDUs from TBs, scheduling information reporting, error correction through HARQ, priority handling, and logical channel prioritization.
[0066] The transmit (TX) processor 316 and receive (RX) processor 370 implement Layer 1 functionality associated with various signal processing functions. Layer 1, which includes the physical (PHY) layer, may include error detection on transport channels, forward error correction (FEC) coding / decoding of transport channels, interleaving, rate matching, mapping onto physical channels, modulation / demodulation of physical channels, and MIMO antenna processing. The TX processor 316 handles the mapping onto signal constellations based on various modulation schemes (e.g., binary phase-shift keying (BPSK), quadrature phase-shift keying (QPSK), M-order phase-shift keying (M-PSK), and M-order quadrature amplitude modulation (M-QAM)). The coded and modulated symbols are then separated into parallel streams. Each stream is then mapped to an OFDM subcarrier, multiplexed with a reference signal (e.g., a pilot) in the time and / or frequency domain, and then combined using an inverse fast Fourier transform (IFFT) to produce a physical channel carrying a time-domain OFDM symbol stream. The OFDM stream is spatially precoded to produce multiple spatial streams. Channel estimates from a channel estimator 374 may be used to determine the coding and modulation schemes, as well as for spatial processing. The channel estimates may be derived from a reference signal and / or channel condition feedback transmitted by the UE 350. Each spatial stream may then be provided to a different antenna 320 via a separate transmitter 318Tx. Each transmitter 318Tx may modulate a radio frequency (RF) carrier using a corresponding spatial stream for transmission.
[0067] At the UE 350, each receiver 354Rx receives a signal via its corresponding antenna 352. Each receiver 354Rx recovers the information modulated onto the RF carrier and provides the information to a receive (RX) processor 356. The TX processor 368 and the RX processor 356 implement Layer 1 functionality associated with various signal processing functions. The RX processor 356 performs spatial processing on the information to recover any spatial streams destined for the UE 350. If multiple spatial streams are destined for the UE 350, they may be combined into a single OFDM symbol stream by the RX processor 356. The RX processor 356 then converts the OFDM symbol stream from the time domain to the frequency domain using a fast Fourier transform (FFT). The frequency-domain signal includes a separate OFDM symbol stream for each subcarrier of the OFDM signal. The symbols and reference signals on each subcarrier are recovered and demodulated by determining the most likely signal constellation point transmitted by the base station 310. These soft decisions may be based on channel estimates calculated by the channel estimator 358. The soft decisions are then decoded and deinterleaved to recover the data and control signals originally sent on the physical channel by base station 310. The data and control signals are then provided to a controller / processor 359, which implements layer 3 and layer 2 functionality.
[0068] The controller / processor 359 may be associated with a memory 360 that stores program codes and data. The memory 360 may be referred to as a computer-readable medium. In the UL, the controller / processor 359 provides demultiplexing between transport and logical channels, packet reassembly, decryption, header decompression, and control signal processing to recover IP packets. The controller / processor 359 is also responsible for error detection using an ACK and / or NACK protocol to support HARQ operations.
[0069] Similar to the functionality described in conjunction with DL transmissions by the base station 310, the controller / processor 359 provides RRC layer functionality associated with system information (e.g., MIB, SIB) acquisition, RRC connection, and measurement reporting; PDCP layer functionality associated with header compression / decompression and security (ciphering, deciphering, integrity protection, integrity verification); RLC layer functionality associated with delivery of upper layer PDUs, error correction through ARQ, concatenation, segmentation, and reassembly of RLC SDUs, resegmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, multiplexing of MAC SDUs onto TBs, demultiplexing of MAC SDUs from TBs, scheduling information reporting, error correction through HARQ, priority handling, and logical channel prioritization.
[0070] Channel estimates derived by the channel estimator 358 based on a reference signal or feedback transmitted by the base station 310 may be used by the TX processor 368 to select the appropriate coding and modulation schemes and to facilitate spatial processing. The spatial streams generated by the TX processor 368 may be provided to different antennas 352 via separate transmitters 354Tx. Each transmitter 354Tx may modulate an RF carrier with a corresponding spatial stream for transmission.
[0071] UL transmissions are processed at the base station 310 in a manner similar to that described in conjunction with the receiver functionality at the UE 350. Each receiver 318Rx receives a signal through its corresponding antenna 320. Each receiver 318Rx recovers information modulated onto an RF carrier and provides the information to the RX processor 370.
[0072] The controller / processor 375 may be associated with a memory 376 that stores program codes and data. The memory 376 may be referred to as a computer-readable medium. In the UL, the controller / processor 375 provides demultiplexing between transport and logical channels, packet reassembly, decryption, header decompression, and control signal processing to recover IP packets. The controller / processor 375 is also responsible for error detection using an ACK and / or NACK protocol to support HARQ operations.
[0073] At least one of the TX processor 368, the RX processor 356, and the controller / processor 359 may be configured to combine Figure 1 The camera assisted positioning component 198 is used to perform various aspects.
[0074] At least one of the TX processor 316, the RX processor 370, and the controller / processor 375 may be configured to perform operations related to Figure 1 The positioning configuration component 199 combines various aspects.
[0075] Figure 4 is a diagram 400 illustrating an example of UE positioning based on reference signal measurements (which may also be referred to as "network-based positioning") according to various aspects of the present disclosure. UE 404 may be at time T SRS_TX UL-SRS 412 is sent and at time T PRS_RX Receive DL Positioning Reference Signal (PRS) (DL-PRS) 410. TRP 406 may be at time T SRS_RX Receive UL-SRS 412 and at time T PRS_TX 410. The UE 404 may receive the DL-PRS 410 before transmitting the UL-SRS 412, or may transmit the UL-SRS 412 before receiving the DL-PRS 410. In both cases, the positioning server (eg, location server 168) or the UE 404 may determine the UL-SRS 412 based on the || T SRS_RX – T PRS_TX | – |T SRS_TX – T PRS_RX || to determine RTT 414. Thus, multi-RTT positioning may utilize UE Rx-Tx time difference measurements (ie, |T SRS_TX – T PRS_RX |) and DL-PRS reference signal received power (RSRP) (DL-PRS-RSRP), and the measured TRP Rx-Tx time difference measurement (ie, |T SRS_RX – T PRS_TX|) and UL-SRS-RSRP. UE 404 uses assistance data received from the positioning server to measure the UE Rx-Tx time difference measurement (and / or the DL-PRS-RSRP of the received signal), and TRP 402, 406 uses assistance data received from the positioning server to measure the gNB Rx-Tx time difference measurement (and / or the UL-SRS-RSRP of the received signal). These measurements can be used at the positioning server or UE 404 to determine the RTT, which is used to estimate the position of UE 404. Other methods for determining RTT are possible, such as, for example, using DL-TDOA and / or UL-TDOA measurements.
[0076] The PRS can be defined for network-based positioning (e.g., NR positioning) to enable UEs to detect and measure more neighboring transmit and receive points (TRPs), with multiple configurations supported to enable various deployments (e.g., indoor, outdoor, sub-6, mmW, etc.). To support PRS beam operation, beam scanning can also be configured for the PRS. The UL positioning reference signal can be based on the sounding reference signal (SRS) with enhancements / adjustments for positioning purposes. In some examples, the UL-PRS can be referred to as "SRS for positioning," and a new information element (IE) can be configured for the SRS to facilitate positioning in RRC signaling.
[0077] DL PRS-RSRP can be defined as the linear average of the power contributions (in watts) of the resource elements of the antenna ports carrying the DL PRS reference signal configured for RSRP measurement, within the considered measurement frequency bandwidth. In some examples, for FR1, the reference point for DL PRS-RSRP can be the UE's antenna connector. For FR2, DL PRS-RSRP can be measured based on the combined signal from the antenna elements corresponding to a given receiver branch. For both FR1 and FR2, if the UE uses receiver diversity, the reported DL PRS-RSRP value can be no lower than the corresponding DL PRS-RSRP of any of the individual receiver branches. Similarly, UL SRS-RSRP can be defined as the linear average of the power contributions (in watts) of the resource elements carrying the sounding reference signal (SRS). UL SRS-RSRP can be measured over the configured resource elements, within the considered measurement frequency bandwidth, and during the configured measurement occasions. In some examples, for FR1, the reference point for UL SRS-RSRP can be the antenna connector of the base station (e.g., gNB). For FR2, the UL SRS-RSRP may be measured based on the combined signals from the antenna elements corresponding to a given receiver branch. For FR1 and FR2, if the base station uses receiver diversity, the reported UL SRS-RSRP value may not be lower than the corresponding UL SRS-RSRP of any of the individual receiver branches.
[0078] PRS-Path RSRP (PRS-RSRPP) can be defined as the power of the linear average of the channel response at the i-th path delay of the resource element carrying the DL PRS signal configured for measurement, where the DL PRS-RSRPP for the first path delay is the power contribution corresponding to the first detected path in time. In some examples, the PRS path phase measurement can refer to the phase associated with the i-th path of the channel derived using the PRS resource.
[0079] DL-AoD positioning may utilize the measured DL-PRS-RSRP of downlink signals received at a UE 404 from multiple TRPs 402, 406. The UE 404 uses assistance data received from a positioning server to measure the DL-PRS-RSRP of the received signals, and the resulting measurements, along with the azimuth angle of departure (A-AoD), the zenith angle of departure (Z-AoD), and other configuration information, are used to position the UE 404 relative to neighboring TRPs 402, 406.
[0080] DL-TDOA positioning may utilize DL Reference Signal Time Difference (RSTD) (and / or DL-PRS-RSRP) of downlink signals received at a UE 404 from multiple TRPs 402, 406. The UE 404 uses assistance data received from a positioning server to measure the DL RSTD (and / or DL-PRS-RSRP) of the received signals, and the resulting measurements, along with other configuration information, are used to position the UE 404 relative to neighboring TRPs 402, 406.
[0081] UL-TDOA positioning may utilize the UL relative time of arrival (RTOA) (and / or UL-SRS-RSRP) of uplink signals transmitted from a UE 404 at multiple TRPs 402, 406. The TRPs 402, 406 use assistance data received from a positioning server to measure the UL-RTOA (and / or UL-SRS-RSRP) of the received signals, and the resulting measurements, along with other configuration information, are used to estimate the position of the UE 404.
[0082] UL-AoA positioning may utilize the measured azimuth angle of arrival (A-AoA) and zenith angle of arrival (Z-AoA) of uplink signals sent from a UE 404 at multiple TRPs 402, 406. The TRPs 402, 406 measure the A-AoA and Z-AoA of the received signals using assistance data received from a positioning server, and the resulting measurements, along with other configuration information, are used to estimate the position of the UE 404. For purposes of this disclosure, positioning operations in which a UE provides measurements to a base station / positioning entity / server for use in calculating the UE's position may be described as "UE-assisted," "UE-assisted positioning," and / or "UE-assisted position calculation," while positioning operations in which a UE measures and calculates its own position may be described as "UE-based," "UE-based positioning," and / or "UE-based position calculation."
[0083] Additional positioning methods may be used to estimate the position of the UE 404, such as, for example, UE-side UL-AoD and / or DL-AoA. Note that data / measurements from various techniques may be combined in various ways to increase accuracy, determine and / or enhance certainty, supplement / refine measurements, and / or replace / provide missing information.
[0084] It should be noted that the terms "positioning reference signal" and "PRS" generally refer to specific reference signals used for positioning in NR and LTE systems. However, as used herein, the terms "positioning reference signal" and "PRS" may also refer to any type of reference signal that can be used for positioning, such as, but not limited to, PRS, TRS, PTRS, CRS, CSI-RS, DMRS, PSS, SSS, SSB, SRS, UL-PRS, and so on, as defined in LTE and NR. Furthermore, the terms "positioning reference signal" and "PRS" may refer to either downlink or uplink positioning reference signals, unless the context indicates otherwise. To further distinguish between the types of PRS, downlink positioning reference signals may be referred to as "DL PRS," and uplink positioning reference signals (e.g., SRS, PTRS used for positioning) may be referred to as "UL-PRS." Furthermore, for signals that can be transmitted in both the uplink and downlink (e.g., DMRS, PTRS), these signals may be prepended with "UL" or "DL" to distinguish their direction. For example, "UL-DMRS" may be distinguished from "DL-DMRS."
[0085] In addition to network-based UE positioning techniques, wireless devices (e.g., base stations / TRPs, UEs, etc.) may also be configured to include radar capabilities, which may be referred to as "radio frequency (RF) sensing," "cellular-based RF sensing," and / or simply "sensing." For example, a wireless device may transmit a radar reference signal (RRS) and measure the RRS reflected from one or more objects. Based at least in part on the measurements, the wireless device may determine or estimate the distance between the wireless device and the one or more objects. In another example, a first wireless device may also receive RRS transmitted from one or more wireless devices, where the first wireless device may determine or estimate the distance between the first wireless device and the one or more wireless devices based at least in part on the received RRS. Therefore, in some examples, RF sensing techniques may be used for UE positioning and / or for assisting UE positioning. For the purposes of this disclosure, a device capable of performing RF sensing (e.g., transmitting and / or receiving RRS for detecting objects) may be referred to as an "RF sensing node." For example, an RF sensing node may be a UE, a base station, a TRP, a device capable of transmitting RRS, and / or a device configured to perform radar functionality, etc.
[0086] In addition to Global Navigation Satellite System (GNSS)-based positioning and network-based positioning, various camera-based positioning methods have been developed to provide alternative / additional positioning mechanisms / modes. Camera-based positioning (which may also be referred to as "camera-based visual positioning," "visual positioning," and / or "vision-based positioning") is a positioning mechanism / mode that uses images captured by at least one camera to determine the position of a target (e.g., a UE or vehicle equipped with at least one camera, an object within the field of view of at least one camera, etc.). For example, images captured by a vehicle's dashboard camera can be used to calculate the vehicle's three-dimensional (3D) position and / or 3D orientation as it moves. In some implementations, camera-based positioning can provide centimeter-level and six-degree-of-freedom (6D) positioning. 6D represents how an object moves through 3D space (e.g., 3D position + 3D pose) via linear translation or axial rotation. For example, a single degree of freedom on an object can be controlled by up / down, forward / backward, left / right, pitch, roll, or yaw. Camera-based positioning holds great potential for a variety of applications, particularly in environments with degraded satellite signals.
[0087] In some scenarios, images captured by a camera can also be used to improve the accuracy / reliability of other positioning mechanisms / modes (e.g., GNSS-based positioning, network-based positioning, etc.), which may be referred to as "vision-assisted positioning," "camera-assisted positioning," "camera-assisted position," and / or "camera-assisted perception," among others. For example, while GNSS and / or inertial measurement units (IMUs) can provide good positioning / localization performance, overall positioning performance may degrade due to IMU bias drift when GNSS measurement interruptions occur. Therefore, images captured by a camera can provide valuable information to mitigate errors. For the purposes of this disclosure, a positioning session associated with camera-based positioning or camera-assisted positioning (e.g., a period of time during which one or more entities are configured to determine the position of a UE) may be referred to as a camera-based positioning session or a camera-assisted positioning session. In some examples, camera-based positioning and / or camera-assisted positioning may be associated with the UE's absolute position, the UE's relative position, the UE's orientation, or a combination thereof.
[0088] Figure 5FIG500 is a diagram illustrating an example of camera-assisted positioning according to various aspects of the present disclosure. A vehicle 502 may be equipped with a GNSS system and a set of cameras, which may include a front-facing camera 504 (for capturing a front view of the vehicle 502), a side-facing camera 506 (for capturing a side view of the vehicle 502), and / or a rear-facing camera 508 (for capturing a front view of the vehicle 502). In some examples, the GNSS system may also include or be associated with at least one IMU (e.g., a GNSS+IMU system).
[0089] The GNSS system can estimate the position of vehicle 502 based on receiving GNSS signals transmitted from multiple satellites (e.g., based on performing GNSS-based positioning). However, when GNSS signals are unavailable or weak, such as when vehicle 502 is in an urban area or a tunnel, the estimated position of vehicle 502 may become inaccurate. Therefore, in some implementations, the set of cameras on vehicle 502 can be used to assist with positioning, such as to verify whether the position estimated by the GNSS system based on the GNSS signals is accurate. For example, as shown at 510, an image captured by the front-facing camera 504 of vehicle 502 may include / identify a particular building 512 (which may also be referred to as a feature) with a known location. Vehicle 502 (or the GNSS system or a positioning engine associated with vehicle 502) can determine / verify whether the position estimated by the GNSS system (e.g., longitude and latitude coordinates) is close to the known location of the particular building 512. Thus, with the assistance of the cameras, the accuracy and reliability of GNSS-based positioning can be further improved. For the purposes of this disclosure, a GNSS system associated with a camera (e.g., capable of performing camera-assisted / camera-based positioning) may be referred to as a "GNSS+camera system," or if the GNSS system is also associated with / includes at least one IMU, as a "GNSS+IMU+camera system."
[0090] Although Figure 5 Vehicle 502 is used as an example, but it is for illustration purposes only. The various aspects presented herein may also be applied to other types of transportation (e.g., motorcycles, bicycles, buses, trains, etc.), devices (e.g., UEs on pedestrians), and / or positioning mechanisms / modes (e.g., in conjunction with Figure 4 Furthermore, for the purposes of this disclosure, a positioning mechanism / mode (e.g., GNSS-based positioning, network-based positioning, etc.) that uses at least one sensor (e.g., IMU, camera) to assist in positioning may be referred to as sensor fusion positioning.
[0091] Camera-based / camera-assisted positioning (which may also be referred to as computer vision and / or feature-based positioning in some examples) can use static features (e.g., specific buildings 512) to provide useful measurements to assist in 6D camera pose estimation. Cameras may be affected (susceptible) by, among other things, ambient lighting conditions, image noise, and / or feature availability. For example, when environmental conditions are favorable for the camera, the GNSS / IMU positioning engine may benefit from the visual information provided by the camera. However, when the visual features provided by the camera are of low quality or the information provided by the camera is misleading, the overall positioning performance may be degraded. Furthermore, while an increasing number of sensor fusion positioning technologies involve cameras or vision assistance, the lack of integrity protection for cameras or images captured by cameras may limit their use in safety-critical applications (i.e., safety-critical applications).
[0092] Image spoofing (or simply spoofing), which may also be referred to as a spoofing attack or spoofing signature, is a form of cyberattack in which a malicious / unauthorized device / person provides false or fabricated video / images to another device, typically to disrupt the other device or achieve an illicit purpose. For example, a spoofing attack / signature may include an attempt by a recognition system to successfully identify an imposter as someone else, such as by using a fake photo or video to replace the original owner's identity. Anti-spoofing (or anti-image spoofing) may refer to a set of countermeasures designed to mitigate or prevent spoofing attacks. A non-spoofed image / video or signature may refer to an image / video or signature that is authentic and does not include a false or fabricated video / image. There are several different types of image spoofing and anti-spoofing associated with camera-assisted positioning, which can be categorized as "unintentional spoofing" and "intentional spoofing."
[0093] Figure 6FIG600 is a diagram illustrating an example of unintentional deception according to various aspects of the present disclosure. In one example, unintentional deception may involve a camera (or sensor) extracting misleading / incorrect features from images / video captured by the camera. For example, as shown at 602, a display (e.g., a large screen) on one side of the road may display a video including a virtual object (such as a car 604), and as shown at 606, a billboard on the other side of the road may also display an image with virtual objects (such as multiple cyclists). In some scenarios, these virtual objects may be recognized and tracked as real objects by a vehicle's camera 608, potentially introducing misleading features to the vehicle's positioning / safety systems. For example, camera 608 may determine that car 604 in the display is a real car on the road and / or that the cyclists on the billboard are real cyclists on the road, and camera 608 may feed this incorrect information to the vehicle's positioning / safety systems. This may cause the positioning / safety systems to perform inaccurate and unsafe actions, such as slowing or stopping the vehicle.
[0094] Figure 7 and Figure 8 Figures 700 and 800 illustrate examples of intentional deception according to various aspects of the present disclosure, respectively. In one example, intentional deception may refer to a deceiver intentionally manipulating and broadcasting virtual images in a video. When a deceiver attempts to intentionally deceive the image source of a visually assisted positioning system, the camera of the visually assisted positioning system may not be able to see the actual detection environment. Instead, the virtual image or manipulated image may occupy the field of view (FOV) of the camera. In one example, intentional deception may occur when a deceiver mounts a projector directly in front of a camera to implement virtual images and / or videos with manipulated reality, where the video may be real-time video or pre-recorded video. In another example, intentional deception may occur when a large screen on the back of a truck is broadcasting distorted or modified images (e.g., adding non-existent landmarks, removing existing landmarks, etc.), so that the vehicle behind the truck (or its visually assisted positioning system) may capture incorrect information based on the FOV of the front camera.
[0095] For example, Figure 7As shown in diagram 700 of FIG. 7 , truck 702 may include a safety feature that can capture a front view of truck 702 (e.g., via a front-facing camera) and display the captured front view on the rear of truck 702 via one or more monitors, such as shown at 704 . Displaying the front view of truck 702 can provide drivers behind truck 702 (such as the driver of vehicle 706) with a view of what is occurring in front of truck 702, which can significantly improve road safety. For example, the driver of vehicle 706 may not be able to see traffic ahead of truck 702 because their view may be blocked by truck 702. Consequently, the driver of vehicle 706 may not be able to determine whether it is safe to overtake truck 702. However, by enabling truck 702 to display its front view on the rear of truck 702, the driver of vehicle 706 can make more informed decisions when overtaking truck 702. For example, as shown at 704, if the driver of vehicle 706 sees another vehicle 708 in the oncoming lane, the driver may know that it is unsafe to overtake truck 702. In another example, this safety feature can also reduce the risk of accidents caused by sudden braking or animals crossing the road.
[0096] However, if Figure 8 As shown in diagram 800 of FIGURE 8, a spoofer may intentionally spoof (e.g., hijack) the image source (e.g., the front-facing camera) of truck 702, which may cause truck 702 to display a manipulated / incorrect video (e.g., pre-recorded video, live video from a different road, etc.) on the monitor of truck 702. For example, as shown at 802, if the image source of truck 702 is spoofed, the monitor at the rear of truck 702 may display a video that does not include vehicle 708, rather than displaying the current front view of truck 702 including vehicle 708 approaching in the opposite lane. Consequently, as shown at 804, if the driver of vehicle 706 attempts to overtake truck 702 based on the manipulated video (e.g., that there are no vehicles ahead of truck 702), vehicle 706 may collide with vehicle 708 during the overtaking, potentially resulting in a serious accident.
[0097] Various aspects presented herein provide various anti-spoofing mechanisms / solutions that can improve the security and reliability of camera-based positioning and / or camera-assisted positioning. Various aspects presented herein can enable a UE to identify whether an image captured by the UE's camera is spoofed (e.g., whether it is a real image or a fake / manipulated / virtual image, etc.).
[0098] In one aspect of the present disclosure, a UE can verify whether an image captured by its camera is spoofed by comparing the captured image to an image from a map database (e.g., a map database providing high-definition (HD) maps). If the captured image of a given area has one or more components / features that differ from images of the same given area from the map database, the UE can determine that the captured image is likely spoofed. Furthermore, the UE can store or output an indication of the spoofed image to other entities (e.g., a positioning engine, a location server, etc.) so that the other entities are aware of the spoofed image. Furthermore, after the UE determines that its captured image is spoofed, the UE can perform camera-based positioning and / or camera-assisted positioning without the spoofed image / features. The various aspects presented herein can be applied to (and beneficial for) vision-enhanced, vision-based, and vision-assisted positioning, augmented reality (AR), virtual reality (VR), and extended reality (XR) glasses, automotive camera perception, autonomous driving, and vision-enhanced positioning and navigation in smartphones.
[0099] Figure 9A FIG900A illustrates an example image from a map database (e.g., an HD map database) according to various aspects of the present disclosure. In one example, an image from a map database may include invariant components (which may also be referred to as invariant features) and varying components. Invariant components / features may refer to objects (e.g., in the real world) that are likely to remain unchanged (e.g., remain constant) over a relatively long period of time. For example, as shown at 902, building 904, fire hydrant 906, and / or road sign 908 may be considered invariant components because they are more likely to remain unchanged over a relatively long period of time. On the other hand, tree 910, crosswalk 912, and / or the color of building 904, fire hydrant 906, and road sign 908 may be considered varying components because they are likely to change over a shorter period of time than invariant components. For example, the leaves of tree 910 may drop or change color in different seasons, the shape, location, or color of crosswalk 912 may change after being repainted, and the color of building 904 may fade over time.
[0100] In one aspect, when a UE is configured to verify whether an image captured by its camera at a location is potentially spoofed, the UE may obtain (eg, download) an image associated with the location from a map database, such as Figure 9A900A. In some examples, since obtaining an image from a map in a map database specifies high communication resources, the UE can be configured to trigger the image verification process when the uncertainty of the UE's 6D solution (e.g., position and attitude) is below a specified uncertainty threshold. For example, when the UE determines that its positioning uncertainty is less than X meters (e.g., positioning uncertainty <5 meters) and / or its heading uncertainty is less than Y degrees (e.g., heading uncertainty <10 degrees), etc., the UE can trigger the image verification process.
[0101] Figure 9B FIG900B illustrates an example spoofed image according to various aspects of the present disclosure. After obtaining an image from a map database, the UE may verify the overall consistency between the image from the map database and its captured image (or with the UE's camera vision). If the image captured by the UE is spoofed, the image may include (or not include) features that match the image from the map database. For example, as shown at 914, the spoofed image may include a stop sign 916 (which may be referred to as a spoofed feature) that is not present in the image from the map database (e.g., as shown in FIG900A), and / or the spoofed image may not include a fire hydrant 906 that is present in the image from the map database. Thus, based on the comparison / verification, the UE may determine whether the captured image (or its camera vision) is likely spoofed.
[0102] In one example, the UE may be configured to perform a comparison / verification between its captured images and images from the map database based on a specified order, which may be based on a specified level of certainty / accuracy (e.g., performing additional / more steps for higher levels of certainty / accuracy). For example, the UE may be configured to first perform an object expectation check, wherein the UE may check whether invariant components / features identified in its captured images are also present in images from the map database (e.g., the presence of buildings 904, fire hydrants 906, and / or road signs 908, etc.).
[0103] If the object passes the inspection (e.g., an invariant component / feature identified in its captured image is present in the image from the map database), the UE may be configured to next perform an object position check, wherein the UE may check whether the position of one or more selected components / features from its captured image is consistent with the image from the map database (e.g., within a distance threshold). In other words, the UE may check whether the invariant components / features are in the correct position. For example, as shown at 916, road sign 908 in the spoofed image may be in a different position. If the UE determines that road sign 908 in its captured image is greater than X centimeters / meters (e.g., as shown in diagram 900A) compared to road sign 908 in the image from the map database (e.g., the position error of road sign 908 is >50 cm), the UE may determine that its captured image (or its camera vision) is likely spoofed. Similarly, the UE may store or output an indication of the spoofed image to other entities (e.g., a positioning engine, a location server, etc.) so that other entities are aware of the spoofed image. After the UE determines that its captured image is spoofed, it may perform camera-based positioning and / or camera-assisted positioning without the spoofed image / feature.
[0104] If the object location check also passes (e.g., the location of the invariant component / feature identified in its captured image is within a distance threshold compared to the image from the map database), the UE can also be configured to perform an object attribute check, in which the UE can check whether other object attributes in its captured image, such as varying components / features (e.g., shape, size, color, etc.), also match the image from the map database. For example, the UE can compare whether the color of building 904 or fire hydrant 906 is consistent with the image from the map database. If the color differs (e.g., by a threshold), the UE can determine that its captured image is likely spoofed. Similarly, the UE can store or output an indication of the spoofed image to other entities (e.g., a positioning engine, a location server, etc.) so that other entities can be aware of the spoofed image. After the UE determines that its captured image is spoofed, it can perform camera-based positioning and / or camera-assisted positioning without the spoofed image / feature. On the other hand, if the color also matches, the UE can determine that its captured image (or its camera vision) is not spoofed. The UE can then continue to perform camera-based positioning and / or camera-assisted positioning using its camera. In another example, if the UE is traveling on a road or in a tunnel, the UE may check whether the road heading is straight all the time, wherein if the UE's camera vision is feeding back a curved road, the UE may detect a spoofing event.
[0105] In another aspect of the present disclosure, a UE can verify whether images captured by its camera are spoofed by varying its camera sampling rate. When a camera operates at a sampling rate higher than the video's display rate (or refresh rate), certain patterns or observations may indicate that the image captured by the UE is a virtual image (rather than natural light reflected from an actual object). This method can be used to detect all types of virtual images, regardless of whether they are spoofed.
[0106] Figure 10A FIG1000A is a diagram illustrating an example image captured by a camera from a real environment (eg, based on natural light reflected from real objects) according to various aspects of the present disclosure. As shown in 1002, the image captured by the camera from the real environment can clearly depict objects in the environment.
[0107] Figure 10B FIG1000B is a diagram illustrating an example image captured from a video displayed at a display / refresh rate lower than the sampling rate of the camera according to various aspects of the present disclosure. Figure 6 Images captured by a UE (such as a display shown at 602 of the UE) may include "blank gaps" 1006, which may be used by the UE to identify that its captured images may be associated with virtual images / objects. Thus, if the captured images display certain patterns or observations (e.g., blank gaps, flickering, distortion, etc.) that may occur when the camera is operating at a higher sampling rate than the display / refresh rate of the video, the UE may determine that its captured images (or its camera vision) may be spoofing.
[0108] In another aspect of the present disclosure, the UE may verify whether the image captured by the camera is spoofed by cross-checking the captured image with images captured by the UE's other cameras (or from another UE), and / or by changing the focal length of its camera. Figure 5A UE (e.g., vehicle 502) may include multiple cameras, where the UE can verify whether an image captured by one of its cameras (e.g., the FOV of the rear-facing camera 508) is spoofed by comparing the image captured by that camera with an image captured by another camera (e.g., the FOV of the side-facing camera 506). This cross-check mechanism enables the UE to detect whether an image captured by its camera is based on a virtual image (e.g., a slowly rotating rebroadcast video). If the cross-check reveals an inconsistency (e.g., a large inconsistency), this may indicate a potential image or virtual feature spoofing event. Similarly, the UE may store or output an indication of the spoofed image to other entities (e.g., a positioning engine, a location server, etc.) so that the other entities are aware of the spoofed image. After the UE determines that the image it captured is spoofed, it may perform camera-based positioning and / or camera-assisted positioning without the spoofed image / feature. However, if the image is not spoofed, the UE may continue to perform camera-based positioning and / or camera-assisted positioning using its camera.
[0109] In one example, if multiple cameras or FOVs are available, the UE may derive / estimate the 6D pose (which in some examples may also be referred to as an IMU pose) of each camera / FOV based on its vision. The UE may then cross-check / compare the derived / estimated 6D poses from the different cameras / FOVs to determine if there are any inconsistencies. For example, if the 6D pose derived based on the vision of the rear-facing camera differs from the 6D pose derived based on the vision of the side cameras and / or the vision of the front camera, the UE may determine that the image captured by the rear-facing camera is spoofed. Similarly, the UE may store or output an indication of the spoofed image to other entities (e.g., a positioning engine, a location server, etc.) so that the other entities are aware of the spoofed image. After the UE determines that the image it captured is spoofed, it may perform camera-based positioning and / or camera-assisted positioning without the spoofed image / features. However, if the image is not spoofed, the UE may continue to perform camera-based positioning and / or camera-assisted positioning using its camera. In another example, when using a monostatic camera (e.g., meaning a single camera is used but it is capable of providing multiple FOVs), the UE can change the camera's FOV to detect or eliminate potential virtual image spoofing. This can also improve camera power optimization.
[0110] In another aspect of the present disclosure, a UE may verify whether an image captured by its camera is spoofed by changing the focal length of the camera. For example, the UE may change the focal length of its camera by zooming in (e.g., making the image captured by the camera appear larger and closer) and / or by zooming out (e.g., making the image captured by the camera appear smaller and farther away). If the focal length of the camera changes, the positions of one or more features (e.g., objects) captured by the camera and / or the distances of these positions from the UE derived from a vision-assisted positioning engine (PE) (e.g., based on sensor fusion positioning: GNSS+IMU+camera) may remain the same. Thus, if a feature / object is virtually spoofed (e.g., Figure 9B 916 in the image), the position or distance of the feature / object may also change as the focus is adjusted.
[0111] In another aspect of the present disclosure, the UE may verify whether an image captured by its camera is spoofed based on performing geometric checks on one or more objects in the captured image using additional sensors.
[0112] In one example, based on vision-based or vision-assisted positioning, the UE may be able to identify the distance of one or more objects / features from the UE. Figure 9A , the UE may be able to identify the distance of the fire hydrant 906 from the UE and / or the distance of the road sign 908 from the UE based on vision-based or vision-assisted positioning. If the UE also includes other sensors, such as radar and / or light detection and ranging (Lidar), the UE may also use these sensors to estimate the physical distance of the fire hydrant 906 from the UE and / or the distance of the road sign 908 from the UE. If there is a discrepancy between the distances of one or more objects / features derived from the vision-based or vision-assisted positioning and from the other sensors, it may indicate that the image captured by the camera may be virtually generated or inaccurate (e.g., Figure 9B In other words, the physical distance of an object / feature can be estimated in the vision-assisted localization engine based on focal length (e.g., image depth). If it comes from a virtual object, it may not pass the physical distance check.
[0113] In another example, based on vision-based or vision-assisted positioning, the UE may also be able to identify the velocity of one or more objects / features. Figure 7, truck 702 may be able to identify the speed of vehicle 708 based on vision-based or vision-assisted positioning. If truck 702 also includes other sensors, such as radar and / or lidar, truck 702 may also use these sensors to estimate the speed of vehicle 708. If there is a discrepancy between the speed of vehicle 708 derived from vision-based or vision-assisted positioning and from other sensors, it may indicate that the image captured by the camera of truck 702 may be virtually generated or inaccurate (e.g., Figure 8 In other words, if the relative velocity between the UE and the object / feature does not match the relative velocity derived from the UE (e.g., GNSS+IMU+camera), the object / feature may be virtually created.
[0114] In another example, the UE may be able to identify whether one or more objects captured by its camera are virtual objects by examining the shape of the one or more objects. In some examples, the UE may use radar and / or lidar to identify the shape of the object. For example, referring back to Figure 6 , the shape of the car 604 from the video (e.g., as shown at 602) and / or the shape of the cyclist from the billboard (e.g., as shown at 606) may be flat (e.g., a flat screen shape, a flat rectangular shape, etc.), which may be different from the shape of real objects (e.g., the shape of a real car, the shape of a real cyclist, etc.). Therefore, if the shape of an object identified by the UE's sensor is different from an actual object, the UE may determine that the object may be a virtual object and / or the UE may also determine that the image captured by its camera may be spoofed.
[0115] Figure 11 1100 is a flow chart of a method for wireless communication. The method may be performed by a UE (e.g., UE 104, 404, 902; vehicle 502; truck 702; device 1304). The method may enable the UE to identify whether an image captured by a camera of the UE is spoofed (e.g., a real image or a fake / manipulated / virtual image, etc.).
[0116] At 1102, the UE may obtain a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera, such as in conjunction with Figure 5 For example, as shown at 510, the front camera 504 of the vehicle 502 can obtain a set of images for a camera-assisted positioning session (e.g., GNSS+camera). The set of images can be obtained by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0117] In one example, the camera-assisted positioning session may be associated with the absolute position of the UE, the relative position of the UE, the orientation of the UE, or a combination thereof.
[0118] At 1104, the UE may detect the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session, such as in combination with Figure 9A 、 Figure 9B 、 Figure 10A and Figure 10B For example, as described in conjunction with Figure 9A and Figure 9B As described, the UE may detect that at least one spoofing feature (e.g., a stop sign 916) is present in an image captured by the UE by comparing the captured image with an image from the map data. Detecting the at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0119] In one example, to detect the presence of at least one spoofing feature in the set of images, the UE may: estimate a first distance of at least one object in the set of images; adjust a focal length of at least one first camera; capture a second set of images using the at least one first camera based on the adjusted focal length; estimate a second distance of at least one object based on the second set of images; and determine that at least one spoofing feature is present if the first distance differs from the second distance by at least a distance threshold.
[0120] In another example, to detect the presence of at least one spoofing feature in the set of images, the UE may: estimate a first speed of at least one object in the set of images; estimate a second speed of the at least one object using a non-camera sensor; and determine that at least one spoofing feature is present if the first speed differs from the second speed by at least a speed threshold.
[0121] In another example, to detect the presence of at least one spoofing feature in the set of images, the UE may: estimate a first shape of at least one object in the set of images; estimate a second shape of the at least one object using a non-camera sensor; and determine that at least one spoofing feature exists if the first shape is different from the second shape.
[0122] At 1106, the UE may store or output an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images, such as in conjunction with Figure 9A 、 Figure 9B 、 Figure 10A and Figure 10BFor example, as shown at 510, the UE may store or output an indication of the spoofing image to other entities (e.g., a positioning engine, a location server, etc.) so that the other entities may be aware of the spoofing image. Storing or outputting an indication of at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0123] At 1108, the UE may perform a camera-assisted positioning session based on at least one non-spoofing feature, wherein the at least one non-spoofing feature is different from the at least one spoofing feature, such as in conjunction with Figure 5 、 Figure 9A 、 Figure 9B 、 Figure 10A and Figure 10B For example, as described in conjunction with Figure 9A and Figure 9B As described, after the UE determines that the image it has captured is spoofed, the UE may perform camera-based positioning and / or camera-assisted positioning without the spoofed image / feature. However, if the image is not spoofed, the UE may continue to perform camera-based positioning and / or camera-assisted positioning using its camera. Performing a camera-assisted positioning session based on at least one non-spoofed feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0124] In one example, as shown at 1110, the UE may receive a map of the UE's current location and determine that at least one spoofing feature is present if one or more objects in the set of images do not match the map, such as in combination with Figure 9A and Figure 9B For example, as described in conjunction with Figure 9A and Figure 9B As described, the UE may receive an image associated with the UE's current location from a map database, and the UE may compare its captured image with the image from the map database to determine whether the captured image is spoofed or includes spoofed features / objects. The received map may be provided by, for example, Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0125] In one implementation, the UE may determine that an uncertainty associated with the UE's position or attitude is below a threshold, wherein the map is received in response to the uncertainty being below the threshold.
[0126] In another specific implementation, in order to determine the presence of at least one deceptive feature when one or more objects in the set of images do not match the map, the UE may extract a set of invariant features from the set of images, and if one or more invariant features in the set of invariant features do not exist in the map, the UE may determine that at least one deceptive feature exists.
[0127] In another specific implementation, in order to determine the presence of at least one spoofing feature when one or more objects in the set of images do not match the map, the UE may extract a feature set from the set of images, and if one or more features in the feature set are not located at corresponding positions within a position error threshold, the UE may determine that at least one spoofing feature exists.
[0128] In another implementation, to determine the presence of at least one spoofing feature when one or more objects in the set of images do not match the map, the UE may extract a set of attributes of the one or more objects from the set of images, and if the set of attributes is not present in the map, the UE may determine the presence of the at least one spoofing feature. In some implementations, the set of attributes may correspond to at least one of the following: a shape of the one or more objects, a size of the one or more objects, or a color of the one or more objects.
[0129] In another example, as shown at 1112, the UE may increase the sampling rate of the at least one first camera and determine that at least one spoofing feature is present if the set of images captured by the at least one first camera using the increased sampling rate includes one or more patterns or observations associated with a lower display refresh rate, such as in combination with Figure 9A and Figure 9B For example, as described in conjunction with Figure 9A and Figure 9B As described, the UE may increase the sampling rate of its camera and determine that the captured image is spoofed or includes a virtual object if the image it captures shows a blank gap 1006. Increasing the sampling rate and / or determining that at least one spoofing feature is present may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0130] In another example, as shown at 1114, the UE may capture a second set of images using at least one second camera, cross-check one or more objects between the set of images and the second set of images, and determine that at least one spoofing feature exists if there is a discrepancy between the one or more objects, such as in combination with Figure 5For example, the UE may verify whether an image captured by one of its cameras (e.g., the FOV of the rear camera 508) is spoofed by comparing the image captured by the one camera with the image captured by another camera (e.g., the FOV of the side camera 506). Cross-checking one or more objects between the set of images and the second set of images and / or determining the presence of at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0131] In another example, as shown at 1116, the UE may derive a first IMU pose of the UE based on the set of images, derive a second IMU pose of the UE based on a second set of images captured by at least one second camera of the UE, and determine that at least one spoofing feature exists if the first IMU pose differs from the second IMU pose by at least a pose threshold. For example, if multiple cameras or FOVs are available, the UE may derive / estimate its 6D pose (which may also be referred to as an IMU pose in some examples) based on the vision of each camera / FOV respectively. The UE may then cross-check / compare the derived / estimated 6D poses from different cameras / FOVs to determine if there are inconsistencies. For example, if the 6D pose derived based on the vision of the rear camera is different from the 6D pose derived based on the vision of the side camera and / or based on the vision of the front camera, the UE may determine that the image captured by the rear camera is spoofed. Deriving the IMU pose and / or determining the presence of at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0132] In another example, as shown at 1118, the UE may estimate a first distance of at least one object in the set of images, estimate a second distance of at least one object using a non-camera sensor, and determine that at least one spoofing feature is present if the first distance differs from the second distance by at least a distance threshold, such as in combination with Figure 9A and Figure 9B For example, as described in conjunction with Figure 9A and Figure 9BAs described, the UE may be able to identify the distance of the fire hydrant 906 from the UE and / or the distance of the road sign 908 from the UE based on vision-based or vision-assisted positioning. If the UE also includes other sensors, such as radar and / or lidar, the UE may also use these sensors to estimate the physical distance of the fire hydrant 906 from the UE and / or the distance of the road sign 908 from the UE. If there is a discrepancy between the distances of one or more objects / features derived from the vision-based or vision-assisted positioning and from the other sensors, it may indicate that the image captured by the camera may be virtually generated or inaccurate (e.g., Figure 9B Estimating a first distance of at least one object in the set of images, estimating a second distance of at least one object using a non-camera sensor, and / or determining the presence of at least one spoofing feature may be performed by, for example Figure 13 The system may be executed by the camera-assisted positioning component 198, the camera 1332, the one or more sensor modules 1318, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0133] In another example, the UE may verify that the at least one spoofing feature is a valid spoofing feature based on detecting the presence of the at least one spoofing feature in the set of images. Then, to store or output an indication of the at least one spoofing feature, the UE may store or output an indication of the at least one spoofing feature based on verifying that the at least one spoofing feature is a valid spoofing feature.
[0134] Figure 12 1200 is a flow chart of a method for wireless communication. The method may be performed by a UE (e.g., UE 104, 404, 902; vehicle 502; truck 702; device 1304). The method may enable the UE to identify whether an image captured by a camera of the UE is spoofed (e.g., a real image or a fake / manipulated image, etc.).
[0135] At 1202, the UE may obtain a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera, such as in conjunction with Figure 5 For example, as shown at 510, the front camera 504 of the vehicle 502 can obtain a set of images for a camera-assisted positioning session (e.g., GNSS+camera). The set of images can be obtained by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0136] In one example, the camera-assisted positioning session may be associated with the absolute position of the UE, the relative position of the UE, the orientation of the UE, or a combination thereof.
[0137] At 1204, the UE may detect the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session, such as in combination with Figure 9A 、 Figure 9B 、 Figure 10A and Figure 10B For example, as described in conjunction with Figure 9A and Figure 9B As described, the UE may detect that at least one spoofing feature (e.g., a stop sign 916) is present in an image captured by the UE by comparing the captured image with an image from the map data. Detecting the at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0138] In one example, to detect the presence of at least one spoofing feature in the set of images, the UE may: estimate a first distance of at least one object in the set of images; adjust a focal length of at least one first camera; capture a second set of images using the at least one first camera based on the adjusted focal length; estimate a second distance of at least one object based on the second set of images; and determine that at least one spoofing feature is present if the first distance differs from the second distance by at least a distance threshold.
[0139] In another example, to detect the presence of at least one spoofing feature in the set of images, the UE may: estimate a first speed of at least one object in the set of images; estimate a second speed of the at least one object using a non-camera sensor; and determine that at least one spoofing feature is present if the first speed differs from the second speed by at least a speed threshold.
[0140] In another example, to detect the presence of at least one spoofing feature in the set of images, the UE may: estimate a first shape of at least one object in the set of images; estimate a second shape of the at least one object using a non-camera sensor; and determine that at least one spoofing feature exists if the first shape is different from the second shape.
[0141] At 1206, the UE may store or output an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images, such as in conjunction with Figure 9A 、 Figure 9B 、 Figure 10A and Figure 10B For example, as shown at 510, the UE may store or output an indication of the spoofing image to other entities (e.g., a positioning engine, a location server, etc.) so that the other entities may be aware of the spoofing image. Storing or outputting an indication of at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0142] In one example, at 1208, the UE may perform a camera-assisted positioning session based on at least one non-spoofing feature, wherein the at least one non-spoofing feature is different from the at least one spoofing feature, such as in conjunction with Figure 5 、 Figure 9A 、 Figure 9B 、 Figure 10A and Figure 10B For example, as described in conjunction with Figure 9A and Figure 9B As described, after the UE determines that the image it has captured is spoofed, the UE may perform camera-based positioning and / or camera-assisted positioning without the spoofed image / feature. However, if the image is not spoofed, the UE may continue to perform camera-based positioning and / or camera-assisted positioning using its camera. Performing a camera-assisted positioning session based on at least one non-spoofed feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0143] In another example, the UE may receive a map of the UE's current location and determine that at least one spoofing feature is present if one or more objects in the set of images do not match the map, such as in combination with Figure 9A and Figure 9B For example, as described in conjunction with Figure 9A and Figure 9B As described, the UE may receive an image associated with the UE's current location from a map database, and the UE may compare its captured image with the image from the map database to determine whether the captured image is spoofed or includes spoofed features / objects. The received map may be provided by, for example, Figure 13The camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324, and / or the transceiver 1322 of the device 1304 in the embodiment of the present invention may be executed. In one embodiment, the UE may determine that an uncertainty associated with the position or posture of the UE is below a threshold, wherein the map is received in response to the uncertainty being below the threshold. In another embodiment, to determine that at least one spoofing feature exists when one or more objects in the set of images do not match the map, the UE may extract a set of invariant features from the set of images, and if one or more invariant features in the set of invariant features are not present in the map, the UE may determine that at least one spoofing feature exists. In another embodiment, to determine that at least one spoofing feature exists when one or more objects in the set of images do not match the map, the UE may extract a set of features from the set of images, and if one or more features in the set of features are not located at corresponding positions within a position error threshold, the UE may determine that at least one spoofing feature exists. In another implementation, to determine the presence of at least one spoofing feature when one or more objects in the set of images do not match the map, the UE may extract a set of attributes of the one or more objects from the set of images, and if the set of attributes is not present in the map, the UE may determine the presence of the at least one spoofing feature. In some implementations, the set of attributes may correspond to at least one of the following: a shape of the one or more objects, a size of the one or more objects, or a color of the one or more objects.
[0144] In another example, the UE may increase the sampling rate of the at least one first camera and determine that at least one spoofing feature is present if the set of images captured by the at least one first camera using the increased sampling rate includes one or more patterns or observations associated with a lower display refresh rate, such as in combination with Figure 9A and Figure 9B For example, as described in conjunction with Figure 9A and Figure 9B As described, the UE may increase the sampling rate of its camera and determine that the captured image is spoofed or includes a virtual object if the image it captures shows a blank gap 1006. Increasing the sampling rate and / or determining that at least one spoofing feature is present may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0145] In another example, the UE may capture a second set of images using at least one second camera, cross-check one or more objects between the set of images and the second set of images, and determine that at least one spoofing feature exists if there is a mismatch between the one or more objects, such as in combination with Figure 5For example, the UE may verify whether an image captured by one of its cameras (e.g., the FOV of the rear camera 508) is spoofed by comparing the image captured by the one camera with the image captured by another camera (e.g., the FOV of the side camera 506). Cross-checking one or more objects between the set of images and the second set of images and / or determining the presence of at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0146] In another example, the UE may derive a first IMU pose of the UE based on the set of images, derive a second IMU pose of the UE based on a second set of images captured by at least one second camera of the UE, and determine that at least one spoofing feature exists if the first IMU pose differs from the second IMU pose by at least a pose threshold. For example, if multiple cameras or FOVs are available, the UE may derive / estimate its 6D pose (which may also be referred to as IMU pose in some examples) based on the vision of each camera / FOV respectively. The UE may then cross-check / compare the derived / estimated 6D poses from different cameras / FOVs to determine if there are inconsistencies. For example, if the 6D pose derived based on the vision of the rear camera is different from the 6D pose derived based on the vision of the side camera and / or based on the vision of the front camera, the UE may determine that the image captured by the rear camera is spoofed. Deriving the IMU pose and / or determining the presence of at least one spoofing feature may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0147] In another example, the UE may estimate a first distance of at least one object in the set of images, estimate a second distance of at least one object using a non-camera sensor, and determine that at least one spoofing feature is present if the first distance differs from the second distance by at least a distance threshold, such as in combination with Figure 9A and Figure 9B For example, as described in conjunction with Figure 9A and Figure 9BAs described, the UE may be able to identify the distance of the fire hydrant 906 from the UE and / or the distance of the road sign 908 from the UE based on vision-based or vision-assisted positioning. If the UE also includes other sensors, such as radar and / or lidar, the UE may also use these sensors to estimate the physical distance of the fire hydrant 906 from the UE and / or the distance of the road sign 908 from the UE. If there is a discrepancy between the distances of one or more objects / features derived from the vision-based or vision-assisted positioning and from the other sensors, it may indicate that the image captured by the camera may be virtually generated or inaccurate (e.g., Figure 9B Estimating a first distance of at least one object in the set of images, estimating a second distance of at least one object using a non-camera sensor, and / or determining the presence of at least one spoofing feature may be performed by, for example Figure 13 The system may be executed by the camera-assisted positioning component 198, the camera 1332, the one or more sensor modules 1318, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0148] In another example, the UE may verify that the at least one spoofing feature is a valid spoofing feature based on detecting the presence of the at least one spoofing feature in the set of images. Then, to store or output an indication of the at least one spoofing feature, the UE may store or output an indication of the at least one spoofing feature based on verifying that the at least one spoofing feature is a valid spoofing feature.
[0149] Figure 131300 is a diagram illustrating an example of a hardware implementation for an apparatus 1304. The apparatus 1304 may be a UE, a component of a UE, or may implement UE functionality. In some aspects, the apparatus 1304 may include a cellular baseband processor 1324 (also referred to as a modem) coupled to one or more transceivers 1322 (e.g., a cellular RF transceiver). The cellular baseband processor 1324 may include on-chip memory 1324′. In some aspects, the apparatus 1304 may also include one or more subscriber identity module (SIM) cards 1320 and an application processor 1306 coupled to a secure digital (SD) card 1308 and a screen 1310. The application processor 1306 may include on-chip memory 1306′. In some aspects, the device 1304 may further include a Bluetooth module 1312, a WLAN module 1314, an SPS module 1316 (e.g., a GNSS module), an ultra-wideband (UWB) module 1336, one or more sensor modules 1318 (e.g., a barometric pressure sensor / altimeter; a motion sensor such as an inertial measurement unit (IMU), a gyroscope, and / or an accelerometer; light detection and ranging (LIDAR), radio-aided detection and ranging (RADAR), sound navigation and ranging (SONAR), a magnetometer, audio, and / or other technologies for positioning), an additional memory module 1326, a power source 1330, and / or a camera 1332. The Bluetooth module 1312, the WLAN module 1314, the UWB module 1336, and the SPS module 1316 may include an on-chip transceiver (TRX) (or, in some cases, only a receiver (RX)). The Bluetooth module 1312, WLAN module 1314, UWB module 1336, and SPS module 1316 may include their own dedicated antennas and / or utilize antenna 1380 for communication. The cellular baseband processor 1324 communicates with the UE 104 and / or RUs associated with the network entity 1302 via one or more antennas 1380 through the transceiver 1322. The cellular baseband processor 1324 and the application processor 1306 may each include computer-readable media / memory 1324', 1306', respectively. The additional memory module 1326 may also be considered a computer-readable medium / memory. Each computer-readable medium / memory 1324', 1306', 1326 may be non-transitory. The cellular baseband processor 1324 and the application processor 1306 are each responsible for general processing, including executing software stored on the computer-readable media / memory. This software, when executed by the cellular baseband processor 1324 / application processor 1306, causes the cellular baseband processor 1324 / application processor 1306 to perform the various functions described above. The computer-readable medium / memory may also be used to store data that is manipulated by the cellular baseband processor 1324 / application processor 1306 when executing the software.The cellular baseband processor 1324 / application processor 1306 may be a component of the UE 350 and may include the memory 360 and / or at least one of the TX processor 368, the RX processor 356, and the controller / processor 359. In one configuration, the device 1304 may be a processor chip (modem and / or applications) and include only the cellular baseband processor 1324 and / or the application processor 1306, and in another configuration, the device 1304 may be the entire UE (e.g., see. Figure 3 UE 350 ) and includes additional modules of device 1304.
[0150] As discussed above, camera-assisted positioning component 198 can be configured to obtain a set of images associated with a camera-assisted positioning session, wherein the set of images was captured using at least one first camera. Camera-assisted positioning component 198 can also be configured to detect the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session. Camera-assisted positioning component 198 can also be configured to store or output an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images. Camera-assisted positioning component 198 can be within cellular baseband processor 1324, application processor 1306, or both. Camera-assisted positioning component 198 can be one or more hardware components specifically configured to perform the process / algorithm, implemented by one or more processors configured to perform the process / algorithm, stored on a computer-readable medium for implementation by one or more processors, or some combination thereof. As shown, apparatus 1304 can include a variety of components configured for various functions. In one configuration, the apparatus 1304, and specifically the cellular baseband processor 1324 and / or the application processor 1306, may include means for obtaining a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera. The apparatus 1304 may also include means for detecting the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session. The apparatus 1304 may also include means for storing an indication of the at least one spoofing feature or means for outputting an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images.
[0151] In one configuration, a camera-assisted positioning session may be associated with an absolute position of the UE, a relative position of the UE, an orientation of the UE, or a combination thereof.
[0152] In another configuration, the means for detecting the presence of at least one deception feature in the set of images includes configuring the device 1304 to estimate a first distance of at least one object in the set of images; adjusting a focal length of at least one first camera; capturing a second set of images using the at least one first camera based on the adjusted focal length; estimating a second distance of at least one object based on the second set of images; and determining that at least one deception feature is present if the first distance differs from the second distance by at least a distance threshold.
[0153] In another configuration, the means for detecting the presence of at least one spoofing feature in the set of images includes configuring the device 1304 to estimate a first speed of at least one object in the set of images; estimate a second speed of the at least one object using a non-camera sensor; and determine that the at least one spoofing feature is present if the first speed differs from the second speed by at least a speed threshold.
[0154] In another configuration, the means for detecting the presence of at least one spoofing feature in the set of images includes configuring the device 1304 to estimate a first shape of at least one object in the set of images; estimate a second shape of the at least one object using a non-camera sensor; and determine that the at least one spoofing feature is present if the first shape is different from the second shape.
[0155] In another configuration, the apparatus 1304 may further include means for performing a camera-assisted positioning session based on at least one non-spoofing feature, wherein the at least one non-spoofing feature is different from the at least one spoofing feature.
[0156] In one configuration, the apparatus 1304 may further include: means for receiving a map of the UE's current location; and means for determining the presence of at least one spoofing feature if one or more objects in the set of images do not match the map. In one implementation, the apparatus 1304 may further include means for determining that an uncertainty associated with the UE's position or posture is below a threshold, wherein the map is received in response to the uncertainty being below the threshold. In another implementation, the means for determining the presence of at least one spoofing feature if one or more objects in the set of images do not match the map may include configuring the apparatus 1304 to extract a set of invariant features from the set of images, and determining the presence of the at least one spoofing feature if one or more invariant features in the set of invariant features are not present in the map. In another implementation, the means for determining the presence of at least one spoofing feature if one or more objects in the set of images do not match the map may include configuring the apparatus 1304 to extract a set of features from the set of images, and determining the presence of the at least one spoofing feature if one or more features in the set of features are not located at corresponding locations within a position error threshold. In another implementation, if one or more objects in the set of images do not match the map, the means for determining that at least one spoofing feature is present may include configuring the device 1304 to extract a set of attributes of the one or more objects from the set of images, and determining that the at least one spoofing feature is present if the set of attributes is not present in the map. In some implementations, the set of attributes may correspond to at least one of the following: a shape of the one or more objects, a size of the one or more objects, or a color of the one or more objects.
[0157] In another configuration, the UE may increase the sampling rate of the at least one first camera and determine that at least one spoofing feature is present if the set of images captured by the at least one first camera using the increased sampling rate includes one or more patterns or observations associated with a lower display refresh rate, such as in combination with Figure 9A and Figure 9B For configuration, as described in Figure 9A and Figure 9B As described, the UE may increase the sampling rate of its camera and determine that the captured image is spoofed or includes a virtual object if the image it captures shows a blank gap 1006. Increasing the sampling rate and / or determining that at least one spoofing feature is present may be performed by, for example Figure 13 The method may be executed by the camera-assisted positioning component 198, the camera 1332, the application processor 1306, the cellular baseband processor 1324 and / or the transceiver 1322 of the device 1304.
[0158] In another configuration, the device 1304 may also include: a component for capturing a second set of images using at least one second camera; a component for cross-checking one or more objects between the set of images and the second set of images; and a component for determining the presence of at least one fraudulent feature if there is an inconsistency in the one or more objects.
[0159] In another configuration, the device 1304 may also include: a component for deriving a first IMU pose of the UE based on the set of images; a component for deriving a second IMU pose of the UE based on a second set of images captured by at least one second camera of the UE; and a component for determining that at least one spoofing feature is present if the first IMU pose differs from the second IMU pose by at least a pose threshold.
[0160] In another configuration, the apparatus 1304 may further include: means for estimating a first distance of at least one object in the set of images; means for estimating a second distance of the at least one object using a non-camera sensor; and means for determining that at least one spoofing feature is present if the first distance differs from the second distance by at least a distance threshold.
[0161] In another configuration, the apparatus 1304 may further include means for verifying that the at least one spoofing feature is a valid spoofing feature based on detecting the presence of the at least one spoofing feature in the set of images. The means for storing an indication of the at least one spoofing feature or the means for outputting an indication of the at least one spoofing feature may include configuring the apparatus 1304 to store or output the indication of the at least one spoofing feature based on verifying that the at least one spoofing feature is a valid spoofing feature.
[0162] The means may be the camera assisted positioning component 198 of the apparatus 1304 configured to perform the functions recited by the means. As described above, the apparatus 1304 may include the TX processor 368, the RX processor 356, and the controller / processor 359. Thus, in one configuration, the means may be the TX processor 368, the RX processor 356, and / or the controller / processor 359 configured to perform the functions recited by the means.
[0163] It should be understood that the specific order or hierarchy of blocks in the disclosed process / flowchart is merely illustrative of exemplary methods. It should be understood that the specific order or hierarchy of blocks in the process / flowchart may be rearranged based on design preferences. In addition, some blocks may be combined or omitted. The accompanying method claims present elements of the various blocks in a sample order, but are not limited to the specific order or hierarchy presented.
[0164] The foregoing description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Therefore, the claims are not limited to the various aspects described herein, but should be given the full scope consistent with the language claims. Unless otherwise specified, references to elements in the singular form do not mean "one and only one", but "one or more". Terms such as "if", "when" and "while" do not imply a direct temporal relationship or reaction. That is, these phrases, such as "when...", do not mean immediate action in response to the occurrence of an action or during the occurrence of an action, but simply imply that if the conditions are met, the action will occur, but there is no need for a specific or immediate time limit for the action to occur. The word "exemplary" is used herein to mean "used as an example, instance or illustration". Any aspect described herein as "exemplary" is not necessarily to be interpreted as preferred or having advantages over other aspects. Unless otherwise specified, the term "some" refers to one or more. Combinations such as “at least one of A, B, or C,” “one or more of A, B, or C,” “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, C, or any combination thereof” include any combination of A, B, and / or C, which may include multiple As, multiple Bs, or multiple Cs. Specifically, combinations such as “at least one of A, B, or C,” “one or more of A, B, or C,” “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, C, or any combination thereof” may be only A, only B, only C, A and B, A and C, B and C, or A, B, and C, where any such combination may include one or more members of A, B, or C. A set should be interpreted as a set of elements, where the number of elements is one or more. Thus, for a set of X, X will include one or more elements. If a first device receives data from or sends data to a second device, the data may be received / sent directly between the first and second devices, or indirectly between the first and second devices via a collection of devices. A device configured to "output" data (such as a transmission, signal, or message) may, for example, transmit the data using a transceiver or transmit the data to the device that transmitted the data. A device configured to "obtain" data (such as a transmission, signal, or message) may, for example, receive the data using a transceiver or obtain the data from the device that received the data. All structural and functional equivalents to the elements of the various aspects described throughout this disclosure that are or later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims.Furthermore, nothing disclosed herein is intended to be dedicated to the public, regardless of whether such disclosure is explicitly recited in the claims. Words such as "module," "mechanism," "element," and "device" are not intended to replace the word "component." Thus, no claim element is to be construed as part-plus-function unless the element is explicitly recited using the phrase "component for..."
[0165] As used herein, the phrase "based on" should not be interpreted as referring to a closed set of information, one or more conditions, one or more factors, etc. In other words, the phrase "based on A" (where "A" can be information, a condition, a factor, etc.) should be interpreted as "based at least on A" unless specifically stated differently.
[0166] The following aspects are merely illustrative and may be combined with other aspects or teachings described herein without limitation.
[0167] Aspect 1 is a method for wireless communication at a UE, the method comprising: obtaining a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera; detecting the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session; and storing or outputting an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images.
[0168] Aspect 2 is the method according to aspect 1, further comprising: performing the camera-assisted positioning session based on at least one non-spoofing feature, wherein the at least one non-spoofing feature is different from the at least one spoofing feature.
[0169] Aspect 3 is a method according to Aspect 1 or 2, wherein detecting the presence of the at least one deceptive feature in the set of images includes: receiving a map of the current location of the UE; and determining that the at least one deceptive feature exists if one or more objects in the set of images do not match the map.
[0170] Aspect 4 is a method according to aspect 3, the method further comprising: determining that an uncertainty associated with the position or attitude of the UE is lower than a threshold, wherein the map is received in response to the uncertainty being lower than the threshold.
[0171] Aspect 5 is a method according to aspect 3, wherein if the one or more objects in the set of images do not match the map, determining that the at least one deceptive feature exists includes: extracting a set of invariant features from the set of images; and determining that the at least one deceptive feature exists if one or more invariant features in the set of invariant features do not exist in the map.
[0172] Aspect 6 is a method according to Aspect 3, wherein if the one or more objects in the set of images do not match the map, determining that the at least one deceptive feature exists includes: extracting a feature set from the set of images; and determining that the at least one deceptive feature exists if one or more features in the feature set are not located at corresponding positions within a position error threshold.
[0173] Aspect 7 is a method according to aspect 3, wherein if the one or more objects in the set of images do not match the map, determining that the at least one deception feature exists includes: extracting a set of attributes of the one or more objects from the set of images; and if the set of attributes does not exist in the map, determining that the at least one deception feature exists.
[0174] Aspect 8 is the method of aspect 7, wherein the set of attributes corresponds to at least one of the following: a shape of the one or more objects, a size of the one or more objects, or a color of the one or more objects.
[0175] Aspect 9 is a method according to any one of Aspects 1 to 8, wherein detecting the presence of the at least one deception feature in the set of images includes: increasing the sampling rate of the at least one first camera; and determining that the at least one deception feature exists if the set of images captured by the at least one first camera using the increased sampling rate includes one or more patterns or observations associated with a lower display refresh rate.
[0176] Aspect 10 is a method according to any one of Aspects 1 to 9, wherein detecting the presence of the at least one deceptive feature in the set of images includes: capturing a second set of images using at least one second camera; cross-checking one or more objects between the set of images and the second set of images; and determining that the at least one deceptive feature exists if there is an inconsistency between the one or more objects.
[0177] Aspect 11 is a method according to any one of Aspects 1 to 10, wherein detecting the presence of the at least one deception feature in the set of images includes: deriving a first IMU posture of the UE based on the set of images; deriving a second IMU posture of the UE based on a second set of images captured by at least one second camera of the UE; and determining that the at least one deception feature exists if the first IMU posture differs from the second IMU posture by at least a posture threshold.
[0178] Aspect 12 is a method according to any one of Aspects 1 to 11, wherein detecting the presence of the at least one deception feature in the set of images includes: estimating a first distance of at least one object in the set of images; adjusting the focal length of the at least one first camera; capturing a second set of images using the at least one first camera based on the adjusted focal length; estimating a second distance of the at least one object based on the second set of images; and determining that the at least one deception feature exists if the first distance differs from the second distance by at least a distance threshold.
[0179] Aspect 13 is a method according to any one of aspects 1 to 12, wherein detecting the presence of the at least one deceptive feature in the set of images includes: estimating a first distance of at least one object in the set of images; estimating a second distance of the at least one object using a non-camera sensor; and determining that the at least one deceptive feature exists if the first distance differs from the second distance by at least a distance threshold.
[0180] Aspect 14 is a method according to any one of aspects 1 to 13, wherein detecting the presence of the at least one deception feature in the set of images includes: estimating a first speed of at least one object in the set of images; estimating a second speed of the at least one object using a non-camera sensor; and determining that the at least one deception feature is present if the first speed differs from the second speed by at least a speed threshold.
[0181] Aspect 15 is a method according to any one of Aspects 1 to 14, wherein detecting the presence of the at least one deceptive feature in the set of images includes: estimating a first shape of at least one object in the set of images; estimating a second shape of the at least one object using a non-camera sensor; and determining that the at least one deceptive feature exists if the first shape is different from the second shape.
[0182] Aspect 16 is a method according to any one of aspects 1 to 15, wherein the camera-assisted positioning session is associated with the absolute position of the UE, the relative position of the UE, the orientation of the UE, or a combination thereof.
[0183] Aspect 17 is a method according to any one of Aspects 1 to 16, the method further comprising: based on detecting the presence of the at least one deceptive feature in the set of images, verifying that the at least one deceptive feature is a valid deceptive feature; wherein storing or outputting the indication of the at least one deceptive feature includes: based on verifying that the at least one deceptive feature is the valid deceptive feature, storing or outputting the indication of the at least one deceptive feature.
[0184] Aspect 18 is an apparatus for wireless communication at a UE, the apparatus comprising: a memory; and at least one processor coupled to the memory, and based at least in part on information stored in the memory, the at least one processor is configured to implement any one of aspects 1 to 17.
[0185] Aspect 19 is the apparatus of aspect 18, further comprising at least one of a transceiver or an antenna coupled to the at least one processor.
[0186] Aspect 20 is an apparatus for wireless communication, comprising means for implementing any one of aspects 1 to 17.
[0187] Aspect 21 is a computer-readable medium (eg, non-transitory computer-readable medium) storing computer-executable code, wherein the code, when executed by a processor, causes the processor to implement any one of aspects 1 to 17.
Claims
1. An apparatus for wireless communication at a user equipment (UE), the apparatus comprising: Memory; and at least one processor coupled to the memory, and the at least one processor configured to: obtaining a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera; detecting the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session; as well as Based on the presence of the at least one spoofing feature in the set of images, an indication of the at least one spoofing feature is stored or output.
2. The apparatus of claim 1 , wherein the at least one processor is further configured to: The camera-assisted positioning session is performed based on at least one non-spoofing feature, wherein the at least one non-spoofing feature is different from the at least one spoofing feature.
3. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: receiving a map of the current location of the UE; and If one or more objects in the set of images do not match the map, then it is determined that the at least one deceptive feature is present.
4. The apparatus of claim 3 , wherein the at least one processor is further configured to: It is determined that an uncertainty associated with the position or attitude of the UE is below a threshold, wherein the at least one processor is configured to receive the map in response to the uncertainty being below the threshold.
5. The apparatus of claim 3 , wherein to determine that the at least one spoofing feature is present if the one or more objects in the set of images do not match the map, the at least one processor is configured to: extracting a set of invariant features from the set of images; and If one or more invariant features in the set of invariant features are not present in the map, then it is determined that the at least one deceptive feature is present.
6. The apparatus of claim 3 , wherein to determine that the at least one spoofing feature is present if the one or more objects in the set of images do not match the map, the at least one processor is configured to: extracting a set of features from the set of images; and If one or more features in the feature set are not located at corresponding positions within a position error threshold, then it is determined that the at least one spoof feature is present.
7. The apparatus of claim 3 , wherein to determine that the at least one spoofing feature is present if the one or more objects in the set of images do not match the map, the at least one processor is configured to: extracting a set of attributes of the one or more objects from the set of images; and If the attribute set does not exist in the map, it is determined that the at least one deceptive feature exists.
8. The apparatus of claim 7, wherein the set of attributes corresponds to at least one of: the shape of the one or more objects, the size of the one or more objects, or The color of the one or more objects.
9. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: increasing a sampling rate of the at least one first camera; and The at least one spoofing feature is determined to be present if the set of images captured by the at least one first camera using the increased sampling rate includes one or more patterns or observations associated with a lower display refresh rate.
10. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: capturing a second set of images using at least one second camera; cross-checking one or more objects between the set of images and the second set of images; and If there is an inconsistency between the one or more objects, it is determined that the at least one fraudulent feature exists.
11. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: deriving a first inertial measurement unit (IMU) pose of the UE based on the set of images; deriving a second IMU pose of the UE based on a second set of images captured by at least one second camera of the UE; as well as If the first IMU pose differs from the second IMU pose by at least a pose threshold, then it is determined that the at least one spoofing feature is present.
12. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: estimating a first distance of at least one object in the set of images; adjusting a focal length of the at least one first camera; capturing a second set of images using the at least one first camera based on the adjusted focal length; estimating a second distance of the at least one object based on the second set of images; as well as If the first distance differs from the second distance by at least a distance threshold, then it is determined that the at least one spoofing feature is present.
13. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: estimating a first distance of at least one object in the set of images; estimating a second distance of the at least one object using a non-camera sensor; as well as If the first distance differs from the second distance by at least a distance threshold, then it is determined that the at least one spoofing feature is present.
14. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: estimating a first velocity of at least one object in the set of images; estimating a second velocity of the at least one object using a non-camera sensor; and If the first speed differs from the second speed by at least a speed threshold, then it is determined that the at least one fraudulent feature is present.
15. The apparatus of claim 1 , wherein to detect the presence of the at least one spoofing feature in the set of images, the at least one processor is configured to: estimating a first shape of at least one object in the set of images; estimating a second shape of the at least one object using a non-camera sensor; as well as If the first shape is different from the second shape, then it is determined that the at least one deceptive feature is present.
16. The apparatus of claim 1, wherein the camera-assisted positioning session is associated with an absolute position of the UE, a relative position of the UE, an orientation of the UE, or a combination thereof.
17. The apparatus of claim 1 , wherein the at least one processor is further configured to: verifying, based on the detecting presence of the at least one spoofing feature in the set of images, that the at least one spoofing feature is a valid spoofing feature; Wherein, in order to store or output the indication of the at least one fraudulent feature, the at least one processor is configured to: store or output the indication of the at least one fraudulent feature based on the verification that the at least one fraudulent feature is the valid fraudulent feature.
18. A method of wireless communication at a user equipment (UE), the method comprising: obtaining a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera; detecting the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session; as well as Based on the presence of the at least one spoofing feature in the set of images, an indication of the at least one spoofing feature is stored or output.
19. The method according to claim 18, further comprising: The camera-assisted positioning session is performed based on at least one non-spoofing feature, wherein the at least one non-spoofing feature is different from the at least one spoofing feature.
20. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: receiving a map of the current location of the UE; as well as If one or more objects in the set of images do not match the map, then it is determined that the at least one deceptive feature is present.
21. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: increasing a sampling rate of the at least one first camera; as well as The at least one spoofing feature is determined to be present if the set of images captured by the at least one first camera using the increased sampling rate includes one or more patterns or observations associated with a lower display refresh rate.
22. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: capturing a second set of images using at least one second camera; cross-checking one or more objects between the set of images and the second set of images; as well as If there is an inconsistency between the one or more objects, it is determined that the at least one fraudulent feature exists.
23. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: deriving a first inertial measurement unit (IMU) pose of the UE based on the set of images; deriving a second IMU pose of the UE based on a second set of images captured by at least one second camera of the UE; as well as If the first IMU pose differs from the second IMU pose by at least a pose threshold, then it is determined that the at least one spoofing feature is present.
24. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: estimating a first distance of at least one object in the set of images; adjusting a focal length of the at least one first camera; capturing a second set of images using the at least one first camera based on the adjusted focal length; estimating a second distance of the at least one object based on the second set of images; as well as If the first distance differs from the second distance by at least a distance threshold, then it is determined that the at least one spoofing feature is present.
25. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: estimating a first distance of at least one object in the set of images; estimating a second distance of the at least one object using a non-camera sensor; as well as If the first distance differs from the second distance by at least a distance threshold, then it is determined that the at least one spoofing feature is present.
26. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: estimating a first velocity of at least one object in the set of images; estimating a second velocity of the at least one object using a non-camera sensor; as well as If the first speed differs from the second speed by at least a speed threshold, then it is determined that the at least one fraudulent feature is present.
27. The method of claim 18, wherein detecting the presence of the at least one spoofing feature in the set of images comprises: estimating a first shape of at least one object in the set of images; estimating a second shape of the at least one object using a non-camera sensor; as well as If the first shape is different from the second shape, then it is determined that the at least one deceptive feature is present.
28. The method of claim 18, further comprising: verifying that the at least one spoofing feature is a valid spoofing feature based on detecting the presence of the at least one spoofing feature in the set of images; Wherein storing or outputting the indication of the at least one deceptive feature comprises: storing or outputting the indication of the at least one deceptive feature based on verifying that the at least one deceptive feature is the valid deceptive feature.
29. An apparatus for wireless communication at a user equipment (UE), the apparatus comprising: means for obtaining a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera; means for detecting the presence of at least one spoofing feature in said set of images during said camera-assisted positioning session; and Means for storing or outputting an indication of the at least one spoofing feature based on the presence of the at least one spoofing feature in the set of images.
30. A computer-readable medium storing computer-executable code at a user equipment (UE) that, when executed by a processor, causes the processor to: obtaining a set of images associated with a camera-assisted positioning session, wherein the set of images is captured using at least one first camera; detecting the presence of at least one spoofing feature in the set of images during the camera-assisted positioning session; and Based on the presence of the at least one spoofing feature in the set of images, an indication of the at least one spoofing feature is stored or output.