Positioning and beam alignment based on optical sensing

Abstract

Techniques are provided for assisting positioning of a user equipment (UE) by a wireless network node using optical sensing. In some embodiments, such techniques can include transmitting first configuration information to a UE, the first configuration information indicating how optical sensing data is obtained with the UE; receiving optical sensing data from the UE; and determining angle information regarding a wireless transmission between the at least one wireless network node and the UE based at least on the optical sensing data received from the UE.

Description

[0001] Related Applications

[0002] This application claims the benefit of U.S. application No. 17 / 931,071, filed on September 9, 2022, entitled “POSITIONING AND BEAM ALIGNMENTBASED ON OPTICAL SENSING,” which is assigned to the assignee of this application and is incorporated herein by reference in its entirety. Background Art 1. Technical Field

[0003] The present disclosure relates generally to the fields of wireless communications and computer vision, and more particularly to using optical signals (eg, visual information) obtained by network devices to locate user equipment (UE) or determine beams between wireless network devices.

[0004] 2. Related technologies

[0005] Visual odometry allows the location or orientation of a device to be determined based on images or other optical information. In one application, visual odometry may involve comparing image frames (e.g., in a video sequence) to infer the trajectory of an autonomous vehicle, which refers to a vehicle that includes sensors to perceive the environment around the vehicle. The trajectory of the autonomous vehicle may be expressed in a camera coordinate system or converted to world coordinates. Summary of the invention

[0006] In one aspect of the present disclosure, a method for assisting positioning of a user equipment (UE) is disclosed. In some embodiments, the method includes: transmitting first configuration information to the UE, the first configuration information indicating how to obtain optical sensing data using the UE; receiving optical sensing data from the UE; and determining angle information about wireless transmissions between at least one wireless network node and the UE based at least on the optical sensing data received from the UE.

[0007] In another aspect of the present disclosure, a device of a wireless communication network is disclosed. In some embodiments, the device includes: one or more network interfaces; a memory; and one or more processors, the one or more processors being communicatively coupled to the one or more network interfaces and the memory and configured to: transmit first configuration information to a user equipment (UE), the first configuration information indicating how to obtain optical sensing data using the UE; receive the optical sensing data from the UE; and determine angle information about wireless transmission between at least one wireless network node and the UE based at least on the optical sensing data received from the UE.

[0008] In another aspect of the present disclosure, a method for beam alignment with a user equipment (UE) is disclosed. In some embodiments, the method includes: receiving first configuration information from a network entity, the first configuration information indicating how to obtain optical sensing data using the UE; obtaining the optical sensing data using a sensor of the UE; and obtaining angle information about wireless transmissions between at least one wireless network node and the UE based at least on the optical sensing data.

[0009] In another aspect of the present disclosure, a user equipment (UE) is disclosed. In some embodiments, the UE includes: one or more network interfaces; a memory; a sensor; and one or more processors, the one or more processors being communicatively coupled to the one or more network interfaces, the sensor, and the memory and configured to: receive first configuration information from a network entity, the first configuration information indicating how to obtain optical sensing data using the UE; obtain the optical sensing data using the sensor of the UE; and obtain angle information about wireless transmission between at least one wireless network node and the UE based at least on the optical sensing data.

[0010] This summary is neither intended to identify key features or essential features of the claimed subject matter nor is it intended to be used alone to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of the disclosure, any or all of the drawings, and each claim. The foregoing and other features and examples will be described in more detail in the following description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a diagram of a positioning system according to an embodiment.

[0012] Figure 2 is a diagram of a fifth generation (5G) new radio (NR) positioning system, which illustrates a positioning system (e.g., Figure 1 implementation scheme of a positioning system).

[0013] Figure 3 is a diagram showing an example of how beamforming may be performed according to some embodiments.

[0014] Figure 4 Depicted is a comparison of keypoints between two example images.

[0015] Figure 5 An example of the relative placement of the camera and a point X in the environment is shown.

[0016] Figure 6A simplified diagram of an example positioning scenario is illustrated, where a UE is in a wireless network and is configured to communicate with one or more base stations.

[0017] Figure 7 A simplified diagram illustrating another example positioning scenario, where there are physical obstructions between UEs.

[0018] Figure 8 A simplified diagram illustrating an example beam alignment scenario involving a UE and a base station is illustrated.

[0019] Fig. 9 is a call flow illustrating the exchange of signals between a location server, a base station, and a UE according to some embodiments.

[0020] Fig.10 is a call flow illustrating the exchange of signals between a base station and a UE according to some embodiments.

[0021] Fig.11 is a flow chart of a method of assisting positioning of a UE according to an embodiment.

[0022] Fig.12 is a flowchart of a method of performing beam alignment with a UE according to an implementation scheme.

[0023] Fig.13 is a block diagram of an implementation of a UE that may be utilized in implementations as described herein.

[0024] Fig.14 is a block diagram of an implementation of a base station that may be utilized in implementations as described herein.

[0025] Fig.15 is a block diagram of an embodiment of a computer system that may be utilized in the embodiments described herein.

[0026] Similar reference symbols in the various drawings indicate similar elements according to certain example implementations. In addition, multiple instances of an element may be indicated by adding a letter or hyphen and a second number after the first number of the element. For example, multiple instances of element 110 may be indicated as 110-1, 110-2, 110-3, etc. or 110a, 110b, 110c, etc. When only the first number is used to refer to such an element, any instance of the element should be understood (for example, element 110 in the previous example will refer to elements 110-1, 110-2 and 110-3 or to elements 110a, 110b and 110c). DETAILED DESCRIPTION

[0027] The following description is directed to certain specific implementations for the purpose of describing the innovative aspects of each embodiment. However, one of ordinary skill in the art will readily recognize that the teachings herein can be applied in a variety of different ways. The described specific implementations can be implemented in any device, system, or network capable of sending and receiving radio frequency (RF) signals according to any communication standard, such as any of the following: the Institute of Electrical and Electronics Engineers (IEEE) 802.15.4 standard for ultra-wideband (UWB), the IEEE 802.11 standard (including those identified as Technical standards), Standard, Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM), GSM / General Packet Radio Service (GPRS), Enhanced Data GSM Environment (EDGE), Terrestrial Trunked Radio (TETRA), Wideband CDMA (W-CDMA), Evolution-Data Optimized (EV-DO), 1xEV-DO, EV-DO Revision A, EV-DO Revision B, High Rate Packet Data (HRPD), High Speed ​​Packet Access (HSPA), High Speed ​​Downlink Packet Access (HSDPA), High Speed ​​Uplink Packet Access (HSUPA), Evolved High Speed ​​Packet Access (HSPA+), Long Term Evolution (LTE), Advanced Mobile Phone System (AMPS), or other known signals for communicating within a wireless, cellular or Internet of Things (IoT) network, such as a system utilizing 3G, 4G, 5G, 6G or further implementations thereof.

[0028] As used herein, "RF signals" include electromagnetic waves that transmit information through the space between a transmitter (or transmitting device) and a receiver (or receiving device). As used herein, a transmitter may transmit a single "RF signal" or multiple "RF signals" to a receiver. However, due to the propagation characteristics of each RF signal through multiple channels or paths, a receiver may receive multiple "RF signals" corresponding to each transmitted RF signal.

[0029] Additionally, unless otherwise indicated, references to "reference signals," "positioning reference signals," "reference signals used for positioning," etc. may be used to refer to signals used to locate user equipment (UE). As described in more detail herein, such signals may include any of a variety of signal types, but may not necessarily be limited to positioning reference signals (PRS) as defined in relevant wireless standards.

[0030] In addition, unless otherwise specified, the term "positioning" as used herein may refer to absolute position determination, relative position determination, ranging, or a combination thereof. For the purpose of location or sensing services, such positioning may include and / or be based on timing, angle, phase or power measurements or a combination thereof (which may include RF sensing measurements).

[0031] The following describes positioning of wireless network devices or determining the best beam between wireless network devices based on the type of visual odometry.

[0032] Inferring the trajectory of an autonomous vehicle may involve comparing key points detected over consecutive frames and finding correspondences between the key points. A key point is a spatial location or point of interest in an image. In some cases, a key point may be a pixel in an image that can be tracked frame by frame, such as a corner point of a feature of a building. A key point may also have an associated descriptor that aids the tracking process. Key points and their descriptors are collectively referred to as features of an image. Key points and features may help find changes in the pose or orientation of a sensor (e.g., a camera) between consecutive image frames. For example, a two-dimensional displacement of five or more immovable key points or features detected and tracked across two camera images is sufficient to restore a three-dimensional displacement of the sensor.

[0033] When performing ranging or positioning between two wireless network devices (e.g., between a base station and a UE), it is valuable to know how the wireless network devices are placed relative to each other. In some implementations, such knowledge (e.g., by a wireless network entity) will enable a receiver (e.g., a UE or a base station) to have information for performing ranging or positioning, such as information about angle information, including, for example, information about the angle of arrival (AoA) that the receiver can expect to receive from a transmitter (e.g., a base station or a UE).

[0034] To this end, it is helpful to determine the unit vectors connecting wireless network devices (describing the distance and / or direction between them) so that even in non-line-of-sight (NLOS) environments (e.g., where there are physical obstructions between wireless network devices), angular orientation and other angular information (e.g., expected AoA, azimuth, elevation) can be inferred. It is further helpful to exchange signals between wireless network devices to enable acquisition of this information and performance of enhanced positioning. As will be described in more detail, in one example scenario, the location of an autonomous vehicle (via a UE co-located with the autonomous vehicle) can be determined even using an NLOS signal path with a base station.

[0035] Additional details will follow this initial description of related systems and techniques.

[0036] Figure 1is a simplified illustration of a positioning system 100 according to an embodiment, in which a UE 105, a location server 160, and / or other components of the positioning system 100 may use the techniques provided herein for assisting in positioning or beam alignment with the UE 105. The techniques described herein may be implemented by one or more components of the positioning system 100. The positioning system 100 may include: a UE 105; one or more satellites 110 (also referred to as space vehicles (SVs)) for a global navigation satellite system (GNSS) such as a global positioning system (GPS), GLONASS, Galileo, or BeiDou; a base station 120; an access point (AP) 130; a location server 160; a network 170; and an external client 180. In general, the positioning system 100 may estimate the location of the UE 105 based on RF signals received by and / or transmitted from the UE 105 and the known locations of other components (e.g., GNSS satellites 110, base stations 120, APs 130) that send and / or receive RF signals. Reference Figure 2 Additional details regarding specific location estimation techniques are discussed in more detail.

[0037] It should be pointed out that Figure 1 Only a generalized illustration of various components is provided, any or all of which may be utilized as appropriate, and each component may be repeated as needed. Specifically, although only one UE 105 is illustrated, it should be understood that many UEs (e.g., hundreds, thousands, millions, etc.) may utilize the positioning system 100. Similarly, the positioning system 100 may include more than one UE 105. Figure 1 A greater or lesser number of base stations 120 and / or APs 130 are illustrated. The illustrated connections connecting the various components in the positioning system 100 include data and signaling connections, which may include additional (intermediate) components, direct or indirect physical and / or wireless connections, and / or additional networks. In addition, components may be rearranged, combined, separated, replaced, and / or omitted depending on the desired functionality. In some embodiments, for example, the external client 180 may be directly connected to the location server 160. Those of ordinary skill in the art will recognize many modifications to the illustrated components.

[0038] Depending on the desired functionality, network 170 may include any one of a variety of wireless and / or wired networks. Network 170 may, for example, include any combination of public and / or private networks, local area networks and / or wide area networks, etc. In addition, network 170 may utilize one or more wired and / or wireless communication technologies. In some embodiments, network 170 may include, for example, cellular or other mobile networks, wireless local area networks (WLANs), wireless wide area networks (WWANs), and / or the Internet. Examples of network 170 include long-term evolution (LTE) wireless networks, fifth-generation (5G) wireless networks (also referred to as new radio (NR) wireless networks or 5G NR wireless networks), Wi-Fi WLANs, and the Internet. LTE, 5G, and NR are wireless technologies defined or being defined by the 3rd Generation Partnership Project (3GPP). Network 170 may also include more than one network and / or more than one type of network.

[0039] The base station 120 and the access point (AP) 130 are communicatively coupled to the network 170. In some embodiments, the base station 120 may be owned, maintained and / or operated by a cellular network provider and may employ any of a variety of wireless technologies, as described herein below. Depending on the technology of the network 170, the base station 120 may include a node B, an evolved node B (eNodeB or eNB), a transceiver base station (BTS), a radio base station (RBS), an NR node B (gNB), a next generation eNB (ng-eNB), etc. In the case where the network 170 is a 5G network, the base station 120 as a gNB or ng-eNB may be part of a next generation radio access network (NG-RAN) that may be connected to a 5G core network (5GC). In view of the open radio access network (O-RAN) and / or virtualized radio access network (V-RAN or vRAN) in 5G or higher networks, the functions performed by the base station 120 in the earlier networks (e.g., 3G and 4G) may be divided into different functional components (e.g., radio unit (RU), distributed unit (DU), central unit (CU)) and layers (e.g., L1 / L2 / L3), which may be performed on different devices at different locations connected, for example, via fronthaul connection, midhaul connection, and backhaul connection. As described herein, a "base station" (or ng-eNB, gNB, etc.) may include any or all of these functional components. For example, AP 130 may include a Wi-Fi AP or AP or an AP with cellular capabilities (e.g., 4G LTE and / or 5GNR). Thus, UE 105 can transmit and receive information with network-connected devices such as location server 160 by accessing network 170 via base station 120 using first communication link 133. Additionally or alternatively, because AP 130 can also be communicatively coupled with network 170, UE 105 can communicate with network-connected and Internet-connected devices (including location server 160) using second communication link 135 or via one or more other mobile devices 145.

[0040] As used herein, the term "base station" may generally refer to a single physical transmission point or multiple co-located physical transmission points that may be located at a base station 120. A transmit receive point (TRP) (also referred to as a transmit / receive point) corresponds to this type of transmission point, and the term "TRP" may be used interchangeably with the terms "gNB," "ng-eNB," and "base station" herein. In some cases, a base station 120 may include multiple TRPs—for example, where each TRP is associated with a different antenna or a different antenna array of the base station 120. As used herein, the transmit function of a TRP may be performed using a transmit point (TP), and / or the receive function of a TRP may be performed by a receive point (RP), which may be physically separate or different from the TP. That is, a TRP may include both a TP and an RP. A physical transmission point may include an antenna array of a base station 120 (for example, as in a multiple-input multiple-output (MIMO) system and / or where the base station employs beamforming). The term "base station" may additionally refer to multiple non-co-located physical transmission points, which may be a distributed antenna system (DAS) (a network of spatially separated antennas connected to a common source via a transmission medium) or a remote radio head (RRH) (a remote base station connected to a serving base station).

[0041] As used herein, the term "cell" may generally refer to a logical communication entity for communicating with base station 120, and may be associated with an identifier (e.g., a physical cell identifier (PCID), a virtual cell identifier (VCID)) for distinguishing adjacent cells operating via the same or different carriers. In some examples, a carrier may support multiple cells, and different cells may be configured according to different protocol types that may provide access for different types of devices (e.g., machine type communication (MTC), narrowband Internet of Things (NB-IoT), enhanced mobile broadband (eMBB), or other protocols). In some cases, the term "cell" may refer to a portion of a geographic coverage area (e.g., a sector) on which the logical entity operates.

[0042] Satellites 110 may be used to locate UE 105 in one or more ways. For example, satellites 110 (also referred to as space vehicles (SVs)) may be part of a global navigation satellite system (GNSS), such as a global positioning system (GPS), GLONASS, Galileo, or BeiDou. Positioning using RF signals from GNSS satellites may include measuring multiple GNSS signals at a GNSS receiver of UE 105 to perform code-based and / or carrier-based positioning, which may be highly accurate. Additionally or alternatively, satellites 110 may be used for non-terrestrial network (NTN) based positioning, where satellites 110 may be functionally operable as a TRP (or multiple TPs) of a network (e.g., an LTE and / or NR network) and may be communicatively coupled with network 170. In particular, reference signals (e.g., PRS) sent by satellites 110 for NTN based positioning may be similar to those sent by base stations 120 and may be coordinated by location server 160. In some embodiments, satellites 110 used for NTN based positioning may be different from those used for GNSS based positioning.

[0043] The location server 160 may include a server and / or other computing device configured to determine an estimated location of the UE 105 and / or provide data (e.g., “assistance data”) to the UE 105 to facilitate location measurement and / or location determination by the UE 105. According to some embodiments, the location server 160 may include a Home Secure User Plane Location (SUPL) Location Platform (H-SLP), which may support the SUPL User Plane (UP) location solution defined by the Open Mobile Alliance (OMA) and may support location services for the UE 105 based on subscription information of the UE 105 stored in the location server 160. In some embodiments, the location server 160 may include a Discovery SLP (D-SLP) or an Emergency SLP (E-SLP). The location server 160 may also include an enhanced serving mobile location center (E-SMLC) that supports positioning of the UE 105 using a control plane (CP) positioning solution for LTE radio access of the UE 105. The location server 160 may also include a location management function (LMF) that supports positioning of the UE 105 using a control plane (CP) positioning solution for NR or LTE radio access of the UE 105.

[0044] In the CP positioning solution, from the perspective of the network 170, signaling for controlling and managing the positioning of the UE 105 may use existing network interfaces and protocols and be exchanged as signaling between elements of the network 170 and with the UE 105. In the UP positioning solution, from the perspective of the network 170, signaling for controlling and managing the positioning of the UE 105 may be exchanged between the location server 160 and the UE 105 as data (e.g., data sent using the Internet Protocol (IP) and / or the Transmission Control Protocol (TCP)).

[0045] As previously noted (and discussed in more detail below), the estimated position of UE 105 may be based on measurements of RF signals transmitted from and / or received by UE 105. In particular, these measurements may provide information about the relative distances and / or angles of UE 105 from one or more components in positioning system 100 (e.g., GNSS satellites 110, APs 130, base stations 120). The estimated position of UE 105 may be estimated geometrically (e.g., using multi-angle measurements and / or multilateration) based on the distance and / or angle measurements along with the known positions of the one or more components.

[0046] Although ground components such as AP 130 and base station 120 may be fixed, embodiments are not limited thereto. Mobile components may be used. For example, in some embodiments, the location of UE 105 may be estimated based at least in part on measurements of RF signals 140 communicated between UE 105 and one or more other mobile devices 145 (the one or more other mobile devices may be mobile or fixed). As illustrated, the other mobile devices may include, for example, mobile phone 145-1, vehicle 145-2, static communication / positioning device 145-3, or other static and / or mobile devices capable of providing wireless signals for locating UE 105, or a combination thereof. Wireless signals from mobile device 145 for positioning of UE 105 may include using, for example, (including Bluetooth Low Energy (BLE)), IEEE 802.11x (e.g. ), ultra-wideband (UWB), IEEE802.15x, or a combination thereof. The mobile device 145 may additionally or alternatively use non-RF wireless signals such as infrared signals or other optical technologies to locate the UE 105.

[0047] The mobile device 145 may include other UEs that are communicatively coupled to a cellular network or other mobile network (e.g., network 170). When one or more other mobile devices 145 including a UE are used in the positioning determination of a particular UE 105, the UE 105 whose positioning is to be determined may be referred to as a "target UE", and each of the other mobile devices 145 used may be referred to as an "anchor UE". In order to perform positioning determination on the target UE, the corresponding positioning of the one or more anchor UEs may be known and / or determined jointly with the target UE. Direct communication between one or more other mobile devices 145 and UE 105 may include side links and / or similar device-to-device (D2D) communication technologies. The side link defined by 3GPP is a form of D2D communication under cellular-based LTE and NR standards. UWB may be a technology by which measurements from one or more anchor devices (e.g., mobile devices 145) may be used to facilitate the positioning of a target device (e.g., UE 105).

[0048] According to some embodiments, such as when the UE 105 includes and / or is incorporated into a vehicle, a form of D2D communication used by the mobile device 105 may include vehicle-to-everything (V2X) communication. V2X is a communication standard for vehicles to exchange information about the traffic environment with related entities. V2X may include vehicle-to-vehicle (V2V) communication between vehicles with V2X capabilities, vehicle-to-infrastructure (V2I) communication between vehicles and infrastructure-based equipment (commonly referred to as roadside units (RSUs)), vehicle-to-person (V2P) communication between vehicles and nearby people (pedestrians, cyclists, and other road users), etc. In addition, V2X may use any of a variety of wireless RF communication technologies. For example, cellular V2X (CV2X) is a form of V2X that uses cellular-based communications in a direct communication mode defined by 3GPP, such as LTE (4G), NR (5G), and / or other cellular technologies. Figure 1 The illustrated UE 105 may correspond to a component or device located on a vehicle, RSU, or other V2X entity for communicating V2X messages. In an embodiment in which V2X is used, the static communication / positioning device 145-3 (which may correspond to an RSU) and / or the vehicle 145-2 may therefore communicate with the UE 105 and may be used to determine the location of the UE 105 using techniques similar to those used by the base station 120 and / or AP 130 (e.g., using multi-angle measurements and / or multi-point positioning). It may be further noted that, according to some embodiments, the mobile device 145 (which may include a V2X device), the base station 120, and / or the AP 130 may be used together (e.g., in a WWAN positioning solution) to determine the location of the UE 105.

[0049] The estimated position of UE 105 may be used in a variety of applications, such as to assist a user of UE 105 in direction finding or navigation or to assist another user (e.g., associated with external client 180) in locating UE 105. "Position" is also referred to herein as "position estimate," "estimated position," "position," "position estimate," "position fix," "estimated position," "position fix," or "fix." The process of determining a position may be referred to as "positioning," "position determination," "position determination," or the like. The position of UE 105 may include an absolute position of UE 105 (e.g., latitude and longitude, and possibly altitude) or a relative position of UE 105 (e.g., a position expressed as a distance north or south, east or west, and possibly above or below some other known fixed position (including, for example, the position of base station 120 or AP 130) or some other location (such as the position of UE 105 at some known previous time, or the position of mobile device 145 (e.g., another UE) at some known previous time). The location may be specified as a geodetic location including coordinates, which may be absolute (e.g., latitude, longitude, and optionally altitude), relative (e.g., relative to some known absolute location), or local (e.g., X, Y, and optionally Z coordinates according to a coordinate system defined relative to a local area, such as a factory, warehouse, university campus, shopping mall, stadium, or convention center). The location may alternatively be a city location, and may then include one or more of a street address (e.g., including a country, state, county, city, road and / or street name or label, and / or road or street number) and / or a label or name of a place, a building, a portion of a building, a floor of a building, and / or a room within a building, etc. The location may also include uncertainty or error indications, such as horizontal distances and possibly vertical distances over which the location is expected to have error, or an indication of an area or volume (e.g., a circle or ellipse) within which the UE 105 is expected to be located with a certain confidence level (e.g., 95% confidence).

[0050] The external client 180 may be a web server or remote application that may have some association with the UE 105 (e.g., accessible by a user of the UE 105), or may be a server, application, or computer system that provides location services to one or more other users, which may include obtaining and providing the location of the UE 105 (e.g., to enable services such as friend or relative locating or child or pet locating). Additionally or alternatively, the external client 180 may obtain the location of the UE 105 and provide it to emergency service providers, government agencies, etc.

[0051] As previously noted, the example positioning system 100 may be implemented using a wireless communication network such as an LTE-based or 5GNR-based network. Figure 2A diagram of a 5G NR positioning system 200 is shown, which illustrates an embodiment of a positioning system (e.g., positioning system 100) that implements 5G NR. The 5G NR positioning system 200 can be configured to determine the location of the UE 105 by using access nodes to implement one or more positioning methods. The access nodes can include NR Node Bs (gNBs) 210-1 and 210-2 (collectively referred to as gNBs 210 herein), ng-eNBs 214, and / or WLANs 216. The gNBs 210 and / or ng-eNBs 214 can communicate with the UE 105 to determine the location of the UE 105. Figure 1 The base station 120 corresponds to the WLAN 216, and the WLAN 216 can be connected to the Figure 1 130. Optionally, the 5G NR positioning system 200 may be additionally configured to determine the location of the UE 105 by using the LMF 220 (which may correspond to the location server 160) to implement one or more positioning methods. Here, the 5G NR positioning system 200 includes the UE 105, and components of a 5G NR network, which includes a next generation (NG) radio access network (RAN) (NG-RAN) 235 and a 5G core network (5G CN) 240. The 5G network may also be referred to as an NR network; the NG-RAN 235 may be referred to as a 5G RAN or an NR RAN; and the 5G CN 240 may be referred to as an NG core network.

[0052] The 5G NR positioning system 200 may also utilize information from satellites 110. As previously indicated, satellites 110 may include GNSS satellites from a GNSS system like the Global Positioning System (GPS) or similar systems (e.g., GLONASS, Galileo, BeiDou, Indian Regional Navigation Satellite System (IRNSS)). Additionally or alternatively, satellites 110 may include NTN satellites that may be communicatively coupled with LMF 220 and operatively used as TRPs (or TPs) in NG-RAN 235. Thus, satellites 110 may communicate with one or more gNBs 210.

[0053] It should be noted that Figure 2Only generalized illustrations of various components are provided, any or all of which may be utilized as appropriate, and each of which may be repeated or omitted as desired. Specifically, although only one UE 105 is illustrated, it should be understood that many UEs (e.g., hundreds, thousands, millions, etc.) may utilize the 5G NR positioning system 200. Similarly, the 5G NR positioning system 200 may include a larger (or smaller) number of satellites 110, gNB 210, ng-eNB 214, wireless local area network (WLAN) 216, access and mobility management function (AMF) 215, external client 230, and / or other components. The illustrated connections connecting the various components in the 5G NR positioning system 200 include data and signaling connections, which may include additional (intermediate) components, direct or indirect physical and / or wireless connections, and / or additional networks. In addition, components may be rearranged, combined, separated, replaced, and / or omitted, depending on the desired functionality.

[0054] UE 105 may include and / or be referred to as a device, a mobile device, a wireless device, a mobile terminal, a terminal, a mobile station (MS), a terminal (SET) enabled with secure user plane location (SUPL), or some other name. In addition, UE 105 may correspond to a cellular phone, a smart phone, a laptop computer, a tablet computer, a personal data assistant (PDA), a navigation device, an Internet of Things (IoT) device, or some other portable or mobile device. Typically, although not necessarily, UE 105 may support the use of one or more radio access technologies (RATs) such as GSM, CDMA, W-CDMA, LTE, High Rate Packet Data (HRPD), IEEE 802.11 Bluetooth, Worldwide Interoperability for Microwave Access (WiMAX TM ), 5G NR (e.g., using NG-RAN 235 and 5G CN 240). UE 105 may also support wireless communications using WLAN 216, which is similar to one or more RATs and as previously described with respect to Figure 1 The use of one or more of these RATs may allow UE 105 (e.g., via Figure 2 Elements of the 5G CN 240, not shown, may communicate with external clients 230, or possibly via a Gateway Mobile Location Center (GMLC) 225) and / or allow external clients 230 (e.g., via GMLC 225) to receive location information about UE 105. When implemented in or communicatively coupled with a 5G NR network, Figure 2 The external client 230 may correspond to Figure 1 External client 180.

[0055] UE 105 may include a single entity or may include multiple entities, such as in a personal area network in which a user may employ audio, video and / or data I / O devices and / or body sensors and separate wired or wireless modems. The estimate of the location of UE 105 may be referred to as location, location estimate, location fix, fix, position, position estimate or position fix, and may be geodetic, providing location coordinates (e.g., latitude and longitude) for UE 105, which may or may not include an altitude component (e.g., height above sea level; height above or depth below ground level, floor level or basement level). Alternatively, the location of UE 105 may be expressed as a municipal location (e.g., a postal address or a designation of a point or smaller area in a building, such as a specific room or floor). The location of UE 105 may also be expressed as a region or volume (geographically or municipally defined) within which UE 105 is expected to be located with a certain probability or confidence level (e.g., 67%, 95%, etc.). The location of the UE 105 may also be a relative location, including, for example, a distance and direction or relative X, Y (and Z) coordinates defined relative to some origin at a known location, which may be defined geodetically, in municipal form, or with reference to a point, area, or volume indicated on a map, floor plan, or building plan. In the description contained herein, the use of the term location may include any of these variations unless otherwise indicated. When calculating the location of the UE, local X, Y, and possibly Z coordinates are typically solved, and then the local coordinates are converted to absolute coordinates (e.g., with respect to latitude, longitude, and altitude above or below mean sea level) if necessary.

[0056] Figure 2 The base stations in the NG-RAN 235 shown may correspond to Figure 1 The base stations 120 in the NG-RAN 235 may include gNBs 210. Pairs of gNBs 210 in the NG-RAN 235 may be connected to each other (e.g., Figure 2 The communication interface between the base stations (gNB 210 and / or ng-eNB 214) may be referred to as an Xn interface 237. Access to the 5G network is provided to the UE 105 via wireless communication between the UE 105 and one or more gNBs 210, which may provide wireless communication access to the 5G CN 240 on behalf of the UE 105 using 5G NR. The wireless interface between the base station (gNB 210 and / or ng-eNB 214) and the UE 105 may be referred to as an Uu interface 239. 5G NR radio access may also be referred to as NR radio access or 5G radio access. Figure 2, it is assumed that the serving gNB for UE 105 is gNB 210-1, but other gNBs (e.g., gNB 210-2) may act as serving gNBs if UE 105 moves to another location, or may act as secondary gNBs to provide additional throughput and bandwidth to UE 105.

[0057] Figure 2 The base stations in the illustrated NG-RAN 235 may additionally or alternatively include a next generation evolved Node B (also referred to as ng-eNB) 214. The ng-eNB 214 may be connected to one or more gNBs 210 in the NG-RAN 235—e.g., directly, or indirectly via other gNBs 210 and / or other ng-eNBs. The ng-eNB 214 may provide LTE radio access and / or evolved LTE (eLTE) radio access to the UE 105. Figure 2 Some of the gNBs 210 (e.g., gNB 210-2) and / or ng-eNBs 214 may be configured to function as positioning-only beacons that may transmit signals (e.g., positioning reference signals (PRS)) and / or may broadcast assistance data to assist in positioning UE 105, but may not receive signals from UE 105 or other UEs. Some of the gNBs 210 (e.g., gNB 210-2 and / or another gNB not shown) and / or ng-eNBs 214 may be configured to function as detection-only nodes that may scan for signals containing, for example, PRS data, assistance data, or other location data. Such detection-only nodes may not transmit signals or data to the UE, but may transmit signals or data (related to, for example, PRS, assistance data, or other location data) to other network entities (e.g., one or more components of 5G CN 240, external clients 230, or a controller), which may receive and store the data or use the data to locate at least UE 105. It should be noted that although Figure 2 Only one ng-eNB 214 is shown in FIG, but some embodiments may include multiple ng-eNBs 214. Base stations (e.g., gNB 210 and / or ng-eNB 214) may communicate directly with each other via an Xn communication interface. Additionally or alternatively, the base station may communicate directly or indirectly with other components of the 5G NR positioning system 200 (such as LMF 220 and AMF 215).

[0058] The 5G NR positioning system 200 may also include one or more WLANs 216 that may be connected to a non-3GPP interworking function (N3IWF) 250 in the 5GCN 240 (e.g., in the case of an untrusted WLAN 216). For example, the WLAN 216 may support IEEE 802.11 Wi-Fi access for the UE 105 and may include one or more Wi-Fi APs (e.g., Figure 1 Here, the N3IWF 250 may be connected to other elements in the 5G CN 240, such as the AMF 215. In some embodiments, the WLAN 216 may support another RAT, such as Bluetooth. The N3IWF 250 may provide support for secure access of the UE 105 to other elements in the 5G CN 240 and / or may support interworking of one or more protocols used by the WLAN 216 and the UE 105 with one or more protocols used by other elements of the 5G CN 240 (such as the AMF 215). For example, the N3IWF 250 may support: IPSec tunnel establishment with the UE 105, termination of the IKEv2 / IPSec protocol with the UE 105, termination of the N2 and N3 interfaces with the 5G CN 240 for the control plane and the user plane, respectively, and relay of uplink (UL) and downlink (DL) control plane non-access stratum (NAS) signaling across the N1 interface between the UE 105 and the AMF 215. In some other embodiments, the WLAN 216 may be directly connected to elements in the 5G CN 240 (e.g., Figure 2 215) and not via the N3IWF 250. For example, a direct connection of the WLAN 216 to the 5GCN 240 may occur when the WLAN 216 is a trusted WLAN for the 5GCN 240 and may use a trusted WLAN interworking function (TWIF) ( Figure 2 It should be noted that although Figure 2 Only one WLAN 216 is shown in FIG. 2 , but some embodiments may include multiple WLANs 216 .

[0059] The access node may include any of a variety of network entities that enable communication between the UE 105 and the AMF 215. As noted, this may include a gNB 210, an ng-eNB 214, a WLAN 216, and / or other types of cellular base stations. However, an access node that provides the functionality described herein may additionally or alternatively include a cellular network entity that enables communication with the UE 105. Figure 2Thus, as used in the embodiments described below, the term “access node” may include, but is not necessarily limited to, gNB 210, ng-eNB 214, or WLAN 216.

[0060] In some embodiments, an access node (such as gNB 210, ng-eNB 214, and / or WLAN 216) (alone or in combination with other components of 5G NR positioning system 200) may be configured to: in response to receiving a request for location information from LMF 220, obtain a location measurement for an uplink (UL) signal received from UE 105 and / or obtain a DL location measurement from UE 105 for a downlink (DL) signal received by UE 105 from one or more access nodes. As noted, although Figure 2 The access nodes (gNB 210, ng-eNB 214, and WLAN 216) are depicted as being configured to communicate in accordance with 5G NR, LTE, and Wi-Fi communication protocols, respectively, but access nodes configured to communicate in accordance with other communication protocols may be used, such as, for example, a Node B using a Wideband Code Division Multiple Access (WCDMA) protocol for a Universal Mobile Telecommunications Service (UMTS) Terrestrial Radio Access Network (UTRAN), an eNB using an LTE protocol for an Evolved UTRAN (E-UTRAN), or an eNB using a LTE protocol for a WLAN. For example, in a 4G Evolved Packet System (EPS) that provides LTE radio access to UE 105, the RAN may include E-UTRAN, which may include base stations including eNBs that support LTE radio access. The core network for EPS may include an Evolved Packet Core (EPC). EPS may then include E-UTRAN plus EPC, where Figure 2 , E-UTRAN corresponds to NG-RAN 235 and EPC corresponds to 5GCN 240. The methods and techniques described herein for obtaining the municipal location of UE 105 may be applicable to such other networks.

[0061] The gNB 210 and ng-eNB 214 may communicate with the AMF 215, which communicates with the LMF 220 for positioning functions. The AMF 215 may support the mobility of the UE 105, including cell changes and handovers of the UE 105 from an access node of a first RAT (e.g., a gNB 210, ng-eNB 214, or WLAN 216) to an access node of a second RAT. The AMF 215 may also be involved in supporting signaling connections with the UE 105 and possibly supporting data and voice bearers for the UE 105. The LMF 220 may support positioning the UE 105 using a CP positioning solution when the UE 105 accesses the NG-RAN 235 or the WLAN 216, and may support positioning procedures and methods, including UE-assisted / UE-based and / or network-based procedures / methods, such as Assisted GNSS (A-GNSS), Observed Time Difference of Arrival (OTDOA) (which may be referred to as Time Difference of Arrival (TDOA) in NR), Frequency Difference of Arrival (FDOA), Real-Time Kinematics (RTK), Precise Point Positioning (PPP), Differential GNSS (DGNSS), Enhanced Cell ID (ECID), Angle of Arrival (AoA), Angle of Departure (AoD), WLAN positioning, Round Trip Signal Propagation Delay (RTT), Multi-Cell RTT, and / or other positioning procedures and methods. The LMF 220 may also process a location service request for the UE 105 received, for example, from the AMF 215 or from the GMLC 225. The LMF 220 may be connected to the AMF 215 and / or the GMLC 225. In some embodiments, the network (such as 5GCN 240) may additionally or alternatively implement other types of location support modules, such as an evolved serving mobile location center (E-SMLC) or a SUPL location platform (SLP). It should be noted that in some embodiments, at least a portion of the positioning functions (including determining the location of UE 105) may be performed at UE 105 (e.g., by measuring downlink PRS (DL-PRS) signals sent by wireless nodes (such as gNB 210, ng-eNB 214 and / or WLAN 216), and / or using assistance data provided to UE 105 by, for example, LMF 220).

[0062] The Gateway Mobile Location Center (GMLC) 225 may support location requests for the UE 105 received from the external client 230, and may forward such location requests to the AMF 215 for forwarding by the AMF 215 to the LMF 220. A location response (e.g., including a location estimate for the UE 105) from the LMF 220 may similarly be returned to the GMLC 225 directly or via the AMF 215, and the GMLC 225 may then return the location response (e.g., including the location estimate) to the external client 230.

[0063] A network exposure function (NEF) 245 may be included in the 5GCN 240. The NEF 245 may support secure exposure of capabilities and events regarding the 5GCN 240 and the UE 105 to the external client 230, which may therefore be referred to as an access function (AF) and may enable secure provisioning of information from the external client 230 to the 5GCN 240. The NEF 245 may be connected to the AMF 215 and / or the GMLC 225 for obtaining the location (e.g., municipal location) of the UE 105 and providing the location to the external client 230.

[0064] like Figure 2 As further illustrated, LMF 220 may communicate with gNB 210 and / or with ng-eNB 214 using NR Positioning Protocol Annex (NRPPa) as defined in 3GPP Technical Specification (TS) 38.455. NRPPa messages may be communicated between gNB 210 and LMF 220 and / or between ng-eNB 214 and LMF 220 via AMF 215. Figure 2 As further illustrated, the LMF 220 and the UE 105 may communicate using the LTE Positioning Protocol (LPP) as defined in 3GPP TS 37.355. Here, LPP messages may be passed between the UE 105 and the LMF 220 via the AMF 215 and the serving gNB 210-1 or the serving ng-eNB 214 of the UE 105. For example, the LPP messages may be passed between the LMF 220 and the AMF 215 using messages for service-based operations (e.g., based on the Hypertext Transfer Protocol (HTTP)), and may be passed between the AMF 215 and the UE 105 using the 5G NAS protocol. The LPP protocol may be used to support positioning of the UE 105 using UE-assisted and / or UE-based positioning methods such as A-GNSS, RTK, TDOA, multi-cell RTT, AoD, and / or ECID. The NRPPa protocol may be used to support positioning of UE 105 using network-based positioning methods such as ECID, AoA, uplink TDOA (UL-TDOA), and / or may be used by LMF 220 to obtain location-related information from gNB 210 and / or ng-eNB 214, such as parameters defining DL-PRS transmissions from gNB 210 and / or ng-eNB 214.

[0065] In the case where the UE 105 accesses the WLAN 216, the LMF 220 may use NRPPa and / or LPP to obtain the location of the UE 105 in a manner similar to that just described for the UE 105 accessing the gNB 210 or ng-eNB 214. Thus, NRPPa messages may be passed between the WLAN 216 and the LMF 220 via the AMF 215 and the N3IWF 250 to support network-based positioning of the UE 105 and / or to pass other location information from the WLAN 216 to the LMF 220. Alternatively, NRPPa messages may be passed between the N3IWF 250 and the LMF 220 via the AMF 215 to support network-based positioning of the UE 105 based on location-related information and / or location measurements that are known or accessible to the N3IWF 250 and passed from the N3IWF 250 to the LMF 220 using NRPPa. Similarly, LPP and / or LPP messages may be communicated between the UE 105 and the LMF 220 via the AMF 215, the N3IWF 250, and the serving WLAN 216 of the UE 105 to support UE-assisted or UE-based positioning of the UE 105 by the LMF 220.

[0066] In the 5G NR positioning system 200, positioning methods may be classified as "UE-assisted" or "UE-based". This may depend on where the request to determine the location of the UE 105 originates. For example, where the request originates from the UE (e.g., from an application or "app" executed by the UE), the positioning method may be classified as UE-based. On the other hand, where the request originates from an external client 230, LMF 220, or other device or service within the 5G network, the positioning method may be classified as UE-assisted (or "network-based").

[0067] With UE-assisted positioning methods, the UE 105 may obtain location measurements and transmit these measurements to a location server (e.g., LMF 220) for use in computing a location estimate for the UE 105. For RAT-dependent positioning methods, the location measurements may include one or more of the following for one or more access points of the gNB 210, ng-eNB 214, and / or WLAN 216: received signal strength indicator (RSSI), round trip signal propagation time (RTT), reference signal received power (RSRP), reference signal received quality (RSRQ), reference signal time difference (RSTD), time of arrival (TOA), AoA, receive time-transmit time difference (Rx-Tx), differential AoA (DAoA), AoD, or timing advance (TA). Additionally or alternatively, similar measurements may be made on sidelink signals sent by other UEs, which may be used as anchor points for positioning the UE 105 if the locations of these other UEs are known. The position measurements may additionally or alternatively include measurements for a RAT-independent positioning method, such as GNSS (eg, GNSS pseudoranges with respect to satellites 110 , GNSS code phase, and / or GNSS carrier phase), WLAN, or the like.

[0068] Using the UE-based positioning method, the UE 105 can obtain a position measurement (e.g., which can be the same or similar to the position measurement of the UE-assisted positioning method), and the position of the UE 105 can be further calculated (e.g., with the aid of auxiliary data received from a location server such as LMF 220, SLP or broadcast by gNB 210, ng-eNB 214, or WLAN 216).

[0069] Using a network-based positioning method, one or more base stations (e.g., gNB 210 and / or ng-eNB 214), one or more APs (e.g., in WLAN 216), or N3IWF 250 may obtain location measurements (e.g., measurements of RSSI, RTT, RSRP, RSRQ, AoA, or TOA) of signals sent by UE 105, and / or may receive measurements obtained by UE 105 or, in the case of N3IWF 250, by APs in WLAN 216, and may transmit these measurements to a location server (e.g., LMF 220) for use in calculating a position estimate for UE 105.

[0070] The positioning of UE 105 may also be classified as UL-based, DL-based, or DL-UL-based according to the type of signal used for positioning. For example, in the case where positioning is based only on signals received at UE 105 (e.g., from a base station or other UE), the positioning may be classified as DL-based. On the other hand, in the case where positioning is based only on signals sent by UE 105 (which may be received by, for example, a base station or other UE), the positioning may be classified as UL-based. DL-UL-based positioning includes positioning based on signals sent and received by UE 105, such as RTT-based positioning. Sidelink (SL)-assisted positioning includes signals communicated between UE 105 and one or more other UEs. According to some embodiments, UL, DL, or DL-UL positioning as described herein may be able to use SL signaling as a supplement or replacement for SL, DL, or DL-UL signaling.

[0071] Depending on the type of positioning (e.g., UL-based, DL-based, or DL-UL-based), the type of reference signal used may be different. For example, for DL-based positioning, these signals may include PRS (e.g., DL-PRS sent by a base station or SL-PRS sent by other UEs), which may be used for TDOA measurements, AoD measurements, and RTT measurements. Other reference signals that may be used for positioning (UL, DL, or DL-UL) may include: Sounding Reference Signal (SRS), Channel State Information Reference Signal (CSI-RS), Synchronization Signal (e.g., Synchronization Signal Block (SSB) Synchronization Signal (SS)), Physical Uplink Control Channel (PUCCH), Physical Uplink Shared Channel (PUSCH), Physical Sidelink Shared Channel (PSSCH), Demodulation Reference Signal (DMRS), etc. In addition, the reference signal may be sent in a Tx beam and / or received in an Rx beam (e.g., using beamforming techniques), which may affect angle measurements such as AoD and / or AoA.

[0072] Figure 3 The example includes two base stations 320-1 and 320-2 (which may correspond to Figure 1 Base station 120 and / or Figure 2 2 and / or ng-eNB 214), these antenna arrays can perform beamforming to produce directional beams for transmitting and / or receiving RF signals. Figure 3 Also illustrated is a UE 105 that can also use beamforming to send and / or receive RF signals. Such directional beams are used in 5G NR wireless communication networks. Each directional beam can have a beam width centered in a different direction, so that different beams of the base station 320 can correspond to different areas within the coverage area of ​​the base station 320.

[0073] Different operating modes may enable base stations 320-1 and 320-2 to use a greater or lesser number of beams. For example, in a first operating mode, base station 320 may use 16 beams, in which case each beam may have a relatively wide beam width. In a second operating mode, base station 320 may use 64 beams, in which case each beam may have a relatively narrow beam width. Depending on the capabilities of base station 320, the base station may use any number of beams that base station 320 is capable of forming. The operating mode and / or number of beams may be defined in the relevant wireless standard and may correspond to different directions (e.g., horizontal and vertical) in either or both of the azimuth and elevation angles. Different operating modes may be used to send and / or receive different signal types. Additionally or alternatively, UE 105 may be able to use different numbers of beams, which may also correspond to different operating modes, signal types, etc.

[0074] In some cases, the base station 320 may use beam scanning. Beam scanning is a process by which the base station 320 may transmit RF signals in different directions, typically continuously, using different corresponding beams, effectively "scanning" across a coverage area. For example, the base station 320 may scan 120 degrees or 360 degrees in the azimuth direction, which may be repeated periodically for each beam scan. Each directional beam may include an RF reference signal (e.g., a PRS resource), wherein the base station 320-1 generates a set of RF reference signals including Tx beams 305-a, 305-b, 305-c, 305-d, 305-e, 305-f, 305-g, and 305-h, and the base station 320-2 generates a set of RF reference signals including Tx beams 309-a, 309-b, 309-c, 309-d, 309-e, 309-f, 309-g, and 309-h. As mentioned, because UE 105 may also include an antenna array, it may use beamforming to receive RF reference signals sent by base stations 320-1 and 320-2 to form respective receive beams (Rx beams) 311-a and 311-b. Beamforming in this manner (by base station 320 and optionally by UE 105) may be used to make communications more efficient. They may also be used for other purposes, including making measurements for positioning determination (e.g., AoD and AoA measurements).

[0075] Visual odometry technology

[0076] Figure 4 and Figure 5 The basis of a technique for determining salient information is illustrated, which can be used, for example, to determine the location of a UE or to determine beams between wireless network devices.

[0077] Various techniques known in the related art can be used to detect key points in an image. Examples of such techniques include, but are not limited to, accelerated segmentation test features (FAST), accelerated robust features (SURF), scale-invariant feature transform (SIFT), oriented FAST and rotational BRIEF (binary robust independent elementary features) (ORB) and Harris corner detector. Any of these techniques can be used to detect key points of interest, such as corners, peaks of window structures. In some methods, image patches from an image can be represented by a two-dimensional grid of pixels to determine key points.

[0078] Figure 4 A comparison of key points between two example images 402 and 404 is depicted. Example image 402 may have been acquired by a first sensor (e.g., a first camera), and example image 404 may have been acquired by a second sensor (e.g., a second camera). In certain implementations, other types of spatial sensors may be used, such as radar sensors, lidar sensors. The use of radar or radar sensors as described herein may include RF sensing, and the radar sensor may include a dedicated RF sensor and / or a wireless communication interface (e.g., Fig.13 1330 of Fig.14 1430 in the above table).

[0079] Example image 402 includes a first example keypoint 420 and a second example keypoint 422. Example image 404 includes a first corresponding example keypoint 420' and a second corresponding example keypoint 422'. The first corresponding example keypoint 420' corresponds to the same keypoint or feature as the first example keypoint 420. The second corresponding example keypoint 422' corresponds to the same keypoint or feature as the second example keypoint 422. That is, when substantially the same object (e.g., a building) is captured by two different cameras, the same features (windows, corners, peaks, etc.) may be identified between them. For example, Figure 4 As shown, the first example key point 420 and the first corresponding example key point 420' may correspond to a portion of a window, while the second example key point 422 and the second corresponding example key point 422' may correspond to a portion of an arch. Figure 4 Various other key points within the example images 402 and 404 are indicated via dots and lines connecting corresponding points.

[0080] Based on the key points, the relative camera orientation (e.g., pose, angle) can be inferred using various methods known in the relevant art. As an example, an eight-point algorithm can be used that uses eight or more key points to form and factorize an essential matrix (also called a fundamental matrix) associated with corresponding points in a stereo image. As another example, Nister's five-point solution for determining the essential matrix can be used to estimate relative camera motion.

[0081] Reference now Figure 5 , shows an example of relative placement of cameras 502, 504 and point X in environment 510. Each of points C and C' is an optical axis of a camera (502 or 504), which in some scenarios may each be associated with an object (e.g., a UE, a vehicle, a UE co-located with a vehicle). This may provide stereoscopic vision of point X in environment 510, which is visualized by the skewed surface formed by points X, C, and C'.

[0082] Consider a scenario where the objects are vehicles 503 and 505 and the UE is co-located with each vehicle, such that the positions of the vehicles become substantially at points C and C'. Vehicles 503 and 505 may sense the environment around them using respective cameras associated therewith. For example, the image sensed by the first vehicle 503 may be the first example image 402, and the image sensed by the second vehicle 505 may be the second example image 404, as shown in FIG. Figure 4 Depicted.

[0083] From the corresponding points connected by lines, using the theory of essential matrix decomposition (e.g., the eight-point or five-point method noted above), the relative orientation and relative translation (in scale) between the two objects can be calculated. In the current scene, key points can be determined within the images sensed by vehicles 503 and 505 (e.g., using a technique such as FAST as noted above), and from each key point, the relative orientation of vehicles 503 and 505 can be determined. An example of a key point (e.g., a feature of an object) associated with a point X in the environment 510 is at point x on image plane 512 and point x' on image plane 514, where points x and x' correspond to points C and C', respectively. Point x (key point) can be identified where the line connecting point C and point X in the environment 510 intersects the image plane 512. This line and the image plane 512 do not need to be perpendicular to each other. Point x' (key point corresponding to key point x) can be identified where the line connecting point C' and point X in the environment 510 intersects the image plane 514.

[0084] Relative orientation may refer to the direction of a line from one point (e.g., from C or C') to another point (e.g., to C' to C). Even though the actual locations of C and C' may not be known, a vector (e.g., a unit vector) along the direction CC' may be determined in this way. This provides useful information for knowing where objects (e.g., vehicles 503, 505) are relative to each other in their own coordinate systems.

[0085] The example scenario then illustrates a localization use case and a beam alignment use case for the visual odometry technique described above.

[0086] Figure 6A simplified diagram of an example positioning scenario is illustrated, wherein a UE 602 is in a wireless network and is configured to communicate with one or more base stations 604a-604c. In this scenario, the UE 602 may be able to wirelessly exchange signals with base stations 604a and 604b via wireless signal paths 608a and 608b. For example, the UE 602 may be performing downlink positioning or uplink positioning with one or more of the base stations 604a and 604b. A physical obstruction 606 may exist between the UE 602 and one of the base stations (604c), thereby blocking a line of sight (LOS) signal path 608c at least where the UE 602 is located. The physical obstruction 606 may be a building, a wall, or other object that blocks the signal path 608c.

[0087] In the absence of signal path 608c for UE 602 to utilize, LOS positioning may be difficult or impossible in the case of base station 604c. NLOS positioning may be difficult in the case of wireless RF systems because it is not always possible to know the signal propagation path around obstacles for accurate positioning. However, using the methods described elsewhere herein, NLOS positioning may be feasible or desirable (e.g., the UE may determine or estimate angle information such as expected AoA). Alternatively, if UE 602 traverses to another location (e.g., point 610), signal path 608d may be available between UE 602 and base station 604c, and / or, for example, because of physical obstruction 606, the LOS signal path may no longer be available between UE 602 and other base stations 604a, 604b.

[0088] Figure 7 A simplified diagram of another example positioning scenario is illustrated, where there is a physical obstacle (e.g., barrier 702) between UEs 704 and 706. In this scenario, UEs 704 and 706 may be performing sidelink ranging, but because of barrier 702, their positioning may be in NLOS with each other as shown.

[0089] Here again, even though the receiver (e.g., UE 704) may be in NLOS with the transmitter (e.g., UE 706), it can infer angle information (e.g., expected AoA, azimuth, or elevation). With this angle information known by the receiver, ranging and / or positioning can be performed between UEs even in an NLOS environment.

[0090] As will be described herein, signals may be exchanged between endpoint wireless network devices (eg, UEs, base stations, location servers) to enable information related to and useful for the enhanced positioning described above to be determined even without knowing the location of any device.

[0091] Figure 8A simplified diagram of an example beam alignment scenario involving a UE 802 and a base station 804 is illustrated. In this scenario, a mobile (e.g., co-located with a vehicle) UE 802 connected to a base station may be determining the best transmit and receive beams for their communications. Conventional approaches use, for example, an iterative P1 (beam scanning and selection), P2 (beam refinement and narrower scanning by the transmitter), P3 (beam refinement by the receiver) search process. Conventional beamforming of transmit and receive beams may use information about Figure 3 The scanning process described is completed.

[0092] However, if the UE 802 and / or the base station 804 are made aware of the direction (including, for example, azimuth or elevation) that the other entity is using a unit vector (e.g., determined using an intrinsic matrix as discussed above) connecting the UE 802 and the base station 804, then even if the UE 802 and the base station 804 are unaware of each other's locations, these endpoint devices become able to autonomously select the correct transmit beam and receive beam without an exhaustive search as might be conventionally performed.

[0093] For example, a UE 802 mounted on a vehicle may use directional information (e.g., based on a unit vector) to determine the correct subset of receive beams to measure. Figure 8 In the scenario shown, the best receive beam may be receive beam 806a, and the best transmit beam may be transmit beam 808a. In some scenarios, one or a few candidate beams may be selected given the directional information, for example, two receive beams 806a, 806b and two transmit beams 808a, 808b. These beams may have been significantly narrowed from the numerous beams that conventional beamforming methods start with. Figure 3 802. Transmit beams 305-a, 305-b, 305-c, 305-d, 305-e, 305-f, 305-g, and 305-h and receive beams 309-a, 309-b, 309-c, 309-d, 309-e, 309-f, 309-g, and 309-h are compared. From the reduced candidate beams, beams 806a and 808a can be selected as the best. Since there is no additional search for beams, this can advantageously result in enhanced power savings for, for example, UE 802. In addition, UE 802 in the vehicle can utilize prior information not only for itself but also for (e.g., provided to or used for) other mobile UEs that are inside the vehicle or otherwise co-located with the vehicle to achieve power savings.

[0094] To achieve the above-mentioned beam alignment, signals may be exchanged between endpoint wireless network devices (e.g., UE and base station) so that each of these wireless network devices may determine information related to and useful for transmit and receive beam selection without knowing the location of any device.

[0095] Positioning Solutions

[0096] Using the information that can be determined using the above method, downlink positioning and uplink positioning of the UE can be performed. In one embodiment, a location server (e.g., LMF) can configure a wireless network device such as a UE, a base station, or an RSU to provide optical key points and / or their associated features of the environment sensed by a sensor (e.g., a camera) associated with the UE and a sensor (e.g., a camera) associated with the base station. In some variants, intrinsic parameters of the sensor can also be requested.

[0097] Upon receiving keypoints and / or features from, for example, a UE and a base station, the location server may use the visual odometry techniques described above to determine a translation unit vector between the UE and the base station. Figure 5 The unit vector CC' of may be an example of a unit vector. From the unit vector, the position may then determine auxiliary information such as angle information (including, for example, AoA of the UE and the base station).

[0098] For uplink positioning, the location server may signal the expected AoA from the UE to all base stations involved in positioning the UE.For downlink positioning, the location server may signal the expected AoA from each base station to the UE.

[0099] Fig. 9 900 is a call flow illustrating the exchange of signals between a location server 902, a base station 904, and a UE 906 according to some embodiments. According to various embodiments, the location server 902 may be a LMF, the base station 904 may be a gNB or in some cases an RSU, and the UE 906 may be a mobile UE or a UE co-located with a vehicle, so that the vehicle or the UE 906 may use a sensor such as a camera to obtain optical information such as an image while the vehicle is moving. In some embodiments, the base station 904 may have a sensor (e.g., a camera) associated therewith (e.g., mounted or attached to the base station 904 or proximate to and communicable with the base station 904).

[0100] In some embodiments, at arrow 912 of call flow 900, UE 906 may transmit a signal to location server 902, requesting assistance data "on demand." The assistance data may include AoA information to be used for positioning, and may be obtained, for example, at arrow 924b.

[0101] In some cases, this on-demand request may be optional, for example, if the location server 902 is aware of a schedule for providing assistance data. As will be noted below, the location server 902 may already have visual features from a prior transmission, particularly since the base station 904 and / or its sensors are likely to be static, and there is likely to be no change over time in the intrinsic or extrinsic parameters of the sensors associated with the base station 904.

[0102] An example information element "RequestAssistanceData" used to signal the location server 902 may contain the following information:

[0103]

[0104] Table 1. Examples of information elements .

[0105] In some implementations, sensors and their sensor parameters may be marked as static or dynamic, for example, whether the camera zoom and orientation is fixed (e.g., if mounted on a rotating platform). Dynamic parameters may be indicated by embedding in the image itself (e.g., metadata) or separately. Whether the sensor parameters are static or dynamic may help the location server 902 to correlate pixels across images taken in time and predict where corresponding pixels will be in future frames. This may then enhance keypoint matching across images for better localization.

[0106] In some embodiments, at arrows 914a and 914b, the location server 902 may transmit configuration information to configure the base station 904 and the UE 906. In some cases, the location server 902 may signal the configuration information to the base station 904 to provide visual features upon receiving a request from the UE 906 at arrow 912. In some implementations, the configuration information may include one or more of the following: a keypoint detection method (Harris corner detector, FAST, SURF, etc.), the number of keypoints and associated features to be provided, feature descriptors for one or more of the keypoints, intrinsic parameters of a sensor (e.g., a camera), or a combination thereof.

[0107] In some implementations, the configuration information may include the number of keypoints to be provided based on a strength metric for each keypoint specified as the sharpness of the corners (e.g., as used in the Harris corner measurement). For example, keypoints corresponding to the top N highest corners may be requested. In some cases, the requested keypoints may be uniformly sampled in the image space.

[0108] In some implementations, the configuration information may include information (e.g., optical information such as images, features), e.g., sensed information such as images and / or sensor intrinsic parameters, to be obtained by the base station 904 and the UE 906. In some implementations, the configuration information may include a request for the original image or the compressed image itself.

[0109] In some implementations, the configuration information may include a request for current location information of the base station and / or UE. Although the locations of the base station and / or UE are not required to perform call flow 900, they may be relevant for confirmation of the unit vector (determined at block 922) or other purposes.

[0110] In some implementations, the configuration information may request keypoints to be labeled as static or dynamic, associated with probabilities. For example, a building may be described as static and permanent, whereas a car on a road may be described as temporary). More generally, classical semantic segmentation may be used to classify pixels in an image so that each pixel can be associated with a class. In some cases, the class and / or probability may be a feature descriptor. This may help the location server 902 (or any other network device or node that determines keypoints) match keypoints across images more efficiently, thereby improving positioning accuracy.

[0111] Signaling the configuration by the location server 902 to the base station 904 (arrow 914a) may be performed, for example, via an NRPPa interface, and signaling the configuration by the location server 902 to the UE 906 (arrow 914b) may be performed, for example, via an LPP interface. In some implementations, each signaling may be performed on a periodic, semi-static (based on the availability of resource elements (time-frequency resources)), or on-demand basis. As alluded to below, because the environment around the base station 904 is relatively static, there may be less need for the base station 904 to obtain sensed features.

[0112] At block 916, base station 904 may obtain the sensed features. In some embodiments, a sensor (e.g., a camera) associated with base station 904 may be used to obtain one or more images, wherein the one or more images include the features sensed by the camera. In some cases, a camera may be used to capture a video. In some cases, one or more frames from a video may be extracted or obtained from the video.

[0113] In some embodiments, block 916 may not be performed in some subsequent or some previous executions of call flow 900. While moving objects such as vehicles, people, animals, etc. may change, and while lighting conditions may change throughout the day, the environment around base station 904 is relatively static and does not change in most cases because base station 904 is generally stationary. In such cases of stationary base stations, where extrinsic parameters and / or intrinsic parameters of at least one sensor of the base station have not changed since a previously reported visual feature, base station 904 may indicate so to the location server.

[0114] Thus, in some cases, unless it is known that a sensor associated with base station 904 has changed orientation, optical axis, or position, location server 902 may configure base station 904 to capture optical information, e.g., at a lower frequency or frames per second (periodically), semi-statically, or only when configured to do so (on demand) than UE 906. In some cases, sensed features may be stored at base station 904 or location server 902 for a period of time without configuring base station 904 to capture more features.

[0115] At block 918, the UE 906 may obtain the sensed features. In some embodiments, one or more images or videos may be obtained using a sensor (e.g., a camera) associated with the UE 906, similar to the features sensed by the base station 904. The sensed features may have points in the same environment as those features sensed by the base station 904, in order to enable keypoint detection and determination of unit vectors based on the detected keypoints.

[0116] At arrows 920a and 920b , location server 902 may receive obtained optical information (eg, sensed features) from base station 904 and UE 906 .

[0117] At block 922, the location server 902 may determine a translation unit vector between the location of the base station 904 and the location of the UE 906, and may determine assistance information based on the unit vector. In some embodiments, the unit vector may be determined by: (i) determining key points within a sensed feature (e.g., image) from the base station 904 and the UE 906 (e.g., using a method such as FAST as discussed above); (ii) determining a relative sensor orientation (e.g., using a method of eight-point or five-point essential matrix decomposition as discussed above); and (iii) determining the unit vector based on the relative sensor orientation. In particular, the unit vector indicates a direction from the base station 904 to the UE 906 and vice versa. Based on the unit vector, the location server 902 may determine assistance information. For example, the assistance information may include angle information, such as an expected AoA connecting the base station 904 and the UE 906, because the unit vector indicates a direction between the base station 904 and the UE 906. In some implementations, the relative translation may also be determined in proportion to the relative orientation.

[0118] At arrow 924a, the location server 902 may transmit assistance information (e.g., expected AoA) and / or unit vectors to the base station 904, for example, via NRPPa. At arrow 924b, the location server 902 may transmit assistance information (e.g., expected AoA) and / or unit vectors to the UE 906, for example, via LPP. With this assistance information, the UE 906 may perform downlink positioning with the base station 904, or the base station 904 may perform uplink positioning with the UE 906. Thus, in some cases, the call flow 900 may perform arrows 924a or 924b to perform uplink positioning or downlink positioning, whereas in other cases, both arrows 924a and 924b may be performed.

[0119] In some implementations, the call flow 900 described above may be applied to a first UE and a second UE to perform sidelink positioning (e.g., to Figure 5 scene).

[0120] In the case where a network device such as a UE has multiple sensors or cameras, the UE may handle reporting of sensed features in different ways. As an example, the UE may report images or key points from each camera separately, as if each camera is the only available camera. As an alternative example, the UE may report one image from multiple images of multiple cameras or key points identified in the image, because the key points identified on the image will be substantially the same. Key points may be calculated using images from multiple cameras at a network device (e.g., a UE) with multiple cameras. A reported image may be required to identify and / or describe a key point in the image, toward which a range and / or direction may be reported. As another example, the UE may report the range and / or direction to the key point directly, assuming that the key point can be identified by a descriptor (e.g., a known landmark building) without transmitting any image data.

[0121] Furthermore, in other implementations, the foregoing methods may be applied with optical sensors at other electromagnetic wave frequencies (eg, radar or lidar) to perform keypoint matching across radar or lidar images.

[0122] Beam Alignment Solutions

[0123] In some embodiments, the UE and the base station may exchange optical information including key points of the environment sensed by a sensor (e.g., a camera) associated with the UE and a sensor (e.g., a camera) associated with the base station and / or its associated features. In some variants, intrinsic parameters of the sensor may also be requested. In some specific implementations, RRC signaling may be used to exchange information between the UE and the base station.

[0124] In some embodiments, the UE and the gNB may independently determine the translation unit vector between them. The UE and the gNB may determine assistance information, such as angle information between them (e.g., AoD, AoA, azimuth, elevation), based on the inferred translation unit vectors in their respective coordinates. Based on the assistance information, the UE and the gNB may select appropriate transmit and receive beams (without using conventional scanning methods). In some cases, the UE may use the assistance information to select a dynamic list of beams to measure for each transmit SSB.

[0125] Fig.101 is a call flow 1000 illustrating the exchange of signals between a base station 1004 and a UE 1006 according to some embodiments. According to various embodiments, the base station 1004 may be a gNB or in some cases an RSU, and the UE 1006 may be a mobile UE or a UE co-located with a vehicle, so that the vehicle or the UE 1006 may use a sensor such as a camera to obtain optical information such as an image while the vehicle is moving. In some embodiments, the base station 1004 may have a sensor (e.g., a camera) associated therewith (e.g., mounted or attached to the base station 1004 or in proximity to and communicable with the base station 1004).

[0126] In some embodiments, at arrow 1012 of call flow 1000, base station 1004 may transmit configuration information to UE 1006 to configure UE 1006. In some implementations, the configuration information may include the type of information described elsewhere herein (one or more of keypoint detection methods, feature descriptors, number of keypoints and associated features to be provided, intrinsic parameters of sensors, etc.). In some implementations, the configuration information may include information to be obtained by UE 1006 (e.g., optical information such as images, features), for example, sensed information such as images and / or intrinsic parameters of sensors.

[0127] In some implementations, the base station 1004 may configure the UE 1006 based on one or more of the following: (i) periodic reporting with a specified reporting period (e.g., 1 second, 2 seconds), (ii) on-demand reporting, (iii) conditional or triggered reporting with preconfigured trigger conditions, or a combination thereof. The trigger condition may include one or more of the following: (i) the self-orientation of the UE 1006 changes by more than a threshold during the time window; (ii) the number of new objects detected by the UE 1006 (e.g., compared to the previous frame) exceeds a threshold (e.g., two or more new objects compared to the previous frame), or a combination thereof.

[0128] At arrow 1014, base station 1004 may receive the obtained optical information (e.g., sensed features) from UE 1006, e.g., via RRC. In some embodiments, the obtained information may include one or more images obtained using a camera associated with UE 1006. In some cases, a camera may be used to capture a video. In some cases, one or more frames from a video may be extracted or obtained from the video.

[0129] At block 1016, the base station 1004 may determine a translation unit vector between the location of the base station 1004 and the location of the UE 1006, and may determine assistance information based on the unit vector. In some embodiments, the unit vector may be determined by: (i) determining key points within a sensed feature (e.g., image) from the base station 1004 and the UE 1006 (e.g., using a method such as FAST as discussed above); (ii) determining a relative sensor orientation (e.g., using a method of eight-point or five-point essential matrix decomposition as discussed above); and (iii) determining the unit vector based on the relative sensor orientation. In particular, the unit vector indicates a direction from the base station 1004 to the UE 1006 and vice versa. Based on the unit vector, the base station 1004 may determine assistance information. For example, the assistance information may include angle information, such as an expected AoA connecting the base station 1004 and the UE 1006, because the unit vector indicates a direction between the base station 1004 and the UE 1006. As another example, the assistance information can include information about transmit beams and / or receive beams (eg, candidate beams, or optimal transmit and / or receive beams). In some implementations, the relative translation can also be determined proportionally along with the relative orientation.

[0130] At arrow 1018, the base station 1004 can transmit assistance information and / or a unit vector to the UE 1006. Using this assistance information, the UE 1006 can select and / or use the best transmit or receive beam to communicate with the base station 1004. In some implementations, the base station 1004 can select and / or use the best transmit or receive beam to communicate with the UE 1006.

[0131] In an alternative embodiment, call flow 1000 may execute arrow 1052, where base station 1004 may transmit optical information (e.g., sensed features) from base station 1004 to UE 1006. In some embodiments, one or more images or videos may be obtained using a sensor (e.g., a camera) associated with base station 1004. UE 1006 may also obtain sensed features using a sensor (e.g., a camera) associated with UE 1006. Features sensed by base station 1004 may have points in the same environment as those features sensed by UE 1006, so as to enable key point detection and determination of unit vectors based on the detected key points.

[0132] In some cases, the base station's provision of the sensed characteristics may be a one-time operation, where it is assumed that the characteristics do not change significantly if the base station 1004 is static and the intrinsic and / or extrinsic parameters of the sensor do not change.

[0133] In an alternative embodiment, call flow 1000 may perform block 1054 without performing blocks 1014-1018. At block 1054, UE 1006 may determine a translation unit vector and determine assistance information based on the unit vector. Similar to block 1016, the unit vector may be determined based on key points within a sensed feature (e.g., an image) from base station 1004 and UE 1006 (e.g., using a method such as FAST as discussed above). Based on the unit vector, UE 1006 may determine information about a transmit beam and / or receive beam (e.g., a candidate beam, or an optimal transmit and / or receive beam). Using this information, UE 1006 may select and / or use the optimal transmit or receive beam to communicate with base station 1004.

[0134] In some implementations, the call flow 1000 described above may be applied to a first UE and a second UE to perform sidelink beamforming (e.g., to Figure 5 scene).

[0135] Furthermore, in other implementations, the foregoing methods may be applied with optical sensors at other electromagnetic wave frequencies (eg, radar or lidar) to perform keypoint matching across radar or lidar images.

[0136] method

[0137] Fig.11 is a flow chart of a method 1100 for assisting in positioning a UE according to an embodiment. Fig.11 The functional structures illustrated in one or more of the illustrated blocks may include hardware and / or software components of a computerized device or system, such as a server (e.g., a location server) or a base station (e.g., a gNB). Components of such a computerized device or system may include, for example, a controller device, a computerized system, or a computer-readable device including a storage medium storing computer-readable and / or computer-executable instructions that are configured to cause at least one processor device or computerized device to perform operations when executed by a processor device. Fig.14 and Fig.15 Example components of a base station and a server are illustrated in FIG. 4 and are described in more detail below.

[0138] It should also be noted that the operations of method 1100 may be performed in any suitable order, and need not be Fig.11 In addition, method 1100 may include: Fig.11 The operations depicted in the figure may be additional or fewer than those described in the figure to assist the UE in positioning.

[0139] At box 1110, the functionality may include transmitting first configuration information to the UE, the first configuration information indicating how the UE will be used to obtain optical sensory data. In some embodiments, the first configuration information may include one or more of the following: a method for detecting one or more key points of the first visual information, a feature descriptor for one or more key points of the first visual information, an amount of one or more key points of the first visual information, and a first parameter associated with an image capture device associated with the UE (e.g., an intrinsic or non-intrinsic camera parameter), or a combination thereof. In some embodiments, the configuration information may include a request to associate one or more key points of the first visual information with a classification (e.g., to classify pixels in an image for classical semantic segmentation as discussed above).

[0140] Other configuration information may alternatively or additionally be included, for example, one or more of the following: the number of key points to be provided based on an intensity metric for each key point specified as a sharpness of corner (e.g., as used in Harris corner measurement); information to be obtained by the base station 904 and the UE 906 (e.g., optical information such as images, features), for example, sensed information such as images and / or sensor intrinsic parameters; a request for the original or compressed image itself; or a request for current location information of the base station and / or UE.

[0141] In some embodiments, configuration information may request keypoints to be marked as static or dynamic with associated probabilities so that keypoints can be matched across images more efficiently, thereby improving localization accuracy.

[0142] like Fig.14 or Fig.15 As illustrated, means for performing the functionality at block 1110 may include a processor, a wireless communication interface, and / or other components of a location server or base station.

[0143] At block 1120, the functionality may include receiving optical sensory data from the UE. In some embodiments, such optical sensory data may have been obtained according to the first configuration information, where examples of optical sensory data are noted below.

[0144] like Fig.14 or Fig.15 As illustrated, means for performing the functionality at block 1120 may include a processor, a wireless communication interface, and / or other components of a location server or base station.

[0145] At box 1130, the functionality may include determining angle information about wireless transmissions between at least one wireless network node (e.g., a base station such as a gNB) and the UE based at least on optical sensing data received from the UE.

[0146] In some embodiments, the functionality further includes transmitting second configuration information to at least one wireless network node, the second configuration information indicating how to obtain optical sensing data using at least one wireless network node. In some specific implementations, the optical sensing data received from the UE includes first visual information (e.g., one or more images or frames) sensed by an image capture device (e.g., a camera) associated with the UE, and the optical sensing data received from at least one wireless network includes second visual information (e.g., one or more images or frames) sensed by an image capture device (e.g., a camera) associated with the at least one wireless network node.

[0147] In some implementations, determining the angle information may also be based on optical sensing data received from at least one wireless network node. For example, key point matching and unit vector determination may be performed at the location server using optical sensing data from the UE and the wireless network node.

[0148] More specifically, in some embodiments, determining the angle information may include determining key points associated with optical sensing data received from at least the UE. The key points may be determined using methods described elsewhere herein (e.g., FAST, etc.). Based on the key points, a vector connecting a wireless network node and a UE may be determined. The vector may be determined using methods described elsewhere herein (e.g., 8-point or 5-point essential matrix decomposition). The vector provides angle information, including the direction to another node and the AoA expected from another node. With this angle information as auxiliary information for the UE, positioning may be performed even in NLOS.

[0149] In some cases, the first configuration information may further include a first time interval for sensing the first visual information, the second configuration information may further include a second time interval for sensing the second visual information, and the second time interval is shorter than the first time interval, for example, because the sensing characteristics of the at least one wireless network node may be relatively static compared to the sensing characteristics of the UE. In some specific implementations, the at least one wireless network node may report that there is no new information between the sensing events.

[0150] In some scenarios, the location server may already have visual features from prior transmissions, or be able to access them (assuming that the wireless network nodes are relatively static and there are no changes in intrinsic or extrinsic parameters of the sensors of the wireless network nodes). The functionality may then further include receiving subsequent optical sensing data from the UE but not from the at least one wireless network node, and determining subsequent angular information about wireless transmissions between the at least one wireless network node and the UE based on the optical sensing data received from the at least one wireless network node and the subsequent optical sensing data from the UE.

[0151] In some embodiments, determining angular information about wireless transmissions between at least one wireless network node and the UE includes: determining a relative orientation between an image capture device associated with the at least one wireless network node and an image capture device associated with the UE based on one or more key points of the first visual information and one or more key points of the second visual information; and determining a vector (e.g., a translation unit vector) between the at least one wireless network node and the UE based on the relative orientation, the angular information being determined based on the vector. In some cases, the relative distance can be determined together with the relative orientation.

[0152] In some cases, the first parameter may include an indication of whether one or more parameters of an image capture device associated with the UE are static or dynamic, the second parameter may include an indication of whether one or more parameters of an image capture device associated with at least one wireless network node are static or dynamic, and determining the relative orientation may also be based on the first parameter and the second parameter.

[0153] In some cases, the first configuration information may include a request to associate one or more key points of the first visual information with a classification, the second configuration information may include a request to associate pixels within one or more key points of the second visual information with a classification, and determining the relative orientation may also be based on the classification.

[0154] In some embodiments, the functionality further includes determining an expected positioning metric based at least on angular information about wireless transmissions between the at least one radio network node and the UE. In some implementations, the expected positioning metric includes an expected uplink AoA from the UE or an expected downlink AoA from the at least one radio network node, and the functionality further includes transmitting the expected uplink AoA to the at least one radio network node, the expected uplink AoA configured for use with positioning or beamforming with the UE; or transmitting the expected downlink AoA to the UE, the expected downlink AoA configured for use with positioning or beamforming with the UE.

[0155] like Fig.14 or Fig.15 As illustrated, means for performing the functionality at block 1130 may include a processor and / or other components of a location server or base station.

[0156] Fig.12 is a flow chart of a method 1200 for beam alignment with a UE according to an embodiment. Fig.12The functional structures illustrated in one or more of the illustrated blocks may include hardware and / or software components of a computerized device or system (e.g., UE). Components of such a computerized device or system may include, for example, a controller device, a computerized system, or a computer-readable device, which includes a storage medium storing computer-readable and / or computer-executable instructions, which are configured to cause at least one processor device or computerized device to perform operations when executed by a processor device. Fig.13 Example components of a UE are illustrated in FIG. 4 , and these example components are described in more detail below.

[0157] It should also be noted that the operations of method 1200 may be performed in any suitable order, and need not be Fig.12 In addition, method 1200 may include Fig.12 The operations depicted in FIG. 2 may be additional or fewer operations than those described in FIG. 2 to perform beam alignment with the UE.

[0158] At block 1210, the functionality may include receiving first configuration information from a network entity, the first configuration information indicating how to obtain optical sensing data with a UE.

[0159] Means for performing the functions at block 1210 may include a processor, a wireless communication interface, and / or other components of the UE, such as Fig.13 exemplified.

[0160] At block 1220, the functionality may include obtaining optical sensory data using a sensor of the UE. In some embodiments, such optical sensory data may have been obtained according to the first configuration information, where examples of optical sensory data are noted elsewhere herein.

[0161] Means for performing the functions at block 1220 may include a processor, a wireless communication interface, and / or other components of the UE, such as Fig.13 exemplified.

[0162] At block 1230, the functionality may include obtaining angle information regarding wireless transmissions between at least one wireless network node and the UE based at least on the optical sensing data.

[0163] In some embodiments, the functionality may further include transmitting the obtained optical sensing data to a network entity, the network entity including a location server, and obtaining angle information may include receiving angle information about wireless transmission between at least one wireless network node and the UE from the location server, the angle information being determined based on optical sensing data obtained using sensors of the at least one wireless network node and optical sensing data obtained using sensors of the UE.

[0164] More specifically, in some embodiments, determining the angle information may include determining key points associated with optical sensing data obtained using at least the UE. The key points may be determined using methods described elsewhere herein (e.g., FAST, etc.). Based on the key points, a vector connecting a wireless network node and a UE may be determined. The vector may be determined using methods described elsewhere herein (e.g., 8-point or 5-point essential matrix decomposition). The vector provides angle information, including the direction to another node and the AoA expected from another node. In the case where this angle information is used as auxiliary information for the UE, it is possible to perform the formation of optimal transmit and receive beams that do not require large-scale scanning.

[0165] In certain cases, the angular information regarding wireless transmissions between the at least one radio network node and the UE may include an expected downlink AoA expected from the at least one radio network node, and the functionality may further include positioning the UE based on the expected downlink AoA.

[0166] In some embodiments, obtaining angle information may include receiving an optimal transmit beam, receive beam, or a combination thereof from at least one wireless network node to communicate with at least one wireless network node, and the angle information is determined based on optical sensing data obtained using a sensor of the at least one wireless network node and optical sensing data obtained using a sensor of the UE.

[0167] In some embodiments, the functionality may further include receiving optical sensing data obtained using a sensor of at least one wireless network node, wherein the network entity includes at least one wireless network node, and obtaining the angle information includes determining angle information about wireless transmissions between the at least one wireless network node and the UE based on the optical sensing data obtained using a sensor of the at least one wireless network node and based on the optical sensing data obtained using a sensor of the UE.

[0168] Means for performing the functions at block 1230 may include a processor, a wireless communication interface, and / or other components of the UE, such as Fig.13 exemplified.

[0169] Device

[0170] Fig.13 is available as above (for example, in conjunction with Figures 6 to 12 ) is a block diagram of an implementation of the UE 105 utilizing the above-described method. For example, the UE 105 may execute Fig.12 One or more of the functions of the method shown. It should be noted that Fig.13 It is intended only to provide a generalized illustration of various components, any or all of which may be utilized as appropriate. It may be noted that in some cases, Fig.13The illustrated components may be localized as a single physical device and / or distributed among various networked devices that may be located at different physical locations. In addition, as previously mentioned, the functionality of the UE discussed in the previously described embodiments may be provided by Fig.13 The invention may be executed by one or more of the illustrated hardware and / or software components.

[0171] UE 105 is shown as including hardware elements that may be electrically coupled via bus 1305 (or may otherwise communicate, as appropriate). The hardware elements may include a processor 1310, which may include, but is not limited to, one or more general-purpose processors (e.g., application processors), one or more special-purpose processors (such as digital signal processor (DSP) chips, graphics acceleration processors, application-specific integrated circuits (ASICs), etc.), and / or other processing structures or components. Processor 1310 may include one or more processing units, which may be housed in a single integrated circuit (IC) or multiple ICs. Fig.13 As shown, some embodiments may have a separate DSP 1320, depending on the desired functionality. Position determination and / or other determinations based on wireless communication (discussed below) may be provided in the processor 1310 and / or the wireless communication interface 1330. The UE 105 may also include: one or more input devices 1370, which may include but are not limited to one or more keyboards, touch screens, touch pads, microphones, buttons, dials, switches, etc.; and one or more output devices 1315, which may include but are not limited to one or more displays (e.g., touch screens), light emitting diodes (LEDs), speakers, etc.

[0172] UE 105 may also include a wireless communication interface 1330, which may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device and / or a chipset (such as Device, IEEE 802.11 device, IEEE 802.15.4 device, Wi-Fi device, WiMAX device, WAN device and / or various cellular devices, etc.), etc., the wireless communication interface may enable UE 105 to communicate with other devices as described in the above embodiments. The wireless communication interface 1330 may allow the TRP of the network to communicate (e.g., send and receive) data and signaling, such as via eNB, gNB, ng-eNB, access point, various base stations and / or other access node types and / or other network components, computer systems and / or any other electronic device communicatively coupled to the TRP, as described herein. Communication may be performed via one or more wireless communication antennas 1332 that transmit and / or receive wireless signals 1334. According to some embodiments, the wireless communication antenna 1332 may include multiple discrete antennas, antenna arrays, or any combination thereof. The antenna 1332 may be capable of using beams (e.g., Tx beams and Rx beams) to send and receive wireless signals. Beamforming may be performed using digital and / or analog beamforming techniques with corresponding digital and / or analog circuitry. The wireless communication interface 1330 may include such circuitry.

[0173] Depending on the desired functionality, the wireless communication interface 1330 may include separate receivers and transmitters, or any combination of transceivers, transmitters, and / or receivers to communicate with base stations (e.g., ng-eNBs and gNBs) and other terrestrial transceivers (such as wireless devices and access points). The UE 105 may communicate with different data networks, which may include various network types. For example, the WWAN may be a CDMA network, a time division multiple access (TDMA) network, a frequency division multiple access (FDMA) network, an orthogonal frequency division multiple access (OFDMA) network, a single carrier frequency division multiple access (SC-FDMA) network, a WiMAX (IEEE 802.16) network, etc. A CDMA network may implement one or more RATs, such as WCDMA, etc. Includes IS-95, IS-2000, and / or IS-856 standards. A TDMA network may implement GSM, Digital Advanced Mobile Phone System (D-AMPS), or some other RAT. An OFDMA network may employ LTE, Advanced LTE, 5G NR, and the like. 5G NR, LTE, Advanced LTE, GSM, and WCDMA are described in documents from 3GPP. Described in documents from an organization named "3rd Generation Partnership Project 2" (3GPP2). 3GPP and 3GPP2 documents are publicly available. A wireless local area network (WLAN) may also be an IEEE 802.11x network, while a wireless personal area network (WPAN) may be a Bluetooth network, IEEE 802.15x, or some other type of network. The techniques described herein may also be used for any combination of WWAN, WLAN, and / or WPAN.

[0174] The UE 105 may also include sensors 1340. The sensors 1340 may include, but are not limited to, one or more inertial sensors and / or other sensors (e.g., accelerometers, gyroscopes, cameras, magnetometers, altimeters, microphones, proximity sensors, light sensors, barometers, etc.), some of which may be used to obtain position-related measurements and / or other information.

[0175] Embodiments of UE 105 may also include a global navigation satellite system (GNSS) receiver 1380 that is capable of receiving signals 1384 from one or more GNSS satellites using antenna 1382 (which may be the same as antenna 1332). Positioning based on GNSS signal measurements may be used to supplement and / or incorporate the techniques described herein. GNSS receiver 1380 may extract the location of UE 105 using conventional techniques from GNSS satellites of GNSS systems such as Global Positioning System (GPS), Galileo, GLONASS, Quasi-Zenith Satellite System (QZSS) covering Japan, IRNSS covering India, BeiDou Navigation Satellite System (BDS) covering China, etc. In addition, the GNSS receiver 1380 may be used with various augmentation systems, such as, for example, a satellite-based augmentation system (SBAS), which may be associated with or otherwise enabled for use with one or more global and / or regional navigation satellite systems, such as, for example, the Wide Area Augmentation System (WAAS), the European Geostationary Navigation Overlay Service (EGNOS), the Multifunction Satellite Augmentation System (MSAS), and the Geographic Augmentation Navigation System (GAGAN), among others.

[0176] It may be pointed out that although Fig.131380 is illustrated as a different component, but the embodiments are not limited thereto. As used herein, the term "GNSS receiver" may include hardware and / or software components configured to obtain GNSS measurements (measurements from GNSS satellites). Therefore, in some embodiments, the GNSS receiver may include (as software) a measurement engine executed by one or more processors, such as processor 1310, DSP 1320, and / or a processor within wireless communication interface 1330 (e.g., in a modem). The GNSS receiver may also optionally include a positioning engine that can use the GNSS measurements from the measurement engine to determine the position of the GNSS receiver using an extended Kalman filter (EKF), a weighted least squares method (WLS), a hatch filter, a particle filter, etc. The positioning engine may also be executed by one or more processors, such as processor 1310 or DSP 1320.

[0177] The UE 105 may also include and / or be in communication with memory 1360. The memory 1360 may include, but is not limited to, local and / or network accessible storage, disk drives, drive arrays, optical storage devices, solid-state storage devices such as random access memory (RAM) and / or read-only memory (ROM), which may be programmable, flash-updatable, etc. Such storage devices may be configured to implement any suitable data storage, including, but not limited to, various file systems, database structures, etc.

[0178] The memory 1360 of the UE 105 may also include software elements ( Fig.13 105), including an operating system, device drivers, executable libraries, and / or other code (such as one or more applications), which software elements may include computer programs provided by various embodiments, and / or may be designed to implement methods provided by other embodiments, and / or configure systems provided by other embodiments, as described herein. By way of example only, one or more procedures described with respect to the methods discussed above may be implemented as code and / or instructions in memory 1360 that can be executed by UE 105 (and / or processor 1310 or DSP 1320 within UE 105). Then, in some embodiments, such code and / or instructions may be used to configure and / or adapt a general purpose computer (or other device) to perform one or more operations according to the described methods.

[0179] Fig.14 is available as above (for example, in conjunction with Figures 6 to 12 ) is a block diagram of an implementation scheme of the base station 120 used. It should be noted that Fig.14It is intended only to provide a generalized illustration of the various components, any or all of which may be utilized as appropriate. In some embodiments, base station 120 may correspond to a gNB, ng-eNB, and / or (more generally) a TRP.

[0180] Base station 120 is shown as including hardware elements that may be electrically coupled via bus 1405 (or may communicate in other ways as appropriate). The hardware elements may include processor 1410, which may include, but is not limited to, one or more general-purpose processors, one or more special-purpose processors (such as DSP chips, graphics acceleration processors, ASICs, etc.), and / or other processing structures or components. Fig.14 As shown, some embodiments may have a separate DSP 1420, depending on the desired functionality. According to some embodiments, wireless communication-based position determination and / or other determinations may be provided in the processor 1410 and / or the wireless communication interface 1430 (discussed below). The base station 120 may also include one or more input devices, which may include but are not limited to a keyboard, a display, a mouse, a microphone, buttons, dials, switches, etc.; and one or more output devices, which may include but are not limited to a display, a light emitting diode (LED), a speaker, etc.

[0181] The base station 120 may also include a wireless communication interface 1430, which may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication device and / or a chipset (such as Devices, IEEE 802.11 devices, IEEE 802.15.4 devices, Wi-Fi devices, WiMAX devices, cellular communication facilities, etc.), etc., which can enable the base station 120 to communicate as described herein. The wireless communication interface 1430 can permit the communication (e.g., sending and receiving) of data and signaling to UEs, other base stations / TRPs (e.g., eNBs, gNBs, and ng-eNBs), and / or other network components, computer systems, and / or any other electronic devices described herein. Communication can be performed via one or more wireless communication antennas 1432 that transmit and / or receive wireless signals 1434.

[0182] The base station 120 may also include a network interface 1480, which may include support for wired communication technologies. The network interface 1480 may include a modem, a network card, a chipset, etc. The network interface 1480 may include one or more input and / or output communication interfaces to permit data to be exchanged with a network, a communication network server, a computer system, and / or any other electronic device described herein.

[0183] In many embodiments, the base station 120 may also include memory 1460. The memory 1460 may include, but is not limited to, local and / or network accessible storage, disk drives, drive arrays, optical storage devices, solid-state storage devices (such as RAM and / or ROM), which may be programmable, flash-updatable, etc. Such storage devices may be configured to implement any suitable data storage, including, but not limited to, various file systems, database structures, etc.

[0184] The memory 1460 of the base station 120 may also include software elements ( Fig.14 14), including an operating system, device drivers, executable libraries, and / or other code (such as one or more applications), which software elements may include computer programs provided by various embodiments, and / or may be designed to implement methods provided by other embodiments, and / or configure systems provided by other embodiments, as described herein. By way of example only, one or more procedures described with respect to the methods discussed above may be implemented as code and / or instructions in memory 1460 that may be executed by base station 120 (and / or processor 1410 or DSP 1420 within base station 120). Then, in some embodiments, such code and / or instructions may be used to configure and / or adapt a general purpose computer (or other device) to perform one or more operations according to the described methods.

[0185] Fig.15 is a block diagram of an embodiment of a computer system 1500 that may be used in whole or in part to provide one or more network components (e.g., Figure 1 Location server 160, Figure 2 It should be noted that Fig.15 It is intended only to provide a generalized illustration of the various components, any or all of which may be utilized as appropriate. Fig.15 The broad illustrations show how various system elements can be implemented in a relatively separate or relatively more integrated manner. Fig.15 The illustrated components may be localized to a single device and / or distributed among various networked devices that may be located at different geographical locations.

[0186] The computer system 1500 is shown as including hardware elements that can be electrically coupled via a bus 1505 (or can communicate in other ways as appropriate). The hardware elements may include a processor 1510, which may include but is not limited to one or more general-purpose processors, one or more special-purpose processors (such as digital signal processing chips, graphics acceleration processors, etc.) and / or other processing structures, which can be configured to perform one or more of the methods described herein. The computer system 1500 may also include one or more input devices 1515, which may include but are not limited to a mouse, a keyboard, a camera, a microphone, etc.; and one or more output devices 1520, which may include but are not limited to a display device, a printer, etc.

[0187] The computer system 1500 may also include (and / or communicate with) one or more non-transitory storage devices 1525, which may include, but are not limited to, local and / or network accessible storage devices, and / or may include, but are not limited to, disk drives, drive arrays, optical storage devices, solid-state storage devices such as RAM and / or ROM, which may be programmable, flash-updatable, etc. Such storage devices may be configured to implement any suitable data storage, including, but not limited to, various file systems, database structures, etc. Such data storage may include databases and / or other data structures for storing and managing messages and / or other information to be transmitted to one or more devices via the hub, as described herein.

[0188] The computer system 1500 may also include a communication subsystem 1530, which may include wireless communication technologies managed and controlled by a wireless communication interface 1533, as well as wired technologies (such as Ethernet, coaxial communication, universal serial bus (USB), etc.). The wireless communication interface 1533 may include one or more wireless transceivers, which may transmit and receive wireless signals 1555 (e.g., signals according to 5G NR or LTE) via a wireless antenna 1550. Therefore, the communication subsystem 1530 may include a modem, a network card (wireless or wired), an infrared communication device, a wireless communication device, and / or a chipset, etc., which may enable the computer system 1500 to communicate with any device on the corresponding network (including user equipment (UE), base station and / or other TRP, and / or any other electronic device described herein) on any or all of the communication networks described herein. Therefore, the communication subsystem 1530 can be used to receive and transmit data, as described in the embodiments of this article.

[0189] In many embodiments, the computer system 1500 will also include a working memory 1535, which may include a RAM or ROM device, as described above. The software elements shown as being located within the working memory 1535 may include an operating system 1540, device drivers, executable libraries, and / or other code such as one or more applications 1545, which may include computer programs provided by various embodiments, and / or may be designed to implement methods provided by other embodiments and / or configure systems provided by other embodiments, as described herein. By way of example only, one or more of the procedures described with respect to the methods discussed above may be implemented as code and / or instructions that can be executed by a computer (and / or a processor within a computer); then, in one aspect, such code and / or instructions may be used to configure and / or adapt a general purpose computer (or other device) to perform one or more operations according to the described methods.

[0190] The set of these instructions and / or codes may be stored on a non-transitory computer-readable storage medium (such as the storage device 1525 described above). In some cases, the storage medium may be incorporated into a computer system such as computer system 1500. In other embodiments, the storage medium may be separate from the computer system (e.g., removable media such as an optical disk), and / or may be provided in the form of an installation package so that the storage medium can be used to program, configure, and / or adapt a general-purpose computer using the instructions / code stored thereon. These instructions may take the form of executable code that can be executed by computer system 1500, and / or may take the form of source code and / or installable code that, when compiled and / or installed on computer system 1500 (e.g., using any of a variety of commonly available compilers, installers, compression / decompression utilities, etc.), takes the form of executable code.

[0191] It will be apparent to those skilled in the art that substantial modifications may be made according to specific requirements. For example, customized hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software, such as applets, etc.), or both. In addition, connections to other computing devices such as network input / output devices may be employed.

[0192] With reference to the accompanying drawings, components that may include memory may include non-transient machine-readable media. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any storage medium that participates in providing data that enables a machine to operate in a particular manner. In the embodiments provided above, various machine-readable media may be involved when providing instructions / codes to a processor and / or other devices for execution. Additionally or alternatively, machine-readable media may be used to store and / or carry such instructions / codes. In many specific implementations, computer-readable media are physical and / or tangible storage media. Such media may take many forms, including but not limited to non-volatile media and volatile media. Common forms of computer-readable media include, for example: magnetic and / or optical media, any other physical media with hole patterns, RAM, programmable ROM (PROM), erasable PROM (EPROM), FLASH-EPROM, any other memory chip or memory box, or any other medium from which a computer can read instructions and / or codes.

[0193] The methods, systems and devices discussed herein are examples. Various embodiments may omit, replace or add various procedures or components as appropriate. For example, the features described in certain embodiments may be combined in various other embodiments. Different aspects and elements of the embodiments may be combined in a similar manner. The various components of the drawings provided herein may be embodied in hardware and / or software. In addition, technology may evolve, and therefore many elements are examples, which do not limit the scope of the present disclosure to those specific examples.

[0194] It has proven convenient at times, primarily for reasons of common usage, to refer to such signals as bits, information, values, elements, symbols, characters, variables, terms, numbers, digital symbols, and the like. However, it should be understood that all of these or similar terms are to be associated with the appropriate physical quantities and are merely convenient labels. Unless otherwise specifically stated, as will be apparent from the above discussion, it should be understood that throughout this specification, discussions utilizing terms such as "processing," "computing," "calculating," "determining," "ascertaining," "identifying," "correlating," "measuring," "performing," and the like refer to the actions or processes of a particular device such as a special-purpose computer or similar special-purpose electronic computing device. Thus, in the context of this specification, a special-purpose computer or similar special-purpose electronic computing device is capable of manipulating or transforming signals, typically represented as physical, electronic, electrical, or magnetic quantities in a memory, register, or other information storage device, a transmitting device, or a display device of the special-purpose computer or similar special-purpose electronic computing device.

[0195] As used herein, the terms "and" and "or" may include multiple meanings that are also intended to depend at least in part on the context in which such terms are used. Generally, "or", if used in association with a list, such as A, B, or C, is intended to mean A, B, and C (inclusive meanings are used here) as well as A, B, or C (exclusive meanings are used here). In addition, as used herein, the term "one or more" may be used to describe any feature, structure, or characteristic in the singular form, or may be used to describe a certain combination of features, structures, or characteristics. However, it should be noted that this is merely an illustrative example, and the subject matter claimed is not limited to this example. In addition, the term "at least one of...", if used in association with a list, such as A, B, or C, may be interpreted to mean any combination of A, B, and / or C, such as A, AB, AA, AAB, AABBCCC, etc.

[0196] Several embodiments have been described, and various modifications, alternative configurations, and equivalents may be used without departing from the scope of the present disclosure. For example, the above elements may be components of a larger system, wherein other rules may take precedence over the application of the various embodiments or otherwise modify the application of the various embodiments. In addition, multiple steps may be performed before, during, or after considering the above elements. Accordingly, the above description does not limit the scope of the present disclosure.

[0197] In view of this description, various embodiments may include different combinations of features. Specific implementation examples are described in the following numbered clauses:

[0198] Clause 1. A method for assisting in positioning a user equipment (UE), the method comprising: transmitting first configuration information to the UE, the first configuration information indicating how to utilize the UE to obtain optical sensing data; receiving optical sensing data from the UE; and determining angle information about wireless transmissions between at least one wireless network node and the UE based at least on the optical sensing data received from the UE.

[0199] Clause 2. The method according to Clause 1 further includes transmitting second configuration information to the at least one wireless network node, wherein the second configuration information indicates how to obtain optical sensing data using the at least one wireless network node; wherein determining the angle information is also based on the optical sensing data received from the at least one wireless network node.

[0200] Clause 3. The method according to any of clauses 1-2, further comprising determining an expected positioning metric based at least on the angle information regarding wireless transmissions between the at least one radio network node and the UE.

[0201] Clause 4. A method according to any one of clauses 1-3, wherein the expected positioning metric includes an uplink angle of arrival (AoA) expected from the UE or a downlink AoA expected from the at least one wireless network node; and the method also includes transmitting the uplink AoA to the at least one wireless network node, the uplink AoA being configured for use with positioning or beamforming using the UE; or transmitting the downlink AoA to the UE, the downlink AoA being configured for use with positioning or beamforming using the UE.

[0202] Clause 5. A method according to any one of clauses 1-4, wherein the optical sensing data received from the UE includes first visual information sensed by an image capture device associated with the UE, and the optical sensing data received from the at least one wireless network includes second visual information sensed by an image capture device associated with the at least one wireless network node.

[0203] Clause 6. A method according to any one of clauses 1-5, wherein determining the angle information about the wireless transmission between the at least one wireless network node and the UE comprises: determining the relative orientation between the image capture device associated with the at least one wireless network node and the image capture device associated with the UE based on one or more key points of the first visual information and one or more key points of the second visual information; and determining the vector between the at least one wireless network node and the UE based on the relative orientation, wherein the angle information is determined based on the vector.

[0204] Clause 7. A method according to any one of clauses 1-6, wherein the first configuration information includes a method for detecting the one or more key points of the first visual information, a feature descriptor for the one or more key points of the first visual information, the amount of the one or more key points of the first visual information, and a first parameter associated with the image capture device associated with the UE, or a combination thereof; and the second configuration information includes a method for detecting the one or more key points of the second visual information, a feature descriptor for the one or more key points of the second visual information, the amount of the one or more key points of the second visual information, and a second parameter associated with the image capture device associated with the at least one wireless network node, or a combination thereof.

[0205] Clause 8. A method according to any one of clauses 1-7, wherein the first parameter includes an indication of whether one or more parameters of the image capture device associated with the UE are static or dynamic; the second parameter includes an indication of whether one or more parameters of the image capture device associated with the at least one wireless network node are static or dynamic; and determining the relative orientation is also based on the first parameter and the second parameter.

[0206] Clause 9. A method according to any one of clauses 1-8, wherein the first configuration information includes a request to associate the one or more key points of the first visual information with a classification, and the second configuration information includes a request to associate pixels within the one or more key points of the second visual information with a classification; and determining the relative orientation is also based on the classification.

[0207] Clause 10. A method according to any one of clauses 1-9, wherein the first configuration information also includes a first time interval for sensing the first visual information; the second configuration information also includes a second time interval for sensing the second visual information; and the second time interval is shorter than the first time interval.

[0208] Clause 11. The method according to any one of clauses 1-10 further includes: receiving subsequent optical sensing data from the UE but not from the at least one wireless network node; and determining subsequent angle information about wireless transmissions between the at least one wireless network node and the UE based on the optical sensing data received from the at least one wireless network node and the subsequent optical sensing data from the UE.

[0209] Clause 12. The method according to any of clauses 1-11, further comprising selecting a subset of transmit beams or a subset of receive beams of the at least one radio network node and the UE based on the expected positioning metric.

[0210] Clause 13. The method of any of clauses 1-12, further comprising receiving a request for assistance data from the UE, wherein transmitting the first configuration information is in response to the request for assistance data.

[0211] Clause 14. A device of a wireless communication network, the device comprising: one or more network interfaces; a memory; and one or more processors, the one or more processors being communicatively coupled to the one or more network interfaces and the memory and configured to: transmit first configuration information to a user equipment (UE), the first configuration information indicating how to obtain optical sensing data using the UE; receive the optical sensing data from the UE; and determine angle information about wireless transmissions between at least one wireless network node and the UE based at least on the optical sensing data received from the UE.

[0212] Clause 15. An apparatus according to clause 14, wherein the one or more processors are further configured to transmit second configuration information to the at least one wireless network node, the second configuration information indicating how to obtain optical sensing data using the at least one wireless network node; and wherein determining the angle information is also based on the optical sensing data received from the at least one wireless network node.

[0213] Clause 16. An apparatus according to any one of clauses 14-15, wherein the one or more processors are further configured to determine an expected positioning metric based at least on the angular information about the wireless transmission between the at least one wireless network node and the UE; wherein: the expected positioning metric includes an uplink angle of arrival (AoA) expected from the UE or a downlink AoA expected from the at least one wireless network node; and the one or more processors are further configured to transmit the uplink AoA to the at least one wireless network node, the uplink AoA being configured for use with positioning or beamforming using the UE; or to transmit the downlink AoA to the UE, the downlink AoA being configured for use with positioning or beamforming using the UE.

[0214] Clause 17. An apparatus according to any one of clauses 14-16, wherein the optical sensing data received from the UE includes first visual information sensed by an image capture device associated with the UE, and the optical sensing data received from the at least one wireless network includes second visual information sensed by an image capture device associated with the at least one wireless network node; and wherein, in order to determine the angle information about the wireless transmission between the at least one wireless network node and the UE, the one or more processors are further configured to: determine the relative orientation between the image capture device associated with the at least one wireless network node and the image capture device associated with the UE based on one or more key points of the first visual information and one or more key points of the second visual information; and determine the vector between the at least one wireless network node and the UE based on the relative orientation; and determine the angle information determined based on the vector.

[0215] Clause 18. An apparatus according to any one of clauses 14-17, wherein the first configuration information includes a method for detecting the one or more key points of the first visual information, a feature descriptor for the one or more key points of the first visual information, the amount of the one or more key points of the first visual information, and a first parameter associated with the image capture device associated with the UE, or a combination thereof; and the second configuration information includes a method for detecting the one or more key points of the second visual information, a feature descriptor for the one or more key points of the second visual information, the amount of the one or more key points of the second visual information, and a second parameter associated with the image capture device associated with the at least one wireless network node, or a combination thereof.

[0216] Clause 19. According to the device described in any of clauses 14-18, the one or more processors are also configured to determine an expected positioning metric based at least on the angle information about the wireless transmission between the at least one wireless network node and the UE; and select a subset of the transmitting beams or a subset of the receiving beams of the at least one wireless network node and the UE based on the expected positioning metric.

[0217] Clause 20. An apparatus as described in any of clauses 14-19, the one or more processors being further configured to receive a request for assistance data from the UE, and to transmit the first configuration information in response to the request for assistance data.

[0218] Clause 21. A method for performing beam alignment with a user equipment (UE), the method comprising: receiving first configuration information from a network entity, the first configuration information indicating how to obtain optical sensing data using the UE; obtaining the optical sensing data using a sensor of the UE; and obtaining angle information about wireless transmission between at least one wireless network node and the UE based at least on the optical sensing data.

[0219] Clause 22. The method according to Clause 21 also includes transmitting the obtained optical sensing data to the network entity, the network entity including a location server; wherein obtaining the angle information includes receiving the angle information about the wireless transmission between the at least one wireless network node and the UE from the location server, and the angle information is determined based on the optical sensing data obtained using the sensor of the at least one wireless network node and the optical sensing data obtained using the sensor of the UE.

[0220] Clause 23. A method according to any one of clauses 21-22, wherein the angle information regarding the wireless transmission between the at least one wireless network node and the UE includes an expected downlink angle of arrival (AoA) expected from the at least one wireless network node; and the method also includes determining the positioning of the UE based on the expected downlink AoA.

[0221] Clause 24. A method according to any one of clauses 21-23, wherein obtaining the angle information comprises receiving an optimal transmit beam, a receive beam, or a combination thereof from the at least one wireless network node to communicate with the at least one wireless network node; and the angle information is determined based on optical sensing data obtained using a sensor of the at least one wireless network node and the optical sensing data obtained using the sensor of the UE.

[0222] Clause 25. The method according to any one of clauses 21-24 further includes receiving optical sensing data obtained using a sensor of the at least one wireless network node; wherein: the network entity includes the at least one wireless network node; and obtaining the angle information includes determining the angle information about the wireless transmission between the at least one wireless network node and the UE based on the optical sensing data obtained using the sensor of the at least one wireless network node and based on the optical sensing data obtained using the sensor of the UE.

[0223] Clause 26. A user equipment (UE), the UE comprising: one or more network interfaces; a memory; a sensor; and one or more processors, the one or more processors being communicatively coupled to the one or more network interfaces, the sensor, and the memory and configured to: receive first configuration information from a network entity, the first configuration information indicating how to obtain optical sensing data using the UE; obtain the optical sensing data using the sensor of the UE; and obtain angle information about wireless transmission between at least one wireless network node and the UE based at least on the optical sensing data.

[0224] Clause 27. A UE according to clause 26, wherein the one or more processors are also configured to transmit the obtained optical sensing data to the network entity, the network entity comprising a location server; and wherein, in order to obtain the angle information, the one or more processors are configured to: receive from the location server the angle information about the wireless transmission between the at least one wireless network node and the UE, the angle information being determined based on the optical sensing data obtained using the sensors of the at least one wireless network node and the optical sensing data obtained using the sensors of the UE.

[0225] Clause 28. A UE according to any one of clauses 26-27, wherein the angle information regarding wireless transmissions between the at least one wireless network node and the UE includes an expected downlink angle of arrival (AoA) expected from the at least one wireless network node; and the one or more processors are also configured to determine the positioning of the UE based on the expected downlink AoA.

[0226] Clause 29. A UE according to any one of clauses 26-28, wherein, in order to obtain the angle information, the one or more processors are configured to: receive an optimal transmit beam, a receive beam, or a combination thereof from the at least one wireless network node to communicate with the at least one wireless network node; and the angle information is determined based on optical sensing data obtained using a sensor of the at least one wireless network node and optical sensor data obtained using the sensor of the UE.

[0227] Clause 30. A UE according to any one of clauses 26-29, wherein the one or more processors are also configured to receive optical sensing data obtained using sensors of the at least one wireless network node; wherein: the network entity includes the at least one wireless network node; and in order to obtain the angle information, the one or more processors are configured to determine the angle information about the wireless transmission between the at least one wireless network node and the UE based on the optical sensing data obtained using the sensors of the at least one wireless network node and the optical sensing data obtained using the sensors of the UE.

Claims

1. A method for assisting positioning of a user equipment (UE), the method comprising: Transmitting first configuration information to the UE, where the first configuration information indicates how to obtain optical sensing data using the UE; receiving the optical sensing data from the UE; as well as Angular information regarding wireless transmissions between at least one wireless network node and the UE is determined based at least on the optical sensing data received from the UE.

2. The method according to claim 1, further comprising transmitting second configuration information to the at least one wireless network node, the second configuration information indicating how to obtain optical sensing data using the at least one wireless network node; The determining of the angle information is further based on the optical sensing data received from the at least one wireless network node. 3 . The method of claim 1 , further comprising determining an expected positioning metric based at least on the angular information regarding wireless transmissions between the at least one radio network node and the UE.

4. The method according to claim 3, wherein: The expected positioning metric comprises an uplink angle of arrival (AoA) expected from the UE or a downlink AoA expected from the at least one radio network node; and The method also includes transmitting the uplink AoA to the at least one radio network node, the uplink AoA being configured for use with positioning or beamforming with the UE; or transmitting the downlink AoA to the UE, the downlink AoA being configured for use with positioning or beamforming with the UE.

5. The method of claim 2, wherein the optical sensing data received from the UE includes first visual information sensed by an image capture device associated with the UE, and the optical sensing data received from the at least one wireless network includes second visual information sensed by an image capture device associated with the at least one wireless network node.

6. The method of claim 5, wherein determining the angle information about the wireless transmission between the at least one radio network node and the UE comprises: determining a relative orientation between the image capture device associated with the at least one wireless network node and the image capture device associated with the UE based on one or more key points of the first visual information and one or more key points of the second visual information; as well as Based on the relative orientation, a vector between the at least one radio network node and the UE is determined, wherein the angle information is determined based on the vector.

7. The method according to claim 6, wherein: The first configuration information includes a method for detecting the one or more key points of the first visual information, a feature descriptor for the one or more key points of the first visual information, an amount of the one or more key points of the first visual information, and a first parameter associated with the image capture device associated with the UE, or a combination thereof; and The second configuration information includes a method for detecting the one or more key points of the second visual information, a feature descriptor for the one or more key points of the second visual information, a quantity of the one or more key points of the second visual information, and a second parameter associated with the image capture device associated with the at least one wireless network node, or a combination thereof.

8. The method according to claim 7, wherein: The first parameter comprises an indication of whether one or more parameters of the image capture device associated with the UE are static or dynamic; The second parameter comprises an indication of whether one or more parameters of the image capture device associated with the at least one wireless network node are static or dynamic; and Determining the relative orientation is also based on the first parameter and the second parameter.

9. The method according to claim 6, wherein: The first configuration information includes a request to associate the one or more key points of the first visual information with a classification, and the second configuration information includes a request to associate pixels within the one or more key points of the second visual information with a classification; and Determining the relative orientation is also based on the classification.

10. The method according to claim 5, wherein: The first configuration information further includes a first time interval for sensing the first visual information; The second configuration information further includes a second time interval for sensing the second visual information; and The second time interval is shorter than the first time interval.

11. The method according to claim 2, further comprising: receiving subsequent optical sensing data from the UE but not from the at least one radio network node; as well as Subsequent angular information regarding wireless transmissions between the at least one radio network node and the UE is determined based on the optical sensing data received from the at least one radio network node and the subsequent optical sensing data from the UE.

12. The method of claim 3, further comprising selecting a subset of transmit beams or a subset of receive beams for the at least one radio network node and the UE based on the expected positioning metric.

13. The method of claim 1, further comprising receiving a request for assistance data from the UE, wherein transmitting the first configuration information is in response to the request for assistance data.

14. A device for a wireless communication network, the device comprising: One or more network interfaces; Memory; as well as one or more processors communicatively coupled to the one or more network interfaces and the memory and configured to: transmitting first configuration information to a user equipment (UE), the first configuration information indicating how to obtain optical sensing data using the UE; receiving the optical sensing data from the UE; as well as Angular information regarding wireless transmissions between at least one wireless network node and the UE is determined based at least on the optical sensing data received from the UE.

15. The apparatus of claim 14, wherein the one or more processors are further configured to transmit second configuration information to the at least one wireless network node, the second configuration information indicating how to obtain optical sensing data using the at least one wireless network node; and The determining of the angle information is further based on the optical sensing data received from the at least one wireless network node.

16. The apparatus of claim 14, wherein the one or more processors are further configured to determine an expected positioning metric based at least on the angular information regarding wireless transmissions between the at least one radio network node and the UE; in: The expected positioning metric comprises an uplink angle of arrival (AoA) expected from the UE or a downlink AoA expected from the at least one radio network node; and The one or more processors are further configured to transmit the uplink AoA to the at least one radio network node, the uplink AoA being configured for use with positioning or beamforming with the UE, or to transmit the downlink AoA to the UE, the downlink AoA being configured for use with positioning or beamforming with the UE.

17. The device according to claim 15, wherein: The optical sensing data received from the UE includes first visual information sensed by an image capture device associated with the UE, and the optical sensing data received from the at least one wireless network includes second visual information sensed by an image capture device associated with the at least one wireless network node; and In order to determine the angle information about the wireless transmission between the at least one radio network node and the UE, the one or more processors are further configured to: determining a relative orientation between the image capture device associated with the at least one wireless network node and the image capture device associated with the UE based on the one or more key points of the first visual information and the one or more key points of the second visual information; and determining a vector between the at least one radio network node and the UE based on the relative orientation; and The angle information determined based on the vector is determined.

18. The device according to claim 17, wherein: The first configuration information includes a method for detecting the one or more key points of the first visual information, a feature descriptor for the one or more key points of the first visual information, an amount of the one or more key points of the first visual information, and a first parameter associated with the image capture device associated with the UE, or a combination thereof; and The second configuration information includes a method for detecting the one or more key points of the second visual information, a feature descriptor for the one or more key points of the second visual information, a quantity of the one or more key points of the second visual information, and a second parameter associated with the image capture device associated with the at least one wireless network node, or a combination thereof.

19. The apparatus of claim 14, wherein the one or more processors are further configured to: determining an expected positioning metric based at least on the angular information regarding wireless transmissions between the at least one radio network node and the UE; and A subset of transmit beams or a subset of receive beams of the at least one radio network node and the UE is selected based on the expected positioning metric.

20. The apparatus of claim 14, the one or more processors further configured to receive a request for assistance data from the UE, and to transmit the first configuration information in response to the request for assistance data.

21. A method for beam alignment with a user equipment (UE), the method comprising: receiving first configuration information from a network entity, the first configuration information indicating how to obtain optical sensing data using the UE; Obtaining the optical sensing data using a sensor of the UE; as well as Based at least on the optical sensing data, angle information about wireless transmissions between at least one wireless network node and the UE is obtained.

22. The method of claim 21, further comprising transmitting the obtained optical sensing data to the network entity, the network entity comprising a location server; Obtaining the angle information includes receiving the angle information about the wireless transmission between the at least one wireless network node and the UE from the location server, and the angle information is determined based on optical sensing data obtained using a sensor of the at least one wireless network node and the optical sensing data obtained using the sensor of the UE.

23. The method of claim 22, wherein: The angular information regarding wireless transmissions between the at least one radio network node and the UE comprises an expected downlink angle of arrival (AoA) expected from the at least one radio network node; and The method also includes determining a location of the UE based on the expected downlink AoA.

24. The method of claim 21, wherein: Obtaining the angle information includes receiving an optimal transmit beam, receive beam, or combination thereof from the at least one wireless network node to communicate with the at least one wireless network node; The angle information is determined based on optical sensing data obtained using a sensor of the at least one radio network node and the optical sensing data obtained using the sensor of the UE.

25. The method of claim 21, further comprising receiving optical sensing data obtained using a sensor of the at least one wireless network node; in: The network entity comprises the at least one radio network node; and Obtaining the angle information comprises determining the angle information about wireless transmissions between the at least one radio network node and the UE based on the optical sensing data obtained using the sensor of the at least one radio network node and based on the optical sensing data obtained using the sensor of the UE.

26. A user equipment (UE), the user equipment (UE) comprising: One or more network interfaces; Memory; sensor; as well as one or more processors communicatively coupled to the one or more network interfaces, the sensor, and the memory and configured to: receiving first configuration information from a network entity, the first configuration information indicating how to obtain optical sensing data using the UE; obtaining the optical sensing data using the sensor of the UE; as well as Based at least on the optical sensing data, angle information about wireless transmissions between at least one wireless network node and the UE is obtained.

27. The UE of claim 26, wherein the one or more processors are further configured to transmit the obtained optical sensing data to the network entity, the network entity comprising a location server; and in, In order to obtain the angle information, the one or more processors are configured to: The angle information about the wireless transmission between the at least one wireless network node and the UE is received from the location server, and the angle information is determined based on the optical sensing data obtained using the sensor of the at least one wireless network node and the optical sensing data obtained using the sensor of the UE.

28. The UE according to claim 27, wherein: The angular information regarding wireless transmissions between the at least one radio network node and the UE comprises an expected downlink angle of arrival (AoA) expected from the at least one radio network node; and The one or more processors are further configured to determine a location of the UE based on the expected downlink AoA.

29. The UE according to claim 26, wherein: In order to obtain the angle information, the one or more processors are configured to: receiving an optimal transmit beam, receive beam, or combination thereof from the at least one wireless network node to communicate with the at least one wireless network node; The angle information is determined based on optical sensing data obtained using a sensor of the at least one radio network node and optical sensor data obtained using the sensor of the UE.

30. The UE of claim 26, wherein the one or more processors are further configured to receive optical sensing data obtained using a sensor of the at least one wireless network node; in: The network entity comprises the at least one radio network node; and In order to obtain the angle information, the one or more processors are configured to determine the angle information about the wireless transmission between the at least one wireless network node and the UE based on the optical sensing data obtained using the sensor of the at least one wireless network node and the optical sensing data obtained using the sensor of the UE.

Images