Estimating the location of a user equipment (UE)

Abstract

A method for estimating a location of a target UE. The method includes determining a set of candidate locations for the target UE. The method also includes, for each candidate location in the set of candidate locations, obtaining a simulated vector containing one or more simulated signal characteristic values, SCVs, thereby obtaining a set of simulated vectors where each simulated vector contains one or more simulated SCVs (SSCVs). The method also includes generating a non-simulated vector containing one or more non-simulated SCVs obtained based on one or more signals transmitted by the target UE and / or one or more signals received at the target UE. The method further includes selecting a candidate location from the set of candidate locations based on the set of simulated vectors and the non-simulated vector.

Description

ESTIMATING THE LOCATION OF A USER EQUIPMENT (UE)TECHNICAL FIELD

[0001] Disclosed are embodiments related to estimating the location of user equipment(UE).BACKGROUND

[0002] Digital Twins and 3D Maps

[0003] A digital twin is a digital representation of a physical object or environment (e.g., a city, a factory, a neighborhood, an office, etc.). A digital representation of a physical environment typically takes the form of a digital, three-dimensional (3D) map of the environment.

[0004] When a UE has a camera, a 3D map of the environment in which the UE is present can be used together with images captured by the UE’s camera to estimate the location of the UE within the environment. As used herein, a UE is any mobile device capable of wireless communication with another device. Examples of a UE include: a smartphone, a tablet, a laptop, a sensor, a robot, a vehicle, a drone, a mobile appliance, etc.

[0005] The localization of a UE (e.g., a mobile phone or robot) in respect to a 3D map is a challenging problem that sits at the core of many augmented reality and mapping and navigation applications. The problem is further complicated if the UE is in an area impacted by a natural disaster. In the case of mobile network infrastructure being impacted, radio positioning based on 5G based positioning or Wi-Fi based positioning might not be applicable.

[0006] Visual based positioning of a UE

[0007] A UE with a camera can capture images of the environment in which the UE is located. An image feature detector then can be used to detect features within the images and compare the detected features to features within a 3D map of the environment. Detected features are represented by their pixel coordinates (u, v) within the image and the corresponding descriptor: D = [d1(d2, ••• , dN],

[0008] The descriptor D (vector with typical dimensionality 64 or 128) captures local visual statistics in the vicinity of the feature point (u, v). Accurate key point matching is achieved by calculating distance, e.g., Euclidean norm, between their descriptors: e^ =||Dj — Dj || = / n(din—djn) > where feature descriptor D;is obtained based on the current camera location and descriptors Dj is stored in the 3D map.

[0009] The way UE positions itself in respect to a 3D map is to find best match between feature descriptor Dj and descriptors Dj stored in the 3D map.SUMMARY

[0010] Certain challenges presently exist. For instance, in certain environments (e.g., self-similar visual environments) images captured by a UE at one location within the environment may be similar to images captured by the UE at one or more other locations in the environment. In this case, the extracted local visual cues may not be able to uniquely localize the UE in a 3D map of the environment. The reason is that image feature descriptors D obtained from the UEs’ camera are equally well mapped to several sets of descriptors Dj in various parts of the 3D map. The pure visual based solution cannot resolve the ambiguity caused by multiple similar candidate locations, as they all result in a small errorindicating equality good feature match. This ambiguity in the visual based positioning could be resolved by means of triangulation from several radio access points. Such approach, however, assumes an availability of radio infrastructure for obtaining these positioning measurements. This might not be the case in certain geographical areas, as well in the case of natural disaster that could impact the existing radio infrastructure.

[0011] Accordingly, in one aspect there is provided a method for estimating a location of a target UE. The method includes determining a set of candidate locations for the target UE. The method also includes, for each candidate location in the set of candidate locations, obtaining a simulated vector containing one or more simulated signal characteristic values, SCVs, thereby obtaining a set of simulated vectors where each simulated vector contains one or more simulated SCVs (SSCVs). The method also includes generating a non-simulated vector containing one or more non-simulated SCVs obtained based on one or more signals transmitted by the target UEand / or one or more signals received at the target UE. The method further includes selecting a candidate location from the set of candidate locations based on the set of simulated vectors and the non-simulated vector.

[0012] In another aspect there is provided a computer program comprising instructions which when executed by processing circuitry of an apparatus causes the apparatus to perform any of the methods disclosed herein. In one embodiment, there is provided a carrier containing the computer program wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium. In another aspect there is provided an apparatus that is configured to perform the methods disclosed herein. The apparatus may include memory and processing circuitry coupled to the memory.

[0013] An advantage of the embodiments disclosed herein is that they overcome the above identified “ambiguity” problem, thereby allowing for a visual based positioning of the UE even if the UE is in a self-similar visual environment. Thus, the embodiments provide an enhanced localization service that can be used in, for example, a rescue situation following a natural disaster where a rescuer with a support UE is searching for the owner of a target UE.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings, which are incorporated herein and form part of the specification, illustrate various embodiments.

[0015] FIG. 1 illustrates an environment according to an embodiment.

[0016] FIG. 2 is a flowchart illustrating a process according to an embodiment.

[0017] FIG. 3 illustrates a set of candidate locations.

[0018] FIG. 4 is a block diagram of a UE according to an embodiment.

[0019] FIG. 5 is a block diagram of a network node according to an embodiment.DETAILED DESCRIPTION

[0020] FIG. 1 illustrates an environment 100 according to an embodiment. Two UEs, UE 101 and UE 102, are located within environment 100. UE 101 and UE 102 have the capability to communicate with each other using a device-to-device (D2D) link 150 (e.g., fifth generationRECTIFIED SHEET (Rule 91)(5G) sidelink). Further, in some embodiments, UE 102 has the capability to communicate with a network node 104.

[0021] This disclosure describes a method for estimating the location of UE 101, which is referred to as the “target UE,” with support from UE 102, which is referred to as the “support UE.” The estimated location of the target UE is based on visual cues obtained through the UE 101 ’s camera. Ambiguities in localization, caused by self-similarities in environment 101, are resolved with the help of the support UE. The target UE (in need of positioning itself or to be positioned by a rescue team) might not be even capable of performing computer vision tasks, but at least is capable of capturing one or more images of its surroundings and communicating with the support UE (e.g., via the D2D link). In one embodiment, the ambiguities in the visual-based positioning are resolved by the support UE through an analysis measurements of signals sent to (or received from) the target UE via the D2D link and simulated radio parameters based on candidate locations.

[0022] Example Localization Process

[0023] FIG. 2 is a flow chart illustrating a process 200, according to an embodiment, for estimating the location of the target UE. Process 200 may begin in step s202.

[0024] Step s202 comprises determining set of candidate locations for the target UE. In one embodiment, each location in the set of candidate locations is a point in a 3D space (e.g., a point in the 3D map), e.g., if the set of candidate locations consists of four locations A, B, C and D, then A = (XA, YA, ZA), B = (XB, YB, ZB), C = (Xc, Yc, Zc) , D = (XD, YD, ZD). However, in the general case one could consider the sensor pose, which includes position and orientation, i.e., 6-dim vector with 3 spatial coordinates and 3 angles. FIG. 3 illustrates the four candidate locations (A, B, C, D) for the target UE. In FIG. 3, the location S represents the location of the support UE.

[0025] Step s204 comprises, for each candidate location in the set of candidate locations, obtaining a simulated vector (sR) containing one or more simulated signal characteristic values (SCVs), thereby obtaining a set of simulated vectors where each simulated vector contains one or more simulated SCVs (SSCVs). In one embodiment, before the simulated vectors are obtained, the location of the support UE (S) must first be determined. The support UE may determine its location using its camera and / or other location determination method known in the art.

[0026] In one embodiment, a simulated vector for a candidate location is obtained using a radio propagation model and performing a simulation on how a simulated signal would traverse the physical environment from location S to the candidate location (or from the candidate location to location S). FIG. 3 illustrates simulated signals 301, 302, 303, and 304 received at the support UE from a simulated device at locations A, B, C, and D, respectively. There exist multiple radio propagation models that may be employed, including: free space, one-slope model, dual-slope model, dominant path model, and ray tracing model. Depending on its capabilities, such a model can capture estimation of radio phenomena, from a transmitter to the receiver.

[0027] The radio propagation model is executed with 1) the available 3D map, 2) known location of the support UE (S) and 3) each of the candidate locations of the target UE. For every candidate target location, the radio simulator returns a vector of simulated signal characteristic values (i.e., a “simulated vector”). For example, simulated vectors SRA, SRB, SRC, and SRD, are returned for candidate locations A, B, C, and D, respectively, and each simulated vector contains a set of simulated signal characteristic values. That is, as an example:SRA = [SSCVIA, SSCV2A, SSCV3A];SRB = [SSCVIB, SSCV2B, SSCV3B]; sRc = [SSCVlc, SSCV2c, SSCV3c]; and SRD - [SSCV 1 D, SSCV2D, SSCV3D].

[0028] In one embodiment, SSCVli for i = A, B, C, D, is a SSCV of a first type (e.g., received signal strength indicator (RSSI)); SSCV2i for i = A, B, C, D, is a SSCV of a second type (e.g., angle-of-arrival (AoA)); and SSCV3i for i = A, B, C, D, is a SSCV of a third type (e.g., time of flight (ToF)).

[0029] Step s206 comprises generating a non-simulated vector (mR) containing one or more non-simulated SCVs obtained based on one or more actual signals transmitted by the target UE and received at the support UE and / or one or more signals transmitted by the support UE and received at the target UE. In one embodiment, these one or more signals are transmitted over the D2D link between the target UE and the support UE. As an example, mR = [MR1 , MR2, MR3], whereMR1 = RSSI;MR2 = AoA; andMR2 = ToF.More generally, mR = [MR1, MR2., ..., MRN).

[0030] Step s208 comprises selecting a candidate location from the set of candidate locations based on the set of simulated vectors (sRi) and the non-simulated vector (mR). The selected candidate location is then the estimated location for the target UE. That is, the closeness between the captured signal (e.g., radio signal) measurements and simulated signals, determines the most likely location of the target UE. In other words, out of the set of candidate locations (e.g., A, B, C, D), the location for which simulated radio sR is closest to mR, is selected as the true one. The closeness, could be measured by means of Euclidean distance, weighted Euclidean distance, normalized cross correlation, etc. As an example, in the case of 4 candidate locations, 4 distances are calculated, and the smallest one determines the most likely location of the target UE.In other words, the most likely location (LL) of the target UE is determined as the position providing best match among d(sRA, mR) ... d(sRD, mR), i.e.,LL = argmin ( d(sRe, mR) ) ee{A,B,c,D}v 7

[0031] In one embodiment, with respect to step s202 the set of candidate locations is determined using the below described steps.

[0032] Step 1 : The target UE collects images of its surrounding physical environment.

[0033] Step 2: The images are processed to produce a feature descriptor DrThe images may be processed by the target UE, the support UE, or network node 104. For example, in one embodiment the target UE may provide the images to the support UE and the support UE uses the images to produce Dx. In another embodiment, the target UE may provide the images to the support UE via the D2D link, and the support UE uploads the images to network node 104 (e.g., via radio access network (RAN), and network node 104 uses the images to produce Dx.

[0034] Step 3: After Dxis created, Dxis compared to the features description within a 3D map of the environment to determine if any of the features description within the 3D map match the created feature descriptor. As noted above, a feature descriptor in the 3D maps matches the created feature descriptor by calculating an error value (e.g., the Euclidean norm) indicating the similarity between Dxand the feature descriptor in the 3D map and determining if the calculated error value satisfies a condition (e.g., is less than or less than or equal to an error threshold). If the calculated error value satisfies the condition, then the feature descriptor in the 3D map matches Dx. Each feature descriptor in the 3D map is associated with a unique location. Hence, by determining the set of one or more feature descriptor in the 3D map that are “match” Dx, one can determine the set of candidate locations for the target UE (i.e., each location in the set of candidate location is a location with which one of the matching features descriptors from the 3D map is associated).

[0035] In some embodiments, the set of candidate locations includes at least a first candidate location and a second candidate location, the set of simulated vectors includes at least a first simulated vector containing one or more simulated SCVs based on the first candidate location and a second simulated vector containing one or more simulated SCVs based on the second candidate location, and selecting a candidate location from the set of candidate locations comprises: i) determining a first similarity value indicating a similarity between the nonsimulated vector with the first simulated vector; ii) determining a second similarity value indicating a similarity between the non-simulated vector with the second simulated vector; and iii) comparing first similar value with the second similarity value.

[0036] In some embodiments, generating the non-simulated vector comprises: a support UE, at a known location, receiving one or more signals transmitted by the target UE andgenerating one or more of the non-simulated SCVs based on one or more measurements of the one or more signals transmitted by the target UE, and / or the target UE receiving one or more signals transmitted by the support UE and generating one or more of the non-simulated SCVs based on one or more measurements of the one or more signals transmitted by the support UE.

[0037] In some embodiments, the target UE has a camera, and determining the set of candidate locations for the target UE comprises: obtaining a set of one or more images captured by the camera of the target UE; and determining the set of candidate locations based on the set of images.

[0038] In some embodiments, obtaining the set of images captured by the camera of the target UE comprises the support UE receiving the set of images from the target UE via a device- to-device (e.g., 5G sidelink) connection established between the target UE and the support UE. In some embodiments process 200 also includes the support UE transmitting the set of images to a remove server, wherein the remote server is configured to determine the set of candidate locations using the set of images.

[0039] In some embodiments, the operation of obtaining the set of simulated vectors is performed by the remote server using set of candidate locations and a 3D map of an area in which the target UE is located. In some embodiments the process also includes the remote server transmitting to the support UE information identifying the selected candidate location.

[0040] In some embodiments, the support UE is configured to determine the set of candidate locations using the set of images, while in other embodiments the target UE is configured to determine the set of candidate locations using the set of images.

[0041] In one embodiment, all of the steps of process 200 are performed by network node 104. In this embodiment, the support UE, which may have limited computational power (e.g., the support UE is a low-end handheld device or drone surveying the environment), receives images captured by the camera of the target UE and forwards those images to network node 104. Additionally, the support UE sends to the network node the signal measurements (e.g., the vector mR), and also sends to network node 104 information identifying the support UE’s location (if network node 104 does not already have this information).

[0042] In another embodiment, some or all of the steps are performed the target UE and / or the support UE. In these embodiments, one or more of the target UE or support UE has sufficient computational power to perform feature extraction and execute the decision logic. In this case, the target UE and / or support UE could be a high-end handheld device capable of performing complex computer vision tasks.

[0043] FIG. 4 is a block diagram of a UE (e.g., UE 101 or UE 102), according to some embodiments. As shown in FIG. 4, the UE may comprise: processing circuitry (PC) 402, which may include one or more processors (P) 455 (e.g., one or more general purpose microprocessors and / or one or more other processors, such as an application specific integrated circuit (ASIC), field-programmable gate arrays (FPGAs), and the like); communication circuitry 448, which is coupled to an antenna arrangement 449 comprising one or more antennas and which comprises a transmitter (Tx) 445 and a receiver (Rx) 447 for enabling the UE to transmit data and receive data (e.g., wirelessly transmit / receive data); and a storage unit (a.k.a., “data storage system”) 408, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 402 includes a programmable processor, a computer readable storage medium (CRSM) 442 may be provided. CRSM 442 may store a computer program (CP) 443 comprising computer readable instructions (CRI) 444. CRSM 442 may be a non-transitory computer readable medium, such as, magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory, flash memory), and the like. In some embodiments, the CRI 444 of computer program 443 is configured such that when executed by PC 402, the CRI causes the UE to perform steps described herein (e.g., steps described herein with reference to the flow charts). In other embodiments, the UE may be configured to perform steps described herein without the need for code. That is, for example, PC 402 may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and / or software

[0044] FIG. 5 is a block diagram of network node 104, according to some embodiments. As shown in FIG. 5, network node 104 may comprise: processing circuitry (PC) 502, which may include one or more processors (P) 555 (e.g., one or more general purpose microprocessors and / or one or more other processors, such as an application specific integrated circuit (ASIC), field-programmable gate arrays (FPGAs), and the like), which processors may be co-located in a single housing or in a single data center or may be geographically distributed (e.g., network node104 may be a distributed computing apparatus comprising two or more computers or a monolithic computing apparatus consisting of a single computer); at least one network interface 548 (e.g., a physical interface or air interface) comprising a transmitter (Tx) 545 and a receiver (Rx) 547 for enabling network node 104 to transmit data to and receive data from other nodes connected to network 110 (e.g., an Internet Protocol (IP) network) to which network interface 548 is connected (physically or wirelessly) (e.g., network interface 548 may be coupled to an antenna arrangement comprising one or more antennas for enabling network node 104 to wirelessly transmit / receive data); and a storage unit (a.k.a., “data storage system”) 508, which may include one or more non-volatile storage devices and / or one or more volatile storage devices. In embodiments where PC 502 includes a programmable processor, a computer readable storage medium (CRSM) 542 may be provided. CRSM 542 may store a computer program (CP) 543 comprising computer readable instructions (CRI) 544. CRSM 542 may be a non-transitory computer readable medium, such as, magnetic media (e.g., a hard disk), optical media, memory devices (e.g., random access memory, flash memory), and the like. In some embodiments, the CRI 544 of computer program 543 is configured such that when executed by PC 502, the CRI causes network node 104 to perform steps described herein (e.g., steps described herein with reference to the flow charts). In other embodiments, network node 104 may be configured to perform steps described herein without the need for code. That is, for example, PC 502 may consist merely of one or more ASICs. Hence, the features of the embodiments described herein may be implemented in hardware and / or software.

[0045] While various embodiments are described herein, it should be understood that they have been presented by way of example only, and not limitation. Thus, the breadth and scope of this disclosure should not be limited by any of the above-described exemplary embodiments. Moreover, any combination of the above-described elements in all possible variations thereof is encompassed by the disclosure unless otherwise indicated herein or otherwise clearly contradicted by context.

[0046] As used herein transmitting a message “to” or “toward” an intended recipient encompasses transmitting the message directly to the intended recipient or transmitting the message indirectly to the intended recipient (i.e., one or more other nodes are used to relay the message from the source node to the intended recipient). Likewise, as used herein receiving a message “from” a sender encompasses receiving the message directly from the sender orindirectly from the sender (i.e., one or more nodes are used to relay the message from the sender to the receiving node). Further, as used herein “a” means “at least one” or “one or more.”

[0047] Additionally, while the processes described above and illustrated in the drawings are shown as a sequence of steps, this was done solely for the sake of illustration. Accordingly, it is contemplated that some steps may be added, some steps may be omitted, the order of the steps may be re-arranged, and some steps may be performed in parallel.

Claims

CLAIMS1. A method (200) for estimating a location of a target user equipment, UE (101), the method comprising: determining (s202) a set of candidate locations for the target UE; for each candidate location in the set of candidate locations, obtaining (s204) a simulated vector containing one or more simulated signal characteristic values, SCVs, thereby obtaining a set of simulated vectors where each simulated vector contains one or more simulated SCVs (SSCVs); generating (s206) a non-simulated vector containing one or more non-simulated SCVs obtained based on one or more signals transmitted by the target UE and / or one or more signals received at the target UE; and selecting (s208) a candidate location from the set of candidate locations based on the set of simulated vectors and the non-simulated vector.

2. The method of claim 1 , wherein the set of candidate locations includes at least a first candidate location and a second candidate location, the set of simulated vectors includes at least a first simulated vector containing one or more simulated SCVs based on the first candidate location and a second simulated vector containing one or more simulated SCVs based on the second candidate location, and selecting a candidate location from the set of candidate locations comprises: determining a first similarity value indicating a similarity between the non-simulated vector with the first simulated vector; determining a second similarity value indicating a similarity between the nonsimulated vector with the second simulated vector; and comparing first similar value with the second similarity value.

3. The method of claim 1 or 2, wherein generating the non-simulated vector comprises:a support UE (102), at a known location, receiving one or more signals transmitted by the target UE and generating one or more of the non-simulated SCVs based on one or more measurements of the one or more signals transmitted by the target UE, and / or the target UE receiving one or more signals transmitted by the support UE and generating one or more of the non-simulated SCVs based on one or more measurements of the one or more signals transmitted by the support UE.

4. The method of claim 1, 2, or 3, wherein the target UE has a camera, and determining the set of candidate locations for the target UE comprises: obtaining a set of one or more images captured by the camera of the target UE; and determining the set of candidate locations based on the set of images.

5. The method of claim 4 when dependent on claim 3, wherein obtaining the set of images captured by the camera of the target UE comprises the support UE receiving the set of images from the target UE via a device-to-device (e.g., 5G sidelink) connection established between the target UE and the support UE.

6. The method of claim 5, further comprising the support UE transmitting the set of images to a remove server, wherein the remote server is configured to determine the set of candidate locations using the set of images.

7. The method of claim 6, wherein the operation of obtaining the set of simulated vectors is performed by the remote server using set of candidate locations and a 3D map of an area in which the target UE is located.

8. The method of claim 7, further comprising the remote server transmitting to the support UE information identifying the selected candidate location.

9. The method of any one of claims 1-8, wherein the support UE is configured to determine the set of candidate locations using the set of images.

10. The method of any one of claims 1-8, wherein the target UE is configured to determine the set of candidate locations using the set of images.

11. An apparatus (101, 102, 104) for estimating a location of a target user equipment, UE (101), the apparatus being configured to: determine (s202) a set of candidate locations for the target UE; for each candidate location in the set of candidate locations, obtain (s204) a simulated vector containing one or more simulated signal characteristic values, SCVs, thereby obtaining a set of simulated vectors where each simulated vector contains one or more simulated SCVs (SSCVs); generate (s206) a non-simulated vector containing one or more non-simulated SCVs obtained based on one or more signals transmitted by the target UE and / or one or more signals received at the target UE; and select (s208) a candidate location from the set of candidate locations based on the set of simulated vectors and the non-simulated vector.

12. The apparatus of claim 11, wherein the set of candidate locations includes at least a first candidate location and a second candidate location, the set of simulated vectors includes at least a first simulated vector containing one or more simulated SCVs based on the first candidate location and a second simulated vector containing one or more simulated SCVs based on the second candidate location, and selecting a candidate location from the set of candidate locations comprises: determining a first similarity value indicating a similarity between the non-simulated vector with the first simulated vector; determining a second similarity value indicating a similarity between the nonsimulated vector with the second simulated vector; and comparing first similar value with the second similarity value.

13. The apparatus of claim 11 or 12, wherein generating the non-simulated vector comprises: a support UE (102), at a known location, receiving one or more signals transmitted by the target UE and generating one or more of the non-simulated SCVs based on one or more measurements of the one or more signals transmitted by the target UE, and / or the target UE receiving one or more signals transmitted by the support UE and generating one or more of the non-simulated SCVs based on one or more measurements of the one or more signals transmitted by the support UE.

14. The apparatus of claim 11, 12, or 13, wherein the target UE has a camera, and determining the set of candidate locations for the target UE comprises: obtaining a set of one or more images captured by the camera of the target UE; and determining the set of candidate locations based on the set of images.

15. The apparatus of claim 14 when dependent on claim 13, wherein obtaining the set of images captured by the camera of the target UE comprises the support UE receiving the set of images from the target UE via a device-to-device (e.g., 5G sidelink) connection established between the target UE and the support UE.

16. The apparatus of claim 15, further comprising the support UE transmitting the set of images to a remove server, wherein the remote server is configured to determine the set of candidate locations using the set of images.

17. The apparatus of claim 16, wherein the operation of obtaining the set of simulated vectors is performed by the remote server using set of candidate locations and a 3D map of an area in which the target UE is located.

18. The apparatus of claim 17, further comprising the remote server transmitting to the support UE information identifying the selected candidate location.

19. The apparatus of any one of claims 11-18, wherein the support UE is configured to determine the set of candidate locations using the set of images.

20. The apparatus of any one of claims 11-18, wherein the target UE is configured to determine the set of candidate locations using the set of images.

21. A computer program (443, 543) comprising instructions (444, 544) which when executed by processing circuitry (402, 502) of an apparatus (101, 102, 104) causes the apparatus to perform the method of any one of claims 1-10.

22. A carrier containing the computer program of claim 11, wherein the carrier is one of an electronic signal, an optical signal, a radio signal, and a computer readable storage medium (442, 542).

Images