Radio frequency assisted management of partially occluded regions of interest for visual positioning

Abstract

Techniques for visual localization are disclosed. In an aspect, a localization entity determines a region of interest in one or more images obtained from one or more cameras, where the region of interest corresponds to at least a portion of a target object detected in the one or more images; obtains a set of key visual features associated with the target object based on the one or more images; estimates a three-dimensional dimension of the target object based at least in part on a type of the target object; and stores the set of key visual features and the three-dimensional dimension of the target object for subsequent localization of the target object.

Description

Background Technology

[0001] 1. Technical Field

[0002] All aspects of this disclosure relate to wireless technology.

[0003] 2. Related technical descriptions

[0004] Wireless communication systems have evolved through many generations, including first-generation analog radiotelephone service (1G), second-generation (2G) digital radiotelephone service (including transitional 2.5G and 2.75G networks), third-generation (3G) high-speed data, wireless services with internet capabilities, and fourth-generation (4G) services (e.g., Long Term Evolution (LTE) or WiMax). Currently, many different types of wireless communication systems are in use, including cellular systems and Personal Communication Services (PCS) systems. Known examples of cellular systems include cellular analog Advanced Mobile Phone Systems (AMPS), as well as digital cellular systems based on Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM), and others.

[0005] The fifth-generation (5G) wireless standard, known as New Radio (NR), delivers higher data transfer speeds, more connections, better coverage, and other improvements. According to the Next Generation Mobile Networks Alliance (NGC), the 5G standard is designed to provide higher data rates, more accurate positioning (e.g., based on Positioning Reference Signals (RS-P), such as downlink, uplink, or sidelink Positioning Reference Signals (PRS)), and other technological enhancements compared to previous standards. These enhancements, along with the use of higher frequency bands, advancements in the PRS process and technology, and the high-density deployment of 5G, enable high-accuracy positioning based on 5G. Summary of the Invention

[0006] The following is a simplified summary of the invention relating to one or more aspects disclosed herein. Therefore, this summary should not be considered an exhaustive overview relating to all conceived aspects, nor should it be considered to identify key or decisive elements relating to all conceived aspects or to depict the scope associated with any particular aspect. Thus, the sole purpose of this summary is to present, in a simplified form, certain concepts relating to one or more aspects involving the mechanisms disclosed herein, prior to the detailed description presented below.

[0007] In one aspect, a visual localization method performed by a localization entity includes: determining one or more images obtained from one or more cameras. picture The region of interest in the image, wherein the region of interest corresponds to one or more of the... picture At least a portion of the target object detected in the image; based on the one or more picture The method includes obtaining a set of key visual features associated with the target object; estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and storing the set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0008] In one aspect, a positioning entity includes: one or more memories; one or more transceivers; and one or more processors communicatively coupled to the one or more memories and the one or more transceivers, the one or more processors being configured individually or in combination to: determine one or more [data / images] obtained from one or more cameras. picture The region of interest in the image, wherein the region of interest corresponds to one or more of the... picture At least a portion of the target object detected in the image; based on the one or more picture The method includes obtaining a set of key visual features associated with the target object; estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and storing the set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0009] On one hand, a positioning entity includes: for determining one or more images obtained from one or more cameras. picture The component of the region of interest in the image, wherein the region of interest corresponds to one or more of the components. picture At least a portion of the target object detected in the image; used for... picture The components include: a component for acquiring a set of key visual features associated with the target object; a component for estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and a component for storing the set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0010] In one aspect, a non-transitory computer-readable medium stores computer-executable instructions that, when executed by a positioning entity, cause the positioning entity to perform the following operations: determine one or more images obtained from one or more cameras. picture The region of interest in the image, wherein the region of interest corresponds to one or more of the... picture At least a portion of the target object detected in the image; based on the one or more picture The method includes obtaining a set of key visual features associated with the target object; estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and storing the set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0011] Based on the appendix picture and A detailed description, as well as other objects and advantages associated with the aspects disclosed herein, will be apparent to those skilled in the art. Appendix picture illustrate

[0012] Presenting attached picture To help describe the various aspects of this disclosure, and to provide appendices picture This is for illustrative purposes only and not for limiting any aspect.

[0013] Figure 1 Example wireless communication systems according to various aspects of this disclosure are illustrated.

[0014] Figure 2A , Figure 2B and Figure 2C Example wireless network architectures based on various aspects of this disclosure are illustrated.

[0015] Figures 3A, 3B and Figure 3C It is a simplified framework of several example aspects of components that can be adopted in user equipment (UE), base stations, and network entities and configured to support communications as taught herein. picture .

[0016] Figure 4 This is an example scenario illustrating various aspects of this disclosure. picture One or more targets within the environment may be at least partially occluded by an over-the-top (OTT) camera system monitoring the environment.

[0017] Figure 5 Examples of partial occlusion of a single view according to various aspects of this disclosure are illustrated. picture The impact of visual localization results in the system.

[0018] Figure 6 This illustrates partial occlusion of multiple views according to various aspects of this disclosure. picture The impact of visual localization results in the system picture .

[0019] Figure 7 This is an example scene illustrating the use of three-dimensional (3D) target information to complete the region of interest (RoI) associated with the target, according to various aspects of this disclosure. picture .

[0020] Figure 8 This is an example illustrating outlier RoI detection and removal according to various aspects of this disclosure. picture .

[0021] Figure 9This is an example architecture illustrating radio frequency (RF) assisted management of partially occluded RoIs for visual positioning according to various aspects of this disclosure. picture .

[0022] Figure 10 Example signaling flows for active target registration and 3D feature estimation according to various aspects of this disclosure are illustrated.

[0023] Figure 11 Example signaling flows for outlier RoI detection and removal are illustrated according to various aspects of this disclosure.

[0024] Figure 12 Example methods for visual positioning according to various aspects of this disclosure are illustrated. Detailed Implementation

[0025] Various aspects of this disclosure are described below with reference to the various examples provided for illustrative purposes and related appendices. picture Provided herein. Alternative aspects may be designed without departing from the scope of this disclosure. Additionally, well-known elements of this disclosure will not be described in detail or will be omitted so as not to obscure the relevant details of this disclosure.

[0026] Overall, various aspects involve vision-based localization. Some aspects are more specifically related to radio frequency (RF)-assisted methods for handling partially occluded targets and incomplete regions of interest (RoIs). In some examples, three-dimensional (3D) target information is used to complete the RoI of occluded targets, where knowledge of the target's dimensions in the 3D environment is used to complete the RoI during partial detection. In other examples, RoIs of partially occluded / detected objects are interpreted and marked as outliers, and if these RoIs do not meet an established threshold, they are subsequently removed from the vision localization processing pipeline and localization engine.

[0027] Specific aspects of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages. In some examples, the described techniques can be used to improve vision-based localization by addressing partially occluded RoIs by making the RoI complete based on 3D target information and / or detecting and removing outliers.

[0028] The terms “exemplary” and / or “example” are used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” and / or “example” is not necessarily to be construed as superior to or better than other aspects. Similarly, the term “aspects of this disclosure” does not require that all aspects of this disclosure include the features, advantages, or modes of operation discussed.

[0029] Those skilled in the art will understand that any of a variety of different techniques and methods can be used to represent the information and signals described below. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the following description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof, depending in part on the specific application, in part on the desired design, in part on the corresponding technology, and so on.

[0030] Furthermore, many aspects are described according to a sequence of actions to be performed by elements of, for example, a computing device. It will be appreciated that the various actions described herein can be performed by specific circuitry (e.g., an application-specific integrated circuit (ASIC)), by program instructions executed by one or more processors, or by a combination of both. Additionally, the sequence of actions described herein can be considered to be entirely embodied in any form of non-transitory computer-readable storage medium storing a corresponding set of computer instructions that, when executed, will cause or command the associated processor of the device to perform the functionality described herein. Therefore, various aspects of this disclosure can be embodied in a variety of different forms, all of which are contemplated within the scope of the claimed subject matter. Furthermore, for each aspect described herein, any corresponding form of any such aspect may be described herein as, for example, "logic configured to perform the described actions."

[0031] As used herein, unless otherwise stated, the terms “User Equipment” (UE) and “Base Station” are not intended to be specific or otherwise limited to any particular Radio Access Technology (RAT). Generally, a UE can be any wireless communication device used by a user to communicate over a wireless communication network (e.g., mobile phone, router, tablet computer, laptop computer, consumer asset positioning device, wearable device (e.g., smartwatch, glasses, augmented reality (AR) / virtual reality (VR) headset, etc.), vehicle (e.g., car, motorcycle, bicycle, etc.), Internet of Things (IoT) device, etc.). A UE can be mobile or can (e.g., at certain times) be stationary and can communicate with a Radio Access Network (RAN). As used herein, the term “UE” can be interchangeably referred to as “Access Terminal” or “AT,” “Client Equipment,” “Wireless Equipment,” “Subscriber Equipment,” “Subscriber Terminal,” “Subscriber Station,” “User Terminal” or “UT,” “Mobile Equipment,” “Mobile Terminal,” “Mobile Station,” or variations thereof. Generally, a UE can communicate with a core network via the RAN, and through the core network, a UE can connect to external networks such as the Internet and to other UEs. Of course, other mechanisms for connecting to the core network and / or the Internet are also possible for the UE, such as through wired access networks, wireless local area network (WLAN) networks (e.g., based on the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard, etc.).

[0032] A base station may operate according to one of several RATs to communicate with the UE, depending on the network in which it is deployed, and may alternatively be referred to as an Access Point (AP), Network Node, Node B, Evolved Node B (eNB), Next Generation eNB (ng-eNB), New Radio (NR) Node B (also referred to as gNB or gNodeB), etc. The base station may primarily be used to support the UE's radio access, including supporting data, voice, and / or signaling connections for the supported UE. In some systems, the base station may only provide edge node signaling functions, while in others, it may provide additional control and / or network management functions. The communication link through which the UE can transmit signals to the base station is called an uplink (UL) channel (e.g., reverse traffic channel, reverse control channel, access channel, etc.). The communication link through which the base station can transmit signals to the UE is called a downlink (DL) or forward link channel (e.g., paging channel, control channel, broadcast channel, forward traffic channel, etc.). As used herein, the term "traffic channel (TCH)" may refer to an uplink / reverse traffic channel or a downlink / forward traffic channel.

[0033] The term "base station" can refer to a single physical transmit / receive point (TRP) or multiple physical TRPs that may or may not be co-located. For example, when the term "base station" refers to a single physical TRP, the physical TRP can be the antenna of a base station corresponding to a cell (or several cell sectors) of the base station. When the term "base station" refers to multiple co-located physical TRPs, the physical TRP can be the antenna array of the base station (e.g., as in a multiple-input multiple-output (MIMO) system or where the base station employs beamforming). When the term "base station" refers to multiple non-co-located physical TRPs, the physical TRP can be a distributed antenna system (DAS) (a network of spatially separated antennas connected via a transmission medium to a common source) or a remote radio headend (RRH) (a remote base station connected to a serving base station). Alternatively, a non-co-located physical TRP can be the serving base station from which the UE receives measurement reports and a neighboring base station where the UE is measuring its reference radio frequency (RF) signal. Because, as used herein, a TRP is the point by which a base station transmits and receives radio signals, references to transmitting from or receiving at a base station should be understood to refer to a specific TRP of the base station.

[0034] In some specific implementations supporting UE positioning, the base station may not support the UE's radio access (e.g., it may not support data, voice, and / or signaling connections for the UE), but may instead transmit reference signals to the UE for measurement and / or receive and measure signals transmitted by the UE. Such a base station may be referred to as a positioning beacon (e.g., in the case of transmitting signals to the UE) and / or as a location measurement unit (e.g., in the case of receiving and measuring signals from the UE).

[0035] An “RF signal” refers to an electromagnetic wave of a given frequency that transmits information across the space between a transmitter and a receiver. As used herein, a transmitter may send a single “RF signal” or multiple “RF signals” to a receiver. However, due to the propagation characteristics of RF signals through multipath channels, a receiver may receive multiple “RF signals” corresponding to each transmitted RF signal. The same transmitted RF signal on different paths between the transmitter and receiver may be referred to as a “multipath” RF signal. As used herein, an RF signal may also be referred to as a “wireless signal” or simply a “signal” where the context clearly indicates that the term “signal” refers to a wireless signal or an RF signal.

[0036] Figure 1An example wireless communication system 100 according to various aspects of this disclosure is illustrated. The wireless communication system 100 (which may also be referred to as a wireless wide area network (WWAN)) may include various base stations 102 (labeled "BS") and various UEs 104. Base station 102 may include macro cell base stations (high-power cellular base stations) and / or small cell base stations (low-power cellular base stations). In one aspect, the macro cell base station may include an eNB and / or an ng-eNB (wherein the wireless communication system 100 corresponds to an LTE network), or a gNB (wherein the wireless communication system 100 corresponds to an NR network), or a combination of both, and the small cell base station may include femtocells, picocells, microcells, etc.

[0037] Base station 102 can collectively form a RAN and interface with core network 170 (e.g., evolved packet core (EPC) or 5G core (5GC)) via backhaul link 122, and interface with one or more location servers 172 (e.g., location management function (LMF) or secure user plane location (SUPL) location platform (SLP)) via core network 170. Location server 172 can be part of core network 170 or can be external to core network 170. Location server 172 can be integrated with base station 102. UE 104 can communicate with location server 172 directly or indirectly. For example, UE 104 can communicate with location server 172 via base station 102 currently serving UE 104. UE 104 can also communicate with location server 172 via another path, such as via application server (not shown), via another network, such as via wireless local area network (WLAN) access point (AP) (e.g., AP 150 described below), etc. For signaling purposes, communication between UE 104 and location server 172 can be represented as an indirect connection (e.g., via core network 170, etc.) or a direct connection (e.g., as shown via direct connection 128), wherein, for clarity, from signaling picture Intermediate nodes are omitted (if they exist).

[0038] In addition to other functions, base station 102 may perform functions associated with one or more of the following: transmitting user data, radio channel encryption and decryption, integrity protection, header compression, mobility control functions (e.g., handover, dual connectivity), inter-cell interference coordination, connection establishment and release, load balancing, distribution of non-access stratum (NAS) messages, NAS node selection, synchronization, RAN sharing, multimedia broadcast multicast service (MBMS), subscriber and equipment tracking, RAN information management (RIM), paging, location, and delivery of warning messages. Base stations 102 may communicate with each other directly or indirectly (e.g., via EPC / 5GC) on backhaul link 134, which may be wired or wireless.

[0039] Base station 102 can wirelessly communicate with UE 104. Each base station in base station 102 can provide communication coverage for a corresponding geographic coverage area 110. In one aspect, one or more cells can be supported by base station 102 in each geographic coverage area 110. A “cell” is a logical communication entity used to communicate with a base station (e.g., via a frequency resource, which is referred to as a carrier frequency, component carrier, carrier, frequency band, etc.) and can be associated with an identifier (e.g., Physical Cell Identifier (PCI), Enhanced Cell Identifier (ECI), Virtual Cell Identifier (VCI), Cell Global Identifier (CGI), etc.) used to distinguish cells operating via the same or different carrier frequencies. In some cases, different cells can be configured according to different protocol types that can provide access for different types of UEs (e.g., Machine Type Communication (MTC), Narrowband IoT (NB-IoT), Enhanced Mobile Broadband (eMBB), or other protocol types). Because a cell is supported by a specific base station, the term “cell” can refer to either or both of the logical communication entity and the base station supporting the logical communication entity, depending on the context. Furthermore, since the TRP is typically the physical transmission point of a cell, the terms "cell" and "TRP" can be used interchangeably. In some cases, the term "cell" can also refer to the geographical coverage area of ​​a base station (e.g., a sector), as long as the carrier frequency can be detected and used for communication within a portion of the geographical coverage area 110.

[0040] While the geographic coverage areas 110 of adjacent macro cell base stations 102 may partially overlap (e.g., in handover areas), some areas within geographic coverage areas 110 may substantially overlap with larger geographic coverage areas 110. For example, a small cell base station 102' (labeled "SC" for "small cell") may have a geographic coverage area 110' that substantially overlaps with the geographic coverage areas 110 of one or more macro cell base stations 102. A network that includes both small cell base stations and macro cell base stations can be referred to as a heterogeneous network. A heterogeneous network may also include a home eNB (HeNB) that can provide service to a restricted group referred to as a Closed Subscriber Group (CSG).

[0041] The communication link 120 between base station 102 and UE 104 may include uplink (also known as reverse link) transmission from UE 104 to base station 102 and / or downlink (DL) (also known as forward link) transmission from base station 102 to UE 104. The communication link 120 may use MIMO antenna techniques, including spatial multiplexing, beamforming, and / or transmit diversity. The communication link 120 may use one or more carrier frequencies. Carrier allocation may be asymmetric for the downlink and uplink (e.g., more or fewer carriers may be allocated to the downlink compared to the uplink).

[0042] The wireless communication system 100 may also include a WLAN access point (AP) 150 that communicates with a wireless local area network (WLAN) station (STA) 152 via a communication link 154 in unlicensed spectrum (e.g., 5 GHz). When communicating in unlicensed spectrum, the WLAN STA 152 and / or WLAN AP 150 may perform a free channel assessment (CCA) or listen-before-talk (LBT) process before communication to determine whether the channel is available.

[0043] Small cell base station 102' can operate in licensed and / or unlicensed spectrum. When operating in unlicensed spectrum, small cell base station 102' can employ LTE or NR technology and use the same 5GHz unlicensed spectrum as WLAN AP 150. Small cell base station 102' employing LTE / 5G in unlicensed spectrum can improve the coverage and / or increase the capacity of the access network. NR in unlicensed spectrum may be referred to as NR-U. LTE in unlicensed spectrum may be referred to as LTE-U, Licensed Assisted Access (LAA), or MULTEFIRE. ® .

[0044] The wireless communication system 100 may also include a millimeter-wave (mmW) base station 180, which can operate at mmW and / or near-mmW frequencies to communicate with the UE 182. Extremely high frequency (EHF) is a portion of the electromagnetic spectrum that contains radio frequency (RF). EHF has a range of 30 GHz to 300 GHz, with wavelengths between 1 mm and 10 mm. Radio waves in this band are referred to as millimeter waves. Near-mmW extends down to frequencies of 3 GHz with wavelengths of 100 mm. Ultra-high frequency (SHF) bands extend between 3 GHz and 30 GHz, and are also referred to as centimeter waves. Communication using mmW / near-mmW radio bands has high path loss and relatively short range. The mmW base station 180 and the UE 182 can utilize beamforming (transmit and / or receive) on the mmW communication link 184 to compensate for the extremely high path loss and short range. Furthermore, it should be understood that, in alternative configurations, one or more base stations 102 may also use mmW or near-mmW and beamforming for transmission. Therefore, it should be understood that the foregoing examples are merely illustrative and should not be construed as limiting the various aspects disclosed herein.

[0045] Transmit beamforming is a technique used to focus RF signals in a specific direction. Traditionally, when a network node (e.g., a base station) broadcasts an RF signal, it broadcasts the signal in all directions (omnidirectionally). Using transmit beamforming, the network node determines where a given target device (e.g., a UE) is located (relative to the transmitting network node) and projects a stronger downlink RF signal in that specific direction, thus providing the receiving device with a faster and stronger RF signal (in terms of data rate). To change the directivity of the RF signal during transmission, the network node can control the phase and relative amplitude of the RF signal at each of one or more transmitters broadcasting the RF signal. For example, the network node can use an array of antennas (called a "phased array" or "antenna array") that forms an RF beam that can be "manipulated" to be pointed in different directions without actually moving the antennas. Specifically, RF currents from the transmitters are fed to individual antennas with the correct phase relationship, such that radio waves from the individual antennas add up in the desired direction to increase radiation, while canceling out in the undesired direction to suppress radiation.

[0046] Transmit beams can be quasi-co-located, meaning they appear to the receiver (e.g., the UE) as having the same parameters regardless of whether the network node's own transmit antennas are physically co-located. In NR, there are four types of quasi-co-located (QCL) relationships. Specifically, a given type of QCL relationship means that certain parameters of a second reference RF signal on a second beam can be derived based on information about the source reference RF signal on the source beam. Therefore, if the source reference RF signal is QCL type A, the receiver can use the source reference RF signal to estimate the Doppler shift, Doppler spread, average delay, and delay spread of the second reference RF signal transmitted on the same channel. If the source reference RF signal is QCL type B, the receiver can use the source reference RF signal to estimate the Doppler shift and Doppler spread of the second reference RF signal transmitted on the same channel. If the source reference RF signal is QCL type C, the receiver can use the source reference RF signal to estimate the Doppler shift and average delay of the second reference RF signal transmitted on the same channel. If the source reference RF signal is of type QCL D, the receiver can use the source reference RF signal to estimate the spatial reception parameters of a second reference RF signal transmitted on the same channel.

[0047] In receive beamforming, a receiver uses a receive beam to amplify an RF signal detected on a given channel. For example, the receiver may increase the gain setting of an antenna array in a particular direction and / or adjust the phase setting of the antenna array in a particular direction to amplify the RF signal received from that direction (e.g., increase its gain level). Therefore, when a receiver is described as performing beamforming in a certain direction, it means that the beam gain in that direction is high relative to the beam gain along other directions, or that the beam gain in that direction is the highest compared to the beam gain of all other receive beams available to the receiver in that direction. This results in a stronger received signal strength (e.g., reference signal received power (RSRP), reference signal received quality (RSRQ), signal-to-interference-plus-noise ratio (SINR), etc.) of the RF signal received from that direction.

[0048] The transmit and receive beams can be spatially correlated. Spatial correlation means that parameters for a second beam (e.g., transmit or receive beam) for a second reference signal can be derived based on information about a first beam (e.g., receive or transmit beam) for a first reference signal. For example, a UE can use a specific receive beam to receive a reference downlink reference signal (e.g., a synchronization signal block (SSB)) from a base station. The UE can then form a transmit beam for transmitting an uplink reference signal (e.g., a sounding reference signal (SRS)) to that base station based on the parameters of the receive beam.

[0049] It is important to note that, depending on the entity forming the "downlink" beam, the beam can be either a transmit beam or a receive beam. For example, if the base station is forming a downlink beam to transmit a reference signal to the UE, the downlink beam is a transmit beam. However, if the UE is forming a downlink beam, the downlink beam is a receive beam for receiving the downlink reference signal. Similarly, depending on the entity forming the "uplink" beam, the beam can be either a transmit beam or a receive beam. For example, if the base station is forming an uplink beam, the uplink beam is an uplink receive beam, while if the UE is forming an uplink beam, the uplink beam is an uplink transmit beam.

[0050] The electromagnetic spectrum is typically subdivided into various categories, bands, channels, etc., based on frequency / wavelength. In 5G NR, two initial operating bands have been designated as frequency ranges FR1 (410MHz to 7.125GHz) and FR2 (24.25GHz to 52.6GHz). It should be understood that although a portion of FR1 is greater than 6GHz, in various documents and articles, FR1 is often (interchangeably) referred to as the "sub-6GHz" band. A similar naming issue sometimes occurs with FR2, which is often (interchangeably) referred to as the "millimeter wave" band in documents and articles, although this differs from the designation used by the International Telecommunication Union. ® Extremely high frequency (EHF) bands (30 GHz to 300 GHz) are designated as “millimeter wave” bands.

[0051] The frequencies between FR1 and FR2 are generally referred to as mid-band frequencies. Recent 5G NR studies have designated the operating bands for these mid-band frequencies as the frequency range designation FR3 (7.125 GHz to 24.25 GHz). Bands falling within FR3 can inherit FR1 and / or FR2 characteristics, thus effectively extending the features of FR1 and / or FR2 to mid-band frequencies. Additionally, higher frequency bands are currently being explored to extend 5G NR operation beyond 52.6 GHz. For example, three higher operating bands have been designated as the frequency range designations FR4a or FR4-1 (52.6 GHz to 71 GHz), FR4 (52.6 GHz to 114.25 GHz), and FR5 (114.25 GHz to 300 GHz). Each of these higher frequency bands falls within the EHF band.

[0052] In light of the foregoing, unless otherwise specifically stated, it should be understood that, as used herein, the term "below 6 GHz" and the like can broadly refer to frequencies less than 6 GHz, within FR1, or including intermediate frequency band frequencies. Furthermore, unless otherwise specifically stated, it should be understood that, as used herein, the term "millimeter wave" and the like can broadly refer to frequencies that can include intermediate frequency band frequencies, within FR2, FR4, FR4-a or FR4-1 and / or FR5, or within the EHF band.

[0053] In multi-carrier systems such as 5G, one of the carrier frequencies is referred to as the "primary carrier," "anchor carrier," "primary serving cell," or "PCell," and the remaining carrier frequencies are referred to as "secondary carriers," "secondary serving cells," or "SCell." In carrier aggregation, the anchor carrier is the carrier operating on the primary frequency (e.g., FR1) used by UE 104 / 182 and the cell, where UE 104 / 182 performs an initial Radio Resource Control (RRC) connection establishment procedure or initiates an RRC connection re-establishment procedure. The primary carrier carries all common and UE-specific control channels and can be a carrier on a licensed frequency (however, this is not always the case). The secondary carrier is a carrier operating on a second frequency (e.g., FR2) that can be configured and used to provide additional radio resources once an RRC connection is established between UE 104 and the anchor carrier. In some cases, the secondary carrier can be a carrier on an unlicensed frequency. Secondary carriers may contain only the necessary signaling information and signals. For example, since the primary uplink and primary downlink carriers are typically UE-specific, the UE-specific signaling information and signals may not be present in the secondary carrier. This means that different UEs 104 / 182 within a cell can have different downlink primary carriers. The same applies to the uplink primary carrier. The network can change the primary carrier of any UE 104 / 182 at any time. This is done, for example, to balance the load on different carriers. Since a "serving cell" (whether PCell or SCell) corresponds to the carrier frequency / component carrier through which a base station communicates, the terms "cell," "serving cell," "component carrier," and "carrier frequency" can be used interchangeably.

[0054] For example, still refer to Figure 1One of the frequencies used by macro cell base station 102 can be an anchor carrier (or "PCell"), and the other frequencies used by macro cell base station 102 and / or mmW base station 180 can be secondary carriers ("SCell"). Simultaneous transmission and / or reception on multiple carriers allows UE 104 / 182 to significantly increase its data transmission and / or reception rates. For example, compared to the data rate obtained by a single 20MHz carrier, two aggregated 20MHz carriers in a multi-carrier system would theoretically result in a doubling of the data rate (i.e., 40MHz).

[0055] The wireless communication system 100 may also include a UE 164, which can communicate with the macro cell base station 102 via communication link 120 and / or with the mmW base station 180 via mmW communication link 184. For example, the macro cell base station 102 may support PCells and one or more SCells for the UE 164, and the mmW base station 180 may support one or more SCells for the UE 164.

[0056] In some cases, UE 164 and UE 182 may be able to communicate via sidelink. A sidelink-capable UE (SL-UE) can communicate with base station 102 via communication link 120 using the Uu interface (i.e., the air interface between the UE and the base station). SL-UEs (e.g., UE 164, UE 182) can also communicate directly with each other via radio sidelink 160 using the PC5 interface (i.e., the air interface between sidelink-capable UEs). Radio sidelink (or simply "sidelink") is an adaptation of core cellular network (e.g., LTE, NR) standards that allows direct communication between two or more UEs without the need for communication through a base station. Sidelink communication can be unicast or multicast and can be used for device-to-device (D2D) media sharing, vehicle-to-vehicle (V2V) communication, vehicle-to-everything (V2X) communication (e.g., cellular V2X (cV2X) communication, enhanced V2X (eV2X) communication, emergency rescue applications, etc. One or more SL-UEs in a group of SL-UEs utilizing sidelink communication may be located within the geographical coverage area 110 of base station 102. Other SL-UEs in this group may be outside the geographical coverage area 110 of base station 102, or may be unable to receive transmissions from base station 102 for other reasons. In some cases, the groups of SL-UEs communicating via sidelink communication may utilize a one-to-many (1:M) system, where each SL-UE transmits to every other SL-UE in the group. In some cases, base station 102 facilitates the scheduling of resources used for sidelink communication. In other cases, sidelink communication is performed between the individual SL-UEs without involving base station 102.

[0057] On one hand, the sidelink 160 can operate via a wireless communication medium of interest that can be shared with other vehicles and / or infrastructure access points and other RATs for wireless communication. "Medium" can include one or more time, frequency, and / or space communication resources (e.g., covering one or more channels across one or more carriers) associated with wireless communication between one or more transmitter / receiver pairs. On another hand, the medium of interest may correspond to at least a portion of unlicensed frequency bands shared among various RATs. While different licensed frequency bands have been reserved for certain communication systems (e.g., by government entities such as the U.S. Federal Communications Commission (FCC), these systems (particularly those employing small cell access points) have recently extended their operation to unlicensed National Information Infrastructure (U-NII) bands used by Wireless Local Area Network (WLAN) technologies (most notably the IEEE 802.11x WLAN technology commonly referred to as "Wi-Fi"). Example systems of this type include various variants of CDMA, TDMA, FDMA, Orthogonal FDMA (OFDMA), Single-Carrier FDMA (SC-FDMA), and so on.

[0058] It should be noted that, although Figure 1 Only two of these UEs are exemplified as SL-UEs (i.e., UE 164 and UE 182), but any UE exemplified can be an SL-UE. Furthermore, although only UE 182 is described as capable of beamforming, any UE exemplified (including UE 164) can be capable of beamforming. When SL-UEs are capable of beamforming, they can beamform towards each other (i.e., towards other SL-UEs), towards other UEs (e.g., UE 104), towards base stations (e.g., base station 102, base station 180, small cell 102', access point 150), etc. Therefore, in some cases, UE 164 and UE 182 can utilize beamforming via sidelink 160.

[0059] exist Figure 1 In the example, the UE shown (for simplicity, in) Figure 1Any UE (shown as a single UE 104) can receive signal 124 from one or more Earth-orbiting spacecraft (SV) 112 (e.g., satellites). In one aspect, SV 112 may be part of a satellite positioning system that allows UE 104 to use as an independent source of location information. Satellite positioning systems typically include a system of transmitters (e.g., SV 112) positioned such that a receiver (e.g., UE 104) can determine its location on or above the Earth based at least in part on positioning signals (e.g., signal 124) received from the transmitters. Such transmitters typically transmit signals marked with a set number of repeating pseudo-random noise (PN) codes. While typically located in SV 112, transmitters may sometimes be located at ground-based control stations, base stations 102, and / or other UEs 104. UE 104 may include one or more dedicated receivers specifically designed to receive signal 124 in order to derive geographic location information from SV 112.

[0060] In a satellite positioning system, the use of signal 124 can be enhanced by various satellite-based augmentation systems (SBAS), which may be associated with or otherwise made capable of being used with one or more global and / or regional navigation satellite systems. For example, SBAS may include augmentation systems that provide integrity information, differential correction, etc., such as Wide Area Augmentation System (WAAS), European Geostationary Navigation Overlap Service (EGNOS), Multifunctional Satellite Augmentation System (MSAS), GPS-assisted geographic augmentation navigation, or GPS and geographic augmentation navigation system (GAGAN). Therefore, as used herein, a satellite positioning system may include any combination of one or more global and / or regional navigation satellites associated with such one or more satellite positioning systems.

[0061] On one hand, SV 112 may additionally or alternatively be part of one or more non-terrestrial networks (NTNs). In an NTN, SV 112 connects to an earth station (also referred to as a ground station, NTN gateway, or gateway), which in turn connects to elements in the 5G network, such as the modified base station 102 (without a ground antenna) or network nodes in a 5GC. This element, in turn, provides access to other elements in the 5G network and ultimately to entities outside the 5G network, such as internet web servers and other user equipment. Thus, as a replacement or supplement to communication signals from the ground base station 102, UE 104 can receive communication signals (e.g., signal 124) from SV 112.

[0062] The wireless communication system 100 may also include one or more UEs, such as UE 190, which are indirectly connected to one or more communication networks via one or more device-to-device (D2D) peer-to-peer (P2P) links (referred to as "side links"). Figure 1 In one example, UE 190 has a D2D P2P link 192 with one of UEs 104 connected to one of the base stations 102 (e.g., UE 190 can indirectly obtain cellular connectivity through this D2D P2P link), and has a D2D P2P link 194 with a WLAN STA 152 connected to a WLAN AP 150 (UE 190 can indirectly obtain WLAN-based Internet connectivity through this D2D P2P link). In one example, D2D P2P links 192 and 194 can utilize any known D2D RAT (such as LTE Direct (LTE-D), Wi-Fi Direct). ® ,Bluetooth ® (etc.) to support.

[0063] Figure 2A An example wireless network architecture 200 is illustrated. For instance, the 5GC 210 (also referred to as the Next Generation Core (NGC)) can be functionally viewed as control plane (C-plane) functions 214 (e.g., UE registration, authentication, network access, gateway selection, etc.) and user plane (U-plane) functions 212 (e.g., UE gateway functions, access to data networks, IP routing, etc.), which work together to form the core network. The user plane interface (NG-U) 213 and the control plane interface (NG-C) 215 connect the gNB 222 to the 5GC 210, specifically to user plane functions 212 and control plane functions 214, respectively. In an additional configuration, the ng-eNB 224 can also connect to the 5GC 210 via the NG-C 215 to the control plane function 214 and the NG-U 213 to the user plane function 212. Furthermore, the ng-eNB 224 can communicate directly with the gNB 222 via a backhaul connection 223. In some configurations, the next-generation RAN (NG-RAN) 220 may have one or more gNBs 222, while other configurations include one or more of both ng-eNBs 224 and gNBs 222. Either or both of the gNBs 222 or ng-eNBs 224 can communicate with one or more UEs 204 (e.g., any of the UEs described herein).

[0064] Another optional aspect may include a location server 230, which can communicate with the 5GC 210 to provide location assistance to the UE 204. The location server 230 may be implemented as multiple separate servers (e.g., physically separate servers, different software modules on a single server, different software modules distributed across multiple physical servers, etc.), or alternatively, each may correspond to a single server. The location server 230 may be configured to support one or more location services for the UE 204, which may be connected to the location server 230 via the core network, the 5GC 210, and / or via the Internet (not illustrated). Furthermore, the location server 230 may be integrated into a component of the core network, or alternatively, may be located outside the core network (e.g., a third-party server, such as an original equipment manufacturer (OEM) server or a service server).

[0065] Figure 2B illustrates another example wireless network architecture 240.5GC 260 (which can be compared with...). Figure 2AThe 5GC 210 (corresponding to 5GC 210) can be functionally considered as a control plane function provided by the Access and Mobility Management Function (AMF) 264 and a user plane function provided by the User Plane Function (UPF) 262, which work together to form the core network (i.e., 5GC 260). The functions of AMF 264 include: registration management, connection management, reachability management, mobility management, lawful interception, transmission of session management (SM) messages between one or more UEs 204 (e.g., any of the UEs described herein) and the Session Management Function (SMF) 266, a transparent proxy service for routing SM messages, access authentication and access authorization, transmission of short message service (SMS) messages between UE 204 and the Short Message Service Function (SMSF) (not shown), and Secure Anchoring Functionality (SEAF). AMF 264 also interacts with the Authentication Server Function (AUSF) (not shown) and UE 204 and receives an intermediate key established as a result of the UE 204's authentication process. In the case of UMTS (Universal Mobile Telecommunications System) Subscriber Identity Module (USIM) authentication, AMF 264 retrieves security material from the AMF. AMF 264 also includes Security Context Management (SCM). The SCM receives a key from the SEAF and uses this key to derive an access network-specific key. AMF 264 functionality also includes location service management for regulatory services, transmission of location service messages between UE 204 and Location Management Function (LMF) 270 (which acts as location server 230), transmission of location service messages between NG-RAN 220 and LMF 270, Evolved Packet System (EPS) bearer identifier allocation for EPS interoperability, and UE 204 mobility event notification. Furthermore, AMF 264 also supports non-3GPP... ® (Third Generation Partner Program) Access network functionality.

[0066] The functions of UPF 262 include: acting as an anchor point for intra-RAT / inter-RAT mobility (where applicable), acting as an external Protocol Data Unit (PDU) session point interconnecting to a data network (not shown), providing packet routing and forwarding, packet inspection, user plane policy rule enforcement (e.g., strobing, redirection, traffic steering), lawful eavesdropping (user plane collection), traffic usage reporting, quality of service (QoS) processing for the user plane (e.g., uplink / downlink rate enforcement, reflective QoS marking in the downlink), uplink traffic verification (Service Data Flow (SDF) to QoS flow mapping), transport-level packet marking in the uplink and downlink, downlink packet buffering and downlink data notification triggering, and delivering and forwarding one or more "end markers" to the source RAN node. UPF 262 can also support the delivery of location service messages between UE 204 and location servers (such as SLP 272) on the user plane.

[0067] The functions of SMF 266 include session management, UE Internet Protocol (IP) address allocation and management, selection and control of user plane functions, service orientation configuration at UPF 262 for routing services to the correct destination, partial control of policy enforcement and QoS, and downlink data notification. The interface through which SMF 266 communicates with AMF 264 is called the N11 interface.

[0068] Another optional aspect may include an LMF 270, which can communicate with the 5GC 260 to provide location assistance to the UE 204. The LMF 270 can be implemented as multiple separate servers (e.g., physically separate servers, different software modules on a single server, different software modules distributed across multiple physical servers, etc.), or alternatively, each can correspond to a single server. The LMF 270 can be configured to support one or more location services for the UE 204, which can connect to the LMF 270 via the core network, the 5GC 260, and / or via the Internet (not illustrated). SLP 272 can support similar functions to LMF 270, but while LMF 270 can communicate with AMF 264, NG-RAN 220, and UE 204 on the control plane (e.g., using interfaces and protocols designed to transmit signaling messages rather than voice or data), SLP 272 can communicate with UE 204 and external clients (e.g., third-party server 274) on the user plane (e.g., using protocols designed to carry voice and / or data, such as Transmit Control Protocol (TCP) and / or IP).

[0069] Another optional aspect may include a third-party server 274, which can communicate with LMF 270, SLP 272, 5GC 260 (e.g., via AMF 264 and / or UPF 262), NG-RAN 220, and / or UE 204 to obtain location information (e.g., location estimation) of UE 204. Therefore, in some cases, the third-party server 274 may be referred to as a Location Services (LCS) client or an external client. The third-party server 274 may be implemented as multiple separate servers (e.g., physically separate servers, different software modules on a single server, different software modules distributed across multiple physical servers, etc.), or alternatively, each may correspond to a single server.

[0070] User plane interface 263 and control plane interface 265 connect 5GC 260, and specifically connect UPF 262 and AMF 264 to one or more gNB 222 and / or ng-eNB 224 in NG-RAN 220. The interface between gNB 222 and / or ng-eNB 224 and AMF 264 is referred to as the "N2" interface, while the interface between gNB 222 and / or ng-eNB 224 and UPF 262 is referred to as the "N3" interface. The gNB 222 and / or ng-eNB 224 of NG-RAN 220 can communicate directly with each other via backhaul connection 223, referred to as the "Xn-C" interface. One or more of gNB 222 and / or ng-eNB 224 can communicate with one or more UEs 204 via a radio interface referred to as the "Uu" interface.

[0071] The functionality of the gNB 222 is divided among the gNB Central Unit (gNB-CU) 226, one or more gNB Distributed Units (gNB-DU) 228, and one or more gNB Radio Units (gNB-RU) 229. The gNB-CU 226 is a logical node that includes base station functions other than those specifically allocated to the gNB-DU 228, including user data delivery, mobility control, radio access network sharing, location, session management, etc. More specifically, the gNB-CU 226 typically hosts the Radio Resource Control (RRC), Serving Data Adaptation Protocol (SDAP), and Packet Data Convergence Protocol (PDCP) protocols of the gNB 222. The gNB-DU 228 is a logical node that typically hosts the Radio Link Control (RLC) and Media Access Control (MAC) layers of the gNB 222. Its operation is controlled by the gNB-CU 226. One gNB-DU 228 can support one or more cells, and a cell is supported by only one gNB-DU 228. The interface 232 between gNB-CU 226 and one or more gNB-DU 228 is referred to as the "F1" interface. The physical (PHY) layer functionality of gNB 222 is typically managed by one or more independent gNB-RU 229s, which perform functions such as power amplification and signal transmission / reception. The interface between gNB-DU 228 and gNB-RU 229 is referred to as the "Fx" interface. Therefore, UE 204 communicates with gNB-CU 226 via the RRC, SDAP, and PDCP layers, with gNB-DU 228 via the RLC and MAC layers, and with gNB-RU 229 via the PHY layer.

[0072] The deployment of communication systems such as 5G NR systems can be arranged in a variety of ways using various components or parts. In a 5G NR system or network, network nodes, network entities, network mobility elements, RAN nodes, core network nodes, network elements, or network equipment (such as base stations or one or more units (or components) performing base station functions) can be implemented in aggregated or decomposed architectures. For example, base stations (such as Node B (NB), evolved NB (eNB), NR base stations, 5GNB, AP, TRP, cells, etc.) can be implemented as aggregated base stations (also known as standalone base stations or monolithic base stations) or decomposed base stations.

[0073] Aggregated base stations can be configured to utilize a radio protocol stack that is physically or logically integrated within a single RAN node. Decentralized base stations can be configured to utilize a protocol stack that is physically or logically distributed across two or more units, such as one or more central or centralized units (CUs), one or more distributed units (DUs), or one or more radio units (RUs). In some respects, the CU may be implemented within a RAN node, and one or more DUs may co-located with the CU, or alternatively, may be geographically or virtually distributed across one or more other RAN nodes. DUs may be implemented to communicate with one or more RUs. Each of the CUs, DUs, and RUs may also be implemented as a virtual unit, namely a virtual central unit (VCU), a virtual distributed unit (VDU), or a virtual radio unit (VRU).

[0074] Base station type operation or network design can consider the aggregation characteristics of base station functionality. For example, decomposed base stations can be used in Integrated Access Backhaul (IAB) networks, Open Radio Access Networks (O-RAN) (such as those developed by the O-RAN Alliance), and other similar networks. ® Decomposition can be used in proposed network configurations or virtualized radio access networks (vRAN, also known as cloud radio access networks (C-RAN)). Decomposition can include distributing functionality across two or more units in various physical locations, as well as virtually distributing the functionality of at least one unit, which enables flexibility in network design. Various units in a decomposed base station or decomposed RAN architecture can be configured to communicate wirelessly with at least one other unit.

[0075] Figure 2C An example disaggregated base station architecture 250 according to various aspects of this disclosure is illustrated. The disaggregated base station architecture 250 may include one or more central units (CUs) 280 (e.g., gNB-CU 226) that can communicate directly with the core network 267 (e.g., 5GC 210, 5GC 260) via a backhaul link, or indirectly with the core network 267 via one or more disaggregated base station units (such as a near real-time (near-RT) RAN intelligent controller (RIC) 259 via an E2 link or a non-real-time (non-RT) RIC 257 associated with a Service Management and Orchestration (SMO) framework 255, or both). CUs 280 may communicate with one or more duplex units (DUs) 285 (e.g., gNB-DU 228) via a corresponding midhaul link (e.g., an F1 interface). DUs 285 may communicate with one or more radio units (RUs) 287 (e.g., gNB-RU 229) via a corresponding fronthaul link. RU 287 can communicate with the corresponding UE 204 via one or more radio frequency (RF) access links. In some implementations, UE 204 can be served by multiple RU 287s simultaneously.

[0076] Each unit in the cells (i.e., CU 280, DU 285, RU 287, and near-RT RIC 259, non-RT RIC 257, and SMO frame 255) may include or be coupled to one or more interfaces configured to receive or transmit signals, data, or information (collectively, signals) via wired or wireless transmission media. Each unit in the cells, or an associated processor or controller providing instructions to the communication interfaces of these units, may be configured to communicate with one or more other units via transmission media. For example, these units may include wired interfaces configured to receive signals or transmit signals to one or more other units via wired transmission media. Additionally, these units may include wireless interfaces that may include receivers, transmitters, or transceivers (such as RF transceivers) configured to receive signals or transmit signals to one or more other units, or both, via wireless transmission media.

[0077] In some aspects, the CU 280 can host one or more higher-level control functions. Such control functions may include RRC, PDCP, Service Data Adaptation Protocol (SDAP), etc. Each control function can be implemented using an interface configured to communicate signaling with other control functions hosted by the CU 280. The CU 280 can be configured to handle user plane functionality (i.e., Central Unit-User Plane (CU-UP)), control plane functionality (i.e., Central Unit-Control Plane (CU-CP)), or a combination thereof. In some implementations, the CU 280 can be logically split into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP units can communicate bidirectionally with the CU-CP units via an interface such as an E1 interface. The CU 280 can be implemented to communicate with the DU 285 for network control and signaling, as needed.

[0078] DU 285 may correspond to a logic unit that includes one or more base station functions for controlling the operation of one or more RU 287s. In some aspects, DU 285 may be at least partially based on functional partitioning (such as that provided by the 3rd Generation Partnership Project (3GPP)). ®The DU285 is functionally partitioned to host one or more of the RLC layer, MAC layer, and one or more high-PHY layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation, and demodulation). In some respects, the DU285 may further host one or more low-PHY layers. Each layer (or module) may be implemented using an interface configured to communicate signals with other layers (and modules) hosted by the DU285 or with control functions hosted by the CU280.

[0079] Lower-layer functionality can be implemented by one or more RU 287s. In some deployments, the RU 287 controlled by the DU 285 may correspond to a logical node that hosts RF processing functions or low-PHY layer functions (such as performing Fast Fourier Transform (FFT), Inverse FFT (iFFT), digital beamforming, Physical Random Access Channel (PRACH) extraction and filtering, or both, based at least in part on functional decomposition (such as lower-layer functional decomposition). In such architectures, the RU 287 may be implemented to handle over-the-air (OTA) communications with one or more UE 204s. In some specific implementations, the real-time and non-real-time aspects of control plane and user plane communications with the RU 287 may be controlled by the corresponding DU 285. In some scenarios, this configuration allows the DU 285 and CU 280 to be implemented in cloud-based RAN architectures such as vRAN architectures.

[0080] SMO framework 255 can be configured to support RAN deployment and provisioning of both non-virtualized and virtualized network elements. For non-virtualized network elements, SMO framework 255 can be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which can be managed via operation and maintenance interfaces such as the O1 interface. For virtualized network elements, SMO framework 255 can be configured to interact with cloud computing platforms such as Open Cloud (O-Cloud) 269 to perform network element lifecycle management (such as instantiating virtualized network elements) via cloud computing platform interfaces such as the O2 interface. Such virtualized network elements may include, but are not limited to, CU 280, DU 285, RU 287, and near-RT RIC 259. In some implementations, SMO framework 255 can communicate with hardware aspects of the 4G RAN, such as Open eNB (O-eNB) 261, via the O1 interface. Additionally, in some implementations, SMO framework 255 can communicate directly with one or more RU 287s via the O1 interface. SMO framework 255 may also include a non-RT RIC 257 configured to support the functionality of SMO framework 255.

[0081] The non-RT RIC 257 can be configured to include logical functions enabling non-real-time control and optimization of RAN elements and resources, including artificial intelligence / machine learning (AI / ML) workflows for model training and updates, or policy-based guidance for applications / features in the near-RT RIC 259. The non-RT RIC 257 can be coupled to or communicate with the near-RT RIC 259, such as via an A1 interface. The near-RT RIC 259 can be configured to include logical functions enabling near real-time control and optimization of RAN elements and resources via an interface, such as an E2 interface, through data collection and actions, connecting one or more CU 280s, one or more DU 285s, or both, and O-eNBs to the near-RT RIC 259.

[0082] In some implementations, to generate AI / ML models to be deployed in the near-RT RIC 259, the non-RT RIC 257 may receive parameters or external enrichment information from an external server. This information can be utilized by the near-RT RIC 259 and may be received from non-network data sources or network functions at the SMO framework 255 or the non-RT RIC 257. In some examples, the non-RT RIC 257 or the near-RT RIC 259 may be configured to tune RAN behavior or performance. For example, the non-RT RIC 257 may monitor long-term trends and patterns of performance and perform corrective actions using the AI / ML model via the SMO framework 255 (such as reconfiguration via O1) or by creating RAN management policies (such as A1 policies).

[0083] Figures 3A, 3B and Figure 3C Several example components (represented by corresponding boxes) are illustrated, which can be incorporated into UE 302 (which may correspond to any UE described herein), base station 304 (which may correspond to any base station described herein), and network entity 306 (which may correspond to or embody any network function described herein, including location server 230 and LMF 270, or alternatively may be independent of UE 302). Figure 2A and Figure 2B depicts the NG-RAN 220 and / or 5GC 210 / 260 infrastructure (such as a private network) to support the operations described herein. It should be understood that these components can be implemented in different specific implementations in different types of devices (e.g., in an ASIC, in a system-on-a-chip (SoC), etc.). The illustrated components can also be incorporated into other devices in the communication system. For example, other devices in the system may include components similar to those described as providing similar functionality. Furthermore, a given device may contain one or more of these components. For example, a device may include multiple transceiver components that enable the device to operate on multiple carriers and / or communicate via different technologies.

[0084] UE 302 and base station 304 each include one or more Wireless Wide Area Network (WWAN) transceivers 310 and 350, which provide components (e.g., components for transmitting, components for receiving, components for measuring, components for tuning, components for blocking transmission, etc.) for communication via one or more wireless communication networks (not shown), such as NR networks, LTE networks, GSM networks, etc. WWAN transceivers 310 and 350 may each be connected to one or more antennas 316 and 356 for communication with other network nodes (such as other UEs, access points, base stations (e.g., eNB, gNB), etc.) via at least one designated RAT (e.g., NR, LTE, GSM, etc.) through a wireless communication medium of interest (e.g., a time / frequency resource set in a specific spectrum). WWAN transceivers 310 and 350 can be configured in different ways to transmit and encode signals 318 and 358 (e.g., messages, indications, information, etc.) according to a specified RAT, and conversely, to receive and decode signals 318 and 358 (e.g., messages, indications, information, pilots, etc.). Specifically, WWAN transceivers 310 and 350 each include: one or more transmitters 314 and 354 for transmitting and encoding signals 318 and 358, respectively; and one or more receivers 312 and 352 for receiving and decoding signals 318 and 358, respectively.

[0085] In at least some cases, UE 302 and base station 304 each further include one or more short-range wireless transceivers 320 and 360, respectively. Short-range wireless transceivers 320 and 360 can be connected to one or more antennas 326 and 366, respectively, and provide access over a wireless communication medium of interest via at least one designated RAT (e.g., Wi-Fi, LTE Direct, Bluetooth). ® ZIGBEE ® Z-WAVE ® Components (e.g., components for transmitting, components for receiving, components for measuring, components for tuning, components for blocking transmission, etc.) that enable communication between PC5, Dedicated Short-Range Communication (DSRC), Wireless Access for Vehicle Environments (WAVE), Near Field Communication (NFC), Ultra-Wideband (UWB), etc.) and other network nodes (such as other UEs, access points, base stations, etc.). Short-range transceivers 320 and 360 can be configured in different ways to transmit and encode signals 328 and 368 (e.g., messages, indications, information, etc.) respectively according to a specified RAT, and conversely, to receive and decode signals 328 and 368 (e.g., messages, indications, information, pilots, etc.) respectively. Specifically, the short-range wireless transceiver 320 and short-range wireless transceiver 360 each include: one or more transmitters 324 and 364 respectively for transmitting and encoding signals 328 and 368, and one or more receivers 322 and 362 respectively for receiving and decoding signals 328 and 368. As a specific example, the short-range wireless transceiver 320 and short-range wireless transceiver 360 can be Wi-Fi transceivers, Bluetooth transceivers, etc. ® Transceiver, Zigbee ® and / or Z-WAVE ® Transceivers, NFC transceivers, UWB transceivers, or vehicle-to-vehicle (V2V) and / or vehicle-to-everything (V2X) transceivers.

[0086] In at least some cases, UE 302 and base station 304 also include satellite signal interfaces 330 and 370, each satellite signal interface including one or more satellite signal receivers 332 and 372, and optionally including one or more satellite signal transmitters 334 and 374, respectively. In some cases, base station 304 may be a terrestrial base station that can communicate with a spacecraft (e.g., spacecraft 112) via satellite signal interface 370. In other cases, base station 304 may be a spacecraft (or other non-terrestrial entity) that uses satellite signal interface 370 to communicate with terrestrial networks and / or other spacecraft.

[0087] Satellite signal receivers 332 and 372 can be connected to one or more antennas 336 and 376, respectively, and can provide components for receiving and / or measuring satellite positioning / communication signals 338 and 378, respectively. When satellite signal receivers 332 and 372 are satellite positioning system receivers, satellite positioning / communication signals 338 and 378 can be Global Positioning System (GPS) signals, Global Navigation Satellite System (GLONASS) signals, Galileo signals, BeiDou signals, Indian Regional Navigation Satellite System (NAVIC), Quasi-Zenith Satellite System (QZSS) signals, etc. When satellite signal receivers 332 and 372 are non-terrestrial network (NTN) receivers, satellite positioning / communication signals 338 and 378 can be communication signals originating from a 5G network (e.g., carrying control and / or user data). Satellite signal receivers 332 and 372 can include any suitable hardware and / or software for receiving and processing satellite positioning / communication signals 338 and 378, respectively. Satellite signal receivers 332 and 372 may request appropriate information and operations from other systems, and in at least some cases, use measurements obtained by any suitable satellite positioning system algorithm to perform calculations to determine the locations of UE 302 and base station 304, respectively.

[0088] Optional satellite signal transmitters 334 and 374 (when present) can be connected to one or more antennas 336 and 376, respectively, and can be provided with components for transmitting satellite positioning / communication signals 338 and 378, respectively. When satellite signal transmitter 374 is a satellite positioning system transmitter, the satellite positioning / communication signal 378 can be a GPS signal, GLONASS signal, etc. ® Signals include Galileo signals, BeiDou signals, NAVIC signals, and QZSS signals. When satellite signal transmitters 334 and 374 are NTN transmitters, satellite positioning / communication signals 338 and 378 can be communication signals originating from a 5G network (e.g., carrying control and / or user data). Satellite signal transmitters 334 and 374 can include any suitable hardware and / or software for transmitting satellite positioning / communication signals 338 and 378, respectively. Satellite signal transmitters 334 and 374 can request appropriate information and operations from other systems.

[0089] Base station 304 and network entity 306 each include one or more network transceivers 380 and 390, which provide components (e.g., transmitting components, receiving components, etc.) for communicating with other network entities (e.g., other base stations 304, other network entities 306). For example, base station 304 may use one or more network transceivers 380 to communicate with other base stations 304 or network entities 306 via one or more wired or wireless backhaul links. Similarly, network entity 306 may use one or more network transceivers 390 to communicate with one or more base stations 304 via one or more wired or wireless backhaul links, or to communicate with other network entities 306 via one or more wired or wireless core network interfaces.

[0090] Transceivers can be configured to communicate via wired or wireless links. A transceiver (whether wired or wireless) includes transmitter circuitry (e.g., transmitters 314, 324, 354, 364) and receiver circuitry (e.g., receivers 312, 322, 352, 362). In some embodiments, the transceiver may be an integrated device (e.g., implementing transmitter and receiver circuitry in a single device), in some embodiments it may include separate transmitter and receiver circuitry, or in other embodiments it may be implemented in a different manner. The transmitter and receiver circuitry of a wired transceiver (e.g., network transceiver 380 and network transceiver 390 in some embodiments) may be coupled to one or more wired network interface ports. Wireless transmitter circuitry (e.g., transmitters 314, 324, 354, 364) may include or be coupled to multiple antennas (e.g., antennas 316, 326, 356, 366), such as an antenna array, which allows the corresponding device (e.g., UE 302, base station 304) to perform transmit beamforming, as described herein. Similarly, wireless receiver circuitry (e.g., receivers 312, 322, 352, 362) may include or be coupled to multiple antennas (e.g., antennas 316, 326, 356, 366), such as an antenna array, which allows the corresponding device (e.g., UE 302, base station 304) to perform receive beamforming, as described herein. In one aspect, the transmitter and receiver circuitry may share the same multiple antennas (e.g., antennas 316, 326, 356, 366), such that the corresponding device may perform only receive or only transmit at a given time, rather than both receive and transmit simultaneously. Wireless transceivers (e.g., WWAN transceivers 310 and 350, short-range wireless transceivers 320 and 360) may also include network listening modules (NLMs) for performing various measurements.

[0091] As used herein, various wireless transceivers (e.g., transceivers 310, 320, 350, and 360 in some specific embodiments, and network transceivers 380 and 390) and wired transceivers (e.g., network transceivers 380 and 390 in some specific embodiments) may generally be described as "transceiver," "at least one transceiver," or "one or more transceivers." Therefore, whether a particular transceiver is a wired or wireless transceiver can be inferred from the type of communication performed. For example, backhaul communication between network devices or servers typically involves signaling via a wired transceiver, while wireless communication between a UE (e.g., UE 302) and a base station (e.g., base station 304) will typically involve signaling via a wireless transceiver.

[0092] UE 302, base station 304, and network entity 306 also include other components that can be used in conjunction with the operation disclosed herein. UE 302, base station 304, and network entity 306 each include one or more processors 342, 384, and 394 for providing functionality related to, for example, wireless communication, and for providing other processing functionality. Thus, processors 342, 384, and 394 may provide components for processing, such as components for determining, components for calculating, components for receiving, components for transmitting, components for indicating, etc. In one aspect, processors 342, 384, and 394 may include, for example, one or more general-purpose processors, multi-core processors, central processing units (CPUs), ASICs, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), other programmable logic devices or processing circuits, or various combinations thereof.

[0093] UE 302, base station 304, and network entity 306 each include memory circuitry implementing memories 340, 386, and 396 (e.g., each including a memory device) for maintaining information (e.g., information indicating reserved resources, thresholds, parameters, etc.). Therefore, memories 340, 386, and 396 can provide components for storage, retrieval, maintenance, etc. In some cases, UE 302, base station 304, and network entity 306 may each include positioning components 348, 388, and 398. Positioning components 348, 388, and 398 may be hardware circuitry that is part of or coupled to processors 342, 384, and 394, respectively, which, when executed, enable UE 302, base station 304, and network entity 306 to perform the functionality described herein. In other aspects, positioning components 348, 388, and 398 may be external to processors 342, 384, and 394 (e.g., part of a modem processing system, integrated with another processing system, etc.). Alternatively, positioning components 348, 388, and 398 may be memory modules stored in memories 340, 386, and 396, respectively, which, when executed by processors 342, 384, and 394 (or modem processing system, another processing system, etc.), enable UE 302, base station 304, and network entity 306 to perform the functionality described herein. Figure 3A illustrates possible locations for positioning component 348, which may be part of, for example, one or more WWAN transceivers 310, memory 340, one or more processors 342, or any combination thereof, or may be a standalone component. Figure 3B illustrates possible locations for positioning component 388, which may be part of, for example, one or more WWAN transceivers 350, memory 386, one or more processors 384, or any combination thereof, or may be a standalone component. Figure 3C Possible locations for the positioning component 398 are illustrated. The positioning component may be part of, for example, one or more network transceivers 390, memory 396, one or more processors 394, or any combination thereof, or may be a standalone component.

[0094] UE 302 may include one or more sensors 344 coupled to one or more processors 342 to provide components for sensing or detecting motion and / or orientation information independent of motion data derived from signals received by one or more WWAN transceivers 310, one or more short-range wireless transceivers 320, and / or satellite signal interfaces 330. By way of example, sensor 344 may include accelerometers (e.g., microelectromechanical systems (MEMS) devices), gyroscopes, geomagnetic sensors (e.g., compasses), altimeters (e.g., barometric altimeters), and / or any other type of motion detection sensor. Furthermore, sensor 344 may include multiple different types of devices and combine their outputs to provide motion information. For example, sensor 344 may use a combination of multi-axis accelerometers and orientation sensors to provide the ability to calculate positioning in two-dimensional (2D) and / or three-dimensional (3D) coordinate systems.

[0095] In addition, UE 302 includes a user interface 346 that provides components for providing instructions to a user (e.g., audible and / or visual instructions) and / or for receiving user input (e.g., when the user actuates a sensing device such as a keypad, touchscreen, microphone, etc.). Although not shown, base station 304 and network entity 306 may also include user interfaces.

[0096] Referring more specifically to one or more processors 384, in the downlink, IP packets from network entity 306 can be provided to processor 384. One or more processors 384 can implement functionality for the RRC layer, Packet Data Convergence Protocol (PDCP) layer, Radio Link Control (RLC) layer, and Media Access Control (MAC) layer. One or more processors 384 may provide: RRC layer functionality associated with broadcasting system information (e.g., Master Information Block (MIB), System Information Block (SIB)), RRC connection control (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), inter-RAT mobility, and measurement configuration for UE measurement reporting; PDCP layer functionality associated with header compression / decompression, security (encryption, decryption, integrity protection, integrity verification), and handover support functions; RLC layer functionality associated with the delivery of upper-layer PDUs, error correction via Automatic Repeat Request (ARQ), concatenation, segmentation, and reassembly of RLC Service Data Units (SDUs), resegmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, scheduling information reporting, error correction, priority processing, and logical channel priority ordering.

[0097] Transmitter 354 and receiver 352 implement Layer 1 (L1) functionality associated with various signal processing functions. Layer 1, including the physical (PHY) layer, may include: error detection on the transport channel, forward error correction (FEC) decoding / decoding of the transport channel, interleaving, rate matching, mapping to the physical channel, modulation / demodulation of the physical channel, and MIMO antenna processing. Transmitter 354 processes the mapping to the signal constellation based on various modulation schemes (e.g., binary phase shift keying (BPSK), quadrature phase shift keying (QPSK), M-phase shift keying (M-PSK), M-quadrature amplitude modulation (M-QAM)). The decoded and modulated symbols can then be split into parallel streams. Each stream can then be mapped to orthogonal frequency division multiplexing (OFDM) subcarriers, multiplexed with a reference signal (e.g., pilot) in the time and / or frequency domains, and then combined using inverse fast Fourier transform (IFFT) to produce a physical channel carrying a stream of time-domain OFDM symbols. The OFDM symbol stream is spatially pre-decoded to generate multiple spatial streams. Channel estimates from the channel estimator can be used to determine the decoding and modulation schemes, as well as for spatial processing. The channel estimates can be derived from a reference signal transmitted by UE 302 and / or channel condition feedback. Each spatial stream can then be provided to one or more different antennas 356. The transmitter 354 can use the corresponding spatial stream to modulate an RF carrier for transmission.

[0098] At UE 302, receiver 312 receives signals via its corresponding antenna 316. Receiver 312 recovers the information modulated onto the RF carrier and provides this information to one or more processors 342. Transmitter 314 and receiver 312 implement Layer 1 functionality associated with various signal processing functions. Receiver 312 can perform spatial processing on the information to recover any spatial streams destined for UE 302. If multiple spatial streams are destined for UE 302, they can be combined by receiver 312 into a single OFDM symbol stream. Receiver 312 then uses a Fast Fourier Transform (FFT) to transform the OFDM symbol stream from the time domain to the frequency domain. The frequency domain signal comprises a separate OFDM symbol stream for each subcarrier of the OFDM signal. The symbols on each subcarrier, along with the reference signal, are recovered and demodulated by determining the most probable signal constellation point transmitted by base station 304. These soft decisions can be based on a channel estimate calculated by a channel estimator. The soft decisions are then decoded and deinterleaved to recover the data and control signals originally transmitted by base station 304 on the physical channel. Then, data and control signals are provided to one or more processors 342, which implement layer 3 (L3) and layer 2 (L2) functionality.

[0099] In the downlink, one or more processors 342 provide demultiplexing, packet reassembly, decryption, header decompression, and control signal processing between the transport and logical channels to recover IP packets from the core network. One or more processors 342 are also responsible for error detection.

[0100] Similar to the functionality described in conjunction with downlink transmissions performed by base station 304, one or more processors 342 provide: RRC layer functionality associated with system information (e.g., MIB, SIB) acquisition, RRC connectivity, and measurement reporting; PDCP layer functionality associated with header compression / decompression and security (encryption, decryption, integrity protection, integrity verification); RLC layer functionality associated with the delivery of upper-layer PDUs, error correction via ARQ, concatenation, segmentation, and reassembly of RLC SDUs, resegmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, multiplexing of MAC SDUs onto transport blocks (TBs), demultiplexing of MAC SDUs from TBs, scheduling information reporting, error correction via Hybrid Automatic Repeat Request (HARQ), priority processing, and logical channel priority ordering.

[0101] The channel estimate derived by the channel estimator from the reference signal or feedback transmitted by the base station 304 can be used by the transmitter 314 to select an appropriate decoding and modulation scheme and facilitate spatial processing. The spatial stream generated by the transmitter 314 can be provided to different antennas 316. The transmitter 314 can use the corresponding spatial stream to modulate the RF carrier for transmission.

[0102] Uplink transmissions are processed at base station 304 in a manner similar to that described in conjunction with the receiver function at UE 302. Receiver 352 receives signals via its corresponding antenna 356. Receiver 352 recovers the information modulated onto the RF carrier and provides this information to one or more processors 384.

[0103] In the uplink, one or more processors 384 provide demultiplexing, packet reassembly, decryption, header decompression, and control signal processing between the transport channel and the logical channel to recover IP packets from UE 302. IP packets from one or more processors 384 can be provided to the core network. One or more processors 384 are also responsible for error detection.

[0104] For convenience, UE 302, base station 304 and / or network entity 306 are shown in Figures 3A, 3B and Figure 3C The examples shown herein include various components that can be configured according to the various examples described herein. However, it should be understood that the illustrated components may have different functionalities in different designs. In particular, Figures 3A to 3B are shown in the figures. Figure 3C Various components are optional in alternative configurations, and various aspects include configurations that can vary due to design choices, cost, device usage, or other considerations. For example, in the case of Figure 3A, a particular implementation of UE 302 may omit the WWAN transceiver 310 (e.g., wearable devices, tablets, personal computers (PCs), or laptops may have Wi-Fi and / or Bluetooth). ® In some cases, the short-range wireless transceiver 320 (e.g., cellular only), the satellite signal interface 330, or the sensor 344 may be omitted. In another example, in the case of FIG3B, a specific implementation of base station 304 may omit the WWAN transceiver 350 (e.g., a Wi-Fi "hotspot" access point without cellular capability), the short-range wireless transceiver 360 (e.g., cellular only), or the satellite signal interface 370, etc. For the sake of brevity, various alternative configurations are not illustrated herein, but will be readily understood by those skilled in the art.

[0105] Various components of UE 302, base station 304, and network entity 306 can be communicatively coupled to each other via data buses 308, 382, ​​and 392, respectively. In one aspect, data buses 308, 382, ​​and 392 can form or be part of the communication interfaces of UE 302, base station 304, and network entity 306, respectively. For example, in cases where different logical entities are embodied in the same device (e.g., gNB and location server functionality integrated into the same base station 304), data buses 308, 382, ​​and 392 can provide communication between these logical entities.

[0106] Figures 3A, 3B and Figure 3C The components can be implemented in various ways. In some specific implementations, Figures 3A, 3B, and... Figure 3CThe components can be implemented in one or more circuits, such as, for example, one or more processors and / or one or more ASICs (which may include one or more processors). Here, each circuit may use and / or combine at least one memory component for storing information or executable code used by the circuit to provide that functionality. For example, some or all of the functionalities represented by blocks 310 to 346 may be implemented by the processor and memory components of UE 302 (e.g., by executing appropriate code and / or by appropriate configuration of the processor components). Similarly, some or all of the functionalities represented by blocks 350 to 388 may be implemented by the processor and memory components of base station 304 (e.g., by executing appropriate code and / or by appropriate configuration of the processor components). Moreover, some or all of the functionalities represented by blocks 390 to 398 may be implemented by the processor and memory components of network entity 306 (e.g., by executing appropriate code and / or by appropriate configuration of the processor components). For simplicity, various operations, actions, and / or functions are described herein as being performed "by the UE," "by the base station," "by the network entity," etc. However, as will be understood, such operations, actions and / or functions can actually be performed by specific components or combinations of components of the UE 302, base station 304, network entity 306, etc. (such as processors 342, 384, 394, transceivers 310, 320, 350 and 360, memories 340, 386 and 396, positioning components 348, 388 and 398, etc.).

[0107] In some designs, network entity 306 may be implemented as a core network component. In other designs, network entity 306 may operate differently from the network operator or cellular network infrastructure (e.g., NG RAN 220 and / or 5GC 210 / 260). For example, network entity 306 may be a component of a private network that can be configured to communicate with UE 302 via base station 304 or independently of base station 304 (e.g., via a non-cellular communication link such as Wi-Fi).

[0108] Visual positioning technologies used to determine the location / position of target objects are being deployed or are expected to be deployed in multiple scenarios, such as IoT monitoring (e.g., retail stores and warehouses, where customers, employees and / or assets are equipped with UEs or other RF devices) and public safety and surveillance (e.g., public spaces, museums and / or campuses where people carry UEs).

[0109] The primary goal of visual positioning technology is to locate / track the UE or other RF devices (also referred to as target devices). To achieve visual positioning, in addition to an RF-based positioning system within the positioning environment, multiple cameras (known as over-the-top (OTT) cameras) also monitor the environment. Vision has the potential to significantly enhance the performance of positioning engines. Vision-based systems typically identify… picture Regions of interest (RoIs) (such as bounding boxes) are then used to determine the three-dimensional (3D) world localization of targets contained within the RoI (bounding box).

[0110] However, even when the target is within the camera's field of view, vision-based systems are still susceptible to occlusion. Particularly challenging situations arise when a relevant target is detected but partially occluded (e.g., by a walkway). Such partial occlusion can adversely affect the performance of the vision components of the localization engine that needs to be addressed.

[0111] Figure 4 Figure 400 illustrates an example scenario according to various aspects of this disclosure, wherein one or more targets within the environment may be at least partially occluded by an OTT camera system monitoring the environment. Figure 4 In the example, denoted as “A”, “B”, and “C”, three targets (e.g., a person, an automated guided vehicle (AGV)) are carrying three UEs (or have UEs attached to them) within the example environment. The illustrated environment includes an RF-based positioning system with at least two access points (APs) and a vision-based positioning system with at least four cameras (e.g., surveillance cameras). Figure 4 It also shows camera view from one of the OTT cameras. picture Example picture Like. Figure 4 As shown, targets B and C are in the field of view of this camera. picture The middle part is partially obscured, while target A is fully visible.

[0112] Regarding the visual component of the positioning engine, there are two operating modes: (1) single view picture Pattern and (2) multi-view picture Mode. In single-view picture In this mode, the relevant target is detected by a single OTT camera, while in multi-view... picture In this mode, the relevant target is detected by multiple OTT cameras.

[0113] Figure 5 Examples of partial occlusion of a single view according to various aspects of this disclosure are illustrated. picture The impact of visual localization results in the system. Specifically, Figure 500 illustrates where targets (e.g., people, AGVs) are located in the field of vision from the camera. picture of pictureThe scene is fully visible within the image. In contrast, Figure 550 illustrates a scene where targets (e.g., people, AGVs) are visible only from the camera's view. picture of picture Like a scene where the interior is partially obscured.

[0114] As shown in Figure 550, in a single view picture In this mode, visual localization is performed by first detecting the RoI containing the target and then back-projecting the centroid of the lower portion of the RoI (e.g., corresponding to the user's foot touching the floor) onto the ground plane of the 3D world coordinate system. If the detection is partially occluded (i.e., if the lower portion of the RoI is not visible), then single-view localization is used. picture The visual localization estimation in the system will be very poor.

[0115] For multiple views picture The system detects the effects of partial occlusion slightly differently. Figure 6 This illustrates partial occlusion of multiple views according to various aspects of this disclosure. picture Figure 600 illustrates the impact of visual localization results in the system. Specifically, Figure 600 illustrates the impact of visual localization results in three different systems. picture Image / Camera View picture A scene that captures two targets (e.g., a person and an AGV).

[0116] In multi-view picture In this mode, detections from multiple cameras are combined to produce a final visual localization estimate. This is similar to single-view... picture System, view picture Some views in picture Partial occlusion in the image may manifest as a decrease in positioning performance. More specifically, in multi-view... picture In the system, object detections first need to be cross-view before they can be fused together. picture Matching. Therefore, due to limited visibility, partial occlusion may make the matching process difficult to operate properly, which may lead to cross-view matching. picture Target mismatch (i.e., partially visible targets are mistakenly identified as different targets), such as Figure 6 As illustrated, this will further degrade the performance of vision-based positioning.

[0117] In this disclosure, an RF-assisted method is proposed for processing partially occluded targets and incomplete RoIs (such as...) by means of... Figures 4 to 6 In the cases shown: (1) use 3D information to make it complete and / or (2) mark it as an outlier (and remove it). The methods disclosed in general (but not exclusively) relate to the above references. Figure 5 and Figure 6 The described single view pictureModes and multi-view picture model.

[0118] Referring to the RoI completeness method using 3D target information, in this method, knowledge of the size (e.g., height and / or width) of the target in the 3D world is available (e.g., estimated) and can be used to complete the corresponding RoI (e.g., bounding box) upon receiving partial detection. This method is typically used in single-view... picture This is very useful in the mode, where at most one camera view picture The relevant target has been detected.

[0119] Figure 7 This is an example scene 700 illustrating the use of 3D target information to complete the RoI associated with the target according to various aspects of this disclosure. For example... Figure 7 As shown, the target (e.g., a person, AGV) is in the camera view from a camera (not shown). picture of picture The image is partially obscured.

[0120] Using 3D information to ensure the complete existence of an RoI involves several key functions / processes and their corresponding information requirements. The first function is unique target identification. In this function, the relevant target is detected as an RoI (e.g., a bounding box), and key visual features associated with its visual appearance are extracted. The type of key visual features can be based on the type of target object. For example, for human targets (such as customers in a store), key visual features might be associated with the color combination of their clothing or the items they are carrying (e.g., a handbag, if any). Storing such features allows for the unique identification of human targets while minimizing privacy concerns. For workers / employees or autonomous robots (e.g., AGVs), key features can be associated with unique visual identifiers associated with the target, such as visible name tags, employee numbers, color-coded clothing (e.g., different colored protective helmets in a warehouse or construction area), and / or quick-response (QR) codes. In the case of boxes or other non-human assets, key features can be associated with the shape, color, barcode, and / or QR code of the asset.

[0121] This first function may also include semantic segmentation. This particular aspect relates to the concept of detecting a target, either completely or partially. For example, if the target is a person, the semantic segmentation component can further process the detected RoI to determine whether both the person's feet and head are visible.

[0122] The second function is target 3D size knowledge acquisition. In this function, the 3D size of the relevant target is estimated. The specific implementation of this function may depend on the target application. For example, the following guidelines can be considered for potential deployment in a typical retail store where customers / staff are the main RF devices / UE-carrying targets. Specifically, human height is key 3D target information. Human width (in terms of the diameter of the equivalent cylinder representing the human) is also useful. Height can be estimated by: (1) using only vision when semantic segmentation is enabled and both the human's feet and head are visible; or (2) fusing RF-based estimation of UE localization when the target is partially occluded and only the reference RoI segment (e.g., the head) is visible.

[0123] It should be noted that in the second scenario, the specific implementation may require further testing or adjustments to the UE's performance. picture A sensible assessment of the position relative to a visible reference segment is needed to estimate the height. When the RF device / UE is not directly visible, this can be achieved using side information related to the state of the UE device (e.g., whether the UE device is in an active call, connected state, etc.).

[0124] The third function is target information storage and management. In this function, target information (e.g., key feature descriptors, 3D information) is stored along with the unique identifier of the RF device / UE (e.g., MAC address). It's important to note that even if the initially detected RoI is incomplete, new active targets can be stored as long as 3D information can be inferred. This can be determined from the RoI generation and semantic segmentation module (if available), depending on the application type. For example, when an application is tracking a person, the RoI is considered complete when both the person's feet and head have been detected (which can be inferred from the semantic segmentation module). This function periodically updates active items to capture more complete information or any visual changes. When a new active target is detected, the RF information of the active RF device / UE can be used for cross-correlation to determine the MAC ID of the new target.

[0125] This information can be stored for a specific period of time. If no target is detected after the specified period threshold (indicating that the target is no longer in the monitored environment or may have changed its visual appearance), the item for the corresponding device will be removed / deleted.

[0126] Referring to outlier RoI detection and removal methods, in this paradigm, RoIs that partially occlude / detect targets are interpreted and marked as outliers, and if these RoIs do not meet specific criteria, they are subsequently removed from the vision processing pipeline and localization engine. Outlier detection / removal methods in multi-view... picture Such systems are often useful because (1) they are inherently redundant and (2) they naturally address the limitations of visibility across multiple viewports due to partially occluded targets. pictureThe problem is the incorrect association of targets.

[0127] Figure 8 Figure 800 illustrates an example of outlier RoI detection and removal according to various aspects of this disclosure. Figure 8 In the example, similar to Figure 6 In the illustrated scenario, two targets (e.g., a person, an AGV) are within the field of view of at least three cameras. Figure 8 As shown, the same target appears in the view of two different cameras. picture It was detected in the middle, but in the camera view picture One of the cameras picture The interior is partially obscured. (The remaining camera view...) picture Another target was detected (partially occluded and further removed as an outlier). The details of outlier removal are described further below.

[0128] For example, the complete RoI methodology contains several key functional, process, and information requirements related to this methodology. The first function is to cross multiple views... picture single vision picture Visual localization. As discussed above, partially detected targets with an incomplete RoI can lead to significant visual localization errors. However, if targets are detected from multiple viewpoints... picture The detection is available (some of which are shown) picture For targets that are fully visible, partial occlusion-based vision can be used. picture The obtained visual localization estimate or localization error (e.g., localization error relative to RF-based localization estimate) is used as an outlier detection criterion.

[0129] To detect such outliers, the following general framework can be used. First, cross all relevant camera views... picture Detect targets and correlate them. Secondly, use a single view. picture The mode spans all relevant camera views picture Perform visual localization. Third, perform clustering of visual localization estimates, identify candidate outliers, and associate the corresponding scores with the candidate outliers (e.g., distance from the cluster center). Fourth, compare the scores of the candidate outliers with a pre-determined outlier threshold and remove severe outliers (e.g., outliers exceeding the outlier threshold).

[0130] It should be noted that retained outliers can further undergo the RoI complete process from the RoI complete approach. It should also be noted that in some cases, the entire cluster may be declared an outlier. This can happen when all detections are poor.

[0131] In some cases, undesirable observations of a given target obtained during the clustering process may end up clustered in different groups. In such cases, multiple targets may be identified as belonging to the same cluster. The second function described below can be used to handle such situations.

[0132] The second function involves cross-correlation with RF-based information and / or key visual and / or 3D features. In this function, RF-based localization-related information can be used to identify outliers in visual detection, particularly across camera views. picture In the case of mismatch, the RF-based localization estimate for the corresponding RF device / UE can be cross-correlated with the vision-based localization information from the first function, and additional metrics for outlier detection can be calculated based on the cross-correlation results.

[0133] However, it should be noted that the usefulness of RF measurements and RF-based location estimation can depend on the specific technologies and setups available. For example, location estimation based on the Wi-Fi Reference Signal Strength Indicator (RSSI) typically provides only a coarse location estimate over a large area indicating the device's location. Such input may be useful across large and relatively sparse monitoring areas, whereas Wi-Fi Round Trip Time (RTT) or Bluetooth based on Electronic Shelf Tags (ESL) are more suitable. ® Low-power (BLE) RSSI positioning can provide high-fidelity positioning estimates comparable to vision-based estimates and may be useful for resolving ambiguities in crowded aisles with high-precision cross-correlation.

[0134] Furthermore, if additional information (such as key feature descriptors, 3D dimensions) has been previously calculated and stored for a given RF device / UE, this information can be cross-correlated with information from concurrent RoI detection and appropriate metrics can be derived to detect any potential outliers and prevent mismatches.

[0135] Figure 9 Figure 900 illustrates an example architecture for RF-assisted management of a partially occluded RoI for visual positioning according to various aspects of this disclosure. Figure 9 The illustrated example architecture includes multiple OTT cameras 910, multiple RF devices / UEs 920, and a server 930. It should be noted that, although... Figure 9 Four OTT cameras 910 are shown (two of which are OTT cameras attached). picture (The markings indicate this), but as will be understood, there may be more or fewer than four OTT cameras 910. Similarly, although Figure 9 Three RF devices / UEs 920 (each labeled "UE") are shown, but as will be understood, there may be more or fewer than three RF devices / UEs 920.

[0136] First, referring to server 930, server 930 can be an LMF (e.g., LMF 270) or a proprietary deployment-dependent server. Server 930 manages the localization engine and related pipelines (RF-based processing pipeline 932, vision-based processing pipeline 934, and hybrid processing pipeline (not shown)). Server 930 also manages incomplete RoI management module 936, which implements the functions, processes, and memory related to the RoI integrity and outlier removal methods described above. Incomplete RoI management module 936 may correspond to or be incorporated into... Figure 3C The localization component 398 is included. It should be noted that server 930 can also manage / train / fine-tune machine learning models for RoI generation, key feature extraction, and semantic segmentation, or the server can use Software as a Service (SaaS) and maintain connections to one or more external servers storing such models.

[0137] Referring now to the RF device / UE 920, each relevant target object in the environment (e.g., a customer or store employee using a proprietary store application) is associated with an RF-enabled device / UE 920 having a unique MAC address. The specific RF technology may depend on the actual deployment and the actual location of the device in the environment. Relevant measurements and technologies for industrial IoT in indoor environments include, for example, Wi-Fi RSSI, Wi-Fi RTT, ESL-BLE RSSI, and UWB. As described above, for the second function of outlier removal methods, the practicality of each of these technologies (or any hybrid / fusion combination) for managing partially occluded RoIs depends on their availability and the specific setup of the target relative to the monitored environment and other active targets.

[0138] Referring now to OTT camera 910, OTT camera 910 can be a closed-circuit television (CCTV) camera or other surveillance camera. Each OTT camera 910 maintains a connection to server 930. While OTT camera 910 can be used for other purposes (such as security monitoring, in which case server 930 will already have access to the relevant camera), the proposed technology also allows for situations where OTT camera 910 is part of a different infrastructure (e.g., an ESL-BLE mount camera), and therefore OTT camera can be activated upon request from server 930.

[0139] Figure 10 An example signaling flow 1000 for active target registration and 3D feature estimation according to various aspects of this disclosure is illustrated. The example signaling flow 1000 can be provided by... Figure 9 The OTT camera 910, RF device / UE 920 and server 930 are used to perform this.

[0140] Figure 11Example signaling flow 1100 for outlier detection and removal (RoI) according to various aspects of this disclosure is illustrated. Example signaling flow 1100 can be provided by... Figure 9 The OTT camera 910, RF device / UE 920 and server 930 are used to perform this.

[0141] While both the RoI full approach and outlier removal approach can be used as individual techniques, as will be understood, in multi-view... picture Within the system, these two methods can be used in a nested manner. Specifically, in multi-view... picture In the system, outlier removal methods can be used first to identify outliers. Then, severe outliers (i.e., outliers with scores above a predetermined threshold) or targets for which no 3D information is available (e.g., new active targets or specific assets) can be removed from the vision processing pipeline (e.g., vision-based processing pipeline 934). Subsequently, the RoI full approach can be applied to the remaining partially occluded RoIs. RF components of the localization engine (e.g., RF measurement or RF-based localization estimation implemented by RF-based processing pipeline 932) can be used at various stages of both approaches, depending on RF availability and the specific setup / scenario of the environment.

[0142] Figure 12 An example method 1200 for visual positioning according to various aspects of this disclosure is illustrated. In one aspect, method 1200 may be performed by a positioning entity (e.g., server 930).

[0143] At 1210, the positioning entity determines one or more images obtained from one or more cameras. picture The region of interest in the image, where the region of interest corresponds to one or more picture At least a portion of the target object detected in the image. In one aspect, operation 1210 may be performed by one or more network transceivers 390, one or more processors 394, memory 396 and / or positioning components 398, any or all of which may be considered as components for performing the operation.

[0144] At 1220, the location entity is based on one or more... picture Like obtaining a set of key visual features associated with a target object. In one aspect, operation 1220 may be performed by one or more network transceivers 390, one or more processors 394, memory 396 and / or positioning components 398, any or all of which can be considered as components for performing the operation.

[0145] At 1230, the positioning entity estimates the three-dimensional dimensions of the target object based at least in part on the type of the target object. In one aspect, operation 1230 can be performed by one or more network transceivers 390, one or more processors 394, memory 396, and / or positioning components 398, any or all of which can be considered as components for performing the operation.

[0146] At 1240, the location entity stores the set of key visual features and three-dimensional dimensions of the target object for subsequent location of the target object. In one aspect, operation 1240 can be performed by one or more network transceivers 390, one or more processors 394, memory 396, and / or location components 398, any or all of which can be considered as components used to perform the operation.

[0147] As will be understood, the technical advantage of method 1200 is that it improves vision-based localization performance by addressing occluded visual targets.

[0148] As can be seen in the detailed description above, different features are grouped together in the examples. This method of disclosure should not be construed as implying that the example clauses contain more features than those explicitly mentioned in each clause. picture Conversely, the various aspects of this disclosure may include fewer features than those of the individual example clauses disclosed. Therefore, the following clauses should be regarded accordingly as incorporated into the description, where each clause may serve as a separate example. Although each dependent clause may refer in the clause to a specific combination with one of the other clauses, the aspect of that dependent clause is not limited to that specific combination. It should be understood that other example clauses may also include combinations of aspects of a dependent clause with the subject matter of any other dependent or independent clause, or any feature combined with other dependent and independent clauses. The various aspects disclosed herein expressly include these combinations unless explicitly stated or readily inferred that a particular combination is not intended for use (e.g., contradictory aspects, such as defining an element as both an electrical insulator and an electrical conductor). Furthermore, it is contemplated that aspects of a clause may be included in any other independent clause, even if that clause does not directly depend on the independent clause.

[0149] Specific implementation examples are described in the following numbered clauses: Clause 1. A method for visual positioning performed by a positioning entity, the method comprising: determining one or more images obtained from one or more cameras (e.g., one or more cameras of a monitoring system and / or one or more cameras of one or more AGVs). picture Region of interest in an image, wherein the region of interest corresponds to one or more of the above. picture At least a portion of the target object detected in the image (e.g., a person, AGV, movable asset, etc.); based on the one or more of the above. picture The method includes obtaining a set of key visual features associated with the target object; estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and storing the set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0150] Clause 2. The method according to Clause 1, wherein the type of the set of key visual features is based on the type of the target object.

[0151] Clause 3. The method according to any one of Clauses 1 to 2, wherein: the region of interest includes the entirety of the target object, and the three-dimensional dimensions are estimated based on the dimensions of the region of interest.

[0152] Clause 4. The method according to any one of Clauses 1 to 2, wherein: the region of interest includes only a portion of the target object, and the three-dimensional dimensions are based on a radio frequency (RF)-based location estimate associated with the target object, the target object being in one or more... picture The estimated position is relative to the region of interest, or both.

[0153] Clause 5. The method according to Clause 4, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the user equipment (UE) associated with the target object.

[0154] Clause 6. The method according to any one of Clauses 1 to 5, wherein the set of key visual features and the three-dimensional dimensions of the target object are stored together with a unique identifier of the target object.

[0155] Clause 7. The method described in Clause 6, wherein the unique identifier of the target object is a unique identifier of the UE associated with the target object.

[0156] Clause 8. The method according to any one of Clauses 1 to 7, the method further comprising: deleting the set of key visual features and the three-dimensional dimensions of the target object after a first threshold time period.

[0157] Clause 9. The method according to Clause 8, wherein the set of key visual features and the three-dimensional dimensions of the target object are based on data from the one or more cameras. picture The target object was not detected during the second threshold time period and was deleted after the first threshold time period.

[0158] Clause 10. The method according to any one of Clauses 1 to 9, wherein: the one or more cameras comprise a plurality of cameras, and from each of the plurality of cameras... picture The target object was detected in the image.

[0159] Clause 11. The method according to Clause 10, the method further comprising: based on the data from each of the plurality of cameras... picture This includes determining multiple location estimates of the target object and performing outlier removal on the multiple location estimates of the target object.

[0160] Clause 12. The method according to Clause 11, wherein performing the outlier removal comprises: aggregating the plurality of location estimates of the target object into one or more clusters; determining a set of candidate outliers of the plurality of location estimates; assigning a score to each of the set of candidate outliers; and removing candidate outliers from the set of candidate outliers that have a score greater than an outlier threshold.

[0161] Clause 13. The method according to Clause 12, wherein the score is based on the distance between the candidate outlier and the center of the nearest cluster in the one or more clusters.

[0162] Clause 14. The method according to any one of Clauses 11 to 13, wherein performing the outlier removal comprises: determining a RF-based location estimate associated with the target object; and removing the location estimate based on the distance between the location estimate of the plurality of location estimates of the target object and the RF-based location estimate being greater than a threshold.

[0163] Clause 15. The method according to Clause 14, wherein the threshold is based on the type of the RF-based location estimation.

[0164] Clause 16. The method according to any one of Clauses 14 to 15, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the UE associated with the target object.

[0165] Clause 17. The method according to any one of Clauses 1 to 16, the method further comprising: sending a request for RF measurements obtained by a UE associated with the target object to the UE; receiving the RF measurements obtained by the UE from the UE; and determining a location estimate of the UE based on the RF measurements.

[0166] Clause 18. The method according to any one of Clauses 1 to 17, the method further comprising: sending a request for a state of a UE associated with the target object to the UE; receiving the state of the UE from the UE; and further estimating the three-dimensional dimensions of the target object based on the state of the UE.

[0167] Clause 19. A positioning entity comprising: one or more memories; one or more transceivers; and one or more processors communicatively coupled to the one or more memories and the one or more transceivers, the one or more processors being individually or in combination configured to: determine one or more [data / images] obtained from one or more cameras. picture Region of interest in an image, wherein the region of interest corresponds to one or more of the above. picture At least a portion of the target object detected in the image; based on the one or more picture The method includes obtaining a set of key visual features associated with the target object; estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and storing the set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0168] Clause 20. The positioning entity as described in Clause 19, wherein the type of the set of key visual features is based on the type of the target object.

[0169] Clause 21. The positioning entity according to any one of Clauses 19 to 20, wherein: the region of interest includes the entirety of the target object, and the three-dimensional dimensions are estimated based on the dimensions of the region of interest.

[0170] Clause 22. The location entity according to any one of Clauses 19 to 20, wherein: the region of interest includes only the portion of the target object, and the three-dimensional dimensions are based on a radio frequency (RF)-based location estimate associated with the target object, the target object being located in one or more... picture The estimated position is relative to the region of interest, or both.

[0171] Clause 23. The location entity as described in Clause 22, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the user equipment (UE) associated with the target object.

[0172] Clause 24. The positioning entity according to any one of Clauses 19 to 23, wherein the set of key visual features and the three-dimensional dimensions of the target object are stored together with a unique identifier of the target object.

[0173] Clause 25. The location entity as described in Clause 24, wherein the unique identifier of the target object is a unique identifier of the UE associated with the target object.

[0174] Clause 26. The positioning entity according to any one of Clauses 19 to 25, wherein the one or more processors are further configured individually or in combination to: delete the set of key visual features and the three-dimensional dimensions of the target object after a first threshold time period.

[0175] Clause 27. The positioning entity as described in Clause 26, wherein the set of key visual features and the three-dimensional dimensions of the target object are based on data from the one or more cameras. picture The target object was not detected during the second threshold time period and was deleted after the first threshold time period.

[0176] Clause 28. The positioning entity according to any one of Clauses 19 to 27, wherein: the one or more cameras comprise a plurality of cameras, and from each of the plurality of cameras... picture The target object was detected in the image.

[0177] Clause 29. The positioning entity as described in Clause 28, wherein said one or more processors are further configured individually or in combination to: based on the data from each of said plurality of cameras... picture This includes determining multiple location estimates of the target object and performing outlier removal on the multiple location estimates of the target object.

[0178] Clause 30. The location entity as described in Clause 29, wherein the one or more processors configured to perform the outlier removal includes the one or more processors configured individually or in combination to: aggregate the plurality of location estimates of the target object into one or more clusters; determine a set of candidate outliers of the plurality of location estimates; assign a score to each of the candidate outliers in the set of candidate outliers; and remove candidate outliers from the set of candidate outliers that have a score greater than the outlier threshold.

[0179] Clause 31. The location entity as described in Clause 30, wherein the score is based on the distance between the candidate outlier and the center of the nearest cluster in the one or more clusters.

[0180] Clause 32. The location entity according to any one of Clauses 29 to 31, wherein the one or more processors configured to perform the outlier removal includes the one or more processors configured individually or in combination to: determine a RF-based location estimate associated with the target object; and remove the location estimate based on the distance between the location estimate of the target object and the RF-based location estimate being greater than a threshold.

[0181] Clause 33. The location entity as described in Clause 32, wherein the threshold is based on the type of the RF-based location estimation.

[0182] Clause 34. The location entity according to any one of Clauses 32 to 33, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the UE associated with the target object.

[0183] Clause 35. The positioning entity according to any one of Clauses 19 to 34, wherein the one or more processors are further configured individually or in combination to: send a request for RF measurements obtained by a UE associated with the target object to the UE via the one or more transceivers; receive the RF measurements obtained by the UE from the UE via the one or more transceivers; and determine a location estimate of the UE based on the RF measurements.

[0184] Clause 36. The positioning entity according to any one of Clauses 19 to 35, wherein the one or more processors are further configured individually or in combination to: send a request for a state of a UE associated with the target object to the UE via the one or more transceivers; receive the state of the UE from the UE via the one or more transceivers; and further estimate the three-dimensional dimensions of the target object based on the state of the UE.

[0185] Clause 37. A positioning entity, the positioning entity comprising: for determining one or more images obtained from one or more cameras. picture The component of the region of interest in the image, wherein the region of interest corresponds to the one or more of the components. picture At least a portion of the target object detected in the image; for use based on the one or more picture The components include: a set of key visual features associated with the target object; a set of components for estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and a set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0186] Clause 38. The positioning entity as described in Clause 37, wherein the type of the set of key visual features is based on the type of the target object.

[0187] Clause 39. The positioning entity according to any one of Clauses 37 to 38, wherein: the region of interest includes the entirety of the target object, and the three-dimensional dimensions are estimated based on the dimensions of the region of interest.

[0188] Clause 40. The location entity according to any one of Clauses 37 to 38, wherein: the region of interest includes only the portion of the target object, and the three-dimensional dimensions are based on a radio frequency (RF)-based location estimate associated with the target object, the target object being in one or more... picture The estimated position is relative to the region of interest, or both.

[0189] Clause 41. The location entity as described in Clause 40, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the user equipment (UE) associated with the target object.

[0190] Clause 42. The positioning entity according to any one of Clauses 37 to 41, wherein the set of key visual features and the three-dimensional dimensions of the target object are stored together with a unique identifier of the target object.

[0191] Clause 43. The location entity as described in Clause 42, wherein the unique identifier of the target object is a unique identifier of the UE associated with the target object.

[0192] Clause 44. The positioning entity according to any one of Clauses 37 to 43, the positioning entity further comprising: a component for deleting the set of key visual features and the three-dimensional dimensions of the target object after a first threshold time period.

[0193] Clause 45. The positioning entity as described in Clause 44, wherein the set of key visual features and the three-dimensional dimensions of the target object are based on data from the one or more cameras. picture The target object was not detected during the second threshold time period and was deleted after the first threshold time period.

[0194] Clause 46. The positioning entity according to any one of Clauses 37 to 45, wherein: the one or more cameras comprise a plurality of cameras, and from each of the plurality of cameras... picture The target object was detected in the image.

[0195] Clause 47. The positioning entity as described in Clause 46, the positioning entity further comprising: for positioning based on the data from each of the plurality of cameras. picture The components include a component for determining multiple location estimates of the target object, and a component for performing outlier removal on the multiple location estimates of the target object.

[0196] Clause 48. The location entity as described in Clause 47, wherein the components for performing the outlier removal include: components for aggregating the plurality of location estimates of the target object into one or more clusters; components for determining a set of candidate outliers of the plurality of location estimates; components for assigning a score to each of the set of candidate outliers; and components for removing candidate outliers from the set of candidate outliers that have a score greater than an outlier threshold.

[0197] Clause 49. The location entity as described in Clause 48, wherein the score is based on the distance between the candidate outlier and the center of the nearest cluster in the one or more clusters.

[0198] Clause 50. The location entity according to any one of Clauses 47 to 49, wherein the component for performing the outlier removal comprises: a component for determining an RF-based location estimate associated with the target object; and a component for removing the location estimate based on the distance between the location estimate of the plurality of location estimates of the target object and the RF-based location estimate being greater than a threshold.

[0199] Clause 51. The location entity as described in Clause 50, wherein the threshold is based on the type of the RF-based location estimation.

[0200] Clause 52. The location entity according to any one of Clauses 50 to 51, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the UE associated with the target object.

[0201] Clause 53. The positioning entity according to any one of Clauses 37 to 52, the positioning entity further comprising: a component for sending a request to the UE for RF measurements obtained by the UE associated with the target object; a component for receiving the RF measurements obtained by the UE from the UE; and a component for determining a location estimate of the UE based on the RF measurements.

[0202] Clause 54. The positioning entity according to any one of Clauses 37 to 53, the positioning entity further comprising: a component for sending a request for a state of a UE associated with the target object to the UE; a component for receiving the state of the UE from the UE; and a component for further estimating the three-dimensional dimensions of the target object based on the state of the UE.

[0203] Clause 55. A non-transitory computer-readable medium storing computer-executable instructions, when executed by a positioning entity, causing the positioning entity to perform the following operations: determine one or more images obtained from one or more cameras. picture Region of interest in an image, wherein the region of interest corresponds to one or more of the above. picture At least a portion of the target object detected in the image; based on the one or more picture The method includes obtaining a set of key visual features associated with the target object; estimating the three-dimensional dimensions of the target object based at least in part on the type of the target object; and storing the set of key visual features and the three-dimensional dimensions of the target object for subsequent localization of the target object.

[0204] Clause 56. The non-transitory computer-readable medium as described in Clause 55, wherein the type of said set of key visual features is based on said type of said target object.

[0205] Clause 57. A non-transitory computer-readable medium according to any one of Clauses 55 to 56, wherein: the region of interest includes the entirety of the target object, and the three-dimensional dimensions are estimated based on the dimensions of the region of interest.

[0206] Clause 58. A non-transitory computer-readable medium according to any one of Clauses 55 to 56, wherein: the region of interest includes only the portion of the target object, and the three-dimensional dimensions are based on a radio frequency (RF)-based location estimate associated with the target object, the target object being in one or more... picture The estimated position is relative to the region of interest, or both.

[0207] Clause 59. The non-transitory computer-readable medium as described in Clause 58, wherein the RF-based location estimation associated with the target object includes an RF-based location estimation of the user equipment (UE) associated with the target object.

[0208] Clause 60. A non-transitory computer-readable medium according to any one of Clauses 55 to 59, wherein the set of key visual features and the three-dimensional dimensions of the target object are stored together with a unique identifier of the target object.

[0209] Clause 61. The non-transitory computer-readable medium as described in Clause 60, wherein the unique identifier of the target object is a unique identifier of the UE associated with the target object.

[0210] Clause 62. The non-transitory computer-readable medium according to any one of Clauses 55 to 61, the non-transitory computer-readable medium further comprising, when executed by the positioning entity, computer-executable instructions causing the positioning entity to perform the following operation: delete the set of key visual features and the three-dimensional dimensions of the target object after a first threshold time period.

[0211] Clause 63. The non-transitory computer-readable medium as described in Clause 62, wherein the set of key visual features and the three-dimensional dimensions of the target object are based on data from the one or more cameras. picture The target object was not detected during the second threshold time period and was deleted after the first threshold time period.

[0212] Clause 64. A non-transitory computer-readable medium according to any one of Clauses 55 to 63, wherein: the one or more cameras comprise a plurality of cameras, and from each of the plurality of cameras... picture The target object was detected in the image.

[0213] Clause 65. The non-transitory computer-readable medium according to Clause 64, further comprising, when executed by the positioning entity, computer-executable instructions causing the positioning entity to perform the following operations based on data from each of the plurality of cameras: picture This includes determining multiple location estimates of the target object and performing outlier removal on the multiple location estimates of the target object.

[0214] Clause 66. The non-transitory computer-readable medium according to Clause 65, wherein the computer-executable instructions that, when executed by the positioning entity, cause the positioning entity to perform the outlier removal include computer-executable instructions that, when executed by the positioning entity, cause the positioning entity to: aggregate the plurality of location estimates of the target object into one or more clusters; determine a set of candidate outliers of the plurality of location estimates; assign a score to each of the set of candidate outliers; and remove candidate outliers from the set of candidate outliers that have a score greater than an outlier threshold.

[0215] Clause 67. The non-transitory computer-readable medium as described in Clause 66, wherein the score is based on the distance between the candidate outlier and the center of the nearest cluster in the one or more clusters.

[0216] Clause 68. A non-transitory computer-readable medium according to any one of Clauses 65 to 67, wherein the computer-executable instructions that, when executed by the positioning entity, cause the positioning entity to perform the outlier removal include computer-executable instructions that, when executed by the positioning entity, cause the positioning entity to: determine a RF-based location estimate associated with the target object; and remove the location estimate based on the fact that a distance between one of the plurality of location estimates of the target object and the RF-based location estimate is greater than a threshold.

[0217] Clause 69. The non-transitory computer-readable medium as described in Clause 68, wherein the threshold is based on the type of the RF-based location estimation.

[0218] Clause 70. A non-transitory computer-readable medium according to any one of Clauses 68 to 69, wherein the RF-based location estimation associated with the target object includes an RF-based location estimation of a UE associated with the target object.

[0219] Clause 71. The non-transitory computer-readable medium according to any one of Clauses 55 to 70, the non-transitory computer-readable medium further comprising, when executed by the positioning entity, computer-executable instructions that cause the positioning entity to: send a request for RF measurements obtained by a UE associated with the target object to the UE; receive the RF measurements obtained by the UE from the UE; and determine a location estimate of the UE based on the RF measurements.

[0220] Clause 72. The non-transitory computer-readable medium according to any one of Clauses 55 to 71, the non-transitory computer-readable medium further comprising, when executed by the positioning entity, computer-executable instructions that cause the positioning entity to: send a request for a state of a UE associated with the target object to the UE; receive the state of the UE from the UE; and further estimate the three-dimensional dimensions of the target object based on the state of the UE.

[0221] Those skilled in the art will understand that information and signals can be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.

[0222] Furthermore, those skilled in the art will understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above in general terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application; however, such implementation decisions should not be construed as departing from the scope of this disclosure.

[0223] The various exemplary logic blocks, modules, and circuits described in conjunction with the aspects disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic components, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternative embodiments, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.

[0224] The methods, sequences, and / or algorithms described in conjunction with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or a combination of both. The software module may reside in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. Example storage media are coupled to a processor such that the processor can read information from and write information to the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal (e.g., a UE). Alternatively, the processor and storage medium may reside as discrete components in the user terminal.

[0225] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted via a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, including any medium that facilitates the transfer of a computer program from one place to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage devices, disk storage devices or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and is accessible to a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disks and optical discs include: compact optical discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.

[0226] While the foregoing disclosure illustrates exemplary aspects of this disclosure, it should be noted that various changes and modifications may be made herein without departing from the scope of this disclosure as defined by the appended claims. For example, the functions, steps, and / or actions of the method claims according to aspects of this disclosure described herein need not be performed in any particular order. Furthermore, no component, function, action, or instruction described or claimed herein should be construed as critical or essential unless explicitly stated otherwise. Additionally, as used herein, the terms “set,” “group,” etc., are intended to include one or more of the stated elements. Furthermore, as used herein, the terms “having,” “comprising,” “including,” etc., do not exclude the presence of one or more additional elements (e.g., element “having” A may also have B). Furthermore, the phrase “based on” is intended to mean “at least partially based on” unless otherwise explicitly stated. Furthermore, as used herein, the term “or” is intended to be open-ended when used in a series and is interchangeable with “and / or” unless otherwise explicitly stated (e.g., if used in conjunction with “any” or “only one”), or these alternatives are mutually exclusive (e.g., “one or more” should not be interpreted as “one and more”). Additionally, although components, functions, actions, and instructions may be described or claimed in the singular, plural forms may also be considered unless explicitly stated to be limited to the singular. Therefore, as used herein, the articles “a,” “an,” “the,” and “described” are intended to include one or more of the stated elements. Additionally, as used herein, the terms “at least one” and “one or more” include “one” component, function, action, or instruction that performs or is capable of performing the described or claimed functionality, and also include “two or more” components, functions, actions, or instructions that perform or are capable of performing the described or claimed functionality in combination.

Claims

1. A positioning entity, the positioning entity comprising: One or more memory units; One or more transceivers; and One or more processors, communicatively coupled to one or more memories and one or more transceivers, wherein the one or more processors are configured individually or in combination to: Determine a region of interest in one or more images obtained from one or more cameras, wherein the region of interest corresponds to at least a portion of a target object detected in the one or more images; Based on the one or more images, obtain a set of key visual features associated with the target object; The three-dimensional dimensions of the target object are estimated, at least in part, based on the type of the target object; as well as The set of key visual features and the three-dimensional dimensions of the target object are stored for subsequent localization of the target object.

2. The positioning entity according to claim 1, wherein the type of the set of key visual features is based on the type of the target object.

3. The positioning entity according to claim 1, wherein: The region of interest includes the entire target object, and The three-dimensional dimensions are estimated based on the dimensions of the region of interest.

4. The positioning entity according to claim 1, wherein: The region of interest includes only a portion of the target object, and The three-dimensional dimensions are estimated based on radio frequency (RF)-based location estimation associated with the target object, the estimated position of the target object relative to the region of interest in one or more images, or both.

5. The location entity of claim 4, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the user equipment (UE) associated with the target object.

6. The positioning entity according to claim 1, wherein the set of key visual features and the three-dimensional dimensions of the target object are stored together with a unique identifier of the target object.

7. The positioning entity of claim 6, wherein the unique identifier of the target object is a unique identifier of the UE associated with the target object.

8. The positioning entity of claim 1, wherein the one or more processors are further configured individually or in combination to: The set of key visual features and the three-dimensional dimensions of the target object are deleted after the first threshold time period.

9. The positioning entity of claim 8, wherein the set of key visual features and the three-dimensional dimensions of the target object are deleted after the first threshold time period based on the fact that the target object was not detected in images from the one or more cameras for a second threshold time period.

10. The positioning entity according to claim 1, wherein: The one or more cameras include multiple cameras, and The target object was detected in images from each of the plurality of cameras.

11. The positioning entity of claim 10, wherein the one or more processors are further configured individually or in combination to: Multiple location estimates of the target object are determined based on the images from each of the plurality of cameras; and Outlier removal is performed on the multiple location estimates of the target object.

12. The location entity of claim 11, wherein the one or more processors configured to perform the outlier removal includes the one or more processors individually or in combination configured to perform the following operations: The multiple location estimates of the target object are aggregated into one or more clusters; Determine a set of candidate outliers from the plurality of location estimates; Assign a score to each candidate outlier in the set of candidate outliers; as well as Candidate outliers that have a score greater than the outlier threshold are removed from the set of candidate outliers.

13. The location entity of claim 12, wherein the score is based on the distance between the candidate outlier and the center of the nearest cluster in the one or more clusters.

14. The location entity of claim 11, wherein the one or more processors configured to perform the outlier removal includes the one or more processors individually or in combination configured to perform the following operations: Determine the RF-based location estimate associated with the target object; and The location estimate is discarded based on the fact that the distance between the location estimate of the target object and the RF-based location estimate is greater than a threshold.

15. The location entity of claim 14, wherein the threshold is based on the type of the RF-based location estimation.

16. The positioning entity of claim 14, wherein the RF-based location estimation associated with the target object includes the RF-based location estimation of the UE associated with the target object.

17. The positioning entity of claim 1, wherein the one or more processors are further configured individually or in combination to: A request for RF measurements obtained by a UE associated with the target object is sent to the UE via the one or more transceivers; Receive the RF measurements obtained by the UE from the UE via the one or more transceivers; and The location estimate of the UE is determined based on the RF measurement.

18. The positioning entity of claim 1, wherein the one or more processors are further configured individually or in combination to: A request for the state of the UE associated with the target object is sent to the UE via the one or more transceivers; Receive the state of the UE from the UE via the one or more transceivers; and The three-dimensional dimensions of the target object are further estimated based on the state of the UE.

19. A method for visual localization performed by a localization entity, the method comprising: Determine a region of interest in one or more images obtained from one or more cameras, wherein the region of interest corresponds to at least a portion of a target object detected in the one or more images; Based on the one or more images, obtain a set of key visual features associated with the target object; The three-dimensional dimensions of the target object are estimated, at least in part, based on the type of the target object; as well as The set of key visual features and the three-dimensional dimensions of the target object are stored for subsequent localization of the target object.

20. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by a positioning entity, cause the positioning entity to: Determine a region of interest in one or more images obtained from one or more cameras, wherein the region of interest corresponds to at least a portion of a target object detected in the one or more images; Based on the one or more images, obtain a set of key visual features associated with the target object; The three-dimensional dimensions of the target object are estimated, at least in part, based on the type of the target object; as well as The set of key visual features and the three-dimensional dimensions of the target object are stored for subsequent localization of the target object.

Images