Visual overhead monitoring for collaborative simultaneous localization and mapping
By utilizing a cooperative simultaneous localization and mapping (C-SLAM) system, which employs radio frequency measurement and visual channel state information, the problem of low efficiency in localization and mapping in wireless communication systems is solved, achieving higher positioning accuracy and mapping efficiency.
Patent Information
- Application Number
- CN202480051523.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-08-18
- Filing Date
- 2024-07-09
- Publication Date
- 2026-03-20
AI Technical Summary
Existing wireless communication systems suffer from inefficiency and insufficient accuracy in positioning and mapping, especially in complex environments where high-accuracy positioning and mapping are difficult to achieve.
By utilizing the cooperative simultaneous localization and mapping (C-SLAM) system, radio frequency (RF) measurements and visual channel state information (vCSI) from multiple intelligent agent devices, combined with the collaborative processing of network entities, the accuracy and efficiency of localization and mapping can be improved.
It achieves higher positioning accuracy and map building efficiency in complex environments, improving the overall performance of wireless communication systems.
Smart Images

Figure CN121713501A_ABST
Abstract
Description
Background Technology
[0001] 1. Technical Field
[0002] All aspects of this disclosure relate to wireless technology.
[0003] 2. Related technical descriptions
[0004] Wireless communication systems have evolved through many generations, including first-generation analog radiotelephone service (1G), second-generation (2G) digital radiotelephone service (including transitional 2.5G and 2.75G networks), third-generation (3G) high-speed data, wireless services with internet capabilities, and fourth-generation (4G) services (e.g., Long Term Evolution (LTE) or WiMax). Currently, many different types of wireless communication systems are in use, including cellular systems and Personal Communication Services (PCS) systems. Known examples of cellular systems include cellular analog Advanced Mobile Phone Systems (AMPS), as well as digital cellular systems based on Code Division Multiple Access (CDMA), Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Global System for Mobile Communications (GSM), and others.
[0005] The fifth-generation (5G) wireless standard, known as New Radio (NR), delivers higher data transfer speeds, more connections, better coverage, and other improvements. According to the Next Generation Mobile Networks Alliance (NGC), the 5G standard is designed to provide higher data rates, more accurate positioning (e.g., based on Positioning Reference Signals (RS-P), such as downlink, uplink, or sidelink Positioning Reference Signals (PRS)), and other technological enhancements compared to previous standards. These enhancements, along with the use of higher frequency bands, advancements in the PRS process and technology, and the high-density deployment of 5G, enable high-accuracy positioning based on 5G. Summary of the Invention
[0006] The following is a simplified summary of the invention relating to one or more aspects disclosed herein. Therefore, this summary should not be considered an exhaustive overview relating to all conceived aspects, nor should it be considered to identify key or decisive elements relating to all conceived aspects or to depict the scope associated with any particular aspect. Thus, the sole purpose of this summary is to present, in a simplified form, certain concepts relating to one or more aspects involving the mechanisms disclosed herein, prior to the detailed description presented below.
[0007] In one aspect, a method of communication performed by a network entity includes: receiving multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; receiving visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system; and sending a command to at least one of the multiple agent devices to the at least one agent device, wherein the command is at least partially based on the vCSI.
[0008] In one aspect, a network entity includes: one or more memories; one or more transceivers; and one or more processors communicatively coupled to the one or more memories and the one or more transceivers, the one or more processors being individually or in combination configured to: receive multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system via the one or more transceivers, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; receive visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system via the one or more transceivers; and send a command to at least one agent device among the multiple agent devices via the one or more transceivers, wherein the command is based at least in part on the vCSI, the RF measurements, or both.
[0009] In one aspect, a network entity includes: components for receiving multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; components for receiving visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system; and components for sending a command to at least one of the multiple agent devices to the at least one agent device, wherein the command is based at least in part on the vCSI, the RF measurements, or both.
[0010] In one aspect, a non-transitory computer-readable medium stores computer-executable instructions that, when executed by a network entity, cause the network entity to: receive multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; receive visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system; and send a command to at least one of the multiple agent devices to that at least one agent device, wherein the command is based at least in part on the vCSI, the RF measurements, or both.
[0011] Based on the accompanying drawings and detailed description, other objects and advantages associated with the aspects disclosed herein will be apparent to those skilled in the art. Attached Figure Description
[0012] The accompanying drawings are provided to help describe various aspects of this disclosure, and are provided for illustrative purposes only and not to limit the aspects.
[0013] Figure 1 Example wireless communication systems according to various aspects of this disclosure are illustrated.
[0014] Figure 2A , Figure 2B and Figure 2C Example wireless network architectures based on various aspects of this disclosure are illustrated.
[0015] Figure 3A , Figure 3B and Figure 3C It is a simplified block diagram of several examples of components that can be used in user equipment (UE), base stations and network entities and configured to support communications as taught herein.
[0016] Figure 4 This is a diagram illustrating examples of several intelligent agents navigating around various environmental landmarks according to various aspects of this disclosure.
[0017] Figure 5 This is an illustration of an example cooperative simultaneous localization and mapping (C-SLAM) system according to various aspects of this disclosure.
[0018] Figure 6 This is an illustration of an example industrial monitoring scenario based on various aspects of this disclosure.
[0019] Figure 7 Examples of various triggering events that may lead to the exchange of map information between agents according to various aspects of this disclosure are illustrated.
[0020] Figure 8This is a diagram illustrating an example message stream for server-initiated Visual Channel State Information (vCSI) assistance functionality according to various aspects of this disclosure.
[0021] Figure 9 This is a diagram illustrating an example message flow for an agent-initiated vCSI assistive function according to various aspects of this disclosure.
[0022] Figure 10 Example methods of communication according to various aspects of this disclosure are illustrated. Detailed Implementation
[0023] Various aspects of this disclosure are provided in the following description and accompanying drawings of various examples provided for illustrative purposes. Alternative aspects may be devised without departing from the scope of this disclosure. Additionally, well-known elements of this disclosure will not be described in detail or will be omitted so as not to obscure the relevant details of this disclosure.
[0024] The various aspects collectively involve cooperative simultaneous localization and mapping (C-SLAM). Some aspects more specifically involve enhanced C-SLAM using visual over-the-top monitoring systems. In some examples, a network entity (e.g., a server) receives multiple sensor reports from multiple agent devices, which include at least radio frequency (RF) measurements obtained by the multiple agent devices. The network entity also receives visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from the visual monitoring system. The network entity then sends a command to at least one of the multiple agent devices, wherein the command is at least partially based on the vCSI.
[0025] Specific aspects of the subject matter described in this disclosure can be implemented to achieve one or more of the following potential advantages. In some examples, by receiving vCSI and sending commands based on vCSI, the described techniques can be used to improve C-SLAM performance using vCSI.
[0026] The terms “exemplary” and / or “example” are used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” and / or “example” is not necessarily to be construed as superior to or better than other aspects. Similarly, the term “aspects of this disclosure” does not require that all aspects of this disclosure include the features, advantages, or modes of operation discussed.
[0027] Those skilled in the art will understand that any of a variety of different techniques and methods can be used to represent the information and signals described below. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the following description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof, depending in part on the specific application, in part on the desired design, in part on the corresponding technology, and so on.
[0028] Furthermore, many aspects are described according to a sequence of actions to be performed by elements of, for example, a computing device. It will be appreciated that the various actions described herein can be performed by specific circuitry (e.g., an application-specific integrated circuit (ASIC)), by program instructions executed by one or more processors, or by a combination of both. Additionally, the sequence of actions described herein can be considered to be entirely embodied in any form of non-transitory computer-readable storage medium storing a corresponding set of computer instructions that, when executed, will cause or command the associated processor of the device to perform the functionality described herein. Therefore, various aspects of this disclosure can be embodied in a variety of different forms, all of which are contemplated within the scope of the claimed subject matter. Furthermore, for each aspect described herein, any corresponding form of any such aspect may be described herein as, for example, "logic configured to perform the described actions."
[0029] As used herein, unless otherwise stated, the terms “User Equipment” (UE) and “Base Station” are not intended to be specific or otherwise limited to any particular Radio Access Technology (RAT). In general, a UE can be any wireless communication device used by a user to communicate over a wireless communication network (e.g., mobile phone, router, tablet computer, laptop computer, consumer asset positioning device, wearable device (e.g., smartwatch, glasses, augmented reality (AR) / virtual reality (VR) headset, etc.), vehicle (e.g., car, motorcycle, bicycle, etc.), Internet of Things (IoT) device, etc.). A UE can be mobile or can (e.g., at certain times) be stationary and can communicate with a Radio Access Network (RAN). As used herein, the term “UE” can be interchangeably referred to as “Access Terminal” or “AT,” “Client Equipment,” “Wireless Equipment,” “Subscriber Equipment,” “Subscriber Terminal,” “Subscriber Station,” “User Terminal” or “UT,” “Mobile Equipment,” “Mobile Terminal,” “Mobile Station,” or variations thereof. Overall, a UE can communicate with a core network via the RAN, and through the core network, a UE can connect to external networks such as the Internet and to other UEs. Of course, other mechanisms for connecting to the core network and / or the Internet are also possible for the UE, such as through wired access networks, wireless local area network (WLAN) networks (e.g., based on the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard, etc.).
[0030] A base station may operate according to one of several RATs to communicate with the UE, depending on the network in which it is deployed, and may alternatively be referred to as an Access Point (AP), Network Node, Node B, Evolved Node B (eNB), Next Generation eNB (ng-eNB), New Radio (NR) Node B (also referred to as gNB or gNodeB), etc. The base station may primarily be used to support the UE's radio access, including supporting data, voice, and / or signaling connections for the supported UE. In some systems, the base station may only provide edge node signaling functions, while in others, it may provide additional control and / or network management functions. The communication link through which the UE can transmit signals to the base station is called an uplink (UL) channel (e.g., reverse traffic channel, reverse control channel, access channel, etc.). The communication link through which the base station can transmit signals to the UE is called a downlink (DL) or forward link channel (e.g., paging channel, control channel, broadcast channel, forward traffic channel, etc.). As used herein, the term “Traffic Channel (TCH)” may refer to the uplink / reverse traffic channel or the downlink / forward traffic channel.
[0031] The term "base station" can refer to a single physical transmit / receive point (TRP) or multiple physical TRPs that may or may not be co-located. For example, when the term "base station" refers to a single physical TRP, the physical TRP can be the antenna of a base station corresponding to a cell (or several cell sectors) of the base station. When the term "base station" refers to multiple co-located physical TRPs, the physical TRP can be the antenna array of the base station (e.g., as in a multiple-input multiple-output (MIMO) system or where the base station employs beamforming). When the term "base station" refers to multiple non-co-located physical TRPs, the physical TRP can be a distributed antenna system (DAS) (a network of spatially separated antennas connected via a transmission medium to a common source) or a remote radio headend (RRH) (a remote base station connected to a serving base station). Alternatively, a non-co-located physical TRP can be the serving base station from which the UE receives measurement reports and a neighboring base station where the UE is measuring its reference radio frequency (RF) signal. Because, as used herein, a TRP is the point by which a base station transmits and receives radio signals, references to transmitting from or receiving at a base station should be understood to refer to a specific TRP of the base station.
[0032] In some specific implementations supporting UE positioning, the base station may not support the UE's radio access (e.g., it may not support data, voice, and / or signaling connections for the UE), but may instead transmit reference signals to the UE for measurement and / or receive and measure signals transmitted by the UE. Such a base station may be referred to as a positioning beacon (e.g., in the case of transmitting signals to the UE) and / or as a location measurement unit (e.g., in the case of receiving and measuring signals from the UE).
[0033] An “RF signal” refers to an electromagnetic wave of a given frequency that transmits information across the space between a transmitter and a receiver. As used herein, a transmitter may send a single “RF signal” or multiple “RF signals” to a receiver. However, due to the propagation characteristics of RF signals through multipath channels, a receiver may receive multiple “RF signals” corresponding to each transmitted RF signal. The same transmitted RF signal on different paths between the transmitter and receiver may be referred to as a “multipath” RF signal. As used herein, where the context clearly indicates that the term “signal” refers to a wireless signal or RF signal, an RF signal may also be referred to as a “wireless signal” or simply a “signal.”
[0034] Figure 1An example wireless communication system 100 according to various aspects of this disclosure is illustrated. The wireless communication system 100 (which may also be referred to as a wireless wide area network (WWAN)) may include various base stations 102 (labeled "BS") and various UEs 104. Base station 102 may include macro cell base stations (high-power cellular base stations) and / or small cell base stations (low-power cellular base stations). In one aspect, the macro cell base station may include an eNB and / or an ng-eNB (wherein the wireless communication system 100 corresponds to an LTE network), or a gNB (wherein the wireless communication system 100 corresponds to an NR network), or a combination of both, and the small cell base station may include femtocells, picocells, microcells, etc.
[0035] Base station 102 can collectively form a RAN and interface with core network 170 (e.g., evolved packet core (EPC) or 5G core (5GC)) via backhaul link 122, and interface with one or more location servers 172 (e.g., location management function (LMF) or secure user plane location (SUPL) location platform (SLP)) via core network 170. Location server 172 can be part of core network 170 or can be external to core network 170. Location server 172 can be integrated with base station 102. UE 104 can communicate with location server 172 directly or indirectly. For example, UE 104 can communicate with location server 172 via base station 102 currently serving UE 104. UE 104 can also communicate with location server 172 via another path, such as via application server (not shown), via another network, such as via wireless local area network (WLAN) access point (AP) (e.g., AP 150 described below), etc. For signaling purposes, communication between UE 104 and location server 172 can be represented as an indirect connection (e.g., via core network 170, etc.) or a direct connection (e.g., as shown via direct connection 128), wherein intermediate nodes (if present) are omitted from the signaling diagram for clarity.
[0036] In addition to other functions, base station 102 may perform functions associated with one or more of the following: transmitting user data, radio channel encryption and decryption, integrity protection, header compression, mobility control functions (e.g., handover, dual connectivity), inter-cell interference coordination, connection establishment and release, load balancing, distribution of non-access stratum (NAS) messages, NAS node selection, synchronization, RAN sharing, multimedia broadcast multicast service (MBMS), subscriber and equipment tracking, RAN information management (RIM), paging, location, and delivery of warning messages. Base stations 102 may communicate with each other directly or indirectly (e.g., via EPC / 5GC) on backhaul link 134, which may be wired or wireless.
[0037] Base station 102 can wirelessly communicate with UE 104. Each base station in base station 102 can provide communication coverage for a corresponding geographic coverage area 110. In one aspect, one or more cells can be supported by base station 102 in each geographic coverage area 110. A “cell” is a logical communication entity used to communicate with a base station (e.g., via a frequency resource, which is referred to as a carrier frequency, component carrier, carrier, frequency band, etc.) and can be associated with an identifier (e.g., Physical Cell Identifier (PCI), Enhanced Cell Identifier (ECI), Virtual Cell Identifier (VCI), Cell Global Identifier (CGI), etc.) used to distinguish cells operating via the same or different carrier frequencies. In some cases, different cells can be configured according to different protocol types that can provide access for different types of UEs (e.g., Machine Type Communication (MTC), Narrowband IoT (NB-IoT), Enhanced Mobile Broadband (eMBB), or other protocol types). Because a cell is supported by a specific base station, the term “cell” can refer to either or both of the logical communication entity and the base station supporting the logical communication entity, depending on the context. Furthermore, since the TRP is typically the physical transmission point of a cell, the terms "cell" and "TRP" can be used interchangeably. In some cases, the term "cell" can also refer to the geographical coverage area of a base station (e.g., a sector), as long as the carrier frequency can be detected and used for communication within a portion of the geographical coverage area 110.
[0038] While the geographic coverage areas 110 of adjacent macro cell base stations 102 may partially overlap (e.g., in handover areas), some areas within geographic coverage areas 110 may substantially overlap with larger geographic coverage areas 110. For example, a small cell base station 102' (labeled "SC" for "small cell") may have a geographic coverage area 110' that substantially overlaps with the geographic coverage areas 110 of one or more macro cell base stations 102. A network that includes both small cell base stations and macro cell base stations may be referred to as a heterogeneous network. A heterogeneous network may also include a home eNB (HeNB) that can provide service to a restricted group referred to as a Closed Subscriber Group (CSG).
[0039] The communication link 120 between base station 102 and UE 104 may include uplink (also known as reverse link) transmission from UE 104 to base station 102 and / or downlink (DL) (also known as forward link) transmission from base station 102 to UE 104. The communication link 120 may use MIMO antenna techniques, including spatial multiplexing, beamforming, and / or transmit diversity. The communication link 120 may use one or more carrier frequencies. Carrier allocation may be asymmetric for the downlink and uplink (e.g., more or fewer carriers may be allocated to the downlink compared to the uplink).
[0040] The wireless communication system 100 may also include a WLAN access point (AP) 150 that communicates with a wireless local area network (WLAN) station (STA) 152 via a communication link 154 in unlicensed spectrum (e.g., 5 GHz). When communicating in unlicensed spectrum, the WLAN STA 152 and / or WLAN AP 150 may perform a free channel assessment (CCA) or listen-before-talk (LBT) process before communication to determine whether the channel is available.
[0041] Small cell base station 102' can operate in licensed and / or unlicensed spectrum. When operating in unlicensed spectrum, small cell base station 102' can employ LTE or NR technology and use the same 5GHz unlicensed spectrum as WLAN AP 150. Small cell base station 102' employing LTE / 5G in unlicensed spectrum can improve the coverage and / or increase the capacity of the access network. NR in unlicensed spectrum may be referred to as NR-U. LTE in unlicensed spectrum may be referred to as LTE-U, Licensed Assisted Access (LAA), or MULTEFIRE. ® .
[0042] The wireless communication system 100 may also include a millimeter-wave (mmW) base station 180, which can operate at mmW and / or near-mmW frequencies to communicate with the UE 182. Extremely high frequency (EHF) is a portion of the electromagnetic spectrum that contains radio frequency (RF). EHF has a range of 30 GHz to 300 GHz, with wavelengths between 1 mm and 10 mm. Radio waves in this band are referred to as millimeter waves. Near-mmW extends down to frequencies of 3 GHz with wavelengths of 100 mm. Ultra-high frequency (SHF) bands extend between 3 GHz and 30 GHz, and are also referred to as centimeter waves. Communication using mmW / near-mmW radio bands has high path loss and relatively short range. The mmW base station 180 and the UE 182 can utilize beamforming (transmit and / or receive) on the mmW communication link 184 to compensate for the extremely high path loss and short range. Furthermore, it should be understood that, in alternative configurations, one or more base stations 102 may also use mmW or near-mmW and beamforming for transmission. Therefore, it should be understood that the foregoing examples are merely illustrative and should not be construed as limiting the various aspects disclosed herein.
[0043] Transmit beamforming is a technique used to focus RF signals in a specific direction. Traditionally, when a network node (e.g., a base station) broadcasts an RF signal, it broadcasts the signal in all directions (omnidirectionally). Using transmit beamforming, the network node determines where a given target device (e.g., a UE) is located (relative to the transmitting network node) and projects a stronger downlink RF signal in that specific direction, thus providing the receiving device with a faster and stronger RF signal (in terms of data rate). To change the directivity of the RF signal during transmission, the network node can control the phase and relative amplitude of the RF signal at each of one or more transmitters broadcasting the RF signal. For example, the network node can use an array of antennas (called a "phased array" or "antenna array") that forms an RF beam that can be "manipulated" to be pointed in different directions without actually moving the antennas. Specifically, RF currents from the transmitters are fed to individual antennas with the correct phase relationship, such that radio waves from the individual antennas add up in the desired direction to increase radiation, while canceling out in the undesired direction to suppress radiation.
[0044] Transmit beams can be quasi-co-located, meaning they appear to the receiver (e.g., the UE) as having the same parameters regardless of whether the network node's own transmit antennas are physically co-located. In NR, there are four types of quasi-co-located (QCL) relationships. Specifically, a given type of QCL relationship means that certain parameters of a second reference RF signal on a second beam can be derived based on information about the source reference RF signal on the source beam. Therefore, if the source reference RF signal is QCL type A, the receiver can use the source reference RF signal to estimate the Doppler shift, Doppler spread, average delay, and delay spread of the second reference RF signal transmitted on the same channel. If the source reference RF signal is QCL type B, the receiver can use the source reference RF signal to estimate the Doppler shift and Doppler spread of the second reference RF signal transmitted on the same channel. If the source reference RF signal is QCL type C, the receiver can use the source reference RF signal to estimate the Doppler shift and average delay of the second reference RF signal transmitted on the same channel. If the source reference RF signal is of type QCL D, the receiver can use the source reference RF signal to estimate the spatial reception parameters of a second reference RF signal transmitted on the same channel.
[0045] In receive beamforming, a receiver uses a receive beam to amplify an RF signal detected on a given channel. For example, the receiver may increase the gain setting of an antenna array in a particular direction and / or adjust the phase setting of the antenna array in a particular direction to amplify the RF signal received from that direction (e.g., increase its gain level). Therefore, when a receiver is described as performing beamforming in a certain direction, it means that the beam gain in that direction is high relative to the beam gain along other directions, or that the beam gain in that direction is the highest compared to the beam gain of all other receive beams available to the receiver in that direction. This results in a stronger received signal strength (e.g., reference signal received power (RSRP), reference signal received quality (RSRQ), signal-to-interference-plus-noise ratio (SINR), etc.) of the RF signal received from that direction.
[0046] The transmit and receive beams can be spatially correlated. Spatial correlation means that parameters for a second beam (e.g., transmit or receive beam) for a second reference signal can be derived based on information about a first beam (e.g., receive or transmit beam) for a first reference signal. For example, a UE can use a specific receive beam to receive a reference downlink reference signal (e.g., a synchronization signal block (SSB)) from a base station. The UE can then form a transmit beam for transmitting an uplink reference signal (e.g., a sounding reference signal (SRS)) to that base station based on the parameters of the receive beam.
[0047] It is important to note that, depending on the entity forming the "downlink" beam, the beam can be either a transmit beam or a receive beam. For example, if the base station is forming a downlink beam to transmit a reference signal to the UE, the downlink beam is a transmit beam. However, if the UE is forming a downlink beam, the downlink beam is a receive beam for receiving the downlink reference signal. Similarly, depending on the entity forming the "uplink" beam, the beam can be either a transmit beam or a receive beam. For example, if the base station is forming an uplink beam, the uplink beam is an uplink receive beam, while if the UE is forming an uplink beam, the uplink beam is an uplink transmit beam.
[0048] The electromagnetic spectrum is typically subdivided into various categories, bands, channels, etc., based on frequency / wavelength. In 5G NR, two initial operating bands have been designated as frequency ranges FR1 (410MHz to 7.125GHz) and FR2 (24.25GHz to 52.6GHz). It should be understood that although a portion of FR1 is greater than 6GHz, in various documents and articles, FR1 is often (interchangeably) referred to as the "sub-6GHz" band. A similar naming issue sometimes occurs with FR2, which is often (interchangeably) referred to as the "millimeter wave" band in documents and articles, although this differs from the designation used by the International Telecommunication Union.® Extremely high frequency (EHF) bands (30 GHz to 300 GHz) are designated as “millimeter wave” bands.
[0049] The frequencies between FR1 and FR2 are generally referred to as mid-band frequencies. Recent 5G NR studies have designated the operating bands for these mid-band frequencies as the frequency range designation FR3 (7.125 GHz to 24.25 GHz). Bands falling within FR3 can inherit FR1 and / or FR2 characteristics, thus effectively extending the features of FR1 and / or FR2 to mid-band frequencies. Additionally, higher frequency bands are currently being explored to extend 5G NR operation beyond 52.6 GHz. For example, three higher operating bands have been designated as the frequency range designations FR4a or FR4-1 (52.6 GHz to 71 GHz), FR4 (52.6 GHz to 114.25 GHz), and FR5 (114.25 GHz to 300 GHz). Each of these higher frequency bands falls within the EHF band.
[0050] In light of the foregoing, unless otherwise specifically stated, it should be understood that, as used herein, the term "below 6 GHz" and the like can broadly refer to frequencies less than 6 GHz, within FR1, or including intermediate frequency band frequencies. Furthermore, unless otherwise specifically stated, it should be understood that, as used herein, the term "millimeter wave" and the like can broadly refer to frequencies that can include intermediate frequency band frequencies, within FR2, FR4, FR4-a or FR4-1 and / or FR5, or within the EHF band.
[0051] In multi-carrier systems such as 5G, one of the carrier frequencies is referred to as the "primary carrier," "anchor carrier," "primary serving cell," or "PCell," and the remaining carrier frequencies are referred to as "secondary carriers," "secondary serving cells," or "SCell." In carrier aggregation, the anchor carrier is the carrier operating on the primary frequency (e.g., FR1) utilized by UE 104 / 182 and the cell, where UE 104 / 182 performs an initial Radio Resource Control (RRC) connection establishment procedure or initiates an RRC connection re-establishment procedure. The primary carrier carries all common and UE-specific control channels and can be a carrier on a licensed frequency (however, this is not always the case). The secondary carrier is a carrier operating on a second frequency (e.g., FR2) that can be configured and used to provide additional radio resources once an RRC connection is established between UE 104 and the anchor carrier. In some cases, the secondary carrier can be a carrier on an unlicensed frequency. Secondary carriers may contain only the necessary signaling information and signals. For example, since the primary uplink and primary downlink carriers are typically UE-specific, the UE-specific signaling information and signals may not be present in the secondary carrier. This means that different UEs 104 / 182 within a cell can have different downlink primary carriers. The same applies to the uplink primary carrier. The network can change the primary carrier of any UE 104 / 182 at any time. This is done, for example, to balance the load on different carriers. Since a "serving cell" (whether PCell or SCell) corresponds to the carrier frequency / component carrier through which a base station communicates, the terms "cell," "serving cell," "component carrier," and "carrier frequency" can be used interchangeably.
[0052] For example, still refer to Figure 1 One of the frequencies used by the macro cell base station 102 can be an anchor carrier (or "PCell"), and the other frequencies used by the macro cell base station 102 and / or the mmW base station 180 can be secondary carriers ("SCell"). Simultaneous transmission and / or reception on multiple carriers allows the UE 104 / 182 to significantly increase its data transmission and / or data reception rates. For example, compared to the data rate obtained by a single 20MHz carrier, two aggregated 20MHz carriers in a multi-carrier system would theoretically result in a doubling of the data rate (i.e., 40MHz).
[0053] The wireless communication system 100 may also include a UE 164, which can communicate with the macro cell base station 102 via communication link 120 and / or with the mmW base station 180 via mmW communication link 184. For example, the macro cell base station 102 may support PCells and one or more SCells for the UE 164, and the mmW base station 180 may support one or more SCells for the UE 164.
[0054] In some cases, UE 164 and UE 182 may be able to communicate via sidelink. A sidelink-capable UE (SL-UE) can communicate with base station 102 via communication link 120 using the Uu interface (i.e., the air interface between the UE and the base station). SL-UEs (e.g., UE 164, UE 182) can also communicate directly with each other via radio sidelink 160 using the PC5 interface (i.e., the air interface between sidelink-capable UEs). Radio sidelink (or simply "sidelink") is an adaptation of core cellular network (e.g., LTE, NR) standards that allows direct communication between two or more UEs without the need for communication through a base station. Sidelink communication can be unicast or multicast and can be used for device-to-device (D2D) media sharing, vehicle-to-vehicle (V2V) communication, vehicle-to-everything (V2X) communication (e.g., cellular V2X (cV2X) communication, enhanced V2X (eV2X) communication, emergency rescue applications, etc. One or more SL-UEs in a group of SL-UEs utilizing sidelink communication may be located within the geographical coverage area 110 of base station 102. Other SL-UEs in this group may be outside the geographical coverage area 110 of base station 102, or may be unable to receive transmissions from base station 102 for other reasons. In some cases, the groups of SL-UEs communicating via sidelink communication may utilize a one-to-many (1:M) system, where each SL-UE transmits to every other SL-UE in the group. In some cases, base station 102 facilitates the scheduling of resources used for sidelink communication. In other cases, sidelink communication is performed between the individual SL-UEs without involving base station 102.
[0055] On one hand, the sidelink 160 can operate via a wireless communication medium of interest that can be shared with other vehicles and / or infrastructure access points and other RATs for wireless communication. "Medium" can include one or more time, frequency, and / or space communication resources (e.g., covering one or more channels across one or more carriers) associated with wireless communication between one or more transmitter / receiver pairs. On another hand, the medium of interest may correspond to at least a portion of unlicensed frequency bands shared among various RATs. While different licensed frequency bands have been reserved for certain communication systems (e.g., by government entities such as the U.S. Federal Communications Commission (FCC), these systems (particularly those employing small cell access points) have recently expanded their operation to unlicensed frequency bands such as those used by wireless local area network (WLAN) technologies (most notably the IEEE 802.11x WLAN technology commonly referred to as "Wi-Fi"). Example systems of this type include various variants of CDMA, TDMA, FDMA, orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), and so on.
[0056] It should be noted that, although Figure 1 Only two of these UEs are exemplified as SL-UEs (i.e., UE 164 and UE 182), but any UE exemplified can be an SL-UE. Furthermore, although only UE 182 is described as capable of beamforming, any UE exemplified (including UE 164) can be capable of beamforming. When SL-UEs are capable of beamforming, they can beamform towards each other (i.e., towards other SL-UEs), towards other UEs (e.g., UE 104), towards base stations (e.g., base station 102, base station 180, small cell 102', access point 150), etc. Therefore, in some cases, UE 164 and UE 182 can utilize beamforming via sidelink 160.
[0057] exist Figure 1 In the example, the UE shown (for simplicity, in) Figure 1Any UE (shown as a single UE 104) can receive signal 124 from one or more Earth-orbiting spacecraft (SV) 112 (e.g., satellites). In one aspect, SV 112 may be part of a satellite positioning system that allows UE 104 to use as an independent source of location information. Satellite positioning systems typically include a system of transmitters (e.g., SV 112) positioned such that a receiver (e.g., UE 104) can determine its location on or above the Earth based at least in part on positioning signals (e.g., signal 124) received from the transmitters. Such transmitters typically transmit signals marked with a set number of repeating pseudo-random noise (PN) codes. While typically located in SV 112, transmitters may sometimes be located at ground-based control stations, base stations 102, and / or other UEs 104. UE 104 may include one or more dedicated receivers specifically designed to receive signal 124 in order to derive geographic location information from SV 112.
[0058] In a satellite positioning system, the use of signal 124 can be enhanced by various satellite-based augmentation systems (SBAS), which may be associated with or otherwise made capable of being used with one or more global and / or regional navigation satellite systems. For example, SBAS may include augmentation systems that provide integrity information, differential correction, etc., such as Wide Area Augmentation System (WAAS), European Geostationary Navigation Overlap Service (EGNOS), Multifunctional Satellite Augmentation System (MSAS), GPS-assisted geographic augmentation navigation, or GPS and geographic augmentation navigation system (GAGAN). Therefore, as used herein, a satellite positioning system may include any combination of one or more global and / or regional navigation satellites associated with such one or more satellite positioning systems.
[0059] On one hand, SV 112 may additionally or alternatively be part of one or more non-terrestrial networks (NTNs). In an NTN, SV 112 connects to an earth station (also referred to as a ground station, NTN gateway, or gateway), which in turn connects to elements in the 5G network, such as the modified base station 102 (without a ground antenna) or network nodes in a 5GC. This element, in turn, provides access to other elements in the 5G network and ultimately to entities outside the 5G network, such as internet web servers and other user equipment. Thus, as a replacement or supplement to communication signals from the ground base station 102, UE 104 can receive communication signals (e.g., signal 124) from SV 112.
[0060] The wireless communication system 100 may also include one or more UEs, such as UE 190, which are indirectly connected to one or more communication networks via one or more device-to-device (D2D) peer-to-peer (P2P) links (referred to as "side links"). Figure 1 In one example, UE 190 has a D2D P2P link 192 with one of UEs 104 connected to one of the base stations in base station 102 (e.g., UE 190 can indirectly obtain cellular connectivity through this D2D P2P link), and has a D2D P2P link 194 with a WLAN STA 152 connected to WLAN AP 150 (UE 190 can indirectly obtain WLAN-based Internet connectivity through this D2D P2P link). In one example, D2D P2P links 192 and 194 can utilize any known D2D RAT (such as LTE Direct (LTE-D), Wi-Fi Direct). ® ,Bluetooth ® (etc.) to support.
[0061] Figure 2A An example wireless network architecture 200 is illustrated. For instance, the 5GC 210 (also referred to as the Next Generation Core (NGC)) can be functionally viewed as control plane (C-plane) functions 214 (e.g., UE registration, authentication, network access, gateway selection, etc.) and user plane (U-plane) functions 212 (e.g., UE gateway functions, access to data networks, IP routing, etc.), which work together to form the core network. The user plane interface (NG-U) 213 and the control plane interface (NG-C) 215 connect the gNB 222 to the 5GC 210, specifically to user plane functions 212 and control plane functions 214, respectively. In an additional configuration, the ng-eNB 224 can also connect to the 5GC 210 via the NG-C 215 to the control plane function 214 and the NG-U 213 to the user plane function 212. Furthermore, the ng-eNB 224 can communicate directly with the gNB 222 via a backhaul connection 223. In some configurations, the next-generation RAN (NG-RAN) 220 may have one or more gNBs 222, while other configurations include one or more of both ng-eNBs 224 and gNBs 222. Either or both of the gNBs 222 or ng-eNBs 224 can communicate with one or more UEs 204 (e.g., any of the UEs described herein).
[0062] Another optional aspect may include a location server 230 that can communicate with the 5GC 210 to provide location assistance to the UE 204. The location server 230 may be implemented as multiple separate servers (e.g., physically separate servers, different software modules on a single server, different software modules distributed across multiple physical servers, etc.), or alternatively, each may correspond to a single server. The location server 230 may be configured to support one or more location services for the UE 204 that can be connected to the location server 230 via the core network, the 5GC 210, and / or via the Internet (not illustrated). Furthermore, the location server 230 may be integrated into a component of the core network, or alternatively, may be located outside the core network (e.g., a third-party server, such as an original equipment manufacturer (OEM) server or a service server).
[0063] Figure 2B Another example wireless network architecture 240.5GC 260 is illustrated (which can be used with...). Figure 2AThe 5GC 210 (corresponding to 5GC 210) can be functionally considered as a control plane function provided by the Access and Mobility Management Function (AMF) 264 and a user plane function provided by the User Plane Function (UPF) 262, which work together to form the core network (i.e., 5GC 260). The functions of AMF 264 include: registration management, connection management, reachability management, mobility management, lawful interception, transmission of session management (SM) messages between one or more UEs 204 (e.g., any of the UEs described herein) and the Session Management Function (SMF) 266, a transparent proxy service for routing SM messages, access authentication and access authorization, transmission of short message service (SMS) messages between UE 204 and the Short Message Service Function (SMSF) (not shown), and Secure Anchoring Functionality (SEAF). AMF 264 also interacts with the Authentication Server Function (AUSF) (not shown) and UE 204 and receives an intermediate key established as a result of the UE 204's authentication process. In the case of UMTS (Universal Mobile Telecommunications System) Subscriber Identity Module (USIM) authentication, AMF 264 retrieves security material from the AMF. AMF 264 also includes Security Context Management (SCM). The SCM receives a key from the SEAF and uses this key to derive an access network-specific key. AMF 264 functionality also includes location service management for regulatory services, transmission of location service messages between UE 204 and Location Management Function (LMF) 270 (which acts as location server 230), transmission of location service messages between NG-RAN 220 and LMF 270, Evolved Packet System (EPS) bearer identifier allocation for EPS interoperability, and UE 204 mobility event notification. Furthermore, AMF 264 also supports non-3GPP... ® (Third Generation Partner Program) Access network functionality.
[0064] The functions of UPF 262 include: acting as an anchor point for intra-RAT / inter-RAT mobility (where applicable), acting as an external Protocol Data Unit (PDU) session point interconnecting to a data network (not shown), providing packet routing and forwarding, packet inspection, user plane policy rule enforcement (e.g., strobing, redirection, traffic steering), lawful eavesdropping (user plane collection), traffic usage reporting, quality of service (QoS) processing for the user plane (e.g., uplink / downlink rate enforcement, reflective QoS marking in the downlink), uplink traffic verification (Service Data Flow (SDF) to QoS flow mapping), transport-level packet marking in the uplink and downlink, downlink packet buffering and downlink data notification triggering, and delivering and forwarding one or more "end markers" to the source RAN node. UPF 262 can also support the delivery of location service messages between UE 204 and location servers (such as SLP 272) on the user plane.
[0065] The functions of SMF 266 include session management, UE Internet Protocol (IP) address allocation and management, selection and control of user plane functions, service orientation configuration at UPF 262 for routing services to the correct destination, partial control of policy enforcement and QoS, and downlink data notification. The interface through which SMF 266 communicates with AMF 264 is called the N11 interface.
[0066] Another optional aspect may include an LMF 270, which can communicate with the 5GC 260 to provide location assistance to the UE 204. The LMF 270 can be implemented as multiple separate servers (e.g., physically separate servers, different software modules on a single server, different software modules distributed across multiple physical servers, etc.), or alternatively, each can correspond to a single server. The LMF 270 can be configured to support one or more location services for the UE 204, which can connect to the LMF 270 via the core network, the 5GC 260, and / or via the Internet (not illustrated). SLP 272 can support similar functions to LMF 270, but while LMF 270 can communicate with AMF 264, NG-RAN 220, and UE 204 on the control plane (e.g., using interfaces and protocols designed to transmit signaling messages rather than voice or data), SLP 272 can communicate with UE 204 and external clients (e.g., third-party server 274) on the user plane (e.g., using protocols designed to carry voice and / or data, such as Transmit Control Protocol (TCP) and / or IP).
[0067] Another optional aspect may include a third-party server 274 that can communicate with LMF 270, SLP 272, 5GC 260 (e.g., via AMF 264 and / or UPF 262), NG-RAN 220, and / or UE 204 to obtain location information (e.g., location estimation) of UE 204. Therefore, in some cases, the third-party server 274 may be referred to as a Location Services (LCS) client or an external client. The third-party server 274 may be implemented as multiple separate servers (e.g., physically separate servers, different software modules on a single server, different software modules distributed across multiple physical servers, etc.), or alternatively, each may correspond to a single server.
[0068] User plane interface 263 and control plane interface 265 connect 5GC 260, and specifically connect UPF 262 and AMF 264 to one or more gNB 222 and / or ng-eNB 224 in NG-RAN 220. The interface between gNB 222 and / or ng-eNB 224 and AMF 264 is referred to as the "N2" interface, while the interface between gNB 222 and / or ng-eNB 224 and UPF 262 is referred to as the "N3" interface. The gNB 222 and / or ng-eNB 224 of NG-RAN 220 can communicate directly with each other via backhaul connection 223, referred to as the "Xn-C" interface. One or more of gNB 222 and / or ng-eNB 224 can communicate with one or more UEs 204 via a radio interface referred to as the "Uu" interface.
[0069] The functionality of the gNB 222 is divided among the gNB Central Unit (gNB-CU) 226, one or more gNB Distributed Units (gNB-DU) 228, and one or more gNB Radio Units (gNB-RU) 229. The gNB-CU 226 is a logical node that includes base station functions other than those specifically allocated to the gNB-DU 228, including user data delivery, mobility control, radio access network sharing, location, session management, etc. More specifically, the gNB-CU 226 typically hosts the Radio Resource Control (RRC), Serving Data Adaptation Protocol (SDAP), and Packet Data Convergence Protocol (PDCP) protocols of the gNB 222. The gNB-DU 228 is a logical node that typically hosts the Radio Link Control (RLC) and Media Access Control (MAC) layers of the gNB 222. Its operation is controlled by the gNB-CU 226. One gNB-DU 228 can support one or more cells, and a cell is supported by only one gNB-DU 228. The interface 232 between gNB-CU 226 and one or more gNB-DU 228 is referred to as the "F1" interface. The physical (PHY) layer functionality of gNB 222 is typically managed by one or more independent gNB-RU 229s, which perform functions such as power amplification and signal transmission / reception. The interface between gNB-DU 228 and gNB-RU 229 is referred to as the "Fx" interface. Therefore, UE 204 communicates with gNB-CU 226 via the RRC, SDAP, and PDCP layers, with gNB-DU 228 via the RLC and MAC layers, and with gNB-RU 229 via the PHY layer.
[0070] The deployment of communication systems such as 5G NR systems can be arranged in a variety of ways using various components or parts. In a 5G NR system or network, network nodes, network entities, network mobility elements, RAN nodes, core network nodes, network elements, or network equipment (such as base stations or one or more units (or components) performing base station functions) can be implemented in aggregated or decomposed architectures. For example, base stations (such as Node B (NB), evolved NB (eNB), NR base stations, 5GNB, access points (APs), transmit / receive points (TRPs), or cells, etc.) can be implemented as aggregated base stations (also known as self-contained base stations or monolithic base stations) or decomposed base stations.
[0071] Aggregated base stations can be configured to utilize a radio protocol stack that is physically or logically integrated within a single RAN node. Decentralized base stations can be configured to utilize a protocol stack that is physically or logically distributed across two or more units, such as one or more central or centralized units (CUs), one or more distributed units (DUs), or one or more radio units (RUs). In some respects, the CU may be implemented within a RAN node, and one or more DUs may co-located with the CU, or alternatively, may be geographically or virtually distributed across one or more other RAN nodes. DUs may be implemented to communicate with one or more RUs. Each of the CUs, DUs, and RUs may also be implemented as a virtual unit, namely a virtual central unit (VCU), a virtual distributed unit (VDU), or a virtual radio unit (VRU).
[0072] Base station type operation or network design can consider the aggregation characteristics of base station functionality. For example, decomposed base stations can be used in Integrated Access Backhaul (IAB) networks, Open Radio Access Networks (O-RAN) (such as those developed by the O-RAN Alliance), and other similar networks. ® This can be used in proposed network configurations or virtualized radio access networks (vRAN, also known as cloud radio access networks (C-RAN)). Decomposition can include distributing functionality across two or more units in various physical locations, as well as virtually distributing the functionality of at least one unit, which allows for flexibility in network design. Various units in a decomposed base station or decomposed RAN architecture can be configured to communicate wirelessly with at least one other unit.
[0073] Figure 2C An example disaggregated base station architecture 250 according to various aspects of this disclosure is illustrated. The disaggregated base station architecture 250 may include one or more central units (CUs) 280 (e.g., gNB-CU 226) that can communicate directly with the core network 267 (e.g., 5GC 210, 5GC 260) via a backhaul link, or indirectly with the core network 267 via one or more disaggregated base station units (such as a near real-time (near-RT) RAN intelligent controller (RIC) 259 via an E2 link or a non-real-time (non-RT) RIC 257 associated with a Service Management and Orchestration (SMO) framework 255, or both). CUs 280 may communicate with one or more duplex units (DUs) 285 (e.g., gNB-DU 228) via a corresponding midhaul link (e.g., an F1 interface). DUs 285 may communicate with one or more radio units (RUs) 287 (e.g., gNB-RU 229) via a corresponding fronthaul link. RU 287 can communicate with the corresponding UE 204 via one or more radio frequency (RF) access links. In some implementations, UE 204 can be served by multiple RU 287s simultaneously.
[0074] Each of the units (i.e., CU 280, DU 285, RU 287, and near-RT RIC 259, non-RT RIC 257, and SMO frame 255) may include or be coupled to one or more interfaces configured to receive or transmit signals, data, or information (collectively, signals) via wired or wireless transmission media. Each unit in the unit, or an associated processor or controller providing instructions to the communication interfaces of these units, may be configured to communicate with one or more other units via transmission media. For example, these units may include wired interfaces configured to receive signals or transmit signals to one or more other units via wired transmission media. Additionally, these units may include wireless interfaces that may include receivers, transmitters, or transceivers (such as RF transceivers) configured to receive signals or transmit signals to one or more other units, or both, via wireless transmission media.
[0075] In some aspects, the CU 280 can host one or more higher-level control functions. Such control functions may include RRC, PDCP, Service Data Adaptation Protocol (SDAP), etc. Each control function can be implemented using an interface configured to signal to other control functions hosted by the CU 280. The CU 280 can be configured to handle user plane functionality (i.e., Central Unit-User Plane (CU-UP)), control plane functionality (i.e., Central Unit-Control Plane (CU-CP)), or a combination thereof. In some implementations, the CU 280 can be logically split into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP units can communicate bidirectionally with the CU-CP units via an interface such as an E1 interface. The CU 280 can be implemented to communicate with the DU 285 for network control and signaling as needed.
[0076] DU 285 may correspond to a logic unit that includes one or more base station functions for controlling the operation of one or more RU 287s. In some aspects, DU 285 may be at least partially based on functional partitioning (such as that provided by the 3rd Generation Partnership Project (3GPP)). ®The DU285 is functionally partitioned to host one or more of the RLC layer, MAC layer, and one or more high-PHY layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation, and demodulation). In some respects, the DU285 can further host one or more low-PHY layers. Each layer (or module) can be implemented using an interface configured to communicate signals with other layers (and modules) hosted by the DU285 or with control functions hosted by the CU280.
[0077] Lower-layer functionality can be implemented by one or more RU 287s. In some deployments, an RU287 controlled by a DU 285 may correspond to a logical node that hosts RF processing functions or low-PHY layer functions (such as performing Fast Fourier Transform (FFT), Inverse FFT (iFFT), digital beamforming, Physical Random Access Channel (PRACH) extraction and filtering, or both, based at least in part on functional decomposition (such as lower-layer functional decomposition). In this architecture, the RU 287 can be implemented to handle over-the-air (OTA) communications with one or more UE 204s. In some specific implementations, the real-time and non-real-time aspects of control plane and user plane communications with the RU 287 can be controlled by the corresponding DU 285. In some scenarios, this configuration enables the implementation of the DU 285 and CU 280 in cloud-based RAN architectures (such as vRAN architectures).
[0078] SMO framework 255 can be configured to support RAN deployment and provisioning of both non-virtualized and virtualized network elements. For non-virtualized network elements, SMO framework 255 can be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which can be managed via operation and maintenance interfaces such as the O1 interface. For virtualized network elements, SMO framework 255 can be configured to interact with cloud computing platforms such as Open Cloud (O-Cloud) 269 to perform network element lifecycle management (such as instantiating virtualized network elements) via cloud computing platform interfaces such as the O2 interface. Such virtualized network elements may include, but are not limited to, CU 280, DU 285, RU 287, and near-RT RIC 259. In some implementations, SMO framework 255 can communicate with the hardware aspects of the 4G RAN (such as Open eNB (O-eNB) 261) via the O1 interface. Additionally, in some implementations, SMO framework 255 can communicate directly with one or more RU 287s via the O1 interface. SMO framework 255 may also include a non-RT RIC 257 configured to support the functionality of SMO framework 255.
[0079] The non-RT RIC 257 can be configured to include logical functions enabling non-real-time control and optimization of RAN elements and resources, including artificial intelligence / machine learning (AI / ML) workflows for model training and updates, or policy-based guidance for applications / features in the near-RT RIC 259. The non-RT RIC 257 can be coupled to or communicate with the near-RT RIC 259, such as via an A1 interface. The near-RT RIC 259 can be configured to include logical functions enabling near real-time control and optimization of RAN elements and resources via an interface, such as an E2 interface, connecting one or more CU 280s, one or more DU 285s, or both, and O-eNBs to the near-RT RIC 259.
[0080] In some implementations, to generate AI / ML models to be deployed in the near-RT RIC 259, the non-RT RIC 257 can receive parameters or external enrichment information from an external server. This information can be utilized by the near-RT RIC 259 and can be received from non-network data sources or network functions at the SMO framework 255 or the non-RT RIC 257. In some examples, the non-RT RIC 257 or the near-RT RIC 259 can be configured to tune RAN behavior or performance. For example, the non-RT RIC 257 can monitor long-term trends and patterns in performance and use AI / ML models to perform corrective actions via the SMO framework 255 (such as reconfiguration via O1) or by creating RAN management policies (such as A1 policies).
[0081] Figure 3A , Figure 3B and Figure 3C Several example components (represented by corresponding boxes) are illustrated, which can be incorporated into UE 302 (which may correspond to any UE described herein), base station 304 (which may correspond to any base station described herein), and network entity 306 (which may correspond to or embody any network function described herein, including location server 230 and LMF 270, or alternatively may be independent of UE 302). Figure 2A and Figure 2BThe NG-RAN 220 and / or 5GC 210 / 260 infrastructures depicted herein (such as dedicated networks) are used to support the operations described herein. It should be understood that these components can be implemented in different specific implementations in different types of devices (e.g., in ASICs, in System-on-Chip (SoCs), etc.). The illustrated components can also be incorporated into other devices in the communication system. For example, other devices in the system may include components similar to those described as providing similar functionality. Furthermore, a given device may contain one or more of these components. For example, a device may include multiple transceiver components that enable the device to operate on multiple carriers and / or communicate via different technologies.
[0082] UE 302 and base station 304 each include one or more Wireless Wide Area Network (WWAN) transceivers 310 and 350, which provide components (e.g., components for transmitting, components for receiving, components for measuring, components for tuning, components for blocking transmission, etc.) for communication via one or more wireless communication networks (not shown), such as NR networks, LTE networks, GSM networks, etc. WWAN transceivers 310 and 350 may each be connected to one or more antennas 316 and 356 for communication with other network nodes (such as other UEs, access points, base stations (e.g., eNB, gNB), etc.) via at least one designated RAT (e.g., NR, LTE, GSM, etc.) through a wireless communication medium of interest (e.g., a time / frequency resource set in a specific spectrum). WWAN transceivers 310 and 350 can be configured in different ways to transmit and encode signals 318 and 358 (e.g., messages, indications, information, etc.) according to a specified RAT, and conversely, to receive and decode signals 318 and 358 (e.g., messages, indications, information, pilots, etc.). Specifically, WWAN transceivers 310 and 350 each include: one or more transmitters 314 and 354 for transmitting and encoding signals 318 and 358, respectively; and one or more receivers 312 and 352 for receiving and decoding signals 318 and 358, respectively.
[0083] In at least some cases, UE 302 and base station 304 each further include one or more short-range radio transceivers 320 and 360, respectively. The short-range radio transceivers 320 and 360 can be connected to one or more antennas 326 and 366, respectively, and provide the capability to communicate over a wireless communication medium of interest via at least one designated RAT (e.g., Wi-Fi, LTE Direct, Bluetooth). ® ZIGBEE ® Z-WAVE ®Components (e.g., components for transmitting, components for receiving, components for measuring, components for tuning, components for blocking transmission, etc.) that enable communication between PC5, Dedicated Short Range Communication (DSRC), Wireless Access for Vehicle Environments (WAVE), Near Field Communication (NFC), Ultra Wideband (UWB), etc., and other network nodes (such as other UEs, access points, base stations, etc.). Short-range transceivers 320 and 360 can be configured in different ways to transmit and encode signals 328 and 368 (e.g., messages, indications, information, etc.) respectively according to a specified RAT, and conversely, to receive and decode signals 328 and 368 (e.g., messages, indications, information, pilots, etc.) respectively. Specifically, short-range wireless transceivers 320 and 360 each include: one or more transmitters 324 and 364 for transmitting and encoding signals 328 and 368, respectively; and one or more receivers 322 and 362 for receiving and decoding signals 328 and 368, respectively. As a specific example, short-range wireless transceivers 320 and 360 may be Wi-Fi transceivers, Bluetooth transceivers, etc. ® Transceiver, Zigbee ® and / or Z-WAVE ® Transceivers, NFC transceivers, UWB transceivers, or vehicle-to-vehicle (V2V) and / or vehicle-to-everything (V2X) transceivers.
[0084] In at least some cases, UE 302 and base station 304 also include satellite signal receivers 330 and 370. Satellite signal receivers 330 and 370 may be connected to one or more antennas 336 and 376, respectively, and may provide components for receiving and / or measuring satellite positioning / communication signals 338 and 378, respectively. Where satellite signal receivers 330 and 370 are satellite positioning system receivers, satellite positioning / communication signals 338 and 378 may be Global Positioning System (GPS) signals, Global Navigation Satellite System (GLONASS) signals, etc. ® The signals can include Galileo signals, BeiDou signals, Indian Regional Navigation Satellite System (NAVIC), Quasi-Zenith Satellite System (QZSS), etc. When satellite signal receivers 330 and 370 are non-terrestrial network (NTN) receivers, satellite positioning / communication signals 338 and 378 can be communication signals originating from a 5G network (e.g., carrying control and / or user data). Satellite signal receivers 330 and 370 can include any suitable hardware and / or software for receiving and processing satellite positioning / communication signals 338 and 378, respectively. Satellite signal receivers 330 and 370 may request information and operations from other systems as needed, and in at least some cases, use measurements obtained by any suitable satellite positioning system algorithm to perform calculations to determine the locations of UE 302 and base station 304, respectively.
[0085] Base station 304 and network entity 306 each include one or more network transceivers 380 and 390, which provide components (e.g., transmitting components, receiving components, etc.) for communicating with other network entities (e.g., other base stations 304, other network entities 306). For example, base station 304 may use one or more network transceivers 380 to communicate with other base stations 304 or network entities 306 via one or more wired or wireless backhaul links. Similarly, network entity 306 may use one or more network transceivers 390 to communicate with one or more base stations 304 via one or more wired or wireless backhaul links, or to communicate with other network entities 306 via one or more wired or wireless core network interfaces.
[0086] Transceivers can be configured to communicate via wired or wireless links. A transceiver (whether wired or wireless) includes transmitter circuitry (e.g., transmitters 314, 324, 354, 364) and receiver circuitry (e.g., receivers 312, 322, 352, 362). In some embodiments, the transceiver may be an integrated device (e.g., implementing transmitter and receiver circuitry in a single device), in some embodiments it may include separate transmitter and receiver circuitry, or in other embodiments it may be implemented in a different manner. The transmitter and receiver circuitry of a wired transceiver (e.g., network transceivers 380 and 390 in some embodiments) may be coupled to one or more wired network interface ports. Wireless transmitter circuitry (e.g., transmitters 314, 324, 354, 364) may include or be coupled to multiple antennas (e.g., antennas 316, 326, 356, 366), such as an antenna array, which allows the corresponding device (e.g., UE 302, base station 304) to perform transmit beamforming, as described herein. Similarly, wireless receiver circuitry (e.g., receivers 312, 322, 352, 362) may include or be coupled to multiple antennas (e.g., antennas 316, 326, 356, 366), such as an antenna array, which allows the corresponding device (e.g., UE 302, base station 304) to perform receive beamforming, as described herein. In one aspect, the transmitter and receiver circuitry may share the same multiple antennas (e.g., antennas 316, 326, 356, 366), such that the corresponding device may perform only receive or only transmit at a given time, rather than both receive and transmit simultaneously. Wireless transceivers (e.g., WWAN transceivers 310 and 350, short-range wireless transceivers 320 and 360) may also include network listening modules (NLMs) for performing various measurements.
[0087] As used herein, various wireless transceivers (e.g., transceivers 310, 320, 350, and 360 in some specific embodiments, and network transceivers 380 and 390) and wired transceivers (e.g., network transceivers 380 and 390 in some specific embodiments) are generally referred to as "transceiver," "at least one transceiver," or "one or more transceivers." Therefore, whether a particular transceiver is a wired or wireless transceiver can be inferred from the type of communication performed. For example, backhaul communication between network devices or servers typically involves signaling via a wired transceiver, while wireless communication between a UE (e.g., UE 302) and a base station (e.g., base station 304) will typically involve signaling via a wireless transceiver.
[0088] UE 302, base station 304, and network entity 306 also include other components that can be used in conjunction with the operation disclosed herein. UE 302, base station 304, and network entity 306 each include one or more processors 332, 384, and 394 for providing functionality related to, for example, wireless communication, and for providing other processing functionality. Thus, processors 332, 384, and 394 may provide components for processing, such as components for determining, components for calculating, components for receiving, components for transmitting, components for indicating, etc. In one aspect, processors 332, 384, and 394 may include, for example, one or more general-purpose processors, multi-core processors, central processing units (CPUs), ASICs, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), other programmable logic devices or processing circuits, or various combinations thereof.
[0089] UE 302, base station 304, and network entity 306 each include memory circuitry implementing memories 340, 386, and 396 (e.g., each including a memory device) for maintaining information (e.g., information indicating reserved resources, thresholds, parameters, etc.). Therefore, memories 340, 386, and 396 can provide components for storage, retrieval, maintenance, etc. In some cases, UE 302, base station 304, and network entity 306 may each include C-SLAM components 342, 388, and 398. C-SLAM components 342, 388, and 398 may be hardware circuitry that is part of or coupled to processors 332, 384, and 394, respectively, which, when executed, enable UE 302, base station 304, and network entity 306 to perform the functionality described herein. In other respects, C-SLAM components 342, 388, and 398 may be external to processors 332, 384, and 394 (e.g., as part of a modem processing system, integrated with another processing system, etc.). Alternatively, C-SLAM components 342, 388, and 398 may be memory modules stored in memories 340, 386, and 396, respectively, which enable UE 302, base station 304, and network entity 306 to perform the functionality described herein when executed by processors 332, 384, and 394 (or modem processing system, another processing system, etc.). Figure 3A Possible locations for C-SLAM component 342 are illustrated. This C-SLAM component may be part of, for example, one or more WWAN transceivers 310, memory 340, one or more processors 332, or any combination thereof, or it may be a standalone component. Figure 3B Possible locations for C-SLAM component 388 are illustrated. This C-SLAM component may be part of, for example, one or more WWAN transceivers 350, memory 386, one or more processors 384, or any combination thereof, or it may be a standalone component. Figure 3C Possible locations for C-SLAM component 398 are illustrated. This C-SLAM component may be, for example, part of one or more network transceivers 390, memory 396, one or more processors 394, or any combination thereof, or may be a standalone component.
[0090] UE 302 may include one or more sensors 344 coupled to one or more processors 332 to provide components for sensing or detecting motion and / or orientation information independent of motion data derived from signals received by one or more WWAN transceivers 310, one or more short-range wireless transceivers 320, and / or satellite signal receivers 330. By way of example, sensor 344 may include accelerometers (e.g., microelectromechanical systems (MEMS) devices), gyroscopes, geomagnetic sensors (e.g., compasses), altimeters (e.g., barometric altimeters), and / or any other type of motion detection sensor. Furthermore, sensor 344 may include multiple different types of devices and combine their outputs to provide motion information. For example, sensor 344 may use a combination of multi-axis accelerometers and orientation sensors to provide the ability to calculate positioning in two-dimensional (2D) and / or three-dimensional (3D) coordinate systems.
[0091] In addition, UE 302 includes a user interface 346 that provides components for providing instructions to a user (e.g., audible and / or visual instructions) and / or for receiving user input (e.g., when the user actuates a sensing device such as a keypad, touchscreen, microphone, etc.). Although not shown, base station 304 and network entity 306 may also include user interfaces.
[0092] Referring more specifically to one or more processors 384, in the downlink, IP packets from network entity 306 can be provided to processor 384. One or more processors 384 can implement functionality for the RRC layer, Packet Data Convergence Protocol (PDCP) layer, Radio Link Control (RLC) layer, and Media Access Control (MAC) layer. One or more processors 384 may provide: RRC layer functionality associated with broadcasting system information (e.g., Master Information Block (MIB), System Information Block (SIB)), RRC connection control (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), inter-RAT mobility, and measurement configuration for UE measurement reporting; PDCP layer functionality associated with header compression / decompression, security (encryption, decryption, integrity protection, integrity verification), and handover support functions; RLC layer functionality associated with the delivery of upper-layer PDUs, error correction via Automatic Repeat Request (ARQ), concatenation, segmentation, and reassembly of RLC Service Data Units (SDUs), resegmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, scheduling information reporting, error correction, priority handling, and logical channel priority ordering.
[0093] Transmitter 354 and receiver 352 implement Layer 1 (L1) functionality associated with various signal processing functions. Layer 1, including the physical (PHY) layer, may include: error detection on the transport channel, forward error correction (FEC) decoding / decoding of the transport channel, interleaving, rate matching, mapping to the physical channel, modulation / demodulation of the physical channel, and MIMO antenna processing. Transmitter 354 processes the mapping to the signal constellation based on various modulation schemes (e.g., Binary Phase Shift Keying (BPSK), Quadrature Phase Shift Keying (QPSK), M-Phase Shift Keying (M-PSK), M-QAM). The decoded and modulated symbols can then be divided into parallel streams. Each stream can then be mapped to Orthogonal Frequency Division Multiplexing (OFDM) subcarriers, multiplexed with a reference signal (e.g., pilot) in the time and / or frequency domains, and then combined using an Inverse Fast Fourier Transform (IFFT) to produce a physical channel carrying a stream of time-domain OFDM symbols. The OFDM symbol stream is spatially pre-decoded to generate multiple spatial streams. Channel estimates from the channel estimator can be used to determine the decoding and modulation scheme, as well as for spatial processing. Channel estimates can be derived based on reference signals transmitted by UE302 and / or channel condition feedback. Each spatial stream can then be provided to one or more different antennas 356. The transmitter 354 can use the corresponding spatial stream to modulate an RF carrier for transmission.
[0094] At UE 302, receiver 312 receives signals via its corresponding antenna 316. Receiver 312 recovers the information modulated onto the RF carrier and provides this information to one or more processors 332. Transmitter 314 and receiver 312 implement Layer 1 functionality associated with various signal processing functions. Receiver 312 can perform spatial processing on the information to recover any spatial streams destined for UE 302. If multiple spatial streams are destined for UE 302, they can be combined by receiver 312 into a single OFDM symbol stream. Receiver 312 then uses a Fast Fourier Transform (FFT) to transform the OFDM symbol stream from the time domain to the frequency domain. The frequency domain signal includes a separate OFDM symbol stream for each subcarrier of the OFDM signal. Symbols on each subcarrier, along with a reference signal, are recovered and demodulated by determining the signal constellation points most likely to be transmitted by base station 304. These soft decisions can be based on channel estimates calculated by a channel estimator. The soft decisions are then decoded and deinterleaved to recover the data and control signals originally transmitted by base station 304 on the physical channel. Then, data and control signals are provided to one or more processors 332, which implement layer 3 (L3) and layer 2 (L2) functionality.
[0095] In the downlink, one or more processors 332 provide demultiplexing, packet reassembly, decryption, header decompression, and control signal processing between the transport and logical channels to recover IP packets from the core network. One or more processors 332 are also responsible for error detection.
[0096] Similar to the functionality described in conjunction with downlink transmissions performed by base station 304, one or more processors 332 provide: RRC layer functionality associated with system information (e.g., MIB, SIB) acquisition, RRC connectivity, and measurement reporting; PDCP layer functionality associated with header compression / decompression and security (encryption, decryption, integrity protection, integrity verification); RLC layer functionality associated with the delivery of upper-layer PDUs, error correction via ARQ, concatenation, segmentation, and reassembly of RLC SDUs, resegmentation of RLC data PDUs, and reordering of RLC data PDUs; and MAC layer functionality associated with mapping between logical channels and transport channels, multiplexing of MAC SDUs onto transport blocks (TBs), demultiplexing of MAC SDUs from TBs, scheduling information reporting, error correction via Hybrid Automatic Repeat Request (HARQ), priority handling, and logical channel priority ordering.
[0097] The channel estimate derived by the channel estimator from the reference signal or feedback transmitted by the base station 304 can be used by the transmitter 314 to select an appropriate decoding and modulation scheme and facilitate spatial processing. The spatial stream generated by the transmitter 314 can be provided to different antennas 316. The transmitter 314 can use the corresponding spatial stream to modulate the RF carrier for transmission.
[0098] Uplink transmissions are processed at base station 304 in a manner similar to that described in conjunction with the receiver function at UE 302. Receiver 352 receives signals via its corresponding antenna 356. Receiver 352 recovers the information modulated onto the RF carrier and provides this information to one or more processors 384.
[0099] In the uplink, one or more processors 384 provide demultiplexing, packet reassembly, decryption, header decompression, and control signal processing between the transport channel and the logical channel to recover IP packets from UE 302. IP packets from one or more processors 384 can be provided to the core network. One or more processors 384 are also responsible for error detection.
[0100] For convenience, UE 302, base station 304 and / or network entity 306 are in Figure 3A , Figure 3B and Figure 3CThe example shown herein includes various components that can be configured according to the various examples described herein. However, it should be understood that the illustrated components may have different functionalities in different designs. In particular, Figures 3A to 3C Various components are optional in alternative configurations, and various aspects include configurations that can vary due to design choices, cost, equipment usage, or other considerations. For example, in Figure 3A In certain cases, specific implementations of UE 302 may omit WWAN transceiver 310 (e.g., wearable devices, tablets, personal computers (PCs), or laptops may have Wi-Fi and / or Bluetooth). ® (e.g., cellular only), or the short-range wireless transceiver 320 can be omitted (e.g., cellular only), or the satellite signal receiver 330 can be omitted, or the sensor 344 can be omitted, etc. For example, in Figure 3B In certain cases, specific implementations of base station 304 may omit WWAN transceiver 350 (e.g., a Wi-Fi "hotspot" access point without cellular capabilities), or short-range wireless transceiver 360 (e.g., cellular only), or satellite signal receiver 370, etc. For the sake of brevity, examples of various alternative configurations are not provided herein, but will be readily understood by those skilled in the art.
[0101] Various components of UE 302, base station 304, and network entity 306 can be communicatively coupled to each other via data buses 334, 382, and 392, respectively. In one aspect, data buses 334, 382, and 392 can form or be part of the communication interfaces of UE 302, base station 304, and network entity 306, respectively. For example, in cases where different logical entities are embodied in the same device (e.g., gNB and location server functionality integrated into the same base station 304), data buses 334, 382, and 392 can provide communication between these different logical entities.
[0102] Figure 3A , Figure 3B and Figure 3C The components can be implemented in various ways. In some specific implementations, Figure 3A , Figure 3B and Figure 3CThe components can be implemented in one or more circuits, such as, for example, one or more processors and / or one or more ASICs (which may include one or more processors). Here, each circuit may use and / or combine at least one memory component for storing information or executable code used by the circuit to provide that functionality. For example, some or all of the functionalities represented by blocks 310 to 346 may be implemented by the processor and memory components of UE 302 (e.g., by executing appropriate code and / or by appropriate configuration of the processor components). Similarly, some or all of the functionalities represented by blocks 350 to 388 may be implemented by the processor and memory components of base station 304 (e.g., by executing appropriate code and / or by appropriate configuration of the processor components). Moreover, some or all of the functionalities represented by blocks 390 to 398 may be implemented by the processor and memory components of network entity 306 (e.g., by executing appropriate code and / or by appropriate configuration of the processor components). For simplicity, various operations, actions, and / or functions are described herein as being performed "by the UE," "by the base station," "by the network entity," etc. However, as will be understood, such operations, actions and / or functions can actually be performed by specific components or combinations of components of the UE 302, base station 304, network entity 306, etc., such as processors 332, 384, 394, transceivers 310, 320, 350 and 360, memory 340, 386 and 396, C-SLAM components 342, 388 and 398, etc.
[0103] In some designs, network entity 306 may be implemented as a core network component. In other designs, network entity 306 may operate differently from the network operator or cellular network infrastructure (e.g., NG RAN 220 and / or 5GC 210 / 260). For example, network entity 306 may be a component of a private network that can be configured to communicate with UE 302 via base station 304 or independently of base station 304 (e.g., via a non-cellular communication link such as Wi-Fi).
[0104] Cooperative Simultaneous Localization and Mapping (C-SLAM) is an emerging field built upon the success of traditional Simultaneous Localization and Mapping (SLAM). In traditional SLAM (i.e., single-agent SLAM), an "agent" (e.g., a robot or other mobile autonomous or semi-autonomous device) aims to simultaneously track its position within that environment while building an accurate map of its surroundings using environmental landmarks. In C-SLAM (also known as multi-agent SLAM), multiple agents collaboratively perform SLAM by exchanging / processing local observable information, aiming to accurately locate themselves within a shared map of the environment while building a common map of the environment.
[0105] C-SLAM systems are envisioned for use in a variety of applications, including industrial monitoring and transportation, public surveillance, autonomous driving, marine exploration, mission-critical applications, and more. Figure 4 Figure 400 illustrates examples of several intelligent agents navigating around various environmental landmarks according to various aspects of this disclosure.
[0106] Due to its collaborative and distributed nature, C-SLAM presents unique challenges in areas such as communication, resource management, computational cost, and robustness. Therefore, this disclosure provides techniques for addressing some of these key challenges in C-SLAM by introducing a new functional block into the logical architecture of a C-SLAM system. This new block receives and processes visual information provided by surveillance camera deployments in industrial monitoring, thereby enabling over-the-top (OTT) management of the C-SLAM system. This functional block is referred to herein as the Visual Over-the-Top (vOTT) Monitoring Block.
[0107] Figure 5 This is a diagram 500 illustrating an example C-SLAM system according to various aspects of this disclosure. For example... Figure 5 As shown, the C-SLAM system has two main functional blocks: a front-end block and a back-end block. The front-end block is responsible for collecting sensor reports, formatting data, extracting relevant features / information, and loop closure. The back-end block fuses (formatted / preprocessed) sensor data and performs inference (specifically, localization and map building).
[0108] Other key functional components include a control center that receives outputs from the front-end and back-end blocks and determines control and radio resource management (RRM) primitives for the actuator / agent. The communication network supports the flow of information between the agent / actuator and C-SLAM functions. The RRM blocks in the communication network maintain connections to the C-SLAM system for agent scheduling purposes, and so on.
[0109] A C-SLAM system can be implemented in a centralized or distributed manner. In this disclosure, it is considered a logical entity and can be centralized or distributed.
[0110] Various challenges exist in C-SLAM systems due to bottlenecks caused by information requirements at both the front-end and back-end. The front-end block performs the following key functions: (1) landmark estimation, (2) ranging, (3) intra-agent loop closure, and (4) inter-agent loop closure. Landmark estimation uses sensing and / or visual sensors to identify, match, and locate environmental landmarks (e.g., key points, features). This function enables each agent to construct a local agent-specific map of the environment. A key challenge is the identification of sufficiently descriptive landmarks at certain indoor locations. For example, insufficient detection can lead to significant drift in location estimation, resulting in agents getting stuck or lost (i.e., agent failure).
[0111] Reference ranging methods use radio frequency (RF), inertial measurement units (IMUs), vision, and / or sensing sensors to provide an initial estimate of an agent's pose within its local map. In C-SLAM systems, agents can also use ranging methods with each other by exchanging local RF / vision / IMU / sensing information (e.g., via a D2D link). A key challenge is the drift (i.e., error accumulation) associated with tracking algorithms based on RF / IMU inputs.
[0112] Referring to the closed loop within the agent's body, this function determines whether the agent has already visited a location (e.g., through location recognition) and corrects for any ranging drift based on that determination. A key challenge is perceptual aliasing, which can occur when the agent visits a location that is confused with previously visited locations.
[0113] Referring to inter-agent loop closure, direct inter-agent loop closure determines whether two or more agents have directly met through direct sensing or by exchanging map information with each other. Indirect inter-agent loop closure determines whether two or more agents have visited the same location at different times. Indirect inter-agent loop closure indicates overlap between individual agent maps. These overlaps are important for building a consistent global map in a C-SLAM system. A key challenge is the significant communication, processing, and memory resource requirements, as entire local agent maps need to be exchanged (between agents and / or between agents and the server) to enable map overlap detection. This can lead to severe bottlenecks.
[0114] The backend block performs the inference function, namely, state estimation. State estimation uses initialization provided by the frontend block to jointly estimate the local map and agent pose. A key challenge in C-SLAM is the requirement for distributed consensus. Therefore, additional attention needs to be paid to inter-agent link failures / damage.
[0115] This disclosure focuses on industrial monitoring applications with access to visual channel state information (vCSI). Figure 6This is a diagram 600 illustrating an example industrial monitoring scenario according to various aspects of this disclosure. For example... Figure 6 As shown, in an industrial monitoring scenario, in addition to the system / network of intelligent agents 602 (e.g., robots), the system also includes a static / fixed camera infrastructure / deployment of a collection of monitoring cameras 606, which partially or completely observe the intelligent agents 602 moving around in the environment. It should be noted that the terms "fixed" and "static" imply the assumption that the parameters of the cameras 606 (such as internal parameters (e.g., camera settings) and external parameters (e.g., camera position)) are known. This camera deployment is referred to herein as a visual over-the-top (vOTT) monitoring system.
[0116] The camera system generates visual information, or vCSI, which is represented as an image stream. vCSI from the vOTT monitoring system can be used to address the key C-SLAM challenges discussed above. A dedicated function block, known as the vOTT monitoring block, can be used for this purpose. This function block performs several key functions designed to solve various C-SLAM challenges. These functions include vCSI-assisted loop monitoring management, vCSI-assisted landmark estimation, vCSI-assisted agent pose estimation, and vCSI-assisted agent link monitoring.
[0117] Referring to vCSI-assisted loop detection and management, loop closure in C-SLAM (as discussed above) requires detecting overlaps in agent maps, which then need to be propagated between agents or between agents and the server, resulting in significant communication, memory, and processing overhead. Using vCSI information can significantly reduce this overhead because it can be used to instruct agents to exchange or transmit relevant (e.g., partial) map information only when necessary (e.g., upon the occurrence of a specific triggering event).
[0118] Figure 7Various example triggering events that may lead to the exchange of map information between agents according to various aspects of this disclosure are illustrated. Figure 710 illustrates an event in which two or more agents (e.g., agent 602) meet or are about to meet. This event can be triggered when a proximity metric falls below a threshold. The proximity metric (e.g., distance between two agents, trajectories of two agents, and / or velocities of two agents) can be determined at least in part based on RF measurements (e.g., object detection measurements) obtained by one or both agents. Figure 730 illustrates an event in which two or more agents (e.g., agent 602) visit the same location at different times. This event can be triggered when a new agent visits a location previously visited by one or more other agents. The location of the agent can be determined using wireless positioning techniques based on the transmission and / or reception of RF measurements made by the agent. Figure 750 illustrates an event in which an agent (e.g., agent 602) visits a previously visited location. This event can be triggered when an agent visits a location it has previously visited (resolved perceptual aliasing). Similarly, the agent's location can be determined using wireless positioning technology based on the transmission and / or reception of RF measurements made by the agent.
[0119] Referring now to a pseudo-format for information storage that can be used by vCSI-assisted loop monitoring and management functions, the vOTT monitoring block uses vCSI (imagery) to identify locations and agents. Each location is described by a list of indicators, which can be landmark locations or even simply coordinates on a grid. Each agent is described by its location (i.e., position) at a given timestamp. The vOTT monitoring block calculates and maintains a proximity metric between agent pairs (e.g., the relative distance between agent pairs). It also maintains a list of agents that have visited a given location along with the timestamp of their visit. If frequent changes in the environment are expected, the list of agents visiting a given location can also be dynamic, and agents can be removed after a certain amount of time (because the location may have changed, and if an agent revisits the location, an updated map will be needed). The vOTT monitoring block also maintains application-specific thresholds for relevant triggering events (e.g., such as...). Figure 7 (as illustrated); if these thresholds are exceeded, an event is triggered.
[0120] Referring now to vCSI-assisted agent pose estimation, ranging refers to the estimation of an agent's pose (i.e., position and orientation), which is later used to initialize the front-end state estimation algorithm. This is typically accomplished using the fusion of measurements from different sensors, such as RF sensing (based on RF measurements) and IMU information. vCSI can also be used to estimate the pose of a visible agent. A combination of visual localization (or hybrid visual / RF localization) and modern techniques, including readily available machine learning, can be used to determine the agent's orientation from images (or videos). This can be triggered when the local agent ranging function fails to exceed a confidence threshold (e.g., due to sensor malfunction).
[0121] Referring now to vCSI used for landmark estimation, vCSI can help an agent navigate its immediate environment by guiding it to locations with highly descriptive landmarks. This prevents the agent from getting lost or stuck in non-descriptive locations, which can be common indoors. Various triggering events exist for this functionality. For example, a vOTT block can determine from an image that feature-rich landmarks (e.g., landmarks with high-contrast colors, landmarks with easily identifiable shapes, etc.) are nearby (e.g., within a threshold distance of the agent's line of sight) but outside the agent's line of sight. As another example, an agent might get stuck in a low-light location (e.g., a long, dimly lit passageway). In both examples, the vOTT block can use the image to determine that the agent is lost and compute trajectory indications to help the agent move.
[0122] Referring now to vCSI used for state estimation and inference (i.e., agent link monitoring), in C-SLAM, the inference of agent trajectories and maps operates on a distributed connectivity graph. This means consensus is required (i.e., trajectory estimates need to be defined and the global map needs to be consistent). Reaching consensus can be difficult due to sporadic link failures between agents or even security attacks that break some links in the graph (e.g., interference). Given the limited communication range of agents, vCSI can be used to help identify potentially vulnerable links. As an example triggering event, using images, through visual inspection of occlusion between agents, links between agents (e.g., D2D links) can be determined to be either line-of-sight (LOS) or non-line-of-sight (NLOS). This information can then be used to decide whether to command agents to use certain edges (i.e., links) in the graph.
[0123] Now consider the nodes of a vOTT monitoring system. A server (e.g., a location server such as the LMF 270 or a third-party server) processes vCSI (i.e., images) and necessary / requested information from agents, extracts / stores vCSI information, and forwards commands / instructions / instructions to various functional blocks of the C-SLAM system. Most of the functions involved in generating vCSI rely on visual localization. Therefore, the server also runs a localization engine (which can be hybrid, fusing both visual and RF-based information). The server also maintains a local database / memory unit / module to store information related to location and agents. In some cases, the server may also maintain a connection to an external cloud that runs advanced machine learning algorithms for various visual processing tasks, including, for example, object detection and similar functions.
[0124] The cameras and camera infrastructure of a vOTT monitoring system can be a network of surveillance cameras deployed to monitor the environment, for example, Figure 6 As illustrated. However, it should be noted that, although Figure 6 The example shows four security cameras, but a vOTT monitoring system can have more or fewer cameras. Furthermore, the cameras can be positioned differently. The cameras maintain connections to a server that can access specific images for further processing as needed.
[0125] The various types of nodes in the vOTT monitoring system also include UE / RF devices attached to the agents. Each agent is assumed to have a UE or other type of RF device (e.g., Wi-Fi communication device, Bluetooth). ® Communication equipment, UWB communication equipment, and other types of sensors. The UE / RF device communicates with the server; thus, depending on the specific implementation, the server can directly request additional information (such as RF measurements) from the agent, or the server can directly disseminate control commands and resource allocations to the agent.
[0126] There are various initiation and execution aspects. Each vOTT monitoring function block is triggered by an event or a set of events. On the other hand, each vOTT monitoring function can be initiated by a server or a UE / RF device (i.e., an agent). Below are various server-side initiation examples. As a first example, in an indirect agent-to-agent loop closure, the server maintains a list of locations and agents accessing them. Upon detecting that an agent has accessed one of its previously visited locations, the server triggers a process in which it commands the involved agents to share relevant portions of their local maps and related information among themselves. In other specific implementations, the server may query the control center of the C-SLAM system to disseminate necessary information to the indicated UE / RF device. Similar reasoning applies to direct agent-to-agent and agent-to-agent loop closures.
[0127] As a second example, the server can also initiate connections to the control center of the C-SLAM system or directly to the UE / RF device to notify agents that the state on the inter-agent link may change based on vCSI (e.g., the link is becoming NLOS and therefore more vulnerable). This will have even greater implications in the event of security vulnerabilities and interference attacks.
[0128] As an example initiated by the UE side, the agent can maintain a local confidence metric relative to the number and quality of landmarks and their estimates, as well as its own local pose estimate. When these thresholds are exceeded, indicating low confidence, the agent may directly request vCSI assistance from the server or the relevant control block of the C-SLAM system.
[0129] Figure 8 This is a diagram 800 illustrating an example message flow for server-initiated vCSI accessibility functionality according to various aspects of this disclosure. At stage 805, one or more cameras 806 (e.g., surveillance cameras) provide images of the environment containing one or more agents 802 to server 870 (e.g., a location server or a third-party server). Camera 806 may correspond to... Figure 6 The illustrated camera 606, and the intelligent agent 802 can correspond to Figure 6 The illustrated agent 602. The image can be a series of discrete images or a video stream. At stage 810, server 870 processes and stores the received image (i.e., vCSI).
[0130] At stage 815, server 870 sends a request to agent 802 for additional information (e.g., RF measurements, sensor measurements, and / or any other information required to determine whether the events described above have been triggered). For example, this request could be for sensor reports, as described above. At stage 820, agent 802 responds to server 870 using the requested additional information (e.g., sensor reports). At stage 825, server 870 determines, at least in part, whether any triggering conditions for any event have been triggered, as described above (e.g., as referenced). Figure 7If not, the process returns to stage 815, and server 870 can request further additional information from agent 802. If any triggering condition has been met, at stage 830, server 870 determines an event-specific command (including instructions / instructions / assignments) to agent 802 based at least in part on the vCSI received at stage 805, the additional information, or both. At stage 835, server 870 transmits the event-specific command (including instructions / instructions / assignments) to agent 802.
[0131] Figure 9 This is a diagram 900 illustrating an example message flow for agent-initiated vCSI assistive functions according to various aspects of this disclosure. At stage 905, one or more cameras 906 (e.g., surveillance cameras) provide an image of the environment containing one or more agents 902 to a server 970 (e.g., a location server or a third-party server). Camera 906 may correspond to... Figure 6 The illustrated camera 606, and the intelligent agent 902 can correspond to Figure 6 The illustrated agent 602. The image can be a series of discrete images or a video stream. At stage 910, server 870 processes and stores the received image (i.e., vCSI).
[0132] At stage 915, agent 902 determines, at least in part, whether any triggering condition for any event has been triggered based on its own RF / sensor measurements, as described above (e.g., as referenced). Figure 7 If any triggering conditions are met, at stage 920, agent 902 sends a request for assistance to server 970. Specifically, this request can be sent to ( Figure 6 (Example) C-SLAM control center.
[0133] At stage 925, server 970 transmits a request to the involved agent 902 for additional information (e.g., RF measurements, sensor measurements, and any other information required to determine whether the event described above has been triggered). For example, this request could be for a sensor report, as described above. At stage 930, agent 902 responds to server 970 using the requested additional information (e.g., sensor reports). At stage 935, server 970 determines an event-specific command (including instructions / instructions / assignments) to agent 902 based at least in part on the vCSI received at stage 905, the additional information received at stage 930, or both. At stage 940, server 970 transmits the event-specific command (including instructions / instructions / assignments) to agent 902.
[0134] Figure 10An example method 1000 for communication according to various aspects of this disclosure is illustrated. In one aspect, method 1000 can be performed by a network entity (e.g., server 870, server 970).
[0135] At point 1010, the network entity receives multiple sensor reports from multiple agent devices (e.g., agents 602, 802, 902) in the C-SLAM system. These multiple sensor reports include at least RF measurements obtained by the multiple agent devices, such as those in... Figure 8 Phase 820 and Figure 9 At stage 930. In one respect, operation 1010 may be performed by one or more network transceivers 390, one or more processors 394, memory 396 and / or C-SLAM components 398, any or all of these components may be considered as parts for performing this operation.
[0136] At position 1020, the network entity is detected from the visual monitoring system (e.g., Figure 6 The illustrated vOTT monitoring system receives vCSI data associated with the environment of multiple smart agent devices, such as in... Figure 8 Phase 805 and Figure 9 At stage 905. In one respect, operation 1020 may be performed by one or more network transceivers 390, one or more processors 394, memory 396 and / or C-SLAM components 398, any or all of these components may be considered as parts for performing this operation.
[0137] At point 1030, the network entity sends a command to at least one intelligent agent device among a plurality of intelligent agent devices, as shown in... Figure 8 Phase 835 and Figure 9 At 940, the command is at least partially based on vCSI. In one aspect, operation 1030 may be performed by one or more network transceivers 390, one or more processors 394, memory 396 and / or C-SLAM components 398, any or all of which may be considered as components for performing this operation.
[0138] As will be understood, the technical advantage of Method 1000 is the improved C-SLAM performance based on the use of vCSI. That is, using vCSI from vOTT systems helps C-SLAM systems solve technical problems and challenges that are typically difficult, as discussed above, ultimately leading to improved performance. Furthermore, due to the use of vCSI, Method 1000 provides more efficient RRM and radio access, including reduced network traffic.
[0139] As can be seen in the detailed description above, different features are grouped together in the examples. This manner of disclosure should not be construed as an intention to have more features than those explicitly mentioned in each clause. Rather, the various aspects of this disclosure may include fewer features than those in the individual example clauses disclosed. Therefore, the following clauses should be regarded accordingly as incorporated into the description, where each clause may serve as a separate example. Although each dependent clause may refer in the clause to a specific combination with one of the other clauses, the aspect of that dependent clause is not limited to that specific combination. It should be understood that other example clauses may also include combinations of aspects of a dependent clause with the subject matter of any other dependent or independent clause, or combinations of any feature with other dependent and independent clauses. The various aspects disclosed herein explicitly include these combinations unless explicitly stated or readily inferred that a particular combination is not intended for use (e.g., contradictory aspects, such as defining an element as both an electrical insulator and an electrical conductor). Furthermore, it is contemplated that aspects of a clause may be included in any other independent clause, even if that clause does not directly depend on the independent clause.
[0140] Specific implementation examples are described in the following numbered clauses: Clause 1. A method of communication performed by a network entity, the method comprising: receiving multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; receiving visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system; and sending a command to at least one of the multiple agent devices to the at least one agent device, wherein the command is based at least in part on the vCSI, the RF measurements, or both.
[0141] Clause 2. The method according to Clause 1, the method further comprising: determining, at least in part, whether one or more triggering conditions for at least one C-SLAM event have been met based on the vCSI, the RF measurement, or both, wherein the command is sent in response to the determination that the one or more triggering conditions have been met; and sending a request to the plurality of agent devices for reports from the plurality of sensors, wherein the plurality of sensor reports are received in response to the request.
[0142] Clause 3. The method according to Clause 2, wherein the at least one C-SLAM event is one of the following: a loop monitoring and management event, a landmark estimation event, an agent pose estimation event, an agent link monitoring event, or any combination thereof.
[0143] Clause 4. The method according to Clause 3, wherein: the at least one C-SLAM event includes the cyclic monitoring and management event, and the one or more triggering conditions include: a determination at least in part based on the vCSI, the RF measurement, or both, regarding the at least one agent device and at least one second agent device among the plurality of agent devices being within a threshold distance from each other; a determination at least in part based on the vCSI, the RF measurement, or both, regarding the at least one agent device and at least one third agent device among the plurality of agent devices accessing the same location at different times; or any combination thereof.
[0144] Clause 5. The method according to Clause 4, wherein the command includes a command to exchange a portion of map information between the at least one intelligent agent device and the at least one second intelligent agent device, the at least one third intelligent agent device, or both.
[0145] Clause 6. The method according to Clause 5, wherein the portion of the map information comprises: a first area within a threshold distance of the projection intersection of the trajectories of the at least one agent device and the at least one second agent device, a second area within a threshold distance of the same location accessed by the at least one agent device and the at least one third agent device, or any combination thereof.
[0146] Clause 7. The method according to any one of Clauses 3 to 6, wherein: the at least one C-SLAM event includes the landmark estimation event, and the one or more triggering conditions include: at least in part based on the determination of the vCSI regarding one or more feature-rich landmarks within a threshold distance of the at least one agent device, at least in part based on the determination of the vCSI regarding the at least one agent being stuck, or any combination thereof.
[0147] Clause 8. The method according to Clause 7, wherein the command includes a trajectory indication determined at least in part based on the vCSI.
[0148] Clause 9. The method according to any one of Clauses 3 to 8, wherein: the at least one C-SLAM event includes the agent pose estimation event, and the one or more triggering conditions include the at least one agent device's ranging function failing to exceed a confidence threshold.
[0149] Clause 10. The method according to Clause 9, wherein the command includes an estimate of the pose of the at least one agent device determined at least in part based on the vCSI, the RF measurement, or both.
[0150] Clause 11. The method according to any one of Clauses 3 to 10, wherein: the at least one C-SLAM event includes the agent link monitoring event, and the one or more triggering conditions include: at least in part based on the determination by the vCSI that a first link between the at least one agent device and the at least one second agent device is a non-line-of-sight (NLOS) link.
[0151] Clause 12. The method according to Clause 11, wherein the command includes instructions to use a second link between the at least one intelligent agent device and at least one third intelligent agent device.
[0152] Clause 13. The method according to any one of Clauses 1 to 12, the method further comprising: determining the location of the plurality of agent devices within the environment based at least in part on the vCSI; and determining one or more locations in the environment based at least in part on the vCSI, wherein each of the one or more locations is described by a list of indicators, and wherein each of the plurality of agent devices is described by a location in the one or more locations and a timestamp when the agent device is located at the location.
[0153] Clause 14. The method according to Clause 13, the method further comprising: maintaining a list of intelligent agent devices among the plurality of intelligent agent devices that have accessed a given location among the one or more locations; and removing the intelligent agent device from the list of intelligent agent devices based on a threshold time period elapsed since the intelligent agent device last accessed the given location.
[0154] Clause 15. The method according to any one of Clauses 1 to 14, the method further comprising: determining a proximity metric between pairs of agent devices among the plurality of agent devices based at least in part on the vCSI, the RF measurement, or both.
[0155] Clause 16. The method according to any one of Clauses 1 to 15, the method further comprising: receiving from the at least one agent device a request for vCSI assistance in the event of at least one C-SLAM event; and sending to the plurality of agent devices a request for the plurality of sensor reports, wherein the plurality of sensor reports are received in response to the request.
[0156] Clause 17. The method according to Clause 16, wherein the command is sent in response to the receipt of reports from the plurality of sensors.
[0157] Clause 18. The method according to any one of Clauses 16 to 17, wherein the at least one C-SLAM event comprises: a confidence level associated with the number and quality of landmarks in the environment being below a threshold, a confidence level associated with the estimated pose of the at least one agent device being below a threshold, or any combination thereof.
[0158] Clause 19. The method according to any one of Clauses 1 to 18, wherein: the visual monitoring system includes a plurality of monitoring cameras, and the plurality of intelligent agent devices include a plurality of industrial robots equipped with user equipment or RF communication devices.
[0159] Clause 20. A network entity comprising: one or more memories; one or more transceivers; and one or more processors communicatively coupled to the one or more memories and the one or more transceivers, the one or more processors being individually or in combination configured to: receive multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system via the one or more transceivers, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; receive visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system via the one or more transceivers; and send a command to at least one of the multiple agent devices via the one or more transceivers to the at least one agent device, wherein the command is based at least in part on the vCSI, the RF measurements, or both.
[0160] Clause 21. The network entity as described in Clause 20, wherein the one or more processors are further configured individually or in combination to: determine, at least in part, whether one or more triggering conditions for at least one C-SLAM event have been met based on the vCSI, the RF measurement, or both, wherein the command is sent in response to the determination that the one or more triggering conditions have been met; and to send a request for the plurality of sensor reports to the plurality of agent devices via the one or more transceivers, wherein the plurality of sensor reports are received in response to the request.
[0161] Clause 22. The network entity as described in Clause 21, wherein the at least one C-SLAM event is one of the following: a cyclic monitoring and management event, a landmark estimation event, an agent pose estimation event, an agent link monitoring event, or any combination thereof.
[0162] Clause 23. A network entity as described in Clause 22, wherein: the at least one C-SLAM event includes the cyclic monitoring and management event, and the one or more triggering conditions include: a determination, at least in part, based on the vCSI, the RF measurement, or both, regarding the at least one agent device and at least one second agent device among the plurality of agent devices being within a threshold distance from each other; a determination, at least in part, based on the vCSI, the RF measurement, or both, regarding the at least one agent device and at least one third agent device among the plurality of agent devices accessing the same location at different times; or any combination thereof.
[0163] Clause 24. The network entity as described in Clause 23, wherein the command includes a command to exchange a portion of map information between the at least one agent device and the at least one second agent device, the at least one third agent device, or both.
[0164] Clause 25. A network entity as described in Clause 24, wherein the portion of the map information includes: a first area within a threshold distance of the projection intersection of the trajectories of the at least one agent device and the at least one second agent device, a second area within a threshold distance of the same location accessed by the at least one agent device and the at least one third agent device, or any combination thereof.
[0165] Clause 26. A network entity pursuant to any one of Clauses 22 to 25, wherein: the at least one C-SLAM event includes the landmark estimation event, and the one or more triggering conditions include: at least in part based on the determination of the vCSI regarding one or more feature-rich landmarks within a threshold distance of the at least one agent device, at least in part based on the determination of the vCSI regarding the at least one agent being stuck, or any combination thereof.
[0166] Clause 27. The network entity as described in Clause 26, wherein the command includes a trajectory indication determined at least in part based on the vCSI.
[0167] Clause 28. A network entity according to any one of Clauses 22 to 27, wherein: the at least one C-SLAM event includes the agent pose estimation event, and the one or more triggering conditions include the at least one agent device's ranging function failing to exceed a confidence threshold.
[0168] Clause 29. The network entity as described in Clause 28, wherein the command includes an estimate of the pose of the at least one agent device determined at least in part based on the vCSI, the RF measurement, or both.
[0169] Clause 30. A network entity pursuant to any one of Clauses 22 to 29, wherein: the at least one C-SLAM event includes the agent link monitoring event, and the one or more triggering conditions include: a determination, at least in part, based on the vCSI, that a first link between the at least one agent device and the at least one second agent device is a non-line-of-sight (NLOS) link.
[0170] Clause 31. The network entity as described in Clause 30, wherein the command includes instructions to use a second link between the at least one intelligent agent device and at least one third intelligent agent device.
[0171] Clause 32. A network entity according to any one of Clauses 20 to 31, wherein the one or more processors are further configured individually or in combination to: determine the location of the plurality of agent devices within the environment based at least in part on the vCSI; and determine one or more locations in the environment based at least in part on the vCSI, wherein each of the one or more locations is described by a list of indicators, and wherein each of the plurality of agent devices is described by a location in the one or more locations and a timestamp when the agent device is located at the location.
[0172] Clause 33. The network entity as described in Clause 32, wherein the one or more processors are further configured individually or in combination to: maintain a list of agent devices among the plurality of agent devices that access a given location among the one or more locations; and to remove the agent device from the list of agent devices based on a threshold time period elapsed since the agent device last accessed the given location.
[0173] Clause 34. A network entity pursuant to any one of Clauses 20 to 33, wherein the one or more processors are further configured individually or in combination to determine a proximity metric between pairs of agent devices among the plurality of agent devices based at least in part on the vCSI, the RF measurement, or both.
[0174] Clause 35. A network entity pursuant to any one of Clauses 20 to 34, wherein the one or more processors are further configured individually or in combination to: receive, via the one or more transceivers, a request for vCSI assistance in the event of at least one C-SLAM event from the at least one agent device; and transmit via the one or more transceivers a request for the plurality of sensor reports to the plurality of agent devices, wherein the plurality of sensor reports are received in response to the request.
[0175] Clause 36. The network entity as described in Clause 35, wherein the command is sent in response to the receipt of reports from the plurality of sensors.
[0176] Clause 37. A network entity pursuant to any one of Clauses 35 to 36, wherein the at least one C-SLAM event comprises: a confidence level associated with the number and quality of landmarks in the environment being below a threshold, a confidence level associated with the estimated pose of the at least one agent device being below a threshold, or any combination thereof.
[0177] Clause 38. A network entity pursuant to any one of Clauses 20 to 37, wherein: the visual monitoring system comprises a plurality of surveillance cameras, and the plurality of intelligent agent devices comprises a plurality of industrial robots equipped with user equipment or RF communication devices.
[0178] Clause 39. A network entity comprising: means for receiving multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; means for receiving visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system; and means for sending a command to at least one of the multiple agent devices, wherein the command is based at least in part on the vCSI, the RF measurements, or both.
[0179] Clause 40. The network entity as described in Clause 39, further comprising: means for determining, at least in part, based on the vCSI, the RF measurement, or both, whether one or more triggering conditions for at least one C-SLAM event have been met, wherein the command is sent in response to the determination that the one or more triggering conditions have been met; and means for sending a request to the plurality of agent devices for the plurality of sensor reports, wherein the plurality of sensor reports are received in response to the request.
[0180] Clause 41. The network entity as described in Clause 40, wherein the at least one C-SLAM event is one of the following: a cyclic monitoring and management event, a landmark estimation event, an agent pose estimation event, an agent link monitoring event, or any combination thereof.
[0181] Clause 42. A network entity as described in Clause 41, wherein: the at least one C-SLAM event includes the cyclic monitoring and management event, and the one or more triggering conditions include: a determination, at least in part, based on the vCSI, the RF measurement, or both, regarding the at least one agent device and at least one second agent device among the plurality of agent devices being within a threshold distance from each other; a determination, at least in part, based on the vCSI, the RF measurement, or both, regarding the at least one agent device and at least one third agent device among the plurality of agent devices accessing the same location at different times; or any combination thereof.
[0182] Clause 43. The network entity as described in Clause 42, wherein the command includes a command to exchange a portion of map information between the at least one intelligent agent device and the at least one second intelligent agent device, the at least one third intelligent agent device, or both.
[0183] Clause 44. A network entity as described in Clause 43, wherein the portion of the map information includes: a first area within a threshold distance of the projection intersection of the trajectories of the at least one agent device and the at least one second agent device, a second area within a threshold distance of the same location accessed by the at least one agent device and the at least one third agent device, or any combination thereof.
[0184] Clause 45. A network entity pursuant to any one of Clauses 41 to 44, wherein: the at least one C-SLAM event includes the landmark estimation event, and the one or more triggering conditions include: at least in part based on the determination of the vCSI regarding one or more feature-rich landmarks within a threshold distance of the at least one agent device, at least in part based on the determination of the vCSI regarding the at least one agent being stuck, or any combination thereof.
[0185] Clause 46. The network entity as described in Clause 45, wherein the command includes a trajectory indication determined at least in part based on the vCSI.
[0186] Clause 47. A network entity according to any one of Clauses 41 to 46, wherein: the at least one C-SLAM event includes the agent pose estimation event, and the one or more triggering conditions include the at least one agent device's ranging function failing to exceed a confidence threshold.
[0187] Clause 48. The network entity as described in Clause 47, wherein the command includes an estimate of the pose of the at least one agent device determined at least in part based on the vCSI, the RF measurement, or both.
[0188] Clause 49. A network entity pursuant to any one of Clauses 41 to 48, wherein: the at least one C-SLAM event includes the agent link monitoring event, and the one or more triggering conditions include: a determination, at least in part, based on the vCSI, that a first link between the at least one agent device and the at least one second agent device is a non-line-of-sight (NLOS) link.
[0189] Clause 50. The network entity as described in Clause 49, wherein the command includes instructions to use a second link between the at least one intelligent agent device and at least one third intelligent agent device.
[0190] Clause 51. A network entity according to any one of Clauses 39 to 50, the network entity further comprising: components for determining the location of the plurality of agent devices within the environment based at least in part on the vCSI; and components for determining one or more locations in the environment based at least in part on the vCSI, wherein each of the one or more locations is described by a list of indicators, and wherein each of the plurality of agent devices is described by a location in the one or more locations and a timestamp when the agent device is located at the location.
[0191] Clause 52. The network entity as described in Clause 51, further comprising: components for maintaining a list of intelligent agent devices among the plurality of intelligent agent devices that have accessed a given location among the one or more locations; and components for removing the intelligent agent device from the list of intelligent agent devices based on a threshold time period elapsed since the intelligent agent device last accessed the given location.
[0192] Clause 53. The network entity according to any one of Clauses 39 to 52, the network entity further comprising: a component for determining a proximity metric between pairs of agent devices among the plurality of agent devices based at least in part on the vCSI, the RF measurement, or both.
[0193] Clause 54. The network entity according to any one of Clauses 39 to 53, the network entity further comprising: means for receiving from the at least one agent device a request for vCSI assistance in the event of at least one C-SLAM event; and means for sending to the plurality of agent devices a request for the plurality of sensor reports, wherein the plurality of sensor reports are received in response to the request.
[0194] Clause 55. The network entity as described in Clause 54, wherein the command is sent in response to the receipt of reports from the plurality of sensors.
[0195] Clause 56. A network entity pursuant to any one of Clauses 54 to 55, wherein the at least one C-SLAM event comprises: a confidence level associated with the number and quality of landmarks in the environment being below a threshold, a confidence level associated with the estimated pose of the at least one agent device being below a threshold, or any combination thereof.
[0196] Clause 57. A network entity pursuant to any one of Clauses 39 to 56, wherein: the visual monitoring system comprises a plurality of surveillance cameras, and the plurality of intelligent agent devices comprises a plurality of industrial robots equipped with user equipment or RF communication devices.
[0197] Clause 58. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by a network entity, cause the network entity to: receive multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; receive visual channel state information (vCSI) associated with the environment in which the multiple agent devices are located from a visual monitoring system; and send a command to at least one of the multiple agent devices, wherein the command is based at least in part on the vCSI, the RF measurements, or both.
[0198] Clause 59. The non-transitory computer-readable medium according to Clause 58, further comprising computer-executable instructions, which, when executed by the network entity, cause the network entity to: determine, at least in part, based on the vCSI, the RF measurement, or both, whether one or more triggering conditions for at least one C-SLAM event have been met, wherein the command is sent in response to the determination that the one or more triggering conditions have been met; and send a request to the plurality of intelligent agent devices for reports from the plurality of sensors, wherein the plurality of sensor reports are received in response to the request.
[0199] Clause 60. The non-transitory computer-readable medium pursuant to Clause 59, wherein the at least one C-SLAM event is one of the following: a cyclic monitoring and management event, a landmark estimation event, an agent pose estimation event, an agent link monitoring event, or any combination thereof.
[0200] Clause 61. The non-transitory computer-readable medium according to Clause 60, wherein: the at least one C-SLAM event includes the cyclic monitoring and management event, and the one or more triggering conditions include: a determination at least in part based on the vCSI, the RF measurement, or both, regarding at least one second agent device among the at least one agent device and at least one third agent device among the plurality of agent devices being within a threshold distance from each other; a determination at least in part based on the vCSI, the RF measurement, or both, regarding at least one third agent device among the at least one agent device and at least one third agent device visiting the same location at different times; or any combination thereof.
[0201] Clause 62. The non-transitory computer-readable medium as described in Clause 61, wherein the command includes a command to exchange a portion of map information between the at least one intelligent agent device and the at least one second intelligent agent device, the at least one third intelligent agent device, or both.
[0202] Clause 63. The non-transitory computer-readable medium as described in Clause 62, wherein the portion of the map information comprises: a first region within a threshold distance of the projection intersection of the trajectories of the at least one intelligent agent device and the at least one second intelligent agent device, a second region within a threshold distance of the same location accessed by the at least one intelligent agent device and the at least one third intelligent agent device, or any combination thereof.
[0203] Clause 64. A non-transitory computer-readable medium according to any one of Clauses 60 to 63, wherein: the at least one C-SLAM event includes the landmark estimation event, and the one or more triggering conditions include: at least in part based on the determination of the vCSI regarding one or more feature-rich landmarks within a threshold distance of the at least one agent device, at least in part based on the determination of the vCSI regarding the at least one agent being stuck, or any combination thereof.
[0204] Clause 65. The non-transitory computer-readable medium as described in Clause 64, wherein the command includes a trajectory indication determined at least in part based on the vCSI.
[0205] Clause 66. A non-transitory computer-readable medium according to any one of Clauses 60 to 65, wherein: the at least one C-SLAM event includes the agent pose estimation event, and the one or more triggering conditions include the at least one agent device's ranging function failing to exceed a confidence threshold.
[0206] Clause 67. The non-transitory computer-readable medium pursuant to Clause 66, wherein the command comprises an estimate of the pose of the at least one agent device determined at least in part based on the vCSI, the RF measurement, or both.
[0207] Clause 68. A non-transitory computer-readable medium according to any one of Clauses 60 to 67, wherein: the at least one C-SLAM event includes the agent link monitoring event, and the one or more triggering conditions include: a determination, at least in part based on the vCSI, that a first link between the at least one agent device and the at least one second agent device is a non-line-of-sight (NLOS) link.
[0208] Clause 69. The non-transitory computer-readable medium pursuant to Clause 68, wherein the commands include instructions for using a second link between the at least one intelligent agent device and at least one third intelligent agent device.
[0209] Clause 70. A nontransitory computer-readable medium according to any one of Clauses 58 to 69, the nontransitory computer-readable medium further comprising computer-executable instructions, which, when executed by the network entity, cause the network entity to: determine the location of the plurality of agent devices within the environment based at least in part on the vCSI; and determine one or more locations in the environment based at least in part on the vCSI, wherein each of the one or more locations is described by a list of indicators, and wherein each of the plurality of agent devices is described by a location in the one or more locations and a timestamp when the agent device is located at the location.
[0210] Clause 71. The non-transitory computer-readable medium according to Clause 70 further includes computer-executable instructions that, when executed by the network entity, cause the network entity to: maintain a list of intelligent agent devices among the plurality of intelligent agent devices that have accessed a given location among the one or more locations; and remove the intelligent agent device from the list of intelligent agent devices based on a threshold time period elapsed since the intelligent agent device last accessed the given location.
[0211] Clause 72. The non-transitory computer-readable medium according to any one of Clauses 58 to 71, the non-transitory computer-readable medium further comprising computer-executable instructions that, when executed by the network entity, cause the network entity to: determine, at least in part, a proximity metric between pairs of agent devices among the plurality of agent devices based on the vCSI, the RF measurement, or both.
[0212] Clause 73. A nontransitory computer-readable medium according to any one of Clauses 58 to 72, the nontransitory computer-readable medium further comprising computer-executable instructions, which, when executed by the network entity, cause the network entity to: receive from the at least one agent device a request for vCSI assistance in the event of at least one C-SLAM event; and send requests to the plurality of agent devices for the plurality of sensor reports, wherein the plurality of sensor reports are received in response to the requests.
[0213] Clause 74. The non-transitory computer-readable medium as described in Clause 73, wherein the command is transmitted in response to the receipt of reports from the plurality of sensors.
[0214] Clause 75. A non-transitory computer-readable medium according to any one of Clauses 73 to 74, wherein the at least one C-SLAM event comprises: a confidence level associated with the number and quality of landmarks in the environment being below a threshold, a confidence level associated with the estimated pose of the at least one agent device being below a threshold, or any combination thereof.
[0215] Clause 76. A non-transitory computer-readable medium according to any one of Clauses 58 to 75, wherein: the visual monitoring system includes a plurality of monitoring cameras, and the plurality of intelligent agent devices include a plurality of industrial robots equipped with user equipment or RF communication devices.
[0216] Those skilled in the art will understand that information and signals can be represented using any of a variety of different techniques and arts. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be mentioned throughout the above description can be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, light fields or optical particles, or any combination thereof.
[0217] Furthermore, those skilled in the art will understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the aspects disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and steps have been described above in general terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art can implement the described functionality in different ways for each specific application; however, such implementation decisions should not be construed as departing from the scope of this disclosure.
[0218] The various exemplary logic blocks, modules, and circuits described in conjunction with the aspects disclosed herein may be implemented or performed using a general-purpose processor, a digital signal processor (DSP), an ASIC, a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic components, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but in alternative embodiments, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration.
[0219] The methods, sequences, and / or algorithms described in conjunction with the aspects disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or a combination of both. The software module may reside in random access memory (RAM), flash memory, read-only memory (ROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the art. Example storage media are coupled to a processor such that the processor can read information from and write information to the storage medium. Alternatively, the storage medium may be integral with the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal (e.g., a UE). Alternatively, the processor and storage medium may reside as discrete components in the user terminal.
[0220] In one or more examples, the described functionality may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on or transmitted via a computer-readable medium. A computer-readable medium includes both computer storage media and communication media, which includes any medium that facilitates the transfer of a computer program from one place to another. A storage medium may be any available medium accessible to a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and is accessible to a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included within the definition of a medium. As used herein, disks and optical discs include: compact optical discs (CDs), laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs. Disks typically reproduce data magnetically, while optical discs reproduce data optically using lasers. Combinations of these should also be included within the scope of computer-readable media.
[0221] While the foregoing disclosure illustrates exemplary aspects of this disclosure, it should be noted that various changes and modifications may be made herein without departing from the scope of this disclosure as defined by the appended claims. For example, the functions, steps, and / or actions of the method claims according to aspects of this disclosure described herein need not be performed in any particular order. Furthermore, no component, function, action, or instruction described or claimed herein should be construed as critical or essential unless explicitly stated otherwise. Additionally, as used herein, the terms “set,” “group,” etc., are intended to include one or more of the stated elements. Furthermore, as used herein, the terms “having,” “comprising,” “including,” etc., do not exclude the presence of one or more additional elements (e.g., element “having” A may also have B). Furthermore, the phrase “based on” is intended to mean “at least partially based on” unless otherwise explicitly stated. Furthermore, as used herein, the term “or” is intended to be open-ended when used in a series and is interchangeable with “and / or” unless otherwise expressly stated (e.g., if used in conjunction with “any” or “only one”), or these alternatives are mutually exclusive (e.g., “one or more” should not be interpreted as “one and more”). Additionally, although components, functions, actions, and instructions may be described or claimed in the singular, plural forms may also be considered unless expressly stated as limited to the singular. Therefore, as used herein, the articles “a,” “an,” “the,” and “the” are intended to include one or more of the described elements. Additionally, as used herein, the terms “at least one” and “one or more” include “one” component, function, action, or instruction that performs or is capable of performing the described or claimed functionality, and also include “two or more” components, functions, actions, or instructions that perform or are capable of performing the described or claimed functionality in combination.
Claims
1. A method for communication performed by a network entity, the method comprising: Receive multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; Receive visual channel state information (vCSI) associated with the environment in which the plurality of intelligent agent devices are located from the visual monitoring system. as well as Send a command to at least one of the plurality of intelligent agent devices, wherein the command is based at least in part on the vCSI, the RF measurement, or both.
2. The method according to claim 1, further comprising: The determination of whether one or more triggering conditions for at least one C-SLAM event have been met is based at least in part on the vCSI, the RF measurement, or both, wherein the command is sent in response to the determination that the one or more triggering conditions have been met; as well as Send requests for reports from the plurality of sensor devices to the plurality of intelligent agent devices, wherein the plurality of sensor reports are received in response to the requests.
3. The method of claim 2, wherein the at least one C-SLAM event is one of the following: Cyclic monitoring and management of events, Landmark estimation event, Intelligent agent pose estimation event. Intelligent agent link monitoring events, or Any combination of them.
4. The method according to claim 3, wherein: The at least one C-SLAM event includes the cyclic monitoring and management event, and The one or more triggering conditions include: Based at least in part on the determination of the at least one agent device and at least one second agent device among the plurality of agent devices within a threshold distance from each other, based on the vCSI, the RF measurement, or both. At least in part based on the determination of the at least one intelligent agent device and at least one third intelligent agent device among the plurality of intelligent agent devices accessing the same location at different times, or Any combination of them.
5. The method of claim 4, wherein the command includes a command to exchange a portion of map information between the at least one agent device and the at least one second agent device, the at least one third agent device, or both.
6. The method of claim 5, wherein the portion of the map information comprises: A first region within a threshold distance of the intersection of the projected trajectories of the at least one intelligent agent device and the at least one second intelligent agent device. A second area within a threshold distance of the same location accessed by the at least one intelligent agent device and the at least one third intelligent agent device, or Any combination of them.
7. The method according to claim 3, wherein: The at least one C-SLAM event includes the landmark estimation event, and The one or more triggering conditions include: At least in part, based on the determination of the vCSI regarding one or more feature-rich landmarks within a threshold distance of the at least one agent device, At least in part based on the vCSI's determination that the at least one agent is stuck, or Any combination of them.
8. The method of claim 7, wherein the command includes a trajectory indication determined at least in part based on the vCSI.
9. The method according to claim 3, wherein: The at least one C-SLAM event includes the agent pose estimation event, and The one or more triggering conditions include the ranging function of at least one agent device failing to exceed the confidence threshold.
10. The method of claim 9, wherein the command includes an estimate of the pose of the at least one agent device determined at least in part based on the vCSI, the RF measurement, or both.
11. The method according to claim 3, wherein: The at least one C-SLAM event includes the agent link monitoring event, and The one or more triggering conditions include: The determination that the first link between the at least one agent device and the at least one second agent device is a non-line-of-sight (NLOS) link is based at least in part on the vCSI.
12. The method of claim 11, wherein the command includes instructions to use a second link between the at least one intelligent agent device and the at least one third intelligent agent device.
13. The method according to claim 1, further comprising: The locations of the plurality of intelligent agent devices within the environment are determined at least in part based on the vCSI; as well as One or more locations in the environment are determined at least in part based on the vCSI, wherein each of the one or more locations is described by a list of indicators, and wherein each of the plurality of agent devices is described by a location in the one or more locations and a timestamp when the agent device is located at the location.
14. The method according to claim 13, further comprising: Maintain a list of intelligent agent devices among the plurality of intelligent agent devices that access a given location among the one or more locations; as well as The agent device is removed from the list of agent devices based on a threshold time period elapsed since the agent device last visited the given location.
15. The method according to claim 1, further comprising: The proximity metric between pairs of agent devices among the plurality of agent devices is determined at least in part based on the vCSI, the RF measurement, or both.
16. The method according to claim 1, further comprising: Receive a request for vCSI assistance in the event of at least one C-SLAM event from the at least one agent device; as well as Send requests for reports from the plurality of sensor devices to the plurality of intelligent agent devices, wherein the plurality of sensor reports are received in response to the requests.
17. The method of claim 16, wherein the command is sent in response to the receipt of reports from the plurality of sensors.
18. The method of claim 16, wherein the at least one C-SLAM event comprises: The confidence level associated with the number and quality of landmarks in the environment is below a threshold. The confidence level associated with the estimated pose of the at least one intelligent agent device is below a threshold, or Any combination of them.
19. The method according to claim 1, wherein: The visual monitoring system includes multiple surveillance cameras, and The multiple intelligent agent devices include multiple industrial robots equipped with user equipment or RF communication devices.
20. A network entity, the network entity comprising: One or more memory units; One or more transceivers; and One or more processors, communicatively coupled to one or more memories and one or more transceivers, wherein the one or more processors are configured individually or in combination to: The system receives multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system via one or more transceivers, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; The visual channel state information (vCSI) associated with the environment in which the plurality of intelligent agent devices are located is received from the visual monitoring system via the one or more transceivers. as well as Commands to at least one of the plurality of intelligent agent devices are sent via the one or more transceivers to the at least one intelligent agent device, wherein the commands are based at least in part on the vCSI, the RF measurement, or both.
21. The network entity of claim 20, wherein the one or more processors are further configured individually or in combination to: The determination of whether one or more triggering conditions for at least one C-SLAM event have been met is based at least in part on the vCSI, the RF measurement, or both, wherein the command is sent in response to the determination that the one or more triggering conditions have been met; and Requests for the plurality of sensor reports are sent to the plurality of intelligent agent devices via the one or more transceivers, wherein the plurality of sensor reports are received in response to the requests.
22. The network entity of claim 21, wherein the at least one C-SLAM event is one of the following: Cyclic monitoring and management of events, Landmark estimation event, Intelligent agent pose estimation event. Intelligent agent link monitoring events, or Any combination of them.
23. The network entity according to claim 22, wherein: The at least one C-SLAM event includes the cyclic monitoring and management event, and The one or more triggering conditions include: Based at least in part on the determination of the at least one agent device and at least one second agent device among the plurality of agent devices within a threshold distance from each other, based on the vCSI, the RF measurement, or both. At least in part based on the determination of the at least one intelligent agent device and at least one third intelligent agent device among the plurality of intelligent agent devices accessing the same location at different times, or Any combination of them.
24. The network entity according to claim 22, wherein: The at least one C-SLAM event includes the landmark estimation event, and The one or more triggering conditions include: At least in part, based on the determination of the vCSI regarding one or more feature-rich landmarks within a threshold distance of the at least one agent device, At least in part based on the vCSI's determination that the at least one agent is stuck, or Any combination of them.
25. The network entity according to claim 22, wherein: The at least one C-SLAM event includes the agent pose estimation event, and The one or more triggering conditions include the ranging function of at least one agent device failing to exceed the confidence threshold.
26. The network entity according to claim 22, wherein: The at least one C-SLAM event includes the agent link monitoring event, and The one or more triggering conditions include: The determination that the first link between the at least one agent device and the at least one second agent device is a non-line-of-sight (NLOS) link is based at least in part on the vCSI.
27. The network entity of claim 20, wherein the one or more processors are further configured individually or in combination to: The locations of the plurality of agent devices within the environment are determined at least in part based on the vCSI; and One or more locations in the environment are determined at least in part based on the vCSI, wherein each of the one or more locations is described by a list of indicators, and wherein each of the plurality of agent devices is described by a location in the one or more locations and a timestamp when the agent device is located at the location.
28. The network entity of claim 20, wherein the one or more processors are further configured individually or in combination to: Receive, via the one or more transceivers, a request for vCSI assistance in the event of at least one C-SLAM event from the at least one agent device; and Requests for the plurality of sensor reports are sent to the plurality of intelligent agent devices via the one or more transceivers, wherein the plurality of sensor reports are received in response to the requests.
29. A network entity, the network entity comprising: Components for receiving multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; A component for receiving visual channel state information (vCSI) associated with the environment in which the plurality of intelligent agent devices are located from a visual monitoring system; and Components for sending commands to at least one of the plurality of agent devices, wherein the commands are based at least in part on the vCSI, the RF measurement, or both.
30. A non-transitory computer-readable medium storing computer-executable instructions, which, when executed by a network entity, cause the network entity to: Receive multiple sensor reports from multiple agent devices in a cooperative simultaneous localization and mapping (C-SLAM) system, the multiple sensor reports including at least radio frequency (RF) measurements obtained by the multiple agent devices; Receive visual channel state information (vCSI) associated with the environment in which the plurality of intelligent agent devices are located from the visual monitoring system. as well as Send a command to at least one of the plurality of intelligent agent devices, wherein the command is based at least in part on the vCSI, the RF measurement, or both.