Network node joint coordination exploration for user equipment radio parameter optimization

Abstract

Technologies include during a configuration exchange, after a key neighbor gNB is identified by a source gNB, the source gNB identifies an interpretation of a key performance indicator (KPI) and a bitstring indicator as a quantitative change in the KPI. In one embodiment, a bit string indicator is transmitted by a key neighbor gNB to a source gNB to reflect a change in cell KPI due to a previous action of the source gNB, the KPI in the source cell is measured to generate its own bit string, a combined bit string is generated based on the received bit string and the generated bit string, and an action is taken based on the combined bit string.

Description

Technical Field

[0001] This specification relates to telecommunication systems. Background Art

[0002] A communication system may be a facility that enables communication between two or more nodes or devices (such as fixed communication devices or mobile communication devices). Signals may be carried on a wired carrier or a wireless carrier.

[0003] An example of a cellular communication system is the architecture standardized by the Third Generation Partnership Project (3GPP). Recent developments in this area are often referred to as the Long Term Evolution (LTE) of the Universal Mobile Telecommunications System (UMTS) radio access technology. E-UTRA (Evolved UMTS Terrestrial Radio Access) is the air interface of 3GPP's LTE upgrade path for mobile networks. In LTE, a base station or access point (AP), which is called an enhanced node AP (eNB), provides wireless access within a coverage area or cell. In LTE, a mobile device or mobile station is called a user equipment (UE). LTE has included many improvements or developments.

[0004] The global bandwidth shortage for wireless carriers has stimulated the consideration of using underutilized millimeter wave (mmWave) spectrum for, for example, future broadband cellular communication networks. mmWave (or very high frequency) can, for example, include a frequency range between 30 and 300 gigahertz (GHz). Radio waves in this frequency band can, for example, have a wavelength from ten millimeters to one millimeter, giving it the name millimeter band or millimeter wave. The amount of wireless data is likely to increase significantly in the next few years. Various technologies have been used to try to solve this challenge, including obtaining more spectrum, having smaller cell sizes, and using improved technologies that achieve more bits / second / hertz. One factor that can be used to obtain more spectrum is to move to higher frequencies, for example, above 6 GHz. For fifth-generation wireless systems (5G), access architectures for deploying cellular radio equipment that employ mmWave radio spectrum have been proposed. Other example spectrums can also be used, such as cmWave radio spectrum (e.g., 3-30 GHz). Summary of the Invention

[0005] According to an example implementation, a method includes: identifying, by a source network node controlling a source cell, a key neighbor network node controlling a key neighbor cell based on a cluster of user equipment connected to the source network node, a key neighbor node for which a change in a radio transmission parameter value for the cluster of user equipment affects the key neighbor node. The method also includes identifying, by the source network node, an interpretation of a bit string indicator for indicating a change in a key performance indicator at the source cell and the key neighbor cell. The method also includes performing, by the source network node, a first action resulting in a change in the radio transmission parameter value for the cluster of user equipment. The method also includes receiving, by the source network node, a first value of the bit string indicator from the key neighbor network node, the first value indicating a change in the key performance indicator at the key neighbor cell. The method also includes generating, by the source network node, a second value of the bit string indicator based on a measurement of the key performance indicator at the source cell.

[0006] According to an example implementation, an apparatus includes at least one processor and at least one memory, including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to at least: identify, by a source network node controlling a source cell, a key neighbor network node controlling a key neighbor cell based on a cluster of user equipment connected to the source network node, a change in a radio transmission parameter value for the cluster of user equipment affecting the key neighbor node. The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to at least: identify, by the source network node, an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell. The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to at least: perform, by the source network node, a first action resulting in a change in the radio transmission parameter value for the cluster of user equipment. The at least one memory and the computer program code are further configured to, with the at least processor, cause the apparatus to at least: receive, by the source network node, from the key neighbor network node, a first value of the bit string indicator, the first value indicating a change in the key performance indicator at the key neighbor cell. The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to at least: generate, by the source network node, a second value of the bit string indicator based on measurements of the key performance indicator at the source cell.

[0007] According to an example implementation, an apparatus includes means for, by a source network node controlling a source cell, identifying a key neighbor network node controlling a key neighbor cell based on a cluster of user equipment connected to the source network node, wherein a change in a radio transmission parameter value for the cluster of user equipment affects the key neighbor node. The apparatus also includes means for, by the source network node, identifying an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell. The apparatus also includes means for, by the source network node, performing a first action, the first action resulting in a change in the radio transmission parameter value for the cluster of user equipment. The apparatus also includes means for, by the source network node, receiving a first value of the bit string indicator from the key neighbor network node, the first value indicating a change in the key performance indicator at the key neighbor cell. The apparatus also includes means for, by the source network node, generating a second value of the bit string indicator based on a measurement of the key performance indicator at the source cell.

[0008] According to an example implementation, a computer program product includes a computer-readable storage medium storing executable code. The executable code, when executed by at least one data processing device, is configured to cause a source network node controlling a source cell to identify a key neighbor network node controlling a key neighbor cell based on a cluster of user equipment connected to the source network node, wherein a change in a radio transmission parameter value for the cluster of user equipment affects the key neighbor node. The executable code is further configured to cause the at least one data processing device to identify an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell. The executable code is further configured to cause the at least one data processing device to perform a first action at the source network node, the first action resulting in a change in the radio transmission parameter value for the cluster of user equipment. The executable code is further configured to cause the at least one data processing device to receive a first value of the bit string indicator from the key neighbor network node at the source network node, the first value indicating a change in the key performance indicator at the key neighbor cell. The executable code is further configured to cause the at least one data processing device to generate a second value of the bit string indicator at the source network node based on a measurement of the key performance indicator at the source cell.

[0009] According to an example implementation, a method includes receiving, by a key neighbor network node controlling a key neighbor cell, a configuration message from a source node controlling a source cell, the configuration message identifying an interpretation of a bit string indicator for indicating a change in a key performance indicator at the source cell and the key neighbor cell. The method also includes receiving, by the key neighbor network node, an indication of a change in a radio transmission parameter value for a cluster of user equipment. The method also includes sending, by the key neighbor node, to the source node a first bit string indicator indicating a change in the key performance indicator at the key neighbor cell.

[0010] According to an example implementation, an apparatus includes at least one processor and at least one memory, including computer program code, the at least one memory and the computer program code being configured to, with the at least one processor, cause the apparatus to at least: receive, by a key neighbor network node controlling a key neighbor cell, a configuration message from a source node controlling a source cell, the configuration message identifying an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell. The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to at least: receive, by the key neighbor network node, an indication of a change in a radio transmission parameter value for a cluster of user equipment. The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to at least: send, by the key neighbor node, to the source node, a first bit string indicator indicating a change in the key performance indicator at the key neighbor cell.

[0011] According to an example implementation, an apparatus includes means for receiving, by a key neighbor network node controlling a key neighbor cell, a configuration message from a source node controlling a source cell, the configuration message identifying an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell. The apparatus also includes means for receiving, by the key neighbor network node, an indication of a change in a radio transmission parameter value for a cluster of user equipment. The apparatus also includes means for sending, by the key neighbor node, to the source node, a first bit string indicator indicating a change in the key performance indicator at the key neighbor cell.

[0012] According to an example implementation, a computer program product includes a computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to: receive, by a key neighbor network node controlling a key neighbor cell, a configuration message from a source node controlling a source cell, the configuration message identifying an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell; receive, by the key neighbor network node, an indication of a change in a radio transmission parameter value for a cluster of user equipment; and send, by the key neighbor node, to the source node a first bit string indicator indicating a change in the key performance indicator at the key neighbor cell.

[0013] The details of one or more examples of implementations are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a block diagram of a digital communication network according to an example implementation.

[0015] Figure 2 is a flow chart illustrating a process for coordinating key performance indicator measurements with selected key neighbor gNBs, according to an example implementation.

[0016] Figure 3 is a flow chart illustrating a process of reinforcement learning-based exploration and exploitation, according to an example implementation.

[0017] Figure 4 is a flow chart illustrating a process for reinforcement learning-based inter-gNB coordination steps for uplink transmit power control (UL-TPC), according to an example implementation.

[0018] Figure 5 is a signaling diagram illustrating a process of coordinated discovery according to an example implementation.

[0019] Figure 6 is a flow chart illustrating a process for coordinating key performance indicator measurements with selected key neighbor gNBs, according to an example implementation.

[0020] Figure 7 is a flow chart illustrating a process for coordinating key performance indicator measurements with a source neighbor gNB, according to an example implementation.

[0021] Figure 8 is a block diagram of a node or wireless station (e.g., a base station / access point, a relay node, or a mobile station / user equipment) according to an example implementation. Specific implementation method

[0022] The principles of the present disclosure will now be described with reference to some exemplary embodiments. It should be understood that these embodiments are described only for illustrative purposes and are intended to assist those skilled in the art in understanding and implementing the present disclosure without implying any limitation on the scope of the present disclosure. The disclosure described herein can be implemented in various ways in addition to the manner described below.

[0023] The terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the example embodiments. As used herein, unless the context clearly indicates otherwise, the singular forms "a," "an," and "the" are intended to include the plural forms as well. It should also be understood that the terms "comprises," "includes," "has," "has," "contains," and / or "comprising," when used herein, specify the presence of stated features, elements, and / or components, etc., but do not preclude the presence or addition of one or more other features, elements, components, and / or combinations thereof.

[0024] Figure 1 is a block diagram of a digital communication system such as wireless network 130 according to an example implementation. Figure 1In a wireless network 130, user devices 131, 132, and 133 (which may also be referred to as mobile stations (MSs) or user equipment (UEs)) may connect to (and communicate with) a base station (BS) 134, which may also be referred to as an access point (AP), an enhanced Node B (eNB), a gNB (which may be a 5G base station), or a network node. At least a portion of the functionality of an access point (AP), a base station (BS), or an (e)NodeB (eNB) may also be performed by any node, server, or host that may be operably coupled to a transceiver (such as a remote radio head). BS (or AP) 134 provides wireless coverage within a cell 136 that includes user devices 131, 132, and 133. Although only three user devices are shown connected or attached to BS 134, any number of user devices or BSs may be provided. BS 134 is also connected to a core network 150 via an interface 151. This is merely one simple example of a wireless network, and other examples may also be used.

[0025] As an example, a user device (user terminal, user equipment (UE)) may refer to a portable computing device including a wireless mobile communication device operating with or without a subscriber identity module (SIM), including but not limited to the following types of devices: mobile station (MS), mobile phone, cellular phone, smart phone, personal digital assistant (PDA), mobile phone, device using a wireless modem (alarm device or measurement device, etc.), laptop and / or touch screen computer, tablet computer, tablet phone, game console, notebook, vehicle and multimedia device. It should be understood that a user device may also be an almost dedicated uplink-only device, an example of which is a camera or video camera that uploads images or video clips to a network.

[0026] In LTE (as an example), the core network 150 may be referred to as an evolved packet core (EPC), which may include a mobility management entity (MME) that may handle or assist in mobility / serving cell changes of user equipment between BSs, one or more gateways that may forward data and control signals between the BSs and a packet data network or the Internet, and other control functions or blocks.

[0027] The various example implementations may be applied to a variety of wireless technologies, wireless networks, such as LTE, LTE-A, 5G (New Radio or NR), cmWave and / or mmWave band networks, or any other wireless network or use case. LTE, 5G, cmWave, and mmWave band networks are provided as illustrative examples only, and the various example implementations may be applied to any wireless technology / wireless network. The various example implementations may also be applied to a variety of different applications, services, or use cases, such as, for example, ultra-reliable low-latency communications (URLLC), Internet of Things (IoT), time-sensitive communications (TSC), enhanced mobile broadband (eMBB), massive machine type communications (MMTC), vehicle-to-vehicle (V2V), vehicle-to-device, etc. Each of these use cases or UE types may have its own set of requirements.

[0028] Enabling, coordinating, and distributing machine learning (ML)-assisted functions across NG-RAN and 5GC entities is a topic of ongoing research and development. Future NR specifications may feature study items on ML-enabled radio resource management (RRM) functions, including ML assistance in UEs. An example use case is open-loop power control (OLPC) parameter optimization based on reinforcement learning (RL) to achieve various goals, such as throughput or energy efficiency (EE) enhancement, but the framework is general and can be applied to a variety of similar RRM use cases that utilize reinforcement learning optimization.

[0029] There are multiple examples of ML-based algorithms for NG-RAN that provide various RRM improvements. Since many of these ML-based RRM algorithms, such as packet scheduling, link adaptation, and beam management, are executed and distributed on the network side, the corresponding ML / RL agents are most likely deployed at each gNB to capture individual cell performance. In this context, the resulting problem is the execution of multi-agent independent learning mechanisms, which can potentially lead to undesirable radio interference, conflicting optimization objectives, and QoS degradation. One example involves optimizing UE transmission parameter settings for UL OLPC for different UE clusters in a serving cell. However, such RL-based algorithms are executed per cell, with each gNB acting as an independent learning agent, jointly exploring and exploiting operations on the same timescale. RL-based algorithms also generate conflicts during the warm-up / exploration phase, resulting in more random actions taken by each agent, which leads to undesirable degradation in cell throughput. This is primarily due to the unknown impact of each individual action on neighboring cells and agents.

[0030] Compared to conventional techniques that perform reinforcement learning that causes interference with a critical neighbor gNB, the improved technique includes: during a configuration exchange, after a critical neighbor gNB is identified by a source gNB, the source gNB identifying a key performance indicator (KPI) and an interpretation of a bit string indicator as a quantified change in the KPI, sending the bit string indicator by the critical neighbor gNB to the source gNB to reflect the change in the cell KPI due to a previous action by the source gNB, measuring the KPI in the source cell to generate its own bit string, generating a combined bit string based on the received bit string and the generated bit string, and taking action based on the combined bit string.

[0031] The improved technique allows the RL agent to operate at a cell to optimize the KPI of that cell without causing undue interference to the KPIs of neighboring cells.

[0032] This paper discloses an inter-gNB coordinated exploration mechanism for instances of the same ML / RL-based algorithm running in each gNB to achieve joint optimization of UE radio parameters across all gNBs. A compact form of bit-string (n-bit) information exchange between gNBs is used to explicitly reflect changes in cell KPIs (e.g., throughput) in other gNBs and also to determine the state and reward function of the RL agent algorithm running in each gNB. Key RL algorithm parameters (such as state space, action space, reward function, Q-table, etc.) are specifically designed for use with bit-string metrics.

[0033] To achieve this, there is a new inter-gNB coordination mechanism that considers a joint exploration and training phase between the source gNB (i.e., gNB0) and a key neighbor gNB (i.e., gNB1) running a specific ML-based RRM algorithm for assistance and coordination, where the optimal adjustment for UE radio parameter settings (e.g., OLPC parameters) is derived at the end of each joint exploration phase for each UE served by gNB0. The joint coordinated exploration between gNBs operates in the following main steps.

[0034] Enables execution of a neighbor selection algorithm that relies on clustering of UEs in a given cell. The goal is to identify key neighbor gNBs for coordination so that every action taken by the source gNB has a direct impact on the associated neighbor cells. The configuration message defines the general parameters and scope of the coordination mechanism, which includes the exchange of UE clustering information between gNBs, the selection of coordination-based parameters (i.e., coordination time step), the selection of radio KPIs / QoS to be measured, and the interpretation of bit string metrics (quantized change (delta) in the estimated cell KPIs / QoS). The coordination signaling configuration exchange between the source gNB and the selected relevant target gNB is assumed to occur via XnAP. The source gNB sends a coordinated discovery request to the key neighbor gNB, which is then performed upon receiving an ACK response at the source gNB from the key neighbor gNB. Instead of sending the full KPI information, a first bit string indicator is sent from the key neighbor gNB(s) to the source gNB to reflect the quantitative change (delta) in the estimated cell KPI / QoS. o This is an indicator of the quantified value of a KPI, or an indication of the change (positive or negative) of a KPI relative to a gNB reference / target KPI, or below / above a threshold KPI, by a specific step size. A second bit string indicator is derived at the source gNB to reflect the quantitative change (delta) in the estimated cell KPI / QoS. o This indicator is derived in the same way as the indicator received from the neighboring gNB. A combined bit string indicator is generated at the source gNB from the first bit string indicator and the second bit string indicator to create a joint indicator to be used in the ML-enabled algorithm executed in the source gNB. For example, the joint metric can be used to determine the RL state and calculate the reward value for various actions on this state space. In addition, other RL-specific properties such as action space or Q-table entries can be specially designed by using the combined bit string metric. The Coordination Exploration Phase is terminated by a Coordination Termination Request from the source gNB and ACK responses from the neighboring gNB(s).

[0035] The main improvement lies in a coordination mechanism designed specifically for ML / RL-based agents executing independently in gNBs. It relies on a novel coordination signaling structure between coordinated neighboring gNBs, which utilizes bit string indicators to reflect the estimated KPI / QoS changes in both the serving cell and key neighboring cells. The improved technique is applicable to coordinated RL exploration mechanisms between gNBs for various use cases, such as UL PC parameter optimization, beam selection, coordinated scheduling, etc.

[0036] Figure 2 is a flow chart illustrating the process of coordinating key performance indicator measurements with selected key neighbor gNBs.

[0037] At 200, the algorithm begins and the RL entity is initialized at each source gNB (gNB0) for exploration. gNB0 configures and selects the expected inputs for the neighbor selection algorithm, i.e., UE measurement reports (including RSRP), mobility pattern, traffic type, etc., and determines whether to create multiple clusters in the cell.

[0038] At 201, according to the configuration at 200, gNB0 outputs a UE cluster ID, which is then converted into a key neighboring gNB ID for coordination and assistance.

[0039] At 202, instead of exchanging complete KPI information (e.g., simply quantifying cell throughput), the following information elements are included in the inter-gNB configuration message. 202a. Select the radio KPI / QoS to be measured, and 202b. Bit string indicator interpretation, ie, the quantitative change (increment) in the measured cell KPI / QoS.

[0040] At 203 , the normal exploration and development phase specific to RL-based implementation of radio parameter adjustment for UEs in different clusters is run. • 203a. The coordinated exploration phase may be triggered based on several events: a) periodic timer, b) UE radio / traffic condition changes, c) cluster information being outdated, eg due to mobility. 203b. Based on the information delivered in 203a, check whether it is time to run a coordinated exploration. If not, return to 203a. Otherwise, move forward to 205.

[0041] At 204, the coordinated signaling elements of the exploration execution phase are executed. The inter-gNB RL coordination algorithm is run sequentially or concurrently between the source gNB and key neighbor gNBs. To improve the efficiency of signaling and subsequent steps applied to the local RL agent, a three-step process is implemented. 204a. A first bit string indicator is sent from the key neighbor gNB(s) to the source gNB to reflect the quantitative change (delta) in the estimated cell KPI / QoS. 204b. A second bit string indicator is derived at the source gNB to reflect the quantitative change (delta) in the estimated cell KPI / QoS. 204c. The combined bit string indicator is combined at the source gNB based on the first bit string indicator and the second bit string indicator to create a joint indicator to be used in the ML-enabled algorithm executed in the source gNB.

[0042] At 205 , a termination condition check is performed for the coordinated exploration. If not, return to step 204 a. Otherwise, proceed to step 206 .

[0043] At 206, outputs / actions are generated from the ML-enabled function, such as a) an optimal action command containing the UL PC parameters determined at gNB0, and b) the achieved KPI / QoS requirements.

[0044] The RL-based algorithm is used in normal execution mode without coordination and in inter-gNB coordination mode at, for example, 203 and 204, respectively. Here, the explicit algorithm design is described below only for the UL-TPC use case at 204 in the coordination scenario between the source gNB and the target gNB.

[0045] Figure 3 is a flowchart illustrating the process of exploration and exploitation based on reinforcement learning. Figure 3 The following figure shows the main algorithm flow for the exploration mode with a higher exploration probability and the exploitation mode with a lower exploration probability. The two execution modes of the RL-based inter-gNB coordination algorithm are distinguished by a warm-up condition, which can be defined by i) a specific time period, ii) a time difference error of ∈, and iii) an update of the Q-table entries. During this phase, the agent will choose more random actions to explore the environment. This further results in updating a large portion of the Q-table entries, which then becomes the starting point for the improved learning algorithm run after the exploration / warm-up period. Subsequently, during the normal exploitation phase, system performance can be quite stable and optimal, as more actions are transferred from the learned knowledge recorded via the Q-table.

[0046] At 301 , the algorithm begins.

[0047] At 302 , RL entities such as hyperparameters, state space, action space, and Q-table are initialized.

[0048] At 303, the RL-based inter-gNB coordination algorithm runs in warm-up / exploration mode. Here, the RL loop is executed with a high exploration probability.

[0049] At 304 , it is determined whether the preheating condition is met. If so, the algorithm continues to 305 . If not, the algorithm returns to 303 .

[0050] At 305, the RL-based inter-gNB coordination algorithm runs in normal execution / utilization mode. The RL loop is executed with low exploration probability.

[0051] Figure 4 is a flow chart illustrating the process of inter-gNB coordination steps based on reinforcement learning for open loop power control (OLPC). Figure 4 In

[15] , we explain an RL-based inter-gNB coordination algorithm applied to the OLPC use case. The modeling assumptions for the MDP, including the state space, action space, and reward function, are as follows. Briefly, i) the state space is constructed based on the combined bit string information at the source gNB and neighboring gNBs, ii) the action space for OLPC is a finite set containing a constant adaptive variable Δ (in dB) used to adjust the P0 value, and iii) the reward function is a weighted sum of the cell throughputs from the two cells.

[0052] At 401 , the RL agent determines an action a(t) for the configured UE.

[0053] At 402, the RL agent enters the RL environment. At 402a, the UE receives an updated action a(t) from the RL agent. The UE then takes the action a(t) in the current time period t. RL The updated ULPC parameter P0 is applied internally. At 402b, gNB0 calculates and buffers the experienced cell throughput and generates a first bitstream indicator. At 402c, gNB1 calculates and buffers the experienced cell throughput and generates the first bitstream indicator, which is then transferred to gNB0.

[0054] At 403, it is determined whether it is time for the next RL iteration, for example, whether t=t next If yes, the RL agent proceeds to 404; if no, the RL agent returns to 402.

[0055] At 404, the RL agent performs the RL-based inter-gNB coordination algorithm reward calculation. There are two options: 1) a discrete reward based on a bit string metric, or 2) a weighted sum of the cell throughputs from the coordinated cells.

[0056] At 405, the RL agent updates the Q-table for the current state and action. The entries of the Q-table are updated with the discounted reward values.

[0057] At 406, the RL agent executes the RL-based inter-gNB coordination algorithm state transition s′(t). The new state s′(t) is determined based on the combined bit string indicator.

[0058] In 407, t next Increment t RL , and the RL agent returns to 401.

[0059] For RL modeling, the RL entity follows the form of a Markov decision process (MDP), i.e., a state space, an action space, a reward function, and a Q-table. Without loss of generality, in the following, the source gNB is denoted as gNB0, and the detected critical neighboring gNB is denoted as gNB1. Environment: Radio cellular network. Agent: A single agent at gNB0 controls the power settings of all served UEs in a cell or a cluster of UEs in a cell. Goal: Optimize its own cell performance, such as throughput, spectral efficiency (SE), and energy efficiency (EE), with minimal impact on KPIs / QoS at gNB1. • Number of cells used for coordination: all M neighboring cells. State space: The execution of each action may result in a state transition, so the combined binary indicators at gNB0 can better define the state space S = {00, 01, 10, 11}. Action space, modeled according to use cases, such as: o (OLPC is enabled): A constant adaptive variable Δ in dB is used to adjust the P0 value (refer to NC322706). An example of a complete action space may be A={-3Δ,-Δ,0,+Δ,+3Δ}. o (CLPC enabled): Closed-loop power adjustment δ(v) is performed according to the 2-bit TPC command with field values ​​0, 1, 2, 3, which results in a complete action space A = {-1dB, 0dB, +1dB, +3dB}. Reward function: The reward function can be discrete, designed based on a combined binary metric at gNB0, or continuous by using the weighted sum of the cell throughputs Tput0, Tput1 of gNB0 and gNB1. Option 1: Where τ is a positive constant with τ > 0, and S > 1 is a scaling factor such that '00' represents the worst case in the overall reward function. Note that this formula allows for linking state indices to different reward values. However, reward function design is not limited to this case. Option 2: R(s,a) = ω × Tput0 + (1-ω) × Tput1 Q-table: A single Q-table is used and divided into M different blocks corresponding to different neighboring cells used for coordination. The size / dimension of each block of the Q-table is determined by the state space and action space.

[0060] Figure 5 is a signaling diagram illustrating an example process 500 for coordinated exploration.

[0061] From 501 to 502, the preparation phase occurs, including executing the UE clustering-based neighbor selection algorithm for coordination exploration. At 501, the UE receives a measurement report from the UE. As a result, at 502, gNB0 assigns the corresponding cluster ID to the UE, and the key neighbor gNB, gNB1, is known to gNB0 for coordination.

[0062] At 503 to 504, a coordination signaling configuration exchange occurs between the source gNB and the target gNB. Use Xn-based signaling between the source gNB (i.e., gNB0) and the target selected key neighbor gNB (i.e., gNB1). Configuration signals include but are not limited to the following key information elements: o Information about the current cluster ID C0 at gNB0 (whose associated key neighbor gNB is gNB1), and information about the current cluster ID C1 at gNB1 (whose key neighbor gNB is gNB0). With this information, only the RL agent at gNB0 controls the radio parameter adaptation for C0, while the RL agent at gNB1 controlling C1 remains in idle mode for assistance and coordination. o The coordination time period is chosen so that coordinated exploration only occurs within a predefined time window. Otherwise, the agent executes the normal RL execution mode. o Selection of radio KPIs to measure between gNBs, such as cell throughput, spectral efficiency, energy efficiency, latency, etc. o Selection of a bit string indication message to measure KPI variation, a binary indicator, or an extended bit string indicator (as explained in the Extensions section - Bit String). In addition, when the bit string indication message is selected, related parameters need to be configured, such as the threshold(s) for throughput variation margin in kpbs. o Selection of RL-based algorithm ID with related MDP entities including state space, action space, reward function, Q-table, etc. The design of RL-based algorithm is related to the selected indicator message. οOther RL-based RRM coordination mechanisms.

[0063] At 505-508, the RL-based algorithm initialization phase occurs at the source gNB. At 505, after receiving the gNB coordination signaling configuration ACK from gNB1, the cell KPI / QoS is measured at gNB0 as a reference case for evaluating the own cell performance changes in the subsequent coordination exploration step. • At 506, the RL entity including the Q-table is initialized for exploration. At 507 to 508, a coordinated exploration signaling exchange occurs between gNB0 and gNB1. Then, when an ACK is received at gNB0 from gNB1, the joint exploration begins.

[0064] At 509 to 522, a coordinated discovery execution phase occurs. At 509 to 513, exploration begins at gNB0: the UE adjusts its current radio transmission parameters based on the action commands sent by gNB0, which in turn will cause interference to gNB1. Based on the actions taken at gNB0, gNB1 measures its estimated cell KPI / QoS, i.e., cell throughput. At 514, gNB1 sends a sequence of first bit string indicators to reflect the change in its cell KPI / QoS (i.e., cell throughput) due to gNB0's latest action. For example, a binary indicator (0 / 1) is used as a baseline reference case. When gNB1's cell throughput increases, gNB1 sends a binary indicator of 1 to gNB0. Otherwise, a binary indicator of 0 is sent to gNB0. In general, a bit string indicator can have more than a single bit. At 515 to 516, gNB0 measures its cell KPI / QoS and generates a sequence of second bit string indicators to reflect the change in its cell KPI / QoS due to its last action (i.e., cell throughput). For example, a binary indicator (0 / 1) is used as a baseline reference case. When gNB0's cell throughput increases, gNB0 generates a binary indicator of 1, and vice versa. At 517, a reward function is calculated based on the combined 2n-bit bit string metrics from both gNB1 and gNB0. For example, using a binary metric (0 / 1) as a baseline reference case, four state indices are generated in the RL state space: 00, 01, 10, and 11. At 518, the Q table is updated and the new action is determined. The update rule follows the Bellman equation Where α is the learning rate and γ is the discount factor. At 519-521, the UE adjusts its current radio transmission parameters based on the action command sent by gNB0, which in turn will cause interference to gNB1. At 522, the coordinated exploration is repeated.

[0065] From 523 to 524, there is a coordinated exploration termination signaling request and response between gNB0 and gNB1. When gNB0 receives an ACK from gNB1, the joint exploration stops.

[0066] The following discussion provides some details about the neighbor selection algorithm described above. The algorithm runs at the source gNB (i.e., gNB0) and its goal is to verify a list of potential neighbor gNBs that can have an explicit impact on the different UE groups / clusters served by gNB0. First, the algorithm starts. The source gNB collects relevant input data to drive the clustering method. In the example below, Slowly fading RSRP is collected from UE measurement reports to build clusters in cells. Based on RSRP measurements, create UE clusters in the source gNB, where each cluster corresponds to a key neighbor gNB for coordination. In this step, multiple UE clusters (each with at least one UE) are created based on the following example criteria. Reference option (strong RSRP rule): UEs belong to a cluster if they share a key neighbor cell, as determined by the strongest neighbor cell RSRP measurement. By doing so, the creation of each UE cluster served by gNB0 will result in the highest interference correlation per unique cooperating gNB. UE clusters are ranked in gNB0 based on predefined criteria that rank key neighbor gNBs for joint exploration. This ranking method generates a ranked list of key neighbor gNBs. Possible criteria for ranking UE clusters may be: a) the number of UEs in each cluster, b) the average serving cell RSRP value for each cluster, c) the average strongest neighbor cell RSRP value for each cluster, d) the average RSRPdiff value for each cluster, etc. o UE clusters containing fewer UEs are discarded. Or given the minimum number of UEs in each cluster is n min , for the number of UEs n <n min This step aims to reduce the potential list of critical neighboring gNBs to achieve an acceptable coordination complexity. A list of i) UE cluster IDs served by gNB0 and ii) selected key neighboring gNB IDs is generated for the following coordination exploration phase.

[0067] Example 1-1: Figure 6 6 is a flow chart illustrating a process for coordinating key performance indicator measurements with a selected key neighbor gNB. Operation 610 includes, by a source network node controlling a source cell, identifying a key neighbor network node controlling a key neighbor cell based on a cluster of user equipment connected to the source network node, where a change in a radio parameter value for the cluster of user equipment affects the key neighbor node. Operation 620 includes, by the source network node, interpreting a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor. Operation 630 includes, by the source network node, performing a first action that results in a change in a radio transmission parameter value for the user equipment. Operation 640 includes, by the source network node, receiving a first value of the bit string indicator from the key neighbor network node, the first value indicating a change in the key performance indicator at the key neighbor cell. Operation 650 includes, by the source network node, generating a second value of the bit string indicator based on measurements of the key performance indicator at the source cell.

[0068] Example 1-2: An example implementation according to Example 1-1, wherein the second value of the bit string pointer is obtained in the same manner as the first value of the bit string pointer.

[0069] Example 1-3: According to the example implementation of Examples 1-1 to 1-2, it also includes: the source network node combining the first value of the bit string indicator and the second value of the bit string indicator to generate a combined bit string indicator; and the source network node performing a second action based on the combined bit string indicator.

[0070] Example 1-4: An implementation according to the example implementation of Example 1-3, wherein performing the second action includes identifying the combined bit string indicator as a state in a state space of a reinforcement learning algorithm; generating a reward for the reinforcement learning algorithm based on the state; and identifying a second action from the reinforcement learning algorithm based on the reward function.

[0071] Example 1-5: An implementation according to the example implementation of Example 1-4, wherein the first value of the bit string indicator is a first set of n bits, the second value of the bit string indicator is a second set of n bits, the combined bit string indicator includes the set of n bits, and if the set of n bits is all ones, then the reward is a positive integer, and if the set of n bits is all zeros, then the reward is a negative integer. Note that this is an example, and other rewards are possible.

[0072] Example 1-6: The example implementation of Examples 1-1 to 1-5, wherein identifying the key neighbor network node controlling the key neighbor cell comprises identifying a cluster of user equipment connected to the source network node based on neighbor cell reference signal received power measurements.

[0073] Example 1-7: The example implementation of Examples 1-1 to 1-6, wherein the key performance indicator includes at least one of cell throughput, spectral efficiency, energy efficiency, or latency.

[0074] Example 1-8: An apparatus comprising means for performing the method of any of Examples 1-1 to 1-7.

[0075] Example 1-9: A computer program product comprising a non-transitory computer-readable storage medium and storing executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to perform the method of any one of Examples 1-1 to 1-7.

[0076] Example 2-1: Figure 7 7 is a flow chart illustrating a process 700 for compressing channel state information. Operation 710 includes receiving, by a key neighbor network node controlling a key neighbor cell, a configuration message from a source node controlling a source cell, the configuration message identifying an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell. Operation 720 includes receiving, by the key neighbor network node, an indication of a change in a radio transmission parameter value for a cluster of user equipment. Operation 730 includes sending, by the key neighbor node, a first bit string indicator indicating a change in the key performance indicator at the key neighbor cell to the source node.

[0077] Example 2-2: The example implementation of Example 2-1, wherein the key performance indicator includes at least one of cell throughput, spectral efficiency, energy efficiency, or latency.

[0078] Example 2-3: An apparatus comprising means for performing the method of any of Examples 2-1 to 2-2.

[0079] Example 2-4: A computer program product includes a non-transitory computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to perform the method of any one of Examples 2-1 to 2-2.

[0080] List of example abbreviations: gNB fifth generation Node B ML Machine Learning NG-RAN Next Generation Radio Access Network OLPC Open-Loop Power Control PUSCH Physical Uplink Shared Channel RL reinforcement learning QL Q Learning QoS Quality of Service SON self-organizing network TPC Transmit Power Control UE User Equipment UL Uplink

[0081] Figure 8 8 is a block diagram of a wireless station (e.g., an AP, BS, e / gNB, NB-IoT UE, UE, or user equipment) 800 according to an example implementation. The wireless station 800 may include, for example, one or more (two in this example) RF (radio frequency) or wireless transceivers 802A, 802B, each of which includes a transmitter for transmitting signals (or data) and a receiver for receiving signals (or data). The wireless station also includes a processor or control unit / entity (controller) 804 and a memory 906, where the processor 804 is used to execute instructions or software and control the transmission and reception of signals, and the memory 906 is used to store data and / or instructions.

[0082] The processor 804 may also make decisions or determinations, generate time slots, subframes, packets or messages for transmission, decode received time slots, subframes, packets or messages for further processing, and other tasks or functions described herein. For example, the processor 604, which may be a baseband processor, may generate messages, packets, frames, or other signals for transmission via the wireless transceiver 802 (802A or 802B). The processor 804 may control the transmission of signals or messages over a wireless network and may control the reception of signals or messages, etc., via a wireless network (e.g., after down-conversion by the wireless transceiver 802). The processor 804 may be programmable and capable of executing software or other instructions stored in a memory or other computer medium to perform the various tasks and functions described above, such as one or more of the tasks or methods described above. The processor 804 may be (or may include), for example, hardware, programmable logic, a programmable processor executing software or firmware, and / or any combination thereof. For example, using other terminology, the processor 804 and the transceiver 802 (802A or 802B) together may be considered a wireless transmitter / receiver system.

[0083] In addition, reference Figure 8 , the controller (or processor) 808 can execute software and instructions and can provide overall control for the station 800 and can Figure 8 Other systems not shown provide controls, such as controlling input / output devices (e.g., display, keypad), and / or may execute software for one or more applications that may be provided on wireless station 800, such as, for example, an email program, audio / video applications, a word processor, a voice over IP application, or other applications or software.

[0084] Additionally, a storage medium may be provided that includes stored instructions that, when executed by a controller or processor, may cause the processor 804 or other controller or processor to perform one or more of the functions or tasks described above.

[0085] According to another example implementation, the RF or wireless transceiver(s) 802A / 802B may receive signals or data and / or transmit or send signals or data. The processor 804 (and possibly the transceiver 802A / 802B) may control the RF or wireless transceiver 802A or 802B to receive, send, broadcast, or transmit signals or data.

[0086] However, the embodiments are not limited to the systems given as examples, and those skilled in the art may apply the solutions to other communication systems. Another example of a suitable communication system is the 5G concept. 5G uses multiple-input multiple-output (MIMO) antennas, many more base stations or nodes than LTE (the so-called small cell concept), including macro sites that operate in collaboration with smaller sites, and may also employ various radio technologies to achieve better coverage and enhanced data rates.

[0087] It will be appreciated that future networks will likely utilize Network Function Virtualization (NFV), a network architecture concept that proposes virtualizing network node functions into "building blocks" or entities that can be operatively connected or linked together to provide services. A virtualized network function (VNF) may include one or more virtual machines that use standard or general-purpose servers rather than custom hardware to run computer program code. Cloud computing or data storage may also be utilized. In radio communications, this may mean that node operations may be performed at least in part in a server, host, or node that is operatively coupled to a remote radio head. Node operations may also be distributed across multiple servers, nodes, or hosts. It will also be appreciated that the division of labor between core network operations and base station operations may be different from that of LTE or may not even exist.

[0088] Implementations of the various techniques described herein may be implemented in digital electronic circuit systems or computer hardware, firmware, software, or a combination thereof. Implementations may be implemented as computer program products, i.e., computer programs tangibly embodied in an information carrier (e.g., in a machine-readable storage device or in a propagated signal) for execution by a data processing apparatus (e.g., a programmable processor, a computer, or multiple computers) or for controlling the operation of a data processing apparatus. Implementations may also be provided on a computer-readable medium or a computer-readable storage medium, which may be a non-transitory medium. Implementations of the various techniques may also include implementations provided via transient signals or media, and / or programs and / or software implementations that may be downloaded via the Internet or other (multiple) networks (wired and / or wireless networks). In addition, implementations may be provided via machine type communications (MTC) and also via the Internet of Things (IoT).

[0089] A computer program may be in source code form, object code form, or some intermediate form, and it may be stored on some carrier, distribution medium, or computer-readable medium, which may be any entity or device capable of carrying the program. Such carriers include, for example, recording media, computer memory, read-only memory, optical and / or electrical carrier signals, telecommunications signals, and software distribution packages. Depending on the required processing power, a computer program may be executed on a single electronic digital computer or distributed among multiple computers.

[0090] In addition, implementations of the various techniques described herein may use cyber-physical systems (CPS) (systems of collaborative computing elements that control physical entities). CPS can implement and utilize a large number of interconnected ICT devices (sensors, actuators, processor microcontrollers, etc.) embedded in physical objects at different locations. Mobile cyber-physical systems (where the physical systems involved have inherent mobility) are a subcategory of cyber-physical systems. Examples of mobile physical systems include mobile robots and electronic devices transported by humans or animals. The growth in the popularity of smartphones has increased interest in the field of mobile cyber-physical systems. Therefore, various embodiments of the techniques described herein may be provided via one or more of these techniques.

[0091] Computer programs, such as the computer program(s) described above, may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for a computing environment or a portion thereof. A computer program may be deployed to execute on one or more computers at one site, or distributed across multiple sites and interconnected by a communication network.

[0092] The method steps may be performed by one or more programmable processors executing a computer program or portion of a computer program to perform functions by operating on input data and generating output. The method steps may also be performed by an apparatus, and the apparatus may be implemented as special-purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application-specific integrated circuit).

[0093] For example, processors suitable for executing a computer program include both general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer, chip, or chipset. Typically, the processor will receive instructions and data from a read-only memory or a random access memory, or both. Elements of a computer may include at least one processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may also include or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices (e.g., magnetic, magneto-optical, or optical disks) for storing data. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, including, for example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into special-purpose logic circuitry.

[0094] To provide for interaction with a user, implementations may be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or liquid crystal display (LCD) monitor) for displaying information to the user and a user interface (e.g., a keyboard and a pointing device, such as a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide for interaction with the user; for example, feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input.

[0095] The implementation can be implemented in a computing system that includes a back-end component, such as a data server, or a middleware component, such as an application server, or a front-end component, such as a client computer with a graphical user interface or a web browser through which a user can interact with the implementation, or any combination of such back-end, middleware, or front-end components. The components can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0096] Although certain features of the described implementations have been shown as described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art. Therefore, it should be understood that the appended claims are intended to cover all modifications and changes that fall within the contemplation of the various embodiments.

Claims

1. A device comprising: at least one processor; as well as at least one memory including computer program code; The at least one memory and the computer program code are configured to cause the apparatus to at least: identifying, by a source network node controlling a source cell, a key neighbor network node controlling a key neighbor cell based on a cluster of user equipment connected to the source network node, a change in a radio transmission parameter value for the cluster of user equipment affecting the key neighbor node; identifying, by the source network node, an interpretation of a bit string indicator for indicating a change in a key performance indicator at the source cell and the key neighbor cell; performing, by the source network node, a first action resulting in a change of the radio transmission parameter value for the cluster of user equipment; receiving, by the source network node, a first value of the bit string indicator from the key neighbor network node, the first value indicating a change in the key performance indicator at the key neighbor cell; as well as A second value of the bit string indicator is generated by the source network node based on a measurement of the key performance indicator at the source cell. 2 . The apparatus according to claim 1 , wherein the second value of the bit string indicator is obtained in the same manner as the first value of the bit string indicator.

3. The apparatus of claim 1 , wherein the at least one memory and the computer program code are further configured to cause the apparatus to at least: combining, by the source network node, the first value of the bit string indicator and the second value of the bit string indicator to produce a combined bit string indicator; and A second action is performed by the source network node based on the combined bit string indicator.

4. The apparatus of claim 3, wherein the at least one memory and the computer program code configured to perform the second action are further configured to cause the apparatus to at least: identifying the combined bit string indicator as a state in a state space of a reinforcement learning algorithm; generating a reward for the reinforcement learning algorithm based on the state; as well as The second action is identified from the reinforcement learning algorithm based on the reward.

5. The apparatus of claim 4 , wherein the first value of the bit string indicator is a first set of n bits, the second value of the bit string indicator is a second set of n bits, the combined bit string indicator comprises a set of n bits, and the reward is a positive integer if the set of n bits is all ones and a negative integer if the set of n bits is all zeros.

6. The apparatus of claim 1 , wherein the at least one memory and the computer program code configured to identify the key neighbor network node controlling the key neighbor cell are further configured to cause the apparatus to at least: The cluster of user equipment connected to the source network node is identified based on neighbor cell reference signal received power measurements.

7. The apparatus according to claim 1, wherein the key performance indicator comprises at least one of cell throughput, spectrum efficiency, energy efficiency or latency.

8. A method comprising: identifying, by a source network node controlling a source cell, a key neighbor network node controlling a key neighbor cell based on a cluster of user equipment connected to the source network node, wherein a change in a radio transmission parameter value for the cluster of user equipment affects the key neighbor node; identifying, by the source network node, an interpretation of a bit string indicator for indicating a change in a key performance indicator at the source cell and the key neighbor cell; performing, by the source network node, a first action resulting in a change of the radio transmission parameter value for the user equipment; receiving, by the source network node, a first value of the bit string indicator from the key neighbor network node, the first value indicating a change in the key performance indicator at the key neighbor cell; as well as A second value of the bit string indicator is generated by the source network node based on a measurement of the key performance indicator at the source cell.

9. The method of claim 8, wherein the second value of the bit string indicator is obtained in the same manner as the first value of the bit string indicator.

10. The method according to claim 8, further comprising: combining, by the source network node, the first value of the bit string indicator and the second value of the bit string indicator to produce a combined bit string indicator; as well as A second action is performed by the source network node based on the combined bit string indicator.

11. The method of claim 10, wherein performing the second action comprises: identifying the combined bit string indicator as a state in a state space of a reinforcement learning algorithm; generating a reward for the reinforcement learning algorithm based on the state; as well as The second action is identified from the reinforcement learning algorithm based on the reward.

12. The method of claim 11 , wherein the first value of the bit string indicator is a first set of n bits, the second value of the bit string indicator is a second set of n bits, the combined bit string indicator comprises a set of n bits, and the reward is a positive integer if the set of n bits is all ones and a negative integer if the set of n bits is all zeros.

13. The method of claim 8, wherein identifying the key neighbor network node that controls the key neighbor cell comprises: The cluster of user equipment connected to the source network node is identified based on neighbor cell reference signal received power measurements.

14. The method according to claim 8, wherein the key performance indicator comprises at least one of cell throughput, spectrum efficiency, energy efficiency or latency.

15. An apparatus comprising: at least one processor; as well as at least one memory including computer program code; The at least one memory and the computer program code are configured to cause the apparatus to at least: receiving, by a key neighbor network node controlling a key neighbor cell, a configuration message from a source node controlling a source cell, the configuration message identifying an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell; receiving, by the critical neighbor network node, an indication of a change in a radio transmission parameter value for a cluster of user equipment; as well as A first bit string indicator indicating a change in the key performance indicator at the key neighbor cell is sent by the key neighbor node to the source node.

16. The apparatus according to claim 15, wherein the key performance indicator comprises at least one of cell throughput, spectrum efficiency, energy efficiency or latency.

17. A method comprising: receiving, by a key neighbor network node controlling a key neighbor cell, a configuration message from a source node controlling a source cell, the configuration message identifying an interpretation of a bit string indicator indicating a change in a key performance indicator at the source cell and the key neighbor cell; receiving, by the critical neighbor network node, an indication of a change in a radio transmission parameter value for a cluster of user equipment; as well as A first bit string indicator indicating a change in the key performance indicator at the key neighbor cell is sent by the key neighbor node to the source node.

18. The method according to claim 17, wherein the key performance indicator comprises at least one of cell throughput, spectrum efficiency, energy efficiency or latency. 19 . A computer program product comprising a non-transitory computer-readable storage medium and storing executable code, the executable code being configured to cause at least one data processing device to perform the method according to claim 8 when executed by the at least one data processing device.

20. An apparatus comprising means for performing the method according to claim 8.

Images