Antenna panel switching based on machine learning

Abstract

The present disclosure relates to machine learning-based antenna panel switching. A method is disclosed, including selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm. Determining a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received on the selected antenna panel. Sending or receiving a second signal via the selected antenna panel if the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels.

Description

Technical Field

[0001] The following exemplary embodiments relate to wireless communications. Background Art

[0002] Since resources are limited, it is desirable to optimize the use of network resources. Terminal devices may be utilized to enable better utilization of resources and to give users of the terminal devices an enhanced user experience. Summary of the invention

[0003] The scope of protection sought for the various exemplary embodiments is given by the independent claims. Exemplary embodiments and features described in this specification that do not fall within the scope of the independent claims, if any, are to be construed as examples useful for understanding the various exemplary embodiments.

[0004] According to one aspect, a device is provided, comprising at least one processor and at least one memory comprising computer program code, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the device to: select an antenna panel from a plurality of antenna panels using a machine learning algorithm; determine a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received on the selected antenna panel; and send or receive a second signal via the selected antenna panel if the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels.

[0005] According to another aspect, a device is provided, comprising: a device for selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm; determining a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received on the selected antenna panel; and sending or receiving a second signal via the selected antenna panel if the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels.

[0006] According to another aspect, a system is provided that includes at least a terminal device and a base station, wherein the terminal device is configured to: select an antenna panel from a plurality of antenna panels using a machine learning algorithm; receive one or more first signals from the base station via the selected antenna panel; determine a first long-term reward value associated with the selected antenna panel based at least in part on the one or more first signals received on the selected antenna panel; and send or receive a second signal via the selected antenna panel when the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels; and wherein the base station is configured to: send one or more first signals to the terminal device; and receive a second signal from or send a second signal to the terminal device.

[0007] According to another aspect, a system is provided that includes at least a terminal device and a base station, wherein the terminal device includes a device for: selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm; receiving one or more first signals from the base station via the selected antenna panel; determining a first long-term reward value associated with the selected antenna panel based at least in part on the one or more first signals received on the selected antenna panel; and sending or receiving a second signal via the selected antenna panel when the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels; and wherein the base station includes a device for: sending the one or more first signals to the terminal device; and receiving a second signal from or sending a second signal to the terminal device.

[0008] According to another aspect, a method is provided, comprising selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm; determining a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received on the selected antenna panel; and sending or receiving a second signal via the selected antenna panel if the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels.

[0009] According to another aspect, a computer program is provided, comprising instructions for causing a device to perform at least the following: select an antenna panel from a plurality of antenna panels using a machine learning algorithm; determine a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received on the selected antenna panel; and send or receive a second signal via the selected antenna panel when the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels.

[0010] According to another aspect, a computer program is provided, comprising instructions for causing a device to perform at least the following: select an antenna panel from a plurality of antenna panels using a machine learning algorithm; determine a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received on the selected antenna panel; and send or receive a second signal via the selected antenna panel when the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels.

[0011] According to another aspect, a non-transitory computer-readable medium is provided, comprising program instructions for causing a device to perform at least the following: select an antenna panel from a plurality of antenna panels using a machine learning algorithm; determine a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received on the selected antenna panel; and send or receive a second signal via the selected antenna panel if the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Hereinafter, various exemplary embodiments will be described in more detail with reference to the accompanying drawings, in which

[0013] Figure 1 An exemplary embodiment of a cellular communication network is shown;

[0014] Figure 2 shows a signaling diagram according to an exemplary embodiment;

[0015] Figure 3 shows a two-dimensional state space for machine learning according to an exemplary embodiment;

[0016] Figure 4 and Figure 5 shows a flow chart according to some exemplary embodiments;

[0017] Figure 6 An exemplary embodiment with linear UE movement is shown;

[0018] Figure 7 An exemplary embodiment with a rotating UE is shown;

[0019] Figure 8 and 9 An apparatus according to an exemplary embodiment is shown. DETAILED DESCRIPTION

[0020] The following embodiments are exemplary. Although the specification may refer to "one", "an" or "some" embodiments in several places in the text, this does not necessarily mean that each refers to the same embodiment, or that a particular feature applies only to a single embodiment. Individual features of different embodiments may also be combined to provide other embodiments.

[0021] In the following, different exemplary embodiments will be described using a radio access architecture based on Long Term Evolution Advanced (LTE Advanced, LTE-A) or New Radio (NR, 5G) as an example of an access architecture to which the exemplary embodiments may be applied, however the exemplary embodiments are not limited to such an architecture. It is clear to a person skilled in the art that the exemplary embodiments may also be applied to other types of communication networks with suitable devices by appropriately adjusting parameters and procedures. For appropriate Some examples of other options may be Universal Mobile Telecommunications System (UMTS) Radio Access Network (UTRAN or E-UTRAN), Long Term Evolution (LTE, same as E-UTRA), Wireless Local Area Network (WLAN or Wi-Fi), Worldwide Interoperability for Microwave Access (WiMAX), Personal Communications Service (PCS), Wideband Code Division Multiple Access (WCDMA), systems using Ultra-Wideband (UWB) technology, sensor networks, Mobile Ad Hoc Networks (MANET) and Internet Protocol Multimedia Subsystem (IMS) or any combination thereof.

[0022] Figure 1 An example of a simplified system architecture is shown, only some elements and functional entities are shown, which are all logical units, the implementation of which may differ from what is shown. Figure 1 The connections shown are logical connections; the actual physical connections may be different. It will be apparent to those skilled in the art that the system may also include Figure 1 Other functions and structures than those shown.

[0023] However, the exemplary embodiments are not limited to the systems given as examples, but a person skilled in the art may apply the solution to other communication systems having the necessary characteristics.

[0024] Figure 1 The example of FIG. 1 shows a portion of an exemplary radio access network.

[0025] Figure 1 User equipment 100 and 102 are shown, which are configured to be wirelessly connected to an access node (such as (e / g) NodeB) 104 providing a cell on one or more communication channels in the cell. The physical link from the user equipment to the (e / g) NodeB can be called an uplink or a reverse link, and the physical link from the (e / g) NodeB to the user equipment can be called a downlink or a forward link. It should be understood that the (e / g) NodeB or their functions can be implemented by using any node, host, server or access point entity suitable for such use.

[0026] The communication system may include more than one (e / g)NodeB, in which case the (e / g)NodeBs may also be configured to communicate with each other via wired or wireless links designed for this purpose. These links may be used for signaling purposes. The (e / g)NodeB may be a computing device configured to control the radio resources of the communication system to which it is coupled. The NodeB may also be referred to as a base station, an access point or any other type of interface device including a relay station capable of operating in a wireless environment. The (e / g)NodeB may include or be coupled to a transceiver. From the transceiver of the (e / g)NodeB, a connection to an antenna unit may be provided, which establishes a bidirectional radio link to a user equipment. The antenna unit may include multiple antennas or antenna elements. The (e / g)NodeB may also be connected to a core network 110 (CN or next generation core NGC). Depending on the system, the counterpart on the CN side may be a serving gateway (S-GW, routing and forwarding user data packets), a packet data network gateway (P-GW) for providing connectivity of a user equipment (UE) to an external packet data network, or a mobile management entity (MME), etc.

[0027] User equipment (also referred to as UE, user device, user terminal, terminal device, etc.) shows a type of device to which resources on the air interface can be allocated and assigned, and therefore any features described herein using user equipment can be implemented using corresponding devices such as relay nodes. An example of such a relay node can be a layer 3 relay (self-backhaul relay) towards a base station.

[0028] User equipment may refer to portable computing devices including wireless mobile communication devices that operate with or without a subscriber identity module (SIM), including but not limited to the following types of devices: mobile stations (mobile phones), smart phones, personal digital assistants (PDAs), handheld devices, devices using wireless modems (alarms or measuring devices, etc.), laptop and / or touch screen computers, tablet computers, game consoles, notebooks and multimedia devices. It should be understood that user equipment may also be almost exclusively uplink-only devices, examples of which may be cameras or video cameras that load images or video clips to the network. User equipment may also be a device with the ability to operate in an Internet of Things (IoT) network, which is a scenario in which objects can be provided with the ability to transmit data on a network without human-to-human or human-to-computer interaction. User equipment may also use the cloud. In some applications, a user device may include a small portable device with a radio component (such as a watch, headphones or glasses), and may perform computing in the cloud. A user device (or a layer 3 relay node in some exemplary embodiments) may be configured to perform one or more user equipment functions. A user device may also be called a subscriber unit, a mobile station, a remote terminal, an access terminal, a user terminal, a terminal device or User Equipment (UE), to mention just a few names or devices.

[0029] The various techniques described here can also be applied to computer-physical systems (CPS) (systems of cooperating computing elements that control physical entities). CPS can implement and utilize a large number of interconnected ICT devices (sensors, actuators, processors microcontrollers, etc.) embedded in physical objects at different locations. Mobile cyber-physical systems, where the physical systems in question can have inherent mobility, are a subclass of cyber-physical systems. Examples of mobile physical systems include mobile robots and electronic devices transported by humans or animals.

[0030] Furthermore, although the apparatus is described as a single entity, different units, processors and / or memory units may be implemented ( Figure 1 Not all are shown).

[0031] 5G may allow the use of multiple-input-multiple-output (MIMO) antennas, many more base stations or nodes than LTE (the so-called small cell concept), including macro sites that operate in cooperation with smaller stations and employ various radio technologies depending on service needs, usage and / or available spectrum. 5G mobile communications may support a wide range of use cases and related applications, including video streaming, augmented reality, different ways of sharing data and various forms of machine-type applications (such as (massive) machine-type communications (mMTC), including vehicle safety, different sensors and real-time control). 5G can be expected to have multiple radio interfaces, namely sub-6GHz, cmWave and mmWave, and also integrate with existing legacy radio access technologies such as LTE. Integration with LTE may be implemented, at least in the early stages, as a system in which macro coverage may be provided by LTE and 5G radio interface access may be obtained from small cells by aggregation to LTE. In other words, 5G may support inter-RAT operability (e.g. LTE-5G) and inter-RI operability (inter-radio interface operability, e.g. sub-6GHz-cmWave, sub-6GHz-cmWave-mmWave). One of the concepts considered for use in 5G networks could be network slicing, where multiple independent and dedicated virtual subnetworks (network instances) can be created within the same infrastructure to run services with different requirements for latency, reliability, throughput, and mobility.

[0032] The current architecture in LTE networks can be fully distributed in the radio and fully centralized in the core network. Low latency applications and services in 5G may require content to be close to the radio, which leads to local outages and multi-access edge computing (MEC). 5G can enable analysis and knowledge generation at the data source. This approach may require the use of resources that may not be continuously connected to the network such as laptops, smart phones, tablets and sensors. MEC can provide a distributed computing environment for application and service hosting. It can also have the ability to store and process content near cellular users for faster response times. Edge computing can cover a wide variety of technologies such as wireless sensor networks, mobile data acquisition, mobile signature analysis, collaborative distributed peer-to-peer self-organizing networking and processing, and can also be classified as local cloud / fog computing and grid / grid computing, open air computing, mobile edge computing, small clouds, distributed data storage and retrieval, autonomous self-healing networks, remote cloud services, augmented and virtual reality, data caching, Internet of Things (massive connectivity and / or latency critical), critical communications (autonomous vehicles, traffic safety, real-time analysis, time-critical control, health care applications).

[0033] The communication system can also communicate with or utilize services provided by other networks such as the public switched telephone network or the Internet 112. The communication network can also support the use of cloud services, for example, at least a portion of the core network operations can be performed as a cloud service (this is in the Figure 1 114). The communication system may also include a central control entity or the like, providing facilities for networks of different operators to collaborate, for example in spectrum sharing.

[0034] Edge cloud can be brought into the radio access network (RAN) by leveraging network function virtualization (NVF) and software defined networking (SDN). Using edge cloud can mean performing access node operations at least partially in a server, host or node that is operably coupled to a remote radio head or base station including the radio part. Node operations can also be distributed among multiple servers, nodes or hosts. The application of cloud RAN architecture can implement RAN real-time functions performed on the RAN side (in distributed unit DU 104) and non-real-time functions performed in a centralized manner (in centralized unit CU108).

[0035] It should also be understood that the labor distribution between core network operations and base station operations may be different from that of LTE, or even non-existent. Some other technological advances that can be used may be big data and all-IP, which may change the way networks are constructed and managed. 5G (or New Radio, NR) networks may be designed to support multiple hierarchical structures, where MEC servers may be placed between the core and base stations or NodeBs (gNBs). It should be understood that MEC may also be applied to 4G networks.

[0036] 5G can also make use of satellite communications to enhance or supplement the coverage of 5G services, for example by providing backhaul. Possible use cases could be to provide service continuity for machine-to-machine (M2M) or Internet of Things (IoT) devices or for passengers in vehicles, or to ensure service availability for critical communications and future rail / maritime / aeronautical communications. Satellite communications can make use of geosynchronous earth orbit (GEO) satellite systems, but can also make use of low earth orbit (LEO) satellite systems, in particular megaconstellations (systems in which hundreds of (nano) satellites are deployed). Each satellite 106 in a megaconstellation can cover several satellite-enabled network entities creating a terrestrial cell. A terrestrial cell can be created by a terrestrial relay node 104 or by a gNB located on the ground or in a satellite.

[0037] It is obvious to a person skilled in the art that the described system is only an example of a part of a radio access system, and in practice, the system may include multiple (e / g)NodeBs, user equipment may have access to multiple radio cells, and the system may also include other devices, such as physical layer relay nodes or other network elements, etc. In addition, in the geographical area of ​​the radio communication system, multiple different kinds of radio cells and multiple radio cells may be provided. The radio cell may be a macro cell (or umbrella cell), which may be a large cell with a diameter of tens of kilometers, or a smaller cell such as a micro cell, a femto cell or a pico cell. Figure 1 The (e / g)NodeB of a cell can provide any type of these cells. A cellular radio system can be implemented as a multi-layer network including multiple types of cells. In a multi-layer network, one access node can provide one or more types of cells, so multiple (e / g)NodeBs may be required to provide this network structure.

[0038] In order to meet the needs of improving the deployment and performance of communication systems, the concept of "plug and play" (e / g) NodeB can be introduced. In addition to the home (e / g) NodeB (H(e / g) NodeB), the network that can use "plug and play" (e / g) NodeB can include the home NodeB gateway or HNB-GW ( Figure 1 (not shown). An HNB Gateway (HNB-GW) that may be installed within an operator's network may aggregate traffic from a large number of HNBs back to the core network.

[0039] A terminal device, which may also be referred to as a UE, may include multiple antenna panels. However, some UEs may be restricted to having only one antenna panel active at a time for radio reception and / or transmission, because operating multiple simultaneously active antenna panels may increase hardware complexity and / or energy consumption. If only one of multiple antenna panels is allowed to be active at a time, a mechanism for swapping the active antenna panel and performing layer 1, L1, measurements (including L1 filtering) may be required. However, such a mechanism may involve a trade-off between serving cell connectivity performance, such as throughput, latency and / or beam failures, and mobility performance, such as handover failure probability and / or ping-pong events, i.e., when the UE performs a handover and then reconnects directly back to the source cell. L1 may also be referred to as the physical layer. For example, when sending channel state information CSI to the gNB, the UE may provide reference signal received power RSRP measurements at L1.

[0040] Each of the plurality of antenna panels may have a directional radiation pattern pointing in a different spatial direction. The UE may be subject to time-varying rotation and / or movement, and there may be time-varying occurrences of local scattering and / or blocking, such as a hand changing position on the UE. The radiation pattern may refer to the variation of the power of the radio waves radiated by the antenna panel as a function of the direction away from the antenna panel, i.e., the way the antenna panel distributes its energy in space.

[0041] During reception and / or transmission of data with its serving cell, the UE may use the best antenna panel, i.e. the antenna panel corresponding to the highest estimated radio signal quality, e.g. measured as RSRP and / or the highest received signal to interference plus noise ratio SINR. The antenna panel may comprise an antenna array, i.e. a plurality of antenna elements, to which the UE may apply beamforming. Base stations such as gNBs may also be equipped with beamforming, e.g. a beam grid, GOB, and therefore use medium access control, MAC, beam-based tracking. In order to ensure that the UE uses its best antenna panel for reception and / or transmission to its serving cell and / or beam, the UE may also perform measurements on its other antenna panels. For example, the UE may perform measurements on synchronization signal blocks SSB and / or channel state information reference signals CSI-RS sent from a gNB beam belonging to its serving cell.

[0042] However, due to the significant difference in the radiation patterns of the antenna panels, the following may occur: the UE may be scheduled to have a downlink or uplink transmission that is assumed to be received or sent using its best antenna panel, and if the UE is performing measurements on different antenna panels, it may not be able to receive transmissions from its serving cell or send to its serving cell when only one antenna panel is active for radio reception and / or transmission at each moment. This may negatively affect the quality of experience QoE. QoE can be defined as the subjective acceptability of the quality of telecommunication services perceived by the user. UE measurements performed on alternative antenna panels may also be required for mobility actions, and therefore they may not be avoided or delayed too much. When the UE moves from one cell to another, for example in radio resource control RRC connected mode, switching may also involve a change in the best antenna panel. However, if too few or too late measurements are performed on antenna panels pointing to the target cell for possible switching, more radio link failures and / or switching failures may occur.

[0043] For example, as described in 3GPP TS 38.300, a UE may be configured to perform measurements on gNB beams from one or more cells, and such measurements may be subject to L1 and L3 filtering before being reported to the serving gNB. L3 may also be referred to as the RRC layer. L3 filtering parameters and potential reporting criteria may be configured, for example, via RRC signaling. However, the gNB may not be aware of at which time the (multiple) UE antenna panels are active.

[0044] In RRC connected mode, the UE may perform radio link monitoring in the active bandwidth part BWP based on one or more reference signals, such as SSB and / or CSI-RS, and a signal quality threshold configured by the base station. For example, the UE may declare a radio link failure if one or more of the following criteria are met: 1) expiration of a timer started after an indication of a radio problem from the physical layer, 2) failure of the random access procedure, and / or 3) radio link control failure. After the radio link failure, the UE may remain in RRC connected mode, select a suitable cell, and initiate RRC reestablishment. Alternatively, the UE may enter RRC idle mode, for example if a suitable cell is not found within a certain period of time after declaring a radio link failure.

[0045] For beam failure detection, the gNB may configure the UE with one or more beam failure detection reference signals, such as SSB and / or CSI-RS, and the UE may declare beam failure, for example, when the number of beam failure instances indicated from the physical layer reaches a predefined threshold before a predefined timer expires. After beam failure is detected, the UE may trigger beam failure recovery by initiating a random access procedure on the cell and selecting a suitable beam to perform beam failure recovery. If the gNB has provided dedicated random access resources for certain beams, these may be prioritized by the UE. Beam failure recovery may be considered complete when the random access procedure is completed.

[0046] Some exemplary embodiments may utilize machine learning (ML) in a device such as a UE to automatically learn or predict how to efficiently perform antenna panel swaps and L1 measurements in order to balance tradeoffs between key performance indicators (KPIs) affected by how the UE performs antenna panel swaps and L1 measurements. In other words, some exemplary embodiments may provide ML-driven UE antenna panel control to exploit, for example, a tradeoff between link performance (such as throughput, outages, and / or latency) and radio link failures, beam failures, and / or switching failures. Some exemplary embodiments may be used, for example, in a UE where only one antenna panel may be allowed to be active at a time. As a non-limiting example, reinforcement learning (RL) may be used with gNB-to-UE signaling to guide the UE on how to compromise between multiple KPIs.

[0047] An example of a KPI may be QoE, QoS, class index, i.e. 5QI, in accordance with 5G Quality of Service. Meeting QoS requirements according to 5QI may require the UE to use its best antenna panel towards the serving cell. During the moment when such measurements are performed, measurements on other antenna panels may result in degraded performance. On the other hand, for a UE using its best antenna panel, it may be necessary to occasionally perform measurements on other antenna panels, as conditions may change such that one of the other antenna panels becomes the best panel.

[0048] Another example of a KPI may be a beam failure declaration. If a beam failure declaration occurs, it may trigger a beam failure recovery procedure involving random access and thus a temporary interruption of the serving cell communication link, i.e. equivalent to a degraded QoE. If the UE is subject to, for example, movement, rotation and / or local scattering changes, the best antenna panel may change more frequently and thus the UE may perform more frequent antenna panel swaps to make measurements on its antenna panel. Therefore, the beam failure probability may be affected by how and when the UE swaps the active antenna panel.

[0049] Another example of a KPI may be a radio link failure rate and / or a handover failure rate. A handover failure is a radio link failure that may occur during a handover. A handover may need to be performed in a timely manner for a UE served by its best cell. When a UE moves from a first cell to a second cell, the best antenna panel for the UE for the second cell may be different from that for the first cell. For efficient handover performance, the UE may therefore occasionally perform measurements on its antenna panel that points to one or more target cells to which it may be handed over.

[0050] As a non-limiting example, some exemplary embodiments may be used for NR at 28 GHz in a UE including multiple antenna panels (e.g., three or four antenna panels), for example. However, it should be understood that some exemplary embodiments may also be used in other frequency regions, and the number of antenna panels may vary.

[0051] Reinforcement learning can be based on states, denoted s, actions, denoted a, and rewards, denoted r. The state s can be defined as a tuple of values, which can also be called features, which describes the environment for the learning software agent in a way that is relevant to the current problem. For example, the environment can be defined as the set of all possible states that the software agent can be in. The action a determines the next state of the software agent. In other words, performing the action results in a transition to a new state. The reward r can be defined as a multi-objective scalar function that expresses the goal of the software agent in a numerical value. The interaction of the software agent with the environment over time can be described by a sample of experiences represented by a tuple (s t, a t , r t+1 ,s t+1 ), where t is a discrete time counter that describes a state transition as a result of applying an action to the environment and receiving a reward. The goal of RL can be to determine a policy p from a sample of experiences that suggests the next action to take in response to the current state in order to maximize the long-term cumulative reward. Thus, the RL algorithm maps rewards to previously taken actions. The policy can be defined as a function that takes a state as input and returns an action as output. A policy can include a mapping or probability distribution of state-action pairs, which can be used to suggest actions in response to the current state based on a reward function parameterized by a set of adjustable weights. A policy can indicate the probability of each action when the software agent is in a particular state, and the action with the highest probability can be returned as the suggested action.

[0052] Figure 2 A signaling diagram according to an exemplary embodiment is illustrated, in which a UE may run a machine learning scheme for antenna panel switching. A base station such as a gNB configures 201 the UE with relative weights of a reward function for the UE, such as serving cell link performance, beam failure declaration, and / or handover failure declaration. The weight may be a weighting factor for each KPI used in the reward function to estimate the reward at each time step as a weighted sum. The configuration of the weights may be performed, for example, as part of a radio resource control RRC connection establishment process. The weights may be sent, for example, as a vector of scalar weights (e.g., three scalar weights), where a value between 0 and 1 indicates the relative importance of the KPI associated with the scalar weight. In other words, the weights may indicate how to balance the tradeoffs between different KPIs, such as maximizing the time that the UE receives on its best antenna panel, i.e., maximizing the serving cell QoE, while minimizing radio link failures, handover failures, and / or beam failures. The weighted KPIs may be normalized before determining or calculating the weighted sum of the KPIs. Normalization can be performed, for example, by assigning or mapping scalar reward values ​​to KPI values, i.e., normalizing achievable real KPI values ​​or their range of interest to the range of scalar reward values. As a non-limiting example, the range of scalar reward values ​​can be -50 to +50. However, reward values ​​do not have to be integers, as other data types can also be used. As another example for performing normalization, reward values ​​can be assigned to KPIs by setting a reward of +1 for good KPI values ​​(e.g., values ​​within a first predefined range) and a reward value of -1 for bad KPI values ​​(e.g., values ​​within a second predefined range).

[0053] Alternatively, the gNB may explicitly configure which KPIs are to be used in the reward function for the UE and how they should be weighted. The gNB may configure the UE, for example, based on the service requirements of the UE. The service requirements may be based, for example, on a 5G QoS class index, i.e., 5QI, in the transport header of the data packet, and on corresponding QoS parameters, which are signaled via the CN interface when a packet session is established. The 5QI is defined, for example, in 3GPP TS23.501 and may include information such as resource type, packet delay budget, packet error rate, data burst volume, etc. As a non-limiting example, the gNB may set different weights, for example for ultra-reliable low latency communication, URLLC, enhanced mobile broadband, eMBB, services and / or high and low mobility users. Once configured, the UE may perform machine learning based on antenna panel switching based on the configured weights of the reward function.

[0054] The UE and the gNB may then perform radio resource management RRM signaling 202 with each other for mobility purposes, such as timely handover. For example, the UE may perform RRM measurements, such as reference signal received power RSRP and / or reference signal received quality RSRQ, and then send the measurement results to the gNB. Based on the measurements, the gNB may then, for example, estimate the probability that the UE will undergo a handover in the near future. For example, if the UE reports a high RSRP and / or RSRQ indicating that the UE is far from a cell edge, the probability of a handover in the near future may be low.

[0055] The gNB may then update 203 the relative weights for the UE based on the RRM signaling 202. As an example, if the gNB estimates a low probability for the UE to experience a handover in the near future, the gNB may configure a low or even zero weight for radio link failure and / or handover failure for the UE to use in its reward function. As a result, the UE may further relax or reduce antenna panel switching that it would otherwise perform for the purpose of a timely handover. The gNB may also configure the UE with different weights for the reward function depending on its RRC state, e.g., having different weights in RRC idle mode and RRC connected mode. However, for enabling the machine learning based scheme for the UE to operate efficiently, it may be beneficial if the weights are not updated too frequently.

[0056] In another exemplary embodiment, a machine learning algorithm or RL algorithm may operate in an exploration mode, where the software agent performs actions randomly, for example with a certain probability, to collect information about the environment. This may be beneficial in order to allow the software agent to experience all or most possible states so that the average reward can be maximized in the long term.

[0057] In some exemplary embodiments, if Figure 2 If the signaling illustrated in FIG. 1 is not present, the UE may still perform machine learning based antenna panel switching, for example based on a hard-coded reward function, or a reward function otherwise determined to be appropriate, for example based on the 5QI information.

[0058] Figure 3 The figure shows a two-dimensional state space for machine learning according to an exemplary embodiment. The state space includes multiple states 301. The first dimension, i.e., the X-axis, represents a quantized version of the quality of the serving cell measured, for example, by the received RSRP. It should be understood that RSRP is used as an example only in this article, and the quality of the serving cell can also be measured, for example, by the SINR received on the CSI-RS. The second dimension, i.e., the Y-axis, represents a quantized version of the quality of the strongest neighboring cell measured, for example, by RSRP. In other words, the state 301 in the state space can be defined by quantized measurements of the quality of the serving cell and the quality of the strongest neighboring cell. A neighboring cell can be defined as another cell adjacent to the serving cell. The diagonal area 301 includes a state indicating a high probability of switching.

[0059] Figure 3 The state space illustrated in may reflect the overall performance, so it may be desirable to have the UE operate in a state as far to the right as possible, i.e. with a high serving cell quality. At each action, the new state of the UE may be determined by its measurements on the active antenna panel, i.e. the UE measures the serving cell quality, such as RSRP. However, in order to determine which antenna panel corresponds to the highest received serving cell quality, the UE may also be required to periodically perform measurements on other antenna panels. The UE may then determine that the antenna panel corresponding to the highest received quality is its best antenna panel. When the UE switches to performing measurements on another antenna panel compared to its current best antenna panel, it may experience a 10-30dB drop in serving cell quality, while the best neighboring cell quality may be experienced stronger. If the serving cell is using a beam grid, the serving cell quality may be measured as the reception quality through the selected beam, which is notified to the UE as part of the MAC-based beam management process. The UE may also measure the RSRP from neighboring cells. If the neighboring cell also operates with a beam grid, the neighboring cell RSRP may be the RSRP corresponding to the strongest beam from the cell. For example, RSRP may be measured on beamformed SSBs from a cell.

[0060] In another exemplary embodiment, a state space including three or more dimensions may be used. For example, in addition to Figure 3In addition to the two dimensions illustrated, a third dimension may be used to represent the filtered beam failure rate, a fourth dimension may be used to represent the filtered radio link failure rate, and / or a fifth dimension may be used to represent the filtered switching failure rate. Any combination of three or more of these dimensions may be used in this exemplary embodiment. The reward function may be defined as a weighted average of the remaining KPIs, i.e., KPIs that are not in the state space dimensions used, including QoS metrics. The type and dimensionality of the state space used in the UE may depend on the UE implementation and processing capabilities. At the time of initial call establishment, the serving base station may be informed of the UE's capabilities in terms of the dimensions of the implemented state space and the corresponding reward function so that the base station can select appropriate scalar weights.

[0061] Figure 4 A flow chart according to an exemplary embodiment for machine learning based antenna panel switching is shown. Figure 4 The exemplary embodiment shown may be used, for example, Figure 3 The state space shown. Figure 4 The functions shown may be performed by a device such as a UE.

[0062] In block 401, a strategy is used to select an action to be taken at the next moment based on the current state and a reward function. The selected action indicates which antenna panel of a plurality of antenna panels included in the UE is to be used for measurements, e.g., during the next SSB burst. The indicated antenna panel may be the same as the currently active antenna panel, or it may be another antenna panel.

[0063] In block 402, the selected action is performed, i.e. the indicated antenna panel is used to measure quality metrics, such as RSRP, of the serving cell and one or more neighbor cells. These measurements may be stored, for example, in an internal database of the UE, so that the UE can keep track of the strongest neighbor cell measurements received from its antenna panel and update 403 the state based on these measurements, i.e. after performing the action.

[0064] In block 404, the KPIs included in the reward function are updated based on the measurements. A new long-term reward scalar value may then be determined or calculated as a weighted average of the KPIs, for example by using weights received from the base station. The long-term time value may depend on the KPI to be maximized. The time period over which the average return or reward is maximized may be another important factor. For example, the environment may be assumed to be constant for a sufficiently long time to allow the agent to learn an optimal policy. As a non-limiting example, the time range for the long-term reward scalar value may be 100ms-1000ms, over which the reward may be maximized until the environment changes too much and the RL algorithm needs to relearn the optimal policy.

[0065] As a non-limiting example, the reward function may be a weighted average of KPIs such as the UE's experienced serving cell quality, experienced long-term beam failure rate, experienced long-term radio link failure rate and / or handover failure rate. However, it should be understood that the reward function may also include different and / or additional KPIs.

[0066] In order to maximize long term performance, low pass filtering may be used to iteratively update the KPI before weighting the KPI and determining an updated reward function. For example, for the serving cell quality KPI, an infinite impulse response (IIR) filter may be used to determine the average RSRP or SINR experienced. If the UE switches to an antenna panel pointing away from the serving cell, this may be indicated by a lower serving cell RSRP or SINR, and therefore by a degradation in serving cell quality. The IIR may operate on samples expressed in dB or dBm such that a geometric mean is determined, as this may more accurately represent serving cell performance. Similarly, for beam failures and swap failures, the ratio of these errors may be determined during a sliding window of a predefined length. Beam tracking errors may occur more frequently than swaps, and so the sliding window length used to monitor the beam failure rate may be shorter than the window used to determine the swap failure rate. For example, the length of the sliding window used by the UE to determine the beam failure rate and the swap failure rate may be as Figure 2 A portion of the signaling shown is signaled from a base station to a UE.

[0067] In block 405, the policy may be updated, for example, using a Q-learning RL algorithm or a deep Q-learning algorithm. The purpose of the policy may be to optimize long-term performance based on a desired tradeoff between KPIs as indicated by weights. The UE may learn the Q function via iterative updates using the Bellman equation. For example, the Q values ​​for individual state-action pairs may be stored and updated in a data structure such as a table. As another example, a convolutional neural network may be used for learning of a data structure table. Convolutional neural networks may be beneficial, for example, for high-dimensional states and continuous states.

[0068] Figure 4 The process shown may be iterative such that after block 405, it returns to block 401 and continues from there. For example, the process may run at 20 ms intervals synchronized with SSB bursts transmitted from one or more cells.

[0069] In some exemplary embodiments, the UE may return to use its best antenna panel after block 405. Alternatively, if the most recently measured serving cell quality on the selected antenna panel is better than the measurements recorded in the database of the antenna panel previously determined to be the best antenna panel, then after block 405, the selected antenna panel may be updated to be the best antenna panel. The UE may then use the best antenna panel for transmission and / or reception. However, if the UE is in probing mode, it may use a randomly selected antenna panel.

[0070] In another exemplary embodiment, Figure 3 The process shown can be used for RRC idle mode by using the camping cell quality instead of the serving cell quality. Additionally, the base station can configure the UE to have different weights for its reward function for use in RRC idle mode. For example, metrics such as beam failure and switching failure may be irrelevant to the UE in RRC idle mode. In addition, the camping cell quality target may not be as strict as for a UE in RRC connected to a URLLC type of service, for example.

[0071] Given the finite length of the RRC connected mode state or the RRC idle mode state, the RL algorithm can operate in an episodic task mode, which means that the antenna panel selection strategy is updated and applied independently between consecutive RRC connected modes or RRC idle modes of the same UE. For example, two separate RL algorithms can be run independently, i.e., each algorithm optimizes its own strategy for the connected mode and the idle mode respectively.

[0072] Figure 5A flow chart according to an exemplary embodiment is shown. In box 501, an antenna panel is selected from a plurality of antenna panels by using a machine learning algorithm. In box 502, a first long-term reward value associated with the selected antenna panel is determined based at least in part on one or more first signals received on the selected antenna panel. In box 503, it is evaluated whether the determined first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels. In other words, the one or more second long-term reward values ​​may be associated with other antenna panels included in the plurality of antenna panels. The one or more second long-term reward values ​​may be previously determined long-term reward values, which may be stored in, for example, an internal or external database (e.g., a table), and may be obtained from, for example, an internal or external database (e.g., a table) for evaluation. If the first long-term reward value exceeds one or more second long-term reward values ​​(503: yes), then in box 504, a second signal is sent or received via the selected antenna panel. If the first long term reward value does not exceed one or more second long term reward values ​​( 503 : NO), a second signal may be sent or received via another antenna panel (eg, the antenna panel associated with the highest long term reward value) and the process may return to block 501 .

[0073] In other words, maximizing the overall long-term reward value can be the main purpose of the software agent. Different reward values ​​for selecting each antenna panel can be used in each time step. These immediate reward values ​​can be calculated based on the weighted reward function described previously, possibly including some filtering. The long-term reward value can then be determined as, for example, the average or sum of consecutive immediate reward values ​​over a predefined time window.

[0074] In some exemplary embodiments of applying Q-learning, the long-term reward value may be a Q-value, i.e., a state value, or at least based on a Q-value. Figure 5 The first long-term reward value described may be based on the first Q value, and one or more second long-term reward values ​​may be based on one or more second Q values. A second signal may then be sent or received via the antenna panel associated with the highest Q value. The estimate of the Q value may be based on the immediate reward value for each action and time step calculated using the weighted reward function described previously.

[0075] Above through Figure 2 , Figure 4 and Figure 5 The functions and / or blocks described are not in an absolute chronological order, and some of them may be executed simultaneously or in an order different from that described. Other functions and / or blocks may also be executed between or within them.

[0076] Figure 6An exemplary embodiment with linear UE movement is shown. In this exemplary embodiment, UE 601 moves from west to east along trajectory 602. The UE may include four antenna panels, where the main lobes of the antenna panels point to the north, east, south and west, respectively. In this case, the best antenna panel is the one pointing to the west, i.e., towards the serving cell 603. In other words, when the UE measures on the antenna panel pointing to the west, it experiences a higher serving cell quality than on the other antenna panels, and it is in a Figure 3 On the other hand, when the UE measures on the antenna panel pointing east, i.e., towards the neighbor cell 604, it experiences lower serving cell quality but higher neighbor cell quality than on the west antenna panel, so it is in Figure 3 The states at the top of the state space are shown. When the UE measures on the antenna panel pointing north or south, the serving cell quality is less than the west antenna panel, and the neighbor cell quality is less than the east antenna panel. Therefore, the UE can learn to prioritize measurements on the west antenna panel and the east antenna panel, and when it approaches the boundary between the two cells, it can measure on the east panel more often so that the swap can be performed in time. Otherwise, the UE may experience a swap failure, resulting in a decrease in the reward function and an adjustment of the strategy to avoid the swap failure from happening again.

[0077] Figure 7 An exemplary embodiment with a rotating UE is shown. In this exemplary embodiment, the UE 701 rotates 702, so the best antenna panel will switch as the UE rotates. The UE may not be subject to a swap in the near future, so it may only show the serving cell 703. At a first moment, the first antenna panel may be the best antenna panel. When the UE rotates, the second antenna panel may become the best antenna panel at a second moment. If the UE does not switch to the second antenna panel at the second moment, it may experience a beam failure, a degradation in the serving cell quality, and / or a radio link failure. Therefore, the UE can learn over time to prioritize measurements on the first antenna panel and the second antenna panel during subsequent rotation periods corresponding to the first moment and the second moment, while giving low priority to measurements on other antenna panels during these periods.

[0078] Technical advantages provided by some exemplary embodiments may be that they may enable a device (such as a UE) to automatically learn when to perform measurements on antenna panels other than an active antenna panel in order to determine which antenna panel provides the best signal quality corresponding to a long-term average time period. By switching to the antenna panel that provides the best signal quality at the appropriate time, the serving cell quality may be increased, and radio link failures, beam failures, and / or switching failures may be reduced.

[0079] Figure 8 An apparatus 800 according to an exemplary embodiment is shown, which may be an apparatus such as a terminal device or included in a terminal device. The apparatus 800 includes a processor 810. The processor 810 interprets computer program instructions and processes data. The processor 810 may include one or more programmable processors. The processor 810 may include programmable hardware with embedded firmware, and may alternatively or additionally include one or more application specific integrated circuits ASICs.

[0080] The processor 810 is coupled to the memory 820. The processor is configured to read data to the memory 820 and write data from the memory 820. The memory 820 may include one or more memory units. The memory unit may be volatile or non-volatile. It should be noted that in some exemplary embodiments, there may be one or more non-volatile memory units and one or more volatile memory units, or alternatively, one or more non-volatile memories, or alternatively, one or more volatile memories. The volatile memory may be, for example, a RAM, a DRAM, or an SDRAM. The non-volatile memory may be, for example, a ROM, a PROM, an EEPROM, a flash memory, an optical storage device, or a magnetic storage device. In general, the memory may be referred to as a non-transitory computer-readable medium. The memory 820 stores computer-readable instructions executed by the processor 810. For example, the non-volatile memory stores computer-readable instructions, and the processor 810 executes the instructions using a volatile memory for temporarily storing data and / or instructions.

[0081] The computer readable instructions may be pre-stored in the memory 820, or alternatively or additionally, they may be received by the device via an electromagnetic carrier signal and / or may be copied from a physical entity such as a computer program product. Execution of the computer readable instructions causes the device 800 to perform one or more functions described above.

[0082] In the context of this document, "memory" or "computer-readable medium" or "computer-readable medium" may be any non-transitory medium or media or means that can contain, store, communicate, propagate, or transport instructions for use by or in connection with an instruction execution system, apparatus, or device (e.g., a computer).

[0083] The device 800 may also include or be connected to an input unit 830. The input unit 830 may include one or more interfaces for receiving input. The one or more interfaces may include, for example, one or more temperature sensors, motion sensors and / or orientation sensors, one or more cameras, one or more accelerometers, one or more microphones, one or more buttons and / or one or more touch detection units. In addition, the input unit 830 may include an interface to which an external device may be connected.

[0084] The device 800 may also include an output unit 840. The output unit may include or be connected to one or more displays capable of presenting visual content, such as light emitting diodes, LEDs, displays, liquid crystal displays, LCDs, and liquid crystal on silicon, LCoS displays. The output unit 840 may also include one or more audio outputs. The one or more audio outputs may be, for example, speakers.

[0085] The device 800 also includes a connection unit 850. The connection unit 850 implements a wireless connection with one or more external devices. The connection unit 850 includes at least one transmitter and at least one receiver that can be integrated into the device 800 or can be connected to the device 800. The at least one transmitter includes at least one transmitting antenna, and the at least one receiver includes at least one receiving antenna. The connection unit 850 may include an integrated circuit or a group of integrated circuits that provide wireless communication capabilities for the device 800. Alternatively, the wireless connection may be a hardwired application-specific integrated circuit ASIC. The connection unit 850 may include one or more components controlled by a corresponding control unit, such as a power amplifier, a digital front end DFE, an analog-to-digital converter ADC, a digital-to-analog converter DAC, a frequency converter, a (de)modulator and / or an encoder / decoder circuit device.

[0086] It should be noted that the apparatus 800 may also include Figure 8 Various components are not shown in the figure. The various components can be hardware components and / or software components.

[0087] Fig. 9 The apparatus 900 shows an exemplary embodiment of an apparatus such as or included in a base station (such as a gNB). The apparatus may include, for example, a circuit device or a chipset applicable to a base station to implement some of the described exemplary embodiments. The apparatus 900 may be an electronic device including one or more electronic circuits. The apparatus 900 may include a communication control circuit device 910 such as at least one processor, and at least one memory 920 including computer program code (software) 922, wherein the at least one memory and the computer program code (software) 922 are configured to, together with the at least one processor, cause the apparatus 900 to perform some of the above exemplary embodiments.

[0088] The memory 920 may be implemented using any suitable data storage technology, such as semiconductor-based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. The memory may include a configuration database for storing configuration data. For example, the configuration database may store a current neighbor cell list, and in some exemplary embodiments, may store the structure of frames used in detected neighbor cells.

[0089] The apparatus 900 may also include a communication interface 930, which includes hardware and / or software for implementing communication connectivity according to one or more communication protocols. The communication interface 930 may provide the apparatus with radio communication capabilities to communicate in a cellular communication system. The communication interface may, for example, provide a radio interface to a terminal device. The apparatus 900 may also include another interface toward a core network (such as a network coordinator device) and / or an access node to a cellular communication system. The apparatus 900 may also include a scheduler 940 configured to allocate resources.

[0090] As used in this application, the term "circuitry" may refer to one or more or all of the following:

[0091] a hardware circuit implementation only (e.g., implementation only in analog and / or digital circuitry) and

[0092] b A combination of hardware circuitry and software, such as (if applicable):

[0093] a combination of analog and / or digital hardware circuit(s) and software / firmware, and

[0094] ii any portion of hardware processor(s) (including digital signal processor(s)), software and memory(s) with software that work together to enable a device such as a mobile phone to perform various functions) and

[0095] c Hardware circuit(s) and / or processor(s) that require software (e.g., firmware) to operate, such as microprocessor(s) or a portion of microprocessor(s) but may not be present when the software is not needed to operate.

[0096] This definition of circuitry applies to all uses of this term in this application, including in any claims. As another example, as used in this application, the term circuitry also covers an implementation of only a hardware circuit or processor (or multiple processors) or a portion of a hardware circuit or processor and its accompanying software and / or firmware. The term circuitry also covers (for example and if applicable to a particular claim element) a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in a server, a cellular network device, or other computing or network device.

[0097] The techniques and methods described herein may be implemented by various components. For example, these techniques may be implemented in hardware (one or more devices), firmware (one or more devices), software (one or more modules), or a combination thereof. For hardware implementations, the (multiple) devices of the exemplary embodiments may be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), graphics processing units (GPUs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof. For firmware or software, the implementation may be performed by a module (e.g., a process, a function, etc.) of at least one chipset that performs the functions described herein. The software code may be stored in a memory unit and executed by a processor. The memory unit may be implemented within or outside the processor. In the latter case, it may be communicatively coupled to the processor via various components known in the art. Additionally, the components of the system described herein may be rearranged and / or supplemented by additional components to facilitate support for implementation of various aspects of the description thereof, etc., and they are not limited to the precise configurations set forth in the given figures, as will be understood by those skilled in the art.

[0098] It is obvious to those skilled in the art that, as technology advances, the concepts of the present invention can be implemented in various ways. The embodiments are not limited to the above exemplary embodiments, but can be varied within the scope of the claims. Therefore, all words and expressions should be interpreted broadly, and they are intended to illustrate rather than limit the exemplary embodiments.

Claims

1. A communication device comprising at least one processor and at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to, together with the at least one processor, cause the device to: selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm; determining a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received at the selected antenna panel, wherein the first long-term reward value is an average or sum of consecutive instantaneous reward values ​​over a first time window, wherein the consecutive instantaneous reward values ​​are based at least in part on a plurality of key performance indicators, the plurality of key performance indicators comprising at least a serving cell quality metric and a beam failure rate; In an event that the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels, a second signal is transmitted or received via the selected antenna panel.

2. The apparatus of claim 1, wherein the machine learning algorithm is based on reinforcement learning.

3. The apparatus of claim 1 , wherein the machine learning algorithm is based on a state space comprising at least two dimensions, wherein a first dimension indicates the serving cell quality metric and a second dimension indicates a neighbor cell quality metric.

4. The apparatus of claim 1 , wherein the antenna panel is selected using a strategy for selecting an action corresponding to the selected antenna panel based at least in part on a current state and a reward function, wherein the reward function is based on a plurality of weights.

5. An apparatus according to claim 4, wherein the multiple weights are scalar values ​​as follows, which indicate the priority for maximizing the serving cell quality metric, minimizing the beam failure rate, minimizing the radio link failure rate and / or minimizing the switching failure rate. The apparatus of claim 4 , wherein the apparatus is further caused to obtain the plurality of weights from a base station.

7. The apparatus according to claim 6, wherein the apparatus is further configured to: measuring a quality metric associated with one or more of said first signals received on said selected antenna panels; The state, the reward function, and the policy are updated based at least in part on the measured quality metric.

8. The apparatus of claim 7, wherein the policy is updated by updating Q-values ​​for one or more state-action pairs using Q-learning.

9. The apparatus of claim 7, wherein the apparatus is further configured to: sending the measured quality metric to the base station; An updated plurality of weights is received from the base station.

10. The apparatus according to claim 7, wherein the quality metric comprises a reference signal received power value and / or a signal to interference and noise ratio value.

11. The apparatus of claim 1, wherein one or more of the second long-term reward values ​​are obtained from a database.

12. The apparatus according to any one of claims 1 to 11, wherein one or more of the first signals include at least a serving cell signal and a neighbor cell signal.

13. A communication system, comprising at least: Terminal equipment and base stations; The terminal device is configured as follows: selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm; receiving one or more first signals from the base station via the selected antenna panel; determining a first long-term reward value associated with the selected antenna panel based at least in part on one or more of the first signals received at the selected antenna panel, wherein the first long-term reward value is an average or sum of consecutive instantaneous reward values ​​over a first time window, wherein the consecutive instantaneous reward values ​​are based at least in part on a plurality of key performance indicators, the plurality of key performance indicators comprising at least a serving cell quality metric and a beam failure rate; transmitting or receiving a second signal via the selected antenna panel if the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels; The base station is configured as follows: Sending one or more of the first signals to the terminal device; A second signal is received from the terminal device or a second signal is sent to the terminal device.

14. A method for communication, comprising: selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm; determining a first long-term reward value associated with the selected antenna panel based at least in part on one or more first signals received at the selected antenna panel, wherein the first long-term reward value is an average or sum of consecutive instantaneous reward values ​​over a first time window, wherein the consecutive instantaneous reward values ​​are based at least in part on a plurality of key performance indicators, the plurality of key performance indicators comprising at least a serving cell quality metric and a beam failure rate; In an event that the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels, a second signal is transmitted or received via the selected antenna panel.

15. A computer program product comprising instructions for causing an apparatus to perform at least the following: selecting an antenna panel from a plurality of antenna panels using a machine learning algorithm; determining a first long-term reward value associated with a selected antenna panel based at least in part on one or more first signals received at the selected antenna panel, wherein the first long-term reward value is an average or sum of consecutive instantaneous reward values ​​over a first time window, wherein the consecutive instantaneous reward values ​​are based at least in part on a plurality of key performance indicators, the plurality of key performance indicators comprising at least a serving cell quality metric and a beam failure rate; In an event that the first long-term reward value exceeds one or more second long-term reward values ​​associated with at least a subset of the plurality of antenna panels, a second signal is transmitted or received via the selected antenna panel.

Images