User device machine learning action decision and evaluation

Abstract

Techniques for implementing actions in a wireless network based on machine learning process output include a user equipment (UE) operating a machine learning process communicating with an action to take with a network node (gNB) based on the output of the machine learning process.

Description

Technical Field

[0001] This description relates to telecommunication systems. Background Art

[0002] A communication system may be a facility that enables communication between two or more nodes or devices, such as fixed or mobile communication devices.Signals may be carried on wired or wireless carriers.

[0003] An example of a cellular communication system is the architecture being standardized by the Third Generation Partnership Project (3GPP). The latest development in this area is generally referred to as the Long Term Evolution (LTE) of the Universal Mobile Telecommunications System (UMTS) radio access technology. E-UTRA (Evolved UMTS Terrestrial Radio Access) is the air interface of the 3GPP's mobile network LTE upgrade path. In LTE, a base station or access point (AP) (which is referred to as an enhanced node AP (eNB)) provides wireless access within a coverage area or cell. In LTE, a mobile device or mobile station is referred to as a user equipment (UE). LTE has included multiple improvements or developments.

[0004] The global bandwidth shortage faced by wireless carriers has, for example, prompted consideration of using underutilized millimeter wave (mmWave) spectrum for future broadband cellular communication networks. mmWave (or extremely high frequency) can include, for example, a frequency range between 30 and 300 gigahertz (GHz). The wavelength of radio waves in this band can be, for example, from ten to one millimeter, so it is called millimeter band or millimeter wave. In the next few years, the amount of wireless data will likely increase significantly. Various technologies have been used to try to meet this challenge, including obtaining more spectrum, reducing cell size, and using improved technology to achieve more bits / s / Hz. One element that can be used to obtain more spectrum is to move to higher frequencies, for example, above 6GHz. For the fifth generation wireless system (5G), an access architecture that uses mmWave radio spectrum to deploy cellular radio equipment has been proposed. Other example spectrums, such as cmWave radio spectrum (e.g., 3-30GHz), may also be used. Summary of the invention

[0005] According to an example implementation, a method includes executing at least one machine learning process by a user device in a wireless network to generate an output. The method also includes determining, by the user device, an action to be performed based on the output generated by executing at least one machine learning process. The method further includes sending, by the user device in the wireless network, a request to perform the action to a network node serving the user device. The method further includes receiving, by the user device, a message from the network node indicating whether the user device can perform the action. The method further includes, in response to the message indicating that the user device can perform the action, the user device performing the action. The method further includes not performing the action in response to the message indicating that the user device cannot perform the action.

[0006] According to an example implementation, an apparatus includes at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the apparatus to at least: execute at least one machine learning process by a user device in a wireless network to generate an output. The at least one memory and the computer program code are also configured to, together with the at least one processor, cause the apparatus to at least: determine, by the user device based on the output generated by executing at least one machine learning process, an action to be performed. The at least one memory and the computer program code are further configured to, together with the at least one processor, cause the apparatus to at least: send a request to perform the action to a network node serving the user device by the user device in the wireless network. The at least one memory and the computer program code are further configured to, together with the at least one processor, cause the apparatus to at least: receive, by the user device, a message from the network node indicating whether the user device can perform the action. The at least one memory and the computer program code are further configured to, together with the at least one processor, cause the apparatus to at least: perform the action by the user device in response to the message indicating that the user device can perform the action. The at least one memory and the computer program code are further configured to, with the at least one processor, cause the apparatus to at least: in response to the message indicating that the user equipment may not perform the action, not perform the action.

[0007] According to an example implementation, an apparatus includes a component for executing at least one machine learning process by a user device in a wireless network to generate an output. The apparatus also includes a component for determining, by the user device, an action to be performed based on the output generated by executing at least one machine learning process. The apparatus further includes a component for sending a request to perform the action by a user device in the wireless network to a network node serving the user device. The apparatus further includes a component for receiving, by the user device, a message from the network node indicating whether the user device can perform the action. The apparatus further includes a component for performing the action by the user device in response to the message indicating that the user device can perform the action. The apparatus further includes a component for not performing the action in response to the message indicating that the user device cannot perform the action.

[0008] According to an example implementation, a computer program product includes a computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to: perform at least one machine learning process by a user device in a wireless network to generate an output. The computer-readable storage medium also stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to: determine an action to be performed by the user device based on the output generated by performing at least one machine learning process. The computer-readable storage medium further stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to: send a request to perform the action by a user device in the wireless network to a network node serving the user device. The computer-readable storage medium further stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to: receive a message from the network node by the user device indicating whether the user device can perform the action. The computer-readable storage medium further stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to: in response to the message indicating that the user equipment can perform the action, the user equipment performs the action. The computer-readable storage medium further stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to: in response to the message indicating that the user equipment cannot perform the action, not perform the action.

[0009] According to an example implementation, a method includes receiving, by a network node of a wireless network, a request for an action to be performed by the user equipment from the user equipment, the network node serving the user equipment in the wireless network, the action being determined by the user equipment based on an output of at least one machine learning process. The method also includes generating, by the network node, a corresponding prediction of a result of the user equipment performing the action on the wireless network. The method further includes sending, by the network node, a message to the user equipment indicating whether the user equipment can perform the action based on the corresponding prediction.

[0010] According to an example implementation, an apparatus includes at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the apparatus to at least: receive, by a network node of a wireless network, a request for an action to be performed by a user device from the user device, the network node serving the user device in the wireless network, the action being determined by the user device based on an output of at least one machine learning process. The at least one memory and the computer program code are further configured to: generate, by the network node, a corresponding prediction of a result of the user device performing the action on the wireless network. The at least one memory and the computer program code are further configured to: send, by the network node, a message to the user device indicating whether the user device can perform the action, based on the corresponding prediction.

[0011] According to an example implementation, an apparatus includes a component for receiving, by a network node of a wireless network, a request for an action to be performed by the user equipment from the user equipment, wherein the network node serves the user equipment in the wireless network, the action being determined by the user equipment based on an output of at least one machine learning process. The apparatus also includes a component for generating, by the network node, a corresponding prediction of a result of the user equipment performing the action on the wireless network. The apparatus further includes a component for sending, by the network node, to the user equipment a message indicating whether the user equipment can perform the action based on the corresponding prediction.

[0012] According to an example implementation, a computer program product includes a computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to: receive, by a network node of a wireless network, a request for an action to be performed by the user equipment from the user equipment, the network node serving the user equipment in the wireless network, the action being determined by the user equipment based on the output of at least one machine learning process. The executable code, when executed by at least one data processing device, is further configured to cause the at least one data processing device to: generate, by the network node, a corresponding prediction of a result of the user equipment performing the action on the wireless network. The executable code, when executed by at least one data processing device, is further configured to cause the at least one data processing device to: send, by the network node, a message to the user equipment indicating whether the user equipment can perform the action based on the corresponding prediction.

[0013] According to an example implementation, a method includes executing, by a user device in a wireless network, at least one machine learning process to produce an output. The method also includes determining, by the user device, an action to be performed based on the output produced by executing the at least one machine learning process. The method further includes executing, by the user device, the action to produce an execution of the action. The method further includes sending, by the user device, a first message to a network node, the first message including an indication of an estimated local result of the execution of the action. The method further includes receiving, by the user device, a second message from the network node, the second message indicating an impact of the execution of the action on the wireless network globally.

[0014] According to an example implementation, an apparatus includes at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the apparatus to at least: execute at least one machine learning process by a user device in a wireless network to generate an output. The at least one memory and the computer program code are also configured to: determine, by the user device, an action to be performed based on the output generated by executing at least one machine learning process. The at least one memory and the computer program code are further configured to: execute the action by the user device to generate the execution of the action. The at least one memory and the computer program code are further configured to: send a first message by the user device to a network node, the first message including an indication of an estimated local result of the execution of the action. The at least one memory and the computer program code are further configured to: receive a second message by the user device from the network node, the second message indicating the impact of the execution of the action on the global wireless network.

[0015] According to an example implementation, an apparatus includes a component for executing at least one machine learning process by a user device in a wireless network to produce an output. The apparatus also includes a component for determining, by the user device, an action to be performed based on the output produced by executing at least one machine learning process. The apparatus further includes a component for executing the action by the user device to produce an execution of the action. The apparatus further includes a component for sending a first message by the user device to a network node, wherein the first message includes an indication of an estimated local result of the execution of the action. The apparatus further includes a component for receiving a second message by the user device from the network node, wherein the second message indicates an impact of the execution of the action on the wireless network globally.

[0016] According to an example implementation, a computer program product includes a computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to: execute at least one machine learning process by a user device in a wireless network to generate an output. The executable code, when executed by at least one data processing device, is also configured to: determine, by the user device, an action to be performed based on the output generated by executing at least one machine learning process. The executable code, when executed by at least one data processing device, is further configured to: execute the action by the user device to generate the execution of the action. The executable code, when executed by at least one data processing device, is further configured to: send a first message by the user device to a network node, the first message including an indication of an estimated local result of the execution of the action. The executable code, when executed by at least one data processing device, is further configured to: receive a second message by the user device from the network node, the second message indicating the impact of the execution of the action on the wireless network globally.

[0017] According to an example implementation, a method includes receiving, by a network node of a wireless network, a first message from a user device, the network node serving the user device in the wireless network, the first message indicating that the user device has performed an action to produce the execution of the action, the action being determined based on an output of a machine learning process, the first message also indicating an estimated local result of the execution of the action. The method also includes generating, by the network node, a corresponding prediction of a global result of the user device performing the action on the wireless network. The method further includes sending, by the network node, a second message to the user device based on the prediction of the global result of the user device performing the action on the wireless network, the second message indicating whether the user device should continue to perform the action.

[0018] According to an example implementation, the apparatus includes at least one processor and at least one memory including computer program code, the at least one memory and the computer program code being configured to, together with the at least one processor, cause the apparatus to at least: receive a first message from a user device by a network node of a wireless network, the network node serving the user device in the wireless network, the first message indicating that the user device has performed an action to produce the execution of the action, the action being determined based on the output of a machine learning process, the first message also indicating an estimated local result of performing the action. The at least one memory and the computer program code are also configured to: generate, by the network node, a corresponding prediction of a global result of the user device performing the action on the wireless network. The at least one memory and the computer program code are further configured to: based on the prediction of the global result of the user device performing the action on the wireless network, send, by the network node, a second message to the user device, the second message indicating whether the user device should continue to perform the action.

[0019] According to an example implementation, an apparatus includes a component for receiving a first message from a user device by a network node of a wireless network, wherein the network node serves the user device in the wireless network, the first message indicating that the user device has performed an action to produce the execution of the action, the action being determined based on the output of a machine learning process, and the first message also indicating an estimated local result of performing the action. The apparatus also includes a component for generating, by the network node, a corresponding prediction of a global result of the user device performing the action on the wireless network. The apparatus further includes a component for sending, by the network node, a second message to the user device based on the prediction of the global result of the user device performing the action on the wireless network, wherein the second message indicates whether the user device should continue to perform the action.

[0020] The details of one or more implementation examples are set forth in the accompanying drawings and the description below. Other features will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a block diagram of a digital communication network according to an example implementation.

[0022] Figure 2 is a sequence diagram illustrating a process of determining actions to take when executing a machine learning algorithm, according to an example implementation.

[0023] Figure 3 is a sequence diagram illustrating a process of determining actions to take when executing a machine learning algorithm, according to an example implementation.

[0024] Figure 4 is a sequence diagram illustrating a process of determining actions to take when executing a machine learning algorithm, according to an example implementation.

[0025] Figure 5 is a flow diagram illustrating a process of determining actions to take when executing a machine learning algorithm, according to an example implementation.

[0026] Figure 6 is a flow diagram illustrating a process of determining actions to take when executing a machine learning algorithm, according to an example implementation.

[0027] Figure 7 is a flow diagram illustrating a process of determining actions to take when executing a machine learning algorithm, according to an example implementation.

[0028] Figure 8 is a flow diagram illustrating a process of determining actions to take when executing a machine learning algorithm, according to an example implementation.

[0029] Fig. 9 is a block diagram of a node or wireless station (eg, a base station / access point, a relay node, or a mobile station / user equipment) according to an example implementation. DETAILED DESCRIPTION

[0030] The principles of the present disclosure will now be described with reference to some example embodiments. It should be understood that these embodiments are described only for illustrative purposes, and help those skilled in the art to understand and implement the present disclosure, without implying any limitation on the scope of the present disclosure. The disclosure described herein can be implemented in various ways except for the ways described below.

[0031] The terms used herein are only for the purpose of describing specific embodiments and are not intended to limit the exemplary embodiments. As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when used herein, the terms "include", "have", and / or "components" specify the presence of stated features, elements / components, and / or components / assemblies, etc., but do not exclude the presence or addition of one or more other features, elements / components, components / assemblies, and / or combinations thereof.

[0032] Figure 1 is a block diagram of a digital communication system, such as wireless network 130, according to an example implementation. Figure 1In a wireless network 130, user equipment 131, 132, and 133 (which may also be referred to as mobile stations (MS) or user equipment (UE)) may be connected (and communicate) with a base station (BS) 134 (which may also be referred to as an access point (AP), an enhanced Node B (eNB), a gNB (which may be a 5G base station), or a network node). At least some of the functions of an access point (AP), a base station (BS), or an (e) Node B (eNB) may also be performed by any node, server, or host that may be operably coupled to a transceiver (such as a remote radio head). BS (or AP) 134 provides wireless coverage within a cell 136 including user equipment 131, 132, and 133. Although only three user equipment are shown connected or attached to BS 134, any number of user equipment may be provided. BS 134 is also connected to a core network 150 via an interface 151. This is just one simple example of a wireless network, and other examples may be used.

[0033] User equipment (user terminal, user equipment (UE)) may refer to a portable computing device including a wireless mobile communication device operating with or without a subscriber identity module (SIM), including, by way of example, but not limited to, the following types of devices: mobile station (MS), mobile phone, cellular phone, smart phone, personal digital assistant (PDA), handheld device, device using a wireless modem (alarm or measurement equipment, etc.), laptop and / or touch screen computer, tablet computer, tablet phone, game console, notebook computer, vehicle, and multimedia device. It should be understood that a user equipment may also be a nearly exclusive uplink-only device, an example of which is a camera or camcorder that loads images or video clips to a network.

[0034] In LTE (as an example), the core network 150 may be referred to as an evolved packet core (EPC), which may include: a mobility management entity (MME), which may handle or assist in mobility / serving cell changes of user equipment between BSs; one or more gateways, which may forward data and control signals between the BSs and a packet data network or the Internet; and other control functions or blocks.

[0035] Various example implementations may be applied to various wireless technologies, wireless networks, such as LTE, LTE-A, 5G (new radio or NR), cmWave and / or mmWave band networks, or any other wireless network or use case. LTE, 5G, cmWave and mmWave band networks are provided only as illustrative examples, and various example implementations may be applied to any wireless technology / wireless network. Various example implementations may also be applied to a variety of different applications, services or use cases, such as, for example, ultra-reliable low latency communication (URLLC), Internet of Things (IoT), time-sensitive communication (TSC), enhanced mobile broadband (eMBB), massive machine type communication (MMTC), vehicle-to-vehicle (V2V), vehicle-to-device, etc. Each of these use cases or UE types may have its own set of requirements.

[0036] Some Radio Resource Management (RRM) functions in the UE may be supported / driven by Machine Learning (ML) based algorithms (i.e., a combination of ML and rule-based algorithms), most of which will likely be device or network vendor specific. Nonetheless, the use of ML based algorithms may have an impact on the performance of RRM functions, e.g., for antenna panel or beam steering, RRM measurements and feedback (Channel State Information (CSI) feedback), link monitoring, transmit power control (TPC) in the UE. In some cases, this impact may be primarily positive and reflected in performance improvements / gains in various traditional Key Performance Indicators (KPIs), which may also be observable at the NG-RAN level.

[0037] It is therefore desirable to allow the NG-RAN to exploit these potential performance improvements / gains in a "proactive" manner, i.e., beyond simply noticing that some UEs perform better than others.

[0038] Conventional techniques for implementing ML-based actions have signaling protocols between gNB and UE, which may need to be extended to include new information elements and procedures designed specifically for ML-based functions (performed on the UE and / or gNB side). Such new information elements and procedures may allow for more dynamic selection and configuration of ML-based functions available in the UE, and optimal UE reporting of corresponding ML-specific reasoning / assisted outputs.

[0039] This can be illustrated by some specific questions identified as follows:

[0040] • Assumption: There are one or more ML-based algorithms implemented at the UE side for inference and / or training, including exploration, etc. The ML agent acts mainly based on the radio and traffic information available at the UE.

[0041] ○ “ML-Action” refers to any UE-based ML-based radio parameter change or adjustment as a result of running an ML-based algorithm. The result of the action may be kept locally at the UE or it may be reported to the serving gNB.

[0042] o The UE ML algorithm can also potentially use information from the network transmitted at higher layers (RRC or higher) at a slower rate.

[0043] • Problem #1: One UE's "action" will not have much impact on the overall cell performance, but many different actions from different UEs with "selfish behavior" may cause problems.

[0044] • Problem #2: The network may not know the best / optimal radio parameter configuration for each UE since it only has delayed (past) information about their radio and traffic conditions.

[0045] Compared to conventional methods of implementing ML-based actions, the improved technique includes a user equipment (UE) running a machine learning process communicating with a network node (gNB) an action to be taken based on the output of the machine learning process. For example, a goal in a wireless network may be to optimize the transmit power value per UE to maximize uplink (UL) and sidelink UE-side throughput while minimizing the impact on overall UL cell performance. After exchanging configuration information with the gNB, the UE runs a reinforcement learning (RL)-based algorithm using UE local reference signal received power (RSRP) and antenna panel or beam selection information as input. The output of the RL-based algorithm is one of three action identifiers: "power increase by 1DB", "power decrease by 1DB", and "no power adjustment", with corresponding identifiers ID=0, 1, and 2. Based on the output, the UE determines an action (e.g., an appropriate UL power adjustment) and notifies the gNB without performing the action. The gNB analyzes this recommended UL power adjustment for the UE in the context of other UEs served by the gNB in ​​the cell. If the gNB expects that the recommended UL power adjustment will not harm the cell UL performance, the gNB sends an ACK message indicating that the UE can perform the action. However, if the gNB expects performance degradation, the gNB sends a NACK message indicating that the UE should not perform the action.

[0046] It should be noted that the machine learning process refers to running or executing a machine learning algorithm that maps input to output. For example, the machine learning process can generate UE UL (or secondary link) transmit power as an output based on the input local radio conditions. In other words, the output generated by the machine learning process is the result of running or executing the machine learning process on the input.

[0047] It should also be noted that the UE is configured to determine an action based on an output generated by performing a machine learning process. For example, the action taken as a result of the above-mentioned machine learning process is to adjust the UL / side link transmit power. For example, when the machine learning process is performed by the UE, the output can be one of the following three possible actions: "increase power by 1dB", "decrease power by 1dB", "no power adjustment". (In this case, the machine learning algorithm can be a classifier.) Therefore, the action determined by the output of performing the machine learning process can be sent by the UE to the serving gNB. A message to take the action, or perform the action and then send a message to the gNB estimating the result of performing the action.

[0048] In some implementations, the UE does not propose the action to the gNB before performing the action. Instead, in such an embodiment, the UE performs the action and then notifies the serving gNB of the action it has taken. In this case, the gNB stores information about the action and estimates its potential impact on the cell UL performance. If the potential impact is not harmful, the gNB sends an ACK message to the UE, indicating that the UE can continue to perform the action; otherwise, the gNB sends a NACK message to the UE, requesting the UE to stop performing the action.

[0049] The improved techniques for implementing ML based actions described above have advantages over conventional approaches, such as providing the gNB with a level of control over machine learning based actions without requiring knowledge of the exact machine learning implementation in the UE.

[0050] As mentioned above, the improved signaling flows can be used in the two main scenarios described below. The proposed solution may rely on a priori ML capability exchange between the serving gNB and the UE, and may also include some configuration parameters.

[0051] In the first scenario, the UE requests permission from the serving gNB to perform the selected action and potentially also indicates its estimated result:

[0052] 1. The UE and gNB exchange information about capabilities for ML-based radio algorithms.

[0053] a. The UE requests from the gNB one or more ML-action related signaling configurations to be used during execution of one or more ML-based algorithms.

[0054] i. Use RRC signaling to send the request.

[0055] b. The UE receives ML-action related signaling configuration from the gNB, implicitly indicating confirmation that the UE can start using the corresponding ML-based algorithm to take actions (e.g., radio parameter settings).

[0056] i. Use RRC signaling to send ML-action related signaling configuration.

[0057] 2. UE activates ML-based algorithm.

[0058] 3. As a result of the active ML-based algorithm, the UE determines the action to be performed.

[0059] 4. The UE sends an “ML Action Request” message to the serving gNB.

[0060] a. Use RRC or MAC signaling, depending on the time constraints for action execution.

[0061] 5. The gNB evaluates the potential outcome of the requested action taking into account any of the following: available action requests / executions from other served UEs, QoS requirements, traffic conditions (load), UE radio / RF capabilities, etc.

[0062] a. If, after evaluation, it is determined that the requested action will not have any adverse impact on the selected network KPI (e.g., cell throughput degradation, increased physical resource block (PRB) utilization), the gNB sends an “ML Action ACK” message to the UE.

[0063] b. Otherwise, the gNB sends a “ML Action NACK” message to the UE.

[0064] c. Use RRC or MAC signaling, depending on the time constraints for the action execution.

[0065] 6. A UE that receives a "ML Action ACK" message from its serving gNB performs the action. A UE that receives a "ML Action NACK" message from its serving gNB does not perform the action.

[0066] a. In some implementations, the UE sends a “ML Action Result Indicator” message to the serving gNB to indicate the estimated local result of taking or not taking this action.

[0067] i. Use RRC signaling to send this message.

[0068] b. In some implementations, the gNB receiving the “ML Action Result Indicator” message stores the indicated impact,

[0069] And potentially also considering available action indicators from other served UEs to evaluate the global impact factor.

[0070] In the second scenario, the UE indicates to the serving gNB the actions performed and their potential estimation results.

[0071] 1. The UE and gNB exchange information about capabilities for ML-based radio algorithms.

[0072] a. The UE requests from the gNB one or more ML-action related signaling configurations to be used during execution of one or more ML-based algorithms.

[0073] i. Use RRC signaling to send the request.

[0074] b. The UE receives ML-action related signaling configuration from the gNB, implicitly indicating confirmation that the UE can start using the corresponding ML-based algorithm to take actions (e.g., radio parameter settings).

[0075] i. Use RRC signaling to send ML-action related signaling configuration.

[0076] 2. UE activates ML-based algorithm.

[0077] 3. As a result of the active ML-based algorithm, the UE determines the action to be performed.

[0078] 4. The UE performs this action without input from the gNB.

[0079] 5. The UE sends a “ML Action Result Indicator” message to the serving gNB to indicate the estimated local result of taking this action.

[0080] 6. The gNB evaluates the result of the UE ML action taking into account available action requests / executions from other served UEs, QoS requirements, traffic conditions (load), UE radio / RF capabilities, etc.

[0081] 7. The gNB sends a "ML Action ACK / NACK" message to the UE, indicating (at least) to continue (ACK) or stop (NACK) taking these actions.

[0082] a. Use RRC signaling.

[0083] 8. A UE that receives a “ML Action ACK / NACK” message from its serving gNB shall (at least) continue (ACK) or stop (NACK) taking those actions.

[0084] It should be noted that the first and second scenarios above specify new signaling elements (RRC, MAC) and new UE configuration structures. In addition, the second scenario allows for faster responses than the first scenario, but is less conservative in terms of network control.

[0085] Figure 2 is a sequence diagram illustrating a process 200 of determining actions to be taken when executing a machine learning algorithm corresponding to the first scenario described above.

[0086] At 201, in some implementations, the gNB and the UE are exchanging messages regarding machine learning radio resource management (RRM) capability signaling.

[0087] At 202, the UE requests from the gNB one or more signaling configurations to be used during execution of one or more ML-based algorithms.

[0088] ●Use RRC signaling.

[0089] ○ After the UE has transitioned from idle to connected mode, if the UE ML-based algorithm is applied in connected mode, or

[0090] ○ Before the UE transitions to idle / inactive mode, if the UE ML-based algorithm is applied in idle / inactive mode.

[0091] • For each requested ML-based algorithm, the signaling configuration information indicates at least the following items.

[0092] ○ ML-based algorithm ID - to be used by the gNB when aggregating information from several UEs and when sending ACK / NACK messages.

[0093] ■ This ID can be the ID of the ML based algorithm / function (eg HO, beam selection, PC, etc.).

[0094] ■In some implementations, this ID may also include a version number used for continuous updating of the model, as is done for federated learning solutions.

[0095] ■ In some implementations, the algorithm ID may specify the range of the outputs (and possibly inputs) of the ML algorithm without disclosing any implementation details.

[0096] ○ Choice between “UE requests action permission” or “UE sends action indicator” (i.e., first and second scenarios) - the same ML-based algorithm can be configured for both procedures (different IDs) to allow potential handover upon gNB request.

[0097] o The signaling channel to use, or the protocol layer to use (eg, RRC or MAC) (which is selected based on the time constraints for the action execution), and the expected parameters to be configured (information element ID, acceptable value range, etc.).

[0098] o A set of ML action IDs to be used on the configured signaling channels.

[0099] o A set of ML action result IDs corresponding to at least one UE KPI, which the UE uses to evaluate the local impact of ML based actions, e.g., throughput target, throughput with delay constraint, spectral efficiency, energy consumption, etc.

[0100] At 203, the UE receives a signaling configuration from the gNB, implicitly indicating a confirmation that the UE can start using the corresponding ML-based algorithm to take actions (e.g., radio parameter settings).

[0101] ●Use RRC signaling.

[0102] ○ After the UE has transitioned from idle to connected mode, if the ML-based algorithm is applied in connected mode, or

[0103] ○ Before the UE transitions to idle / inactive mode, if the ML based algorithm is applied in idle / inactive mode.

[0104] • The UE may apply the received configuration and disable all other ML based algorithms for which it has not requested or received a configuration.

[0105] • The configuration message may include different evaluation KPIs than requested by the UE.

[0106] At 204, the UE activates the ML algorithm, which generates an output.

[0107] At 205, the UE determines an action to take based on the output.

[0108] At 206, the UE sends an ML Action Request message to the serving gNB.

[0109] ●Use RRC or MAC signaling, as configured in 202 and 203.

[0110] ●The message includes at least the following items.

[0111] ○ ML-based algorithm ID from 202.

[0112] ○ The selected action (e.g., increase or decrease TX power, UE beam selection, etc.) and the associated ID.

[0113] ●In some implementations, the UE sends a ranked list of preferred actions, leaving it to the gNB to select the best action with respect to network KPI performance.

[0114] ● In some implementations, the ML architecture can be split and trained jointly between the UE and the gNB, in which case

[0115] Some of the results will be sent from the UE to the gNB, which is ultimately responsible for generating the best action for each UE.

[0116] At 207, the gNB evaluates the potential outcome of the requested action taking into account available action requests / executions from other served UEs, QoS requirements, traffic conditions (load), UE radio / RF capabilities, etc.

[0117] • This is understood as "admission control" for UE's ML based actions.

[0118] ●Historical data may be used as labeled data for training and testing at the gNB side.

[0119] At 208, the UE receives the "ML Action Response ACK" message, and if the requested action evaluated by the gNB will not have any adverse impact on the selected network KPI (e.g., cell throughput reduction, increase PRB utilization - which may be different from the UE KPI selected above), the gNB sends the "ML Action ACK" message to the UE. Otherwise, the gNB sends the UE a "ML Action NACK" message.

[0120] ● using RRC or MAC signaling, as configured in 202-203 and as used in 206,

[0121] ●The response message includes at least the following items.

[0122] ○ ML-based algorithm ID from 202.

[0123] ○ML-based action ID from 206.

[0124] ● In some implementations, the response message may include a valid time / period for the response (ACK or NACK),

[0125] The UE is expected to apply or not apply the action within this period.

[0126] At 209, after receiving the "ML Action ACK" message from its serving gNB, the UE performs the action.

[0127] • Information from the response message is applied to corresponding ML-based algorithms and ML-based actions.

[0128] ● In some implementations (where the UE sends a ranked list of preferred actions, and the gNB replies with the ID of the best action (from the gNB's perspective)), the UE can use the selected actions to adjust (e.g., retrain) its own ML-based algorithm. For example, the actions selected by the gNB become the labels for the data used as input features (and thus this pair becomes an additional training data point).

[0129] In some implementations, at 210, the UE sends a "ML Action Indicator" message to the serving gNB to indicate the estimated local result of taking or not taking this action.

[0130] ●Use RRC signaling.

[0131] • The UE estimates the local (UE-perceived) impact of the action on the configured KPIs from 202-203.

[0132] In some implementations, at 211, the gNB receiving the "ML action indicator" message stores the indicated impact and potentially also considers available action indicators from other served UEs to evaluate a global impact factor (e.g., the selected cell-based KPI).

[0133] The gNB processes and stores the received information (historical data) to allow adjustment of future “ML Action ACK / NACK” messages to the same UE.

[0134] o In some implementations, the 'learned' impact of the action may also be used in the configuration at 202-203.

[0135] ●The same additional inputs as 207 can be used.

[0136] Figure 3 is a sequence diagram illustrating a process 300 of determining actions to take when executing a machine learning algorithm. Process 300 differs from process 200 in two respects: process 300 allows multiple possible ML actions to be taken; these actions are represented in a preferred action list (PAL), and process 300 uses the PAL to enhance the training set for the ML-based process and uses the enhanced training set to retrain the ML-based process.

[0137] At 301, in some implementations, the gNB and the UE are exchanging messages regarding machine learning radio resource management (RRM) capability signaling.

[0138] At 302, the UE requests from the gNB one or more signaling configurations to be used during execution of one or more ML-based algorithms.

[0139] ●Use RRC signaling.

[0140] ○ After the UE has transitioned from idle to connected mode, if the UE ML-based algorithm is applied in connected mode, or

[0141] ○ Before the UE transitions to idle / inactive mode, if the UE ML-based algorithm is applied in idle / inactive mode.

[0142] • For each requested ML-based algorithm, the signaling configuration information indicates at least the following items.

[0143] ○ ML-based algorithm ID - to be used by the gNB when aggregating information from several UEs and when sending ACK / NACK messages.

[0144] ■ This ID can be the ID of the ML based algorithm / function (eg HO, beam selection, PC, etc.).

[0145] ■In some implementations, this ID may also include a version number used for continuous updating of the model, as is done for federated learning solutions.

[0146] ■ In some implementations, the algorithm ID may specify the range of the outputs (and possibly inputs) of the ML algorithm without disclosing any implementation details.

[0147] ○ Choice between “UE requests action permission” or “UE sends action indicator” (i.e., first and second scenarios) - the same ML-based algorithm can be configured for both procedures (different IDs) to allow potential handover upon gNB request.

[0148] o The signaling channel to use, or the protocol layer to use (eg, RRC or MAC) (which is selected based on the time constraints for the action execution), and the expected parameters to be configured (information element ID, acceptable value range, etc.).

[0149] o A set of ML action IDs to be used on the configured signaling channels.

[0150] o A set of ML action result IDs corresponding to at least one UE KPI, which the UE uses to evaluate the local impact of ML based actions, e.g., throughput target, throughput with delay constraint, spectral efficiency, energy consumption, etc.

[0151] At 303, the UE receives a signaling configuration from the gNB, implicitly indicating confirmation that the UE can start using the corresponding ML-based algorithm to take actions (e.g., radio parameter settings).

[0152] ●Use RRC signaling.

[0153] ○ After the UE has transitioned from idle to connected mode, if the ML-based algorithm is applied in connected mode, or

[0154] ○ Before the UE transitions to idle / inactive mode, if the ML based algorithm is applied in idle / inactive mode.

[0155] • The UE may apply the received configuration and disable all other ML based algorithms for which it has not requested or received a configuration.

[0156] • The configuration message may include different evaluation KPIs than requested by the UE.

[0157] At 304, the UE activates the ML algorithm, which generates an output.

[0158] At 305, the UE determines an action to take based on the output.

[0159] At 306, the UE sends a ML Action Request message including a Preferred Action List (PAL) to the serving gNB.

[0160] ●Use RRC or MAC signaling, as configured in 202 and 203.

[0161] ●The message includes at least the following items.

[0162] ○ ML-based algorithm ID from 202.

[0163] o PAL (e.g., increase or decrease TX power, UE beam selection, etc.) and associated ID.

[0164] ●By providing PAL, the UE allows the gNB to select the best action regarding network KPI performance.

[0165] ● In some implementations, the ML architecture can be split and trained jointly between the UE and the gNB, in which case

[0166] Some of the results will be sent from the UE to the gNB, which is ultimately responsible for generating the best action for each UE.

[0167] At 307, the gNB evaluates the potential outcome of each requested action in the PAL taking into account available action requests / executions from other served UEs, QoS requirements, traffic conditions (load), UE radio / RF capabilities, etc.

[0168] • This is understood as "admission control" for UE's ML based actions.

[0169] ●Historical data may be used as labeled data for training and testing at the gNB side.

[0170] At 308, the UE receives a corresponding "ML Action Response ACK" message for each action in the PAL, and if the action assessed by the gNB will not have any adverse impact on the selected network KPI (e.g., cell throughput reduction, increase PRB utilization - which may be different from the UE KPI selected above), the gNB sends a "ML Action ACK" message to the UE. Otherwise, the gNB sends a "ML Action NACK" message to the UE for the action.

[0171] ● Using RRC or MAC signaling, as configured in 202-203 and as used in 206.

[0172] ●The response message includes at least the following items.

[0173] ○ The ML-based algorithm ID from a 302, or

[0174] ○ML-based action ID from 306.

[0175] ● In some implementations, the response message may include a valid time / period for the response (ACK or NACK),

[0176] The UE is expected to apply or not apply the action within this period.

[0177] At 309, after receiving the "ML Action ACK" message for the k-th action from its serving gNB, the UE performs the k-th action (PAL(k)).

[0178] • Information from the response message is applied to corresponding ML-based algorithms and ML-based actions.

[0179] ●A STOP (NACK) message indicates that the UE should not apply the corresponding action until explicitly enabled by the gNB.

[0180] ● In some implementations, the gNB replies with the ID of the best action (from the gNB's perspective), and the UE can use the selected action to adjust (e.g., retrain) its own ML-based algorithm. For example, the action selected by the gNB becomes the label for the data used as an input feature (and thus this pair becomes an additional training data point).

[0181] ●When a distributed multi-agent Actor-Critic RL based algorithm is used at the UE and gNB, the network can additionally feed back RL-related parameters / variables to the UE, such as the estimated network-level reward.

[0182] In some implementations, at 310, the UE sends a "ML Action Indicator" message to the serving gNB to indicate the estimated local result of taking or not taking this action.

[0183] ●Use RRC signaling.

[0184] • The UE estimates the local (UE-perceived) impact of the action on the configured KPIs from 202-203.

[0185] In some implementations, at 311, the gNB receiving the "ML action indicator" message stores the indicated impact and potentially also considers available action indicators from other served UEs to evaluate a global impact factor (e.g., the selected cell-based KPI).

[0186] The gNB processes and stores the received information (historical data) to allow adjustment of future “ML Action ACK / NACK” messages to the same UE.

[0187] o In some implementations, the "learned" impact of the action may also be used in the configuration at 202-203.

[0188] ●The same additional inputs as 207 can be used.

[0189] At 312, the UE uses PAL(k) to enhance the training set. Once the network confirms PAL(k), it then becomes the label of the data set used to first generate the decision (i.e., at 305). In other words, the real data and the results of the ML block become additional training points, where the label has been reviewed by the network. By doing so, the training set (which initially only contains training data generated during the training process) is enhanced with training data generated during normal operation (i.e., after ML has been deployed and applied).

[0190] The UE retrains the ML-based algorithm using the enhanced training set at 313. By doing so, the ML model learns what the NW preference is, i.e., which action the network will prefer to select from the PAL.

[0191] Figure 4 is a sequence diagram illustrating a process 400 of determining actions to be taken when executing a machine learning algorithm corresponding to the second scenario described above.

[0192] At 401, in some implementations, the gNB and the UE are exchanging messages regarding machine learning radio resource management (RRM) capability signaling.

[0193] At 402, the UE requests from the gNB one or more signaling configurations to be used during execution of one or more ML-based algorithms.

[0194] ●Use RRC signaling.

[0195] ○ After the UE has transitioned from idle to connected mode, if the UE ML-based algorithm is applied in connected mode, or

[0196] ○ Before the UE transitions to idle / inactive mode, if the UE ML-based algorithm is applied in idle / inactive mode.

[0197] • For each requested ML-based algorithm, the signaling configuration information indicates at least the following items.

[0198] ○ ML-based algorithm ID - to be used by the gNB when aggregating information from several UEs and when sending ACK / NACK messages.

[0199] ■ This ID can be the ID of the ML based algorithm / function (eg HO, beam selection, PC, etc.).

[0200] ■In some implementations, this ID may also include a version number used for continuous updating of the model, as is done for federated learning solutions.

[0201] ■ In some implementations, the algorithm ID may specify the range of the outputs (and possibly inputs) of the ML algorithm without disclosing any implementation details.

[0202] ○ Choice between “UE requests action permission” or “UE sends action indicator” (i.e. first and second scenarios) - the same ML-based algorithm can be configured for both procedures (different IDs) to allow potential handover upon gNB request.

[0203] o The signaling channel to use, or the protocol layer to use (eg, RRC or MAC) (which is selected based on the time constraints for the action execution), and the expected parameters to be configured (information element ID, acceptable value range, etc.).

[0204] o A set of ML action IDs to be used on the configured signaling channels.

[0205] o A set of ML action result IDs corresponding to at least one UE KPI, which the UE uses to evaluate the local impact of ML based actions, e.g., throughput target, throughput with delay constraint, spectral efficiency, energy consumption, etc.

[0206] At 403, the UE receives a signaling configuration from the gNB, implicitly indicating confirmation that the UE can start using the corresponding ML-based algorithm to take actions (e.g., radio parameter settings).

[0207] ●Use RRC signaling.

[0208] ○ After the UE has transitioned from idle to connected mode, if the UE ML-based algorithm is applied in connected mode, or

[0209] ○ Before the UE transitions to idle / inactive mode, if the UE ML-based algorithm is applied in idle / inactive mode.

[0210] • The UE may apply the received configuration and disable all other ML-based algorithms for which it has not requested or received a configuration.

[0211] • The configuration message may include different evaluation KPIs than requested by the UE.

[0212] At 404, the UE activates the ML algorithm, which generates an output.

[0213] At 405, the UE determines an action to take based on the output.

[0214] At 406, the UE performs the action without first requesting permission from the gNB as in Scenario 1.

[0215] At 407, the UE sends a "ML Action Indicator" message to the serving gNB, including an indication of the estimated local result of taking the determined action.

[0216] ●Use RRC signaling.

[0217] ●The message includes at least the following items.

[0218] ○ ML-based algorithm ID from 402.

[0219] Action ID.

[0220] ○Action result ID.

[0221] ○ The UE may be able to learn from its actions and interactions with the RAN and provide this as additional “after action”

[0222] information.

[0223] At 408, the gNB evaluates the potential outcome of the requested action taking into account available action requests / executions from other served UEs, QoS requirements, traffic conditions (load), UE radio / RF capabilities, etc.

[0224] ●In some implementations, when the impact of the action is always detrimental to the gNB, the gNB may instruct the UE to switch to using the method from scenario 1.

[0225] • In some implementations, a validity time / period is provided for the response (ACK or NACK), during which the UE is expected to apply or not apply the action.

[0226] At 409, the UE receives a "ML Action Affecting ACK / NACK" message sent to the UE indicating (at least) to continue (ACK) or to stop (NACK) taking the action.

[0227] ●Use RRC signaling.

[0228] • The Continue (ACK) message instructs the UE to continue using the configured ML-based algorithm and to apply the actions that have been confirmed. The signaling from step 6 for this action may be relaxed so that it occurs with higher periodicity.

[0229] ●A STOP (NACK) message indicates that the UE should not apply the corresponding action until explicitly enabled by the gNB.

[0230] ●In some implementations, when a distributed multi-agent Actor-Critic RL based algorithm is used at the UE and gNB, the network may additionally feed back RL related parameters / variables to the UE, such as the estimated network-level reward.

[0231] ●In some implementations, the gNB instructs the UE to switch to a method for use scenario 1.

[0232] At 410, a UE that receives a "ML Action Affects ACK / NACK" message from its serving gNB will (at least) continue (ACK) or stop (NACK) taking those actions.

[0233] • When a Continue (ACK) message is received, the UE continues to use the configured ML-based algorithm and applies the actions that have been acknowledged. The signaling from step 6 for this action may be relaxed so that it occurs with higher periodicity.

[0234] ●When a STOP (NACK) message is received, the UE shall not apply the corresponding action until explicitly enabled by the gNB.

[0235] ●In some implementations, when a distributed multi-agent Actor-Critic RL based algorithm is used at the UE and gNB, the network may additionally provide RL related parameters / variables to the UE, such as the estimated network-level reward.

[0236] ●In some implementations, the gNB instructs the UE to switch to a method for use scenario 1.

[0237] In scenarios 1 and 2, following are the specific details of the case when the UE ML-based algorithm is applied in RRC idle / inactive mode.

[0238] • RRC connected mode configurations are stored and used separately, and they may apply different ML-based algorithms than those used in RRC idle / inactive mode.

[0239] • Whenever the UE enters RRC idle / inactive mode, the configuration information from 402-403 may be reused.

[0240] • The UL UE RRC message may be sent in a similar manner as the Tracking Area Update message.

[0241] ●Instead of using DL gNB RRC messages, PDCCH or broadcast information is used to transmit the "ML action ACK / NACK" or "ML action impact ACK / NACK" message (e.g., simplified to 1 bit).

[0242] Example 1-1: Figure 5 is a flow chart illustrating a process 500 of determining an action to be taken when executing a machine learning algorithm. Operation 510 includes executing at least one machine learning process by a user device in a wireless network to produce an output. Operation 520 includes determining, by the user device, an action to be performed based on the output produced by executing at least one machine learning process. Operation 530 includes sending, by a user device in a wireless network, a request to perform the action to a network node serving the user device. Operation 540 includes receiving, by the user device, a message from the network node indicating whether the user device can perform the action. Operation 550 includes, in response to the message indicating that the user device can perform the action, performing the action by the user device. Operation 560 includes, in response to the message indicating that the user device cannot perform the action, not performing the action.

[0243] Example 1-2: The example implementation according to Example 1-1 further includes: sending a request for at least one machine learning process signaling configuration to be used during the execution of at least one machine learning-based process; and receiving at least one machine learning action signaling configuration.

[0244] Example 1-3: An example implementation according to any of Examples 1-1 and 1-2, wherein a request for at least one machine learning action signaling configuration is sent via radio resource control signaling.

[0245] Example 1-4: An example implementation according to Example 1-3, wherein the user equipment performs at least one machine learning process in a connected state, and wherein radio resource control signaling is used to send in response to the user equipment transitioning from an idle state to a connected state.

[0246] Example 1-5: An example implementation according to Examples 1-3 to 1-4, wherein the user equipment performs at least one machine learning process in an idle or inactive state, and wherein radio resource control signaling is used to send in response to the user equipment transitioning from a connected state to an idle or inactive state.

[0247] Example 1-6: According to the example implementation described in Examples 1-2 to 1-5, further comprising: in response to a request for at least one machine learning process signaling configuration to be used during execution of at least one machine learning process, receiving at least one machine learning process signaling configuration from a network node, the at least one machine learning process signaling configuration indicating that the user equipment can perform at least one machine learning process.

[0248] Example 1-7: The example implementation of Examples 1-5 to 1-6 further includes: sending a message to the network node indicating an estimate of a result of performing the action.

[0249] Example 1-8: An example implementation according to any one of Examples 1-1 to 1-7, wherein each machine learning process of at least one machine learning process has a corresponding machine learning process identifier; and wherein the message indicating whether the user device can perform the action includes a reference to the corresponding machine learning process identifier that generated the action.

[0250] Example 1-9: An example implementation according to Example 1-8, wherein the corresponding machine learning process identifier includes a version number of at least one machine learning process.

[0251] Example 1-10: An example implementation according to Examples 1-8 to 1-9, wherein the corresponding machine learning process identifier includes a range of outputs of the at least one machine learning process.

[0252] Example 1-11: According to the example implementation described in Examples 1-1 to 1-10, it further includes: measuring the output of at least one machine learning process to generate process measurements.

[0253] Example 1-12: An example implementation according to Examples 1-10 to 1-11, wherein the process measurement corresponds to a key performance indicator (KPI), and wherein the method further includes: assigning a result identifier to the KPI, the result identifier identifying the result, and wherein the corresponding machine learning process identifier includes the result identifier.

[0254] Example 1-13: An example implementation according to any one of Examples 1-1 to 1-12, wherein the request to perform the action includes identifiers of multiple machine learning processes, which are arranged in order of values ​​of key performance indicators (KPIs).

[0255] Example 1-14: An example implementation according to Example 1-13, wherein at least one machine learning process signaling configuration includes a message indicating a machine learning process among multiple machine learning processes that can be performed by the user equipment.

[0256] Example 1-15: An example implementation according to Examples 1-1 to 1-14, wherein at least one machine learning process generates multiple actions, and wherein the request to perform the action includes corresponding identifiers of the multiple actions.

[0257] Example 1-16: An apparatus comprising components for performing the method according to any one of Examples 1-1 to 1-15.

[0258] Example 1-17: A computer program product comprising a non-transitory computer-readable storage medium and storing executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to perform a method according to any one of Examples 1-1 to 1-16.

[0259] Example 2-1: Figure 6 6 is a flow chart illustrating a process 600 of determining an action to be taken when executing a machine learning algorithm. Operation 610 includes receiving, by a network node of a wireless network from a user equipment, a request for an action to be performed by the user equipment, the network node serving the user equipment in the wireless network, the action being determined by the user equipment based on an output of at least one machine learning process. Operation 620 includes generating, by the network node, a corresponding prediction of a result of the user equipment performing the action on the wireless network. Operation 630 includes sending, by the network node, to the user equipment a message indicating whether the user equipment can perform the action based on the corresponding prediction.

[0260] Example 2-2: An example implementation according to Example 2-1, wherein the request includes a corresponding identifier for at least one machine learning process, and wherein the message includes a corresponding identifier for a corresponding prediction of a result of performing the action.

[0261] Example 2-3: An example implementation according to Example 2-1 or 2-2, wherein generating a corresponding prediction of the result of performing the action is based on at least one of the following: quality of service requirements, traffic conditions, user device radio capabilities, or requests from other user devices in the wireless network.

[0262] Example 2-4: The example implementation according to Example 2-3, wherein the message indicates that the user device can perform the action, and wherein the method further includes: receiving a message from the user device indicating an estimate of a result of performing the action.

[0263] Example 2-5: According to the example implementation described in Example 2-4, it further includes: storing an estimate of the result of using at least one machine learning process; and evaluating the impact of the estimate of the result of using at least one machine learning process.

[0264] Example 2-6: An apparatus comprising components for performing the method according to Example 2-1.

[0265] Example 2-7: A computer program product includes a non-transitory computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to perform the method according to Example 2-1.

[0266] Example 3-1: Figure 7 7 is a flow chart illustrating a process 700 of determining an action to be taken when executing a machine learning algorithm. Operation 710 includes executing, by a user device in a wireless network, at least one machine learning process to produce an output. Operation 720 includes determining, by the user device, an action to be performed based on the output produced by executing the at least one machine learning process. Operation 730 includes executing, by the user device, the action to produce execution of the action. Operation 740 includes sending, by the user device, a first message to a network node, the first message including an indication of an estimated local result of the execution of the action. Operation 750 includes receiving, by the user device, a second message from the network node, the second message indicating an impact of the execution of the action on the wireless network globally.

[0267] Example 3-2: The example implementation according to Example 3-1, wherein the second message indicates that the user equipment should not continue to perform the action, and wherein the method further includes: stopping performing the action.

[0268] Example 3-3: According to the example implementation of Example 3-2, the second message further indicates that the user equipment is to send a request to perform the action to a network node serving the user equipment before subsequent performance of the action.

[0269] Example 3-4: An apparatus comprising means for performing the method according to Example 3-1.

[0270] Example 3-5: A computer program product includes a non-transitory computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to perform the method according to Example 3-1.

[0271] Example 4-1: Figure 88 is a flow chart illustrating a process 800 of determining an action to be taken when executing a machine learning algorithm. Action 810 includes receiving, by a network node of a wireless network from a user device, the network node serving the user device in the wireless network, the first message indicating that the user device has performed an action to produce the execution of the action, the action being determined based on the output of the machine learning process, the first message also indicating an estimated local result of performing the action. Action 820 includes generating, by the network node, a corresponding prediction of a global result of the user device performing the action on the wireless network. Action 830 includes sending, by the network node to the user device, a second message based on the prediction of the global result of the user device performing the action on the wireless network, the second message indicating whether the user device should continue to perform the action.

[0272] Example 4-2: An apparatus comprising components for performing the method according to Example 4-1.

[0273] Example 4-3: A computer program product includes a non-transitory computer-readable storage medium and stores executable code, which, when executed by at least one data processing device, is configured to cause the at least one data processing device to perform the method according to Example 4-1.

[0274] The following example is a use case for scenario 1. The process defined in scenario 1 is applicable to solving the UE UL TPC parameter optimization problem, where the goal is to determine the best transmit power value per UE to maximize the UL / side link UE throughput without significantly affecting the overall UL cell performance. Reinforcement learning can be one of the preferred ML-based algorithms, and it is assumed that the RL agent will be deployed on the UE side to adjust the UE UL (or side link) transmit power based on local radio conditions. It is assumed that these power adjustments are combined with normal UL OLPC and UL CLPC mechanisms.

[0275] In this example, in 202-203, 302-303 or 402-403, the gNB and the UE agree on the type of ML-based algorithm: based on RL, using UE local measurements (RSRP) and antenna panel or beam selection information as input. For example, the RL algorithm is used to determine the application of three possible actions: "power increase 1dB" (ID=0), "power reduction 1dB" (ID=1), "no power adjustment" (ID=2).

[0276] Assume that scenario 1 is used in this example.

[0277] The UE shall autonomously determine the appropriate UL transmit power adjustment (in combination with OLPC and CLPC) and inform the serving gNB about its ML Action ID without applying the action.

[0278] The gNB uses information about the UE’s potential transmit power level and also uses knowledge from all other served UEs to estimate the cell UL performance (global impact).

[0279] ●If the gNB does not estimate any significant drop / degradation from the candidate UE ML action, it sends an ML Action ACK response to the UE.

[0280] ●The UE performs the action and adjusts its transmit power accordingly.

[0281] ●The serving gNB can evaluate the global impact of the UE power adjustment action, so it can also adjust the estimation algorithm used to decide on future similar ML actions (same UE or other UEs).

[0282] Variations on the above examples include using scenario 2, and / or using other types of UE ML algorithms (CNN / DNN / GAN / RNN).

[0283] The following example is a use case for scenario 2. The process defined in scenario 2 is applicable to solving the UE antenna panel selection optimization problem, where the goal is to determine the best transmit / receive per UE so that its link throughput is maximized. Reinforcement learning can be one of the preferred ML-based algorithms. It is assumed that one RL agent is deployed on the UE side to adjust the UE antenna panel (or beam) to be used based on the local radio conditions. Another RL agent is deployed on the gNB side to track the global performance in the cell related to the UE panel selection. This ML architecture allows collaboration with the gNB and fine-tuning of the UE antenna selection while maintaining UE autonomy.

[0284] Assume that scenario 2 is used in this example.

[0285] ●The UE will autonomously determine the antenna panel to be used, switch to the selected antenna panel (ML action), and notify the serving gNB about its ML action ID.

[0286] ●The gNB uses information on the selected antenna panel and, for example, also uses knowledge from all other served UEs to assess the cell UL interference (global impact).

[0287] ● If the gNB does not assess any significant drop / degradation from the UE ML actions taken, it sends an ML action impact ACK response to the UE, including an indication of the cell-level reward metric.

[0288] ○ The UE continues to use the antenna panel selection algorithm, uses the cell-level reward metric indication to adjust its own local reward function, and informs the gNB about its actions (potentially with higher periodicity to reduce signaling load).

[0289] ●If / when the gNB assesses a potential degradation in UL performance in the cell due to UE antenna panel selection, it sends an ML action affecting NACK response to the UE.

[0290] ○ The UE ceases to take actions that have been “rejected” by the serving gNB.

[0291] Variations on the above examples include using scenario 1, and / or using other types of UE ML algorithms (CNN / DNN / GAN / RNN).

[0292] Example abbreviation list:

[0293] BLER Block Error Rate

[0294] BS Base Station, a synonym for 5G gNB

[0295] CU Central Unit (5G CU-gNB)

[0296] DNN Deep Neural Network (5G DU-gNB)

[0297] DU Distributed Unit

[0298] KPI Key Performance Indicator

[0299] LA Link Adaptation

[0300] MCS Modulation and Coding Scheme

[0301] ML Machine Learning

[0302] NN Neural Network

[0303] PC Power Control

[0304] PDU Protocol Data Unit

[0305] RB Resource Block

[0306] RL

[0307] RSRP Reference Signal Received Power

[0308] SE Spectral Efficiency

[0309] TP Throughput

[0310] UL Uplink

[0311] Fig. 9900 is a block diagram of a wireless station (e.g., AP, BS, e / gNB, NB-IoT UE, UE, or user equipment) according to an example implementation. The wireless station 900 may include, for example, one or more RF (radio frequency) or wireless transceivers 902A, 902B, wherein each wireless transceiver includes a transmitter for sending signals (or data) and a receiver for receiving signals (or data). The wireless station also includes: a processor or control unit / entity (controller) 904 to execute instructions or software and control the sending and receiving of signals; and a memory 906 to store data and / or instructions.

[0312] The processor 904 may also make decisions or determinations, generate time slots, subframes, packets or messages for transmission, decode received time slots, subframes, packets or messages for further processing, and other tasks or functions described herein. The processor 904 (which may be a baseband processor) may, for example, generate messages, packets, frames or other signals for transmission via the wireless transceiver 902 (902A or 902B). The processor 904 may control the transmission of signals or messages through a wireless network, and may control the reception of signals or messages, etc., via a wireless network (e.g., after being down-converted by the wireless transceiver 902). The processor 904 may be programmable and may be capable of executing software or other instructions stored in a memory or other computer medium to perform the various tasks and functions described above, such as one or more of the tasks or methods described above. The processor 904 may, for example, be (or may include) hardware, programmable logic, a programmable processor that executes software or firmware, and / or any combination of these. Using other terms, the processor 904 and the transceiver 902 together may be considered, for example, as a wireless transmitter / receiver system.

[0313] In addition, refer to Fig. 9 , the controller (or processor) 908 can execute software and instructions, can provide overall control of the station 900, and can provide Fig. 9 Control of other systems not shown in the figure, such as controlling input / output devices (e.g., display, keyboard), and / or executing software for one or more applications that may be provided on the wireless station 900, such as, for example, an email program, audio / video applications, a word processor, an IP voice application, or other applications or software.

[0314] Additionally, a storage medium may be provided that includes stored instructions that, when executed by a controller or processor, may cause processor 904 or other controllers or processors to perform one or more of the functions or tasks described above.

[0315] According to another example implementation, the RF or wireless transceiver 902A / 902B may receive signals or data, and / or may transmit or send signals or data. The processor 904 (and possibly the transceiver 902A / 902B) may control the RF or wireless transceiver 902A or 902B to receive, send, broadcast or transmit signals or data.

[0316] However, the embodiments are not limited to the systems given as examples, but those skilled in the art may apply the solution to other communication systems. Another example of a suitable communication system is the 5G concept. It is assumed that the network architecture in 5G is very similar to that of Advanced LTE. 5G uses multiple-input multiple-output (MIMO) antennas, has more base stations or nodes than LTE (the so-called small cell concept), including macro sites that operate in conjunction with small stations, and may also use various radio technologies to achieve better coverage and enhanced data rates.

[0317] It should be understood that future networks will likely use network function virtualization (NFV), a network architecture concept that proposes virtualizing network node functions into "building blocks" or entities that can be operationally connected or linked together to provide services. A virtualized network function (VNF) may include one or more virtual machines that use standard or general-purpose servers rather than custom hardware to run computer program code. Cloud computing or data storage may also be used. In radio communications, this may mean that node operations may be performed at least in part in a server, host, or node that is operationally coupled to a remote radio head. Node operations will also be distributed among multiple servers, nodes, or hosts. It should also be understood that the distribution of labor between core network operations and base station operations may be different from that of LTE, or even nonexistent.

[0318] The implementation of the various techniques described herein may be implemented in digital electronic circuits, or in computer hardware, firmware, software, or in a combination thereof. The implementation may be implemented as a computer program product, i.e., a computer program tangibly embodied in an information carrier (e.g., in a machine-readable storage device or in a propagated signal) to be executed by a data processing device (e.g., a programmable processor, a computer, or multiple computers) or to control the operation of a data processing device (e.g., a programmable processor, a computer, or multiple computers). The implementation may also be provided on a computer-readable medium or a computer-readable storage medium, which may be a non-transitory medium. The implementation of the various techniques may also include an implementation provided via a transient signal or medium, and / or a program and / or software implementation that can be downloaded via the Internet or other networks (wired networks and / or wireless networks). In addition, the implementation may be provided via machine type communication (MTC) or via the Internet of Things (IoT).

[0319] A computer program may be in source code form, object code form, or some intermediate form, and it may be stored in some carrier, distribution medium, or computer readable medium, which may be any entity or device capable of carrying the program. Such carriers include, for example, recording media, computer memory, read-only memory, optoelectronic and / or electrical carrier signals, telecommunication signals, and software distribution packages. Depending on the processing power required, a computer program may be executed in a single electronic digital computer, or it may be distributed among multiple computers.

[0320] In addition, the implementation of the various technologies described herein may use a cyber-physical system (CPS) (a system of collaborative computing elements that control physical entities). CPS can enable the implementation and utilization of a large number of interconnected ICT devices (sensors, actuators, processors, microcontrollers, etc.) embedded in physical objects at different locations. Mobile cyber-physical systems (where the physical system in question has inherent mobility) are a subcategory of cyber-physical systems. Examples of mobile physical systems include mobile robots and electronic devices transported by humans or animals. The popularity of smartphones has increased people's interest in the field of mobile cyber-physical systems. Therefore, the various technical implementations described herein can be provided via one or more of these technologies.

[0321] Computer programs (such as the above-mentioned computer programs) can be written in any form of programming language (including compiled or interpreted languages) and can be deployed in any form, including being deployed as a stand-alone program or being deployed as a module, component, subroutine, or other unit or part thereof suitable for use in a computing environment. The computer program can be deployed to be executed on one computer or multiple computers at one site, or distributed between multiple sites and interconnected by a communication network.

[0322] The method steps may be performed by one or more programmable processors executing a computer program or computer program portions to perform functions by operating on input data and generating output. The method steps may also be performed by, and the apparatus may be implemented as, special purpose logic circuitry, e.g., an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).

[0323] As examples, processors suitable for executing computer programs include general and special purpose microprocessors, and any one or more processors of any kind of digital computer, chip or chipset. Typically, the processor will receive instructions and data from a read-only memory or a random access memory or both. The elements of a computer may include at least one processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or may be operatively coupled to the mass storage device to receive data from it or send data to it or both. Information carriers suitable for embodying computer program instructions and data include all forms of non-volatile memory, for example, including semiconductor memory devices, such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CDROM and DVD-ROM disks. The processor and memory may be supplemented by or incorporated into dedicated logic circuits.

[0324] To provide interaction with a user, implementations may be implemented on a computer having a display device (e.g., a cathode ray tube (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; a user interface (such as a keyboard and a pointing device, such as a mouse or trackball) through which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input.

[0325] The implementation may be implemented in a computing system that includes a back-end component (e.g., a data server), or includes a middleware component (e.g., an application server), or includes a front-end component (e.g., a client computer with a graphical user interface or a web browser through which a user can interact with the implementation), or any combination of such back-end, middleware, or front-end components. The components may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.

[0326] While certain features of the described implementations have been illustrated as described herein, many modifications, substitutions, changes, and equivalents will now occur to those skilled in the art. It should therefore be understood that the appended claims are intended to cover all such modifications and changes that are contemplated in the various embodiments.

Claims

1. A device comprising: at least one processor; as well as at least one memory including computer program code; The at least one memory and the computer program code are configured to cause the apparatus to at least: performing at least one machine learning process by a user device in the wireless network to generate an output; determining, by the user device, an action to be performed based on an output generated by executing the at least one machine learning process; A user equipment in a wireless network sends a request for executing the action to a network node serving the user equipment; receiving, by the user equipment, from the network node, a message indicating whether the user equipment can perform the action; In response to the message indicating that the user equipment can perform the action, the user equipment performs the action; as well as In response to the message indicating that the user equipment may not perform the action, not performing the action.

2. The device according to claim 1, wherein: The at least one memory and the computer program code are further configured to cause the apparatus to at least: sending a request for at least one machine learning process signaling configuration to be used during execution of the at least one machine learning based process; as well as Receive at least one of the machine learning action signaling configurations.

3. The device according to claim 1, wherein: A request for at least one machine learning action signaling configuration is sent via radio resource control signaling.

4. The device according to claim 3, wherein: The user equipment performs the at least one machine learning process in a connected state, and The radio resource control signaling is sent in response to the user equipment switching from an idle state to a connected state.

5. The device according to claim 4, wherein: The at least one memory and the computer program code are further configured to cause the apparatus to at least: A message is sent to the network node indicating an estimate of a result of performing the action.

6. The device according to claim 3, wherein: The user equipment performs the at least one machine learning process in an idle or inactive state, and The radio resource control signaling is sent in response to the user equipment switching from a connected state to an idle or inactive state.

7. The device according to claim 2, wherein: The at least one memory and the computer program code are further configured to cause the apparatus to at least: In response to a request for at least one machine learning process signaling configuration to be used during execution of the at least one machine learning process, the at least one machine learning process signaling configuration is received from the network node, and the at least one machine learning process signaling configuration indicates that the user equipment can perform the at least one machine learning process.

8. The device according to claim 1, wherein: Each of the at least one machine learning process has a corresponding machine learning process identifier; Wherein, the message indicating whether the user device can perform the action includes a reference to the corresponding machine learning process identifier that generated the action.

9. The device according to claim 8, wherein: The corresponding machine learning process identifier includes a version number of the at least one machine learning process.

10. The device according to claim 8, wherein: The corresponding machine learning process identifier includes a scope of an output of the at least one machine learning process.

11. The device according to claim 1, wherein: The at least one memory and the computer program code are further configured to cause the apparatus to at least: The output of the at least one machine learning process is measured to produce process measurements.

12. The device according to claim 11, wherein Each of the at least one machine learning process has a corresponding machine learning process identifier; Wherein, the process measurement corresponds to a key performance indicator KPI, Wherein, the at least one memory and the computer program code are further configured to cause the apparatus to at least: assigning a result identifier to the KPI, the result identifier identifying the result, and Wherein, the corresponding machine learning process identifier includes the result identifier.

13. The device according to claim 1, wherein: The request to perform the action includes identifiers of multiple machine learning processes, and the identifiers are arranged in the order of values ​​of key performance indicators KPI.

14. The device according to claim 13, wherein: The at least one machine learning process signaling configuration includes a message indicating a machine learning process among the multiple machine learning processes that the user equipment can perform.

15. The device according to claim 1, wherein: The at least one machine learning process generates a plurality of actions, and The request to perform the action includes corresponding identifiers of the multiple actions.

16. A method comprising: performing at least one machine learning process by a user device in the wireless network to generate an output; determining, by the user device, an action to be performed based on an output generated by executing the at least one machine learning process; A user equipment in a wireless network sends a request for executing the action to a network node serving the user equipment; receiving, by the user equipment, from the network node, a message indicating whether the user equipment can perform the action; In response to the message indicating that the user equipment can perform the action, the user equipment performs the action; as well as In response to the message indicating that the user equipment may not perform the action, not performing the action.

17. An apparatus comprising: at least one processor; as well as at least one memory including computer program code; The at least one memory and the computer program code are configured to cause the apparatus to at least: receiving, by a network node of a wireless network serving user equipment in the wireless network, a request from a user equipment for an action to be performed by the user equipment, the action being determined by the user equipment based on an output of at least one machine learning process; generating, by the network node, a corresponding prediction of a result of the user equipment performing the action on the wireless network; as well as Based on the corresponding prediction, a message is sent by the network node to the user equipment indicating whether the user equipment can perform the action.

18. The device according to claim 17, wherein: The request includes a corresponding identifier for the at least one machine learning process, and Therein, the message comprises a respective identifier of the respective prediction of the outcome of performing the action.

19. The device according to claim 17, wherein: Generating the corresponding prediction of the result of performing the action is based on at least one of: quality of service requirements, traffic conditions, user equipment radio capabilities, or requests from other user equipment in the wireless network.

20. The device according to claim 19, wherein The message indicates that the user equipment can perform the action, and Wherein, the at least one memory and the computer program code are further configured to cause the apparatus to at least: A message is received from the user equipment indicating an estimate of a result of performing the action.

21. The device according to claim 20, wherein: The at least one memory and the computer program code are further configured to cause the apparatus to at least: storing an estimate of a result of employing the at least one machine learning process; and Evaluate the impact of the estimate of the results of employing the at least one machine learning process.

22. A method comprising: receiving, by a network node of a wireless network serving user equipment in the wireless network, a request from a user equipment for an action to be performed by the user equipment, the action being determined by the user equipment based on an output of at least one machine learning process; generating, by the network node, a corresponding prediction of a result of the user equipment performing the action on the wireless network; as well as Based on the corresponding prediction, a message is sent by the network node to the user equipment indicating whether the user equipment can perform the action.

23. An apparatus comprising: at least one processor; as well as at least one memory including computer program code; The at least one memory and the computer program code are configured to cause the apparatus to at least: performing at least one machine learning process by a user device in the wireless network to generate an output; determining, by the user device, an action to be performed based on an output generated by executing the at least one machine learning process; executing the action by the user device to cause execution of the action; sending, by the user equipment, a first message to a network node, the first message comprising an indication of an estimated local result of performance of the action; as well as A second message is received by the user equipment from the network node, the second message indicating the impact of the execution of the action on the wireless network globally.

24. The device according to claim 23, wherein: The second message indicates that the user equipment should not continue to perform the action, and Wherein, the at least one memory and the computer program code are further configured to cause the apparatus to at least: Stops performing the action.

25. The device according to claim 24, wherein: The second message further instructs the user equipment to send a request to a network node serving the user equipment to perform the action prior to subsequent performance of the action.

26. A method comprising: performing at least one machine learning process by a user device in the wireless network to generate an output; determining, by the user device, an action to be performed based on an output generated by executing the at least one machine learning process; executing the action by the user device to cause execution of the action; sending, by the user equipment, a first message to a network node, the first message comprising an indication of an estimated local result of performance of the action; as well as A second message is received by the user equipment from the network node, the second message indicating the impact of the execution of the action on the wireless network globally.

27. An apparatus comprising: at least one processor; as well as at least one memory including computer program code; The at least one memory and the computer program code are configured to cause the apparatus to at least: receiving, by a network node of a wireless network from a user equipment, the network node serving the user equipment in the wireless network, the first message indicating that the user equipment has performed an action to produce execution of the action, the action being determined based on an output of a machine learning process, the first message also indicating an estimated local result of the execution of the action; generating, by the network node, a corresponding prediction of a global outcome of the user equipment performing the action on the wireless network; as well as Based on the prediction of the global result of the user equipment performing the action on the wireless network, a second message is sent by the network node to the user equipment, the second message indicating whether the user equipment should continue to perform the action.

28. A method comprising: receiving, by a network node of a wireless network from a user equipment, the network node serving the user equipment in the wireless network, the first message indicating that the user equipment has performed an action to produce execution of the action, the action being determined based on an output of a machine learning process, the first message also indicating an estimated local result of the execution of the action; generating, by the network node, a corresponding prediction of a global outcome of the user equipment performing the action on the wireless network; as well as Based on the prediction of the global result of the user equipment performing the action on the wireless network, a second message is sent by the network node to the user equipment, the second message indicating whether the user equipment should continue to perform the action.

29. A computer program product comprising a non-transitory computer-readable storage medium and storing executable code, which when executed by at least one data processing device is configured to cause the at least one data processing device to perform the method according to claim 16.

30. An apparatus comprising means for performing the method of claim 16.

Images