Procedure to validate a beam prediction model

Abstract

To validate accuracy of a beam prediction model in a second apparatus, a first apparatus receives from the second apparatus information indicating N strongest predicted beams output by the beam prediction model. The first apparatus further obtains information on M strongest beams at the second apparatus. Then the first apparatus determines whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validates accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams. N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M.

Description

[0001] DESCRIPTION

[0002] TITLE

[0003] PROCEDURE TO VALIDATE A BEAM PREDICTION MODEL

[0004] TECHNICAL FIELD

[0005] Various example embodiments relate to communication systems.

[0006] BACKGROUND

[0007] Wireless communication systems are under constant development. New applications, use cases and industry verticals are to be envisaged. Beamforming and beam management is predicted to become more complex due to factors such as user mobility, a higher number of antennas, and adoption of elevated frequencies. Artificial intelligence, specifically machine learning, may provide an efficient tool for beam management, for example by means of a beam prediction model inferred or trained in a user equipment, or corresponding device. A solution enabling to determine whether such a beam prediction model is accurate enough is desired.

[0008] SUMMARY

[0009] The independent claims define the scope, and different embodiments are defined in dependent claims.

[0010] According to an aspect there is provided a first apparatus comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the first apparatus at least to: receive from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtain information on M strongest beams at the second apparatus; determine whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validate accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams; wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M.

[0011] In an embodiment, combinable with the aspect and other embodiments, the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to obtain the information on M strongest beams at the second apparatus by receiving from the second apparatus in the information further information indicating M strongest measured beams or by deriving the information on M strongest beams from a test set up configuration.

[0012] In an embodiment, combinable with the aspect and other embodiments, the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to: receive, prior to the information, from the second apparatus an indication whether the second apparatus is performing inference or training of the beam prediction model; transmit, when the second apparatus is performing inference or training of the beam prediction model, to the second apparatus configuration information to obtain measurement results on a first set of beams, and configuration information to obtain predictions using as input measurement result on a second set of beams.

[0013] In an embodiment, combinable with the aspect and other embodiments, the second set of beams is a subset of the first set of beams.

[0014] In an embodiment, combinable with the aspect and other embodiments, the second set of beams comprises different beams than the first set of beams.

[0015] In an embodiment, combinable with the aspect and other embodiments, the first set of beams comprises a first number of beams, and the second set of beams comprises a second number of beams, which is at least a quarter of the first number.

[0016] In an embodiment, combinable with the aspect and other embodiments, the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to: transmit, prior to receiving the information, to the second apparatus a command to switch to an artificial intelligence / machine learning mode based functionality, and / or a request for reporting at least N strongest predicted beams.

[0017] In an embodiment, combinable with the aspect and other embodiments, the information indicating a beam is a beam identifier.

[0018] In an embodiment, combinable with the aspect and other embodiments, the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to: transmit to the second apparatus information indicating, whether the accuracy of the beam prediction model is validated.

[0019] According to an aspect there is provided a second apparatus comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: perform inference or training to a beam prediction model outputting X strongest predicted beams; obtain measurement results on beams; input the measurement results to the beam prediction model; transmit to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

[0020] In an embodiment, combinable with the aspect and other embodiments, the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the second apparatus at least to: receive from the first apparatus configuration information to obtain measurement results on a first set of beams, and configuration information to obtain predictions using as input measurement result on a second set of beams; input the measurement results on the second set of beams to the beam prediction model; and transmit to the first apparatus further information indicating M strongest measured beams at the second apparatus, wherein M is a positive integer which is smaller than or equal to N.

[0021] In an embodiment, combinable with the aspect and other embodiments, the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the second apparatus at least to: transmit to the first apparatus at least an indication whether the second apparatus is performing inference or training of the beam prediction model.

[0022] In an embodiment, combinable with the aspect and other embodiments, the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the second apparatus at least to: receive from the first apparatus a command to switch to artificial intelligence / machine learning mode based functionality; transmit, after receiving the command and when in the artificial intelligence / machine learning mode based functionality, to the first apparatus information confirming that the second apparatus is in the artificial intelligence / machine learning mode based functionality.

[0023] According to an aspect there is provided a method comprising at least: receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtaining information on M strongest beams at the second apparatus; determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams; wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M.

[0024] According to an aspect there is provided a method comprising at least: performing inference or training to a beam prediction model outputting X strongest predicted beams; obtaining measurement results on beams; inputting the measurement results to the beam prediction model; transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

[0025] According to an aspect there is provided a computer readable medium comprising instructions which, when executed by an apparatus, cause the apparatus to perform at least one of a first process or a second process, wherein the first process comprises at least: receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtaining information on M strongest beams at the second apparatus; determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams; wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M, and wherein the second process comprises at least: performing inference or training to a beam prediction model outputting X strongest predicted beams; obtaining measurement results on beams; inputting the measurement results to the beam prediction model; transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

[0026] According to an aspect there is provided a non-transitory computer readable medium comprising instructions which, when executed by an apparatus, cause the apparatus to perform at least one of a first process or a second process, wherein the first process comprises at least: receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtaining information on M strongest beams at the second apparatus; determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams; wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M, and wherein the second process comprises at least: performing inference or training to a beam prediction model outputting X strongest predicted beams; obtaining measurement results on beams; inputting the measurement results to the beam prediction model; transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

[0027] According to an aspect there is provided a computer program comprising instructions, which, when executed by an apparatus, cause the apparatus to perform at least one of a first process or a second process, wherein the first process comprises at least: receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtaining information on M strongest beams at the second apparatus; determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams, wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M, and wherein the second process comprises at least: performing inference or training to a beam prediction model outputting X strongest predicted beams; obtaining measurement results on beams; inputting the measurement results to the beam prediction model; transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

[0028] BRIEF DESCRIPTION OF DRAWINGS

[0029] Embodiments are described below, by way of example only, with reference to the accompanying drawings, in which

[0030] Fig. 1 illustrates an exemplified high-level system architecture;

[0031] Fig. 2 illustrates an example functionality of an apparatus;

[0032] Fig. 3 illustrates an example functionality of a device;

[0033] Fig. 4 illustrates an example of information exchange;

[0034] Fig. 5 is a schematic block diagram;

[0035] Fig. 6 is a schematic block diagram; and

[0036] Fig. 7 is a schematic block diagram.

[0037] DETAILED DESCRIPTION OF SOME EMBODIMENTS

[0038] The following embodiments are only presented as examples. Although the specification may refer to “an”, “one”, or “some” embodiment(s) and / or example(s) in several locations, this does not necessarily mean that each such reference is to the same embodiment(s) or example(s), or that a particular feature only applies to a single embodiment and / or single example. Single features of different embodiments and / or examples may also be combined to provide other embodiments and / or examples. Furthermore, words “comprising” and “including” should be understood as not limiting the described embodiments to consist of only those features that have been mentioned and such embodiments may contain also features / structures that have not been specifically mentioned. Further, although terms including ordinal numbers, such as “first”, “second”, etc., may be used for describing various elements, the elements are not restricted by the terms. The terms are used merely for the purpose of distinguishing an element from other elements. For example, a first apparatus could be termed an apparatus or a second apparatus, and correspondingly a second apparatus could be termed an apparatus or a first apparatus without departing from the scope of the present disclosure.

[0039] 5G- Advanced, and beyond future wireless networks aim to support a large variety of services, use cases and industrial verticals, for example unmanned mobility with fully autonomous connected vehicles, other vehicle-to-every thing (V2X) services, or smart environment, e.g. smart industry, smart power grid, or smart city, just to name few examples. To provide variety of services with different requirements, such as enhanced mobile broadband, ultrareliable low latency communication, massive machine type communication, wireless networks are envisaged to adopt network slicing, flexible decentralized and / or distributed computing systems and ubiquitous computing, with local spectrum licensing, spectrum sharing, infrastructure sharing, and intelligent automated management underpinned by mobile edge computing, artificial intelligence, for example machine learning, based tools, cloudification and blockchain technologies. For example, in the network slicing multiple independent and dedicated network slice instances may be created within the same infrastructure to run services that have different requirements on latency, reliability, throughput and mobility. In addition to the above listed features, 6G (sixth generation) networks are expected to adopt short-packet communication, for example. Key features of 6G will include intelligent connected management and control functions, programmability, integrated sensing and communication, reduction of energy footprint, trustworthy infrastructure, scalability and affordability. In addition to these, 6G is also targeting new use cases covering the integration of localization and sensing capabilities into system definition to unifying user experience across physical and digital worlds.

[0040] Various techniques described herein may also be applied to a cyber-physical system (CPS) (a system of collaborating computational elements controlling physical entities). CPS may enable the implementation and exploitation of massive amounts of interconnected ICT devices (sensors, actuators, processors microcontrollers, etc.) embedded in physical objects at different locations. Mobile cyber physical systems, in which the physical system in question has inherent mobility, are a subcategory of cyber-physical systems. Examples of mobile physical systems include mobile robotics and electronics transported by humans or animals.

[0041] It is envisaged that artificial intelligence, Al, / machine learning, ML, based models will be used to improve performance of at least some air-interface functions, providing an AI / ML-enabled radio interface. For example, to improve performance of beam selection related processes, different beam prediction models, that output information, for example identifying information, on strongest beams (best beams) may be used. Herein different examples of a test mechanism to test beam prediction models are discussed. The examples are described herein using principles and terminology of 5G (fifth generation) without limiting the examples, and the terminology used to the 5G. A person skilled in the art may apply the solutions and examples to other communication systems, for example beyond 5G, e.g. 6G, 7G, provided with necessary properties.

[0042] Fig. 1 illustrates an exemplified extremely high-level network architecture only showing some functional entities, all being logical units, whose implementation may differ from what is shown. The connections shown in Fig. 1 are logical connections; the actual physical connections may be different.

[0043] Referring to Fig. 1, a wireless network 100, or a system comprising wireless networks, comprises device components 101 for device functionalities in device domain, access network components 102 for access network functionalities in access network domain, core network components 103 for core network functionalities in core network domain, and data network components 104 for data network functionalities in data network domain.

[0044] A device component 101 may be any electrical device, or apparatus 110, connectable to an access network and configurable to be in a wireless connection on one or more communication channels 122, including one or more control channels, with an access network component 102, e.g. an access network apparatus 120, providing a cell 121, for example. The physical link from the device component 101 to the access network component 102 towards a core network component 103 is called an uplink or a reverse link and the physical link to the device component is called a downlink or a forward link. By way of example rather than limitation, the device component 101 may referred to as a served apparatus, a downlink apparatus, a mobile apparatus, a terminal device, a communication device, a user equipment (UE), a subscriber station (SS), a portable subscriber station, a mobile station (MS), or an access terminal (AT). A non-limiting lists of examples of the device component 101, or what the device component 101 may comprise or be comprised in, include a mobile phone, a cellular phone, a smart phone, a voice over internet protocol (VoIP) phone, a wireless local loop phone, a device using a wireless modem, a portable computer, a desktop computer, laptop-embedded equipment (LEE), laptop-mounted equipment (LME), a smart device, a multimedia device, an image capture terminal device, such as a digital camera, a gaming terminal device, a music storage and playback appliance, a drone, a vehicle, an automated guided vehicle, an autonomous connected vehicle, a vehicle-mounted wireless terminal device, a wireless endpoint, an internet of things device, an industrial internet of things device, a device operating in an industrial and / or an automated processing chain contexts, a consumer electronics device, a consumer internet of things device, a mobile robot, a mobile robot arm, a sensor, a surveillance camera, an eHealth related device, a medical monitoring device, a medical device, for example for remote surgery, a wearable device, such as a smart watch, a smart ring, a head-mounted display (HMD), an on- person device, etc. The device component may also be part of a group of device components seen as one device component, i.e. one mobile apparatus, by the wireless network.

[0045] An access network domain may be based on any kind of an access network, such as a cellular access network, for example 5G network, 5G-Advanced network, 6G network, etc., a non-terrestrial network, a legacy cellular radio access network, for example 4G or older generation network, or a non-cellular access network, for example a wireless local area network, or any combination thereof. To provide the wireless access, the access network comprises access network components 102, such as access network apparatuses 120, or access devices. An access device component 102 may provide one or more cells 121, possibly with different cell accessibility per a cell, but a cell is provided by one access device. However, there may be overlapping cells, for example a macro cell provided by an access device operating in co-operation of access nodes providing smaller cells, such as micro-, femto- or picocells, which overlap at least partly within the macro cell. There are a wide variety of access network components 102. A non-limiting lists of examples of the access network component 102, or what the access network component 102 may comprise or be comprised in, include different types of base stations, such as eNBs, gNBs, split gNBs, transmission-reception points, network- controlled repeaters, nodes operationally coupled to one or more remote radio heads, satellites, donor nodes in integrated access and backhaul (IAB), fixed IAB nodes, mobile IAB nodes mounted on vehicles, for example, etc. At least some of the apparatuses in the access network may provide an abstraction platform to separate abstractions of network functions from the processing hardware.

[0046] Further, it should be noted that some of the components may be multi-domain components. For example a device component 101 may also provide services to other device components, i.e. operate also as an access network component 102, for example be a relay node, or a mobile IAB node, or a mobile termination part in an IAB node. Hence, herein term mobile apparatus is used for device components, or device component functionality in a multi-domain component and term access network apparatus is used for access network components or access network component functionality in a multi-domain component. The core network components 103 form one or more core networks. A core network may be based on a non-standalone core network, for example an LTE-based network, or a standalone access network, for example a 5G core network. However, it should be appreciated that the core network, and the core network components 103, may use any technology that enable network services to be delivered between devices and data networks.

[0047] A data network may be any network, like the internet, an intranet, a wide area network, etc. Different remote monitoring and / or data collection services for different use cases may be reached via the data network and the data network components 104.

[0048] In the illustrated example of Fig. 1, a detail illustrating a test environment 105, or test arrangement, or test architecture, comprises a first apparatus 120 and a second apparatus 110. The first apparatus may 120 may be an access network component, and the second apparatus 110 may be a device network component, or both apparatuses may be access network components, or both apparatuses may be device network components.

[0049] The second apparatus 110 may be configured to transmit and receive data and different reference signals over a plurality of beams 122 (only few beams being illustrated). A beam represents a resource, for example a channel state information reference signal, CSI-RS, resource, or a synchronization signal block, SSB, resource. A beam may be a narrow beam, or a wide beam, which may overlap one or more narrow beams. Beams are separable from one another by means of information indicating a beam, e.g., by means of identifying information. For example a beam may have an identifier, that may be used in reporting to identify the beam whose value(s) are reported. The identifier may be an index value, such as a channel state information reference signal resource index, CRI, value or a synchronized signal block resource index, SSBRI, value. It should be appreciated that any kind of an identifier may be used. Further, other kind of information than identifying information may be used as the information indicating a beam.

[0050] In the illustrated example of Fig. 1, the second apparatus 110 comprises at least one beam prediction model (P-M) 111, and is configured to perform training or inference of the at least one model. A beam prediction model is an AI / ML model, for example a deep learning based model. The beam prediction model 111 may be a spatial beam prediction model, or a time-domain beam prediction model (temporal beam prediction model), or a spatial and timedomain beam prediction model. The spatial beam prediction aim to predict strongest transmis- sion / reception beams in different spatial locations. The time-domain beam prediction aim to predict the most likely beam to use for next time instants. Input to a beam prediction model may be channel quality related measurement results, for example reference signal received power measurement results. However, to test accuracy of the beam prediction model, the detailed structure of the beam prediction model and details on how the training or inference of said model is performed, bear no significance, and hence are not discussed in more detail herein.

[0051] The first apparatus 120, for example a test equipment (TE), or an access network component configured to contain test equipment functionality may be preconfigured with a test configuration, or the test configuration may be transmitted from a service and orchestration management platform, for example, to the apparatus, or part of the test configuration may be preconfigured and part received. The test configuration may comprise a test set up defining e.g. beams, transmission power of beams, accuracy criteria, etc. During a test phase, the first apparatus will determine, whether to validate accuracy of the beam prediction model, as will be described in more detail below. When the first apparatus 120 is an access network component, or any network node configured with the test equipment functionality in a real network environment, it is possible to avoid that the second apparatus 110, or more precisely, a beam prediction model in the second apparatus, performs poorly in the field even though the test in a separate test environment is passed. However, a separate test environment may be used. In a further implementation, a separate test environment is initially used, to initially validate accuracy of the model, and then the network environment is used to determine, whether to validate accuracy of the beam prediction model further training / inference.

[0052] Fig. 2 illustrates an example functionality of an apparatus, for example the first apparatus 120 illustrated in Fig. 1, configured to test, or determine, whether a beam prediction model, or shortly a model, in another apparatus, second apparatus, can predict N strongest beams accurately enough.

[0053] Referring to Fig. 2, the apparatus obtains (block 201) information on M strongest beams at the second apparatus. In other words, the apparatus obtains information on M strongest received beams. In one implementation, the apparatus obtains the information by receiving from the second apparatus information indicating M strongest measured beams. The apparatus and / or another access network component may transmit reference signals over a plurality of beams so that the second apparatus can obtain measurement results to determine M strongest measured beams. In another implementation, the apparatus may obtain the information on M strongest beams by deriving the information from the test set up.

[0054] Further, the apparatus receives (block 202) from the second apparatus information indicating N strongest predicted beams output by a beam prediction model. In the illustrated example the accuracy criteria is that Top-K beams (K strongest beams, wherein K can be one or more) are included in Top-N predicted beams (N strongest predicted beams). N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M. Non-limiting examples of values for M, N and K are given below with Fig. 4.

[0055] The information indicating N strongest predicted beams may comprise, as a further information, the information indicating M strongest measured beams.

[0056] Then the apparatus determines (block 203) whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams. For example, if M strongest beams comprise, starting from the strongest, following beams B9, Bl, B5, B4 and B7, and M strongest predicted beams comprise, starting from the strongest predicted, following beams B7, B2, B3, B4, B6, B9, and a value of K=l, the apparatus determines that B9 is among the M strongest predicted beams. However, if a value of K=3. the apparatus determines that K strongest beams B9, B 1 , and B5 are not all among the M strongest predicted beams, since M strongest predicted beams do not comprise B 1 and B5. In another example, the apparatus may determine that if the accuracy meets or exceeds a preset percentage limit, the accuracy is validated. For example, the accuracy of the above example with value K=3 may be 33 % (one found in both sets).

[0057] The apparatus validates (block 204) the accuracy of the beam prediction model, when (block 203: yes)at least the K strongest beams are among the N strongest predicted beams.

[0058] The apparatus does not validate (block 205) the accuracy of the beam prediction model, when (block 203: no) at least the K strongest beams are not among the N strongest predicted beams.

[0059] In an implementation, the apparatus may be configured to transmit the validation result (validated / non- validated) to the second apparatus. The second apparatus may use the validation result to determine whether to continue inference or training.

[0060] In one example implementation, the apparatus may be configured to perform the functionality described with Fig. 1, when the apparatus receives from the second apparatus an indication whether the second apparatus is performing inference or training of the beam prediction model.

[0061] In one example implementation, combinable with other implementations, the apparatus may be configured, at the beginning of the functionality described with Fig. 2, to transmit to the second apparatus configuration information to obtain measurement results on a first set of beams, and configuration information to obtain predictions using as input measurement result on a second set of beams. Different examples of the first and second set of beams are given below with Fig. 4.

[0062] Fig. 3 illustrates an example functionality of an apparatus, for example the second apparatus 110 illustrated in Fig. 1, configured to predict strongest beams using a beam prediction model, or shortly a model.

[0063] Referring to Fig. 3, the apparatus may perform (301), or may have earlier performed, inference or training to a beam prediction model outputting X strongest predicted beams. In other words, the testing of the beam prediction model may be performed while the inference or training of the model is performed, or the testing may be performed to an earlier trained / inferenced model. Further, it may be that another apparatus has performed training or inferencing, and the model has been downloaded to the apparatus, and it is tested for to verify its suitability to the apparatus.

[0064] The apparatus obtains (block 302) measurement results on beams. For example, even though not illustrated in Fig. 3, it is possible that the apparatus determines a measurement or measurements of the beams according to a configuration received earlier and the apparatus may measure beams accordingly and / or may cause one or more other entities to measure the beams. For example, the apparatus and / or the entities / entity may measure downlink beam reference signals, for example synchronization signal blocks or tracking reference signals or channel state information reference signals. Measurement results may be reference signal received power values. Further, herein it is assumed, for the clarity of description, that the measurement results are accurate enough for for high prediction accuracy, i.e. that their accuracy has in practise no effect to the accuracy of predicted beams.

[0065] The apparatus inputs (block 303) the measurement results, or at least part of the measurement results, to the beam prediction model, which outputs X strongest predicted beams. The apparatus transmits (block 304) to the first apparatus information indicating N strongest predicted beams output by the beam prediction model. X and N are positive integers, N being smaller than or equal to X. Different examples of the information has been described above. When N is smaller than X, overhead reduction is achieved compared to a situation in which X beams are indicated.

[0066] In an implementation, the configuration received earlier may be a configuration to obtain measurements results on a first set of beams and to input to the beam prediction model measurement results on a second set of beams, as will be described in more detail with Fig. 4. Fig. 4 illustrates an information exchange on a high level. Non-limiting examples of the information exchange will be described below. Optional features, i.e. information exchange or functionality, any of which may be omitted or performed within the described non-limited examples are illustrated using dashed lines. Further, it should be appreciated that a detail in an example may be added to another example.

[0067] It may be that the second apparatus is configured (block 4-1), for example by the first apparatus, or by some other network entity, with AI / ML mode for beam management. For example, a model to be trained / inferred, and related parameters for training / inferring may have been transmitted to the second apparatus.

[0068] First example

[0069] In the first example, the first apparatus transmits (message 4-2) a command to switch to AI / ML mode based functionality. The command may be “enable AI / ML based beam management use case”. The goal of message 4-2 is to ensure that the mode of functioning at the second apparatus side is well known at the first apparatus side.

[0070] The second apparatus may then confirm (message 4-3) the activation of the requested mode, or that it already was in the requested mode. The mode may be indicated also indirectly. For example, message 4-3 may be a functionality indication indicating whether the second apparatus is performing inference or training of the beam prediction model.

[0071] The second apparatus then configures (message 4-9) the first apparatus at least to report at least N strongest predicted beams. The second apparatus may configure (message 4- 9) the second apparatus to report measurement results, for example to report M strongest measured beams simultaneously with the N strongest predicted beams.

[0072] The second apparatus then performs (block 4-10) in the illustrated example Top-K beam identifier (ID) prediction. In other words, the second apparatus input measurement results to the beam prediction model that outputs X strongest predicted beams, or in the example predicted beam identifiers. Top-K beam or beams may be a subset of the X strongest predicted beams, starting from the strongest predicted beam.

[0073] The second apparatus reports (message 4-11) at least the predicted Top-K strongest beam identifiers to the first apparatus. The second apparatus may report M strongest measured beams by transmitting corresponding beam identifiers to the first apparatus, either separately (message 4-12) or together with the Top_K predicted beam identifiers. The second apparatus may report M strongest measured beams even when not configured to report them. The first apparatus then determines (block 4-13) the accuracy of the beam prediction model. In other words, the first apparatus determines, whether to validate the accuracy, as described above. The determining may comprise verifying whether the strongest beam identifier (known by the first apparatus or received from the second apparatus) is one of the predicted Top-K strongest beam identifiers.

[0074] Second example

[0075] In a second example, applicable to the spatial domain beam prediction and the timedomain beam prediction, two sets of beams are used, a first set of beams, called in the example Set A, and a second set of beams, called in the example Set B. In the second example, Set B is a subset of Set A, and the beam prediction model is configured to predict Top-K identifiers of Set A beams.

[0076] In the second example, the first apparatus transmits (message 4-2) to the second apparatus the command to switch to AI / ML mode based functionality.

[0077] The second apparatus transmits (message 4-3) to the first apparatus a confirmation that it operates in AI / ML BM mode. (Message 4-3 is optional).

[0078] The second apparatus transmits (message 4-4) to the first apparatus functionality indication, e.g., indication whether the second apparatus performs (runs) inference or training in Top-K downlink beam management in spatial or Top-K downlink beam management in time-domain.

[0079] The first apparatus checks (block 4-5) the indication whether the second apparatus performs inference for Top-K beam ID prediction. In the illustrated example it is assumed that the second apparatus performs inference for Top-K beam ID prediction, and the information exchange may continue.

[0080] The first apparatus transmits (message 4-6) configurations to the second apparatus to measure the whole Set A, i.e. all beams in Set A. For example, message 4-6 may comprise the configuration and channel state information reference signal, CSLRS, resources of whole Set A measurements. The second apparatus will then measure the whole reference signal reception power, RSRP, values of Set A beams.

[0081] The first apparatus transmits (message 4-7) configurations to the second apparatus to measure the whole Set A. Message 4-7 may comprise the configuration and channel state information reference signal, CSI-RS, resources of fixed Set B beams. For example, the second apparatus may be configured to use 16 Set B beams to predict Top-1 or Top-4 or Top-8 of 64 Set A beams. It should be appreciated that Set A beams could be 32 or 64 or 256 beams while Set B beams would be at least quarter of Set A (14 of Set A), e.g., if Set A is 64 beams, Set B should be configured to be 16 beams.

[0082] The first apparatus further transmits (message 4-8) to the second apparatus a request to prepare RSRP values of Top-K strongest beams. The predicted Top-K strongest beams can be within Set B or within Set B including the beams in Set A that are not in Set B (exclude Set B). Purpose of the request (message 4-8) is to tell to the second apparatus to rank the beams and get Top-K beams from measurements. (Message 4-8 is optional.)

[0083] The first apparatus then transmits (message 4-9) to the second apparatus a request to report Top-K beam identifier prediction results. Message 4-9 may contain a request to report Top-K strongest beams identifiers in Set A beams measurements.

[0084] The second apparatus then performs (block 4-10) in the illustrated example Top-K beam identifier (ID) prediction of Set A beams using RSRP values of Set B beams (primary measurements) as input to the beam prediction model, that outputs predicted Top-K strongest beam identifiers of Set A. In other words, only a subset of RSRP values of Set A are input to the beam prediction model.

[0085] The second apparatus then reports (message 4-11) predicted Top-K strongest beam identifiers to the first apparatus. In other words, the output of the beam prediction model, or part of it, is transmitted in message 4-11.

[0086] The second apparatus may report (message 4-12), based on measurement results of the whole Set A, Top-K strongest beam identifiers of Set A beams to the first apparatus. As described above, the first apparatus may have that information available at the first apparatus.

[0087] The first apparatus then determines (block 4-13) accuracy of the beam prediction model. For example, the first apparatus may validate the predicted Top-K strongest beam identifiers with the Top-K strongest beam identifiers of Set A. If the predicted Top-K strongest beam identifiers include the strongest beam identifiers, the first apparatus may compare the predicted Top-K strongest beam identifiers with the measured Top-K strongest beam identifiers. According to one example condition, if they are the same, the test is passed, and the accuracy will be validated; otherwise the model predicts wrong beams, and the test fails resulting to non-validation of the accuracy. According to another condition, if the predicted Top-K strongest beam identifiers includes the identifier of the strongest measured beam, the test is passed, otherwise test is failed. In another option, the first apparatus may know legacy beam(s) having the strongest RSRP, and may compare the predicted Top-K strongest beam identifiers with the legacy beam identifiers. According to one example condition, if they are the same, the test is passed. According to another condition, if the predicted Top-K strongest beam identifiers includes the identifier of the strongest beam, the test is passed, otherwise test is failed. It should be noted that use of the legacy beams having the strongest RSRP may yield to a limited number of beam identifiers than the user of measurement results. Hence the latter may be better for test purposes.

[0088] The first apparatus may report (message 4-15) the test results (e.g., pass / fail or val- idated / non-validated) to the second apparatus, which then may decide (block 4-15), using at least the reported information, whether to continue inferencing / training the beam prediction model.

[0089] Third example

[0090] In the third example, applicable to the spatial domain beam prediction and the timedomain beam prediction, two sets of beams are used, a first set of beams, called in the example Set A, and a second set of beams, called in the example Set B. In the third example, Set B is different from Set A, Set B comprising one or more wide beams, Set A comprising narrow beams. The beam prediction model is configured to predict Top-K identifiers of Set A beams. In other words, wide beam(s) are used to predict narrow beam(s), enabling selecting a best narrow beam from a best wide beam. Per a wide beam in Set B, there will be one or more narrow beams in Set A that may overlap the wide beam.

[0091] In the third example, the first apparatus transmits (message 4-2) to the second apparatus the command to switch to AI / ML mode based functionality.

[0092] The second apparatus transmits (message 4-3) to the first apparatus a confirmation that it operates in AI / ML BM mode. (Message 4-3 is optional).

[0093] The second apparatus transmits (message 4-4) to the first apparatus functionality indication, e.g., indication whether the second apparatus performs (runs) inference or training in Top-K downlink beam management in spatial or Top-K downlink beam management in time-domain.

[0094] The first apparatus checks (block 4-5) the indication whether the second apparatus performs inference for Top-K beam ID prediction. In the illustrated example it is assumed that the second apparatus performs inference for Top-K beam ID prediction, and the information exchange may continue.

[0095] The first apparatus transmits (message 4-6) configurations and synchronization signal block, SSB, or CSI-RS resources for measurement of fixed set B. For example, the second apparatus may be configured to use 16 Set B beams (wide beams) to predict Top-1 or Top-4 or Top- 8 of 64 Set A beams (narrow beams). The first apparatus transmits (message 4-7) a pointer (indicator) that the configured Set B is CSI-RS or SSB. In other words, the first apparatus transmits the pointer to identify to the second apparatus that Set B is wide beam set.

[0096] The first apparatus may transmits (message 4-8) to the second apparatus a request to prepare RSRP values of Top-K strongest beams. Purpose of the request (message 4-8) is to tell to the second apparatus to rank the beams and get Top-K beams from measurements. (Message 4-8 is optional.)

[0097] In an implementation, the first apparatus may transmit, in a separate message, or in message 4-7 or in message 4-8 configurations to the second apparatus to measure the Set A.

[0098] The first apparatus then transmits (message 4-9) to the second apparatus a request to report Top-K beam identifier prediction results. Message 4-9 may contain a request to report Top-K strongest beams identifiers in Set A beams measurements.

[0099] The second apparatus then performs (block 4-10) in the illustrated example Top-K beam identifier (ID) prediction of Set A beams using RSRP values of Set B beams (wide beams) as input to the beam prediction model, that outputs predicted Top-K strongest beam identifiers of Set A (narrow beams).

[0100] The second apparatus then reports (message 4-11) predicted Top-K strongest beam identifiers to the first apparatus. In other words, the output of the beam prediction model, or part of it, is transmitted in message 4-11.

[0101] The second apparatus may report (message 4-12), based on measurement results of Set A, Top-K strongest beam identifiers of Set A beams to the first apparatus. As described above, the first apparatus may have that information available at the first apparatus.

[0102] The first apparatus then determines (block 4-13) accuracy of the beam prediction model. For example, the first apparatus may validate the predicted Top-K strongest beam identifiers with the Top-K strongest beam identifiers of Set A. If the predicted Top-K strongest beam identifiers include the strongest beam identifiers, the first apparatus may compare the predicted Top-K strongest beam identifiers with the measured Top-K strongest beam identifiers. According to one example condition, if they are the same, the test is passed, and the accuracy will be validated; otherwise the model predicts wrong beams, and the test fails resulting to non-validation of the accuracy. According to another condition, if the predicted Top-K strongest beam identifiers includes the identifier of the strongest measured beam, the test is passed, otherwise test is failed. In another option, the first apparatus may know legacy beam(s) having the strongest RSRP, and may compare the predicted Top-K strongest beam identifiers with the legacy beam identifiers. According to one example condition, if they are the same, the test is passed. According to another condition, if the predicted Top-K strongest beam identifiers includes the identifier of the strongest beam, the test is passed, otherwise test is failed. It should be noted that use of the legacy beams having the strongest RSRP may yield to a limited number of beam identifiers than the user of measurement results. Hence the latter may be better for test purposes.

[0103] The first apparatus may report (message 4-15) the test results (e.g., pass / fail or val- idated / non-validated) to the second apparatus, which then may decide (block 4-15), using at least the reported information, whether to continue inferencing / training the beam prediction model.

[0104] As can be seen from the above examples, it is possible to test an AI / ML based beam prediction model for real-time monitoring mechanism. Further, the overhead is reduced significantly since the second apparatus is configured to report the Top-K beams in Set A (not the whole Set A and exclude Set B).

[0105] The blocks, related functions, and information exchange (messages, signaling) described above by means of Fig. 1 to Fig. 4 are in no absolute chronological order, and some of them may be performed simultaneously or in an order differing from the given one. Other functions can also be executed between them or within them, and other information may be sent, and / or other rules applied. For example, the accuracy may be validated using more than one test round during which reference signals are transmitted to obtain unused input to the beam prediction model. Some of the blocks or part of the blocks or one or more pieces of information can also be left out or replaced by a corresponding block or part of the block or one or more pieces of information.

[0106] Fig. 5 illustrates an apparatus 501, e.g. the first apparatus, that may be configured to perform testing or AI / ML based beam prediction model accuracy validation according to some embodiments. Fig. 6 illustrates an apparatus that may implement distributed functionality of the apparatus illustrated in Fig. 5. Fig. 7 illustrates an apparatus, e.g., the second apparatus, that may be configured to at least input measurement results to an AI / ML based beam prediction model and to report its output.

[0107] According to an embodiment, there is provided an apparatus comprising means for receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; means for obtaining information on M strongest beams at the second apparatus; means for determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and means for validating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams; wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M.

[0108] According to an embodiment, there is provided an apparatus comprising means for performing inference or training to a beam prediction model outputting X strongest predicted beams; means for obtaining measurement results on beams; means for inputting the measurement results to the beam prediction model; means for transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

[0109] The apparatus 501, 701 may comprise one or more communication control circuitry 520, 720, such as at least one processor, and at least one memory 530, including one or more algorithms 531, 731, such as a computer program code (software) wherein the at least one memory and the computer program code (software) are configured, with the at least one processor, to cause the apparatus to carry out any one of the exemplified functionalities of a corresponding apparatus, described above with any of Fig. 1 to Fig. 4. Said at least one memory 530, 730 may also comprise at least one database 532, 732.

[0110] Referring to Fig. 5, the one or more communication control circuitries 520 of the apparatus 501 comprise at least an AI / ML testing circuitry 521 which is configured to perform determining and transmitting test related configurations, commands, or request and to determine whether to validate accuracy of AI / ML based beam prediction model. To this end, the AI / ML testing circuitry 521 of the apparatus 501 is configured to carry out at least some of the functionalities described above, e.g., by means of Fig. 1, Fig. 2, and Fig. 4, using one or more individual circuitries.

[0111] Referring to Fig. 5, the memory 530 may be implemented using any suitable data storage technology, such as semiconductor based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory.

[0112] Referring to Fig. 5, the apparatus 501 may further comprise different interfaces 510 such as one or more communication interfaces (TX / RX) comprising hardware and / or software for realizing communication connectivity according to one or more communication protocols. The one or more communication interfaces 510 may enable connecting to the Internet and / or to a core network of a wireless communications network and / or to a radio access network and / or to other apparatuses within range of the apparatus. The one or more communication interface 510 may provide the apparatus with communication capabilities to communicate in a cellular communication system and enable communication to second apparatuses, such as different network nodes or elements or device components, e.g., mobile apparatuses, such as terminal devices or user equipments, for example. The one or more communication interfaces 510 may comprise standard well-known components such as an amplifier, filter, frequencyconverter, (de)modulator, and encoder / decoder circuitries, controlled by the corresponding controlling units, and possibly one or more antennas.

[0113] In an embodiment, as shown in Fig. 6, at least some of the functionalities of the apparatus of Fig. 5 may be shared between two physically separate devices, forming one operational entity. Therefore, the apparatus may be seen to depict the operational entity comprising one or more physically separate devices for executing at least some of the described processes. Thus, the apparatus of Fig. 6, utilizing such shared architecture, may comprise a control unit CU 620, or a remote control unit, such as a host computer or a server computer, operatively coupled (e.g. via a wireless or wired connection) to a remote distributed unit DU 622 located in the first apparatus or in a remote head of the first apparatus. In an embodiment, at least some of the described processes may be performed by the CU 620. In an embodiment, the execution of at least some of the described processes may be shared among the DU 622 and the CU 620.

[0114] Similar to Fig. 5, the apparatus of Fig. 6 may comprise one or more communication control circuitries (CNTL) 520, such as at least one processor, and at least one memory (MEM) 530, including one or more algorithms (PROG) 531, such as a computer program code (software) wherein the at least one memory and the computer program code (software) are configured, with the at least one processor, to cause the apparatus to carry out any one of the exemplified functionalities of the first apparatus, described above, e.g., by means of e.g., by means of Fig. 1, Fig. 2 and Fig. 4.

[0115] Referring to Fig. 7, the one or more communication control circuitry 720 of the apparatus 701 comprise at least a beam predicting circuitry 721, which is configured to predict and report beams, that are discussed with Fig. 1 to Fig. 4. To this end, the beam predicting circuitry 721 of the apparatus 701 is configured to carry out at least some of the functionalities of the second apparatus, described above, e.g., by means of Fig. 1, Fig. 3 and Fig. 4, for example, using one or more individual circuitries.

[0116] Referring to Fig. 7, the memory 730 may be implemented using any suitable data storage technology, such as semiconductor based memory devices, flash memory, magnetic memory devices and systems, optical memory devices and systems, fixed memory and removable memory. Referring to Fig. 7, the apparatus 701 may further comprise different interfaces 710 such as one or more communication interfaces (TX / RX) comprising hardware and / or software for realizing communication connectivity according to one or more communication protocols. The one or more communication interfaces 710 may enable connecting to a radio access network and / or to other apparatuses within range of the apparatus and / or to the Internet and / or to a core network of a wireless communications network. The one or more communication interface 710 may provide the apparatus with communication capabilities to communicate in a cellular communication system and enable communication to different network nodes or elements. The one or more communication interfaces 710 may comprise standard well-known components such as an amplifier, filter, frequency-converter, (de)modulator, and encoder / decoder circuitries, controlled by the corresponding controlling units, and one or more antennas.

[0117] In embodiments, the CU 620 may generate a virtual network through which the CU 620 communicates with the DU 622. In general, virtual networking may involve a process of combining hardware and software network resources and network functionality into a single, software-based administrative entity, a virtual network. Network virtualization may involve platform virtualization, often combined with resource virtualization. Network virtualization may be categorized as external virtual networking which combines many networks, or parts of networks, into the server computer or the host computer (e.g. to the CU). External network virtualization is targeted to optimized network sharing. Another category is internal virtual networking which provides network-like functionality to the software containers on a single system. Virtual networking may also be used for testing the terminal device.

[0118] In embodiments, the virtual network may provide flexible distribution of operations between the DU and the CU. In practice, any digital signal processing task may be performed in either the DU or the CU and the boundary where the responsibility is shifted between the DU and the CU may be selected according to implementation.

[0119] As used in this application, the term ‘circuitry’ may refer to one or more or all of the following: (a) hardware-only circuit implementations, such as implementations in only analog and / or digital circuitry, and (b) combinations of hardware circuits and software (and / or firmware), such as (as applicable): (i) a combination of analog and / or digital hardware circuit(s) with software / firmware and (ii) any portions of hardware processor(s) with software, including digital signal processor(s), software, and memory(ies) that work together to cause an apparatus, such as a terminal device or an access node, to perform various functions, and (c) hardware circuit(s) and processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g. firmware) for operation, but the software may not be present when it is not needed for operation. This definition of ‘circuitry’ applies to all uses of this term in this application, including any claims. As a further example, as used in this application, the term ‘circuitry’ also covers an implementation of merely a hardware circuit or processor (or multiple processors) or a portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ also covers, for example and if applicable to the particular claim element, a baseband integrated circuit for an access node or a terminal device or other computing or network device.

[0120] In an embodiment, at least some of the processes described in connection with Fig. 1 to Fig. 4 may be carried out by an apparatus comprising corresponding means for carrying out at least some of the described processes. Some example means for carrying out the processes may include at least one of the following: detector, processor (including dual-core and multiple-core processors), digital signal processor, controller, receiver, transmitter, encoder, decoder, memory, RAM, ROM, software, firmware, display, user interface, display circuitry, user interface circuitry, user interface software, display software, circuit, antenna, antenna circuitry, and circuitry. In an embodiment, the at least one processor, the memory, and the computer program code form processing means or comprises one or more computer program code portions for carrying out one or more operations according to any one of the embodiments of Fig. 1 to Fig. 4 or operations thereof.

[0121] Embodiments and examples as described may also be carried out in the form of a computer process defined by a computer program or portions thereof. Embodiments of the functionalities described in connection with Fig. 1 to Fig. 4 may be carried out by executing at least one portion of a computer program comprising corresponding instructions. The computer program may be provided as a computer readable medium comprising program instructions stored thereon or as a non-transitory computer readable medium comprising program instructions stored thereon. The computer program may be in source code form, object code form, or in some intermediate form, and it may be stored in some sort of carrier, which may be any entity or device capable of carrying the program. For example, the computer program may be stored on a computer program distribution medium readable by a computer or a processor. The computer program medium may be, for example but not limited to, a record medium, computer memory, read-only memory, electrical carrier signal, telecommunications signal, and software distribution package, for example. The computer program medium may be a non-transitory medium. The term “non-transitory,” as used herein, is a limitation of the medium itself (i.e., tangible, not a signal ) as opposed to a limitation on data storage persistency (e.g., RAM vs. ROM). Coding of software for carrying out the embodiments as shown and described is well within the scope of a person of ordinary skill in the art.

[0122] Even though the embodiments have been described above with reference to examples according to the accompanying drawings, it is clear that the embodiments are not restricted thereto but can be modified in several ways within the scope of the appended claims. Therefore, all words and expressions should be interpreted broadly and they are intended to illustrate, not to restrict, the embodiment. It will be obvious to a person skilled in the art that, as technology advances, the inventive concept can be implemented in various ways. Further, it is clear to a person skilled in the art that the described embodiments may, but are not required to, be com- bined with other embodiments in various ways.

Claims

CLAIMS1. A first apparatus comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the first apparatus at least to: receive from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtain information on M strongest beams at the second apparatus; determine whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validate accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams, wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M.

2. The first apparatus of claim 1, wherein the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to obtain the information on M strongest beams at the second apparatus by receiving from the second apparatus in the information further information indicating M strongest measured beams or by deriving the information on M strongest beams from a test set up configuration.

3. The first apparatus of claim 2, wherein the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to: receive, prior to the information, from the second apparatus an indication whether the second apparatus is performing inference or training of the beam prediction model; transmit, when the second apparatus is performing inference or training of the beam prediction model, to the second apparatus configuration information to obtain measurement results on a first set of beams, and configuration information to obtain predictions using as input measurement result on a second set of beams.

4. The first apparatus of claim 3, wherein the second set of beams is a subset of the first set of beams.

5. The first apparatus of claim 3, wherein the second set of beams comprises different beams than the first set of beams.

6. The first apparatus of claim 3, 4 or 5, wherein the first set of beams comprises a first number of beams, and the second set of beams comprises a second number of beams, which is at least a quarter of the first number.

7. The first apparatus of any of preceding claims, wherein the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to: transmit, prior to receiving the information, to the second apparatus a command to switch to an artificial intelligence / machine learning mode based functionality, and / or a request for reporting at least N strongest predicted beams.

8. The first apparatus of any of preceding claims, wherein the information indicating a beam is a beam identifier.

9. The first apparatus of any of preceding claims, wherein the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the first apparatus at least to: transmit to the second apparatus information indicating, whether the accuracy of the beam prediction model is validated.

10. A second apparatus comprising at least one processor and at least one memory storing instructions that, when executed by the at least one processor, cause the second apparatus at least to: perform inference or training to a beam prediction model outputting X strongest predicted beams; obtain measurement results on beams; input the measurement results to the beam prediction model; transmit to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

11. The second apparatus of claim 10, wherein the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the second apparatus at least to: receive from the first apparatus configuration information to obtain measurement results on a first set of beams, and configuration information to obtain predictions using as input measurement result on a second set of beams; input the measurement results on the second set of beams to the beam prediction model; and transmit to the first apparatus further information indicating M strongest measured beams at the second apparatus, wherein M is a positive integer which is smaller than or equal to N.

12. The second apparatus of claim 10 or 11, wherein the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the second apparatus at least to: transmit to the first apparatus at least an indication whether the second apparatus is performing inference or training of the beam prediction model.

13. The second apparatus of claim 10, 11 or 12, wherein the at least one processor and the at least one memory storing instructions, when executed by the at least one processor, further cause the second apparatus at least to: receive from the first apparatus a command to switch to artificial intelligence / ma- chine learning mode based functionality; transmit, after receiving the command and when in the artificial intelligence / ma- chine learning mode based functionality, to the first apparatus information confirming that the second apparatus is in the artificial intelligence / machine learning mode based functionality.

14. A method comprising at least: receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtaining information on M strongest beams at the second apparatus; determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; andvalidating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams, wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M.

15. A method comprising at least: performing inference or training to a beam prediction model outputting X strongest predicted beams; obtaining measurement results on beams; inputting the measurement results to the beam prediction model; transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

16. A computer readable medium comprising instructions which, when executed by an apparatus, cause the apparatus to perform at least one of a first process or a second process, wherein the first process comprises at least: receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtaining information on M strongest beams at the second apparatus; determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams, wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M, and wherein the second process comprises at least: performing inference or training to a beam prediction model outputting X strongest predicted beams; obtaining measurement results on beams; inputting the measurement results to the beam prediction model; transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model,wherein X and N are positive integers, N being smaller than or equal to X.

17. The computer readable medium of claim 16, wherein the computer readable medium is a non-transitory computer readable medium.

18. A computer program comprising instructions, which, when executed by an apparatus, cause the apparatus to perform at least one of a first process or a second process, wherein the first process comprises at least: receiving from a second apparatus information indicating N strongest predicted beams output by a beam prediction model; obtaining information on M strongest beams at the second apparatus; determining whether at least K strongest beams in the M strongest beams are among the N strongest predicted beams; and validating accuracy of the beam prediction model, when at least the K strongest beams are among the N strongest predicted beams, wherein N, M and K are positive integers, M being smaller than or equal to N and K being smaller than or equal to M, and wherein the second process comprises at least: performing inference or training to a beam prediction model outputting X strongest predicted beams; obtaining measurement results on beams; inputting the measurement results to the beam prediction model; transmitting to a first apparatus information indicating N strongest predicted beams output by the beam prediction model, wherein X and N are positive integers, N being smaller than or equal to X.

Images