METHOD FOR MANAGING RADIO RESOURCES IN A MOBILE NETWORK USING HYBRID MAPPING OF RADIO PROPERTIES

Description

TECHNICAL FIELD

[0001] The present invention relates to the field of cellular networks and more particularly to the management of radio resources or RRM ( Radio Resource Management ) in such a network. It also concerns the field of artificial intelligence and more particularly that of distributed learning ( Distributed Learning). STATE OF THE PRIOR ART

[0002] With the deployment of 5th generation (5G) cellular networks, radio resource management techniques have had to evolve to accommodate new use cases involving highly heterogeneous services in terms of quality of service (QoS). In addition, most 5G networks are themselves heterogeneous in nature, typically using an overlay of a dense layer of small cells or SBS ( Small cell base stationsoperating in particular in the millimeter band, intended to provide short-distance, high-speed coverage, and a low-density layer of macrocells or MBS ( Macro cell base stations ) , operating in the sub-6 GHz band, intended to ensure continuous coverage. This heterogeneity leads to greater complexity in radio resource management or RRM.

[0003] Radio resource management is generally performed by maximizing or minimizing a given objective function. Thus, depending on the strategy considered, one can, for example, maximize the quality of service (QoS) of certain network communications, minimize the frequency of handover operations, minimize the latency of certain communications, or minimize the energy consumption of mobile terminals and / or base stations. Resource management occurs at different levels in the network, for example in the allocation of transmission resources (time slots, frequencies, codes), the association of user terminals (UEs) with base stations, the allocation / configuration of beams, the power allocation, the handover mechanism, the dynamic deployment of base stations or relays, the terminal location, etc.

[0004] For reasons of simplicity, flexibility and scalability, radio resource management methods generally use a mapping of radio characteristics representing the propagation conditions and / or spectral activity in different sub-bands, at different points in the geographical area in which the network is deployed. These radio characteristics are classically obtained from channel state information or CSI ( Channel State Information ) collected by different user terminals at different points, at different frequencies and at different times. The radio characteristics thus mapped then make it possible to estimate interference levels, propagation channel properties and network topology.

[0005] Different radio characteristic maps, more simply called radio maps, are known in the state of the art.

[0006] For example, a received signal strength map or RSS ( Received Signal Strength ) of a plurality of access points (or, conversely, the signal power received from a source by a plurality of receivers) is conventionally used in radio fingerprinting localization methods ( radio fingerprinting ) . An example of an RSS mapping application to dynamically deploy relays in a network is described in J. Chen's paper "Learning radio maps UAV-aided wireless networks: a segmented regression approach" published in Proc. of IEEE Int'l Conf. on Communications, May 2017, pp. 1-6.

[0007] The article by S. Bi et al. entitled "Engineering radio map for wireless resource management" published in IEEE Wireless Communications, February 2019, proposes to generate a power spectral density map or PSD ( Power Spectral Density) by superimposing local maps obtained at different emission points. However, this mapping method requires a large number of nodes to obtain an accurate map of the PSD.

[0008] Finally, the article by C. Studer et al. entitled "Channel charting: locating users within the radio environment using channel state information" published in IEEE Access, vol. 4, 2016, pp. 1-17 describes an unsupervised learning method for a radio map (here a channel characteristics map), from an acquisition of CSI information collected by the different terminals. The channel characteristics map is learned in a representation space ( embedding ) of reduced dimension ( ℝ D ′ ) with respect to the geometric space ( ℝ D ), while preserving local geometric relationships. Specifically, physically close points in the geometric space are transformed into neighboring points in the representation space (continuity constraint). The mapping of channel characteristics into the representation space can then be used by a base station to make handover decisions.

[0009] However, the channel characteristics mapping proposed in this paper contains relatively poor information. Thus, it cannot be used satisfactorily for network radio resource management. In addition, the aforementioned continuity constraint may be inappropriate in some cases. Indeed, mobile terminals located close to each other may have radically different channel characteristics while on the contraryRemote mobile terminals may have relatively similar channel characteristics. Moreover, the generation of mapping of these characteristics is generally carried out centrally, for example within a computing server, which, on the one hand, requires the exchange of a large number of measurement messages within the network and reduces the useful throughput of communications, and, on the other hand, does not allow for effective monitoring of local variations in characteristics due to network heterogeneity. Finally, the channel mapping obtained is specific to a given environment and frequency band: a change of environment or frequency range (for example from the sub-6 GHz band to the millimeter band or vice versa) therefore requires launching a new complete learning campaign.

[0010] Patent document WO 2019 / 211134 A1 discloses methods for managing the performance of a radio access network. The radio access network comprises one or more radio network nodes. A first network node determines a configuration of one or more parameters in the radio access network. The determination is based on one or more machine-implemented reinforcement learning procedures to optimize the performance of the radio access network based on the one or more parameters.

[0011] Patent document WO 2020 / 068127 A1 discloses methods of using machine learning to improve communications between different networks.

[0012] Patent document WO 2012 / 072445 A1 discloses examples of network learning and optimization systems to facilitate adaptation to system state.

[0013] The object of the present invention is therefore to propose a method for managing radio resources based on an enriched (or augmented) radio mapping, which can easily take into account the heterogeneity of the network without resorting to significant computing infrastructures or penalizing the useful throughput of communications, and which does not require a new complete learning phase in the event of a change in the environment or network topology. STATEMENT OF THE INVENTION

[0014] The present invention is defined by a method according to claim 1.

[0015] Advantageously, the reward to be maximized corresponds to a sum of flow rates or quality of service levels on network communications to be maximized, or a handover frequency or energy consumption to be minimized.

[0016] The neighborhood of the point of interest can be defined as a set of neighboring points of the network in the sense of a similarity metric operating in a space representing local observations.

[0017] Alternatively, said neighborhood of the point of interest is determined by means of a previously trained classifier, operating in a space representing local observations.

[0018] Advantageously, the node of interest only decides to perform an action at a time to the extent that one of the local maps of a radio feature at that time differs from the local map of the same radio feature at the previous time, the difference between the two local maps being measured using a Kullback-Leibler divergence.

[0019] Alternatively, the node of interest decides to perform an action at a time only to the extent that the hybrid local mapping at that time differs from the hybrid local mapping at the previous time, the difference between the two hybrid mappings being measured using a Kullback-Leibler divergence.

[0020] The fusion of local radio maps advantageously uses an attention mechanism to H heads, with H < K Or K is the number of radio characteristics.

[0021] In this case, hybrid local mapping can be obtained by means of Φ j a t f = W Φ . α j h ϕ j h p t f ; h = 1 , … , H T Or α j h = α i , j h ; i = 1 , … , P j is the score between a query q j h of the knot N j and the keys associated with the different observations from the neighborhood nodes V j ( t ) , of the node of interest, ϕ j h p t f is the local mapping value of the feature hat point p, at time t and at the frequency f, And W Φ is a matrix of size P j × H Or P j is the number of nodes in the neighborhood V j ( t ) .

[0022] The query of the node of interest can be expressed by a vector q j h = W q , j h o j T Or W q , j h is a matrix of size n × Ω Or n is the dimension of the feature representation space and Ω is the size of the observation vectors and the keys associated with the different observations from the nodes Neither of V j ( t ) can be expressed by vectors k i , j h = W k , j h o j T .

[0023] Finally, the score between the query q j h of the knot N j and a key associated with a node Neither from the neighborhood V j ( t ) can be calculated using α i , j h = softmax q j h , k i , j h n where “.” represents the dot product. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Other characteristics and advantages of the invention will appear on reading a preferred embodiment of the invention, described with reference to the attached figures among which: [ Fig. 1 ] schematically represents a message passing service between nodes in the same neighborhood within the network, used in the context of the present invention; [ Fig. 2 ] schematically illustrates an example of a combination of radio local characteristic maps that can be used in the radio resource management method according to the present invention; [ Fig. 3 ] schematically represents the architecture of a network node making it possible to implement a method of managing radio resources according to an embodiment of the present invention; [ Fig. 4] schematically represents a flowchart of the radio resource management method according to an embodiment of the present invention; [ Fig. 5 ] details the step of determining the neighborhood of a node in the flowchart of the Fig. 4 . DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

[0025] In the following, we will consider a cellular network composed of a plurality of base stations. Without loss of generality and for illustration purposes only, we will assume that this cellular network is heterogeneous. By heterogeneous cellular network we mean a network resulting from the superposition of a layer of small cells (SBS) with low coverage, but potentially capable of offering each UE a high throughput, and a layer of macro-cells (MBS), guaranteeing the continuity of network coverage by offering a greater range. A typical example of application is that of a 5G network in which the SBS cells operate in the millimeter band and the MBS cells operate in the sub-6 GHz band. Those skilled in the art will, however, understand that the association method according to the present invention applies to any cellular network, whether homogeneous or heterogeneous.

[0026] An idea underlying the present invention is to perform a learning of a radio resource management policy from a hybrid local mapping of a plurality of radio characteristics. This mapping is obtained in a distributed manner and no longer centralized as in the prior art. To do this, the present invention uses a message passing service between neighboring nodes, as illustrated in Fig. 1 .

[0027] The nodes of the network, noted Neither , i = 1,.., P can be user terminals (UE), base stations (BS) or even relays. We have represented in Fig. 1 the nodes belonging to the neighborhood V j ( t ) , of a node of interest N j , for example a user terminal, said neighborhood being obtained by means of a similarity metric described later. As the notation indicates, the neighborhood V j ( t) depends on the time t as far as the node environment N j may vary, either as it moves within the network or as nodes leave or enter the area where it is located N j , either due to changes in the propagation environment itself (for example, movement of an obstacle to propagation), or due to a combination of these different causes, in a non-limiting manner.

[0028] Each node belonging to the neighborhood of N j , either Neither ∈ V j ( t ) , makes a local observation, noted oi ( t, f ), of its radio environment in the form of a measurement of at least one radio parameter or signal.

[0029] Generally, such a local observation can be presented in a vector form, for example a line vector o i ( t, f ) ,each element of the observation vector corresponding to the measurement of a parameter or a radio signal, and can depend on the instant, t, and frequency, f, to which (or even the frequency band in which) it is carried out. This local observation may include a measurement of the received signal RSS, an angle of arrival of a received signal, an angle of departure of a transmitted signal, a channel estimate or equivalently, CSI channel state information, a quality of service QoS estimate, an interference level, a mobility class (in a 5G system) or a traffic density request (in a 5G system).

[0030] We will assume in the following that each node is capable of extracting Kradio characteristics of its local observations. These radio characteristics can relate, for example, to network topology, interference level, traffic density, etc. These radio characteristics are coded in the form of messages which are transmitted to the nodes N j . More specifically, in the case of the Fig. 1 , each node Neither ∈ V j ( t ) carrying out a local observation, o i ( t, f ), extracts radio characteristics k = 1,..., K and encodes each of them as a message m i , j k t f which it transmits to the node N j , or: [Math. 1] m i , j k t f = g j k o i t f

[0031] The coding function g j k . usually depends on the node N j whose neighborhood and the radio characteristic considered are considered. It can be seen as a pre-filtering function relative to the characteristic khaving the aim of extracting from local observation o i ( t, f ) information relating to this characteristic. This coding function can take different forms depending on the type of observation and the type of characteristic considered. For example, it can be defined by an association operation ( clustering ), a projection onto a geometric variety, in particular onto an affine variety relative to the characteristic in question, a multiplication by a filtering matrix, a mapping table, a neural network.

[0032] The node of interest N j aggregates messages relating to the same characteristic k, received from its neighbors and calculates the value taken by a local cartographic variable representing this characteristic, in a plurality of positions p = 1,..., P j ( t ) with P j ( t ) = Card ( V j ( t )) , right now tand at the frequency f, or: [Math.2] ϕ j k p t f = σ k m i , j k t f , ∀ i ∈ V j t Or σ k { .} is an aggregation function, invariant under permutation of indices i (the order of the nodes in the neighborhood being unimportant) and scalable, since the aggregation function must be independent of the number of nodes in the neighborhood V j ( t ) . In the following, for reasons of simplification, we will adopt the notation P j instead of P j ( t ) , being well understood that the number of neighboring nodes of the node of interest N j usually depends on time. The set of values ϕ j k p t f for the different positions p = 1,..., P j gives a local mapping ( local feature map ), Φ j k t f of the characteristic k right now t and frequency f, in the neighborhood V j ( t ) , Or Φ j k t f is a vector of size P j ( t ) = Card ( V j ( t )) . More generally, the values ϕ j k p t f are not necessarily scalars but vectors of dimension n ( n being the dimension of the space representing the radio characteristics), the mapping then being defined by a matrix Φ j k t f of size P j × n. If necessary, a matrix size can be provided Pi × n independent of the node of interest considered, Pi ≥ P j with the matrix then being hollow ( sparse ) or presenting interpolated values ​​for points not corresponding to node positions in the neighborhood. In the following, however, we will assume, without loss of generality, that the matrices Φ j k t f are of size P j × n.

[0033] The aggregation function σ k ( .) of the different messages can be, for example, a concatenation, a max-pooling operation, a sum of projections, or even a self-attention mechanism ( self-attention ) as defined in the article by A. Vaswani et al. entitled “Attention is all you need”, published in Proceedings of NIPS 2017, 6.12.2017.

[0034] If necessary, the coding and combination operations can be performed jointly in a graph neural network or GNN (Graph Neural Network), as described in the article by J. Zhou et al. entitled “Graph neural networks: a review of methods and applications” published in Al Open, vol. 1, pp. 57-81, for example in a graph attention network. In such a case, the nodes of the graph are nodes of the cellular network, each node having information corresponding to its local observation o i ( t, f ) and the edges of the graph represent the messages exchanged m i , j k t f .

[0035] Local maps Φ j k t f of the different characteristics, k = 1,.. ,K, are then merged to obtain a hybrid local mapping of these characteristics, also called augmented mapping, Φ j a t f . The way in which fusion is performed depends on the objective function to be optimized (maximized or minimized) for a given radio resource management policy. For example, it could be to maximize a sum of throughputs (over different communications), to maximize a quality of service, to reduce the handover frequency, to reduce energy consumption in a computing scenario at the edge of the network or MEC (Mobile Edge Computing).

[0036] According to a first example of implementation, the fusion can be obtained by means of a simple sum, or even a weighted sum. In other words: [Math.3] Φ j a t f = ∑ k = 1 K a j k Φ j k t f Or a j k , k = 1,.. ,Kare strictly positive real numbers. This assumes in this case that all matrices Φ j k t f are of the same size Pi × n.

[0037] According to a second example of implementation, the augmented mapping can be obtained by means of a sum of matrix products: [Math.4] Φ j a t f = ∑ k = 1 K A j k Φ j k t f Or A j k , k = 1,.., K are matrices of size Pi × P j .

[0038] According to a third example of implementation, the augmented mapping is obtained by means of an attention mechanism. We recall that an attention mechanism aims, schematically, to search in a sequence of inputs, those (here the radio characteristics) which present a certain connection or correlation to best predict an output. An attention mechanism (see article by Vaswani et al. above) implements a query vector ( query ), a key vector ( key) and a value vector ( value ) . For a given query and for each key associated with a value, we calculate an attention score which represents the degree of relevance of this key for the query, the result of the query then being the weighting of the different values ​​by weights representative of the associated scores.

[0039] In this case, a query from the node N j is formed for each characteristic h among H < K characteristics by means of: [Math.5] q j h = W q , j h o j T Or W q , j h is a matrix of size n × Ω Or n is the dimension of the feature representation space and Ω is the size of the observation vectors.

[0040] More precisely, the augmented local mapping is then obtained by: [Math.6] Φ j a t f = W Φ . α j h ϕ j h p t f ; h = 1 , … , H T Or α j h = α i , j h ; i = 1 , .. , P j is the score between the query q j h of the knot N jand the keys associated with the different observations from the nodes of V j ( t ) , namely: [Math.7] α i , j h = softmax q j h . k i , j h n Or k i , j h = W k , j h o j T is the key vector associated with the different observations from the nodes of V j ( t ) and . represents the scalar product; ϕ j h p t f = v i , j h t f ; i = 1 , .. , P j T is a matrix of size P j × n Or v i , j h t f = W v , j h o j T is the value vector associated with these same observations; W Φ is a matrix of size P j × H Or H is the number of attention heads, α j h ϕ j h p t f ; h = 1 , … , H is a matrix of size n × H and therefore the mapping Φ j a t f is a matrix of size P j × n.

[0041] The coefficients a j k in the first example of realization, the matrices A j k in the second embodiment, as well as the matrices W q , j h , W k , j h , W v , j h , W Φoccurring in the third exemplary embodiment, collectively referred to as fusion parameters, are advantageously determined by means of reinforcement learning as described below.

[0042] There Fig. 2 schematically represents a combination of local radio maps according to the first example of embodiment mentioned above.

[0043] The local radio maps obtained by the node are represented in 210 1 , 210 2 ,...,210 K N j from the messages received from its neighboring nodes. For example, the map 210 1 can give the local topology of the network, the map 210 2 can give RSS information of signal received from N j , etc.

[0044] These maps are weighted using weighting coefficients a j k which will have been previously determined in a learning phase, this learning depending on the objective function, then combined in 220.

[0045] The hybrid (or augmented) mapping 230, resulting from the combination of the different radio characteristic maps, is used to make radio resource management decisions.

[0046] There Fig. 3 schematically represents the architecture of a network node making it possible to implement a method of managing radio resources according to an embodiment of the present invention.

[0047] The node considered, 300, is the node of interest N j of the Fig. 1 . We recall that each node N i , i ≠ j, from the neighborhood V j ( t ) makes an observation o i ( t, f ) of its environment, extracts information relating to the radio characteristic k then transmits a message m i , j k t f = g j k o i t f at the knot N j .

[0048] Node 300 has a plurality K of 310 k aggregation modules, k =1,..., K each 310k aggregation module aggregating messages m i , j k t f , i = 1,.., P j , received from the nodes of V j ( t ) to generate a local mapping of the radio characteristic k This mapping is stored in an associated local memory (not shown).

[0049] The local radio maps relating to the different characteristics are then merged in the combination module 320 to provide a hybrid (or augmented) local map. Φ j a t f .

[0050] The node 300 makes a decision relating to the management of radio resources in the decision module 330, so as to optimize an objective function. The combination and decision modules can be implemented, for example, by means of neural networks.

[0051] According to one embodiment, the decision module 330 only makes a new decision when a variation has occurred in one of the maps. Φ j k t f , k = 1,..., K .

[0052] The parameters of the fusion modules and the decision module are determined by learning, advantageously by means of reinforcement learning, as described later.

[0053] There Fig. 4 schematically represents a flowchart of the radio resource management method according to an embodiment of the present invention.

[0054] We again consider a node of interest N j of the network.

[0055] This node 400 carries out in step 410 a local observation of its radio environment, o j ( t, f ) . It extracts information relating to each radio characteristic k = 1,..., K then transmits the messages m j , l k t f = g l k o j t f to the knots N ℓ for which N j ∈ V ℓ ( t ) .

[0056] In parallel, the node N j determines its neighborhood in step 415 V j ( t ) right now t in the sense of a metric as described later, and receives in 425 the messages m i , j k t f network nodes belonging to this neighborhood.

[0057] The knot N j generates local maps in 420 Φ j k t f , k = 1, ..., K of the different radio characteristics and merges them in 440 to obtain the augmented local mapping, Φ j a t f .

[0058] In parallel, local maps Φ j k t f updates are stored in 430 in a local memory 435 of the node, N j .

[0059] The knot N jmakes a radio resource management decision at 460, different options are possible: First, a radio resource management decision can be made at each time increment. Alternatively, such a decision will only be made if one of the current mappings Φ j k t f , k = 1,..., K differs significantly from that generated for the same feature at the previous time Φ j k t − 1 , f , option represented by test 450. Alternatively, such a decision will only be made if the current hybrid mapping Φ j a t f differs significantly from that relating to the previous moment, Φ j a t − 1 , f .Alternatively, finally, such a decision will only be taken if the hybrid mapping observed over a given time window differs significantly from that observed at an instant preceding this window. In all cases, the detection of a variation in the mapping can be obtained from the KL divergence (Kullback-Leibler divergence) between two successive mappings. For example, in the first case, a significant variation will be detected if: [Math.8] ∃ k ∈ 1 , … , K tel que D KL Φ j k t f Φ j k t − 1 , f > ε TH Or D KL is the KL divergence and ε TH is a positive real number representing a predetermined variation threshold.

[0060] The radio resource management policy followed by the node N jis determined by means of reinforcement learning. Recall that a reinforcement learning method is a machine learning method in which an autonomous agent, immersed in an environment, learns actions to perform from experiences, in order to optimize a cumulative reward over time. The agent makes decisions based on its state in the environment according to a Markovian decision process and the environment in return provides it with rewards based on the actions it performs at each moment.

[0061] In this case, the agent is a node in the network, in this case the node N j , and the state of the node in the environment V j ( t ) is represented by the augmented local mapping Φ j a t f . The objective of learning here is to determine the radio resource management strategy π j , Θ a j t Φ j a t f ,i.e. the probability for the node to decide to perform the action aj ( t ) ∈ A Or A is the set of possible actions when the augmented local mapping is Φ j a t f . The whole Θ parameters parameterizing the probability distribution π j , Θ a j t Φ j a t f , as well as the fusion parameters, for example matrices W q , j h , W k , j h , W v , j h , W Φ occurring in the third implementation example are learned by means of end-to-end learning ( end-to- end), that is to say concerning both the combination module 330 and the decision module 340. The learning can be carried out, in a manner known per se, by a strategy gradient method ( policy gradient method ) described in RS Sutton's book "Reinforcement Learning" published by MIT Press in 2018.

[0062] The radio resource management method described above assumes that each node N j can determine its neighborhood (see step 415), and therefore the nodes which will participate in the message passing service.

[0063] The neighborhood V j ( t ) of the node can be determined by selecting (or subsampling) from among the nodes in the network, or a subset thereof, those that satisfy a certain metric δ , operating in a space of representation of local observations: [Math.9] V j t = N i δ o i o j ≤ d ; N i ∈ U

[0064] The metric δ can be a cosine similarity ( cosine similarity ) based on a scalar product of normed vectors. Alternatively, δ can be considered as a classification process where δ ( o i , o j ) defines the probability that the observation o j is in the same class as o i . The classification operator can be trained online in a centralized and supervised manner as shown in Fig. 5 .

[0065] At step 510, the node N j collects local observations from the set U of network nodes or from a subset consisting, for example, of those within reception range of N j . These local observations are represented by vectors o i .

[0066] These vectors are then subject to pre-processing (for example normalization) in 520 before being projected into a reduced-dimensional representation space in 530.

[0067] A K-means clustering algorithm, 540, then makes it possible to group the nodes in the form of aggregates and then to assign, 550, the same label to the nodes of the same aggregate. A classifier can finally be trained offline ( offline ) in a supervised manner on the node-label pairs thus created, 560.

[0068] Once trained, the classifier can determine for a given node, N j ,the nodes of the network or a subset of it that belong to the same aggregate as N j . These nodes form the neighborhood V j ( t ) of N j .

[0069] The method of managing the radio resources of a cellular network from a local map augmented in each node of the network or even in nodes of a certain type only (base stations BS, SBS, MBS, relays, etc.) allows decisions to be made in a distributed manner while taking into account the heterogeneity of the latter. What is more, when all the parameters Θ has been learned for one frequency range (e.g. sub-6 GHz), it can be transferred to combination and decision modules operating in another frequency range (millimeter) for further learning and adaptation to this other range. The transferability of the parameters thus makes it possible to accelerate the learning phase in a new environment.

[0070] By way of illustration, we will describe below two examples of application of a radio resource management method according to the present invention.

[0071] The first example concerns the association of user equipment (UEs) in a 5G network. We recall that an association method aims to determine, for each mobile terminal, the base station (in other words the cell) which will have to serve it, taking into account the needs of all users (throughput, signal-to-noise ratio, latency, etc.) and the constraints relating to the base stations (maximum transmission power, interference level, available bandwidth, etc.). We assume that the 5G network has N m small cells ( small cells ) equipped with SBS base stations operating in the millimeter band and N s macro-cells ( macro cells) equipped with MBS base stations in the sub-6 GHz band. All user terminals are noted here U ( t ) and that of access points, of cardinal N m + N s is noted S .

[0072] Each user terminal or UE N j ∈ U ( t ) can perform an action aj ( t ) ∈ A corresponding to a request for association with a base station (SBS or MBS). A is the set of possible association requests, corresponding here in a one-to-one manner to the set S base stations.

[0073] Observations from the different EUs Neither ∈ V j ( t ) are given by the vectors: [Math.10] o i t f = R t − 1 , D i t , RSS i , b t f , AoA i , b t f , I i , b t − 1 , f , QoS i , b t − 1 , f b ∈ S Or R ( t -1) is the total capacity of the network at the previous instant t -1, D i ( t ) is the flow rate required by the mobile terminal Neither right now t, RSS i,b ( t, f ) is the signal strength received by the UE Neither of the base station b ∈ S right now t and at the frequency f, AoA i,b ( t, f ) is the angle of arrival of the signal received from this base station, I i,b ( t - 1 , f ) is the measured interference level and QoS i,b ( t - 1 , f ) the level of quality of service, as perceived at the previous moment. Note that the total capacity of the network R ( t ) is not known at the time of observation since the decision to associate N j is not yet taken, so it is its value at the previous moment t - 1 which is taken into account in the observation vector. The same applies to interference and quality of service levels.

[0074] The neighborhood V j ( t ) is then determined as described previously in relation to the Fig. 5 .

[0075] The messages m i , j k t f = g j k o i t f received by the node N j are taken into account by means of a multi-headed (H) attention mechanism. The message transmitted by the node N j at the knot N i corresponding to the characteristic h ∈ {1,..., H} is defined by: [Math.11] m i , j h t f = k i , j h v i , j h with k i , j h = W k , j h o j T And k i , j h = W k , j h o j T by taking up the previous notations. The augmented local mapping Φ j a t f is obtained by merging the local mappings, by expressions (6) and (7). The parameters Θ parameterizing the strategy π j , Θ a j t Φ j a t f are chosen in such a way as to maximize the reward over time, for example maximizing the flow rates or QoS levels of the different communications, or minimizing the handover frequency.

[0076] The second example concerns the positioning of mobile access points or MAPs ( Mobile Access Points ) for example access points installed on drones or UAVs ( Unmanned Aerial Vehicles ) in a 5G network. Such mobile access points allow for rapid deployment and reconfiguration of a network, for example in emergency situations or in a theater of operations. Managing radio resources in such a context, including the number of drones and their positioning, is particularly complex due to the mobility of both UEs and MAPs.

[0077] In this example, it is assumed that the MAPs operate in the millimeter frequency range and are directly connected to the backbone network ( backhaul network ). The network also includes N s MBS base stations in the sub-6 GHz band. For simplicity, we consider a single MBS base station (a single macrocell) and a plurality N m of mobile access points MAPs. As before, we note U ( t) all user terminals, each user terminal is assumed to have 2 antennas allowing it to establish either a link with a MAP or a link with an MBS. All access points S is cardinal N m + 1 ( N m MAPs and an MBS), the number of MAPs being chosen equal to the number of aggregates, N c ( t ) , identified in the network. These aggregates can therefore be designated by C b ( t ) , b ∈ {1,..., N c ( t )} , each aggregate b consisting of user equipment N j , j ∈ C b ( t ) . Each mobile hotspot MAP b , can make a decision a b ( t ) ∈ A Or Ais the set of all possible movement actions. For example, this set can consist of 6 incremental movements (positive and negative along the 3 axes X,Y,Z) and a stationary state (no movement at the moment t ).

[0078] The environment of a mobile hotspot MAP b consisting of EUs N j ,j E C b ( t ) , each EU belonging to the aggregate b transmits to the node MAP b a message relating to the characteristic h : [Math.12] m j , b h t f = k j , b h , v j , b h Or k j , b h = W k , b h o j T And v j , b h = W v , b h o j T are respectively the key and value vectors relative to the MAP b . The observation vector is o j is given by o j ( t, f ) = { x j ( t ) , y j ( t ) , R j ( t )} Or ( x j ( t ) , y j ( t )) are the spatial coordinates of the EU N j And R j ( t) is the throughput perceived by this UE.

[0079] The mobile hotspot MAP b aggregates received messages using a multi-attention mechanism ( H ) heads and generates, for each radio characteristic h, a local map Φ b h t f = v j , b h , ∀ j ∈ C b t T . These local mappings are then merged using an attention mechanism, as described in relation to expressions (6) and (7), namely: [Math.13] Φ b a t f = W Φ . α b h ϕ b h p t f ; h = 1 , … , H T Or α b h = α j , b h ; ∀ j ∈ C b t is the score between the query q b h = W q , b h o b T from the mobile hotspot MAP b , Or o b refers to the local observation of MAP b (namely its 3D location) and the key vectors associated with the different observations from the aggregate UEs b : [Math.14] α j , b h = softmax q b h . k j , b h n where, as before, nis the dimension of the radio feature representation space. This score reflects the correlation between the dynamics of the mobile access point and those of the UEs belonging to the aggregate.

[0080] From augmented mapping Φ b a t f , the mobile hotspot MAP b makes a decision a b ( t ) ∈ A by means of the strategy π b , Θ a b t Φ b a t f .

[0081] The whole Θ parameters parameterizing the probability distribution π b , Θ a b t Φ b a t f , as well as the fusion parameters, namely the matrices W q , b h , W k , b h , W v , b h , W F are learned using end-to-end reinforcement learning, aiming to maximize a reward over time.

[0082] As in the previous example, reward maximization can be expressed as maximizing the throughput or QoS levels of the different communications, or minimizing the handover frequency.

Claims

1. Method for managing radio resources in a cellular network comprising a plurality of nodes, wherein for each node of interest of the network, a neighbourhood of this node of interest is determined, each node of said neighbourhood performing a local observation of its environment and by extracting a plurality of radio characteristics, characterised in that: - each node of said neighbourhood encodes each of the radio characteristics in the form of a message and transmits this message to the node of interest; - the node of interest generates a local mapping of each radio characteristic by aggregating the messages encoding this radio characteristic; - the node of interest merges the local mappings using fusion parameters to generate a hybrid local mapping of the different radio characteristics; - the node of interest decides at each time to perform an action from a finite set of possible actions, based on said hybrid local mapping and on a radio resource management strategy defined by a conditional probability parameterised distribution of each action, the set of fusion parameters as well as the set of parameters of the conditional probability distribution being subject to reinforcement learning in order to maximise a reward over time, the learning being dependent on a function-objective of the network.

2. Method for managing radio resources in a cellular network according to claim 1, characterised in that the reward to be maximised corresponds to a sum of bitrates or service quality levels on network communications to be maximised, or a handover frequency or energy consumption to be minimised.

3. Method for managing radio resources in a cellular network according to claim 1 or 2, characterised in that said neighbourhood of the node of interest is defined as a set of neighbouring nodes of the network, considering a similarity metric operating in a representation space of the local observations.

4. Method for managing radio resources in a cellular network according to claim 1 or 2, characterised in that said neighbourhood of the node of interest is determined by means of a previously trained classifier, operating in a representation space of the local observations.

5. Method for managing radio resources in a cellular network according to any one of the preceding claims, characterised in that the node of interest only decides to perform an action at a time if one of the local mappings of a radio characteristic at that time differs from the local mapping of the same radio characteristic at the previous time, the difference between the two local mappings being measured using a Kullback-Leibler divergence.

6. Method for managing radio resources in a cellular network according to one of claims 1 to 4, characterised in that the node of interest only decides to perform an action at a time if the hybrid local mapping at that time differs from the hybrid local mapping at the previous time, the difference between the two hybrid mappings being measured using a Kullback-Leibler divergence.

7. Method for managing radio resources in a cellular network according to any one of the preceding claims, characterised in that the fusion of local radio mappings uses an attention mechanism with H heads, with H < K, where K is the number of radio characteristics.

8. Method for managing radio resources in a cellular network according to claim 7, characterised in that hybrid local mapping is obtained by means of Φ j a t f = WΦ. α j h ϕ j h p t f ; h = 1 , … , H T , where α j h = α i , j h ; i = 1 , .. , P j is the score between a query q j h from the node Nj and keys associated with the different observations from the nodes from the neighbourhood Vj(t), the node of interest, ϕ j h p t f is the local mapping value of the characteristic h at point p, at time t and at frequency f, and WΦ is a matrix of size Pj × H, where Pj is the number of nodes in the neighbourhood Vj(t).

9. Method for managing radio resources in a cellular network according to claim 8, characterised in that the query of the node of interest is expressed by a vector q j h = W q , j h o j T , where W q , j h is a matrix of size n × Ω, where n is the dimension of the characteristic representation space and Ω is the size of the observation vectors and that the keys associated with the different observations from the nodes Ni of Vj(t) are expressed by the vectors k i , j h = W k , j h o j T .

10. Method for managing radio resources in a cellular network according to claim 9, characterised in that the score between the query q j h from the node Nj and a key associated with a node Ni from the neighbourhood Vj(t) is calculated by means of α i , j h = softmax q j h . k i , j h n where "." represents the scalar product.