Managing the delivery of power to the radio head

Abstract

A method is provided for managing the transmission of power from a local battery located proximate to at least one radio head, the method comprising: making a decision for transmitting power from the local battery to the at least one radio head in a future time window (1001). The decision is made by a machine learning model based on (i) a difference determination of output power data statistics including average power demand and peak power demand, and (ii) time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization. The method further comprises outputting a decision for transmitting power from the local battery to the at least one radio head in a future time window (1003).

Description

[Technical field]

[0001] The present disclosure relates generally to methods, and related methods and apparatus, for managing the delivery of power to a radio head from a local battery located in close proximity to the radio head. [Background technology]

[0002] Radio Access Networks (RANs) account for a large portion of the energy consumption of mobile networks. Increasing the number of radio units in a RAN is a common approach to expand capacity and coverage to improve service quality. Adding radio units may include adding more radio units of the same type and / or radio units for different radio access technologies (RATs) and bands. However, such approaches increase energy consumption.

[0003] In some approaches, the site dimensioning (e.g. power cables, circuit breakers, power supply units (PSUs)) is done based on the maximum radio power consumption of each radio unit, which can significantly affect and increase the input rating of the site fuses. Increasing the rating of the site fuses can have a significant impact on operational costs for the operator.

[0004] Future and existing RAN features (e.g., microsleep transmit (Tx), low energy scheduler solutions (LESS), multiple-input multiple-output (MIMO) sleep modes, cell sleep modes, etc.) will increase the sleep time of radio units and reduce power consumption. Such techniques will substantially increase the difference between the average power consumption and the peak power used by an installed radio unit. Summary of the Invention

[0005] Currently, one or more of certain challenges exist: There is a need to manage and / or transmit power in a manner that allows for determining the difference between the average power and the peak power of a wireless head, while still adding new radio access technologies (RATs) in a sustainable manner.

[0006] Certain aspects of the present disclosure and their embodiments may provide solutions to these and other problems.

[0007] According to various embodiments, a method is provided for managing the transmission of power from a local battery located proximate to at least one radio head, the method comprising making a decision on the transmission of power from the local battery to the at least one radio head in a future time window, the decision being made by a machine learning model based on (i) a determination of a difference in output power data statistics including average and peak power demands in a set of past time windows covering a defined period for the at least one radio head, and (ii) time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization in the future time window. The method further comprises outputting the decision on the transmission of power from the local battery to the at least one radio head in the future time window.

[0008] According to another embodiment, a computing device in a communication system for managing the transmission of power from a local battery located in proximity to at least one radio head is provided. The computing device includes at least one processor and at least one memory coupled to the at least one processor and storing program code executed by the at least one processor for performing operations of making a decision on the transmission of power from the local battery to the at least one radio head in a future time window. The decision is made by a machine learning model based on (i) a determination of a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time- and location-dependent cost data of charging and / or discharging of the local battery and grid utilization in the future time window. The operation further includes outputting the decision on the transmission of power from the local battery to the at least one radio head in the future time window.

[0009] According to another embodiment, a computing device in a communication system for managing the transmission of power from a local battery located in proximity to at least one radio head is provided. The computing device is adapted to perform operations including making a decision on the transmission of power from the local battery to the at least one radio head in a future time window. The decision is made by a machine learning model based on (i) a determination of a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time- and location-dependent cost data of charging and / or discharging of the local battery and grid utilization in the future time window. The operations further include outputting the decision on the transmission of power from the local battery to the at least one radio head in the future time window.

[0010] According to another embodiment, a computer program is provided, the computer program including program code executed by a processing circuit of a computing device in a communication system for managing the transmission of power from a local battery located proximate to at least one radio head to the at least one radio head. Execution of the program code causes the computing device to perform operations including making a decision on the transmission of power from the local battery to the at least one radio head in a future time window. The decision is made by a machine learning model based on (i) a determination of a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization in the future time window. The operation further includes outputting the decision on the transmission of power from the local battery to the at least one radio head in the future time window.

[0011] According to another embodiment, a computer program product is provided that includes a non-transitory storage medium including a program code executed by a processing circuit of a computing device in a communication system for managing the transmission of power from a local battery located proximate to at least one radio head to the at least one radio head. Execution of the program code causes the computing device to perform operations including making a decision on the transmission of power from the local battery to the at least one radio head in a future time window. The decision is made by a machine learning model based on (i) a determination of a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization in the future time window. The operation further includes outputting the decision on the transmission of power from the local battery to the at least one radio head in the future time window.

[0012] Certain embodiments may provide one or more of the following technical advantages: As described above, a method of some embodiments may manage the transmission of power to at least one radio head from a local battery located proximate to the at least one radio head. The method includes making a decision on the transmission of power from the local battery to the at least one radio head in a future time window. The decision is based on (i) a determination of a difference in output power data statistics including average and peak power demands in a set of past time windows covering a defined period for the at least one radio head, and (ii) time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization for the future time window. Toand outputting a decision on the supply of power from the local battery to at least one radio head for a future time window. As a result, technical advantages provided by the method may include energy cost savings based on charging and / or discharging the local battery for peak power demands. Furthermore, peak shaving at the radio head may be enhanced based on management of the local battery using ML. Such management may not only provide and / or optimize a local battery charging / discharging policy, but may also extend the life of the local battery due to the determination and output of the decision. [Brief description of the drawings]

[0013] The accompanying drawings, which are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this application, illustrate certain non-limiting embodiments of the inventive concepts.

[0014] [Figure 1] FIG. 1 is a schematic diagram showing a conventional RAN site deployment.

[0015] [Diagram 2] FIG. 2 is a schematic diagram showing the addition of a radio head to an existing conventional RAN site.

[0016] [Diagram 3] 3 is a schematic diagram illustrating an additional radio head of the RAN site of FIG. 2 in accordance with some embodiments of the present disclosure.

[0017] [Figure 4] Figure 1 shows a plot of the input power to four radio heads at a two-sector radio site over the course of a day.

[0018] [Diagram 5] 5 is the plot of FIG. 4 annotated in accordance with some embodiments of the present disclosure.

[0019] [Figure 6] FIG. 2 is a schematic diagram illustrating the state of charge of a local battery according to some embodiments of the present disclosure.

[0020] [Figure 7] 1 is a sequence diagram illustrating operations according to some embodiments of the present disclosure.

[0021] [Figure 8] 1 is a sequence diagram illustrating operations according to some embodiments of the present disclosure.

[0022] [Figure 9] 1 is a block diagram illustrating a computing device according to some embodiments of the present disclosure.

[0023] [Figure 10] and [Figure 11] 1 is a flowchart illustrating the operation of a computing device node according to some embodiments of the present disclosure.

[0024] [Figure 12] 1 is a block diagram of a communication system in which a communication system according to some embodiments of the present disclosure may be implemented.

[0025] [Figure 13] 1 is a block diagram of a network node according to some embodiments of the present disclosure.

[0026] [Figure 14] 1 is a block diagram of a host computer in communication with a user device according to some embodiments of the present disclosure.

[0027] [Figure 15] 1 is a block diagram of a virtualization environment according to some embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] The concept of the present disclosure will now be described more fully with reference to the accompanying drawings, which show examples of embodiments of the concept of the present disclosure. However, the concept of the present disclosure can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the concept of the present disclosure to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Elements of one embodiment may be implicitly assumed to be present / used in another embodiment.

[0029] The following description presents various embodiments of the disclosed subject matter. These embodiments are presented as instructional examples and are not to be construed as limiting the scope of the disclosed subject matter. For example, certain details of the described embodiments may be modified, omitted, or expanded without departing from the scope of the described subject matter.

[0030] Figure 1 is a schematic diagram illustrating a conventional RAN site deployment. As shown in Figure 1, a power source 101 is connected to three 4G radio heads (also called radio units) 103a, 103b, 103c via power supply line 105. The power source 101 also includes a backup battery (labeled "B" in Figure 1).

[0031] Figure 2 is a schematic diagram illustrating the addition of radio heads to an existing conventional RAN site (e.g., the RAN site of Figure 1). As shown in Figure 2, three additional radio heads 204a, 204b, 204c are added to the RAN site of Figure 1 and connected to the power source 101 via three additional power supply lines 105, so that there are six power lines extending from the power source 101 to each of the radio heads 103a, 103b, 103c, 204a, 204b, and 204c, respectively.

[0032] "Peak shaving" is a term that refers to the cutting of power at a utility when demand from customers is high. Several publications discuss architectures that control peak shaving for utilities and / or power transmission lines.

[0033] Currently, one or more challenges of some kind exist.

[0034] Architectures to control peak shaving for utilities and / or power transmission lines behave differently than power systems with integrated local backup batteries at the site for wireless communications, for example as shown in Figures 1 and 2. Wireless communications add non-trivial challenges and new variables with respect to time-varying wireless network loads and targeted key performance indicators (KPIs).

[0035] Existing approaches for site equipment include a backup battery (e.g. B in Figures 1 and 2) used as a backup in case of power outage. The site backup battery B is typically dimensioned according to the site's local regulations (e.g. time) and the average power consumption requirement of the site. As a result, the site dimensioning (e.g. power cables, circuit breakers, PSUs) is often done based on the maximum radio power consumption of each radio unit, which has a very large impact on and increases the input rating of the site fuses. However, increasing the site fuse rating can have a significant negative impact on operational costs for the operator.

[0036] In addition, the approach of adding additional radio units to existing RAN sites (e.g., as shown in Figure 2) lacks the determination of the difference between the average and peak power demands for the radio units, and instead adds more equipment (e.g., more power lines, fuses, etc.) to provide the power demand for the additional radio heads (other than backup battery power) from the grid via a grid-tied source. However, such a determination of the difference between the average and peak power is important to optimize / improve the design and operation of local batteries to make the process of adding new radio access technologies (RATs) more sustainable while reducing the total carbon footprint of wireless network operations.

[0037] There is a need to improve site equipment and methods for transmitting power in a manner that allows for the determination of the difference between the average power and peak power of the radio heads while adding new RATs in a more sustainable manner (e.g., so that peak power needs are supplied from local batteries rather than from the grid).

[0038] Certain aspects of the present disclosure and their embodiments may provide solutions to these and other problems. Certain embodiments of the present disclosure include a method that uses machine learning (ML) to determine the difference between the average power usage and the peak power usage of different radio heads. A local battery provides auxiliary power to handle the peak power consumption of the radio heads. The local battery is located in close proximity to the radio heads. A power line from a site power source provides average power to the radio heads.

[0039] Certain embodiments may provide one or more of the following additional technical advantages: As described above, according to some embodiments, a method for managing the transmission of power from a local battery located in proximity to at least one radio head is provided. The method includes making a decision on the transmission of power from the local battery to the at least one radio head in a future time window. The decision is performed by (i) determining a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) a machine learning model based on time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization for the future time window, and outputting a decision on the supply of power from the local battery to the at least one radio head for the future time window.

[0040] As a result of locating a local battery in close proximity to a radio head, making the determination, and outputting the determination on the transmission of power from the local battery, a reduction in the number or length of power lines (also referred to herein as "cables") required and a reduction in the number of fuses may be provided, and the expansion of a site with additional radio heads may be easily accomplished with fewer or shorter lengths of power lines. For example, when a radio head is added to an existing conventional RAN site (e.g., the RAN site of FIG. 1), based on the employment of a local battery in close proximity to the radio head (e.g., as shown in FIG. 3, further described herein), the number of power lines from the power source (e.g., power source 101) to the local battery (e.g., local battery 301) may remain the same (e.g., three power lines 105). This is in contrast to existing approaches for expanding a site to have additional radio heads, where new power lines need to be installed for the entire distance from the power source to the new radio head. In the example of FIG. 2, three additional power lines are added such that there are six power lines covering the entire distance between the power source (e.g., power source 101) and the radio heads (e.g., radio heads 103a-103b and 104a-104c as shown in FIG. 2). Furthermore, based on the adoption of local battery management for wireless peak power, the operator of the wireless system can provide robustness against grid disturbances since the local battery is an active power source (e.g., not simply a backup power source). Further technical advantages of including a local battery in close proximity to the radio head(s) for managing the delivery of power to the radio head(s) may contribute to a zero-emission RAN (e.g., using the grid to power the radio heads and / or charging a backup battery located at the grid power source), which may be used to charge the local battery. green This may include increased use of energy sources as well as increased network robustness to grid disturbances.

[0041] Figure 3 is a schematic diagram showing additional radio heads 204a, 204b, 204c of Figure 2, but with a RAN site configured according to some embodiments of the present disclosure. The architecture of Figure 3 can realize peak shaving of local radio heads based on installing a local battery 301. According to certain embodiments of the present disclosure, a method for management of peak power consumption of at least one radio head 103a-103c, 204a-204c from the local battery 301 includes differentiation (determining the difference) between average power and radio peak power and managing the transmission of power from two different power sources 101, 301 during normal network operation.

[0042] In FIG. 2, backup battery B is a backup power source, and additional radio heads 204a-204c are added to a site that includes a different RAT, e.g., LTE-enhanced radio head 204c. In contrast, FIG. 3 illustrates a site design that adds a local battery 301, which achieves peak shaving while further adding additional radio heads 204a-204c, while reducing the number of cables and fuses required. According to the exemplary embodiment of FIG. 3, the number of cables 105 from the power source 101 is reduced compared to FIG. 2, based on including the local battery 301 as an active power source (i.e., not a battery backup). In FIG. 2, the addition of radio heads 204a-204c also adds three power cables 105 that cover the entire distance between the power source 101 and the radio heads 204a-204c, such that there are six power lines 105 that cover the entire distance between the power source 101 and the radio heads 103a-103c and 204a-204c, respectively. In contrast, in Figure 3, adding wireless heads 204a-204c does not add power lines 105 between the power source 101 and the local battery 101. Thus, in contrast to Figure 2, the number and / or length of power lines 105 is reduced in Figure 3. Furthermore, in Figure 3, power injected from the grid through the power source 101 can be reduced because the local battery 301 is an active source of power for peak power demand.

[0043] Figure 4 is a plot of the input power to four radio heads (radio heads 1, 2, 3, 4 as shown in the legend) in a two-sector radio site over a defined period of approximately 30 seconds. The solid and dashed lines represent radios 1 and 2 in the first sector. The dotted and dashed lines represent radios 3 and 4 in the second sector. As can be seen from the plot illustrated in Figure 4, the radio heads have average power in watts and power peaks over the period.

[0044] 5 illustrates the plot of FIG. 4 annotated with horizontal lines 501, 503, 505, 507 illustrating the determination of the difference between the average and peak powers of the wireless heads over a period of time in accordance with some embodiments of the present disclosure. It can be seen from the illustrated plot that, according to the exemplary embodiment of FIG. 5, given the separation between the average powers shown by lines 503, 507 and the peak powers shown by lines 501, 505, respectively, with proper dimensioning of the power lines 105, the average powers 503, 507 of the wireless units can be carried and the peak powers 501, 505 can be controlled from the local battery 301.

[0045] 4 and 5 are described with reference to four radio heads and a defined period of approximately 30 seconds, embodiments of the present disclosure are not so limited. Alternatively, any quantity of one or more radio heads may be included, and the defined period may be any period defined in seconds, minutes, hours, days, weeks, etc. Additionally, while two peak power settings and two average power settings are shown for the plot shown in FIG. 5, embodiments of the present disclosure are not so limited. Alternatively, a peak power setting may be identified / determined (as further described herein) for each radio head and / or for one or more radio heads. Additionally, an average power setting may be identified / determined (as further described herein) for each radio head and / or for one or more radio heads. Additionally, although some embodiments are described herein with reference to average power supplied by the power grid and peak power supplied by a local battery, embodiments of the present disclosure are not so limited. Alternatively, an amount of power greater than or less than the peak power may be supplied by the local battery, and the remaining power demand of the radio head may be supplied from the power grid.

[0046] According to certain embodiments of the present disclosure, the ML-based method can manage the charging and discharging policies of the local battery 301 for peak shaving based on the time variation of wireless power peak demand as well as the grid energy price and carbon footprint profile for the time period.

[0047] According to some embodiments, the following observations are taken into account: - There may be significant variations in the input and output power of radio heads. Some radio heads are very active at high output power, while others rarely reach high output power. ●There is a high correlation between the peak power of the radio head relative to the input power from the site power supply (and grid) and the output power of the radio head (which peaks at peak traffic loads). For active time windows during which the radio head is substantially inactive, a differentiation is required between the average radio power and the peak radio power (see e.g. FIG. 4). Such a differentiation can support efficient operation of the local battery 301. ● The output power of a single radio head is highly correlated with time of day (e.g., over the course of an overnight period).

[0048] According to some embodiments, these observations are used to build ML methods for managing a local battery (eg, battery 301).

[0049] According to some embodiments, a defined time period (e.g., a day) is divided into a set of time windows (e.g., hourly) and decision variables are generated for future time windows. The method uses data collected or accessed over time derived from the power consumption of wireless heads (including wireless traffic variations) to train an ML model (e.g., an RL agent).

[0050] Training of the RL agent includes at least the following inputs: (1) local battery capacity, (2) average power consumption of all wireless heads in K past time windows among the set of time windows for a given integer K>0, (3) peak power consumption of all wireless heads in K past time windows among the set of time windows, and (4) charging cost of the local battery and cost of using the grid in the current or future time window. The RL agent optimizes a charging state decision including when to charge the local battery for the start of the current / future time window (e.g., each time window), when and how much to discharge the local battery for the wireless head, and does not transmit power from the local battery to the wireless head (e.g., if the charging state decision is to determine that the amount of discharge from the local battery for the wireless head is zero).

[0051] FIG. 6 is a schematic diagram illustrating a state of charge (SOC) of a local battery according to some embodiments of the present disclosure. As shown in FIG. 6, the SOC level of the local battery ranges from 0% to 100%. The ML model (e.g., RL model) defines / sets the SOC of the local battery for the predicted radio peak power in the future time window(s) (e.g., at each future time window). The SOC level can vary between 0%-100% and is defined at the SOC level depending on the predicted peak power. According to the example embodiment of FIG. 6, the SOC level may be set for the future time window at the SOC level indicated by the three arrows. Although the example of FIG. 6 shows three SOC levels, the present disclosure is not so limited and the SOC level may be defined at any non-zero SOC level depending on the predicted peak power.

[0052] According to some embodiments, the SOC decision (also referred to herein as “decision”) satisfies the following four constraints on the time window: C1. Constraint 1: The local battery discharge level to the radio head for any time window plus the input power (from the power cable) should be equal to the input power required for that radio head at that time. These time- and location-stamped input powers are inputs / constants to the ML model given in the dataset. C2. Constraint 2: The local battery discharge in any time window should not exceed the local battery level at the start of that time window plus the charge profile limit for that time window. C3. Constraint 3: The current local battery level is equal to the sum of the previous local battery level and the charge in this time window. C4. Constraint 4: The local battery level in any time window must not exceed the local battery capacity.

[0053] The objective of the approach is to minimize the total cost, including the cost of charging the local battery and the cost of using the grid, for future time window(s) by allowing the average power to the radio heads to be transmitted from the site power system based on the adoption of the above constraints.

[0054] The ML model is now further described. According to some embodiments, the ML model is an episodic RL agent, where each episode is of a defined duration (e.g., one day). According to some embodiments, the RL agent is trained in the cloud and then installed at the battery management service location (e.g., deployed on the site's local battery).

[0055] According to some embodiments, the RL agent uses a dataset of power consumption of wireless heads over a defined period of time and determines the difference between wireless average power and wireless peak power for one episode or more episodes (e.g., several days).

[0056] According to some embodiments, the actions include determining charging and discharging at least one local battery for the radio head.

[0057] According to some embodiments, the status includes the current local battery level, the current output power of all radio heads, and the current local battery charging cost.

[0058] According to some embodiments, the reward of the RL agent for every state-action pair is to minimize the total cost of input power in the next time window in the set of time windows. The total cost is a weighted sum of the power from the grid and the power from the local battery, weighted based on their own costs. According to some embodiments, the total cost includes, among others, a monetary cost of energy or a carbon footprint index.

[0059] An exemplary embodiment will now be described with reference to Figure 7. Although the exemplary embodiment of Figure 7 is described in the context of Figure 3, which shows one local battery and six radio heads, various embodiments of the present disclosure are not so limited and may include a different number of local batteries and different numbers or types of radio heads. Additionally, although certain embodiments are described with reference to one local battery for ease of explanation, various embodiments of the present disclosure are not so limited and may include any number of local batteries.

[0060] 7, according to an exemplary embodiment, a total of R wireless heads (e.g., wireless heads 103a-103c and 204a-204c), indexed by i∈{1, 2,...,R} (R=6 according to this exemplary embodiment), are connected to a local battery 301 with a charge of C, as shown by connection 701, and further connected to a power grid via a power source 101, as shown by connection 703. In operation 705, a computing device (e.g., computing device 900 described further herein) divides each episode (i.e., a defined period for decision-making, such as every day) into a set of W time windows of predefined duration T, and calculates the time windows B. w denotes the level of the local battery 301 at the beginning of a future time window w from the set of time windows. As shown in operation 709, in every window w, the i-th wireless head of the plurality of wireless heads (e.g., a wireless head as shown for wireless head 103a in FIG. 7) will have a maximum power p iw max , average power p iw avg , and the instantaneous power p iw (t). For each time window w, the local battery 301 has d iw b0. As shown in box 711, in every time window w, the charging of the local battery 301 is performed according to a predefined charging profile h w b (t) and cost profile c w b By following (t), the total charge amount during this time window is

number

number

number

[0061] Continuing to refer to FIG. 7, the following constraints are included for each time window in the set of time windows: First constraint: Input power to the radio head: d iw b +d iw g =p iw (t). Second constraint: Local battery level (this includes constraints 2-4 above): B w +H w b -Td iw b ∈[0,C].

[0062] The operating cost (e.g., electricity cost) in any time window is C_w^b+C_w^g, which is f(d 11 b ,d 12 b ,...,d RW b )=Σ w∈W C w b + Σ i∈R C iw g for the entire episode. Thus, the optimization problem, where f is the reward function, is formulated as follows: Subject to constraints C1 and C2, minimize f(d 11 b ,d 12 b ,...,d RW b )

[0063] The reward function f is therefore given by f(d) for the entire episode constrained by C1 (i.e., in any time window, the local battery discharge level to the radio head plus the power input (from the power cable) should be equal to the input power required by the radio head at that time) and C2 (i.e., the local battery discharge in any time window must not exceed the local battery level at the start of this time window plus the charge profile limit for that time window). 11 b ,d 12 b ,...,d RW b )=Σ w∈W C w b + Σ i∈R C iw g This is to minimize the cost.

[0064] As shown in operation 715, the RL agent of the computing device 900 solves an optimization problem. According to some embodiments, the RL agent calculates the power consumption {p iw (t)}, as well as charging the local battery 301 via the power source 101 and powering the grid c w g Cost profile using (t) To The RL agent can then use that dataset to simulate an environment.

[0065] FIG. 8 is a sequence diagram illustrating operations for managing peak power demand from a local battery to one or more radio heads according to some embodiments of the present disclosure. According to the exemplary embodiment of FIG. 8, the ML model is an RL agent 803. The environment 801 is (i) a simulator based on data collected during training of key performance indicators (KPIs) and power data, or (ii) an actual network where the KPIs and power data are measured after the RL agent 803 and the local battery are installed. The KPIs include, but are not limited to, local battery and cost data including a capacity of the local battery, a first cost for charging the local battery in a future time window, and a second cost for using power from the grid for an average power demand of at least one radio head in a future time window. The power data includes, but is not limited to, (i) an average power demand of at least one radio head in a set of past time windows, (ii) a peak power demand of at least one radio head in a set of past time windows, and (iii) a traffic load of at least one radio head in a set of past time windows.

[0066] 8, at operation 807, the RL agent 803 gets input data from the environment 801, including KPIs and input power consumption. At operation 809, the RL agent 803 maps the input to an environment state and sends the state to the reward computing device 805. The RL agent 803 computes the next action to be taken and sends the action to the reward computing device 805 at operation 811. The reward computing device 805 computes a reward at operation 813. At operation 815, the RL agent 803 receives the reward and then improves the RL model and actions based on the reward at operation 817.

[0067] Continuing to refer to FIG. 8, according to some embodiments, during training, the loop of operations 807-819 continues to determine a local battery management policy (eg, an optimal battery management policy).

[0068] According to some embodiments, after determining the local battery management policy, the trained RL agent 803 is deployed in the real network. According to some embodiments, the RL agent 803 is deployed at every site (e.g., battery management service location) or in the cloud. The deployed RL agent 803 can access data from the network and monitor KPIs for potential retraining / tuning of the RL agent. Operations 807-819 of the loop are executed in the real network to determine the local battery management policy.

[0069] According to some embodiments, the at least one local battery has a set of capabilities for making the at least one local battery an auxiliary power source, including determining and executing active charging and discharging to minimize some cost function (e.g., for an operator).

[0070] According to some embodiments, at least one local battery is installed near the radio head, and the at least one local battery is responsible for only the peak power consumption. The power cable coming from the site power system for the radio head (e.g., a power source 60 m away from the radio head on site) is set to only deliver average power to the radio head. The at least one local battery has a local calculation or SC decision to be responsible for the peak power.

[0071] According to some embodiments, the method determines the difference between the average power and the radio peak power consumption of several radio heads in a state space, and a decision is provided that enables the local battery to transmit only the peak power to the radio heads.

[0072] According to some embodiments, the method calculates daily traffic variations and power demands and enables operation of at least one local battery (eg, charging or discharging for peak power demands).

[0073] FIG. 9 is a block diagram illustrating elements of a computing device 900 (also referred to as a server, cloud-based server, edge server, radio access network node, base station, radio base station, eNodeB / eNB, gNodeB / gNB, or any other functional physical or virtual network node in which a machine learning model may be implemented, as disclosed herein) of a communication network (e.g., communication network QQ100 configured to provide communication as described below with respect to FIG. 12) according to an embodiment of the concepts of the present disclosure. The computing device 900 may be provided as described below with respect to, for example, network nodes QQ110A, QQ110B of FIG. 12, network node QQ300 of FIG. 13, hardware QQ504, and / or virtual machines QQ508A, QQ508B of FIG. 15, all of which should be considered interchangeable in the examples and embodiments described herein, unless otherwise noted, and this is within the intended scope of the present disclosure. As shown, the computing device may include a computer 901 communicatively connected to an ML model 911. The ML model 911 may be an episodic RL agent. The ML model 911 may also include a reward computing unit (e.g., reward computing unit 705). The computer 901 may include a transceiver circuit (e.g., corresponding to a portion of the RF transceiver circuit QQ312 and the radio front-end circuit QQ318 of FIG. 13, also referred to as a transceiver) including a transmitter and a receiver configured to provide uplink and downlink wireless communication with a mobile terminal. The computer may include a network interface circuit 907 (e.g., corresponding to a portion of the communication interface circuit QQ312 and the radio front-end circuit QQ318 of FIG. 13, also referred to as a transceiver) configured to provide communication with other devices or nodes of a communication network (e.g., other network nodes, communication devices, and / or data repositories). centreIt may include a network interface (also referred to as) corresponding to a part of Ace QQ306. The computing device may also include a processing circuit 903 (also referred to as a processor corresponding to the processing circuit QQ302 in FIG. 13, for example) coupled to the transceiver circuit, and a memory circuit 905 (also referred to as a memory corresponding to the memory QQ304 in FIG. 13, for example) coupled to the processing circuit. The memory circuit 905 may include computer-readable program code that, when executed by the processing circuit 903, causes the processing circuit to perform operations according to the embodiments disclosed herein. According to other embodiments, the processing circuit 903 may be defined to include memory such that a separate memory circuit is not required.

[0074] As described herein, the operations of the computing device may be performed by the processing circuit 903, the network interface 907, and / or the transceiver. For example, the processing circuit 903 may control the transceiver to transmit downlink communications to one or more mobile terminals UE via the wireless interface through the transceiver, and / or centre transmit downlink communications to one or more mobile terminals UE via the wireless interface through the transceiver, and / or centreThe processing circuitry 903 may receive uplink communications from one or more communication devices via the network interface. Similarly, the processing circuitry 903 may control the network interface 907 to transmit communications to one or more other devices or network nodes via the network interface 907 and / or receive communications via the network interface from one or more network nodes, radio heads, local batteries, communication devices, etc. Additionally, modules may be stored in the memory 905 that, when their instructions are executed by the processing circuitry 903, may provide instructions such that the processing circuitry 903 performs respective operations (e.g., operations described below with respect to the exemplary embodiments related to a computing device). According to some embodiments, the computing device 900 and / or elements / functionality thereof may be embodied as a virtual node / nodes and / or virtual machines / machines, e.g., as described with respect to FIG. 15.

[0075] According to some other embodiments, the computing device may be implemented as a core network node without a transceiver. According to such embodiments, a transmission to a communication device, network node, radio head, local battery, etc. may be initiated by the computing device 900 such that the transmission to the communication device, network node, etc. is provided through the computing device 900 that includes a transceiver (e.g., through a base station or RAN node). According to embodiments in which the computing device is a RAN node that includes a transceiver, initiating the transmission may include transmitting through the transceiver.

[0076] The operation of a computing device (e.g., a computing device including an ML model 911) (implemented using the structure of FIG. 9) will now be discussed with reference to the flowcharts of FIGS. 10 and 11 according to some embodiments of the present disclosure. In the following description, the computing device may be any of the computing device 900, the network nodes QQ110A, QQ110B, QQ300, QQ606, the hardware QQ504, or the virtual machines QQ508A, QQ508B, but the computing device 900 shall be used to describe the functionality of the operation of the computing device. For example, modules may be stored in the memory 905 of FIG. 9, and these modules may provide instructions such that the processing circuitry 903 performs the respective operations of the flowcharts when the instructions of the modules are executed by the processing circuitry 903 of the respective computing device.

[0077] Referring to FIG. 10, a method is provided for managing the transmission of power from a local battery located in proximity to at least one radio head to at least one radio head, the method being performed by a computing device (900) in a communication system. The method includes making a decision (1001) for a future time window on the transmission of power from the local battery to at least one radio head. The decision is made by a machine learning model based on (i) a determination of a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization in the future time window. The method further includes outputting (1003) a decision on the transmission of power from the local battery to at least one radio head in the future time window.

[0078] According to some embodiments, the output power data statistics include at least (i) an average power demand of the at least one radio head over a set of past time windows, and (ii) a peak power demand of the at least one radio head over a set of past time windows.

[0079] According to some embodiments, the time-location dependent cost data includes a first cost for charging a local battery in a future time window and a second cost for using power from the grid for an average power demand of the at least one wireless head in the future time window.

[0080] According to some embodiments, the decision includes one of the following for a future time window (i) where power is transferred from the local battery to the at least one wireless head during the future time window, the future time window including a period of peak power consumption in the at least one wireless head, (ii) charging the local battery during the future time window, and (iii) not transferring power from the local battery to the at least one wireless head during the future time window.

[0081] According to some embodiments, the determination is made by a machine learning model based on (i) inputting the power data and the battery and cost data into the machine learning model, and (ii) determining a minimized total cost for transmitting power to the at least one wireless head for a future time window, where the future time window is constrained by multiple constraints.

[0082] According to some embodiments, the constraints include (i) a first constraint set as an input power consistency, where the input power to the radio head is equal to the discharge level of the local battery to the radio head plus the power from the grid, and (ii) a second constraint, where the discharge is set such that the amount of discharge from the local battery is less than or equal to the charge level of the local battery at the start of the future time window plus the charge profile limit of the future time window.

[0083] According to some embodiments, the charge profile limits for the future time windows include further constraints, including (i) a third constraint that the current charge level of the local battery is set such that the current charge level is equal to the previous (immediately preceding) charge level of the local battery plus the charge minus the discharge of the local battery in the current time window, and (ii) a fourth constraint that the charge level is set such that the charge level of the local battery in the set of time windows is less than or equal to the charge of the local battery.

[0084] According to some embodiments, the machine learning model is action With pair The state includes a current charge level of the local battery, a current input power of the at least one wireless head, and a current cost to charge the local battery. The action in the state and action pair includes a decision.

[0085] 11, according to some embodiments, the at least one radio head includes a plurality of radio heads. The method further includes dividing the defined period into a set of time windows (1101). The method further includes making a decision (1103) for each time window in the set of time windows, for each radio head.

[0086] Referring again to FIG. 10, according to some embodiments, outputting the decision (903) includes outputting a decision for controlling transmission of power to the at least one wireless head based on the decision.

[0087] According to some embodiments, the power data is offline data, and the determining (1001) and outputting (1003) is performed during training of a machine learning model that uses the offline data.

[0088] According to some embodiments, the power data is online data, the machine learning model is installed in the communication system, and the determination (1001) and output (1003) are performed by the installed machine learning model.

[0089] According to some embodiments, the computing device is located in one of a location proximate to a local battery and a cloud-based location.

[0090] Various operations from the flowchart of FIG. 11 may be optional with respect to some embodiments of the method performed by a computing device.

[0091] FIG. 12 illustrates an example of a communication network QQ100 in which a communication system according to some embodiments may be implemented.

[0092] In the illustrated example, the communication network QQ100 includes a telecommunications network QQ102 including an access network QQ104, such as a radio access network (RAN), and a core network QQ106 including one or more core network nodes QQ108. The access network QQ104 includes one or more access network nodes, such as network nodes QQ110a and QQ110b (one or more of which may be generally referred to as network node QQ110), or any other similar third generation partnership project (3GPP®) access node or non-3GPP® access point. The network node QQ110 facilitates direct or indirect connection of user equipment (UE), such as by connecting UE QQ112a, QQ112b, QQ112c, and QQ112d (one or more of which may be generally referred to as UE QQ112) to the core network QQ106 via one or more wireless connections. The network nodes QQ110A, QQ110B may be cloud-implemented network nodes (e.g., servers) or may be located in the cloud or edge-implemented network nodes (e.g., servers). The network nodes QQ110A, QQ110B facilitate direct or indirect connection of communication devices QQ112a, QQ112b, QQ112c, and QQ112d (one or more of which may be generally referred to as communication devices / UEs QQ112) to the communication network QQ100 via one or more wireless connections.

[0093] wirelessExemplary wireless communication over the connections includes transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared, and / or other types of signals suitable for conveying information without the use of wires, cables, or other conductive materials. Additionally, according to various embodiments, the communication network QQ100 may include any number of wired or wireless networks, network nodes, communication devices, and / or any other components or systems that may facilitate or be involved in the communication of data and / or signals, whether over wired or wireless connections. The communication network QQ100 may include and / or interface with any type of communication, telecommunication, data, cellular, wireless networks, and / or other similar types of systems, including, but not limited to, 5G and / or 6G networks. centre It is possible to use.

[0094] In the illustrated example, core network QQ 106 connects network node QQ 110 to one or more hosts, such as host QQ 116. These connections may be direct or indirect through one or more intermediate networks or devices. In other embodiments, the network nodes may be directly coupled to the hosts. Core network QQ 106 may include hardware Eh A and Software EhThe core network node may include one or more core network nodes (e.g., core network node QQ108) that are configured with a UE, a network node, and / or a host. Features of these components may be substantially similar to those described with respect to the UE, the network node, and / or the host, and therefore, the descriptions are generally applicable to the corresponding components of the core network node QQ108. Exemplary core network nodes include one or more functions of a Mobile Switching Center (MSC), a Mobility Management Entity (MME), a Home Subscriber Server (HSS), an Access and Mobility Management Function (AMF), a Session Management Function (SMF), an Authentication Server Function (AUSF), a Subscription Identifier Unhiding Function (SIDF), a Unified Data Management (UDM), a Security Edge Protection Proxy (SEPP), a Network Exposure Function (NEF), and / or a User Plane Function (UPF).

[0095] Host QQ 116 may be under the ownership or control of a service provider other than the operator or provider of access network QQ 104 and / or telecommunications network QQ 102, and may be operated by or on behalf of the service provider. Host QQ 116 may host a variety of applications to provide one or more services. Examples of such applications include live and pre-recorded audio / video content, data collection services such as searching and compiling data on various ambient conditions detected by multiple UEs, analytics functions, social media, functions for controlling or otherwise interacting with remote devices, functions for an alarm and monitoring center, or any other such functions performed by a server.

[0096] In its entirety, the communication network QQ100 of Figure 12 enables connectivity between UEs, network nodes, and hosts. In that sense, the communication network may include any suitable wireless communication standard, such as Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), and / or any other suitable 2G, 3G, 4G, 5G standard, or applicable future generation standard (e.g., 6G), Wireless Local Area Network (WLAN) standard, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard (WiFi), and / or any Low Power Wide Area Network (LPWAN) standard, such as WiMAX, Bluetooth, Z-Wave, Near Field Communication (NFC), ZigBee, LiFi, and / or any Low Power Wide Area Network (LPWAN) standard, such as LoRa and Sigfox. Specific standards, etc. , but not limited to but , can be configured to operate according to defined rules or procedures.

[0097] In some examples, the telecommunications network QQ102 is a cellular network that implements 3GPP standardized features. Thus, the telecommunications network QQ102 may support network slicing to provide different logical networks to different devices connected to the telecommunications network QQ102. For example, the telecommunications network QQ102 may provide Ultra-Reliable Low-Latency Communication (URLLC) services to some UEs, while providing Enhanced Mobile Broadband (eMBB) services to other UEs, and / or Massive Machine Type Communication (mMTC) / Massive IoT services to additional UEs.

[0098] In an example, the hub QQ114 communicates with the access network QQ104 to facilitate indirect communication between one or more UEs (e.g., UEs QQ112c and / or QQ112d) and a network node (e.g., network node QQ110b). In some examples, the hub QQ114 may be a controller, a router, a content source and analysis, or any of the other communication devices described herein with respect to a UE. For example, the hub QQ114 may be a broadband router that enables access to the core network QQ106 for the UE. As another example, the hub QQ114 may be a controller that sends commands or instructions to one or more actuators in the UE. The instructions or instructions may be received from the UE, the network node QQ110, or by executable code, scripts, processes, or other instructions in the hub QQ114. As another example, the hub QQ114 may be a data collector that serves as a temporary store for UE data and may perform analysis or other processing of the data according to some embodiments. As another example, the hub QQ114 may be a content source. For example, for a UE that is a VR headset, display, loudspeaker, or other media distribution device, Hub QQ 114 may retrieve data related to VR assets, video, audio, or other media, or sensory information via a network node, which Hub QQ 114 then provides directly to the UE, either after performing local processing and / or adding additional local content. In yet another example, Hub QQ 114 acts as a proxy server or orchestrator for the UEs, especially when one or more of the UEs are low energy IoT devices.

[0099] Hub QQ114 may have a constant / persistent or intermittent connection to network node QQ110b. Hub QQ114 may also enable other communication schemes and / or schedules between hub QQ114 and UEs (e.g., UEs QQ112c and / or QQ112d) and between hub QQ114 and core network QQ106. In other embodiments, hub QQ114 is connected to core network QQ106 and / or one or more UEs via a wired connection. Furthermore, hub QQ114 may be configured to connect to an M2M service provider via access network QQ104 and / or to another UE via a direct connection. In some situations, a UE may be connected to a wired or wireless with network node QQ110 while still being connected through hub QQ114 via a connection wireless According to some embodiments, hub QQ 114 may be a dedicated hub, i.e. a hub whose main function is to route communications to / from network node QQ 110b to UEs. According to other embodiments, hub QQ 114 may be a non-dedicated hub, i.e. a device capable of operating to route communications between UEs and network node QQ 110b, but capable of further operating as a communication origin and / or termination point for a particular data channel.

[0100] FIG. 13 illustrates a network node QQ300 according to some embodiments (e.g., the network node QQ300 may include a network node capable of implementing a computing device 900). As used herein, a network node refers to a configured, arranged, and / or operative device that can communicate directly or indirectly with UEs and / or other network nodes or devices in a communication network. Examples of network nodes include, but are not limited to, a server, an access point (AP) (e.g., a wireless access point), a base station (BS) (e.g., a radio base station, a Node B, an evolved Node B (eNB), and an NR Node B (gNB)).

[0101] Base stations may be classified based on the size of the coverage they provide (or, stated differently, their transmission power level) and may therefore be referred to as femto, pico, micro, or macro base stations depending on the size of the coverage provided. A base station may be a relay node or a relay donor node that controls a relay. A network node may also include one or more (or all) parts of a distributed radio base station, such as a centralized digital unit and / or a remote radio unit (RRU), sometimes referred to as a remote radio head (RRH). Such remote radio units may or may not be integrated with an antenna as an antenna-integrated radio. Parts of a distributed radio base station may also be referred to as nodes in a distributed antenna system (DAS).

[0102] Other examples of network nodes include a multi-transmission point (multi-TRP) 5G access node, a multi-standard radio (MSR) equipment such as an MSR BS, a network controller such as a radio network controller (RNC) or a base station controller (BSC), a base transceiver station (BTS), a transmission point, a transmitting node, a multi-cell / multicast coordination entity (MCE), an operation and maintenance (O&M) node, an operation support system (OSS) node, a self-organizing network (SON) node, a positioning node (e.g., an evolved serving mobile location center (E-SMLC)), a minimization of drive testing (MDT), and / or a cloud-implemented server or an edge-implemented server.

[0103] The network node QQ300 includes a processing circuit QQ302, a memory QQ304, a communication interface QQ306, and a power supply QQ308. The network node QQ300 may be composed of multiple physically separate components (e.g., a clustering component, a database (e.g., knowledge graph) component, a NodeB component and an RNC component, or a BTS component and a BSC component, etc.), each of which may have its own respective components. In a scenario in which the network node QQ300 comprises multiple separate components (e.g., clustering and database components), one or more of the separate components may be shared among several network nodes. For example, a single network node may control multiple network nodes, including clustering components and / or databases (e.g., repositories). In such a scenario, each unique pair of network node and component may be considered as a single separate network node in some examples. In some embodiments, the network node QQ300 may be configured to support multiple radio access technologies (RATs). In such an embodiment, some components may be redundant (e.g., separate memories QQ304 for different RATs) and some components may be reused (e.g., the same database or the same antenna QQ310 may be shared by different RATs). Network node QQ300 may also include multiple sets of the various illustrated components for different wireless technologies integrated into network node QQ300, such as GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, RFID (Radio Frequency Identification), or Bluetooth wireless technologies. The wireless technologies may be integrated into the same or different chips or sets of chips and other components within network node QQ300.

[0104] The processing circuitry QQ302 may include one or more combinations of a microprocessor, controller, microcontroller, central processing unit, digital signal processor, application specific integrated circuit, field programmable gate array or other suitable computing device, resources or combinations of hardware, software and / or coded logic operable, alone or in conjunction with other network node QQ300 components, such as memory QQ304, to provide the functionality of network node QQ300.

[0105] In some embodiments, the processing circuit QQ302 comprises a system on a chip (SOC). In some embodiments, the processing circuit QQ302 comprises one or more of a radio frequency (RF) transceiver circuit QQ312 and a baseband processing circuit QQ314. In some embodiments, the radio frequency (RF) transceiver circuit QQ312 and the baseband processing circuit QQ314 may be on separate chips (or sets of chips), boards, or units, such as a radio unit and a digital unit. In alternative embodiments, some or all of the RF transceiver circuit QQ312 and the baseband processing circuit QQ314 may be on the same chip or set of chips, board, or unit.

[0106] The memory QQ304 may include any type of volatile or non-volatile computer readable memory, including, but not limited to, persistent storage, solid state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read only memory (ROM), mass storage media (e.g., hard disk), removable storage media (e.g., flash drive, compact disk (CD) or digital video disk (DVD)), and / or any other volatile or non-volatile non-transitory device readable and / or computer executable memory device that stores information, data and / or instructions that can be used by the processing circuit QQ302. The memory QQ304 may store any suitable instructions, data or information, including applications including one or more of computer programs, software, logic, rules, codes, tables, and / or other instructions, that are executable by the processing circuit QQ302 and usable by the network node QQ300. The memory QQ304 may be used to store any computational results produced by the processing circuit QQ302 and / or any data received via the communication interface QQ306. In some embodiments, the processing circuit QQ302 and the memory QQ304 are integrated.

[0107] The communication interface QQ306 is used in wired or wireless communication of signaling and / or data between network nodes, access networks, and / or UEs. As shown, the communication interface QQ306 includes a port / terminal QQ316 for transmitting and receiving data to and from a network over a wired connection, for example. The communication interface QQ306 also includes a radio front-end circuit QQ318 that is coupled to the antenna QQ310 or may be part of the antenna QQ310 in some embodiments. The radio front-end circuit QQ318 includes a filter QQ320 and an amplifier QQ322. The radio front-end circuit QQ318 may be connected to the antenna QQ310 and the processing circuit QQ302. The radio front-end circuit may be configured to condition signals communicated between the antenna QQ310 and the processing circuit QQ302. The radio front-end circuit QQ318 may accept digital data to be sent to other network nodes or UEs over a wireless connection. The radio front-end circuit QQ318 may convert the digital data into a radio signal having appropriate channel and bandwidth parameters using a combination of filters QQ320 and / or amplifiers QQ322. The radio signal may then be transmitted via antenna QQ310. Similarly, when data is received, antenna QQ310 may collect the radio signal, which may then be converted into digital data by radio front-end circuit QQ318. The digital data may be passed to processing circuit QQ302. In other embodiments, the communication interface may include different components and / or different combinations of components.

[0108] In an alternative embodiment, the network node QQ300 may not include a separate radio front-end circuit QQ318; rather, the processing circuit QQ302 may include the radio front-end circuit and may be connected to the antenna QQ310. Similarly, in some embodiments, all or some of the RF transceiver circuit QQ312 is part of the communication interface QQ306. In yet another embodiment, the communication interface QQ306 includes one or more ports or terminals QQ316, the radio front-end circuit QQ318, and the RF transceiver circuit QQ312 as part of a radio unit (not shown), and the communication interface QQ306 communicates with a baseband processing circuit QQ314 that is part of a digital unit (not shown).

[0109] Antenna QQ310 may include one or more antennas or antenna arrays configured to transmit and / or receive wireless signals. Antenna QQ310 may be coupled to radio front-end circuit QQ318 and may be any type of antenna capable of wirelessly transmitting and receiving data and / or signals. In some embodiments, antenna QQ310 is separate from network node QQ300 and connectable to network node QQ300 through an interface or port.

[0110] The antenna QQ310, the communication interface QQ306 and / or the processing circuit QQ302 may be configured to perform any receiving operation and / or any acquiring operation described herein as being performed by a network node. Any information, data and / or signals may be received from a UE, other network nodes and / or any other network equipment. Similarly, the antenna QQ310, the communication interface QQ306 and / or the processing circuit QQ302 may be configured to perform any transmitting operation described herein as being performed by a network node. Any information, data and / or signals may be transmitted to a UE, other network nodes and / or any other network equipment.

[0111] The power source QQ308 provides power to the various components of the network node QQ300 in a manner appropriate for each component (e.g., at the voltage and current levels required for each component). The power source QQ308 may further include or be coupled to a power management circuit for providing power to the components of the network node QQ300 to perform the functionality described herein. For example, the network node QQ300 may be connectable to an external power source (e.g., a power grid, an electrical outlet) via an input circuit or interface, such as an electrical cable, such that the external power source provides power to the power circuit of the power source QQ308. As a further example, the power source QQ308 may include a source of power in the form of a battery or battery pack connected to or integrated into the power circuit. The battery may provide backup power in case of failure of the external power source.

[0112] Embodiments of network node QQ300 may include additional components beyond those shown in Figure 13 to provide certain aspects of the network node's functionality, including any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, network node QQ300 may include user interface devices that allow for the input of information into network node QQ300 and the output of information from network node QQ300. This may enable a user to perform diagnostics, maintenance, repair, and other management functions on network node QQ300.

[0113] 14 is a block diagram of a host QQ 400, which may be a host of an embodiment of the host QQ 116 of FIG. 12 and may be capable of implementing a computing device 900 in accordance with various aspects described herein. As used herein, the host QQ 400 may be or include hardware and / or software in various combinations, including a standalone server, a blade server, a cloud-implemented server, an edge-implemented server, a distributed server, a virtual machine, a container, or processing resources in a server farm. The host QQ 400 may provide one or more services to one or more UEs.

[0114] The host QQ400 includes a processing circuit QQ402 operably coupled via a bus QQ404 to an input / output interface QQ406, a network interface QQ408, a power supply QQ410, and a memory QQ412. In other embodiments, other components may be included, the functionality of which may be substantially similar to those described with respect to the devices in previous figures, such as Figures 17 and 18, and therefore those descriptions are generally applicable to the corresponding components of the host QQ400.

[0115] The memory QQ412 may include one or more computer programs including one or more host application programs QQ414 and data QQ416, which may include user data, such as data generated by the UE for the host QQ400 or data generated by the host QQ400 for the UE. An embodiment of the host QQ400 may utilize only a subset or all of the illustrated components. The host application programs QQ414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Versatile Video Coding (VVC), High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), MPEG, VP9) and audio codecs (e.g., FLAC, Advanced Audio Coding (AAC), MPEG, G.711), including transcoding for different UE classes, types, or implementations (e.g., handsets, desktop computers, wearable display systems, heads-up display systems). The host application program QQ414 may also provide user authentication and license checks, and may periodically report health, route and content availability to a central node, such as a device in or at the edge of the core network. Thus, the host QQ400 may select and / or point to different hosts for over-the-top services for the UE. The host application program QQ414 may support a variety of protocols, such as HTTP Live Streaming (HLS) protocol, Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), Dynamic Adaptive Streaming over HTTP (MPEG-DASH), etc.

[0116] FIG. 15 is a block diagram illustrating a virtualization environment QQ500 in which functions of a computing device 900 implemented according to some embodiments may be virtualized. In this context, a virtualization means for generating a virtual version of an apparatus or device may include a virtualized hardware platform, storage devices, and networking resources. As used herein, virtualization may be applied to any device or component thereof described herein and relates to an implementation in which at least a portion of its functionality is implemented as one or more virtual components. Some or all of the functions described herein may be implemented as virtual components executed by one or more virtual machines (VMs) implemented in one or more virtual environments QQ500 hosted by one or more hardware nodes, such as a network node, a UE, a core network node, or a hardware computing device acting as a host. Furthermore, in embodiments in which a virtual node does not require wireless connectivity (e.g., a core network node or a host), the node may be virtualized in its entirety.

[0117] Application QQ502 (which may alternatively be referred to as a software instance, a virtual appliance, a network function, a virtual node, a virtual network function, etc.) runs in virtualization environment Q400 to implement some of the features, functionality and / or benefits of some of the embodiments disclosed herein.

[0118] The hardware QQ504 may include a processing circuit, a memory that stores software and / or instructions executable by the processing circuit, which is hardware, and / or hardware devices as described herein, such as a network interface and an input / output interface. The software may be executed by the processing circuit to instantiate one or more virtualization layers QQ506 (also referred to as a hypervisor or virtual machine monitor (VMM)), provide VMQQ508a and QQ508b (one or more of which may be generally referred to collectively as VMQQ508), and / or perform any of the functions, features, and / or benefits described in connection with some embodiments described herein. The virtualization layer QQ506 may present a virtual operating platform that appears as networking hardware to the virtual machine QQ508.

[0119] VMQQ508 may include virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and may be executed by a corresponding virtualization layer QQ506. Various embodiments of an instance of virtual appliance QQ502 may be implemented in one or more of VMQQ508, and the implementation may be done in various ways. Hardware virtualization is referred to in some contexts as network function virtualization (NFV). NFV may be used to aggregate many network equipment types into industry-standard, high-capacity server hardware, physical switches, and physical storage that may be located in data centers and customer premises equipment.

[0120] In the context of NFV, a VMQQ508 may be a software implementation of a physical machine that runs programs as if they were running on a physical, non-virtualized machine. Each VMQQ508 and the portion of the hardware QQ504 on which it runs, whether on hardware dedicated to that VM and / or shared by that VM with other VMs, forms a separate virtual network element. Also in the context of NFV, a virtual network function is responsible for handling a specific network function running in one or more VMQQ508 on top of the hardware QQ504 and corresponds to an application QQ502.

[0121] The hardware QQ504 may be implemented in a standalone network node with generic or specific components. The hardware QQ504 may implement some functions via virtualization. Alternatively, the hardware QQ504 may be part of a larger hardware cluster (such as in a data center or CPE) where multiple hardware nodes work together and are managed via a management and orchestration QQ510, which oversees, among other things, the lifecycle management of the application QQ502. In some embodiments, the hardware QQ504 is coupled to one or more radio units, each of which may include one or more transmitters and one or more receivers, which may be coupled to one or more antennas. The radio units may communicate directly with other hardware nodes via one or more appropriate network interfaces, or may be used in combination with virtual components to provide wireless capabilities to virtual nodes, such as radio access nodes or base stations. In some embodiments, some signaling may be provided with the use of a control system QQ512, which may alternatively be used for communication between the hardware nodes and the radio units.

[0122] Although the computing devices (e.g., network nodes, servers, hosts, etc.) described herein may include the illustrated combination of hardware components, other embodiments may include computing devices having various combinations of components. It should be understood that the computing devices may include any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determining, calculating, obtaining, or similar operations described herein may be performed by processing circuitry that may process information, for example, by transforming the obtained information into other information, comparing the obtained or transformed information with information stored at the network node, and / or performing one or more operations based on the obtained or transformed information, and making a decision as a result of the processing. Additionally, while components are shown as a single box arranged within a larger box or nested within multiple boxes, in reality the communication apparatus and network nodes may comprise multiple different physical components that make up a single illustrated component, and functionality may be partitioned among the separate components. For example, a communications interface may be configured to include any of the components described herein, and / or the functionality of those components may be partitioned between the processing circuitry and the communications interface, In other examples, non-computationally heavy functionality of any of those components may be implemented in software or firmware, and computationally heavy functionality may be implemented in hardware.

[0123] In some embodiments, some or all of the functionality described herein may be provided by a processing circuit executing instructions stored in a memory, which in some embodiments may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functionality may be provided by the processing circuit, such as in a hardwired manner, without executing instructions stored in a separate or discrete device-readable storage medium. In any of these specific embodiments, the processing circuit may be configured to perform the described functionality, whether or not it executes instructions stored in a non-transitory computer-readable storage medium. Benefits provided by such functionality are not limited to the processing circuit alone or to other components of the computing device, but are generally enjoyed by the computing device as a whole, and / or by end users and communication systems (e.g., wireless networks).

[0124] Additional definitions of embodiments are described below.

[0125] In the above description of various embodiments of the inventive concept, it should be understood that the terms used herein are only for describing specific embodiments and are not intended to limit the inventive concept. Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which the inventive concept belongs. Furthermore, it will be understood that terms as defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the present specification and related art, and should not be interpreted in an idealized or overly formal sense unless expressly defined herein.

[0126] When an element is referred to as being "connected," "coupled," "responsive" to another element, or variations thereof, it may be directly connected, coupled, or responsive to the other element, or there may be intervening elements. In contrast, when an element is referred to as being "directly connected," "directly coupled," "directly responsive" to another element, or variations thereof, there are no intervening elements. Like reference numbers refer to like elements throughout. Furthermore, as used herein, "coupled," "connected," "responsive," or variations thereof may include wirelessly coupled, connected, or responsive. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. Well-known features or configurations may not be described in detail for brevity and / or clarity. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0127] In this specification, terms such as first, second, third, etc. may be used to describe various elements / operations, but it should be understood that these elements / operations should not be limited by these terms. These terms are only used to distinguish one element / operation from another. Thus, a first element / operation in some embodiments can be called a second element / operation in other embodiments without departing from the teachings of the inventive concept. The same reference numbers or the same reference symbols refer to the same or similar elements throughout the specification.

[0128] As used herein, the terms "comprise," "comprising," "comprises," "include," "including," "includes," "have," "having," "has," or variations thereof, are open-ended and include one or more stated features, integers, elements, steps, components, or functions, but do not exclude the presence or addition of one or more other features, integers, elements, steps, components, functions, or groups thereof. Additionally, as used herein, the common abbreviation "for example," from the Latin phrase "exempli gratia," may be used to introduce or designate a general example or instance of an item previously mentioned and is not intended to be limiting of such items. The common abbreviation "ie," from the Latin phrase "id est," may be used to designate a particular item from a more general enumeration.

[0129] Exemplary embodiments are described herein with reference to block diagrams and / or flowchart illustrations of computer-implemented methods, apparatus (systems and / or devices), and / or computer program products. It will be understood that blocks of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions can be provided to general-purpose computer circuits, special-purpose computer circuits, and / or processor circuits of other programmable data processing circuits to generate machines such that the instructions, translation and control transistors, values ​​stored in memory locations, and other hardware components in such circuits execute via the processor of the computer and / or other programmable data processing apparatus to implement the functions / operations specified in the block diagrams and / or flowchart blocks or blocks, thereby creating means (functions) and / or structures for implementing the functions / operations specified in the block diagrams and / or flowchart blocks.

[0130] These computer program instructions may also be stored on a tangible computer-readable medium that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored on the computer-readable medium produce an article of manufacture that includes instructions that implement the functions / acts specified in one or more blocks of the block diagrams and / or flowcharts. Thus, embodiments of the inventive concepts may be embodied in hardware and / or software (including firmware, resident software, microcode, etc.*) running on a processor, such as a digital signal processor, which may collectively be referred to as a "circuit," "module," or variations thereof.

[0131] It should be noted that in some alternative embodiments, the functions / acts described in the blocks may occur in a different order than that described in the flowchart. For example, two blocks shown in succession may in fact be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order, depending on the functions / acts involved. Furthermore, the functionality of a given block of the flowcharts and / or block diagrams may be separated into multiple blocks, and / or the functionality of two or more blocks of the flowcharts and / or block diagrams may be at least partially integrated. Finally, other blocks may be added / inserted between the illustrated blocks and / or blocks / acts may be omitted without departing from the scope of the inventive concept. Additionally, although some of the figures include arrows on communication paths to indicate a primary direction of communication, it should be understood that communication may occur in a direction opposite to that depicted arrow.

[0132] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the inventive concept. All such variations and modifications are intended to be included within the scope of the inventive concept herein. Therefore, the subject matter disclosed above should be considered as illustrative and not limiting, and the example embodiments are intended to encompass all such modifications, enhancements, and other embodiments that are within the spirit and scope of the inventive concept. Thus, the scope of the inventive concept should be determined by the broadest permissible interpretation of the present disclosure, including the example embodiments and their equivalents, to the maximum extent permitted by law, and should not be limited or restricted by the foregoing detailed description.

Claims

1. 1. A method for managing transmission of power to at least one radio head from a local battery located proximate to the at least one radio head, the method being performed by a computing device (900) in a communication system, the method comprising: Making a decision on power transmission from the local battery to the at least one radio head in a future time window (1001), the decision being made by a machine learning model based on (i) determining a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time and location dependent cost data of charging and / or discharging of the local battery and grid utilization in the future time window, outputting (1003) the decision on transmitting power from the local battery to the at least one wireless head in the future time window; A method comprising:

2. 2. The method of claim 1, wherein the output power data statistics include at least: (i) an average power demand of the at least one radio head in the set of past time windows; and (ii) a peak power demand of the at least one radio head in the set of past time windows.

3. 3. The method according to claim 1, wherein the time and location dependent cost data comprises a first cost for charging the local battery in the future time window and a second cost for using power from a power grid for an average power demand of the at least one wireless head in the future time window.

4. 4. The method of claim 1 , wherein the determination for the future time window includes one of: (i) transmitting power from the local battery to the at least one radio head during the future time window, where the future time window includes a period of peak power consumption in the at least one radio head; (ii) charging the local battery during the future time window; and (iii) not transmitting power from the local battery to the at least one radio head during the future time window.

5. 5. The method of claim 1 , wherein the determination is made by the machine learning model based on: (i) inputting the power data and the battery and cost data into the machine learning model; and (ii) determining a minimized total cost for transmitting power to the at least one wireless head for the future time window if the future time window is constrained by a number of constraints.

6. 6. The method of claim 5, wherein the plurality of constraints include (i) a first constraint set as an input power consistency such that the input power to the radio head is equal to a discharge level of the local battery to the radio head plus power from the grid, and (ii) a second constraint set as the discharge from the local battery being less than or equal to a charge level of the local battery at a start of the future time window plus a charge profile limit for the future time window.

7. 7. The method of claim 6, wherein the charge profile limit for the future time window includes further constraints including: (i) a third constraint setting the current charge level of the local battery to be the same as a previous charge level of the local battery plus a charge amount minus a discharge amount of the local battery in the current time window; and (ii) a fourth constraint setting the charge level of the local battery to be less than or equal to a charge capacity of the local battery in any one window in the set of time windows.

8. The method according to any one of claims 1 to 7, the machine learning model receives reward feedback for state and action pairs; the reward feedback being a value that minimizes the total cost of input power to the at least one radio head in a next window in the set of time windows; the state includes a current charge level of the local battery, a current input power of the at least one wireless head, and a current cost to charge the local battery; The method, wherein the action in the state and action pair includes the decision.

9. 9. The method according to claim 1, wherein the at least one radio head comprises a plurality of radio heads, further comprising: Dividing the defined period into a set of time windows (1101); making a decision for each radio head for each time window in the set of time windows (1103); A method comprising:

10. 10. The method of claim 1, wherein outputting the decision (1003) comprises outputting the decision for controlling the transmission of power to the at least one radio head based on the decision.

11. The method according to any one of claims 1 to 10, the power data is offline data; The method of claim 1, wherein the making of the determination (1001) and the outputting (1003) are performed during training of the machine learning model with the offline data.

12. The method according to any one of claims 1 to 11, The power data is online data, The machine learning model is installed in a communication system; The method, wherein the making of the decision (1001) and the outputting (1003) are performed by the installed machine learning model.

13. The method of any one of claims 1 to 12, wherein the computing device is located at one of a location proximate to the local battery and a cloud-based location.

14. A computing device (900) in a communication system for managing the transmission of power to at least one radio head from a local battery located proximate to the at least one radio head, the computing device comprising: At least one processor (903); and at least one memory (905) coupled to said at least one processor (903) and configured to store program code that is executed by said at least one processor to perform operations, said operations including: making a decision on transmitting power from the local battery to the at least one radio head in a future time window, the decision being made by a machine learning model based on (i) determining a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time and location dependent cost data of charging and / or discharging of the local battery and grid utilization in the future time window; outputting the decision about transmitting power from the local battery to the at least one wireless head in the future time window; a computing device comprising:

15. A computing device (900) according to claim 14, wherein the at least one memory (905) is connected to the at least one processor (903) and stores program code executed by the at least one processor to perform the operations according to any one of claims 2 to 13.

16. A computing device (900), the computing device being in a communication system for managing transmission of power to at least one radio head from a local battery located proximate to the at least one radio head, the computing device being adapted to perform operations, the operations including: making a decision on transmitting power from the local battery to the at least one radio head in a future time window, the decision being made by a machine learning model based on (i) determining a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time and location dependent cost data of charging and / or discharging of the local battery and grid utilization in the future time window; outputting the decision about transmitting power from the local battery to the at least one wireless head in the future time window; a computing device comprising:

17. A computing device according to claim 16, adapted to perform the operations according to any one of claims 2 to 13.

18. 1. A computer program comprising: program code executed by a processing circuit (903) of a computing device (900) in a communication system for managing transmission of power to at least one radio head from a local battery located proximate to the at least one radio head, whereby execution of the program code causes the computing device to perform operations, the operations comprising: making a decision on transmitting power from the local battery to the at least one radio head in a future time window, the decision being made by a machine learning model based on (i) determining a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time and location dependent cost data of charging and / or discharging of the local battery and grid utilization in the future time window; outputting the decision about transmitting power from the local battery to the at least one wireless head in the future time window; A computer program comprising:

19. A computer program product according to claim 18, wherein execution of said program code causes said computing device (900) to perform the operations according to any one of claims 2 to 13.

20. 1. A computer program product comprising: a non-transitory storage medium comprising program code executed by a processing circuit (903) of a computing device (900) in a communication system, the program code comprising: managing transmission of power to at least one radio head from a local battery located proximate to the at least one radio head, whereby execution of the program code causes the computing device to perform operations, the operations including: making a decision on transmitting power from the local battery to the at least one radio head in a future time window, the decision being made by a machine learning model based on (i) determining a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a defined period for the at least one radio head, and (ii) time and location dependent cost data of charging and / or discharging of the local battery and grid utilization in the future time window; outputting the decision about transmitting power from the local battery to the at least one wireless head in the future time window; 2. A computer program product comprising:

21. 21. A computer program product according to claim 20, wherein execution of said program code causes said computing device (900) to perform the operations according to any one of claims 2 to 13.

Images