Management of power transmission to a wireless head

Abstract

A method is provided for managing the transmission of power from a local battery located proximate to at least one radio head, the method comprising: making a decision for transmitting power from the local battery to the at least one radio head in a future time window (1001). The decision is made by a machine learning model based on (i) a difference determination of output power data statistics including average power demand and peak power demand, and (ii) time- and location-dependent cost data of charging and / or discharging the local battery and grid utilization. The method further comprises outputting a decision for transmitting power from the local battery to the at least one radio head in a future time window (1003).

Description

Technical Field

[0001] The present disclosure generally relates to methods, related methods and apparatuses for managing the delivery (power transmission) of power from a local battery disposed in proximity to a wireless head to the wireless head.

Background Art

[0002] Wireless access networks (RANs) account for the majority of the energy consumption of mobile networks. Increasing the number of wireless units in a RAN is a common approach for expanding capacity and coverage to improve service quality. Adding wireless units can include adding more wireless units of the same type and / or wireless units for various radio access technologies (RATs) and bands. However, such an approach increases energy consumption.

[0003] In some approaches, site sizing design (e.g., power cables, circuit breakers, power supply units (PSUs)) is performed based on the maximum radio power consumption of each wireless unit, which can significantly affect and increase the input rating of site fuses. Increasing the rating of site fuses can have a significant impact on operating costs for the operator (telecommunications carrier).

[0004] Future and existing RAN features (e.g., microsleep transmission (Tx), low energy scheduler solution (LESS), multiple-input multiple-output (MIMO) sleep mode, cell sleep mode, etc.) increase the sleep time of wireless units and reduce power consumption. Such techniques will substantially increase the difference between the average power consumption and the peak power used by the installed wireless units.

Summary of the Invention

[0005] Currently, there are one or more problems of a certain kind. It is necessary to manage and / or transmit power in a way that allows the difference between the average power and peak power of the radio head to be determined while adding new radio access technologies (RATs) in a sustainable way.

[0006] Certain aspects in the present disclosure and their embodiments can provide solutions to these problems or other problems.

[0007] According to various embodiments, there is provided a method executed by a computing device in a communication system for managing the transmission of power from at least one local battery disposed proximate to at least one radio head to the at least one radio head. The method includes making a determination regarding the transmission of power from the at least one local battery to the at least one radio head in a future time window. This determination is made by a machine learning model based on (i) a determination of the difference in output power data statistics including the average power demand and peak power demand in a set of past time windows covering a period defined for the at least one radio head, and (ii) time - location - dependent cost data of charging and / or discharging of the local battery and utilization of the power distribution network in the future time window. The method further includes outputting a determination regarding the transmission of power from the at least one local battery to the at least one radio head in the future time window.

[0008] According to another embodiment, a computing device within a communication system is provided for managing the transmission of power from at least one local battery disposed in proximity to at least one wireless head to the at least one wireless head. The computing device includes at least one processor and at least one memory connected to the at least one processor and storing program code executed by the at least one processor to perform an operation of making a decision regarding the transmission of power from the at least one local battery to the at least one wireless head in a future time window. This decision is made by a machine learning model based on (i) a determination of the difference in output power data statistics including the average power demand and the peak power demand in a set of past time windows covering a period defined for the at least one wireless head, and (ii) time - location - dependent cost data for charging and / or discharging of the local battery and utilization of the power distribution network in the future time window. This operation further includes outputting a decision regarding the transmission of power from the at least one local battery to the at least one wireless head in the future time window.

[0009] According to other embodiments, a computing device within a communication system is provided for managing power transmission from at least one local battery disposed proximate to at least one wireless head to the at least one wireless head. The computing device is adapted to perform operations including making a decision regarding power transmission from the local battery to the at least one wireless head in a future time window. This decision is made by a machine learning model based on (i) a determination of the difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a period defined for the at least one wireless head, and (ii) time - location - dependent cost data for charging and / or discharging of the local battery and power grid utilization in the future time window. This operation further includes outputting a decision regarding power transmission from the local battery to the at least one wireless head in the future time window.

[0010] According to other embodiments, a computer program is provided that includes program code executed by a processing circuit of a computing device, the computing device being within a communication system for managing power transmission from at least one local battery disposed proximate to at least one wireless head to the at least one wireless head. Execution of the program code causes the computing device to perform operations including making a decision regarding power transmission from the local battery to the at least one wireless head in a future time window. This decision is made by a machine learning model based on (i) a determination of the difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a period defined for the at least one wireless head, and (ii) time - location - dependent cost data for charging and / or discharging of the local battery and power grid utilization in the future time window. This operation further includes outputting a decision regarding power transmission from the local battery to the at least one wireless head in the future time window.

[0011] According to another embodiment, there is provided a computer program product including a non-transitory storage medium including program code executed by a processing circuit of a computing device, the computing device being within a communication system for managing power transmission from at least one local battery disposed proximate to at least one wireless head to the at least one wireless head. Execution of the program code causes the computing device to perform operations including making a determination regarding power transmission from the at least one local battery to the at least one wireless head in a future time window. This determination is made by a machine learning model based on (i) a determination of a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a period defined for the at least one wireless head, and (ii) time-location-dependent cost data for charging and / or discharging the local battery and utilization of the power transmission and distribution network for the future time window. The operations further include outputting a determination regarding power transmission from the at least one local battery to the at least one wireless head in the future time window.

[0012] Certain embodiments can provide one or more of the following technical advantages. As described above, the method of some embodiments can manage power transmission from at least one local battery disposed proximate to at least one wireless head to the at least one wireless head. The method includes making a determination regarding power transmission from the at least one local battery to the at least one wireless head in a future time window. This determination is based on (i) a determination of a difference in output power data statistics including average power demand and peak power demand in a set of past time windows covering a period defined for the at least one wireless head, and (ii) time-location-dependent cost data for charging and / or discharging the local battery and utilization of the power transmission and distribution network for the future time window toPerformed by a machine learning model based on and outputting a decision regarding the supply of power from a local battery to at least one wireless head for a future time window. As a result, the technical advantages provided by the present method can include savings in the cost of energy based on charging and / or discharging of the local battery for peak power demand. Further, peak shaving in the wireless head can be enhanced based on the management of the local battery using ML (machine learning). Such management can not only provide and / or optimize a local battery charging / discharging policy, but can also extend the life of the local battery due to the determination and output of decisions.

Brief Description of the Drawings

[0013] Included to provide a further understanding of the present disclosure, the accompanying drawings incorporated in and constituting a part of this application, illustrate specific and non-limiting embodiments of the inventive concept. In the drawings,

[0014]

Figure 1

[0015]

Figure 2

[0016]

Figure 3

[0017]

Figure 4

[0018]

Figure 5

[0019]

Figure 6

[0020]

Figure 7

[0021]

Figure 8

[0022]

Figure 9

[0023]

Figure 10

Figure 11

[0024]

Figure 12

[0025]

Figure 13

[0026]

Figure 14

[0027]

Figure 15

DETAILED DESCRIPTION OF THE INVENTION

[0028] Hereinafter, the concept of the present disclosure will be more fully described with reference to the accompanying drawings showing examples of embodiments of the concept of the present disclosure. However, the concept of the present disclosure can be embodied in many different forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the concept of the present disclosure to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. It may be implicitly assumed that the components of one embodiment are present / used in another embodiment.

[0029] The following description presents various embodiments of the disclosed subject matter. These embodiments are presented as illustrative examples and should not be construed as limiting the scope of the disclosed subject matter. For example, the specific details of the described embodiments may be modified, omitted, or extended without departing from the scope of the described subject matter.

[0030] FIG. 1 is a schematic diagram showing the deployment of a conventional RAN site. As shown in FIG. 1, a power source 101 is connected to three 4G radio heads (also called radio units) 103a, 103b, 103c via a power supply line 105. The power source 101 also includes a backup battery (labeled "B" in FIG. 1).

[0031] FIG. 2 is a schematic diagram showing the addition of radio heads to an existing conventional RAN site (e.g., the RAN site of FIG. 1). As shown in FIG. 2, three additional radio heads 204a, 204b, 204c are added to the RAN site of FIG. 1 and connected to the power source 101 via three additional power supply lines 105, resulting in six power lines extending from the power source 101 to each of the radio heads 103a, 103b, 103c, 204a, 204b, and 204c, respectively.

[0032] "Peak shaving" refers to the term of cutting (reducing) the power in the utility when the demand from the customers is high. Some documents discuss the architecture for controlling peak shaving for the utility and / or power transmission lines.

[0033] Currently, there are one or more problems of a certain kind.

[0034] The architecture for controlling peak shaving for the utility and / or power transmission lines behaves differently from a power system having an integrated local backup battery at a site for wireless communication, as shown in FIGS. 1 and 2, for example. Wireless communication adds non-trivial problems and new variables with respect to the wireless network load that changes over time and the target key performance indicators (KPIs).

[0035] The existing approach for site facilities includes a backup battery (e.g., B in FIGS. 1 and 2) used as a backup in case of a power outage. The site backup battery B is typically sized according to the local rules (e.g., time) of the site and the average power consumption requirement of the site. As a result, the site sizing (e.g., power cables, circuit breakers, PSUs) is often done based on the maximum wireless power consumption of each wireless unit, which has a very large impact on and increases the input rating of the site fuse. However, increasing the site fuse rating can have a significant adverse impact on the operating cost for the operator.

[0036] In addition, an approach of adding additional radio units to an existing RAN site (e.g., as shown in FIG. 2) lacks the determination of the difference between the average power demand and the peak power demand for the radio units. Instead, in order to provide power demand for additional radio heads (other than backup battery power) from the power distribution network via a power source connected to the power distribution network, more equipment (e.g., more power lines, fuses, etc.) is added. However, the determination of such a difference between the average power and the peak power is important for optimizing / improving the design and operation of the local battery in order to make the process of adding new radio access technologies (RATs) more sustainable while reducing the total carbon footprint of the radio network operation.

[0037] There is a need to improve the site facilities and methods for transmitting power in a way that enables the determination of the difference between the average power and the peak power of the radio head while adding new RATs in a more sustainable way (e.g., such that the peak power needs are supplied from the local battery rather than from the power distribution network).

[0038] Certain aspects in the present disclosure and their embodiments can provide solutions to these or other problems. Specific embodiments of the present disclosure include a method of using machine learning (ML) to determine the difference between the average power usage and the peak power usage of a plurality of different radio heads. The local battery provides auxiliary power to handle the peak power consumption of the radio head. The local battery is disposed in proximity to the radio head. The power line from the site power source supplies the average power to the radio head.

[0039] Certain embodiments can provide one or more of the following additional technical advantages. As described above, according to some embodiments, a method is provided for managing the power transmission from at least one local battery disposed proximate to at least one wireless head to the at least one wireless head. The method includes making a determination regarding the power transmission from the at least one local battery to the at least one wireless head in a future time window. This determination is made by (i) determining the difference in output power data statistics including the average power demand and the peak power demand in a set of past time windows covering a period defined for the at least one wireless head, and (ii) a machine learning model based on the time - location - dependent cost data of charging and / or discharging of the local battery and the use of the power distribution network for the future time window, and outputting a determination regarding the supply of power from the at least one local battery to the at least one wireless head for the future time window.

[0040] Placing a local battery in proximity to the wireless head, outputting the determination, and the determination regarding power transmission from the local battery, as a result, may provide a reduction in the number or length of the required power lines (also referred to as "cables" in this specification), and a reduction in the number of fuses. The expansion of a site with additional wireless heads may be easily achieved with fewer or shorter power lines. For example, when a wireless head is added to an existing conventional RAN site (e.g., the RAN site in FIG. 1), based on the adoption of a local battery in proximity to the wireless head (as shown, for example, in FIG. 3 further described in this specification), the number of power lines from the power source (e.g., power source 101) to the local battery (e.g., local battery 301) may be maintained the same (e.g., three power lines 105). This is in contrast to the existing approach for expanding a site to have additional wireless heads, where new power lines need to be installed over the entire distance from the power source to the new wireless head. In the example of FIG. 2, three additional power lines are added such that there are six power lines covering the entire distance between the power source (e.g., power source 101) and the wireless heads (e.g., wireless heads 103a - 103b and 104a - 104c as shown in FIG. 2). Further, based on the adoption of local battery management for wireless peak power, the operator of the wireless system can provide robustness against power grid failures since the local battery is an active power source (e.g., not just a backup power source). A further technical advantage of including a local battery in proximity to the wireless head(s) for managing power transmission to the wireless head(s) is that it can contribute to zero - emission RAN, including the use of an energy source for charging the local battery (e.g., for powering the wireless head and / or for charging a backup battery located at the power grid power source), and an increase in network robustness against power grid failures. Green The use of an energy source, and may include an increase in network robustness against power grid failures.

[0041] FIG. 3 is a schematic diagram showing additional wireless heads 204a, 204b, 204c of FIG. 2, where the RAN site is configured according to some embodiments of the present disclosure. The architecture of FIG. 3 can achieve peak shaving of the local wireless head based on installing a local battery 301. According to a particular embodiment of the present disclosure, a method for managing the peak power consumption of at least one of the wireless heads 103a-103c, 204a-204c from the local battery 301 differentiates (determines the difference) between the average power and the wireless peak power and manages the power transmission from two different power sources 101, 301 during normal network operation.

[0042] In FIG. 2, the backup battery B is a backup power source, and additional wireless heads 204a-204c are added to a site including different RATs, such as the LTE extended wireless head 204c. In contrast, FIG. 3 shows a site design that adds a local battery 301 to achieve peak shaving while further adding additional wireless heads 204a-204c and reducing the number of required cables and fuses. According to an exemplary embodiment of FIG. 3, including the local battery 301 as an active power source (i.e., not a battery backup) reduces the number of cables 105 from the power source 101 compared to FIG. 2. In FIG. 2, the addition of the wireless heads 204a-204c also adds three power cables 105 that cover the total distance between the power source 101 and the wireless heads 204a-204c such that there are six power lines 105 that each cover the total distance between the power source 101 and the wireless heads 103a-103c and 204a-204c. In contrast, in FIG. 3, adding the wireless heads 204a-204c does not add a power line 105 between the power source 101 and the local battery 101. Thus, compared to FIG. 2, in FIG. 3, the number and / or length of the power lines 105 are reduced. Further, in FIG. 3, since the local battery 301 is an active power source for peak power demand, the power input from the grid through the power source 101 can be reduced.

[0043] Figure 4 is a plot of the input power to four radio heads (radio heads 1, 2, 3, 4 as shown in the legend) in a two-sector radio site over a defined period of about 30 seconds. The solid and dashed lines represent radios 1 and 2 of the first sector. The fine dotted and dash-dotted lines represent radios 3 and 4 of the second sector. As can be seen from the plot illustrated in Figure 4, the radio heads have an average power and a power peak in watts over the period.

[0044] Figure 5 shows the plot of Figure 4 annotated with horizontal lines 501, 503, 505, 507, which shows the determination of the difference between the average power and the peak power of the radio head over the period according to some embodiments of the present disclosure. According to the exemplary embodiment of Figure 5, when a separation is provided between the average power indicated by lines 503, 507 and the peak power indicated by lines 501, 505 respectively, it can be seen from the illustrated plot that with an appropriate dimensioning of the power line 105, the average power 503, 507 of the radio unit can be carried, and the peak power 501, 505 can be controlled from the local battery 301.

[0045] Figures 4 and 5 are described with reference to four wireless heads and a defined period of about 30 seconds, but embodiments of the present disclosure are not so limited. Instead, any number of one or more wireless heads may be included, and the defined period may be any period defined in seconds, minutes, hours, days, weeks, etc. Further, two peak power settings and two average power settings are shown for the plot shown in FIG. 5, but embodiments of the present disclosure are not limited thereto. Instead, for each wireless head and / or for one or more wireless heads, peak power settings may be specified / determined (as further described herein). Further, for each wireless device and / or for one or more wireless heads, average power settings may be specified / determined (as further described herein). Further, some embodiments are described herein with reference to average power supplied by a power grid (power transmission and distribution network) and peak power supplied by a local battery, but embodiments of the present disclosure are not so limited. Instead, more or less power than the peak power may be supplied by the local battery, and the remaining power demand of the wireless head may be supplied from the power transmission and distribution network.

[0046] According to certain embodiments of the present disclosure, the ML-based method can manage the charging and discharging policies of the local battery 301 for peak shaving based on the temporal variations of the wireless power peak demand and the power grid energy price and carbon footprint profile over a period of time.

[0047] According to some embodiments, the following observations are considered: ● There may be significant variations in the input and output power of the wireless heads. Some wireless heads become very active with high output power, while others rarely reach high output power. ● There is a high correlation between the peak of the power of the wireless head related to the input power from the site power supply (and the power grid) and the output power of the wireless head (which peaks at the peak of the traffic load). ● For an active time window in which the wireless head is substantially inactive, differentiation (determination of the difference) is required between the average radio power and the peak radio power (see, for example, FIG. 4). Such determination of the difference can support the efficient operation of the local battery 301. ● There is a high correlation between the output power of a single wireless head and time (e.g., during an overnight period).

[0048] According to some embodiments, these observations are used to construct an ML method for managing a local battery (e.g., battery 301).

[0049] According to some embodiments, a defined period (e.g., one day) is divided into a set of time windows (e.g., every hour), and decision variables are generated for future time windows. The method uses data obtained from the power consumption of the wireless head (including wireless traffic variations), which is collected or accessed over time, to train an ML model (e.g., an RL agent).

[0050] The training of the RL agent includes at least the following inputs: (1) the local battery capacity, (2) the average power consumption of all wireless heads in K past time windows out of a set of time windows for a given integer K>0, (3) the peak power consumption of all wireless heads in K past time windows out of a set of time windows, and (4) the charging cost of the local battery and the cost for using the power transmission and distribution network in the current or future time window. The RL agent optimizes the charge state decision, including when to charge the local battery and how much to discharge the local battery for the wireless head, at the start of each current / future time window (e.g., each time window), and does not transmit power from the local battery to the wireless head if the charge state decision determines that the discharge amount from the local battery for the wireless head is zero.

[0051] Figure 6 is a schematic diagram showing the state of charge (SOC) of a local battery according to some embodiments of the present disclosure. As shown in Figure 6, the SOC level of the local battery ranges from 0% to 100%. The ML model (e.g., RL model) defines / sets the SOC of the local battery for the predicted wireless peak power in future time window(s) (e.g., in each future time window). The SOC level can vary between 0% and 100% and is defined at the SOC level according to the predicted peak power. According to an exemplary embodiment of Figure 6, the SOC level may be set for a future time window at the SOC levels indicated by three arrows. The example of Figure 6 shows three SOC levels, but the present disclosure is not so limited, and the SOC level may be defined at any non-zero SOC level according to the predicted peak power.

[0052] According to some embodiments, the SOC determination (also referred to herein as "determination") satisfies the following four constraints for a time window: C1. Constraint 1: The sum of the discharge level of the local battery to the wireless head in any time window and the input power (from the power cable) should be the same as the input power required by the wireless head at that time. These time- and position-stamped input powers are inputs / constants to the ML model given in the dataset. C2. Constraint 2: The discharge amount of the local battery in all time windows should not exceed the sum of the local battery level at the start of that time window and the charging profile limit (restriction) of that time window. C3. Constraint 3: The current local battery level is the same as the sum of the previous local battery level and the charge amount in this time window. C4. Constraint 4: The local battery level in any time window should not exceed the local battery capacity.

[0053] The objective of this approach is to minimize the total cost, including the cost of charging the local battery and the cost of using the power transmission and distribution network, for future time window(s), by enabling the average power to the wireless head to be transmitted from the site power system based on the adoption of the above constraints.

[0054] Next, the ML model is further described. According to some embodiments, the ML model is an episodic RL agent, where each episode is a defined period (e.g., one day). According to some embodiments, the RL agent is trained in the cloud and then installed at the battery management service location (e.g., placed at the local battery of the site).

[0055] According to some embodiments, the RL agent uses a dataset of the power consumption of the wireless head over a defined period and determines the difference between the wireless average power and the wireless peak power for one episode or more episodes (e.g., several days).

[0056] According to some embodiments, the action includes determining the charging and discharging of at least one local battery for the wireless head.

[0057] According to some embodiments, the state includes the current local battery level, the current output power of all wireless heads, and the current local battery charging cost.

[0058] According to some embodiments, the reward of the RL agent in all state-action pairs is to minimize the total cost of the input power in the next time window among a set of time windows. The total cost is a weighted sum of the power from the power transmission and distribution network and the power from the local battery, weighted based on their respective costs. According to some embodiments, the total cost includes, among other things, the monetary cost of energy or the carbon footprint index.

[0059] Next, an exemplary embodiment will be described with reference to FIG. 7. The exemplary embodiment of FIG. 7 is described in the context of FIG. 3 showing one local battery and six wireless heads, but various embodiments of the present disclosure are not so limited and can include different numbers of local batteries and different numbers or types of wireless heads. Further, although a particular embodiment is described with reference to one local battery for ease of explanation, various embodiments of the present disclosure are not so limited and can include any number of local batteries.

[0060] Referring to FIG. 7, according to an exemplary embodiment, a total of R wireless heads (e.g., wireless heads 103a - 103c and 204a - 204c) are indexed by i ∈ {1, 2, ···, R} (in this exemplary embodiment, R = 6) and are connected to a local battery 301 with a charge level C as shown by connection 701. Further, the wireless heads are connected to the power distribution network via a power source 101 as shown by connection 703. In operation 705, a computing device (e.g., computing device 900 further described herein) divides each episode (i.e., a defined period for decision-making such as daily) into a set of W time windows of a pre-defined duration T, and B w represents the level of the local battery 301 at the start of a future time window w from the set of time windows. As shown in operation 709, for all windows w, the i-th wireless head among the plurality of wireless heads (e.g., the wireless head as shown for wireless head 103a in FIG. 7) has a maximum power p iw max , an average power p iw avg , and an instantaneous power p iw (t) at any time t within this time window. For each time window w, the local battery 301 has a d iw bDischarge the i-th radio head at a constant speed of ≧0. As shown in box 711, the charging of the local battery 301 in all time windows w is based on a predefined charging profile h w b (t) and a cost profile c w b By following this, during this time window, the total charge amount is

Number

Number

Number

[0061] Continuing to refer to Figure 7, the following constraints are included in each time window within the set of time windows: First constraint: Input power to the radio head: d iw b +d iw g =p iw (t). Second constraint: Local battery level (which includes the above-mentioned constraints 2 - 4): B w +H w b -Td iw b ∈[0,C].[[]END]]

[0062] The operating cost (e.g., electricity cost) in any time window is \(C_w^b + C_w^g\), which results in a cost of \(f(d 11 b ,d 12 b ,...,d RW b )=\sum_{} w∈W C w b +\sum_{} i∈R C iw g for the entire episode. Therefore, the optimization problem, with \(f\) as the reward function, is formulated as follows: Minimize the following under constraints C1 and C2 f(d 11 b ,d 12 b ,...,d RW b )

[0063] Therefore, the reward function \(f\) is constrained by C1 (i.e., the sum of the power input from the power cable to the discharge level of the local battery to the radio head in any time window should be the same as the input power required by the radio head at that time) and C2 (i.e., the discharge amount of the local battery in all time windows should not exceed the local battery level at the start of this time window plus the charging profile limit of that time window) for the entire episode, and \(f(d 11 b ,d 12 b ,...,d RW b )=\sum_{} w∈W C w b +\sum_{} i∈R C iw g is to minimize the cost.

[0064] As shown in operation 715, the RL agent of computing device 900 solves an optimization problem. According to some embodiments, the RL agent not only accesses historical data of power consumption {p iw (t)} for many past episodes, but also charges local battery 301 via power supply 101 and accesses the cost profile of using the power distribution network c w g (t). The RL agent uses that dataset to simulate the environment. to The RL agent uses that dataset to simulate the environment.

[0065] FIG. 8 is a sequence diagram showing operations for managing peak power demand from a local battery to one or more wireless heads according to some embodiments of the present disclosure. According to the exemplary embodiment of FIG. 8, the ML model is RL agent 803. The environment 801 is either (i) a simulator based on data collected during training of key performance indicators (KPIs) and power data, or (ii) an actual network in which KPIs and power data are measured after RL agent 803 and the local battery are installed. The KPIs include, but are not limited to, local battery capacity, a first cost for charging the local battery in a future time window, and a second cost for using power from the power distribution network for the average power demand of at least one wireless head in a future time window. The power data includes, but is not limited to, (i) the average power demand of at least one wireless head in a set of past time windows, (ii) the peak power demand of at least one wireless head in a set of past time windows, and (iii) the traffic load of at least one wireless head in a set of past time windows.

[0066] Continuing to refer to FIG. 8, in operation 807, the RL agent 803 obtains input data from the environment 801, including KPIs and input power consumption. In operation 809, the RL agent 803 maps the input to an environmental state and sends that state to the reward calculation device 805. In operation 811, the RL agent 803 calculates the next action to be executed and sends that action to the reward calculation device 805. The reward calculation device 805 calculates the reward in operation 813. In operation 815, the RL agent 803 receives the reward and then, in operation 817, improves the RL model and actions based on the reward.

[0067] Continuing to refer to FIG. 8, according to some embodiments, during training, the loop of operations 807-819 is continued to determine a local battery management policy (e.g., an optimal battery management policy).

[0068] According to some embodiments, after determining the local battery management policy, the trained RL agent 803 is deployed in the actual network. According to some embodiments, the RL agent 803 is deployed across all sites (e.g., battery management service locations) or within the cloud. The deployed RL agent 803 can access data from the network and monitors KPIs for potential re-training / tuning of the RL agent. The loop of operations 807-819 is executed in the actual network to determine the local battery management policy.

[0069] According to some embodiments, at least one local battery has a set of functions for using the at least one local battery as an auxiliary power source, which includes proactive charge and discharge decisions and executions to minimize some cost functions (e.g., for the operator).

[0070] According to some embodiments, at least one local battery is installed near the wireless head, and at least one local battery is responsible for only the peak power consumption. The power cable coming from the site power system for the wireless head (e.g., a power source 60 m away from the wireless head of the site) is set to transmit only the average power to the wireless head. At least one local battery has local operations or SC decisions for handling the peak power.

[0071] According to some embodiments, the method determines the difference between the average power and the wireless peak power consumption of several wireless heads in the state space, and provides a decision that enables the local battery to transmit only the peak power to the wireless head.

[0072] According to some embodiments, the method calculates the daily traffic fluctuations and power demand and enables the operation of at least one local battery (e.g., charging or discharging for the peak power demand).

[0073] FIG. 9 is a block diagram showing elements (components) of a computing device 900 (also referred to as a server, cloud-based server, edge server, radio access network node, base station, radio base station, eNodeB / eNB, gNodeB / gNB, or any other functional physical or virtual network node in which a machine learning model may be implemented, as disclosed herein) of a communication network (e.g., communication network QQ100 configured to provide communication as described below with respect to FIG. 12). The computing device 900 may be provided as described below with respect to, for example, network nodes QQ110A, QQ110B of FIG. 12, network node QQ300 of FIG. 13, hardware QQ504, and / or virtual machines QQ508A, QQ508B of FIG. 15, all of which should be considered interchangeable in the examples and embodiments described herein unless otherwise noted, which is within the intended scope of the present disclosure. As shown, the computing device can include a computer 901 communicatively connected to an ML model 911. The ML model 911 may be an episodic RL agent. The ML model 911 may also include a reward calculation device (e.g., reward calculation device 705). The computer 901 may include a transceiver circuit (also called a transceiver, corresponding to a part of the RF transceiver circuit QQ312 and the radio front-end circuit QQ318 of FIG. 13) including a transmitter and a receiver configured to provide uplink and downlink wireless communication with a mobile terminal. The computer may include a network interface circuit 907 (e.g., communication interface of FIG. 13) configured to provide communication with other devices or nodes of the communication network (e.g., other network nodes, communication devices, and / or data repositories). fIt may include a network interface (also referred to as) corresponding to a part of Ace QQ306. The computing device may also include a processing circuit 903 (also referred to as a processor corresponding to the processing circuit QQ302 in FIG. 13, for example) coupled to the transceiver circuit, and a memory circuit 905 (also referred to as a memory corresponding to the memory QQ304 in FIG. 13, for example) coupled to the processing circuit. The memory circuit 905 may include computer-readable program code that, when executed by the processing circuit 903, causes the processing circuit to perform operations according to the embodiments disclosed herein. According to other embodiments, the processing circuit 903 may be defined to include a memory so that a separate memory circuit is not required.

[0074] As described herein, the operations of the computing device may be performed by the processing circuit 903, the network interface 907, and / or the transceiver. For example, the processing circuit 903 may control the transceiver to transmit downlink communication to one or more mobile terminals UE via the wireless interface through the transceiver, and / or f transmit downlink communication to one or more mobile terminals UE via the wireless interface through the transceiver, and / or fIt can receive uplink communication from one or more communication devices via the base. Similarly, the processing circuit 903 controls the network interface 907 to transmit communication to one or more other devices or network nodes via the network interface 907 and / or receive communication via the network interface from one or more network nodes, radio heads, local batteries, communication devices, etc. Further, the modules may be stored in the memory 905, and these modules may provide instructions such that when the instructions of the modules are executed by the processing circuit 903, the processing circuit 903 executes respective operations (e.g., the operations described below with respect to exemplary embodiments regarding computing devices). According to some embodiments, the computing device 900 and / or its elements / functions can be embodied as virtual node(s) / node(s) and / or virtual machine(s) / machine(s), for example, as described with respect to FIG. 15.

[0075] According to some other embodiments, the computing device may be implemented as a core network node without a transceiver. According to such embodiments, transmissions to communication devices, network nodes, radio heads, local batteries, etc. may be initiated by the computing device 900 such that transmissions to communication devices, network nodes, etc. are provided through the computing device 900 including a transceiver (e.g., through a base station or a RAN node). According to embodiments where the computing device is a RAN node including a transceiver, initiating the transmission can include transmitting through the transceiver.

[0076] The operation of a computing device (e.g., a computing device including ML model 911) (implemented using the structure of FIG. 9) is discussed herein with reference to the flowcharts of FIGS. 10 and 11 according to some embodiments of the present disclosure. In the following description, the computing device may be any of computing device 900, network nodes QQ110A, QQ110B, QQ300, QQ606, hardware QQ504, or virtual machines QQ508A, QQ508B, but computing device 900 is used to illustrate the functions of the operation of the computing device. For example, the modules can be stored in memory 905 of FIG. 9, and these modules can provide instructions such that when the instructions of the modules are executed by the processing circuit 903 of each computing device, the processing circuit 903 executes each operation of the flowchart.

[0077] Referring to FIG. 10, there is provided a method executed by a computing device (900) in a communication system for managing power transmission from at least one local battery disposed proximate to at least one wireless head to the at least one wireless head. The method includes making a decision (1001) regarding power transmission from the at least one local battery to the at least one wireless head for a future time window. The decision is made by a machine learning model based on (i) a determination of the difference in output power data statistics including the average power demand and the peak power demand in a set of past time windows covering a period defined for the at least one wireless head, and (ii) time - location - dependent cost data for charging and / or discharging of the local battery and power grid utilization in the future time window. The method further includes outputting (1003) a decision regarding power transmission from the at least one local battery to the at least one wireless head in the future time window.

[0078] According to some embodiments, the output power data statistics include at least (i) the average power demand of at least one radio head in a set of past time windows, and (ii) the peak power demand of at least one radio head in a set of past time windows.

[0079] According to some embodiments, the time and location-dependent cost data includes a first cost for charging a local battery in a future time window and a second cost for using power from a power distribution network for the average power demand at at least one radio head in a future time window.

[0080] According to some embodiments, the determination includes one of the following for a future time window (i), and power is transmitted from a local battery to at least one radio head during the future time window. The future time window includes a period of peak power consumption at at least one radio head, (ii) charging the local battery during the future time window, and (iii) not transmitting power from the local battery to at least one radio head during the future time window.

[0081] According to some embodiments, the determination is made by a machine learning model based on (i) inputting power data and battery and cost data into the machine learning model, and (ii) determining a minimized total cost for power transmission to at least one radio head for a future time window if the future time window is constrained by a plurality of constraints.

[0082] According to some embodiments, the plurality of constraints include: (i) a first constraint set as input power consistency, where the input power to the wireless head is equal to the sum of the power from the power distribution network and the discharge level of the local battery to the wireless head; and (ii) a second constraint where the discharge from the local battery is set such that the amount of discharge is less than or equal to the sum of the charge level of the local battery at the start of a future time window and the charging profile limit for the future time window.

[0083] According to some embodiments, the charging profile limit for a future time window includes further constraints. The further constraints include: (i) a third constraint where the current charge level of the local battery is set such that the current charge level is the same as the previous (immediately preceding) charge level of the local battery plus the difference between the charge amount and the discharge amount of the local battery in the current time window; and (ii) a fourth constraint where the charge level is set such that the charge level of the local battery in the set of time windows is less than or equal to the charge amount of the local battery.

[0084] According to some embodiments, the machine learning model receives reward feedback for a state and action and pair an action. The state includes the current charge level of the local battery, the current input power of at least one wireless head, and the current cost for charging the local battery. The action in the state and action pair includes a decision.

[0085] Referring now to FIG. 11, according to some embodiments, at least one wireless head includes a plurality of wireless heads. The method further includes dividing a defined period into a set of time windows (1101). The method further includes making a decision for each wireless head for each time window in the set of time windows (1103).

[0086] Referring back to FIG. 10, according to some embodiments, outputting a decision (903) includes outputting a decision for controlling power transmission to at least one radio head based on the decision.

[0087] According to some embodiments, the power data is offline data, and the determination (1001) and output (1003) are performed during the training of a machine learning model that uses the offline data.

[0088] According to some embodiments, the power data is online data, the machine learning model is installed in the communication system, and the determination (1001) and output (1003) are performed by the installed machine learning model.

[0089] According to some embodiments, the computing device is disposed at one of a location proximate to a local battery and a cloud-based location.

[0090] Various operations from the flowchart of FIG. 11 may be optional with respect to some embodiments of the method performed by the computing device.

[0091] FIG. 12 shows an example of a communication network QQ100 in which a communication system according to some embodiments may be implemented.

[0092] In an example, the communication network QQ100 includes a telecommunications network QQ102 including an access network QQ104 such as a radio access network (RAN), and a core network QQ106 including one or more core network nodes QQ108. The access network QQ104 includes one or more access network nodes such as network nodes QQ110a and QQ110b (one or more of which may generally be referred to as network node QQ110), or any other similar 3rd Generation Partnership Project (3GPP™) access node or non-3GPP™ access point. The network node QQ110 facilitates a direct or indirect connection of a user equipment (UE) by connecting the UEQQ112a, QQ112b, QQ112c, and QQ112d (one or more of which may generally be referred to as UEQQ112) to the core network QQ106 via one or more wireless connections. The network nodes QQ110A, QQ110B may be cloud-implemented network nodes (e.g., servers), or may be located in cloud or edge-implemented network nodes (e.g., servers). The network nodes QQ110A, QQ110B facilitate a direct or indirect connection of a communication device by connecting the communication devices QQ112a, QQ112b, QQ112c, and QQ112d (one or more of which may generally be referred to as communication device / UE QQ112) to the communication network QQ100 via one or more wireless connections.

[0093] wirelessExemplary wireless communication via a connection includes transmitting and / or receiving wireless signals using electromagnetic waves, radio waves, infrared rays, and / or other types of signals suitable for transmitting information without using wires, cables, or other conductive materials. Further, according to various embodiments, communication network QQ100 may include any number of wired or wireless networks, network nodes, communication devices, and / or any other components or systems that may facilitate or be involved in the communication of data and / or signals, regardless of whether they are via wired or wireless connections. Communication network QQ100 includes, but is not limited to, other similar types of systems including any kind of communication, telecommunication, data, cellular, wireless network, and / or 5G and / or 6G network, and / or may interface f with them.

[0094] In the illustrated example, core network QQ106 connects network node QQ110 to one or more hosts such as host QQ116. These connections may be direct or indirect via one or more intermediate networks or devices. In other examples, the network node may be directly coupled to the host. Core network QQ106 is hardware e and software eIt includes one or more core network nodes (e.g., core network node QQ108) composed of components. The characteristics of these components can be substantially the same as those described with respect to the UE, network node, and / or host, and thus, those descriptions are generally applicable to the corresponding components of the core network node QQ108. Exemplary core network nodes include one or more functions among a mobile switching center (MSC), a mobility management entity (MME), a home subscriber server (HSS), an access and mobility management function (AMF), a session management function (SMF), an authentication server function (AUSF), a subscription identifier non-concealment function (SIDF), an integrated data management (UDM), a security edge protection proxy (SEPP), a network exposure function (NEF), and / or a user plane function (UPF).

[0095] Host QQ116 may be under the ownership or control of a service provider other than the operator or provider of access network QQ104 and / or telecommunications network QQ102, and may be operated by or on behalf of the service provider. Host QQ116 can host various applications to provide one or more services. Examples of such applications include live and pre-recorded audio / video content, data collection services such as searching and compiling data regarding various ambient conditions detected by multiple UEs, analytical functions, social media, functions for controlling or otherwise interacting with remote devices, functions for alarm and monitoring centers, or any other such functions executed by the server.

[0096] Overall, the communication network QQ100 of FIG. 12 enables connectivity between UEs, network nodes, and hosts. In that sense, the communication network can be a Global System for Mobile Communications (GSM) for mobile communications, a Universal Mobile Telecommunications System (UMTS), Long Term Evolution (LTE), and / or other suitable 2G, 3G, 4G, 5G standards, or applicable future generation standards (e.g., 6G), a Wireless Local Area Network (WLAN) standard such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard (WiFi), and / or Worldwide Interoperability for Microwave Access (WiMax), Bluetooth®, Z-Wave, Near Field Communication (NFC), ZigBee, LiFi, and / or any other suitable wireless communication standards such as Low Power Wide Area Network (LPWAN) standards like LoRa and Sigfox specific standards, etc. , not limited to these are , and can be configured to operate according to defined rules or procedures.

[0097] In some examples, the telecommunications network QQ102 is a cellular network implementing 3GPP® standardized features. Thus, the telecommunications network QQ102 can support network slicing to provide different logical networks to different devices connected to the telecommunications network QQ102. For example, the telecommunications network QQ102 can provide ultra-reliable low-latency communication (URLLC) services to some UEs while providing enhanced mobile broadband (eMBB) services to other UEs and / or massive machine type communication (mMTC) / massive IoT services to further UEs.

[0098] In the example, the hub QQ114 communicates with the access network QQ104 to facilitate indirect communication between one or more UEs (e.g., UEs QQ112c and / or QQ112d) and a network node (e.g., network node QQ110b). In some examples, the hub QQ114 may be any of a controller, a router, a content source and analyzer, or other communication devices described herein with respect to the UE. For example, the hub QQ114 may be a broadband router that enables access to the core network QQ106 for the UE. As another example, the hub QQ114 may be a controller that sends commands or instructions to one or more actuators within the UE. The commands or instructions may be received from the UE, the network node QQ110, or executable code, scripts, processes, or other instructions within the hub QQ114. As another example, the hub QQ114 may be a data collector that serves as temporary storage for UE data and, according to some embodiments, may perform analysis or other processing of the data. As another example, the hub QQ114 may be a content source. For example, in the case of a UE that is a VR headset, a display, a loudspeaker, or other media delivery device, the hub QQ114 can retrieve VR assets, video, audio, or other media, or data related to sensory information, via the network node, and then the hub QQ114 provides it directly to the UE either after performing local processing and / or after adding additional local content. In yet another example, the hub QQ114 acts as a proxy server or orchestrator for the UE, especially when one or more of the UEs are low-energy IoT devices.

[0099] Hub QQ114 can have a constant / persistent or intermittent connection to network node QQ110b. Hub QQ114 can also enable another communication method and / or schedule between hub QQ114 and a UE (e.g., UE QQ112c and / or QQ112d), and between hub QQ114 and core network QQ106. In other embodiments, hub QQ114 is connected to core network QQ106 and / or one or more UEs via a wired connection. Additionally, hub QQ114 may be configured to connect to an M2M service provider via access network QQ104 and / or to another UE via a direct connection. In some situations, the UE may establish a connection with network node QQ110 while remaining connected via hub QQ114 via a wired or wireless wireless connection. wireless According to some embodiments, hub QQ114 may be a dedicated hub, i.e., a hub whose main function is to route communications from / to network node QQ110b to / from a UE. According to other embodiments, hub QQ114 may be a non-dedicated hub, i.e., a device that can operate to route communications between a UE and network node QQ110b but can also further operate as a communication origin and / or destination for a specific data channel.

[0100] FIG. 13 shows a network node QQ300 according to some embodiments (e.g., the network node QQ300 can include a network node that can implement a computing device 900). As used herein, a network node refers to a device that is configured, arranged, and / or operable to communicate directly or indirectly with a UE and / or other network nodes or devices in a communication network. Examples of network nodes include, but are not limited to, servers, access points (APs) (e.g., wireless access points), base stations (BSs) (e.g., wireless base stations, Node B, evolved Node B (eNB), and NR Node B (gNB)).

[0101] Base stations can be classified based on the size of the coverage they provide (or, put differently, their transmission power levels), and thus can be referred to as femto base stations, pico base stations, micro base stations, or macro base stations, depending on the size of the coverage provided. A base station may be a relay node or a relay donor node that controls a relay. A network node may also include one or more (or all) parts of a distributed radio base station, such as a centralized digital unit and / or a remote radio unit (RRU), sometimes referred to as a remote radio head (RRH). Such remote radio units may or may not be integrated with an antenna as an antenna-integrated radio. Some parts of a distributed radio base station may sometimes be referred to as nodes in a distributed antenna system (DAS).

[0102] Other examples of network nodes include multi-transmission point (multi-TRP) 5G access nodes, multi-standard radio (MSR) devices such as MSR BS, network controllers such as radio network controllers (RNCs) or base station controllers (BSCs), base transceiver stations (BTSs), transmission points, transmission nodes, multi-cell / multicast coordination entities (MCEs), operation and maintenance (O&M) nodes, operation support system (OSS) nodes, self-organizing network (SON) nodes, positioning nodes (e.g., evolved serving mobile location center (E-SMLC)), minimization of drive tests (MDT), and / or cloud implementation servers or edge implementation servers.

[0103] Network node QQ300 includes a processing circuit QQ302, a memory QQ304, a communication interface QQ306, and a power supply QQ308. Network node QQ300 may be composed of a plurality of physically separate components (e.g., clustering components, database (e.g., knowledge graph) components, NodeB components and RNC components, or BTS components and BSC components, etc.) that each of its respective components may have. In a scenario where network node QQ300 includes a plurality of distinct components (e.g., clustering and database components), one or more of those distinct components may be shared among several network nodes. For example, a single network node may control a plurality of network nodes including a clustering component and / or a database (e.g., repository). In such scenarios, each unique pair of network nodes and components may, in some instances, be considered a single distinct network node. In some embodiments, network node QQ300 may be configured to support a plurality of radio access technologies (RATs). In such embodiments, some components may be made redundant (e.g., separate memories QQ304 for different RATs), and some components may be reused (e.g., the same database, or the same antenna QQ310 may be shared by a plurality of different RATs). Also, network node QQ300 may include a plurality of sets of various exemplary components for various wireless technologies integrated into network node QQ300, such as, for example, GSM, WCDMA, LTE, NR, WiFi, Zigbee, Z-wave, LoRaWAN, RFID (Radio Frequency Identification), or Bluetooth wireless technologies. Those wireless technologies may be integrated into the same or different chips or sets of chips and other components within network node QQ300.

[0104] The processing circuit QQ302 may include one or more combinations of a microprocessor, a controller, a microcontroller, a central processing unit, a digital signal processor, an application specific integrated circuit, a field programmable gate array, or other suitable computing device, resource, or hardware, software, and / or encoded logic that is operable to provide the functionality of the network node QQ300, either alone or in cooperation with other components of the network node QQ300 such as the memory QQ304.

[0105] In some embodiments, the processing circuit QQ302 includes a system on chip (SOC). In some embodiments, the processing circuit QQ302 includes one or more of a radio frequency (RF) transceiver circuit QQ312 and a baseband processing circuit QQ314. In some embodiments, the radio frequency (RF) transceiver circuit QQ312 and the baseband processing circuit QQ314 may be on separate chips (or a set of chips), substrates, or units, such as a radio unit and a digital unit. In alternative embodiments, some or all of the RF transceiver circuit QQ312 and the baseband processing circuit QQ314 may be on the same chip or set of chips, substrate, or unit.

[0106] Memory QQ304 includes, without limitation, any form of volatile or non-volatile computer-readable memory, such as persistent storage, solid-state memory, remotely mounted memory, magnetic media, optical media, random access memory (RAM), read-only memory (ROM), mass storage media (e.g., hard disk), removable storage media (e.g., flash drive, compact disc (CD) or digital video disc (DVD)), and / or any other volatile or non-volatile non-transitory device-readable and / or computer-executable memory device. Memory QQ304 may store a computer program, software, logic, rules, code, table, or other suitable instructions, data, or information that is executable by processing circuit QQ302 and available for use by network node QQ300. Memory QQ304 may be used to store any calculation results generated by processing circuit QQ302 and / or any data received via communication interface QQ306. In some embodiments, processing circuit QQ302 and memory QQ304 are integrated.

[0107] The communication interface QQ306 is used for wired or wireless communication of signaling and / or data between a network node, an access network, and / or a UE. As shown in the figure, the communication interface QQ306 includes, for example, a port / terminal QQ316 for transmitting and receiving data to and from a network over a wired connection. The communication interface QQ306 also includes a radio front-end circuit QQ318 that can be coupled to the antenna QQ310 or in some embodiments is part of the antenna QQ310. The radio front-end circuit QQ318 includes a filter QQ320 and an amplifier QQ322. The radio front-end circuit QQ318 can be connected to the antenna QQ310 and the processing circuit QQ302. The radio front-end circuit may be configured to condition signals communicated between the antenna QQ310 and the processing circuit QQ302. The radio front-end circuit QQ318 can receive digital data to be transmitted to other network nodes or UEs via a wireless connection. The radio front-end circuit QQ318 can convert the digital data into a wireless signal having appropriate channel and bandwidth parameters using a combination of the filter QQ320 and / or the amplifier QQ322. The wireless signal can then be transmitted via the antenna QQ310. Similarly, when data is received, the antenna QQ310 collects the wireless signal, and then the wireless signal can be converted into digital data by the radio front-end circuit QQ318. The digital data can be passed to the processing circuit QQ302. In other embodiments, the communication interface may include different components and / or different combinations of components.

[0108] In certain alternative embodiments, network node QQ300 may not include a separate radio front-end circuit QQ318. Instead, processing circuit QQ302 may include a radio front-end circuit and may be connected to antenna QQ310. Similarly, in some embodiments, all or some of RF transceiver circuit QQ312 is part of communication interface QQ306. In yet another embodiment, communication interface QQ306 includes one or more ports or terminals QQ316, radio front-end circuit QQ318, and RF transceiver circuit QQ312 as part of a wireless unit (not shown), and communication interface QQ306 communicates with baseband processing circuit QQ314, which is part of a digital unit (not shown).

[0109] Antenna QQ310 may include one or more antennas or antenna arrays configured to transmit and / or receive wireless signals. Antenna QQ310 may be coupled to radio front-end circuit QQ318 and may be any type of antenna capable of wirelessly transmitting and receiving data and / or signals. In certain embodiments, antenna QQ310 is separate from network node QQ300 and can be connected to network node QQ300 through an interface or port.

[0110] Antenna QQ310, communication interface QQ306, and / or processing circuit QQ302 may be configured to perform any of the receiving operations and / or certain acquisition operations described herein as being performed by a network node. Any information, data, and / or signals may be received from a UE, other network nodes, and / or any other network equipment. Similarly, antenna QQ310, communication interface QQ306, and / or processing circuit QQ302 may be configured to perform any of the transmitting operations described herein as being performed by a network node. Any information, data, and / or signals may be transmitted to a UE, other network nodes, and / or any other network equipment.

[0111] Power supply QQ308 provides power to the various components of network node QQ300 in a form suitable for each component (e.g., at the voltage and current levels required for each respective component). The power supply QQ308 may further include, or be coupled to, a power management circuit for supplying power to the components of network node QQ300 to perform the functionality described herein. For example, network node QQ300 may be connectable to an external power source (e.g., a power grid, an electrical outlet) via an input circuit or interface such as an electrical cable, whereby the external power source supplies power to the power circuit of power supply QQ308. As a further example, power supply QQ308 may include a source of power in the form of a battery or battery pack connected to or integrated into the power circuit. The battery may provide backup power in case of a failure of the external power source.

[0112] Embodiments of network node QQ300 may include additional components other than those shown in FIG. 13 to provide a functional perspective of the network node that includes any of the functionality described herein and / or any functionality necessary to support the subject matter described herein. For example, network node QQ300 may include user interface devices that enable the input of information to network node QQ300 and the output of information from network node QQ300. This may enable a user to perform diagnostic, maintenance, repair, and other management functions on network node QQ300.

[0113] FIG. 14 is a block diagram of a host QQ400 that can be a host of the embodiment of the host QQ116 of FIG. 12 and can implement the computing device 900 according to the various aspects described herein. As used herein, the host QQ400 can be hardware and / or software in various combinations, including a stand-alone server, a blade server, a cloud-implemented server, an edge-implemented server, a distributed server, a virtual machine, a container, or processing resources within a server farm, or can include them. The host QQ400 can provide one or more services to one or more UEs.

[0114] The host QQ400 includes a processing circuit QQ402, a network interface QQ408, a power supply QQ410, and a memory QQ412 that are operably coupled via a bus QQ404 to an input / output interface QQ406. In other embodiments, other components may be included. The functions of those components may be substantially similar to those described for the devices in the previous figures such as FIGS. 17 and 18, and thus, those descriptions are generally applicable to the corresponding components of the host QQ400.

[0115] Memory QQ412 may include one or more computer programs including one or more host application programs QQ414, and data QQ416 that may include user data such as data generated by the UE for the host QQ400 or data generated by the host QQ400 for the UE. Embodiments of the host QQ400 may utilize only a subset or all of the illustrated components. The host application program QQ414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., VVC (Versatile Video Coding), HEVC (High Efficiency Video Coding), AVC (Advanced Video Coding), MPEG, VP9) and audio codecs (e.g., FLAC, AAC (Advanced Audio Coding), MPEG, G.711) including transcoding for different classes, types or implementations of multiple UEs (e.g., handsets, desktop computers, wearable display systems, head-up display systems). Also, the host application program QQ414 may provide user authentication and license checking and may periodically report health, route and content availability to a central node such as a device within or at the edge of the core network. Thus, the host QQ400 may select and / or indicate different hosts for an over-the-top service for the UE. The host application program QQ414 may support various protocols such as the HLS (HTTP Live Streaming) protocol, RTMP (Real-Time Messaging Protocol), RTSP (Real-Time Streaming Protocol), MPEG-DASH (Dynamic Adaptive Streaming over HTTP), etc.

[0116] FIG. 15 is a block diagram showing a virtualization environment QQ500 in which the functions of a computing device 900 implemented according to some embodiments can be virtualized. In this context, virtualization means for generating a virtual version of an apparatus or device may include a virtual hardware platform, a storage device, and networking resources. As used herein, virtualization can be applied to any device or their components described herein and is related to implementation examples in which at least a part of its functionality is implemented as one or more virtual components. Some or all of the functions described herein are implemented as virtual components executed by one or more virtual machines (VMs) implemented within one or more virtual environments QQ500 hosted by one or more hardware nodes such as a hardware computing device operating as a network node, a UE, a core network node, or a host. Further, in embodiments where the virtual node does not require wireless connectivity (e.g., a core network node or a host), the node may be virtualized as a whole.

[0117] An application QQ502 (alternatively, may be referred to as a software instance, a virtual appliance, a network function, a virtual node, a virtual network function, etc.) operates in a virtualization environment Q400 for implementing some features, functions, and / or benefits of some of the embodiments disclosed herein.

[0118] Hardware QQ504 includes a processing circuit, a memory storing software and / or a set of instructions executable by the hardware processing circuit, and / or hardware devices as described herein, such as a network interface and an input / output interface. The software is executed by the processing circuit to instantiate one or more virtualization layers QQ506 (also referred to as a hypervisor or a virtual machine monitor (VMM)), provide VMs QQ508a and QQ508b (which may generally be collectively referred to as VMs QQ508), and / or execute any of the functions, features, and / or benefits described in relation to several embodiments herein. The virtualization layer QQ506 may present a virtual operating platform that appears as networking hardware to the virtual machines QQ508.

[0119] VM QQ508 includes virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and can be executed by the corresponding virtualization layer QQ506. Various embodiments of instances of virtual appliances QQ502 may be implemented in one or more of the VMs QQ508, and the implementation may be made in various ways. Hardware virtualization is referred to in some contexts as network function virtualization (NFV). NFV can be used to consolidate many types of network devices into industry-standard high-capacity server hardware, physical switches, and physical storage that can be located within data centers and customer premise equipment.

[0120] In the context of NFV, VMQQ508 may be a software implementation of a physical machine that runs a program as if it were running on a physical, non-virtualized machine. Each of VMQQ508, and the portion of the hardware QQ504 that executes the VM, whether it is hardware dedicated to the VM or hardware shared with other VMs by the VM, forms a separate virtual network element. Also in the context of NFV, the virtual network function is responsible for handling the proprietary network functions running in one or more VMQQ508 at the top level of the hardware QQ504 and corresponds to the application QQ502.

[0121] Hardware QQ504 may be implemented in a stand-alone network node with general or proprietary components. Hardware QQ504 may implement some functions via virtualization. Alternatively, hardware QQ504 may be part of a larger class of hardware (such as those within a data center or CPE) where multiple hardware nodes cooperate and are managed via management and orchestration QQ510, which oversees, among other things, the lifecycle management of the application QQ502. In some embodiments, hardware QQ504 is coupled to one or more wireless units, each including one or more transmitters and one or more receivers, which may be coupled to one or more antennas. The wireless units may communicate directly with other hardware nodes via one or more appropriate network interfaces and may be used in combination with virtual components to provide wireless capabilities to virtual nodes such as wireless access nodes or base stations. In some embodiments, some signaling can be provided in conjunction with the use of the control system QQ512, which may alternatively be used for communication between the hardware nodes and the wireless units.

[0122] The computing devices described herein (e.g., network nodes, servers, hosts, etc.) can include the illustrated combinations of hardware components, but other embodiments can include computing devices having various combinations of components. It should be understood that those computing devices can include any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determinations, calculations, acquisitions, or similar operations described herein may be performed by a processing circuit, which may, for example, convert the acquired information into other information, compare the acquired information or the converted information with the information stored in the network node, and / or perform one or more operations based on the acquired information or the converted information, and make a determination as a result of that processing, thereby processing the information. Further, the components are shown as a single box placed within a larger box or nested within multiple boxes, but in reality, the communication devices and network nodes can comprise a plurality of various physical components that make up a single illustrated component, and the functions can be partitioned among separate components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of those components may be partitioned between the processing circuit and the communication interface. In other examples, computationally non-intensive functions of any of such components may be implemented in software or firmware, and computationally intensive functions may be implemented in hardware.

[0123] In some embodiments, some or all of the functionality described herein may be provided by a processing circuit executing a set of instructions stored in memory, and in some embodiments, it may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of that functionality may be provided by a processing circuit in a hardwired manner, etc., without executing instructions stored in a separate or discrete device-readable storage medium. In any of those specific embodiments, the processing circuit can be configured to perform the functionality described, whether or not it executes instructions stored in a non-transitory computer-readable storage medium. The advantages provided by such functionality are not limited to the processing circuit alone or to other components of a computing device, but generally are enjoyed by a computing device, and / or an end user and a communication system (e.g., a wireless network) as a whole.

[0124] Additional definitions of embodiments are described below.

[0125] In the foregoing description of various embodiments of the concepts of the present invention, it should be understood that the terms used herein are for the purpose of describing particular embodiments only and are not intended to limit the concepts of the present invention. Unless otherwise defined, all terms (including technical and scientific terms) used herein shall have the same meaning as commonly understood by one of ordinary skill in the art to which the concepts of the present invention pertain. Further, terms defined as would be commonly understood in a general use dictionary should be interpreted as having a meaning that coincides with the meaning in the context of this specification and the relevant art, and should not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0126] When an element is referred to as being "connected to", "coupled to", "responsive to", or variations thereof, another element, it can be directly connected to, coupled to, or responsive to the other element, or intervening elements may be present. In contrast, when an element is referred to as being "directly connected to", "directly coupled to", "directly responsive to", or variations thereof, another element, no intervening elements are present. The same reference numerals refer to the same element throughout. Further, as used herein, "coupled", "connected", "responsive to", or variations thereof, may include being wirelessly coupled, connected, or responsive. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Well-known functions or constructions may not be described in detail for brevity and / or clarity. The term "and / or" includes any and all combinations of one or more of the associated listed items.

[0127] In this specification, terms such as first, second, third, etc. may be used to describe various elements / acts, but it should be understood that these elements / acts are not to be limited by these terms. These terms are only used to distinguish one element / act from another. Thus, a first element / act in some embodiments may be referred to as a second element / act in other embodiments without departing from the teachings of the concept of the invention. The same reference numerals or the same reference signs indicate the same or similar elements throughout the specification.

[0128] As used herein, the terms "comprise", "comprising", "comprises", "include", "including", "includes", "have", "having", "has" or variations thereof are open-ended and include one or more of the recited features, integers, elements, steps, components or functions but do not preclude the presence or addition of one or more other features, integers, elements, steps, components, functions or groups thereof. Further, as used herein, the common abbreviation "e.g.", which is derived from the Latin phrase "exempli gratia", may be used to introduce or specify a general example or examples of the previously mentioned items and is not intended to limit such items. The common abbreviation "i.e.", which is derived from the Latin phrase "id est", may be used to specify particular items from a more general listing.

[0129] Exemplary embodiments are described herein with reference to block diagrams and / or flowchart diagrams of a computer-implemented method, apparatus (system and / or device), and / or computer program product. It should be understood that the blocks of the block diagrams and / or flowchart diagrams, as well as combinations of blocks in the block diagrams and / or flowchart diagrams, can be implemented by computer program instructions executed by one or more computer circuits. These computer program instructions are provided to the processor circuits of general-purpose computer circuits, dedicated computer circuits, and / or other programmable data processing circuits, and are executed via the processors of the computer and / or other programmable data processing devices, so that instructions, conversions, and control transistors, values stored in memory locations, and other hardware components within such circuits implement the functions / operations specified in the block diagrams and / or flowchart blocks or blocks, thereby generating a machine to create means (functions) and / or structures for implementing the functions / operations specified in the block diagrams and / or flowchart blocks.

[0130] These computer program instructions can also be stored in a tangible computer-readable medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable medium produce a manufacture including instructions for implementing the functions / operations specified in one or more blocks of the block diagrams and / or flowcharts. Accordingly, embodiments of the concepts of the present invention can be embodied in hardware and / or software (including firmware, resident software, microcode, etc.) executed on a processor such as a digital signal processor, which may be collectively referred to as "circuits," "modules," or variations thereof.

[0131] Note that in some alternative embodiments, it should be noted that the functions / operations described within a block may be performed in an order different from the order described within the flowchart. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or the blocks may sometimes be executed in the reverse order depending on the functions / operations involved. Further, the functions of a given block in the flowchart and / or block diagram may be separated into multiple blocks, and / or the functions of two or more blocks in the flowchart and / or block diagram may be at least partially integrated. Finally, other blocks can be added / inserted between the illustrated blocks and / or blocks / operations can be omitted without departing from the scope of the concept of the present invention. Furthermore, although some of the figures include arrows on the communication paths to indicate the main direction of communication, it should be understood that communication can occur in a direction opposite to the drawn arrows.

[0132] Many variations and modifications can be made to the embodiments without substantially departing from the principles of the concept of the present invention. All such variations and modifications are intended to be included herein within the scope of the concept of the present invention. Accordingly, the subject matter disclosed above should be considered illustrative and not restrictive, and the examples of embodiments are intended to encompass all such modifications, enhancements, and other embodiments that are within the spirit and scope of the concept of the present invention. Therefore, the scope of the concept of the present invention should be determined by the broadest permissible interpretation of the present disclosure, including the examples of embodiments and their equivalents, to the maximum extent permitted by law, and should not be limited or restricted by the foregoing detailed description.

Claims

1. A method, executed by a computing device (900) in a communication system, for managing power transmission from a local battery disposed proximate to at least one radio head to the at least one radio head, the method comprising: making a determination (1001) regarding power transmission from the local battery to the at least one radio head in a future time window, the determination being made by a machine learning model based on: (i) a determination of the difference between the average power demand and the peak power demand of output power data statistics in a set of past time windows covering a period defined for the at least one radio head; and (ii) time and location dependent cost data of charging and / or discharging the local battery and using power from the power distribution network in the future time window; outputting (1003) the determination regarding power transmission from the local battery to the at least one radio head in the future time window; A method as described above.

2. The method according to claim 1, wherein the output power data statistics include at least: (i) the average power demand of the at least one radio head in the set of past time windows; and (ii) the peak power demand of the at least one radio head in the set of past time windows.

3. The method according to claim 1, wherein the time and location dependent cost data includes: a first cost for charging the local battery in the future time window; and a second cost for using power from the power distribution network for the average power demand in the at least one radio head in the future time window.

4. The method according to claim 1, wherein the determination for the future time window comprises one of: (i) powering the at least one wireless head from the local battery during the future time window, wherein the future time window includes a period of peak power consumption at the at least one wireless head; (ii) charging the local battery during the future time window; (iii) not powering the at least one wireless head from the local battery during the future time window.

5. The method according to claim 3, wherein the determination is made by the machine learning model based on: (i) inputting power data and battery and cost data into the machine learning model; and (ii) determining a minimized total cost for power transmission to the at least one wireless head for the future time window when the future time window is constrained by a plurality of constraints.

6. The method according to claim 5, wherein the plurality of constraints include: (i) a first constraint set as input power consistency such that the input power to the wireless head is equal to the value obtained by adding the power from the power distribution network to the discharge level of the local battery to the wireless head; and (ii) a second constraint set as the discharge such that the discharge from the local battery is less than or equal to the value obtained by adding the charging profile limit of the future time window to the charging level of the local battery at the start of the future time window.

7. The method according to claim 6, wherein the charging profile limit of the future time window includes further constraints, and the further constraints are: (i) a third constraint that the current charge level of the local battery is set as the current charge level such that it is the same as the previous charge level of the local battery plus the difference between the charge amount and the discharge amount of the local battery in the current time window; and (ii) a fourth constraint that the charge level is set such that the charge level of the local battery is less than or equal to the charge capacity of the local battery in any window in the set of time windows.

8. The method according to claim 5, wherein the machine learning model receives reward feedback for pairs of states and actions, the reward feedback being a value that minimizes the total cost of the input power to the at least one wireless head in the next window in the set of time windows, the state including the current charge level of the local battery, the current input power of the at least one wireless head, and the current cost for charging the local battery, the action in the pair of the state and the action including the determination.

9. The method according to claim 1, wherein the at least one wireless head includes a plurality of wireless heads, and further comprising: dividing the defined period into a set of time windows (1101); making a determination for each wireless head for each time window within the set of time windows (1103).

10. The method according to claim 1, wherein outputting the determination (1003) includes outputting the determination for controlling the power transmission to the at least one wireless head based on the determination.

11. The method according to claim 5, wherein the power data is offline data, and making the determination (1001) and outputting the determination (1003) are performed during the training of the machine learning model using the offline data.

12. The method according to claim 5, wherein the power data is online data, The machine learning model is installed in a communication system, The making of the determination (1001) and the outputting (1003) are performed by the installed machine learning model, a method.

13. The method according to claim 1, wherein the computing device is arranged at one of a position close to the local battery and a cloud-based location.

14. A computing device (900) in a communication system for managing power transmission from a local battery arranged close to at least one radio head to the at least one radio head, wherein the computing device is configured to execute the method according to any one of claims 1 to 13.

15. A computer program for causing a computing device (900) to execute the method according to any one of claims 1 to 13.

16. A computer-readable storage medium storing the computer program according to claim 15.

Images