Control device for radio access network and computer-readable storage medium

Abstract

This control device for a radio access network (RAN) comprises: a generation means configured to acquire, from another control device, second training data in first training data collected by the other control device, and to perform machine learning on the basis of the second training data to generate a learning model; a transmission means configured to transmit the learning model to the other control device; and a control means configured to control an operation in which the other control device selects the second training data from the first training data on the basis of the usefulness in the machine learning of the first training data, and transmits the second training data to the control device.

Description

Radio access network control device and computer-readable storage medium

[0001] The present disclosure relates to radio access network (RAN) control techniques.

[0002] Figure 1 shows the control configuration of the RAN proposed by the Open Radio Access Network (O-RAN) Alliance. As shown in Figure 1, a first control function that performs long-term control and a second control function that performs short-term control are defined. In the O-RAN, the first control function is called the non-real-time RAN intelligent controller (Non-RT RIC), and the second control function is called the near-real-time RAN intelligent controller (Near-RT RIC).

[0003] The first control function and the second control function are connected via an A1 interface. The second control function controls components of the RAN, such as a central unit (CU) and distributed units (DU), via an E2 interface. Controlling the CU and DU includes notifying and setting various parameter values ​​used by the CU and DU in their processing, and instructing the CU and DU to perform certain operations. Furthermore, in the following description, the term "RAN" is used as a general term for its components. Therefore, for example, "controlling the RAN" means controlling the CU and DU, which are components of the RAN. The first control function, the second control function, and the RAN are further connected via an O1 interface. The O1 interface can be used to transmit traffic data, performance data, fault data, etc., measured / detected and accumulated by the RAN.

[0004] Non-Patent Document 1 discloses a configuration that uses machine learning for RAN control. Specifically, Non-Patent Document 1 discloses performing machine learning based on various learning data collected from the RAN, etc. to generate a learning model, performing inference using the learning model, and controlling the RAN according to the inference result. In one of the configurations disclosed in Non-Patent Document 1, a first control function generates the learning model. Then, a second control function controls the RAN using the learning model generated by the first control function.

[0005] Non-Patent Document 2 discloses distributed reinforcement learning, which can achieve high learning efficiency. As shown in FIG. 2, in distributed reinforcement learning, a learning unit performs machine learning based on learning data stored in a storage unit to generate a learning model. The learning unit distributes the learning model to multiple execution units that are distributed. Each execution unit is associated with a "control object." The execution unit determines control details for the control object using the learning model based on the "current state" of the control object, and executes control of the control object based on the determined "control details." The execution unit determines the "next state" of the control object as a result of this control, and evaluates the "degree of achievement" of the control goal. Note that in distributed reinforcement learning, the "control object," "current state," "next state," "control details," and "degree of achievement" described above are referred to as "environment," "state," "next state," "action," and "reward," respectively.

[0006] The execution unit then calculates an evaluation value called a time difference error based on the achievement level, the current state, and the next state. In the following description, the time difference error is referred to as a TD error, and its value is referred to as a TD error value. The TD error value is defined as the sum of the "achievement level" obtained as a result of control and the value determined based on the "next state," minus the value determined based on the "current state" before control. A large TD error value means that the achievement level of the control goal in the "next state" is low, that is, the accuracy of the output of the learning model based on the "current state" is low, or the output error of the learning model based on the "current state" is large.

[0007] The execution unit stores the "current state," "next state," "control content," and "achievement level" as learning data in the storage unit for the next learning by the learning unit. At this time, the execution unit assigns the TD error value calculated for the learning data to the learning data. As described above, a large TD error value of the learning data means that the accuracy of the output of the learning model based on the input data (current state) to the learning model included in the learning data is low. Therefore, using learning data with a large TD error value for machine learning significantly updates the learning model, improving the learning effect. For this reason, the learning unit prioritizes the learning data with a larger TD error value assigned from among the learning data stored in the storage unit for use in machine learning. This can improve the learning efficiency of the learning unit.

[0008] O-RAN Alliance, "AI / ML workflow description and requirements", O-RAN. WG2. AIML-v01.03, July 2021 D. Horgan, et. al. , "Distributed prioritized experience replay", arXiv:1803.00933. , April 2018

[0009] FIG. 3 shows a configuration in which distributed reinforcement learning is applied to an O-RAN. An execution unit that controls the RAN based on a learning model is located in the second control function, while a learning unit that generates the learning model and a storage unit that stores the learning data collected by the execution unit are located in the first control function. However, with the configuration of FIG. 3, the amount of learning data from each execution unit of the second control function becomes too large. For example, assume that there are 2.5 million user equipment (UE) devices in the area covered by the first control function, the size of the learning data acquired from the RAN is 356 bytes per UE, and each execution unit of the second control function in the area covered by the first control function transmits learning data to the first control function at a cycle of 20 times per second. In this case, the first control function would receive 142.4 gigabits of data per second, which is unrealistic. For example, by lengthening the cycle at which each execution unit of the second control function transmits learning data, the amount of learning data received by the first control function can be reduced, but this may reduce learning efficiency.

[0010] According to one aspect of the present invention, a control device of a radio access network (RAN) includes a generation means configured to acquire second learning data from another control device among first learning data collected by the other control device, and perform machine learning based on the second learning data to generate a learning model; a transmission means configured to transmit the learning model to the other control device; and a control means configured to control the operation of the other control device to select the second learning data from the first learning data based on the usefulness of the first learning data in the machine learning and transmit the second learning data to the control device.

[0011] According to the present disclosure, it is possible to prevent a decrease in learning efficiency and to prevent the amount of learning data transmitted between a device that collects learning data and a device that performs machine learning from increasing too much.

[0012] Other features and advantages of the present invention will become apparent from the following description taken in conjunction with the accompanying drawings, in which the same or similar elements are designated by the same reference numerals.

[0013] A control configuration diagram of a RAN according to the background art. A configuration diagram of distributed reinforcement learning according to the background art. A configuration diagram when distributed reinforcement learning is applied to RAN control. A control configuration diagram of a RAN according to an embodiment. A configuration diagram of a second control device according to an embodiment. A configuration diagram of a first control device according to an embodiment. A sequence diagram according to an embodiment.

[0014] Hereinafter, the embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention as claimed, and not all combinations of features described in the embodiments are necessarily essential to the invention. Two or more of the features described in the embodiments may be arbitrarily combined. Furthermore, the same reference numerals are used for the same or similar components, and redundant explanations will be omitted.

[0015] FIG. 4 shows a control configuration of a RAN according to this embodiment. The configuration of FIG. 4 is basically the same as the configuration of FIG. 1. The first control device performs long-term control of the RAN and is, for example, a device that implements the function of Non-RT RIC of the O-RAN. The second control device controls the RAN at a shorter period than the first control device and is, for example, a device that implements the function of Near-RT RIC of the O-RAN. The first control device is connected to one or more second control devices. Note that in the example of FIG. 4, the first control device is connected to three second control devices. The first control device performs machine learning to generate a learning model and distributes the learning model to each second control device. Each second control device is associated with a geographical area and uses the learning model obtained from the first control device to control the components of the RAN located in the associated geographical area. It should be noted that the second control device controls the part of the geographical area of ​​the RAN of the mobile communication network that is associated with the second control device, but the part of the geographical area of ​​the RAN of this mobile communication network that is associated with the second control device will also be referred to as RAN.

[0016] FIG. 5 is a configuration diagram of the second control device. The execution unit 21 acquires a learning model from the first control device and performs inference based on the learning model to control the RAN. The control goal may be, for example, maximizing overall throughput or maximizing the average quality of the wireless section. In this case, for example, the second control device may control beamforming to achieve the control goal. The control goal may also be, for example, minimizing the computer resources used to realize the CU and DU using virtualization technology. In this case, for example, the second control device may control the placement of the CU and DU in the RAN.

[0017] For example, the execution unit 21 inputs the "current state" of the RAN collected from the RAN into the learning model to obtain the output of the learning model, and determines the "control content" of the RAN based on this output to control the RAN. The execution unit 21 then collects the "next state" of the control target as a result of this control from the RAN and evaluates the "achievement level" of the control goal. The execution unit 21 then generates learning data based on the "current state," the "next state," the "control content," and the "achievement level," determines an evaluation value for the learning data, and assigns it to the learning data. The execution unit 21 stores the learning data with the assigned evaluation value in the storage unit 22. In this way, the execution unit 21 has a function of controlling the RAN based on the learning model, a function of collecting learning data, and a function of determining the evaluation value of the learning data. The evaluation value of the learning model indicates the usefulness of the learning model in machine learning for the learning model. Note that high usefulness means that the learning effect of machine learning using the learning data is high. As an example, the above-mentioned TD error value can be used as the evaluation value. In other words, the larger the learning data based on input data to the learning model that increases the error in the output of the learning model, the larger the value that can be used as the evaluation value.In other words, the larger the learning data based on data that decreases the degree of achievement of the control goal for the RAN, the larger the value that can be used as the evaluation value.

[0018] When the TD error value is used as the evaluation value, the larger the evaluation value assigned to the training data, the more useful the training data is. However, the evaluation value is not limited to the TD error value, and any value that changes depending on, for example, the inference error of the training model, i.e., the error in the output of the training model, can be used. In this case, the more the training data is based on input data that increases the error in the inference (output of the training model), the more useful the training data is in machine learning. In other words, the more the training data is based on input data that decreases the accuracy of the inference (output of the training model), the more useful the training data is in machine learning. Note that the smaller the evaluation value of the training data, the more useful the training data is in machine learning.

[0019] The processing unit 23 reads out the learning data to which evaluation values ​​have been assigned that is stored in the storage unit 22 and transmits it to the first control device. In the following description, "learning data to which evaluation values ​​have been assigned" will be simply referred to as "learning data." For example, if the first control device is equipped with a non-RT RIC and the second control device is equipped with a near-RT RIC, the processing unit 23 can transmit the learning data to the first control device via the A1 interface or the O1 interface. In this embodiment, the processing unit 23 (second control device) is configured to operate in at least one of the three operating modes described below.

[0020] First operating mode: In the first operating mode, an upper limit for the transmission rate is set in the processing unit 23, and the processing unit 23 reads learning data from the storage unit 22 and transmits it to the first control device so that the transmission rate to the first control device is equal to or lower than the set upper limit. In other words, the processing unit 23 may transmit learning data at a transmission rate equal to or lower than the upper limit while the amount of learning data stored in the storage unit 22 is small, but controls reading from the storage unit 22 so that the transmission rate does not exceed the upper limit no matter how much the amount of learning data stored in the storage unit 22 increases. When transmitting learning data to the first control device, the processing unit 23 reads the learning data from the storage unit 22 in descending order of usefulness indicated by the evaluation value. For example, if the TD error value is used as the evaluation value, the processing unit 23 reads the learning data from the storage unit 22 in descending order of evaluation value.

[0021] By setting an upper limit on the transmission speed of data transmitted to the first control device, it is possible to prevent the amount of learning data transmitted to the first control device that performs machine learning from increasing too much. Furthermore, since learning data that has been assigned an evaluation value indicating high usefulness in machine learning is transmitted to the first control device with priority, it is possible to prevent a decrease in learning efficiency in the first control device.

[0022] Second operation mode: In the second operation mode, a threshold value for the evaluation value is set in the processing unit 23, and the processing unit 23 reads out from the storage unit 22 only learning data to which an evaluation value indicating usefulness higher than the set threshold value is assigned, and transmits the learning data to the first control device. For example, when the TD error value is used as the evaluation value, the processing unit 23 reads out from the storage unit 22 only learning data whose evaluation value is greater than the threshold value.

[0023] By transmitting only the learning data that satisfies a predetermined condition to the first control device, rather than transmitting all of the learning data collected by the execution unit 21 to the first control device, it is possible to prevent the amount of learning data transmitted to the first control device that performs machine learning from increasing too much. Furthermore, because only the learning data that has been assigned an evaluation value indicating that its usefulness in machine learning is higher than a threshold is transmitted to the first control device, it is possible to prevent a decrease in the learning efficiency of the first control device.

[0024] Third operating mode: In the third operating mode, as in the first operating mode, an upper limit value for the transmission speed is set in the processing unit 23, and the processing unit 23 reads learning data from the storage unit 22 and transmits it to the first control device so that the transmission speed to the first control device is equal to or less than the set upper limit value. However, in the third operating mode, the processing unit 23 is further set with judgment information for judging the value of the read probability (probability value) of the learning data based on the evaluation value of the learning data. Note that the judgment information is set so that the probability value of the learning data increases as the usefulness of the learning data increases. The processing unit 23 judges the probability value of the read probability of the learning data based on the judgment information. Then, the processing unit 23 reads out the learning data stored in the storage unit 22 according to the probability value of each piece of learning data. Note that the higher the probability value of learning data, the higher the probability that it will be read from the storage unit 22.

[0025] Therefore, in the third operation mode, unlike the first operation mode, it is probabilistic that less useful learning data stored in the storage unit 22 is read from the storage unit 23 and transmitted to the first control device before more useful learning data stored in the storage unit 22. For example, if machine learning is performed using only useful learning data, the performance of the learning model may deteriorate due to so-called overfitting. In the third operation mode, less useful learning data is also probabilistically transmitted to the first control device, thereby suppressing performance deterioration of the learning model due to overfitting.

[0026] The determination information may be information indicating a formula or coefficients of the formula for converting the evaluation value into a probability value, or information indicating a lookup table (LUT) showing the relationship between the evaluation value and the probability value.

[0027] In any operation mode, it may happen that less useful learning data is not read out from the storage unit 22. To prevent "old" learning data that has been collected for some time from being used for machine learning, the storage unit 22 may be configured to discard learning data that has been stored in the storage unit 22 for a predetermined period of time.

[0028] FIG. 6 is a configuration diagram of the first control device. The storage unit 12 stores the learning data received from the second control device. The learning unit 11 performs machine learning based on the learning data stored in the storage unit 12 to generate a learning model. At this time, the learning unit 11 can prioritize learning data with a higher usefulness evaluation value for use in machine learning. The learning unit 11 transmits the generated learning model to each second control device. Thus, the learning unit 11 has a generation function for performing machine learning to generate a learning model and a transmission function for transmitting the generated learning model. The generation of learning data by the learning unit 11 and the subsequent transmission to each second control device can be performed at a predetermined interval or when a predetermined criterion is met. For example, the predetermined criterion can be met when a predetermined amount of learning data stored in the storage unit 12 is reached.

[0029] The control unit 13 controls the operation mode of each second control device (processing unit 23). Fig. 7 is a sequence diagram of the process of controlling the operation mode of the processing unit 23. In S1, the control unit 13 transmits a capability request message to the processing unit 23. In response to the capability request message, the processing unit 23 transmits a capability response message to the control unit 13 in S2. The capability response message contains information indicating the operation modes that can be supported (used) by the processing unit 23. The control unit 13 selects the operation mode to actually be used from the operation modes that can be supported by the processing unit 23.

[0030] In S3, the control unit 13 transmits a collection request message including information indicating the operation mode to be actually used to the processing unit 23. The collection request message includes operation information corresponding to the operation mode to be actually used. For example, when the first operation mode is used, the operation information includes information indicating an upper limit of the transmission speed when transmitting the learning data. When the second operation mode is used, the operation information includes information indicating a threshold value. Furthermore, when the third operation mode is used, the operation information includes information indicating an upper limit of the transmission speed when transmitting the learning data and determination information. In S4, the processing unit 23 transmits a collection response message to the control unit 13 as a response to the collection request message. Thereafter, the processing unit 23 performs processing to transmit the learning data to the first control device until the processing unit 23 is notified by the first control device to stop collection.

[0031] Furthermore, the control unit 13 can change the operation mode of the processing unit 23 in response to the collection request message of S3. Furthermore, the control unit 13 can change the operation information used in the operation mode currently used by the processing unit 23 in response to the collection request message of S3. For example, when operating in the first operation mode or the third operation mode, the control unit 13 can change the upper limit of the transmission speed depending on the time period and traffic conditions. This can reduce the amount of learning data transmitted to the first control device during times of heavy traffic and increase the amount of learning data transmitted to the first control device during times of light traffic. Furthermore, when operating in the third operation mode, the control unit 13 can change the determination information depending on whether degradation of learning performance due to overlearning has occurred, and control the probability value calculated from the evaluation value. This can dynamically suppress degradation of learning performance due to overlearning.

[0032] Furthermore, when operating in the second operating mode, the control unit 13 can dynamically update the threshold value according to the amount of learning data transmitted from each second control device. The evaluation value of each learning data item increases on average while the inference performance of the learning model is low, and decreases on average as the inference performance of the learning model improves. Therefore, if the threshold value is kept constant, the amount of learning data transmitted from the second control device to the first control device may become too small as the inference performance of the learning model improves. Therefore, by dynamically controlling the threshold value, it is possible to prevent the amount of learning data transmitted from each second control device from becoming too small.

[0033] In this embodiment, the operation mode is changed or the operation information used in the operation mode is changed in response to the collection request message in S3. However, the operation mode or the operation information used in the operation mode may be changed using a message other than the collection request message, such as a "change request message." Furthermore, instead of the control unit 13 setting the operation information in each second control device, a device other than the first control device and the second control device, or an operator of a mobile communication network, may set the operation information in the second control device. In this case, the control unit 13 does not need to notify the processing unit 23 of the operation information in S3. In the third operation mode, the first control device may set one of the upper limit value and the determination information, and the other may be set by an operator of a mobile communication network, etc. Furthermore, for example, if all processing units 23 are configured with only one operation mode, the processes in S1 and S2 may be omitted. In other words, the control unit 13 does not need to determine the operation modes available to the processing unit 23 and specify one operation mode to be used to the processing unit 23; the processing unit 23 operates according to the configured single operation mode.

[0034] With the above configuration, the first control device can specify the operation mode of the second control device and control the amount of learning data transmitted from each second control device, thereby preventing a decrease in learning efficiency and preventing the amount of learning data transmitted from each second control device to the first control device from increasing too much.

[0035] The first control device and the second control device may be realized by a single device, for example, a single computer. Furthermore, the first control device and the second control device may be realized by multiple devices, for example, multiple computers, that can communicate with each other. As an example, the learning unit 11, the storage unit 12, and the control unit 13 of the first control device may each be implemented in a single device, and the three devices that implement the learning unit 11, the storage unit 12, and the control unit 13 may communicate with each other via a network, thereby constituting a single first control device as a whole. Furthermore, for example, the learning unit 11 and the control unit 13 may be implemented in a single device, and the storage unit 12 may be implemented in a single device, and these two devices may communicate with each other via a network, thereby constituting a single first control device as a whole. The same applies to the second control device.

[0036] The present disclosure further provides a program executable by one or more processors, the program including instructions that, when executed by one or more processors of an apparatus, cause the apparatus to function as a first control device or a second control device. The present disclosure also provides a non-transitory computer-readable storage medium having the program stored thereon. The present disclosure also provides a method performed by a first control device or a second control device, a program for causing an apparatus having one or more processors to perform the method, and a non-transitory computer-readable storage medium having the program stored thereon.

[0037] The invention is not limited to the above-described embodiment, and various modifications and variations are possible within the scope of the gist of the invention.

[0038] This application claims priority based on Japanese Patent Application No. 2023-150825, filed on September 19, 2023, the entire contents of which are incorporated herein by reference.

Claims

1. A control device for a radio access network (RAN), comprising: a generation means configured to acquire second learning data from another control device among first learning data collected by the another control device, and perform machine learning based on the second learning data to generate a learning model; a transmission means configured to transmit the learning model to the another control device; and a control means configured to control the operation of the other control device to select the second learning data from the first learning data based on the usefulness of the first learning data in the machine learning and transmit the second learning data to the control device.

2. The control device according to claim 1, wherein the control means is further configured to obtain from the other control device one or more operating modes of the operation available in the other control device, and to control the operation by specifying one operating mode to be used in the other control device from the one or more operating modes.

3. The control device according to claim 2, wherein the one or more operating modes include a first operating mode in which the other control device selects the second learning data from the first learning data in descending order of the usefulness of the first learning data so that the transmission speed of the second learning data to the control device does not exceed an upper limit value.

4. The control device according to claim 3, wherein said control means is further configured to notify said another control device of said upper limit value when said first operation mode is designated.

5. A control device as claimed in any one of claims 2 to 4, wherein the one or more operating modes include a second operating mode in which the other control device selects the first learning data whose usefulness is higher than a threshold as the second learning data.

6. The control device according to claim 5, wherein said control means is further configured to notify said another control device of said threshold value when said second operating mode is designated.

7. A control device as described in any one of claims 2 to 6, wherein the one or more operating modes include a third operating mode in which the other control device determines a probability value of a selection probability of the first learning data based on the usefulness of the first learning data, and the other control device selects the second learning data from the first learning data with a selection probability based on the probability value of the first learning data so that a transmission speed of the second learning data to the control device does not exceed an upper limit value, and the probability value becomes higher as the usefulness becomes higher.

8. The control device according to claim 7, wherein the control means is further configured to, when specifying the third operating mode, notify the other control device of the upper limit value and judgment information for judging the probability value based on the usefulness.

9. A control device as claimed in any one of claims 1 to 8, wherein the other control device selects the second learning data from the first learning data in descending order of the usefulness of the first learning data so that a transmission speed of the second learning data to the control device does not exceed an upper limit value, and the control means is further configured to control the operation by controlling the upper limit value.

10. A control device as claimed in any one of claims 1 to 8, wherein the other control device selects as the second learning data the first learning data whose usefulness is higher than a threshold value, and the control means is further configured to control the operation by controlling the threshold value.

11. A control device as described in any one of claims 1 to 8, wherein the other control device determines a probability value of a selection probability of the first learning data based on the usefulness of the first learning data, and selects the second learning data from the first learning data with a selection probability based on the probability value of the first learning data so that a transmission speed of the second learning data to the control device does not exceed an upper limit value, the probability value becoming higher as the usefulness becomes higher, and the control means is further configured to control the operation by controlling at least one of the upper limit value and judgment information for determining the probability value based on the usefulness.

12. A control device according to any one of claims 1 to 11, wherein the usefulness of the first learning data increases as the time difference error of the first learning data increases.

13. The control device according to any one of claims 1 to 12, wherein the other control device is a Near-RT RIC.

14. The control device according to any one of claims 1 to 13, wherein the control device is a Non-RT RIC.

15. A computer-readable storage medium storing a program that, when executed by one or more processors of an apparatus having one or more processors, causes the apparatus to function as a control device according to any one of claims 1 to 14.

Images