Communication control system and communication control method

Abstract

To improve the learning efficiency of models and the inference accuracy of trained models in the control of a radio access network.SOLUTION: A communication control system includes a model learning unit provided corresponding to each classification of radio environments around user terminals, a model execution unit provided corresponding to each user terminal, and a storage unit that stores inference data by the model of the model execution unit corresponding to the user terminal associated with one classification of the radio environment, for each classification of the radio environment. The model trained by one model learning unit is supplied to the model execution unit corresponding to the user terminal associated with the classification of the radio environment corresponding to the one model learning unit, and the one model learning unit learns a model that infers predetermined communication control items using the inference data of the classification of the radio environment corresponding to the one model learning unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication control system and a communication control method. [Background technology]

[0002] In recent years, in the "Beyond 5G" system, which is the fifth-generation mobile communication system (5G) and beyond, improvements in communication performance, such as throughput, communication latency, and the number of terminal connections, have been considered to provide a wide variety of services, such as robot control, connected cars, augmented reality (AR), and virtual reality (VR). Artificial intelligence (AI) and machine learning (ML) are attracting attention as key technologies for achieving this. For example, in the radio access network (RAN), the application of reinforcement learning to various applications, such as beamforming control, radio resource allocation, and base station function placement, is being considered in order to maximize network performance using limited network resources.

[0003] Additionally, the O-RAN (Open Radio Access Network) Alliance is studying the openness and intelligence of next-generation radio access networks such as 5G, and is examining the interfaces and workflows required for control using AI and ML. O-RAN defines RICs (RAN Intelligent Controllers), which are controllers that control the RAN. There are two types of RICs: the Non-RT RIC (Non-Real Time RAN Intelligent Controller), which is designed for long-term control, and the Near-RT RIC (Near-Real Time RAN Intelligent Controller), which is designed for short-term control.

[0004] It is being considered to achieve low-latency inference by placing the learning unit with AI and ML functions in the "Non-RT RIC," which has abundant computational resources available, while placing the inference unit that performs inference using a trained model in the "Near-RT RIC." For example, Non-Patent Document 1 proposes placing the "Data Pipeline," which processes training data, in the "Near-RT RIC." Furthermore, Non-Patent Document 2 proposes a procedure in the A1 interface, which is an interface between the "Non-RT RIC" and the "Near-RT RIC," for collecting training data via the "Data Pipeline."

[0005] Furthermore, for example, Non-Patent Document 3 describes a distributed reinforcement learning technology for achieving high learning efficiency in online learning. This distributed reinforcement learning technology is characterized by collecting large amounts of learning data through parallel execution as parallel processing using a distributed architecture, having multiple distributed "Actors" collect learning data (experience) through trial and error with the environment, which is then learned by a single "Learner," and the single "Learner" periodically supplies the unique learned model to each "Actor" to update the model common to each "Actor." [Prior art documents] [Non-patent literature]

[0006] [Non-Patent Document 1] “Discussion on Data Collection for Training in Non-RT RIC”, KDDI, CMCC, O-RAN WG2, July 25 2023 [Non-patent document 2] “Study of AI / ML-Related Procedures for RAN Slice SLA Assurance Use Case”, KDDI, CMCC, O-RAN WG2, June 21, 2023 [Non-patent document 3] D. Horgan, et al. “Distributed Prioritized Experience Replay”, ICLR, 2018 Summary of the Invention [Problem to be solved by the invention]

[0007] When applying distributed reinforcement learning technology to RAN control, one possible approach to reducing the processing load of online learning is to reduce the size of models such as neural networks and the amount of training data. However, for large-scale RANs, reducing the model size or the amount of training data makes it impossible to fully learn the diverse radio environments that may differ for each user equipment (UE), which can lead to a decline in the quality of the trained model and a decrease in the accuracy of inference based on the trained model.

[0008] The present invention has been made in consideration of these circumstances, and its purpose is to improve the learning efficiency of models and the inference accuracy of trained models in the control of radio access networks (RAN). [Means for solving the problem]

[0009] One aspect of the present invention is a communication control system comprising, in a wireless access network, a model learning unit provided corresponding to each classification of wireless environments around user terminals, a model execution unit provided corresponding to each user terminal, and a storage unit that stores, for each classification of wireless environments, inference data based on a model from the model execution unit corresponding to a user terminal associated with a classification of the wireless environment, wherein a model learned by one of the model learning units is supplied to the model execution unit corresponding to the user terminal associated with the classification of the wireless environment corresponding to the one of the model learning units, and the one of the model learning units uses the inference data of the classification of the wireless environment corresponding to the one of the model learning units to learn a model that infers a specified communication control item. One aspect of the present invention is a communication control system that, in the above-mentioned communication control system, further comprises a terminal group organizing unit that organizes user terminal groups according to radio environment classification, and the terminal group organizing unit dynamically changes the organization of the user terminal groups in response to changes in the radio environment around each user terminal.

[0010] One aspect of the present invention is a communication control method comprising, in a wireless access network, a model learning unit provided corresponding to each classification of wireless environments around user terminals, a model execution unit provided corresponding to each user terminal, and a storage unit that stores, for each classification of wireless environments, inference data based on a model from the model execution unit corresponding to a user terminal associated with a classification of the wireless environment, wherein a model learned by one of the model learning units is supplied to the model execution unit corresponding to the user terminal associated with the classification of the wireless environment corresponding to the one of the model learning units, and the one of the model learning units uses the inference data of the classification of the wireless environment corresponding to the one of the model learning units to learn a model that infers a specified communication control item. [Effects of the Invention]

[0011] According to the present invention, it is possible to obtain the effect of improving the learning efficiency of models and the inference accuracy of trained models in the control of radio access networks (RAN). [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram showing an example of the configuration of a communication control system according to a first embodiment. [Figure 2] 4 is a flowchart showing an example of the procedure of a communication control method according to the first embodiment. [Figure 3] FIG. 10 is a block diagram showing an example of the configuration of a communication control system according to a second embodiment. [Figure 4] FIG. 10 is a block diagram showing an example of the configuration of a communication control system according to a third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0014] [First embodiment] Fig. 1 is a block diagram showing an example of the configuration of a communication control system according to the first embodiment. In Fig. 1, the communication control system 1 includes a plurality of model learning units 10 (Learner#1-#N, where N is an integer equal to or greater than 2), storage units 20 (Repository#1-#N) provided corresponding to each of the model learning units 10, and model execution units 30 (Actor for UE1-UEk) provided corresponding to each of the user equipments (UE1-UEk).

[0015] UE1-UEk are associated with classifications of the radio environment around the UE. For example, the classification of the radio environment around the UE is whether the signal strength of an AP (access point) around the UE is "high" or "low." In this case, a signal strength threshold is defined for determining whether the signal strength of the AP around the UE is high or low. Among UE1-UEk, UEs whose signal strength of the AP around the UE is equal to or greater than the signal strength threshold are associated with a classification in which the signal strength of the AP around the UE is "high." On the other hand, among UE1-UEk, UEs whose signal strength of the AP around the UE is less than the signal strength threshold are associated with a classification in which the signal strength of the AP around the UE is "low." This results in the organization of a group of UEs associated with a classification in which the signal strength of the AP around the UE is "high" and a group of UEs associated with a classification in which the signal strength of the AP around the UE is "low."

[0016] Examples of classification of the wireless environment around the UE include classification by the RSRP (Reference Signal Received Power) value (average, total, etc.) of APs around the UE, classification by the SINR (Signal to Interference Noise power Ratio) value (average, total, etc.) of APs around the UE, classification by the UE's moving speed, classification by geographical information of the AP to which the UE is connected, etc. Examples of geographical information of the AP include urban structure, UE-related information, and images (still images, video) taken by a camera located near the AP.

[0017] Examples of classification based on urban structure include classification based on clustering of wireless environments based on building height data, classification based on clustering of wireless environments based on satellite image data, classification based on building area ratio, classification based on building occupancy rate around UE based on point cloud data, etc. The building occupancy rate around UE is an indicator of whether the visibility around UE is good.

[0018] Examples of classification based on UE-related factors include classification based on the density of UE distribution, classification based on the density of AP distribution, and classification based on traffic volume.

[0019] A model learning unit 10 is provided corresponding to each classification of the wireless environment around the UE. For example, if the wireless environment around the UE is classified into two categories, "high" and "low" signal strength of APs around the UE, a model learning unit 10 (Learner #1) corresponding to the classification of "high" signal strength of APs around the UE and a model learning unit 10 (Learner #2) corresponding to the classification of "low" signal strength of APs around the UE are provided.

[0020] A storage unit 20 is provided corresponding to each model learning unit 10. For example, if the wireless environment around the UE is classified into two categories, namely, "high" and "low" signal strength of APs around the UE, a storage unit 20 (Repository #1) corresponding to the category of "high" signal strength of APs around the UE and a storage unit 20 (Repository #2) corresponding to the category of "low" signal strength of APs around the UE are provided.

[0021] The model learning unit 10 learns a model using the storage unit 20 corresponding to itself. For example, if the wireless environment around the UE is classified into two categories, "high" and "low", the signal strength of APs around the UE, the model learning unit 10 (Learner#1) corresponding to the category of "high" signal strength of APs around the UE learns a model using the storage unit 20 (Repository#1) corresponding to the category of "high" signal strength of APs around the UE. On the other hand, the model learning unit 10 (Learner#2) corresponding to the category of "low" signal strength of APs around the UE learns a model using the storage unit 20 (Repository#2) corresponding to the category of "low" signal strength of APs around the UE. In this embodiment, reinforcement learning is used as the model learning method.

[0022] A model execution unit 30 (Actor for UE1-UEk) is provided corresponding to each of the user equipments (UE1-UEk). A model learned by one model learning unit 10 is supplied to the model execution unit 30 corresponding to the user equipment associated with the classification of the wireless environment corresponding to that one model learning unit 10.

[0023] For example, if the wireless environment around the UE is classified into two categories, "high" and "low," the model (Model #1) learned by the model learning unit 10 (Learner #1) corresponding to the "high" category of signal strength of APs around the UE is supplied to the model execution units 30 (Actor for UE1, Actor for UEk) corresponding to UE1 and UEk, respectively, associated with the "high" category of signal strength of APs around the UE. Meanwhile, the model (Model #2) learned by the model learning unit 10 (Learner #2) corresponding to the "low" category of signal strength of APs around the UE is supplied to the model execution unit 30 (Actor for UE2) corresponding to UE2, associated with the "low" category of signal strength of APs around the UE.

[0024] The model execution unit 30 executes a model (trained model) that has been trained by the model learning unit 10. By executing the trained model, one model execution unit 30 takes an action on a UE (target UE) corresponding to that one model execution unit 30, and obtains states and rewards from the target UE as a result of the action.

[0025] For example, if the wireless environment around a UE is classified into two categories, namely, "high" and "low" signal strength of APs around the UE, the model execution unit 30 (Actor for UE1, Actor for UEk) corresponding to the "high" signal strength category of APs around the UE takes action toward the target UE (UE1, UEk) by executing the learned model (Model #1) of the model learning unit 10 (Learner #1) corresponding to the "high" signal strength category of APs around the UE, and obtains a state and a reward from the target UE (UE1, UEk) as a result of the action. On the other hand, the model execution unit 30 (Actor for UE2) corresponding to the "low" signal strength category of APs around the UE takes action toward the target UE (UE2) by executing the learned model (Model #2) of the model learning unit 10 (Learner #2) corresponding to the "low" signal strength category of APs around the UE, and obtains a state and a reward from the target UE (UE2) as a result of the action.

[0026] In the model execution unit 30, the trained model performs inference based on the state and reward resulting from the action taken against the target UE. Inference data (Experience) indicating the result of inference by the trained model of one model execution unit 30 is stored in the storage unit 20 corresponding to that one model execution unit 30.

[0027] For example, if the wireless environment around the UE is classified into two categories, "high" and "low" signal strength of APs around the UE, the inference data (Experience) indicating the results of inference by the trained model (Model #1) of the model execution unit 30 (Actor for UE1, Actor for UEk) corresponding to the category of "high" signal strength of APs around the UE is stored in the storage unit 20 (Repository #1) corresponding to the category of "high" signal strength of APs around the UE. On the other hand, the inference data (Experience) indicating the results of inference by the trained model (Model #2) of the model execution unit 30 (Actor for UE2) corresponding to the category of "low" signal strength of APs around the UE is stored in the storage unit 20 (Repository #2) corresponding to the category of "low" signal strength of APs around the UE.

[0028] One model learning unit 10 uses inference data of the classification of the wireless environment corresponding to that one model learning unit 10 to learn (reinforcement learning) a model that infers a predetermined communication control item.

[0029] For example, if the wireless environment around the UE is classified into two categories, "high" and "low" signal strength of APs around the UE, the model learning unit 10 (Learner #1) corresponding to the category of "high" signal strength of APs around the UE learns a model that infers a predetermined communication control item using inference data (Experience) stored in the storage unit 20 (Repository #1) corresponding to the category of "high" signal strength of APs around the UE. On the other hand, the model learning unit 10 (Learner #2) corresponding to the category of "low" signal strength of APs around the UE learns a model that infers a predetermined communication control item using inference data (Experience) stored in the storage unit 20 (Repository #2) corresponding to the category of "low" signal strength of APs around the UE.

[0030] An example of a communication control item according to this embodiment is a cluster (AP cluster) of APs (access points) around the UE. In the case of the communication control item "AP cluster," the state used by the model can be, for example, the current AP cluster, the SLA (Service Level Agreement), the RSRP value of the APs around the UE, or the SINR value of the APs around the UE. In the case of the communication control item "AP cluster," the model infers (what action should be taken), for example, the action to be taken when the SLA is insufficient or the action to be taken when the SLA is excessive. For example, when the SLA is insufficient, the number of APs belonging to the AP cluster can be increased, while when the SLA is excessive, the number of APs belonging to the AP cluster can be decreased.

[0031] An example of a communication control item according to this embodiment is the selection of APs to be used for interference suppression. In the case of the communication control item "selection of APs to be used for interference suppression," the state used by the model may be, for example, the current AP cluster, a list of APs to be used for interference suppression, the SLA, the RSRP values ​​of APs around the UE, and the SINR values ​​of APs around the UE. In the case of the communication control item "selection of APs to be used for interference suppression," the model infers (what action should be taken), for example, the action to be taken when the SLA is insufficient or the action to be taken when the SLA is excessive. For example, when the SLA is insufficient, the number of APs to be used for interference suppression may be increased, while when the SLA is excessive, the number of APs to be used for interference suppression may be reduced.

[0032] An example of a communication control item according to this embodiment is the placement of a vDU (virtual distributed unit). In the case of the communication control item "vDU placement," the state used by the model may include, for example, the current placement of the vDU on the virtual infrastructure (server identifier (server ID) and base ID), SLA, the RSRP value of the AP around the UE, the SINR value of the AP around the UE, and various server performances. In the case of the communication control item "vDU placement," the item inferred by the model (what action should be taken) may include, for example, a change in the vDU selected by the UE.

[0033] An example of a reward in reinforcement learning according to this embodiment is the difference between the communication speed (throughput) and the SLA. A specific example is "reward = (performance throughput) - (throughput determined by the SLA)." Another specific example is a reward of "1" when the performance throughput is equal to or greater than the throughput determined by the SLA, and a reward of "0" when the performance throughput is less than the throughput determined by the SLA.

[0034] 2 is a flowchart showing an example of the procedure of the communication control method according to this embodiment, which will be described with reference to FIG.

[0035] (Step S1) For each classification of the radio environment around the UE, the model learned by the model learning unit 10 is supplied to the model executing unit 30. Note that an initial model such as a neural network is set in advance in the communication control system 1. In the initial stage of model learning by the model learning unit 10, the initial model is supplied.

[0036] (Step S2) The model execution unit 30 corresponding to each classification of the radio environment around the UE executes its own trained model. As a result, the model execution unit 30 takes action on the target UE and obtains a state and a reward from the target UE as a result of the action. The model execution unit 30 uses the state and reward to obtain inference data from its own trained model. For each classification of the radio environment around the UE, the inference data is sent to the storage unit 20.

[0037] (Step S3) The storage unit 20 stores the inference data transmitted from the model execution unit 30 for each classification of the radio environment around the UE.

[0038] (Step S4) For each classification of the radio environment around the UE, the model learning unit 10 uses the inference data stored in the storage unit 20 to perform model learning (reinforcement learning).

[0039] (Step S5) If the process is to be completed, the process in Fig. 2 is terminated. On the other hand, if the process is to be continued, the process returns to step S1.

[0040] According to this embodiment, model learning is performed for each classification of the radio environment around the UE, thereby improving model learning efficiency. Also, since the same trained model is used for inference for each classification of the radio environment around the UE, inference accuracy is improved. This provides the effect of improving model learning efficiency and inference accuracy using trained models in RAN control.

[0041] [Second embodiment] Fig. 3 is a block diagram showing an example of the configuration of a communication control system according to the second embodiment. In Fig. 3, parts corresponding to those in Fig. 1 are given the same reference numerals, and their explanations will be omitted. In Fig. 3, the communication control system 1a further includes a terminal group organization unit 40 in addition to the communication control system 1 in Fig. 1.

[0042] The terminal group organizing unit 40 organizes UE groups according to the classification of the wireless environment. For example, if the wireless environment around the UE is classified into two categories, "high" and "low," the terminal group organizing unit 40 organizes UE group #1 corresponding to the classification of "high" signal strength of the AP around the UE and UE group #2 corresponding to the classification of "low" signal strength of the AP around the UE.

[0043] The terminal group organizing unit 40 dynamically changes the organization of the UE groups in response to changes in the radio environment around each UE. For example, if the radio environment around the UE is classified into two categories, "high" and "low," the signal strength of the AP around the UE, the terminal group organizing unit 40 dynamically changes the organization of the UE group #1 corresponding to the category of "high" signal strength of the AP around the UE and the UE group #2 corresponding to the category of "low" signal strength of the AP around the UE in response to changes in the signal strength of the AP around each UE.

[0044] The UE groups organized by the terminal group organization unit 40 are used to associate UEs with models for each classification of the wireless environment around the UE. For example, if the classification of the wireless environment around the UE is divided into two categories, "high" and "low," the signal strength of the AP around the UE is associated with the UE group #1 corresponding to the classification of "high" signal strength of the AP around the UE. On the other hand, the model learning unit 10 (Learner #2) corresponding to the classification of "low" signal strength of the AP around the UE is associated with the UE group #2 corresponding to the classification of "low" signal strength of the AP around the UE.

[0045] According to this embodiment, the composition of the UE group is dynamically changed in response to changes in the radio environment around each UE, and therefore, it is possible to adapt to changes in the radio environment around the UE, such as when the UE is moving at high speed.

[0046] [Third embodiment] Fig. 4 is a block diagram showing a configuration example of a communication control system according to the third embodiment. In Fig. 4, parts corresponding to those in Fig. 1 are given the same reference numerals, and their description will be omitted. The communication control system 1b shown in Fig. 4 is a configuration example in which the communication control system 1 in Fig. 1 is applied to a radio access network conforming to the O-RAN specifications.

[0047] In this embodiment, a model is learned by a "Non-RT RIC (Non-Real Time RAN Intelligent Controller)" 100 (first control unit), while an "Near-RT RIC (Near-Real Time RAN Intelligent Controller)" 200 (second control unit) performs inference using the learned model.

[0048] 4, the model learning unit 10 and the storage unit 20 are realized by an application 110 executed by the "Non-RT RIC" 100. Therefore, the model learning unit 10 and the storage unit 20 are realized by the "Non-RT RIC" 100 executing the application 110.

[0049] 4, the model execution unit 30 is realized by an application 210 executed by the "Near-RT RIC" 200. Therefore, the model execution unit 30 is realized by the "Near-RT RIC" 200 executing the application 210.

[0050] The "Non-RT RIC" 100 and the "Near-RT RIC" 200 communicate information using the A1 interface.

[0051] The model execution unit 30 transmits information (actions, states, rewards, etc.) to and from the UE via O-RAN base stations (O-CU (O-RAN Central Unit) and O-DU (O-RAN Distributed Unit)) 300 (300-1, 300-2).

[0052] According to this embodiment, the present invention can be applied to an O-RAN specification radio access network, using a "Non-RT RIC" 100 and a "Near-RT RIC" 200, training a model in the "Non-RT RIC" 100, and performing inference using the trained model in the "Near-RT RIC" 200.

[0053] According to each of the above-described embodiments, it is possible to obtain the effect of improving the learning efficiency of models and the inference accuracy of trained models in controlling RAN.

[0054] This will enable, for example, improvements to the overall quality of services in wireless networks, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), which is to "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0055] Although an embodiment of the present invention has been described in detail above with reference to the drawings, the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present invention.

[0056] In addition, a computer program for realizing the functions of each of the above-described devices may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer system. Note that the "computer system" here may also include hardware such as an OS and peripheral devices. In addition, "computer-readable recording medium" refers to writable non-volatile memory such as a flexible disk, optical magnetic disk, ROM, or flash memory, portable media such as a DVD (Digital Versatile Disc), or a storage device such as a hard disk built into a computer system.

[0057] Furthermore, the term "computer-readable recording medium" also includes those that retain a program for a certain period of time, such as volatile memory (e.g., DRAM (Dynamic Random Access Memory)) within a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program for implementing some of the functions described above, or may be a so-called differential file (differential program) that can implement the functions described above in combination with a program already stored in the computer system. [Explanation of symbols]

[0058] 1...communication control system, 10...model learning unit, 20...storage unit, 30...model execution unit, 40...terminal group organization unit, 100...non-RT RIC, 200...near-RT RIC, 110, 210...application, UE...user terminal

Claims

1. In a radio access network, a model learning unit provided corresponding to each classification of the wireless environment around the user terminal; a model execution unit provided corresponding to each of the user terminals; a storage unit for storing, for each classification of a wireless environment, inference data based on a model of the model execution unit corresponding to a user terminal associated with one classification of the wireless environment; supplying the model trained by one of the model training units to the model execution unit corresponding to the user terminal associated with the classification of the wireless environment corresponding to the one of the model training units; the one of the model learning units uses the inference data of the classification of the wireless environment corresponding to the one of the model learning units to learn a model that infers a predetermined communication control item; Communications control system.

2. a terminal group organizing unit that organizes user terminal groups for each classification of the wireless environment; the terminal group formation unit dynamically changes the formation of the user terminal group in response to changes in the radio environment around each user terminal. The communication control system according to claim 1 .

3. In a radio access network, a model learning unit provided corresponding to each classification of the wireless environment around the user terminal; a model execution unit provided corresponding to each of the user terminals; a storage unit for storing, for each classification of a wireless environment, inference data based on a model of the model execution unit corresponding to a user terminal associated with one classification of the wireless environment; supplying the model trained by one of the model training units to the model execution unit corresponding to the user terminal associated with the classification of the wireless environment corresponding to the one of the model training units; the one of the model learning units uses the inference data of the classification of the wireless environment corresponding to the one of the model learning units to learn a model that infers a predetermined communication control item; Communication control method.

Images