Relay device control device, relay device control method, relay device control program, and wireless communication system

Abstract

This relay device control apparatus is characterized by comprising: a receiving station communication quality collecting unit (5) that collects information about the communication quality of a receiving station; a learning unit (6) that acquires the information collected by the receiving station communication quality collecting unit and calculates a control parameter of a relay device by reinforcement learning for setting, as a reward, a value based on the communication quality of the receiving station; and a relay device control unit (7) that controls the relay device with the control parameter calculated by the learning unit.

Description

[Technical Field]

[0001] The present invention relates to a relay device control device, a relay device control method, a relay device control program, and a wireless communication system. [Background technology]

[0002] In order to handle the increasing mobile traffic in recent years, it is important to improve frequency utilization efficiency. Therefore, attempts have been made to improve signal strength and reduce interference by actively controlling the radio wave propagation environment itself. For example, Non-Patent Document 1 discloses a technology that changes the radio wave propagation environment by combining relay devices such as reflectors. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Riku Omiya and five others, "Wireless Repeater Combination Selection Method for Forming Intelligent Spaces," 2020 IEICE Society Conference Summary of the Invention [Problem to be solved by the invention]

[0004] With conventional technology, it was difficult to dynamically control parameters such as the installation position and reflection angle of relay devices required to create a desired radio wave propagation environment, making it difficult to improve communication quality in real time.

[0005] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a relay device control device, a relay device control method, a relay device control program, and a wireless communication system that can improve communication quality in real time. [Means for solving the problem]

[0006] A relay device control device according to one aspect of the present invention is a relay device control device that controls a relay device that relays a signal transmitted from a transmitting station to a receiving station, the relay device control device comprising: a receiving-station communication quality collection unit that collects information on communication quality of the receiving station; a learning unit that acquires the information collected by the receiving-station communication quality collection unit and calculates control parameters for the relay device through reinforcement learning using a value based on the communication quality of the receiving station as a reward; and a relay device control unit that controls the relay device using the control parameters calculated by the learning unit. The functions of the relay device control device according to one aspect of the present invention can also be realized as a wireless communication system.

[0007] A relay device control method according to another aspect of the present invention is a method for controlling a relay device that relays a signal transmitted from a transmitting station to a receiving station, the method comprising: a receiving station communication quality collection step of collecting information on communication quality of the receiving station; a learning step of calculating control parameters for the relay device by reinforcement learning using a value based on the communication quality of the receiving station as a reward; and a relay device control step of controlling the relay device using the control parameters calculated by the learning step. [Effects of the Invention]

[0008] According to the present invention, it is possible to improve communication quality in real time. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram schematically illustrating a configuration example of a wireless communication system according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the operation of each device constituting the wireless communication system according to the first embodiment. [Figure 3] 2 is a functional block diagram showing main functions of a relay device control device according to the first embodiment. FIG. [Figure 4] 10 is a flowchart showing an example of operation of the relay device control device according to the first embodiment. [Figure 5] FIG. 10 is a functional block diagram illustrating a modified example of the relay device control device according to the first embodiment. [Figure 6] 10 is a flowchart showing a modified example of the relay device control device according to the first embodiment. [Figure 7] 2 is a diagram illustrating an example of a hardware configuration for realizing each function of the wireless communication system and the relay device control device 4 according to the first embodiment. FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described with reference to the accompanying drawings. In this disclosure, overlapping descriptions will be appropriately simplified or omitted. Note that the present disclosure is not limited to the following embodiments. The present disclosure may include various modifications and combinations of the configurations disclosed in the following embodiments without departing from the spirit of the present disclosure.

[0011] Embodiment 1 Fig. 1 is a diagram schematically illustrating a configuration example of a wireless communication system according to embodiment 1. Fig. 2 is a diagram illustrating the operation of each device that configures the wireless communication system according to embodiment 1.

[0012] The wireless communication system according to this embodiment is a system that realizes a good radio wave propagation environment by relaying a signal transmitted from a transmitting station 1 to a receiving station 2 using a relay device 3. The relay device 3 is, for example, a reflector that reflects and relays the radio wave transmitted from the transmitting station 1. The wireless communication system may include multiple relay devices 3.

[0013] The relay device 3 is configured to be able to control parameters such as its installation position and reflection angle. By dynamically controlling the parameters of the relay device 3, it is possible to ensure a good radio wave propagation environment in real time, for example, by following environmental changes such as the movement of obstacles.

[0014] The wireless communication system according to this embodiment includes a relay device control device 4. The relay device control device 4 controls the relay device 3. The relay device control device 4 has a function of dynamically calculating parameters of the relay device 3 to create a desired radio wave propagation environment, and controlling the relay device 3 based on the calculated parameters.

[0015] 2, a transmitting station 1 transmits a signal to a receiving station 2. The receiving station 2 transmits a signal to the transmitting station 1 in response to the received signal. A relay device 3 relays the signal transmitted from the transmitting station 1 to the receiving station 2 and the signal transmitted from the receiving station 2 to the transmitting station 1.

[0016] The signal transmitted from the receiving station 2 to the transmitting station 1 includes information on the communication quality of the receiving station 2. The transmitting station 1 transmits the information on the communication quality of the receiving station 2 included in the signal received from the receiving station 2 to the relay device control device 4. Note that the relay device control device 4 may collect the information on the communication quality of the receiving station 2 from the receiving station 2 or the relay device 3, for example, without going through the transmitting station 1.

[0017] The relay device control device 4 calculates control parameters for the relay device 3 to achieve a target radio wave propagation environment through reinforcement learning using the communication quality of the receiving station 2 as an input value. The relay device control device 4 uses a reinforcement learning algorithm to calculate a value based on the communication quality of the receiving station 2 as a reward and determines control parameters to be tried for the relay device 3. The relay device control device 4 then transmits a control signal to the relay device 3 to try controlling the relay device 3 with the determined control parameters. The relay device 3 that receives the control signal is controlled based on the control parameters determined by the relay device control device 4.

[0018] As shown in FIG. 2 , in the wireless communication system according to this embodiment, the following steps are repeated n times: determining control parameters through reinforcement learning, attempting to control the relay device 3 using the determined control parameters, and determining control parameters through reinforcement learning based on the communication quality of the receiving station 2 after the trial. This allows the parameters of the relay device 3 to be dynamically controlled, thereby improving communication quality in real time. Furthermore, by using reinforcement learning with real-time communication quality as an input value, it is possible to reduce the effort required for parameter determination before building a wireless communication system. For example, there is a method of optically calculating the propagation direction in advance and setting parameters, but this method has the problem that only direct waves can be considered due to time constraints, making it difficult to consider multipath. This embodiment can also solve this problem.

[0019] Examples of communication quality include throughput, received power, RTT (Round Trip Time), CSI (Channel State Information), MCS (Modulation and Coding Sceme), etc. Note that, in addition to communication quality, the input values ​​for reinforcement learning in the relay device control device 4 may also include position information of the receiving station, the moving direction and speed of the receiving station, CSI acquired by another terminal near the receiving station, point cloud data around the transmitting and receiving stations, etc.

[0020] As an example of how to calculate a reward in reinforcement learning in a relay device control device, we will show an example of how to calculate a reward using the measured RTT, which is an example of communication quality, as a variable. As an example, the reward is calculated using equation (1). Equation (1) Reward = {300 (average RTT + minimum RTT + maximum RTT)} / 30 Here, the average RTT, minimum RTT, and maximum RTT refer to the values ​​over the previous n seconds. The above formula (1) uses the average RTT (≦100), minimum RTT (≦100), and maximum RTT (≦100) over the previous n seconds as variables, normalized so that the maximum reward is 100 and the minimum is 0. Using the statistical value over the previous n seconds instead of the instantaneous RTT value enables stable control that suppresses changes in reward due to RTT fluctuations.

[0021] Note that the above formula (1) is just an example. The calculation of the reward in reinforcement learning should be performed so that the reward is large when the communication quality is good, such as when the RTT value is small, and the reward is small when the communication quality is poor, such as when the RTT value is large.

[0022] Furthermore, the relay device 3 and the receiving station 2 do not necessarily have a one-to-one correspondence. One relay device 3 may relay signals to multiple receiving stations 2. In this case, information on the communication quality of the multiple receiving stations 2 is input to the relay device control device 4, and parameters are calculated by reinforcement learning. For example, the populations of the average RTT, maximum RTT, and minimum RTT values ​​in equation (1) are the communication quality of all receiving stations 2.

[0023] 3 is a functional block diagram showing main functions of the relay device control device 4 according to the embodiment 1. The relay device control device according to the embodiment includes a receiving station communication quality collecting unit 5, a learning unit 6, and a relay device control unit 7.

[0024] The receiving station communication quality collecting unit 5 collects information on the communication quality of the receiving station 2. The learning unit 6 acquires the information collected by the receiving station communication quality collecting unit 5 and calculates control parameters for the relay device 3 by reinforcement learning using a value based on the communication quality of the receiving station 2 as a reward. The relay device control unit 7 controls the relay device 3 using the control parameters calculated by the learning unit 6.

[0025] 4 is a flowchart showing an example of the operation of the relay device control device 4 according to the first embodiment. When communication is being performed among the transmitting station 1, the receiving station 2, and the relay device 3, the receiving station communication quality collecting unit 5 collects information on the communication quality of the receiving station 2 (S01). Then, the learning unit 6 acquires the collected information on the communication quality of the receiving station 2 (S02).

[0026] The learning unit 6 uses the acquired communication quality information as an input value to calculate control parameters for the relay device 3 through reinforcement learning to achieve a target radio wave propagation environment (S03). The relay device control unit 7 controls the relay device 3 using the control parameters calculated by the learning unit 6 (S04).

[0027] Steps S01 to S04 are repeated while communication is being performed among the transmitting station 1, the receiving station 2, and the relay device 3. This allows the relay device 3 to be dynamically controlled, and communication quality is improved in real time.

[0028] 5 is a functional block diagram showing a modified example of the relay device control device 4 according to the first embodiment. FIG. 6 is a flowchart showing a modified example of the relay device control device 4 according to the first embodiment.

[0029] The relay device control device 4 may include a receiving station location information collection unit 8 that collects location information of the receiving station 2 and a search range calculation unit 9 that sets a search range for reinforcement learning. Steps S11 and S12 in FIG. 6 are similar to steps S01 and S02 in FIG. 4 , and therefore will not be described here. In this modification, the search range calculation unit 9 calculates the distance and direction to the receiving station 2 from the relay device 3 using the information collected by the receiving station location information collection unit 8, and sets a search range for reinforcement learning in the learning unit 6 (S13). That is, the receiving station location information collection unit 8 narrows the search range for reinforcement learning in the learning unit 6 by performing optical calculations based on the location information of the receiving station 2. The learning unit 6 calculates control parameters for the relay device 3 within the narrowed search range (S14). The relay device control unit 7 controls the relay device 3 using the control parameters calculated by the learning unit 6 (S15). Steps S11 to S15 are repeated while communication is being performed between the transmitting station 1, the receiving station 2, and the relay device 3. According to this modification, the calculation result of the control parameters by reinforcement learning can be made more appropriate depending on the actual position of the receiving station 2.

[0030] The wireless communication system and the relay device control device 4 configured as described above can improve communication quality in real time. Note that the functions of the wireless communication system and the relay device control device 4 can also be realized as a wireless communication method.

[0031] The wireless communication system according to the present embodiment can be realized as a single device such as the relay device control device 4, or can be realized by a plurality of devices working together. Furthermore, each function of the relay device control device 4 may be realized by a plurality of devices working together.

[0032] The functions of the wireless communication system and the relay device control device according to the above-described embodiments and modifications may be partially or entirely realized using hardware such as an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), or an FPGA (Field Programmable Gate Array). The functions of the wireless communication system may be realized by a combination of dedicated hardware and software. Furthermore, the functions of the wireless communication system may be partially or entirely configured as a program executed by a processor such as a CPU. The program may be recorded on a computer-readable storage medium.

[0033] For example, the wireless communication system can be realized using a computer and a program, and the program can be recorded on a storage medium or provided via a network.

[0034] 7 is a diagram illustrating an example of a hardware configuration for realizing each function of the wireless communication system and the relay device control device 4 according to the first embodiment. As illustrated in FIG. 7, each function of the wireless communication system and the relay device control device 4 is realized by, for example, an input unit 100, an output unit 110, a communication unit 120, a CPU 130, a memory 140, and a HDD 150. The input unit 100, the output unit 110, the communication unit 120, the CPU 130, the memory 140, and the HDD 150 are connected via a bus 160 and function as a computer. In addition, the computer configured by the input unit 100, the output unit 110, the communication unit 120, the CPU 130, the memory 140, and the HDD 150 can input and output data to and from a computer-readable storage medium 170.

[0035] The input unit 100 is, for example, a keyboard and a mouse, etc. The output unit 110 is, for example, a display device such as a display, etc. The communication unit 120 is, for example, a wireless network interface.

[0036] The CPU 130 controls each unit constituting the wireless communication system and the relay device control device 4, and performs predetermined processing, etc. The memory 140 and the HDD 150 function as storage units that store various types of data, etc.

[0037] The storage medium 170 stores a program for executing each function of the wireless communication system and the relay device control device 4. Note that the architecture configuring the wireless communication system and the relay device control device 4 is not limited to the example shown in FIG.

[0038] Furthermore, the term "computer" here includes hardware such as the OS and peripheral devices. "Computer-readable storage media" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs.

[0039] Furthermore, the "computer-readable storage medium" may be a medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. Also, the "computer-readable storage medium" may be a medium that stores a program for a certain period of time, such as volatile memory inside a computer that is a server or client. [Industrial Applicability]

[0040] The relay device control device, the relay device control method, the relay device control program, and the wireless communication system according to the present invention can be applied to, for example, a mobile base station that provides wireless communication. [Explanation of symbols]

[0041] 1 transmitting station 2 receiving stations 3. Relay Devices 4. Relay device control device 5. Receiving station communication quality collection unit 6. Learning Department 7 Relay device control section 8. Receiving station location information collection unit 9 Search range calculation section 100 Input section 110 Output section 120 Communications Department 130 CPU 140 memory 150 HDD 160 Bus 170 Storage medium

Claims

1. A relay device control device controls a relay device that relays a signal transmitted from a transmitting station to a receiving station, a receiving station communication quality collection unit that collects information on communication quality of the receiving station; a learning unit that acquires the information collected by the receiving station communication quality collecting unit and calculates a control parameter of the relay device by reinforcement learning using a value based on the communication quality of the receiving station as a reward; a relay device control unit that controls the relay device using the control parameters calculated by the learning unit; A relay device control device comprising:

2. A relay device control device as described in Claim 1, wherein the relay device is a reflector that reflects and relays the radio waves transmitted by the transmitting station.

3. A relay device control device as described in claim 1 or claim 2, wherein the control parameters of the relay device are at least one of an installation position and a reflection angle.

4. A method for controlling a relay device that relays a signal transmitted from a transmitting station to a receiving station, comprising: a receiving station communication quality collection step of collecting information on communication quality of the receiving station; a learning step of calculating a control parameter of the relay device by reinforcement learning using a value based on the communication quality of the receiving station as a reward; a relay device control step of controlling the relay device using the control parameters calculated in the learning step; A method for controlling a relay device, comprising:

5. A method for controlling a relay device as described in claim 1, wherein the relay device is a reflector that reflects and relays the radio waves transmitted by the transmitting station.

6. A relay device control method as described in claim 4 or claim 5, wherein the control parameters of the relay device are at least one of an installation position and a reflection angle.

7. 4. A relay device control program for causing a computer to function as each unit of the relay device control device according to claim 1.

8. In a wireless communication system in which a signal transmitted from a transmitting station to a receiving station is relayed by a relay device, a receiving station communication quality collection unit that collects information on communication quality of the receiving station; a learning unit that acquires the information collected by the receiving station communication quality collecting unit and calculates a control parameter of the relay device by reinforcement learning using a value based on the communication quality of the receiving station as a reward; a relay device control unit that controls the relay device using the control parameters calculated by the learning unit; A wireless communication system comprising:

9. The wireless communication system described in Claim 1, wherein the relay device is a reflector that reflects and relays the radio waves transmitted by the transmitting station.

10. A wireless communication system as described in claim 8 or claim 9, wherein the control parameters of the relay device are at least one of an installation position and a reflection angle.

Images