Learning device and system for learning selection of connection destination base station for mobile device

Abstract

To determine an appropriate base station to which a mobile device should be connected among multiple base stations.SOLUTION: There is provided a learning device that learns the selection of a base station to which a mobile device should be connected within a communication area where the mobile device can connect to multiple base stations with different communication methods. A learning device includes: a mobile body position management unit that manages a position of the mobile device within the communication area; a reward management unit that manages a reward value in reinforcement learning for each of the multiple base stations at each position in the communication area of the mobile device, the reward value representing the reward for selecting the base station as the connection destination at the position; and a learning model generation unit that generates a learning model for selecting the base station to which the mobile device should be connected through reinforcement learning based on the position of the mobile device and the reward value.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a mobile device and a learning device that learns to select a base station to which the mobile device should connect. [Background technology]

[0002] In recent years, various wireless communication networks have been used in parallel. For example, there are networks provided by telecommunications carriers called public 5G (or 4G, LTE) and networks provided by general companies and local governments that are not telecommunications carriers called local 5G (or 4G, LTE), and the communication areas of the latter are provided in spots that overlap with the communication areas of the former. In such overlapping areas, mobile terminals can select and use either network. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] "Issues and Solutions for Interoperability between Private LTE and Public LTE", [online], Internet Initiative Japan, Inc., [Retrieved July 28, 2023], Internet<https: / / www.slideshare.net / IIJ_techlog / iijmio-meeting-26-ltelte> Summary of the Invention [Problem to be solved by the invention]

[0004] However, for example, in an environment such as the one described above, when a mobile terminal moves from a public 5G area to a local 5G area, communication with the public 5G network may continue and communication may not be switched to the local 5G network (see, for example, non-patent document 1). [Means for solving the problem]

[0005] According to one aspect of the present invention, there is provided a learning device that learns to select a base station to which a mobile device should connect in a communication area capable of connecting to multiple base stations with different communication methods, the learning device comprising: a mobile position management unit that manages the position of the mobile device in the communication area; a reward management unit that manages a reward value in reinforcement learning for each of the multiple base stations for each position of the mobile device in the communication area, where the reward value represents a reward for selecting the base station as a connection destination at that position; and a learning model generation unit that generates a learning model for selecting a base station to which the mobile device should connect by reinforcement learning based on the position of the mobile device and the reward value.

[0006] Furthermore, according to one aspect of the present invention, the mobile device position management unit may manage one or more predetermined movement routes in the communication area of ​​the mobile device as the position of the mobile device, and the learning model generation unit may generate the learning model by reinforcement learning based on each position on the one or more movement routes of the mobile device and the reward value corresponding to each position.

[0007] According to one aspect of the present invention, the travel route is a route spanning a first communication area in which it is possible to connect to a base station using a first communication method but not a base station using a second communication method, and a second communication area in which it is possible to connect to both a base station using the first communication method and a base station using the second communication method, and in the second communication area, the reward value for a base station using the second communication method may be set to a higher value than the reward value for a base station using the first communication method.

[0008] In addition, according to one embodiment of the present invention, the first communication method may include a communication method using any of public 5G, public 4G, and public LTE, and the second communication method may include a communication method using any of local 5G, local 4G, local LTE, and wireless LAN.

[0009] According to another aspect of the present invention, the learning model is adapted to convert the position of the mobile device into a state s t and selecting each base station among the plurality of base stations. t The action value Q(s t ,a t ) may be a model configured with a neural network based on deep reinforcement learning.

[0010] In addition, according to one aspect of the present invention, there is provided a mobile device equipped with the learning model generated by the learning device, the mobile device comprising: a position acquisition unit that acquires a current position of the mobile device; a base station selection unit that uses the learning model to select one base station from among a plurality of base stations in the communication area based on the acquired current position of the mobile device; and a communication unit that communicates with the base station selected by the base station selection unit. Effect of the Invention

[0011] According to the present invention, it is possible to determine an appropriate base station to which a mobile device should connect from among a plurality of base stations. [Brief description of the drawings]

[0012] [Figure 1] 1 is a diagram showing a schematic configuration of a system to which a learning device according to an embodiment of the present invention is applied. [Diagram 2] 1 is a block diagram showing a functional configuration of a learning device according to an embodiment of the present invention. [Diagram 3] 1 is an example of a management table for a mobile device position management unit to manage the positions of mobile devices and for a reward management unit to manage reward values ​​for learning. [Figure 4] FIG. 2 is a schematic diagram showing state transitions of a mobile device. [Diagram 5] FIG. 1 is a diagram showing an example of a learning model (DQN). [Figure 6] FIG. 13 is a diagram showing another example of a learning model (Fixed Target Q-Network). [Figure 7] FIG. 11 is a diagram showing another example of a movement path. [Figure 8] 1 is a block diagram showing a functional configuration of a mobile device according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0014] 1 is a diagram showing a schematic configuration of a system to which a learning device according to an embodiment of the present invention is applied. The system 10 includes a learning device 100, a mobile device 200, and a plurality of base stations 300 (300A, 300B). The mobile device 200 can perform wireless communication with the plurality of base stations 300. Each base station 300 forms a wireless communication area (hereinafter, communication area) that enables the mobile device 200 to wirelessly communicate with the base station 300.

[0015] In the example of FIG. 1, the communication area includes a first communication area CA1 and a second communication area CA2 contained within the first communication area CA1. The first communication area CA1 is an area in which the mobile device 200 can communicate with the base station 300A but cannot communicate with the base station 300B. The second communication area CA2 is an area in which the mobile device 200 can communicate with both the base station 300A and the base station 300B. In other words, the base station 300A forms a communication area that spans one large area that includes both the first communication area CA1 and the second communication area CA2. Meanwhile, the base station 300B forms a smaller communication area (the second communication area CA2) within the communication area formed by the base station 300A.

[0016] For example, when the system 10 in Fig. 1 is used together with a 5G (fifth generation), 4G (fourth generation), and / or LTE (Long Term Evolution) mobile communication system, the communication area formed by the base station 300A (i.e., the entire area including the first communication area CA1 and the second communication area CA2) may be a public 5G area, a public 4G area, or a public LTE area, and the communication area formed by the base station 300B (the second communication area CA2) may be a local 5G area, a local 4G area, or a local LTE area. The communication area formed by the base station 300B (the second communication area CA2) is not limited to a mobile communication area, and may be a communication area capable of communication by wireless LAN such as WiFi. A public 5G (4G, LTE) area is a communication area with a wide coverage area that is built and managed by a telecommunications carrier that provides mobile communication services (mobile phones and data communications), while a local 5G (4G, LTE) area is a communication area with a relatively narrow coverage area that is built and managed by general companies or local governments that are not telecommunications carriers.

[0017] For example, the mobile device 200 can communicate with the base station 300A in the public 5G area (i.e., the first communication area CA1 and the second communication area CA2) using a first communication method (e.g., a 5G mobile communication method using a first frequency band), and can communicate with the base station 300B in the local 5G area (i.e., the second communication area CA2) using a second communication method different from the first communication method (e.g., a 5G mobile communication method using a second frequency band different from the first frequency band). Also, for example, when the public 5G area and the communication area by wireless LAN coexist, the mobile device 200 can communicate with the base station 300B in the wireless LAN communication area (i.e., the second communication area CA2) using a second communication method (wireless LAN communication method) different from the first communication method (5G mobile communication method) used in the public 5G area. Therefore, in the second communication area CA2, the mobile device 200 selects whether to communicate with the base station 300A conforming to the first communication method via public 5G, or to communicate with the base station 300B conforming to the second communication method via local 5G (or, for example, wireless LAN), and communicates with either the base station 300A or the base station 300B based on the selection.

[0018] In addition, public 5G (4G, LTE) and local 5G (4G, LTE) usually use different frequency bands even if they are the same generation of communication standards. In this specification, the difference in frequency bands is considered to be a concept included in the difference in communication method.

[0019] It should be noted that the arrangement of the communication areas shown in Fig. 1 is merely an example for explanation and should not be construed as limiting. For example, the size of the second communication area CA2 may be much larger than that shown in Fig. 1, or multiple second communication areas CA2 may exist within the first communication area CA1. Also, although only one base station 300A forming a public area (e.g., a public 5G area) is shown in Fig. 1, the public area may be formed by multiple base stations 300A, and the mobile device 200 may communicate with one of the base stations 300A depending on the location within the public area. The same applies to a local area (e.g., a local 5G area).

[0020] The mobile device 200 may be, for example, a remotely controlled or autonomous air, land, or sea mobile machine such as a drone, or a mobile vehicle (such as an automobile) including an autonomous vehicle, or an information terminal such as a smartphone carried and moved by a human being, or any other type of device capable of communicating with the base station 300 and capable of moving within a communication area.

[0021] As shown in FIG. 1, the mobile device 200 moves along a certain moving path MP in a communication area. The moving path MP includes three sections, for example, a section #1 that first passes through a first communication area CA1, a section #2 that passes through a second communication area CA2 after the section #1, and a section #3 that passes through the first communication area CA1 again after the section #2. In this example, for example, it may be desirable or required for the mobile device 200 to communicate by local 5G (or wireless LAN) rather than public 5G in the section #2 while moving along the moving path MP. The system 10 according to the present embodiment includes a learning device 100 that learns which communication method (for example, public 5G / local 5G) base station 300A or 300B should be selected as a communication connection destination by the mobile device 200 moving on such a moving path MP.

[0022] 2 is a block diagram showing a functional configuration of a learning device 100 according to an embodiment of the present invention. The learning device 100 includes a mobile object position management unit 110, a reward management unit 120, a learning model generation unit 130, and a learning model 140 generated by the learning model generation unit 130. The learning device 100 can be realized by a computer including a processor and a memory.

[0023] The mobile object position management unit 110 of the learning device 100 is configured to manage the position of the mobile object device 200 in the communication area. The position of the mobile object device 200 is each position on a moving path MP as shown in FIG. 1. The moving path MP may be, for example, a route set in advance. Each position on the moving path MP can be represented by three-dimensional position coordinates (x, y, z). The mobile object position management unit 110 manages (for example, stores in a storage unit such as a database) the three-dimensional position coordinates of each position on one or more moving paths MP set in advance. The one or more moving paths MP set in advance can be appropriately set as a route for executing learning in the learning device 100.

[0024] The reward management unit 120 of the learning device 100 is configured to manage a reward value used in learning (reinforcement learning) of the base station 300 to be connected to the mobile device 200. Specifically, the reward management unit 120 manages a reward value for each of the multiple base stations 300 for each position of the mobile device 200 in the communication area when the base station 300 is selected as the connection destination. For example, in the example of FIG. 1 described above in which the communication area is formed by the base stations 300A and 300B, the reward management unit 120 manages (for example, stores in a storage unit such as a database) a reward value when the base station 300A is the connection destination and a reward value when the base station 300B is the connection destination for each position in the communication area, for example, each position on the movement route MP.

[0025] The reward value is set so that the reward value for the base station 300 that the mobile device 200 desires to select as a connection destination is greater than the reward value for the base station 300 that is not. For example, in the example described above with reference to FIG. 1, that is, in an example where communication by local 5G (or wireless LAN) is desired in section #2 in the movement route MP, the reward value for selecting the base station 300B corresponding to local 5G (or wireless LAN) in section #2 is set to a value (e.g., "2") greater than the reward value (e.g., "1") for selecting the base station 300A corresponding to public 5G in section #2. Such a reward value setting can be set in advance in consideration of an appropriate design guideline as to which base station of the multiple base stations 300 the mobile device 200 desires to connect or should connect to depending on its position (position on the movement route MP).

[0026] 3 is an example of a management table referenced by the mobile object location management unit 110 and the reward management unit 120 so that the mobile object location management unit 110 manages the location of the mobile object device 200 and the reward management unit 120 manages the reward value for learning. In this example, the location P i1 ~P i2 represents each position within section #1 on the moving path MP illustrated in FIG. 1, and position P i3 ~P i4 represents each position in section #2 on the moving route MP, and position P i5 ~P i6 represents each position within section #3 on the moving route MP. In other words, this management table stores the position P i1 , …, P i2 , …, P i3 , …, P i4 , …, P i5 , …, P i6 The setting is that the items move in the order of

[0027] In addition, in this example of the management table, in sections #1 and #3 on the moving route MP, the mobile device 200 can only communicate with the base station 300A, so the reward value for the base station 300A is set to "1" and the reward value for the base station 300B is set to "0". On the other hand, in section #2 on the moving route MP, where communication with both the base stations 300A and 300B is possible, the reward value for the base station 300A is set to "1" and the reward value for the base station 300B is set to "2". That is, in section #2 on the moving route MP, a higher reward value is set for the base station 300B (e.g., local 5G) than for the base station 300A (e.g., public 5G). Therefore, by performing learning regarding the selection of the base station 300 using a management table in which such reward values ​​are set, it becomes possible to determine one appropriate base station 300 to which the mobile device 200 should connect from among the multiple base stations 300.

[0028] 4 is a schematic diagram showing state transitions of the mobile device 200. This state transition diagram is managed (stored) by, for example, the learning model generation unit 130 of the learning device 100, and is used to generate the learning model 140. The state of the mobile device 200 changes from state s1 to state s2, from state s2 to state s3, ..., state s t From state s t+1 In this embodiment, the "state" of the mobile device 200 is the location of the mobile device 200 in the communication area. For example, the state of the mobile device 200 is the location (P i1 In each state (i.e., each position), the mobile device 200 sequentially transitions according to N actions a1, a2, ..., a N For example, the mobile device 200 may be in state s t-1 In the new state s t Then, it transitions to state s t In the state s t+1The mobile device 200 can transition to the next "action". Each "action" that the mobile device 200 can take includes designating one of the multiple base stations 300 as a connection destination candidate. For example, the action a1 of the mobile device 200 may be designating the base station 300A corresponding to public 5G as a connection destination candidate, and the action a2 of the mobile device 200 may be designating the base station 300B corresponding to local 5G (or wireless LAN) as a connection destination candidate. The number N of actions that the mobile device 200 can take is equal to the total number of base stations 300 with which the mobile device 200 can communicate within the communication area (for example, N=2 in the example of FIG. 1).

[0029] Returning to FIG. 2, the learning model generation unit 130 of the learning device 100 is configured to generate a learning model 140 for determining a base station 300 with which the mobile device 200 will actually communicate, by reinforcement learning based on the state (position) of the mobile device 200 and a reward value for each of the multiple base stations 300. For example, as the learning model 140, as shown in FIG. 5, t is input to the input layer, and the state s t All possible actions that the mobile device 200 can take in (selection of one base station 300 from multiple base stations 300) t The action value Q(s t ,a t ) from each node of the output layer. Alternatively, as shown in FIG. 6, the learning model 140 may be based on a Fixed Target Q-Network that uses two neural networks (main network and target network) with the same configuration as in FIG. 5. In the DQN in FIG. 5 and the Fixed Target Q-Network in FIG. 6, the action value Q(s t ,a t ) and the reward value provided by the reward management unit 120 to update the parameters (weights and thresholds) of the neural network, thereby training the learning model 140 (i.e., generating the learning model 140).

[0030] In addition, well-known methods can be applied as a method for learning neural networks in DQN and Fixed Target Q-Network. Therefore, in this specification, a description of the specific learning procedure is omitted.

[0031] In the above explanation, it is assumed that only one movement route MP is set in the communication area as shown in FIG. 1, but multiple movement routes may be set as shown in FIG. 7, and the neural network may be trained for each case in which the mobile device 200 moves along each of these movement routes.

[0032] 8 is a block diagram showing a functional configuration of a mobile device 200 according to an embodiment of the present invention. The mobile device 200 includes at least a position acquisition unit 210, a base station selection unit 220, a learning model 230, and a communication unit 240. The mobile device 200 may include functional units related to other functions (for example, a drive unit such as an engine or a motor for movement), but a description of these will be omitted here.

[0033] The position acquisition unit 210 of the mobile device 200 is configured to acquire a current position of the mobile device 200 moving within a communication area. For example, the position acquisition unit 210 may be configured to receive a GPS signal and acquire current GPS position information of the mobile device 200 from the received GPS signal.

[0034] The base station selection unit 220 of the mobile device 200 is configured to use the learning model 230 to select one base station 300 from among the multiple base stations 300 in the communication area based on the current location of the mobile device 200. The learning model 140 generated (i.e., trained) by the learning device 100 is installed in the mobile device 200 as the learning model 230. The current location of the mobile device 200 is input to an input layer of the learning model 230, and the learning model 230 calculates each action value Q(s t ,at ) from each node of the output layer. The base station selection unit 220 outputs the action value Q(s t ,a t ), the base station 300 corresponding to the largest action value is selected as the base station to which the mobile device 200 should actually connect.

[0035] The communication unit 240 of the mobile device 200 communicates with the base station 300 thus selected by the base station selection unit 220. This allows the mobile device 200 to communicate with an appropriate base station 300 according to its current location.

[0036] Although the embodiment of the present invention has been described above, the present invention is not limited to this, and various modifications are possible without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0037] 10. System 100 Learning Device 110 Mobile object position management section 120 Compensation Management Department 130 Learning model generation unit 140 Learning Model 200 Mobile device 210 Position acquisition part 220 Base station selection unit 230 Learning Model 240 Communications Department 300A, B base station CA1 First communication area CA2 2nd communication area MP Movement Path

Claims

1. A learning device that learns selection of a base station to which a mobile device should connect in a communication area in which the mobile device can connect to a plurality of base stations having different communication methods, comprising: a mobile location management unit that manages the location of the mobile device within the communication area; a reward management unit that manages a reward value in reinforcement learning for each of the plurality of base stations for each of the positions in the communication area of ​​the mobile device, the reward value representing a reward for selecting the base station as a connection destination at the position; a learning model generation unit that generates a learning model for selecting a base station to which the mobile device should connect by reinforcement learning based on the position of the mobile device and the reward value; A learning device comprising:

2. the mobile object location management unit manages one or more preset movement routes in the communication area of ​​the mobile object device as the location of the mobile object device; the learning model generation unit generates the learning model by reinforcement learning based on each position on the one or more movement routes of the mobile device and the reward value corresponding to each position; The learning device according to claim 1 .

3. the movement route is a route spanning a first communication area in which a base station according to a first communication method can be connected but a base station according to a second communication method cannot be connected, and a second communication area in which a base station according to both the first communication method and the second communication method can be connected, In the second communication area, the reward value for the base station according to the second communication method is set to a value higher than the reward value for the base station according to the first communication method. The learning device according to claim 2 .

4. The learning device of claim 3, wherein the first communication method includes a communication method using any one of public 5G, public 4G, and public LTE, and the second communication method includes a communication method using any one of local 5G, local 4G, local LTE, and wireless LAN.

5. The learned model converts the position of the mobile device into a state s ｔ and selecting each base station among the plurality of base stations. ｔ The action value Q (s ｔ , a ｔ 2. The learning device according to claim 1, wherein the model is configured by a neural network based on deep reinforcement learning, and outputs a training result based on the training result.

6. A mobile device equipped with the learning model generated by the learning device according to any one of claims 1 to 5, a location acquisition unit for acquiring a current location of the mobile device; a base station selection unit that uses the learning model to select one base station from among a plurality of base stations in the communication area based on the acquired current location of the mobile device; a communication unit that communicates with the base station selected by the base station selection unit; A mobile device comprising:

Images