Apparatus and method for transferring data in a manner appropriate to the server status.

Abstract

The purpose of the present disclosure is to adjust a server load and network efficiency by selecting a forwarding method on the basis of a server status.　The present disclosure is a device provided to a communication system that establishes a communication path between a data source and a server and uses the established communication path to transmit data, said device: determining whether to establish the communication path on the basis of the server status; allowing the data source to transmit data using a communication method that establishes the communication path upon determining that the communication path is to be established; and allowing the data source to transmit data using a communication method that does not establish the communication path upon determining that the communication path is not to be established.

Description

Technical Field

[0001] This disclosure relates to communication control for data collection.

Background Art

[0002] With the spread of IoT (Internet of Things) and remote work, communication traffic has been increasing year by year. When transferring a large amount of data generated from various sensors to a server, in the conventional TCP (Transmission Control Protocol) method, the communication processing on the receiving side may become a bottleneck. If the CPU (Central Processing Unit) load of the server increases due to the receiving process, there is a possibility of a decrease in receiving processing performance and an increase in power consumption.

[0003] Since RDMA (Remote Direct Memory Access) does not perform memory copying, it has the effect of reducing the CPU load of the server, which was a bottleneck in the conventional TCP method. However, since a lossless communication network is required to perform data transfer using RDMA, it cannot be directly used in the current wide-area network.

[0004] On the other hand, as methods of using RDMA in a network where packet loss occurs, a method of using PFC (Priority Flow Control) and a method of providing a retransmission control mechanism in the server have been considered, but there are problems such as the possibility of deadlock and an increase in server load.

[0005] In order to realize data transfer by RDMA in a wide-area network, a communication system has been proposed in which a communication path is established between a data source and a server, and the data source and the server perform RDMA communication using the established communication path (for example, Non-Patent Document 1).

Prior Art Documents

Non-Patent Documents

[0006] [Non-Patent Document 1] Ryohei Tsugami, Tatsuya Fukui, "Implementation Evaluation of a Communication Path Utilization Control Method for Wide-Area Multi-Terminal RDMA Data Acquisition," Proceedings of the 2022 IEICE General Conference, 2B-6-23. [Overview of the project] [Problems that the invention aims to solve]

[0007] Sending all data using the path-cooperative RDMA method results in poorer network efficiency compared to the TCP method, while sending all data using the TCP method leads to high server load. Therefore, this disclosure aims to adjust server load and network efficiency by selecting the transfer method based on the server status. [Means for solving the problem]

[0008] This disclosure relates to a communication system that transmits data from a data source to a server, A first communication method for establishing a communication path from the data source to the server, A second communication method that does not establish a communication path from the data source to the server, Any of these can be adopted. Based on the status of the server, the first communication method and the second communication method are used interchangeably.

[0009] The first communication method described above is, for example, a path-cooperative RDMA method. The second communication method is, for example, the TCP method.

[0010] The apparatus of this disclosure is Based on the status of the aforementioned server, it is determined whether or not to establish the communication path. When it is determined that the communication path has been established, the data source is instructed to transmit data using the communication method that established the communication path. If it is determined that the aforementioned communication path cannot be established, the data source is instructed to send data using a communication method that does not establish the aforementioned communication path.

[0011] The device of this disclosure is any device provided in the communication system, for example, the data source, or a path controller that manages the communication path connecting the data source and the server.

[0012] The device disclosed herein functions as a path controller that controls the communication path from a data source to a server, Based on the status of the aforementioned server, it is determined whether or not to establish the communication path. When it is determined that the communication path has been established, the communication path has been established, If it is determined that the aforementioned communication path cannot be established, the data source is notified of the communication method that does not establish the aforementioned communication path.

[0013] The status of the server is the CPU load of the server, When the CPU load of the server exceeds a threshold, it is determined that the communication path is established. If the CPU load of the server is below a threshold, it may be determined that the communication path will not be established.

[0014] The status of the server is the memory load of the server, When the memory load of the aforementioned server exceeds a threshold, it is determined that the communication path will not be established. The server may determine to establish the communication path when the server's memory load is below a threshold.

[0015] The device described herein functions as a data source that transmits data to a server, (i) A first communication method for establishing a communication path from the data source to the server, (ii) A second communication method that does not establish the aforementioned communication path, It is possible to send data using both, Based on the status of the server, at least one of the first communication method and the second communication method is used.

[0016] The device of the present disclosure is a device that functions as a server for receiving data from a data source, (i) A first communication method for establishing a communication path from the data source to the server, (ii) A second communication method that does not establish the communication path, and can receive data using both of them, Based on the status of the server, at least one of the first communication method and the second communication method is used.

[0017] The method of the present disclosure may be a method executed by the communication system of the present disclosure. Also, the method of the present disclosure may be a method executed by the device of the present disclosure. The device of the present disclosure is any device provided in the communication system of the present disclosure, including a data source, a server, and a path controller.

[0018] The program of the present disclosure is a program for realizing a computer as the device of the present disclosure, and is a program for realizing the method of the present disclosure on a computer.

[0019] In addition, the above disclosures can be combined as much as possible.

Advantages of the Invention

[0020] According to the present disclosure, by selecting a transfer method based on the status of the server, the server load and network efficiency can be adjusted.

Brief Description of the Drawings

[0021] [Figure 1] An example of the system configuration of the present disclosure is shown. [Figure 2] An example of the configuration of a communication network is shown. [Figure 3] An example of the configuration of a data source is shown. [Figure 4]An example server configuration is shown. [Figure 5] An example of a path controller configuration is shown. [Figure 6] An example of a path setting location table is shown below. [Figure 7] An example of a sensing data transmission sequence is shown. [Figure 8] Examples of data transfer methods selected in this disclosure are shown. [Figure 9] Examples of data transfer methods selected in this disclosure are shown. [Figure 10] Examples of data transfer methods selected in this disclosure are shown. [Modes for carrying out the invention]

[0022] Embodiments of this disclosure will be described in detail below with reference to the drawings. However, this disclosure is not limited to the embodiments shown below. These examples are illustrative, and this disclosure can be implemented in various modified and improved forms based on the knowledge of those skilled in the art. In this specification and in the drawings, components with the same reference numerals refer to the same components.

[0023] (Path-linked RDMA method) Figure 1 shows an example of the system configuration of this disclosure. The system of this disclosure establishes a communication path between a data source 10 and a server 40, and the data source 10 and server 40 perform RDMA communication using the established communication path. In this embodiment, an example of the system of this disclosure is shown that includes a path controller 30.

[0024] The apparatus of this disclosure, including the data source 10, server 40, and path controller 30, can also be implemented by a computer and a program, and the program can be recorded on a recording medium or provided through a communication network.

[0025] Figure 2 shows an example of the configuration of the communication network 80. The communication network 80 is composed of multiple network devices 81. In this embodiment, the network devices 81 used between the data source 10 and the server 40 are predetermined. For example, the data source 10A and the server 40 are connected by network devices 81A and 81B, the data source 10B and the server 40 are connected by network devices 81A and 81C, and the data source 10C and the server 40 are connected by network devices 81D and 81E.

[0026] Each network device 81A to 81E knows which network device to forward the data to when it receives data from data sources 10A to 10C destined for server 40. Therefore, even if data sources 10A to 10C do not know which network device 81 to use, they can send the data to server 40 by specifying it as the server. Thus, this disclosure enables lossless and high-bandwidth data transfer when a communication path is established.

[0027] Figure 3 shows an example configuration of the data source 10. The data source 10 is a device that holds sensing data from the user. The data source 10 includes, for example, a data transmission / reception unit 11, an arithmetic processing unit 12, and a storage unit 13. The arithmetic processing unit 12 acquires sensing data from any sensor 50 and stores it in the storage unit 13. The data transmission / reception unit 11 transmits the sensing data to the server 40.

[0028] The data transmission / reception unit 11 transmits and receives arbitrary information with the server 40, as well as with the path controller 30. The communication method, such as the communication network and communication standard, may be the same for the server 40 and the path controller 30, but may also be different.

[0029] The storage unit 13 may store a transmission policy that defines the timing for transmitting sensing data. In this case, the data transmission / reception unit 11 transmits the sensing data according to the transmission policy.

[0030] Figure 4 shows an example of the configuration of the server 40. The server 40 is a device that collects various data, including sensing data, from the data source 10. The server 40 includes, for example, a data transmission / reception unit 41, an arithmetic processing unit 42, and a storage unit 43. The data transmission / reception unit 41 receives sensing data from any data source 10. The arithmetic processing unit 42 acquires sensing data from any data source 10 and stores it in the storage unit 43.

[0031] The data transmission / reception unit 41 transmits and receives arbitrary information with the data source 10, as well as with the path controller 30. The communication method, such as the communication network and communication standard, may be the same for the data source 10 and the path controller 30, but may also be different.

[0032] Figure 5 shows an example configuration of the path controller 30. The path controller 30 is a device that controls communication paths and can establish a communication path from the data source 10 to the server 40. The path controller 30 includes, for example, a path setting control unit 31, a path setting unit 32, and a path setting location table 33. The path setting location table 33 stores information on network devices 81 that transfer sensing data from each data source 10. Based on the path setting location table 33, the path setting control unit 31 determines which network devices 81 to be used for the communication path connecting the data source 10 and the server 40, and controls the settings of the network devices 81. The path setting unit 32 sets the network devices 81 according to instructions from the path setting control unit 31. As a result, a communication path connecting the data source 10 and the server 40 is established.

[0033] Figure 6 shows an example of the path setting location table 33. In the case of data source 10A shown in Figure 2, the network device 81 information includes identification information for network devices 81A and 81B that connect data source 10A and server 40, and communication path setting information (establishment / release) for network devices 81A and 81B.

[0034] The path setting control unit 31 transmits and receives arbitrary information with the data source 10, and also transmits and receives arbitrary information with the server 40. The communication method, such as the communication network and communication standard, may be the same for the data source 10 and the server 40, but may also be different.

[0035] Figure 7 shows an example of the transmission sequence of sensing data when using the path-cooperative RDMA method. When sensing data is generated, data source 10B sends a request to the path controller 30 to establish a communication path (S101). When the path controller 30 receives a request to establish a communication path, it configures the communication paths for network devices 81A and 81C and establishes the communication path (S102). Once the establishment of the communication path is complete, the path controller 30 sends a notification of completion of communication path establishment to the data source 10B (S103). When data source 10B receives notification that the communication path has been established, it sends sensing data to server 40 (S104). Once data source 10B has finished sending sensing data to server 40, it sends a request to the path controller 30 to release the communication path (S105). When the path controller 30 receives a request to release the communication path, it releases the communication paths of the network devices 81A and 81C and sends a communication path release completion notification to the data source 10B indicating that the release of the communication path has been completed (S106).

[0036] The sequence shown in Figure 7 is an example of a transmission policy in which a communication path is established and data is transferred immediately after data generation. However, there are other transmission policies, such as one that transfers data after a certain amount of data has accumulated, or one that transfers data at a predetermined time. This disclosure is applicable to any transmission policy.

[0037] Using the path-cooperative RDMA method for data transfer from data source 10 to server 40 reduces the load on server 40's receiving processing, but it worsens the network efficiency of communication network 80. On the other hand, if the TCP method is used for data transfer from data source 10 to server 40, the network efficiency of the communication network 80 does not deteriorate, but the CPU load on server 40 increases due to the receiving processing.

[0038] Therefore, in this disclosure, the data source 10 is made capable of transmitting data using both path-linked RDMA (first communication method) and TCP (second communication method), and the server 40 is made capable of receiving data using both path-linked RDMA (first communication method) and TCP (second communication method), and the method of transferring data from the data source 10 to the server 40 is used selectively based on the status of the server 40.

[0039] Specifically, in this disclosure, the path controller 30 establishes a communication path between the data source 10 and the server 40. Therefore, in this disclosure, the path controller 30 acquires the status of the server 40 and selects a data transfer method based on the status of the server 40. This will be explained in detail below.

[0040] (First Embodiment) Figure 8 shows an example of the data transfer method selection in this embodiment. In this embodiment, the status of server 40 is the CPU load of server 40, and the path-cooperative RDMA method is selected when the CPU load of server 40 is high.

[0041] The path controller 30 obtains the CPU usage rate from the server 40 (S111). The timing of obtaining the CPU usage rate from the server 40 is arbitrary, but it can be done periodically, for example.

[0042] The path controller 30 determines whether the CPU usage is above a threshold (S112). If the CPU usage exceeds a threshold (Yes in S112), the path controller 30 selects the path-cooperative RDMA method and notifies the data source 10 that it will use the path-cooperative RDMA method (S113). This causes the data source 10 to wait for the communication path to be established. If the CPU usage is below the threshold (No in S112), the path controller 30 selects the TCP method and notifies the data source 10 that the TCP method will be used (S114).

[0043] Each data source 10 uses the notified method when it sends data to the server 40. Specifically, if the data source 10 is notified of the path-cooperative RDMA method, it will send data from the data source 10 to the server 40 using the path-cooperative RDMA method. If the data source 10 is notified of the TCP method, it will send data from the data source 10 to the server 40 using the TCP method.

[0044] The threshold in step S112 can be determined based on the CPU load of server 40. For example, the threshold is a predetermined percentage of the total CPU available for communication processing. For instance, if the CPU load (usage) for data collection communication exceeds 70% of the total CPU available for communication processing, a path-cooperative RDMA method with a lower CPU load can be used so as not to exceed 100%.

[0045] Furthermore, while CPU usage was used as the threshold in this embodiment, the disclosure is not limited thereto. For example, the threshold in this embodiment can be any information that can determine the degree of CPU load.

[0046] Thus, this embodiment uses either the path-cooperative RDMA method or the TCP method based on the CPU usage rate on the server 40. This allows the embodiment to adjust the balance between server load and network load.

[0047] In this embodiment, the timing at which the path controller 30 executes step S112 is arbitrary and can, for example, be performed periodically. In this case, the data source 10 periodically acquires information on which communication method to use to transmit data, and when transmitting according to its own transmission policy, it transmits the data using the communication method determined by the last acquired information.

[0048] The timing at which the path controller 30 executes step S112 may be when the communication path from the data source 10 to the server 40 is established. For example, when the communication path establishment request shown in step S101 of Figure 7 is received. In this case, the path controller 30 can establish the communication path when executing step S114.

[0049] Furthermore, the path controller 30 may have data transmission information from the data source 10 in advance, such as when the time for sending data from the data source 10 to the server 40 is predetermined. In this case, the path controller 30 may execute steps S111 and S112 at predetermined times.

[0050] (Second embodiment) Figure 9 shows an example of data transfer method selection in this embodiment. In this embodiment, the status of server 40 is the memory load of server 40, and the path-cooperative RDMA method is selected within the range determined by the memory load of server 40.

[0051] The path controller 30 obtains the memory usage rate from the server 40 (S121). The timing of obtaining the memory usage rate from the server 40 is arbitrary, but it can be done periodically, for example.

[0052] The path controller 30 determines whether the memory usage rate is above a threshold (S122). If the memory usage is below the threshold (No in S122), the path-cooperative RDMA method is selected, and the data source 10 is notified that the path-cooperative RDMA method will be used (S123). This causes the data source 10 to wait for the establishment of a communication path. If the memory usage exceeds a threshold (Yes in S122), the TCP method is selected, and the data source 10 is notified that the TCP method will be used (S124).

[0053] Each data source 10 uses the notified method when it transmits data to the server 40, similar to the first embodiment.

[0054] Here, the threshold in step S122 can be determined based on the memory load of the server 40. For example, the threshold is a predetermined percentage of the total memory available for communication processing. For example, if the memory load (usage) for data collection exceeds 70% of the amount available for communication processing, the TCP method is used, which does not require fixed memory allocation for each communication flow.

[0055] Furthermore, while memory usage was used as the threshold in this embodiment, the disclosure is not limited thereto. For example, the threshold in this embodiment can be any information that can determine the degree of memory load.

[0056] Thus, in this embodiment, when there is a possibility of affecting various applications on the server 40, the TCP method is used to prevent impact on those applications. Otherwise, the CPU load on the server 40 can be reduced by transmitting data using the path-cooperative RDMA method to the extent that it does not affect various applications on the server 40.

[0057] The timing of when the path controller 30 executes step S122, and the timing of when the path controller 30 obtains the CPU usage rate of the server 40 in step S121, are the same as in the first embodiment.

[0058] (Third embodiment) Figure 10 shows an example of data transfer method selection in this embodiment. In this embodiment, the status of server 40 is both CPU load and memory load, and path-cooperative RDMA and TCP methods are used depending on both loads.

[0059] The controller 30 obtains the CPU usage and memory usage of the server 40 (S131) ​​and determines whether the CPU usage is above a threshold (S132) or whether the memory usage is above a threshold (S133). If CPU usage is above the threshold and memory usage is below the threshold, the path-cooperative RDMA method is selected and the system waits for path assignment (S134). If CPU usage is below a threshold, or memory usage is above a threshold, the TCP method is selected and the data source is notified (S135).

[0060] Here, the threshold value in step S132 is the same as in the first embodiment, and the threshold value in step S133 is the same as in the second embodiment.

[0061] Thus, this embodiment can maintain load balancing on server 40 by switching between path-cooperative RDMA and TCP methods based on both the CPU load and memory load of server 40.

[0062] (Fourth embodiment) This disclosure may use both path-cooperative RDMA and TCP methods. In this case, the path controller 30 adjusts the ratio of path-cooperative RDMA to TCP methods based on at least one of the CPU load or memory load of the server 40.

[0063] For example, the controller 30 can mix data sources using the path-linked RDMA method (data source A) and data sources using the TCP method (data source B) by changing the frequency of decisions made using a flowchart similar to that of the first and second embodiments.

[0064] (Other embodiments) The above-described embodiment shows an example in which the path controller 30 selects a data transfer method, but the disclosure is not limited thereto. For example, the data source 10 may acquire the server load and select a data transfer method based on it. In this case, the data source 10 may send a request to the controller 30 to establish a communication path only when it has selected the path cooperation RDMA method.

[0065] Furthermore, while this embodiment demonstrates an example of using both path-cooperative RDMA and TCP, this disclosure is not limited thereto. For example, any communication method that establishes a communication path may be adopted instead of or in conjunction with path-cooperative RDMA. Also, any communication method that does not establish a communication path may be adopted instead of or in conjunction with TCP.

[0066] (Effects of this disclosure) As described above, this disclosure selects a transfer method based on the status of server 40, such as the memory load and CPU load of server 40. This allows the disclosure to adjust server load and network efficiency.

[0067] Furthermore, the path controller 30 may use either the path-cooperative RDMA method or the TCP method depending on the status of the communication network 80, in addition to the status of the server 40. For example, the path controller 30 may determine the delay time or the load on the communication network 80 using the path-cooperative RDMA method. [Explanation of Symbols]

[0068] 10, 10A, 10B, 10C: Data source 11, 41: Data transmission / reception unit 12, 42: Arithmetic Processing Unit 13, 43: Storage section 30: Path Controller 31: Path Setting Control Unit 32: Path setting section 33: Path setting location table 40: Server 50: Sensor 80: Communication Network 81A, 81B, 81C, 81D, 81E: Network equipment

Claims

1. A device in a communication system that establishes a communication path between a data source and a server, and transmits data using the established communication path, Based on the status of the aforementioned server, it is determined whether or not to establish the communication path. When it is determined that the aforementioned communication path has been established, the server is instructed to transmit data to the data source using a communication method that establishes the aforementioned communication path, which results in a relatively low processing load on the server and low network efficiency. If it is determined that the aforementioned communication path cannot be established, the server's processing load will be relatively high, and the data will be sent to the data source using a communication method that does not establish the aforementioned communication path, which is more network efficient. Device.

2. The device is a path controller that manages the communication path connecting the data source and the server. When it is determined that the aforementioned communication path is established, the communication path is established in order to reduce the processing load on the server and enable the use of a communication method with low network efficiency, If it is determined that the aforementioned communication path cannot be established, the server will be notified of a communication method that does not establish the aforementioned communication path, which would result in a relatively higher processing load on the server and higher network efficiency. The apparatus according to claim 1.

3. The status of the aforementioned server is the CPU load of the aforementioned server, When the CPU load of the server exceeds a threshold, it is determined that the communication path is established. When the CPU load of the server is below a threshold, it is determined that the communication path will not be established. The apparatus according to claim 1.

4. The status of the server is the memory load of the server, When the memory load of the server exceeds a threshold, it is determined that the communication path will not be established. When the memory load of the server is below a threshold, it is determined that the communication path will be established. The apparatus according to claim 1.

5. A device that functions as a data source for sending data to a server, (i) A first communication method for establishing a communication path from the data source to the server, (ii) A second communication method that does not establish the aforementioned communication path, It is possible to send data using both, Based on the status of the aforementioned server, it is determined whether or not to establish the communication path. When it is determined that the aforementioned communication path has been established, the data is transmitted using the first communication method, which results in a relatively low processing load on the server and low network efficiency. If it is determined that the aforementioned communication path cannot be established, the data is transmitted using the second communication method, which has a relatively high processing load on the server and high network efficiency. Device.

6. A device that functions as a server that receives data from a data source, (i) A first communication method for establishing a communication path from the data source to the server, (ii) A second communication method that does not establish the aforementioned communication path, Data can be received using both methods. Based on the status of the device, it is determined whether or not to establish the communication path. When it is determined that the aforementioned communication path has been established, the device receives data using the first communication method, which has a relatively low processing load and low network efficiency. If it is determined that the aforementioned communication path cannot be established, the processing load on the device itself will be relatively high, and the data will be received using the second communication method, which has high network efficiency. Device.

7. A method executed by a path controller that controls the communication path from a data source to a server, Based on the status of the aforementioned server, it is determined whether or not to establish the communication path. When it is determined that the aforementioned communication path is established, the communication path is established in order to reduce the processing load on the server and enable the use of a communication method with low network efficiency, If it is determined that the aforementioned communication path cannot be established, the server will be notified of a communication method that does not establish the aforementioned communication path, which would result in a relatively higher processing load on the server and higher network efficiency. method.

8. A method used by a data source to send data to a server, The aforementioned data source is (i) A first communication method for establishing a communication path from the data source to the server, (ii) A second communication method that does not establish the aforementioned communication path, It is possible to send data using both, Based on the status of the aforementioned server, it is determined whether or not to establish the communication path. When it is determined that the aforementioned communication path has been established, the data is transmitted using the first communication method, which results in a relatively low processing load on the server and low network efficiency. If it is determined that the aforementioned communication path cannot be established, the data is transmitted using the second communication method, which has a relatively high processing load on the server and high network efficiency. method.

9. A method performed by a server that receives data from a data source, The aforementioned server, (i) A first communication method for establishing a communication path from the data source to the server, (ii) A second communication method that does not establish the aforementioned communication path, Data can be received using both methods. Based on the status of the device, it is determined whether or not to establish the communication path. When it is determined that the aforementioned communication path has been established, the device receives data using the first communication method, which has a relatively low processing load and low network efficiency. If it is determined that the aforementioned communication path cannot be established, the processing load on the device itself will be relatively high, and the data will be received using the second communication method, which has high network efficiency. method.

10. A program for a computer to implement each of the functions provided in the apparatus described in any one of claims 1 to 6.

Images