Data collection system

The data collection system addresses bottlenecks in data transmission by calculating and optimizing data transfer requirements, ensuring efficient data transmission and resource utilization in wide area networks, thereby enhancing the satisfaction of data transfer requirements and reducing server load.

WO2024246991A1PCT designated stage expired Publication Date: 2024-12-05NIPPON TELEGRAPH & TELEPHONE CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
PCT/JP2023/019721
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-12-05

AI Technical Summary

Technical Problem

Conventional data collection systems face bottlenecks in CPU load and power consumption when transmitting large amounts of data from IoT sensors, and RDMA over wide area networks is hindered by packet loss and potential deadlocks, leading to increased server load.

Method used

A data collection system that calculates the degree of satisfaction with data transfer requirements for each data source, using this index to determine a transmission policy that optimizes resource usage and delay requirements, allowing data transmission while maintaining resource usage within a predetermined threshold.

Benefits of technology

The system effectively satisfies data transfer requirements for more data sources by optimizing transmission policies based on resource usage and delay satisfaction, maximizing resource utilization and reducing server load, even in wide area networks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure JP2023019721_05122024_PF_FP_ABST
    Figure JP2023019721_05122024_PF_FP_ABST
Patent Text Reader

Abstract

The present disclosure is a transmission control device comprising: a function for calculating, for each of a plurality of data sources, a degree of satisfaction with respect to a data transfer requirement in the data sources; and a function for controlling transmission of the plurality of data sources so that the sum of the degrees of satisfaction in the plurality of data sources increases.
Need to check novelty before this filing date? Find Prior Art

Description

Data Collection System

[0001] The present disclosure relates to a transmission control technique in a data collection system.

[0002] With the spread of the Internet of Things (IoT) and remote work, communication traffic continues to increase year by year. When transmitting large volumes of data generated by a wide variety of sensors to a server, conventional communication methods can cause a bottleneck in communication processing on the receiving side. Furthermore, an increased load on the central processing unit (CPU) can lead to a decrease in reception processing performance and an increase in power consumption.

[0003] RDMA (Remote Direct Memory Access) has the effect of reducing CPU load by eliminating memory copying, which was a bottleneck in the conventional TCP (Transmission Control Protocol). However, RDMA data transmission requires a low-loss network, so it cannot be directly used in current wide-area networks.

[0004] Methods of using RDMA in networks where packet loss occurs include using PFC (Priority Flow Control) and providing a retransmission control mechanism in the server, but these methods have issues such as the possibility of deadlock and increased server load.

[0005] To achieve data transmission using RDMA over a wide area network, a system has been proposed in which data sources send data transfer requirements to a controller, the controller receives the requirements, determines a transmission policy for each data source, and distributes it to the data sources. Each data source autonomously requests the controller to establish a communication path based on the transmission policy, and transmits data after the communication path is established (see, for example, Patent Document 1 and Non-Patent Document 1).

[0006] WO2023 / 012878

[0007] Ryohei Tsugami and Tatsuya Fukui, "Study on communication path usage control method for wide-area multi-terminal RDMA data collection," IEICE Technical Report, vol. 122, no. 14, CS2022-7, pp. 25-30, May 2022.

[0008] Although data transmission requirements are set for each data transmitted from a data source, the data transmission requirements may not always be met during system operation. Therefore, the present disclosure aims to use the satisfaction level of the data transmission requirements as an index and determine a transmission policy based on this.

[0009] The data collection system of the present disclosure includes the transmission control device of the present disclosure, a plurality of data sources that transmit data in accordance with control from the transmission control device, and a server that collects data transmitted from the plurality of data sources.

[0010] The transmission control device of the present disclosure has a function of calculating the degree of satisfaction with data transfer requirements for multiple data sources for each data source, and a function of controlling the transmission of the multiple data sources so that the sum of the degrees of satisfaction for the multiple data sources is increased.

[0011] A transmission control device according to the present disclosure executes a control method according to the present disclosure, which includes a procedure for calculating, for each of a plurality of data sources, a degree of satisfaction with a data transfer requirement for the data sources, and a procedure for controlling transmission of the plurality of data sources so as to maximize the sum of the degrees of satisfaction for the plurality of data sources.

[0012] The data transfer requirements may include a delay requirement. In this case, the transmission control device according to the present disclosure may acquire delays at the plurality of data sources and calculate, for each data source, a ratio of the delay to the delay requirement as a degree of satisfaction with the data transfer requirements.

[0013] The transmission control device of the present disclosure may calculate resource usage amounts used by the multiple data sources in the communication network, and increase the sum of the satisfaction levels so that the resource usage amounts are within a predetermined threshold. In this case, the transmission control device of the present disclosure may calculate the resource usage amounts using generated data sizes, data generation frequencies, and data accumulation amounts of the multiple data sources.

[0014] The transmission control of the multiple data sources may include the amount of data accumulated in at least one of the multiple data sources, and the multiple data sources may transmit the data when the data accumulation amount set in their own devices has been accumulated.

[0015] The above disclosures can be combined as much as possible.

[0016] According to the present disclosure, the satisfaction level of the data transfer requirements is used as an index and a transmission policy is determined based on the index, so that the data collection system that collects data according to the present disclosure can satisfy the data transfer requirements of a larger number of data sources.

[0017] 1 shows an example of a system configuration of the present disclosure; 2 shows an example of a data transmission sequence; 3 shows an example of a server configuration; 4 shows an example of a data source configuration; 5 shows an example of information stored in a requirement table of each data source; 6 shows an example of a configuration of a transmission control device; 7 shows an example of an operation in the transmission control device; 8 shows an example of a transmission policy determination flow in a transmission policy determination unit;

[0018] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings indicate the same components.

[0019] 1 shows an example of a system configuration of the present disclosure. The system of the present disclosure includes a plurality of data sources 10, a transmission control device 20, a path management device 30, and a server 40. The plurality of data sources 10, the transmission control device 20, the path management device 30, and the server 40 are connected via a communication network 80.

[0020] The data source 10 is a terminal that has user-side data such as sensing data acquired by a sensor 50. The server 40 is a terminal that collects data from the data source 10. The communication network 80 is any means that enables packet transfer, and may be wired or wireless, and may include a public network.

[0021] 2 shows an example of a data transmission sequence. In the data collection system of the present disclosure, the path management device 30 establishes a communication path, thereby transmitting data from the data source 10 to the server 40. At this time, the data collection system of the present disclosure executes the following sequence: The data source 10 requests establishment of a communication path (S101). The path management device 30 establishes the communication path (S102), and then transmits a notification that the communication path has been established (S103). The data source 10 transmits data (S104), and then transmits a request to release the communication path (S105). The path management device 30 releases the communication path, and then transmits a notification that the communication path has been released (S106).

[0022] The timing of step S101 in the data transmission sequence is determined by the transmission policy of the data source 10. Examples of transmission policies include the following: First transmission policy: Data is transmitted immediately after it is generated. Second transmission policy: Data is transmitted when a predetermined amount of data has been accumulated in the device.

[0023] 2 shows an example in which the data source 10 transmits data in accordance with the first transmission policy. The time from when the data is generated until the data is transmitted (S104) is the delay. The data in the data source 10 includes both strict and loose delay requirements. When the delay requirement is loose, the first transmission policy is not required, and the second transmission policy may be used. Therefore, in the following embodiment, the satisfaction level with the delay requirement is used as an index to indicate the satisfaction level with the data transfer requirement, and the transmission policy is determined based on this.

[0024] 3 shows an example configuration of the server 40. The server 40 includes a data receiving unit 41, a memory 42, and an application 43. The data receiving unit 41 receives data from each data source 10. The memory 42 stores data from the sensor 50. The application 43 collects data from the sensor 50. The application 43 is any application that collects any data detected or generated by a user terminal, a sensor, an in-vehicle system, or the like.

[0025] 4 shows an example of the configuration of the data source 10. The data source 10 includes a requirement notification unit 11, a transmission policy receiving unit 12, a data transmission timing control unit 13, a communication path setting unit 14, a data transmission unit 15, a communication path release unit 16, a generated data storage unit 17, a requirement table 18, and a transmission policy table 19.

[0026] The generated data storage unit 17 stores data from the sensor 50. The requirement notification unit 11 reads data transfer requirements from the requirement table 18 and notifies the transmission control device 20 of the read data transfer requirements. The transmission policy receiving unit 12 receives a transmission policy from the transmission control device 20. The data transmission timing control unit 13 controls the communication path setting unit 14, the data transmitting unit 15, and the communication path release unit 16 in accordance with the transmission policy table 19. The communication path setting unit 14 sends a path establishment request to the path management device 30. The data transmitting unit 15 transmits data stored in the generated data storage unit 17 to the server 40. The communication path release unit 16 sends a path release request to the path management device 30. The requirement table 18 stores data transfer requirements for each data source 10. The transmission policy table 19 stores policies for transmitting data from the data source 10 to the server 40.

[0027] 5 shows an example of information stored in the requirement table 18 for each data source. The requirement table contains, as data transfer requirements, the allowable delay time d i_req The data generation frequency and generated data size are stored in association with the application. The allowable delay time is the allowable delay time from when data is generated until it reaches the server 40, and corresponds to the delay requirement. The generated data size is the amount of data generated at one time in the sensor 50.

[0028] 6 shows an example of the configuration of the transmission control device 20. The transmission control device 20 includes a requirement receiving unit 21, a transmission policy determining unit 22, a transmission policy distribution unit 23, and a transmission policy determining rule 24.

[0029] The transmission policy determination unit 22 has a function of calculating the degree of satisfaction of the data transfer requirements for each of the multiple data sources 10, and a function of controlling the transmission of the multiple data sources 10 so as to maximize the sum of the satisfaction levels for the multiple data sources 10. In this embodiment, the data transfer requirements are defined as the allowable delay time d i_req the transmission policy determination unit 22 determines the average delay time d iand a function to acquire the allowable delay time d as a satisfaction level for the data transfer requirements. i_req Average delay time d i An example will be described in which the function of calculating the ratio of each data source 10 is provided.

[0030] 7 shows an example of the operation of the transmission control device 20. Each data source 10 notifies the transmission control device 20 of data transfer requirements based on the information in the requirements table 18 (S201). The data transfer requirements include the allowable delay time d i_req , data occurrence frequency, and generated data size. The requirement receiving unit 21 of the transmission control device 20 receives the data transfer requirements from the data source 10. The transmission policy determining unit 22 determines a transmission policy for each data source 10 in accordance with the data transfer requirements in accordance with the transmission policy determination rules 24 (S202). The transmission policy distribution unit 23 of the transmission control device 20 distributes the determined transmission policy to each data source 10 (S203).

[0031] 8 shows an example of a transmission policy determination flow in the transmission policy determination unit 22. The transmission policy determination unit 22 acquires the data transfer requirements of each data source 10 (S301), and calculates the delay requirement satisfaction level S for each ith data source 10. i (S302), and the resource usage amount NR i The total resource usage amount NR of all data sources 10 is calculated (S303), and the total resource usage amount NR is equal to or less than the threshold (S304), and the delay requirement satisfaction degree S i (S305) to maximize the sum S of the delay requirement satisfaction scores S of each data source 10. i is adjusted (S302).

[0032] Delay requirement satisfaction S i When the sum S of the data stored in the i-th data source 10 reaches a maximum (Yes in S305), the transmission policy distribution unit 23 transmits the data stored in the i-th data source 10 to the i-th data source 10. iAs a result, the data storage amount T i The amount of data stored is T i The data source 10, whose transmission policy table 19 stores the data stored amount T i When the data is accumulated, the data is transmitted.

[0033] In step S301, the data transfer requirement is the allowable delay time d i_req , average delay time d i , data occurrence frequency F i , generated data size P i Here, the generated data size P i is the amount of data generated at one time in data source i.

[0034] In step S302, the delay requirement satisfaction S of the data source i is calculated. i is the average delay time d of data source i i and the allowable delay time d i_req Using S, it can be expressed as follows: i = d i / d i_req

[0035] Here, the amount of data stored T i For data source i, where i is the generated data size P of data source i i , data occurrence frequency F i and data accumulation amount T i Using the above, it can be calculated using the following formula: i =F i *P i / T i

[0036] In step S303, the resource usage amount NR of the i-th data source 10 is calculated. i can be expressed as follows: Here, BW is the communication bandwidth used, and H is the path establishment overhead.

[0037] Therefore, the total resource usage NR in the communication network 80 used by all data sources 10 can be expressed by the following equation:

[0038] In this way, the transmission policy determination unit 22 determines the generated data size P i , data occurrence frequency F i , and the amount of data stored T i and a function of increasing the sum S of satisfaction levels so that the total resource usage NR is within a predetermined threshold.

[0039] The threshold value in step S304 is the total amount of network resources that can be used in the communication network 80, and can be set arbitrarily. i The sum S is expressed by the following equation.

[0040] In step S306, the transmission policy distribution unit 23 transmits the delay requirement satisfaction degree S to the i-th data source 10. i The amount of data stored according to T i is transmitted as a transmission policy. i The specific calculation method can be exemplified as follows:

[0041] (For data sources with strict delay requirements) The delay requirement satisfaction level S of data source i with strict delay requirements i = 1, the data accumulation amount T of the data source i is calculated from the formula (1). i is obtained by the following formula: T i =F i *P i / d i_req (4) where BW and H are the average delay times d i was excluded because it is included in

[0042] Data occurrence frequency F i = 10 times / s, generated data size P i = 64 KB, allowable delay time d i_reqFor data source i with a data size of 10 ms, equation (4) gives 64000 KB = 64 MB.

[0043] (Data Sources with Less Strict Delay Requirements) The delay requirement satisfaction of a data source i with strict delay requirements is expressed as, for example, S i = 0.5, then, from equation (1), the data accumulation amount T i is obtained by the following formula: T i = 2 * F i *P i / d i_req (5) where BW and H are the average delay times d i was excluded because it was included.

[0044] Data occurrence frequency F i = 10 times / s, generated data size P i = 64 KB, allowable delay time d i_req For data source i with a data size of 10 ms, equation (5) gives 128000 KB = 128 MB.

[0045] When each data source 10 receives the data accumulation amount as a transmission policy, it transmits the data when the data accumulation amount set in the data source 10 is reached. For example, if the delay requirement of data source A is strict and the data accumulation amount T i is 64 MB, the data source A requests the path management device 30 to establish a communication path when 64 MB of sensing data is accumulated. For example, if the delay requirement of the data source B is not strict and the data accumulation amount T i is 128 MB, the data source B requests the path management device 30 to establish a communication path when 128 MB of sensing data is accumulated.

[0046] As described above, the data collection system of this embodiment is i is used as an index, and the delay requirement satisfaction S i The data accumulation amount T i In this way, the present embodiment determines the delay requirement satisfaction level S i A transmission policy that determines the data transmission timing of each data source 10 can be determined so that the sum S of the above equations is maximized.

[0047] Here, in this embodiment, the delay requirement satisfaction level S is calculated within the range of the resources of the communication network 80. i Therefore, in this embodiment, each data source 10 transmits data based on the transmission policy, thereby enabling transmission control to be performed so as to make the most effective use of the resources of the communication network 80.

[0048] (Other embodiments) The delay requirement satisfaction level S i The transmission control based on the data accumulation amount T i The parameter is not limited to the above, but may be any parameter in the transmission policy. Therefore, the present disclosure enables efficient data transmission to the server 40 even when there is a delay in a wide area network.

[0049] In the above embodiment, the delay requirement satisfaction level S i The transmission control was performed using the index, but the delay requirement satisfaction S i Instead of or in addition to the above, the degree of satisfaction with any requirement included in the data transfer requirements can be used as an index.

[0050] Furthermore, in the above embodiment, an example is shown in which the transmission control device 20 and the path management device 30 are arranged separately, but these control devices may be provided in a common device, or may be composed of multiple devices arranged in a distributed manner.

[0051] Furthermore, in the above-described embodiment, an example was shown in which the data collection system includes a path management device 30, but the data collection system of the present disclosure is not limited to this. For example, the data collection system of the present disclosure may collect data from the data source 10 to the server 40 without using a communication path established by the path management device 30.

[0052] The transmission control device 20 of the present disclosure can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided via a network. The program of the present disclosure is a program for causing a computer to realize each functional unit provided in the transmission control device 20 according to the present disclosure, and is a program for causing a computer to execute each procedure provided in the method executed by the transmission control device 20 according to the present disclosure.

[0053] 10, 10A, 10B: Data source 20: Transmission control device 30: Path management device 40: Server 50: Sensor 11: Requirement notification unit 12: Transmission policy receiving unit 13: Data transmission timing control unit 14: Communication path setting unit 15: Data transmitting unit 16: Communication path releasing unit 17: Generated data storage unit 18: Requirement table 19: Transmission policy table 21: Requirement receiving unit 22: Transmission policy determination unit 23: Transmission policy distribution unit 24: Transmission policy determination rule 41: Data receiving unit 42: Memory 43: Application

Claims

1. A transmission control device having a function of calculating, for each of a plurality of data sources, a degree of satisfaction with data transfer requirements; and a function of controlling transmission of the plurality of data sources so as to maximize the sum of the degrees of satisfaction of the plurality of data sources.

2. A transmission control device as described in claim 1, wherein the data transfer requirements include delay requirements, and the transmission control device has a function of acquiring delays in the multiple data sources, and a function of calculating, for each data source, a ratio of delay to the delay requirement as a degree of satisfaction with the data transfer requirements.

3. A transmission control device as described in claim 1, comprising: a function for calculating the amount of resource usage used by the multiple data sources in the communication network; and a function for increasing the sum of the satisfaction levels so that the amount of resource usage is within a predetermined threshold.

4. The transmission control device according to claim 3, further comprising a function for calculating the resource usage amount using the generated data size, data generation frequency, and data accumulation amount of the plurality of data sources.

5. A data collection system comprising: a transmission control device according to any one of claims 1 to 4; a plurality of data sources which transmit data in accordance with control from said transmission control device; and a server which collects data transmitted from said plurality of data sources.

6. The data collection system of claim 5, wherein the transmission control of the multiple data sources includes a data accumulation amount to be accumulated in at least one of the multiple data sources, and the multiple data sources perform the data transmission when data of the data accumulation amount set for the device itself has been accumulated.

7. A control method comprising: a step of a transmission control device calculating, for each of a plurality of data sources, a degree of satisfaction with data transfer requirements for the plurality of data sources; and a step of controlling transmission of the plurality of data sources so as to maximize the sum of the degrees of satisfaction for the plurality of data sources.

8. A program for causing a computer to realize each function of the transmission control device according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Method for providing action guide information and electronic device supporting method

    US20180345081A1

  • System, method, and program for collecting data

    WO2023012878A1