Data collection system

Abstract

The present disclosure provides a server connected to a data source via a communication network, wherein the server: receives, from the data source, a data generation notification indicating that data has been generated; determines, using information obtained by the server, whether to acquire data from the data source; and transmits, to the data source, a data request for causing a path establishment request for establishing a communication path between the data source and the server to be transmitted to the data source on the basis of the result of the determination.

Description

Data Collection System

[0001] The present disclosure relates to communication control for data collection.

[0002] A data collection system has been proposed that collects data from data sources such as user terminals and sensors to a server (see, for example, Patent Document 1). In Patent Document 1, the data source determines whether to transmit data and makes a request to establish a communication path based on the determination result.

[0003] Remote Direct Memory Access (RDMA) is being considered as a method for transferring data at high speed (see, for example, Patent Document 1 and Non-Patent Document 1). RDMA transfers data from the memory of a local computer to the memory of a different remote computer by DMA (direct data transfer between peripheral devices, main memory (RAM), etc., without going through the CPU).

[0004] PCT / JP2021 / 028660

[0005] RFC5040 (https: / / www.rfc-editor.org / rfc / rfc5040.html)

[0006] To transfer data using RDMA in a wide-area communication network, a communication path must be set up for each flow in advance. Therefore, when collecting data from multiple data sources, a large amount of network resources is required. In order for multiple data sources to use limited network resources, it is necessary to adjust the timing of data transfers to avoid traffic collisions from each data source.

[0007] Therefore, an object of the present disclosure is to make it possible to avoid traffic collisions in a data collection system that collects data from data sources.

[0008] The system of the present disclosure includes the server of the present disclosure and a plurality of data sources connected to the server via a communication network.

[0009] The server of the present disclosure is connected to a data source via a communication network and executes a method of the present disclosure, which includes receiving a data occurrence notification from the data source indicating that data has occurred, determining whether to acquire data from the data source using information obtained by the server, and transmitting a data request to the data source based on the result of the determination, causing the data source to transmit a path establishment request to establish a communication path between the data source and the server.

[0010] The server of the present disclosure may have a function for performing processing using data acquired from the data source, and may determine to acquire data from the data source when the function uses data from the data source.

[0011] The above disclosures can be combined as much as possible.

[0012] According to the present disclosure, traffic collisions can be avoided in a data collection system that collects data from data sources.

[0013] 1 shows an example of a system configuration of the present disclosure. 2 shows an example of a sequence executed by the system of the present disclosure. 3 shows an example of a configuration of a data source 10. 4 shows an example of a configuration of a path management unit 40. 5 shows an example of a path setting location table. 6 shows an example of a configuration of a transmission management unit 60. 7 shows an example of a configuration of a server 30. 8 shows an example of a requirement table. 9 shows an example of a transmission policy table. 10 shows an example of a data notification table. 11 is a flow diagram showing an example of the operation of a data transmission timing control function. 12 shows an example of a sequence executed by the system of the present disclosure. 13 shows an example of a configuration of a data source 10.

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. Note that the present disclosure is not limited to the embodiments shown below. These implementation examples are merely illustrative, and the present disclosure can be implemented in various forms with various modifications and improvements based on the knowledge of those skilled in the art. Note that components with the same reference numerals in this specification and drawings indicate the same components.

[0015] Transmitting data every time it is generated consumes resources in the communication network and the server, leading to packet loss and degradation of server performance. The timing of data transfer can also be determined at the data source, as in Patent Document 1. However, from the perspective of maintenance of sensors and data sources installed in remote locations and from the perspective of system management, it is desirable to have as few functions as possible installed in the sensors and data sources. The data source cannot determine when the server and application require data. Therefore, in the present disclosure, the server determines the timing of data transmission.

[0016] In the first embodiment, the server sends a data request to the data source when data is needed, and the data request triggers the establishment of a communication path. The "data request" is a signal that causes the data source 10 to send a path establishment request in order to establish a communication path between the data source 10 and the server 30. In this embodiment, RDMA data transfer is possible over a wide area network to establish a communication path. Furthermore, data can be sent and received when the server and the application truly need the data. Furthermore, the reduced frequency of data transmission reduces the load on the server and the data source.

[0017] The data source of the present disclosure may determine the timing of sending a data occurrence notification indicating that data has occurred. In a second embodiment, the data occurrence notification is sent at the timing determined by the data source. This reduces the number of times the data occurrence notification is sent, thereby reducing the consumption of communication network resources.

[0018] The present disclosure can be combined as described in detail below.

[0019] (First embodiment) Fig. 1 shows an example of a system configuration of the present disclosure. The system of the present disclosure includes a plurality of data sources 10, a server 30, a path management unit 40, and a transmission management unit 60. The plurality of data sources 10, the server 30, the path management unit 40, and the transmission management unit 60 are connected via a communication network 20. The data sources 10 are terminals that have user-side sensing data. The server 30 is a terminal that collects data from the data sources 10.

[0020] The path management unit 40 has a function of managing communication paths from the multiple data sources 10 to the server 30. The transmission management unit 60 has a function of managing transmission control of sensing data from the multiple data sources 10 to the server 30. In this embodiment, an example is shown in which the transmission management unit 60 and the path management unit 40 are separately arranged, but these management units may be provided in a common device or may be composed of multiple devices that are distributed.

[0021] 2 shows an example of a sequence executed by the system of the present disclosure. When the multiple data sources 10 acquire sensing data from the sensors 50, they transmit a data generation notification to the server 30 indicating that data has been generated (S11). When the server 30 receives the data generation notification, it determines whether or not to acquire the sensing data. In this embodiment, an example is shown in which, when a first data generation notification is received, in step S31, it determines that the sensing data is to be "held" and not acquired, and when a second data generation notification is received, in step S32, it decides to acquire the sensing data.

[0022] When it is decided in step S31 to acquire sensing data, the server 30 transmits a data request to the data source 10 (S32). When the data source 10 receives the data request, it transmits a path establishment request to the path management unit 40 (S12). When the path management unit 40 receives the path establishment request from the data source 10, it establishes a path in accordance with the path establishment request (S13). This establishes a communication path between the data source 10 and the server 30. The server 30 acquires sensing data from the data source 10 using the established communication path (S14). Then, when the data source 10 has completed transmitting the sensing data, it transmits a path release request to the path management unit 40 (S15). This releases the communication path between the data source 10 and the server 30.

[0023] In step S14, the data source 10 may transmit the sensing data to the server 30 using any method, such as TCP (Transmission Control Protocol) or RDMA. For example, in step S11, the data source 10 may transmit the amount of sensing data and the memory address where the sensing data is stored, in addition to the data generation notification. In this case, after receiving the path establishment response in step S12, an RDMA connection establishment request / response is sent to both parties to confirm that the path has been established. Then, in step S14, the server 30 can read the sensing data from the memory in the data source 10 based on the memory address and data amount included in the data generation notification.

[0024] In this way, the server 30 of this embodiment receives a data generation notification from the data source 10, uses the information obtained by the server 30 to determine whether or not to acquire data from the data source 10, and sends a data request to the data source 10 based on the result of the determination.

[0025] 3 shows an example of the configuration of the data source 10. The data source 10 includes a data storage function 11, a data occurrence notification function 12, a data transmission / reception function 13, and a communication path request function 17. The data storage function 11 stores sensing data from a sensor 50. When the sensing data is stored in the data storage function 11, the data occurrence notification function 12 sends a "data occurrence notification" to the server 30. The data transmission / reception function 13 sends the sensing data stored in the data storage function 11 to the server 30. The communication path request function 17 sends a path establishment request and a path release request to the path management unit 40.

[0026] 4 shows an example configuration of the path management unit 40. The path management unit 40 includes a request receiving unit 41, a device control unit 42, and a path setting location table. The path setting location table manages setting information (establishment / release) of communication paths from the data source 10 to the server 30. The request receiving unit 41 receives path establishment requests and path release requests from the data source 10 and the server 30. The device control unit 42 references the path setting location table and transmits control signals to network devices in accordance with the path establishment requests and path release requests received by the request receiving unit 41, thereby establishing and releasing communication paths.

[0027] 5 shows an example of a path setting location table. The path setting location table stores information about the network devices connecting the data source 10 to the server 30. For example, in the case of data source 10A, a communication path from the data source 10A to the server 30 can be established and released by controlling the settings of network device A and network device B. For data source 10B, these are network devices A and C, and for data source 10C, these are network devices D and E.

[0028] 6 shows an example configuration of the transmission management unit 60. The transmission management unit 60 includes a requirement receiving function 61, a transmission policy determination function 62, a transmission policy sending function 63, and a transmission policy determination rule. The requirement receiving function 61 receives data transfer requirements from the server 30. The transmission policy determination function 62 determines a transmission policy for each data source 10 in accordance with the data transfer requirements in accordance with the transmission policy determination rule. The transmission policy sending function 63 sends the determined transmission policy to the server 30.

[0029] The transmission policy includes a transmission start timing and a transmission end timing, and can be exemplified as follows: First transmission policy: Data is transmitted immediately after it is generated, and transmission ends immediately. Second transmission policy: Data is transmitted after a certain amount of data has accumulated, and transmission ends immediately. Third transmission policy: Data is transmitted immediately after it is generated, and transmission ends after a certain amount of time has passed. Fourth transmission policy: Data is transmitted after a certain amount of data has accumulated, and transmission ends after a certain amount of time has passed.

[0030] The transmission policy determination rule determines a transmission policy for each data source 10 and each application 30A according to the data transfer requirements. For example, the following can be exemplified: (i) If the allowable delay time is equal to or less than a set time and the data generation frequency is equal to or less than a set value, the first transmission policy described above is determined; (ii) If the allowable delay time is greater than a set time and the data generation frequency is equal to or less than a set value, the second transmission policy described above is determined; (iii) If the allowable delay time is equal to or less than a set time and the data generation frequency is greater than a set value, the third transmission policy described above is determined.

[0031] FIG. 7 shows an example configuration of the server 30. The server 30 includes an application 30A, a data generation notification receiving function 31, a data receiving function 32, a communication path request function 33, a requirement notification function 34, a transmission policy receiving function 35, and a data transmission timing control function 36. The application 30A is an arbitrary application that executes arbitrary processing using sensing data collected from the data sources 10. The data generation notification receiving function 31 receives data generation notifications from each data source 10. The data receiving function 32 transmits data requests to each data source 10 and receives sensing data from each data source 10. The requirement notification function 34 reads data transfer requirements from the requirement table and transmits the read data transfer requirements to the transmission management unit 60. The transmission policy receiving function 35 receives a transmission policy from the transmission management unit 60 and stores it in the transmission policy table 19. The data transmission timing control function 36 controls the transmission timing of the sensing data. In this embodiment, an example of control by the data transmission timing control function 36 is shown, in which the transmission timing of a data request transmitted by the data receiving function 32 is controlled.

[0032] 8 shows an example of the requirements table. In the requirements table, the allowable delay time, data generation frequency, and data generation amount of each data source 10 are stored as data transfer requirements, linked to the application 30A. The allowable delay time is the allowable delay time from when data is generated until it reaches the server. The data generation amount is the amount of data generated at one time in the sensor 50. The data transfer requirements may also include application requirements.

[0033] When the server 30 receives the transmission policy from the transmission management unit 60, it stores it in the transmission policy table 19. As a result, a transmission policy suitable for each data source 10 is stored in the transmission policy table of each data source 10, as shown in FIG.

[0034] The data transmission timing control function 36 uses a "data notification table" as shown in FIG. 10 to manage the time of reception of a data generation notification from the data source 10 as the data generation time. In this embodiment, an example is shown in which the data notification table manages the generation time and data amount of the sensing data together with the memory address in each data source 10. Then, the data transmission timing control function 36 uses information stored in the data notification table, the requirement table, and the transmission policy table to determine whether or not to acquire sensing data from the data source 10. This determination can be made using any information available in the server 30.

[0035] The information of the server 30 is, for example, information about the application 30A. When the application 30A uses the sensing data of the data source 10, the data transmission timing control function 36 determines to acquire the sensing data from the data source 10. The application 30A may be a wide variety of applications depending on the sensor 50.

[0036] 11 shows an example of the operation of the data transmission timing control function 36 based on a data generation notification. The data transmission timing control function 36 waits until it receives a data generation notification (S321), and when the amount of data generation included in the data generation notification is equal to or greater than a set value (S323), it starts acquiring sensing data from the data source 10 (S324). This allows the server 30 to acquire all of the sensing data from the data source 10 at once.

[0037] 11 , when the application 30A requires sensing data (S235), the data transmission timing control function 36 may execute step S324 to start acquiring the sensing data required by the application 30A. As an example, a case will be described in which the sensor 50 is a camera and the application 30A performs person tracking. In steps S321 and S322, the data transmission timing control function 36 determines to acquire sensing data from the data source 10 when the amount of sensing data accumulated in the data source 10 becomes equal to or greater than a threshold.

[0038] When the application 30A detects a person or a person behaving suspiciously, the application 30A transmits a request for sensing data to the data transmission timing control function 36 (S325). Then, the data transmission timing control function 36 executes step S31 and determines to acquire sensing data from the data source 10. As a result, the server 30 starts acquiring sensing data from the data source 10 that has transmitted video / images of the detected person, and continuously receives the sensing data and provides it to the application 30A. At this time, when the server 30 receives a data occurrence notification from the corresponding data source 10, the server 30 sets the data transmission timing control function 36 so that it determines to acquire sensing data in step S31.

[0039] In the above-described step S31, step S31 is executed when a data generation notification is received, but as in the present embodiment, the data transmission timing control function 36 may execute step S31 by an interrupt process triggered by a request for sensing data from the application 30A. In this way, the data transmission timing control function 36 can determine to acquire sensing data from the data source 10 at both the timing of threshold determination and application interrupt.

[0040] When the interrupt process of step S325 occurs, the data transmission timing control function 36 may transition to a waiting state for the interrupt process. In this case, the data transmission timing control function 36 does not need to determine to acquire sensing data from the data source 10 until it receives a request for sensing data from the next application 30A.

[0041] Second Embodiment In this embodiment, similar to the first embodiment, a transmission management unit 60 is further provided, which controls the timing of transmitting a data generation notification.

[0042] 12 shows an example of a sequence executed by the system of the present disclosure. In this embodiment, the data source 10 executes step S21 before step S11. In step S21, when the data source 10 acquires sensing data from the sensor 50, it determines whether or not to transmit a data occurrence notification based on the transmission policy. In this embodiment, an example is shown in which it is determined in steps S21-1 and S21-2 that the data occurrence notification is to be "pending" and not to be transmitted, and then a decision is made to transmit the data occurrence notification in step S21-3.

[0043] 13 shows an example of the configuration of the data source 10. In addition to the functions of the first embodiment, the data source 10 of this embodiment further includes a requirement notification function 14, a transmission policy receiving function 15, and a data transmission timing control function 16. The requirement notification function 14 reads data transfer requirements from the requirement table and transmits the read data transfer requirements to the transmission management unit 60. The transmission policy receiving function 15 receives a transmission policy from the transmission management unit 60. The data transmission timing control function 16 controls the data occurrence notification function 12 in accordance with the transmission policy table.

[0044] The requirements table and the transmission policy table are as described in the first embodiment. When the data source 10 receives a transmission policy from the transmission management unit 60, it stores it in the transmission policy table 19. As a result, a transmission policy suitable for each data source 10 is stored in the transmission policy table of each data source 10, as shown in Fig. 6. The data occurrence notification function 12 makes the determination in step S21 in accordance with the transmission policy table.

[0045] Other Embodiments The device of the present invention can also be realized by a computer and a program, and the program can be recorded on a recording medium or provided via a network.

[0046] Furthermore, in this embodiment, an example has been shown in which the data collected by the server 30 is sensing data, but the present disclosure can be applied to any data that is required to be collected, such as user terminals, sensors, and in-vehicle systems.

[0047] 10: Data source 11: Data storage function 12: Data occurrence notification function 13: Data transmission / reception function 14: Requirement notification function 15: Transmission policy reception function 16: Data transmission timing control function 17: Communication path request function 20: Communication network 30: Server 30A: Application 31: Data occurrence notification reception function 32: Data reception function 33: Communication path request function 34: Requirement notification function 35: Transmission policy reception function 36: Data transmission timing control function 40: Path management unit 41: Request reception unit 42: Device control unit 50: Sensor 60: Transmission management unit 61: Requirement reception function 62: Transmission policy determination function 63: Transmission policy transmission function

Claims

1. A server connected to a data source via a communication network, the server receiving a data occurrence notification from the data source indicating that data has been generated, using information obtained by the server to determine whether or not to acquire data from the data source, and based on the result of the determination, sending a data request to the data source to cause the data source to send a path establishment request to establish a communication path between the data source and the server.

2. The server according to claim 1, wherein the server has a function for performing processing using data acquired from the data source, and when the data from the data source is used in the function, the server determines that data is acquired from the data source.

3. A system comprising: a server according to claim 1 or 2; and a plurality of data sources connected to the server via a communication network.

4. A method executed by a server connected to a data source via a communication network, comprising: receiving a data occurrence notification from the data source indicating that data has occurred; using information obtained by the server, determining whether to acquire data from the data source; and, based on the result of the determination, sending a data request to the data source to cause the data source to send a path establishment request to establish a communication path between the data source and the server.

Images