Generation device, generation method, and generation program

Abstract

A generation device 40 comprises a generation unit 41 that uses the arrival time of data from a data stream group determined in advance as a basis to generate a data arrangement plan for arranging, in a continuous storage area, data from a plurality of data streams having the same data cycle or similar data characteristics among the data stream group.

Description

Generation device, generation method, and generation program

[0001] The present disclosure relates to a generation device, a generation method, and a generation program.

[0002] It is expected that communication infrastructure with its advantages of ultra-low latency and massive connectivity will promote smart industries such as remote factory control and distributed energy resource control. To achieve this, computing infrastructure must also be able to transfer and process data in deterministic time with ultra-low latency and massive connectivity.

[0003] For example, the time it takes to collect data from a real-space data source and complete processing such as AI (Artificial Intelligence) analysis must be kept deterministically below a predetermined value. Even when collecting data from multiple data sources at the same time and performing multi-dimensional analysis, the time it takes must be kept deterministically below a predetermined value.

[0004] In this regard, techniques for improving the efficiency of packet transmission and reception include methods for dividing large data received by the kernel or a network interface card (NIC) into sizes that are easy to process, and for integrating small processed data into sizes that are easy to communicate (Non-Patent Document 1).

[0005] “Segmentation Offloads”, “Generic Segmentation Offload”, “Generic Receive Offload”, The Linux Kernel documentation, [online], [Retrieved December 12, 2020], <URL: https: / / docs.kernel.org / networking / segmentation-offloads.html>

[0006] However, data partitioning and data integration cannot be dynamically optimized according to the situation. Furthermore, when data partitioning is performed, a huge amount of data must be processed individually, which increases the number of I / O operations to the storage area where the data is temporarily stored.

[0007] The present disclosure has been made in consideration of the above circumstances, and an object of the present disclosure is to provide a technology that can improve the time required to process data into a storage area.

[0008] A generation device according to one aspect of the present disclosure includes a generation unit that generates a data placement plan for placing data from multiple data streams within a data stream group that have the same data period or similar data characteristics in consecutive storage areas, based on the arrival times of data from the data stream group that have been determined in advance.

[0009] A generation method according to one aspect of the present disclosure is a generation method performed by a generation device, which generates a data placement plan for placing data from multiple data streams within a data stream group that have the same data period or similar data characteristics in consecutive storage areas based on the arrival times of data from the data stream group that have been determined in advance.

[0010] A generation program according to one aspect of the present disclosure causes a computer to function as the generation device.

[0011] According to the present disclosure, it is possible to provide a technique that can reduce the time required to process data in a storage area.

[0012] FIG. 1 is a diagram showing an example of the overall configuration of a system according to this embodiment. FIG. 2 is a diagram showing an example of the configuration of a video analysis system. FIG. 3 is a diagram showing an example of a method for generating a data placement plan and a data readout plan. FIG. 4 is a diagram showing an example (first example) of processor placement. FIG. 5 is a diagram showing an example (first example) of a data placement plan and a data readout plan. FIG. 6 is a diagram showing an example (second example) of processor placement. FIG. 7 is a diagram showing an example (second example) of a data placement plan and a data readout plan. FIG. 8 is a diagram showing an example (third example) of processor placement. FIG. 9 is a diagram showing an example (third example) of a data placement plan and a data readout plan. FIG. 10 is a diagram showing an example of a method for generating a data transfer plan. FIG. 11 is a diagram showing a processing flow for generating each plan. FIG. 12 is a diagram showing a processing flow for processing a data stream. FIG. 13 is a diagram showing the hardware configuration of a generation device.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0014] [Summary of the Disclosure] In the case of general-purpose operating systems currently widely used in cloud computing, etc., scheduling is performed with emphasis on fairness in processor usage time among tasks, so the wait time until a processor is assigned to a task is constantly fluctuating. In addition, interrupt processing is performed with the highest priority regardless of the priority of the task, so the processing time is also constantly fluctuating.

[0015] Therefore, this disclosure attempts to build a real-time system that guarantees no fluctuations in waiting time or processing time. In particular, optimization is performed to reduce processing waiting time and processor idle time, taking into account the I / O processing load on the storage area. However, this assumes that the data arrival time (including the data arrival time pattern) is known in advance. Communication networks in which the data arrival time is known in advance (communication means with deterministic data arrival) are existing technologies.

[0016] Specifically, based on this premise, a data placement plan is generated that places data from multiple data streams (groups of data that flow intermittently over time) with the same data period or similar data characteristics in a contiguous storage area, and a data reading plan is generated in advance that bulks the data from the multiple data streams from that contiguous storage area and reads it all at once.

[0017] This reduces the number of I / O operations to the storage area, thereby reducing the time required to process data in the storage area.

[0018] [System Configuration Example] In this embodiment, an example will be described in which video analysis is performed by aggregating video from multiple cameras.

[0019] 1 is a diagram showing an example of the overall configuration of a system 1 according to this embodiment. The system 1 includes a plurality of cameras 10, a plurality of processing devices 20, a user terminal 30, and a generating device 40. These are connected to each other via a communication network 50 so as to be able to communicate with each other.

[0020] 2 is a diagram showing an example of the configuration of a video analysis system. A group of video data streams output from multiple cameras 10 are distributed to a user terminal 30 via one or more processing devices 20 in series and / or in parallel.

[0021] The processing device 20 includes a NIC 21 that receives multiple data streams from multiple cameras 10, a memory 22 that stores the multiple data streams, and one or more processors 23 that process the multiple data streams. The multiple processors 23 are connected in multiple stages. Video data is processed sequentially by a first processor 23 a, a second processor 23 b, and a third processor 23 c.

[0022] In the case of Figure 2(b), the first processing device 20a is, for example, a video termination device. The first processor 23a performs color correction and edge enhancement on frames of the data stream. The second processor 23b changes the resolution of frames of the data stream to a resolution that can be processed by a video analysis device. The second processing device 20b is, for example, a video analysis device, and analysis processing is performed by a third processor 23c.

[0023] However, the processing device 20 may be a processing device that executes any processing. The processor 23 may also be a processor that executes any processing. The number of processing devices 20 and the number of processors 23 are also arbitrary. The processor is, for example, a CPU, a GPU, an FPGA, etc. The memory 22 may be a shared memory. A sensor may be used instead of the camera 10.

[0024] [Configuration of the Generation Device] In contrast to the above-described video analysis system, this embodiment includes a generation device 40 as shown in Fig. 1. The generation device 40 includes a generation unit 41, a transmission unit 42, and a storage unit 43.

[0025] The generation unit 41 has the function of allocating a memory area in advance to temporarily store each data of a data stream group, generating a plan to write each data into the allocated memory area when the data arrives, and generating a plan to bulk each data from the memory area and read it all at once.

[0026] Specifically, the generation unit 41 has a function to generate a data placement plan for placing, in a contiguous memory area, data from multiple data streams related to the same or similar tasks, that is, data from multiple data streams with the same data period or similar data characteristics, from the data of the data stream group that has arrived, based on the arrival times of the data of the data stream group that have been determined in advance (including the data arrival time pattern).

[0027] The generating unit 41 has a function of generating a data readout plan for reading the data of the plurality of data streams from the continuous memory area all at once.

[0028] The generation unit 41 has a function of generating a data transfer plan in which, after an earlier processor among the multi-stage first processor 23a to third processor 23c has processed all of the data of the multiple data streams, the data of the multiple data streams is transferred all at once to a later processor.

[0029] The generating unit 41 has a function of generating a software program for referencing the data placement plan, the data readout plan, and the data transfer plan.

[0030] The generating unit 41 can also generate the data placement plan, the data read plan, and the data transfer plan as a single plan.

[0031] The transmitting unit 42 has a function of transmitting the software program to one or more processing devices 20 .

[0032] The storage unit 43 has a function of storing the above software programs.

[0033] The storage unit 43 has a function of storing data necessary for generating the data placement plan, the data readout plan, and the data transfer plan. The necessary data includes, for example, processor time slot design information (maximum processor execution time, maximum communication time between processors, etc.), and placement information (contents of processing tasks (data characteristics, etc.), input data volume, processor placement). This data can be acquired, for example, from an actual machine in a test environment.

[0034] [Method for Generating Data Allocation Plan and Data Readout Plan] FIG. 3 is a diagram showing an image of generating a data allocation plan and a data readout plan, and an image of data processing based on these plans.

[0035] Even if multiple frames are generated at the same time by multiple different cameras 10, there will be a phase shift due to processing fluctuations within the camera 10 and the line length to the first processing device 20a being different for each camera 10, and the frames will not necessarily arrive at the same time.

[0036] Therefore, assuming that the arrival time of data is known in advance, a data placement plan is generated in advance that bulks I / O processing of data from data streams with the same timing or similar characteristics and allocates it to contiguous memory areas, and a data reading plan is generated in advance that reads data from those contiguous memory areas all at once, thereby reducing the number of I / Os to memory and optimizing to reduce data processing wait times and processor free time.

[0037] 3, it is known in advance that data streams A, C, D, and E are 30 fps, and data streams B and F are 15 fps. It is also known in advance that at a given time, the frames of these data streams will arrive in the following order: A → C → D → B → E → F.

[0038] In this case, the generation unit 41 generates a data placement plan that places 30 fps frames A1, C1, D1, and E1 in consecutive memory areas R11 to R14, in that order, and 15 fps frames B1 and F1 in consecutive memory areas R21 to R22, in that order, based on the arrival times of data from the data stream group that have been determined in advance.

[0039] Similarly, for the subsequent 30 fps frames A2, C2, D2, and E2, the generation unit 41 generates a data placement plan for allocating the frames in consecutive memory areas in that order. Similarly, for the subsequent 15 fps frames B2 and F2, the generation unit 41 generates a data placement plan for allocating the frames in consecutive memory areas in that order.

[0040] (Method of generating a data reading plan) In the above example, the generation unit 41 generates a data reading plan in which 30 fps frames A1, C1, D1, and E1 arranged in consecutive memory areas R11 to R14 are read consecutively in that order all at once, and 15 fps frames B1 and F1 arranged in consecutive memory areas R21 to R22 are read consecutively in that order all at once.

[0041] Similarly, for the subsequent frames A2, C2, D2, and E2 at 30 fps, the generator 41 generates a data readout plan for reading the frames from consecutive memory areas at once in that order. Similarly, for the subsequent frames B2 and F2 at 15 fps, the generator 41 generates a data readout plan for reading the frames from consecutive memory areas at once in that order.

[0042] (Specific examples of data placement plans and data reading plans) (First example) Consider a case where, as shown in Figure 4, multiple processors are arranged in multiple stages (horizontally on the page) and in parallel (vertically on the page), and the processing of three 30 fps data streams is consolidated into one data pipeline.

[0043] In this case, the generation unit 41 generates a data placement plan for placing frames A1, B1, and C1 input to the first data pipeline in consecutive memory areas in that order. The generation unit 41 also generates a data readout plan for reading frames A1, B1, and C1 in that order from consecutive memory areas all at once.

[0044] 5 shows a data placement plan and a data read plan corresponding to the configuration of the first data pipeline in FIG. 4. At time 0, the first processor 23a acquires all of the data in a single read operation because frames A1, B1, and C1 are arranged in a contiguous area. Next, at time 20, the second processor 23b acquires the processing results for frames A1, B1, and C1 of the first processor 23a in a single read operation. The same is true for data acquisition by the third processor 23c. Finally, at time 65, the third processor 23c writes the processing results for frames A1, B1, and C1 to a contiguous shared memory in a single output operation.

[0045] In the drawing, a "●" indicates an available processor.

[0046] (Second Example) As shown in Fig. 6, it is also possible to allocate the processing of data in a data stream in units of frames, which are smaller processing units. In this case, the generation unit 41 generates a data placement plan and a data read plan as shown in Fig. 7. At time 0 and time 75, frames A1 to J1 are subjected to I / O processing in consecutive memory areas.

[0047] (Third Example) As shown in Fig. 8, it is also possible to reduce the number of first processors 23a and third processors 23c to two each, divide 300 fps (equivalent to 10 data streams) into 150 fps (equivalent to 5 data streams), and allocate the data to the two first processors 23a. In this case, the generation unit 41 generates a data placement plan and a data readout plan as shown in Fig. 9. At time 0 and time 75, frames A1 to J1 are I / O processed in consecutive memory areas.

[0048] (Supplementary Note) When the memory 22 is a shared memory that can be commonly accessed by multiple processors 23, it is preferable to plan to execute multiple data write processes between data read processes by each processor 23. This can further reduce the time required to process data in the memory area.

[0049] [Method for generating a data transfer plan] Figure 10 is a diagram showing an image of generating a data transfer plan and an image of data processing based on that plan. The control unit 14 generates a data transfer plan for transferring, for example, four 30 fps frames A1, C1, D1, and E1 all at once to a downstream processor. Note that since multiple frames are transferred all at once, delays occur as the data is stored in a buffer, but the plan should be such that the delay falls within a predetermined delay requirement.

[0050] Based on the above data transfer plan, the nth (n is a natural number) processor 23 receives four frames A1, C1, D1, and E1 all at once from the n-1th processor in accordance with the data transfer plan, processes them all, and then transfers the four frames A1, C1, D1, and E1 all at once to the n+1th processor.

[0051] As a result, as shown in Figure 10(a), the number of data input / output operations for the nth processor for the four frames A1, C1, D1, and E1 is a total of two. In contrast, in the conventional case, as shown in Figure 10(b), the number of data input / output operations is a total of eight. The number of data input / output operations between processors is reduced to one-fourth of the conventional number, thereby reducing the number of data input / output operations between processors.

[0052] The data placement plan, the data readout plan, and the data transfer plan are generated by the generating device 4. The generating device 4 may generate each plan as is based on the plan input by the user. The generating device 4 may autonomously generate each plan using data required to generate each plan. The generating device 4 may generate each plan collectively at once. The generating device 4 may utilize machine learning or the like when generating the task allocation plan.

[0053] [Operation for Generating Each Plan] FIG. 11 is a diagram showing a processing flow for generating each plan.

[0054] Step S11: The generating unit 41 generates a data allocation plan for allocating data of a plurality of data streams having the same data period in consecutive memory areas based on the arrival times of data of the data stream group that have been determined in advance.

[0055] Step S12: The generation unit 41 generates a data readout plan for reading the data of the plurality of data streams from the continuous memory areas all at once.

[0056] Step S13: The generation unit 41 generates a data transfer plan in which, after a first-stage processor among the multiple multi-stage processors has processed all of the data of the multiple data streams, the data of the multiple data streams is transferred all at once to a second-stage processor.

[0057] Step S14: The generating unit 41 generates software programs related to the generated data placement plan, data read plan, and data transfer plan. The transmitting unit 42 transmits the software programs to one or more processing devices 20.

[0058] [Data Stream Processing Operation] FIG. 12 is a diagram showing the processing flow of a data stream.

[0059] Step S21: The NIC 21 of the processing device 20 executes the above software program, and when data of the data stream group arrives, it places each frame of multiple data streams with the same data period in a consecutive memory area based on the data placement plan used by the execution.

[0060] Step S22: Each processor 23 of the processing device 20 executes the above software program and, based on the data reading plan used by the execution, reads out each frame (each frame of multiple data streams with the same data period) from consecutive memory areas all at once.

[0061] Step S23: Each processor 23 of the processing device 20 executes the software program, processes all of the frames based on the data transfer plan used by the execution, and then transfers all of the frames together at once to the subsequent processor 23. Thereafter, the last processor 23 of the processing device 20 allocates the processed frames in a continuous memory area based on the data allocation plan.

[0062] [Application Examples] (Application Example 1) For example, the present invention can be applied to cases where safe driving of a vehicle is supported.

[0063] This is a use case in which the risk of collisions between vehicles and between vehicles and pedestrians at intersections is evaluated and predicted from multiple perspectives through integrated analysis of multiple cameras, and real-time feedback (through traffic lights, signs, speakers, etc.) is provided to vehicles and pedestrians to prevent accidents before they occur.

[0064] To realize such a use case, it is necessary to process everything from collecting camera footage to assessing and predicting the risk of accidents and issuing warnings to drivers and pedestrians with low and deterministic latency.

[0065] Therefore, by applying this embodiment, it is possible to prevent accidents such as right-turn collisions and head-on collisions, which frequently occur at intersections. Specifically, it is possible to reduce collisions with oncoming vehicles that occur due to a driver's miscalculation or poor visibility when turning right, or due to a driver ignoring a traffic light. It is also possible to reduce accidents that occur at intersections without traffic lights due to driver inattention or poor visibility.

[0066] (Application Example 2) The technology of this embodiment is highly effective in cases where high-speed, low-latency processing is required. The following use cases are possible application examples.

[0067] One use case is short-term power supply and demand adjustment. Power consumption data for factories, data centers, homes, etc., as well as power supply data for energy conservation and automobile batteries, are collected and analyzed (sensing and risk assessment) with high frequency and low latency, and a stable power supply is achieved by adjusting the demand and supply in response to adjustment requests from the grid based on the results of near-future power supply and demand forecasts.

[0068] One use case is low-latency trading, where automated trading systems in financial markets (market participant systems that receive data from the market, analyze the received data, automatically generate buy and sell orders, and send the generated orders to the exchange system) perform real-time analysis using various ever-changing market data in order to improve operational performance.

[0069] [Effect] According to this embodiment, the generation unit 41 of the generation device 40 generates a data placement plan for placing data from multiple data streams with the same data period or similar data characteristics within the arriving data stream group into consecutive memory areas based on the arrival times of the data from the data stream group that have been determined in advance, thereby reducing the number of times data is input into the memory 22 and the time required to process the data into the memory area.

[0070] Furthermore, according to this embodiment, the generation unit 41 of the generation device 40 generates a data reading plan that reads the data of the above-mentioned multiple data streams from consecutive memory areas all at once, thereby reducing the number of times data is output from the memory 22 and further reducing the time required to process the data into the memory area.

[0071] Furthermore, according to this embodiment, the generation unit 41 of the generation device 40 generates a data transfer plan in which, after an earlier processor among multiple processors has processed all of the data of the multiple data streams, the data of the multiple data streams is transferred all at once to a later processor, thereby reducing the number of data transfers and shortening the data transfer time between processors.

[0072] [Others] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0073] The generation device 40 of the present embodiment described above can be realized, for example, by using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in Fig. 13. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, thereby realizing each function of the generation device 40.

[0074] The generating device 40 may be implemented by one computer. The generating device 40 may be implemented by multiple computers. The generating device 40 may be a virtual machine implemented on a computer.

[0075] The program for the generation device 40 can be stored in a computer-readable recording medium such as a HDD, SSD, USB memory, CD, or DVD. The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the generation device 40 can also be distributed via a communication network.

[0076] REFERENCE SIGNS LIST 1 System 10 Camera 20 Processing device 21 NIC 22 Memory 23a First processor 23b Second processor 23c Third processor 30 User terminal 40 Generation device 41 Generation unit 42 Transmission unit 43 Storage unit 50 Communication network 901 CPU 902 Memory 903 Storage 904 Communication device 905 Input device 906 Output device

Claims

1. A generating device comprising: a generating unit that generates a data placement plan that places data from multiple data streams with the same data period or similar data characteristics within a data stream group in consecutive storage areas based on the arrival times of data from the data stream group that have been determined in advance.

2. The generating device according to claim 1, wherein the generating unit generates a data readout plan for reading data of the plurality of data streams from the continuous storage area all at once.

3. The generating device according to claim 1, wherein the generating unit generates a data transfer plan in which, after a first-stage processor among a plurality of multi-stage processors has processed all of the data of the plurality of data streams, the data of the plurality of data streams is transferred all at once to a second-stage processor.

4. A generation method performed by a generation device, which generates a data placement plan for placing data from multiple data streams with the same data period or similar data characteristics within a data stream group in consecutive storage areas based on the arrival times of data from the data stream group that have been determined in advance.

5. A generating program that causes a computer to function as the generating device according to any one of claims 1 to 3.

Images