Generation device, generation method, and generation program

Abstract

A generation device 40 includes a generation unit 41 that, on the basis of the arrival time of each piece of data of a plurality of data streams determined in advance, generates a task allocation plan for allocating the processing of each piece of data of the plurality of data streams to a plurality of parallel multi-stage processors, so as to aggregate the processing, in data stream units, into one or more columns among a plurality of columns of multi-stage processors, or to aggregate the processing, in data piece units, into one or more stages among the plurality of multi-stage processors of the plurality of columns.

Description

Generation device, generation method, and generation program

[0001] The present disclosure relates to a generation device, a generation method, and a generation program.

[0002] It is expected that communication infrastructure with its advantages of ultra-low latency and massive connectivity will promote smart industries such as remote factory control and distributed energy resource control. To achieve this, computing infrastructure must also be able to transfer and process data in deterministic time with ultra-low latency and massive connectivity.

[0003] For example, the time it takes to collect data from a real-space data source and complete processing such as AI (Artificial Intelligence) analysis must be kept deterministically below a predetermined value. Even when collecting data from multiple data sources at the same time and performing multi-dimensional analysis, the time it takes must be kept deterministically below a predetermined value.

[0004] Technologies for transferring data at deterministic times include TSN (Time Sensitive Network), which transfers data with punctuality over Ethernet by time synchronization between devices or time-division transmission, and DMA (Direct Memory Access), which transfers data directly between memories without going through the CPU or OS. These technologies make it possible to deterministically transfer data at deterministic times.

[0005] In order to make the overall processing time deterministic, task execution time must also be deterministic. In the case of general-purpose operating systems currently widely used in cloud computing, scheduling is performed with an emphasis on fairness in processor usage time among tasks, so the wait time until a processor is assigned to a task is constantly fluctuating. In addition, interrupts are processed with the highest priority regardless of the task priority, so processing time is also constantly fluctuating.

[0006] As a method for suppressing such fluctuations in waiting time and processing time, there is a method of exclusively allocating a processor for processing a predetermined purpose (Non-Patent Documents 1 and 2).

[0007] “Scheduling Algorithms for Multiprogramming in a Hard Real-Time Environment”, Journal of the Association for Computing Machinery, Vol. 20, No. 1, January 1973, p. 46-p. 61, [online], [Retrieved February 19, 2020], <URL: https: / / www.cs.ru.nl / ~hooman / DES / liu-layland.pdf> “Deadline Task Scheduling”, [online], [searched on February 19, 2020], <URL: https: / / www.kernel.org / doc / html / latest / scheduler / sched-deadline.html>

[0008] However, if a processor is occupied by a specific task, the processor cannot execute other processes even when there is an idle time when the processor is not processing a task, and the processor utilization rate drops by the amount of the idle time.

[0009] Furthermore, since a general-purpose OS generally shares a processor with other tasks, it is difficult to predict the influence of other tasks, and it is difficult to complete processing within a definite time.

[0010] The present disclosure has been made in consideration of the above circumstances, and an object of the present disclosure is to provide a technology that can achieve both determinism in task processing time and improved processor utilization.

[0011] A generation device according to one aspect of the present disclosure includes a generation unit that generates a task allocation plan for allocating tasks to multiple parallel multi-stage processors based on the arrival time of each piece of data in multiple data streams that has been determined in advance, so that processing of each piece of data in the multiple data streams is aggregated on a data stream basis to one or more columns of multiple columns of multi-stage processors, or is aggregated on a data unit basis to one or more columns of the multiple columns of multi-stage processors.

[0012] A generation method of one aspect of the present disclosure is a generation method performed by a generation device, which generates a task allocation plan to assign tasks to multiple parallel multi-stage processors based on the arrival time of each piece of data in multiple data streams that has been determined in advance, so that the processing of each piece of data in the multiple data streams is aggregated on a data stream basis to one or more columns of multiple columns of multi-stage processors, or is aggregated on a data unit basis to one or more columns of multiple columns of multi-stage processors.

[0013] A generation program according to one aspect of the present disclosure causes a computer to function as the generation device.

[0014] According to the present disclosure, it is possible to provide a technology that can achieve both determinism in task processing time and improved processor utilization.

[0015] FIG. 1 is a diagram illustrating an example of the overall configuration of a system according to this embodiment. FIG. 2 is a diagram illustrating an example of the configuration of a video analysis system. FIG. 3 is a diagram illustrating an example (first example) of a task allocation plan generation method. FIG. 4 is a diagram illustrating an example (first example) of a task allocation plan. FIG. 5 is a diagram illustrating an example (first example) of a task allocation plan. FIG. 6 is a diagram illustrating an example (first example) of a task allocation plan. FIG. 7 is a diagram illustrating an example (first example) of a task allocation plan. FIG. 8 is a diagram illustrating an example (second example) of a task allocation plan generation method. FIG. 9 is a diagram illustrating an example (second example) of a task allocation plan. FIG. 10 is a diagram illustrating an example (third example) of a task allocation plan generation method. FIG. 11 is a diagram illustrating an example (third example) of a task allocation plan. FIG. 12 is a diagram illustrating an example (other example) of a task allocation plan generation method. FIG. 13 is a diagram illustrating an example (other example) of a task allocation plan generation method. FIG. 14 is a diagram illustrating a process flow for generating a task allocation plan. FIG. 15 is a diagram illustrating a data stream processing flow. FIG. 16 is a diagram illustrating a hardware configuration of a generation device.

[0016] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In the description of the drawings, the same parts are designated by the same reference numerals and the description thereof will be omitted.

[0017] [Summary of the Disclosure] The disclosure consolidates multiple data streams (groups of data that flow intermittently in a time series) and generates a task allocation plan to processors that can process the data streams while observing time constraints through time division. This makes it possible to achieve both deterministic task processing times and improved processor utilization. However, this assumes that the data arrival times (including data arrival time patterns) are known in advance. Communication networks in which data arrival times are known in advance (communication means with deterministic data arrival) are existing technologies.

[0018] In other words, this disclosure focuses on the fact that data arrival times are deterministic, and generates a logical task execution plan in advance based on processor time slot design information and placement information to increase processor utilization, thereby achieving both the two conflicting goals of improving processor utilization and task execution time constraints.

[0019] [System Configuration Example] In this embodiment, an example will be described in which video analysis is performed by aggregating video from multiple cameras.

[0020] 1 is a diagram showing an example of the overall configuration of a system 1 according to this embodiment. The system 1 includes a plurality of cameras 10, a plurality of processing devices 20, a user terminal 30, and a generating device 40. These are connected to each other via a communication network 50 so as to be able to communicate with each other.

[0021] 2 is a diagram showing an example of the configuration of a video analysis system. Multiple data streams output from multiple cameras 10 are distributed to a user terminal 30 via one or multiple processing devices 20 in series and / or in parallel.

[0022] The processing device 20 includes a NIC 21 that receives multiple data streams from multiple cameras 10, a memory 22 that stores the multiple data streams, and one or more processors 23 that process the multiple data streams. The multiple processors 23 are connected in multiple stages. Video data is processed sequentially by a first processor 23 a, a second processor 23 b, and a third processor 23 c.

[0023] In the case of Figure 2(b), the first processing device 20a is, for example, a video termination device. The first processor 23a performs color correction and edge enhancement on frames of the data stream. The second processor 23b changes the resolution of frames of the data stream to a resolution that can be processed by a video analysis device. The second processing device 20b is, for example, a video analysis device, and analysis processing is performed by a third processor 23c.

[0024] However, the processing device 20 may be a processing device that executes any processing. The processor 23 may also be a processor that executes any processing. The number of processing devices 20 and the number of processors 23 are also arbitrary. The processor is, for example, a CPU, a GPU, an FPGA, etc. The memory 22 may be a shared memory. A sensor may be used instead of the camera 10.

[0025] [Configuration of the Generation Device] In contrast to the above-described video analysis system, this embodiment includes a generation device 40 as shown in Fig. 1. The generation device 40 includes a generation unit 41, a transmission unit 42, and a storage unit 43.

[0026] The generation unit 41 has a function of generating a task allocation plan for allocating tasks to multiple parallel multi-stage processors based on the arrival time of each piece of data in multiple data streams that has been determined in advance, so that the processing of each piece of data in the multiple data streams is aggregated in units of data streams to one or more columns of multiple multi-stage processors, or is aggregated in units of data to one or more columns of multiple multi-stage processors.

[0027] The generation unit 41 has a function of generating the task allocation plan so that the utilization rates of the plurality of processors are the same.

[0028] The generation unit 41 has a function of generating a software program for referring to the task allocation plan.

[0029] The transmitting unit 42 has a function of transmitting the software program to one or more processing devices 20 .

[0030] The storage unit 43 has a function of storing the above software programs.

[0031] The storage unit 43 has a function for storing data necessary for generating a task allocation plan. The necessary data includes, for example, processor time slot design information (maximum processor execution time, maximum communication time between processors, etc.) and placement information (contents of processing tasks (data characteristics, etc.), input data volume, processor placement). This data can be obtained, for example, from the actual machine in the test environment.

[0032] 2, data streams are sequentially processed by a first processor 23a, a second processor 23b, and a third processor 23c, which are connected in multiple stages. In the following example, a case will be described in which 10 data streams are sequentially processed by a plurality of first processors 23a, a plurality of second processors 23b, and a plurality of third processors 23c, which are arranged in multiple stages (horizontal direction on the page) and in parallel (vertical direction on the page).

[0033] (First Example) In this example, data refers to one frame of data that constitutes the data stream output by the camera.

[0034] As shown in Figure 3, the time slot length of each of the first processor 23a and the third processor 23c is 5 ms. Therefore, the processing capacity of each of the first processor 23a and the third processor 23c is 1 s / 5 ms = 200 fps. The time slot length of the second processor 23b is 10 ms. Therefore, the processing capacity of the second processor 23b is 1 s / 10 ms = 100 fps. The frames of the ten data streams A to J are 30 fps.

[0035] The connection of the first processor 23a, the second processor 23b, and the third processor 23c, which are connected in multiple stages, is called a data pipeline. The processing capacity of one data pipeline is 100 fps, because it is dominated by the second processor 23b, which has the lowest processing capacity. Therefore, one data pipeline can process 100 fps / 30 fps≒3 data streams.

[0036] 3, three data streams A to C are aggregated into a first data pipeline, three data streams D to F are aggregated into a second data pipeline, and three data streams G to I are aggregated into a third data pipeline. Data stream J is processed in a fourth data pipeline. In other words, in the first example, processing is aggregated on a data stream-by-data stream basis.

[0037] At this time, three data streams are input to each of the first to third data pipelines. From the processor's point of view, the processing of each of the three data streams is performed by switching between them in a time-division manner, which results in overhead processing of "searching for available free processor resources after the data of a data stream arrives."

[0038] Therefore, in the first example, we consider omitting overhead processing, assuming that data arrives via an optical communication network that can guarantee accurate data arrival. That is, in the first example, we assume that the arrival times of data in a data stream are known in advance, and generate a static task allocation plan in advance that determines which processor will process which stream frame at what time based on the known data arrival times (including the data arrival time pattern). Note that an optical communication network is an example of a communication means that has deterministic data arrival.

[0039] In summary, in the first example, a task allocation plan is generated in advance based on the arrival time of each piece of data in multiple data streams, which is known in advance, so that the processing of each piece of data in those multiple data streams is aggregated on a data stream basis.

[0040] 4 to 7 are diagrams showing examples of task allocation plans corresponding to the configuration of Fig. 3. Fig. 4 shows a task allocation plan for data streams A to C. Three data streams A to C are aggregated into a first data pipeline.

[0041] For data stream A, the task allocation plan is as follows: frame A1 is read from shared memory at time 0, frame A1 is processed by the first processor 23a of the first data pipeline for 5 ms, frame A1 is then processed by the second processor 23b of the first data pipeline for 10 ms, and frame A1 is then processed by the third processor 23c of the first data pipeline for 5 ms, and frame A1 is stored in shared memory after all processing is completed.

[0042] For data stream B, the task allocation plan is such that frame B1 is read from the shared memory at time 0, and after frame A1 is processed by the first processor 23a, frame B1 is processed by the first processor 23a.

[0043] For data stream C, the task allocation plan is such that frame C1 is read from the shared memory at time 0, and after frame B1 is processed by the first processor 23a, frame C1 is processed by the first processor 23a.

[0044] In the drawing, a "●" indicates an available processor.

[0045] In this way, in the first example, a task allocation plan is generated based on the arrival time of each piece of data in multiple data streams, which is known in advance, so the process of dynamically searching for available free processor resources can be omitted.Furthermore, since the task allocation plan is generated so that the processing of each piece of data in the multiple data streams is aggregated on a data stream basis, it is possible to achieve both deterministic task processing times and improved processor utilization.

[0046] (Second Example) In the first example, the utilization rate of each of the first to third data pipelines is 30 fps x 3 / 100 fps = 90% because three data streams are input to each. On the other hand, the utilization rate of the fourth data pipeline is 30 fps x 1 / 100 fps = 30%. Only 30 fps of the 100 fps processing capacity is being used.

[0047] Therefore, in the second example and the third example described below, a task allocation plan is generated to reduce the number of idle processors.

[0048] 8, the processing of each data stream in multiple data streams is distributed in small processing units of frames. For example, frame A1 of data stream A is distributed to the first data pipeline, frame A2 is distributed to the second data pipeline, frame A3 is distributed to the third data pipeline, and frame A4 is distributed to the first data pipeline. In other words, by distributing data in frame units, a plan is made to process 10 / 3 = 3.33 data streams with one data pipeline.

[0049] 9 is a diagram showing an example of a task allocation plan corresponding to the configuration of FIG. 8. The processes of frames A1 to J1 are repeatedly assigned to the first to third data pipelines in sequence. The first processor 23a of the first data pipeline is also assigned to process frame J1 at time 20. The second processor 23b of the first data pipeline is also assigned to process frame J1 at times 40 to 45. The third processor 23c of the first data pipeline is also assigned to process frame J1 at time 50.

[0050] The frame period of data stream A is 1 / 30 fps = 33.3 ms, and each processor 23 can process 33.3 ms / 50 ms ≈ 6 frames within one period. Therefore, even under the condition that the processing capacity of one data pipeline is 100 fps, it is possible to further allocate a fourth frame J1 to data stream A.

[0051] In this way, in the second example, data stream processing is allocated on a frame-by-frame basis, allowing for more effective use of processor free time, further improving the determinism of task processing time and processor utilization. In the above example, the utilization rate of one data pipeline (= utilization rate of the second processor 23b) can be improved to 30 fps x 3.33 lines / 100 fps = 99%. In addition, the three processors corresponding to the fourth data stream can be eliminated.

[0052] (Third Example) The processing capacity of each of the first processor 23a and the third processor 23c is 200 fps, but in the second example, only up to 300 fps×⅓=100 fps is used.

[0053] Therefore, in the third method, as shown in FIG. 10, the number of first processors 23a and the number of third processors 23c are reduced to two, and 300 fps (= 10 data streams) is divided into 150 fps (= 5 data streams) and allocated to the two first processors 23a.

[0054] Since the utilization rate of the second processors 23b was improved to 99% in the second example, the number of second processors 23b is left at three. In this case, it is preferable to make the utilization rates of the three second processors 23b the same. For example, the output of one of the two first processors 23a is divided into 100 fps and 50 fps, and each is assigned to the second processors 23b in the first and second rows. The output of the other first processor 23a is divided into 50 fps and 100 fps, and each is assigned to the second processors 23b in the second and third rows.

[0055] Fig. 11 is a diagram showing an example of a task allocation plan corresponding to the configuration of Fig. 10. Comparing it with Fig. 9, it can be seen that the idle time of the processors can be used more effectively. In the above example, the utilization rates of the first processor 23a and the third processor 23c can be improved from 30 fps x 3.33 lines / 200 fps = 50% to 30 fps x 5 lines / 200 fps = 75%.

[0056] In this way, in the third example, the number of processors is reduced while the data stream processing is distributed on a frame-by-frame basis, thereby further improving the determinism of task processing times and the utilization rate of processors.

[0057] (Other Examples) It is also possible to combine the first to third examples. For example, task allocation plans such as those shown in Figs. 12 and 13 may be generated.

[0058] (Supplementary Note) The task allocation plan is generated by the generating device 4. The generating device 4 may generate the task allocation plan as is based on the task allocation plan input by the user. The generating device 4 may also autonomously generate the task allocation plan using data required to generate the task allocation plan. In this case, it is preferable to equalize the utilization rates of multiple processors in each stage and / or each column as much as possible. The generating device 4 may utilize machine learning or the like when generating the task allocation plan.

[0059] [Task Allocation Plan Generation Operation] FIG. 14 is a diagram showing a processing flow for generating a task allocation plan.

[0060] Step S11: Based on the arrival time of each data of the plurality of data streams that has been determined in advance, the generation unit 41 generates a task allocation plan for allocating the processing of each data of the plurality of data streams to multiple parallel multi-stage processors so that the processing of each data of the plurality of data streams is aggregated in one or more columns of multiple columns of multi-stage processors on a data stream basis, or is aggregated in one or more columns of multiple columns of multi-stage processors on a data data basis.

[0061] Step S12: The generating unit 41 generates a software program related to the generated task allocation plan, and the transmitting unit 42 transmits the software program to one or more processing devices 20.

[0062] [Data Stream Processing Operation] FIG. 15 is a diagram showing the processing flow of a data stream.

[0063] Step S21: Each processor 23 of the processing device 20 executes the software program, and when each piece of data from the multiple data streams arrives, it processes each piece of data sequentially based on the task allocation plan used by the execution.

[0064] [Application Examples] (Application Example 1) For example, the present invention can be applied to cases where safe driving of a vehicle is supported.

[0065] This is a use case in which the risk of collisions between vehicles and between vehicles and pedestrians at intersections is evaluated and predicted from multiple perspectives through integrated analysis of multiple cameras, and real-time feedback (through traffic lights, signs, speakers, etc.) is provided to vehicles and pedestrians to prevent accidents before they occur.

[0066] To realize such a use case, it is necessary to process everything from collecting camera footage to assessing and predicting the risk of accidents and issuing warnings to drivers and pedestrians with low and deterministic latency.

[0067] Therefore, by applying this embodiment, it is possible to prevent accidents such as right-turn collisions and head-on collisions, which frequently occur at intersections. Specifically, it is possible to reduce collisions with oncoming vehicles that occur due to a driver's miscalculation or poor visibility when turning right, or due to a driver ignoring a traffic light. It is also possible to reduce accidents that occur at intersections without traffic lights due to driver inattention or poor visibility.

[0068] (Application Example 2) The technology of this embodiment is highly effective in cases where high-speed, low-latency processing is required. The following use cases are possible application examples.

[0069] One use case is short-term power supply and demand adjustment. Power consumption data for factories, data centers, homes, etc., as well as power supply data for energy conservation and automobile batteries, are collected and analyzed (sensing and risk assessment) with high frequency and low latency, and a stable power supply is achieved by adjusting the demand and supply in response to adjustment requests from the grid based on the results of near-future power supply and demand forecasts.

[0070] One use case is low-latency trading, where automated trading systems in financial markets (market participant systems that receive data from the market, analyze the received data, automatically generate buy and sell orders, and send the generated orders to the exchange system) perform real-time analysis using various ever-changing market data in order to improve operational performance.

[0071] [Effects] According to this embodiment, the generation unit 41 of the generation device 40 generates a task allocation plan for allocating tasks to multiple parallel multi-stage processors based on the arrival time of each piece of data in multiple data streams that is determined in advance, so that the processing of each piece of data in the multiple data streams is aggregated on a data stream basis to one or more columns of multiple columns of multi-stage processors, or is aggregated on a data unit basis to one or more columns of the multiple columns of multi-stage processors. This makes it possible to achieve both the determinism of task processing time and improved processor utilization.

[0072] Furthermore, according to this embodiment, the generation unit 41 of the generation device 40 generates the task allocation plan so that the utilization rates of the multiple processors are the same, thereby further improving the determinism of task processing times and processor utilization rates.

[0073] [Others] The present disclosure is not limited to the above-described embodiments, and various modifications are possible within the scope of the gist of the present disclosure.

[0074] The generation device 40 of the present embodiment described above can be realized, for example, by using a general-purpose computer system including a CPU 901, a memory 902, a storage 903, a communication device 904, an input device 905, and an output device 906, as shown in Fig. 16. The memory 902 and the storage 903 are storage devices. In this computer system, the CPU 901 executes a predetermined program loaded onto the memory 902, thereby realizing each function of the generation device 40.

[0075] The generating device 40 may be implemented by one computer. The generating device 40 may be implemented by multiple computers. The generating device 40 may be a virtual machine implemented on a computer.

[0076] The program for the generation device 40 can be stored in a computer-readable recording medium such as a HDD, SSD, USB memory, CD, or DVD. The computer-readable recording medium is, for example, a non-transitory recording medium. The program for the generation device 40 can also be distributed via a communication network.

[0077] REFERENCE SIGNS LIST 1 System 10 Camera 20 Processing device 21 NIC 22 Memory 23a First processor 23b Second processor 23c Third processor 30 User terminal 40 Generation device 41 Generation unit 42 Transmission unit 43 Storage unit 50 Communication network 901 CPU 902 Memory 903 Storage 904 Communication device 905 Input device 906 Output device

Claims

1. A generating device comprising: a generating unit that generates a task allocation plan for allocating tasks to a plurality of parallel multi-stage processors based on the arrival time of each piece of data in a plurality of data streams that has been determined in advance, so that the processing of each piece of data in the plurality of data streams is aggregated in units of data streams to one or more columns of a plurality of multi-stage processors, or is aggregated in units of data to one or more columns of a plurality of multi-stage processors.

2. The generating device according to claim 1, wherein the generating unit generates the task allocation plan so that the utilization rates of the plurality of processors are the same.

3. A generation method performed by a generation device, which generates a task allocation plan to allocate tasks to multiple parallel multi-stage processors based on the arrival time of each piece of data in multiple data streams that has been determined in advance, so that the processing of each piece of data in the multiple data streams is aggregated in units of data streams to one or more columns of multiple columns of multi-stage processors, or is aggregated in units of data to one or more columns of multiple columns of multi-stage processors.

4. A generating program that causes a computer to function as the generating device according to claim 1 or 2.

Images