Information processing device

Abstract

To reduce time required for processing using a convolution neural network.SOLUTION: A control device of an information processor includes: a data acquisition section that acquires data D indicating a detection result from a sensor group at a predetermined cycle; and a processing section that processes the data from the data acquisition section by a CNN. The processing section acquires the data D(data D1, D2, ...) from the data acquisition section at each predetermined cycle. Assuming that the pieces of data D1 to D10 are required for one processing using CNN and a part of the processing for a convolution layer can be executed when the pieces of data D1 to D3 are gathered, the processing section executes an executable task using the pieces of data D1 to D3 at a time point of acquiring the data D3 without waiting for the data D1 to D10 to be gathered. Then, the processing section sequentially executes executable processing each time the data D is acquired.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device that uses a convolutional neural network. [Background technology]

[0002] International Publication No. 2020 / 194465 (Patent Document 1) discloses a neural network circuit that divides a convolution operation into a convolution operation in the spatial direction and a convolution operation in the channel direction and executes them separately. This neural network circuit includes a 1×1 convolution operation circuit that performs convolution in the channel direction, an SRAM that stores the operation result of the 1×1 convolution operation circuit, and an N×N convolution operation circuit that performs convolution in the spatial direction on the operation result stored in the SRAM. By storing the operation result of the 1×1 convolution operation circuit in the SRAM, the memory bottleneck of the N×N convolution operation circuit is avoided. The memory bottleneck refers to the time required to read data required for one convolution operation from memory exceeding the time required for one convolution operation (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2020 / 194465 Summary of the Invention [Problem to be solved by the invention]

[0004] It is known that when a Convolutional Neural Network (CNN) is used, the amount of calculation required becomes enormous. For example, in embedded systems, etc., a CPU (such as a microcomputer) with lower processing power than a general-purpose CPU (Central Processing Unit) may be used from the viewpoint of cost reduction. When a CNN is used with a CPU with low processing power, there is a possibility that the processing will take a long time. Therefore, a method for shortening the time required for processing using CNN is desired.

[0005] The present disclosure has been made to solve the above-mentioned problems, and its purpose is to reduce the time required for processing using a convolutional neural network. [Means for solving the problem]

[0006] The information processing device according to the present disclosure uses a convolutional neural network. The information processing device includes a data acquisition unit that acquires input data, and a processing unit that processes the input data by the convolutional neural network. The processing unit includes a convolutional layer that performs a convolutional process on the input data, a pooling layer that performs a pooling process on the output of the convolutional layer, and a fully connected layer that performs a fully connected process with the output of the pooling layer as an input. When an input that can execute at least a part of the processes included in the convolutional process, the pooling process, and the fully connected process is obtained, the processing unit executes the part of the processes.

[0007] According to the above configuration, when inputs for executing at least a part of the processes included in the convolution process, the pooling process, and the full connection process are available, the processing unit executes the part of the processes. That is, the processing unit does not wait until all inputs for executing the processes using a convolutional neural network (CNN) (convolution process, pooling process, and full connection process) are available before executing the processes, but executes the processes that can be executed at any time. Since data acquisition and the process using CNN are executed in parallel, the time required for the process can be shortened compared to the case where the process is executed after all inputs for executing the process using CNN are available. Effect of the Invention

[0008] According to the present disclosure, it is possible to reduce the time required for processing using a convolutional neural network. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an information processing device according to an embodiment. [Diagram 2] FIG. 2 is a diagram for explaining a detailed configuration of a processing unit. [Diagram 3] FIG. 1 is a diagram for explaining an overview of conventional processing using CNN. [Figure 4] FIG. 2 is a diagram for explaining an overview of a process using CNN executed by a processing unit according to an embodiment. [Diagram 5] This is a diagram (part 1) to explain the processing of the convolution layer. [Figure 6] This is a diagram (part 2) to explain the processing of the convolution layer. [Figure 7] FIG. 13 is a diagram for explaining the processing of a fully connected layer. [Figure 8] FIG. 11 is a diagram illustrating a processing time for each task in a comparative example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference characters and their description will not be repeated.

[0011] 1 is a diagram showing a configuration of an information processing device 1 according to the present embodiment. The information processing device 1 according to the present embodiment processes input data using a convolutional neural network (CNN) and outputs a classification result (processing result) for the input data. The information processing device 1 includes a control device 2, a storage device 3, and a communication device 4.

[0012] The storage device 3 includes, for example, a read only memory (ROM) and a random access memory (RAM). The storage device 3 stores programs and the like executed by the control device 2. The communication device 4 is configured to be capable of two-way communication with an external device.

[0013] The control device 2 includes a data acquisition unit 21, a processing unit 23, and an output unit 25. The control device 2 executes a program stored in the storage device 3 to function as the data acquisition unit 21, the processing unit 23, and the output unit 25. The control device 2 may include a memory (not shown) that stores the above-mentioned program.

[0014] The data acquisition unit 21 acquires data indicating the detection results from the sensor group 10. In this embodiment, the sensor group 10 includes first to fifth sensors 5 to 9. Any sensors can be applied as the first to fifth sensors 5 to 9. When the information processing device 1 is used in, for example, an engine ECU (Electronic Control Unit) that controls a vehicle engine, the sensor group 10 may include an outside air temperature sensor, an air pressure sensor, an engine (radiator) water temperature sensor, an engine oil temperature sensor, a fuel temperature sensor, a fuel pressure sensor, and / or an A / F sensor. The data acquisition unit 21 acquires data indicating the detection results from each of the first to fifth sensors 5 to 9 at a predetermined cycle, and outputs the acquired data to the processing unit 23.

[0015] The processing unit 23 uses the CNN to process the data (input data) received from the data acquisition unit 21, and outputs to the output unit 25 the classification result (processing result) for the input data.

[0016] FIG. 2 is a diagram for explaining a detailed configuration of the processing unit 23. The processing unit 23 includes convolution layers 231, 232, pooling layers 233, 234, and a fully connected layer 235. The convolution layers 231, 232 and the pooling layers 233, 234 extract features from input data. The convolution layers 231, 232 execute a convolution process using a filter (kernel) of a predetermined size. The pooling layers 233, 234 execute a pooling process that two-dimensionally reduces the convolution result to extract a valid value. Note that FIG. 2 shows an example in which two convolution layers 231, 232 and two pooling layers 233, 234 are included in the processing unit 23, but the number of these layers can be changed as appropriate.

[0017] The fully connected layer 235 includes an input layer, an intermediate layer, and an output layer. The input layer is composed of a plurality of units. The output of the pooling layer 234, which has been converted to one dimension, is input to each unit.

[0018] The intermediate layer is composed of multiple layers. FIG. 2 illustrates an example in which the number of layers in the intermediate layer is two, but the number of layers in the intermediate layer can be changed as appropriate. Each layer in the intermediate layer is composed of multiple units. Each unit is connected to each unit in the previous layer and each unit in the next layer. Each unit multiplies each output value from each unit in the previous layer by a weight and accumulates the multiplication results. Next, each unit adds (or subtracts) a predetermined bias to each of the accumulation results, inputs the addition result (or subtraction result) to a predetermined activation function (e.g., a ramp function or a sigmonite function), and outputs the output value of the activation function to each unit in the next layer.

[0019] The output layer is composed of one or more units. The number of units in the output layer can be changed as appropriate. Each unit in the output layer is connected to each unit in the final layer of the intermediate layer. Each unit in the output layer receives an output value from each unit in the final layer of the intermediate layer, multiplies each output value by a weight, and accumulates the multiplication results. The multiplication results are input to a predetermined activation function (e.g., a ramp function or a sigmonite function). The output value of the activation function indicates, for example, a probability.

[0020] Generally, in processing using CNN, the first processing (processing of the convolutional layer in this embodiment) is started when all data required for one processing using CNN, that is, data for executing processing of the convolutional layer, the pooling layer, and the fully connected layer, is collected. For example, when all data from the sensor group required for executing processing of the convolutional layer is collected, processing of the convolutional layer is executed. Also, when all data from the convolutional layer required for executing processing of the pooling layer is collected, processing of the pooling layer is executed. Also, when all data from the pooling layer required for executing processing of the fully connected layer is collected, processing of the fully connected layer is executed.

[0021] Here, an execution period is set for the task executed by the processing unit 23 of the control device 2 according to the present embodiment. In the following, the execution period of the task is also referred to as a "task period." The task period is set to, for example, several ms to several tens of ms. Depending on the processing capacity of the processing unit 23, a series of processes (jobs) from acquisition of input data to output of results may take more than the task period. In such a case, the process is divided into a range that can be calculated in the task period, and the series of processes are divided and executed. However, if the series of processes (jobs) are divided and executed for each layer, and if the process of each layer is executed after the input required for executing the process of each layer is obtained, the time required for the process will be long overall. Furthermore, if it takes a certain amount of time to acquire input data from the sensor group 10, the time required for the process will be even longer.

[0022] Fig. 3 is a diagram for explaining an outline of a conventional process using CNN. Fig. 4 is a diagram for explaining an outline of a process using CNN executed by a processing unit 23 according to the present embodiment. In Figs. 3 and 4, it is assumed that data D1 to D10 from a sensor group 10 are required for one process using CNN. The sensor group 10 outputs data Dn at predetermined intervals, and a data acquisition unit 21 of the control device 2 acquires the data Dn. n is a natural number. The data Dn includes detection values ​​detected by the first to fifth sensors 5 to 9.

[0023] With reference to FIG. 3, in the conventional processing using CNN, first, the data D1 to D10 are waited for to be collected. It takes 10 cycles until the data D1 to D10 are collected. When the data D1 to D10 are collected, the processing is divided for each layer to a range that can be calculated in one task cycle. For example, the processing is divided for each layer or for each channel. As shown in FIG. 2, in the example where the CNN includes two convolution layers, for example, the processing is divided into the processing of the first layer and the processing of the second layer. When the processing is divided in this way, the processing load differs for each task, and there is a possibility that the processing time differs. For example, assuming that the first layer of the convolution layer has 8 filters, the second layer has 4 filters, and the calculation of one filter takes 1 ms, the first task takes 8 ms of processing time, and the second task takes 4 ms of processing time. In this way, when the processing time required for each task differs, when a task that requires a processing time close to the task cycle is executed, a case may occur where the processing cannot be completed within the task cycle due to some factor. Furthermore, in an embedded system, for example, it is desirable to equalize the processing load in each task period.

[0024] With reference to FIG. 4, the processing unit 23 according to this embodiment does not wait until all the data D1 to D10 are available, but executes tasks that can be executed using the data available at that time in sequence. For example, assuming a case where part of the processing of the convolutional layer 231 can be executed when the data D1 to D3 are available, the processing unit 23 executes tasks that can be executed using the data D1 to D3 at the time when the data D1 to D3 are available. Then, when the data D4 is acquired in the next control cycle (predetermined cycle), the processing unit 23 executes tasks that can be executed using the data D1 to D4. The processing unit 23 executes tasks that can be executed each time data is added, and executes tasks that can be executed regardless of whether the processing is in the convolutional layer, the pooling layer, or the fully connected layer. In other words, even if the processing in the convolutional layer is not completed, if there is a task that can be executed in the pooling layer or / and the fully connected layer, the task is executed. The processing in the convolutional layer, the pooling layer, and the fully connected layer is executed in parallel.

[0025] Furthermore, the processing unit 23 according to the present embodiment divides the processing based on the processing load, not on the unit of layers or channels. More specifically, the processing unit 23 divides the processing so as to level the processing load of one task. Using the example of FIG. 3 again, the processing is divided so that up to six filters in the first layer of the convolution layer are processed in the first task, and two filters in the first layer and four filters in the second layer are processed in the second task. By purposely limiting the processing to up to six filters in the first task, the processing time of each task becomes 6 ms, and the processing load between tasks is leveled. By leveling the processing load between tasks, it is possible to stabilize the system, for example, in an embedded system. The processing load that is the target for leveling the processing load between tasks is also referred to as the "target processing load". By appropriately setting the target processing load for the task cycle, it is possible to suppress the occurrence of a case where the processing is not completed within the task cycle.

[0026] 5 and 6 are diagrams for explaining the processing of the convolution layer. In FIG. 5 and FIG. 6, an example in which a 3×3 filter is used will be explained. In addition, the slide is 1 and there is no padding.

[0027] At time T1, the data acquiring unit 21 acquires data D1 from the sensor group 10. The data D1 includes the detection results of the first to fifth sensors 5 to 9. At this point in time, no processing can be started, so the processing unit 23 waits for further data to be collected. Similarly, at time T2, the data acquiring unit 21 acquires data D2 from the sensor group 10. At this point in time, no processing can be started, so the processing unit 23 waits for further data to be collected. The interval between time T1 and time T2 is the above-mentioned predetermined period.

[0028] At time T3, the data acquisition unit 21 acquires data D3 from the sensor group 10. When the data D3 is acquired, the processing unit 23 executes the processing of the convolution layer 231 (first layer) since the processing unit 23 can execute the processing of the convolution layer 231 using the input data (data D1 to D3) that have been prepared up to that point. That is, at time T3, the processing unit 23 calculates the top row of the first layer buffer.

[0029] At time T4, the data acquisition unit 21 acquires data D4 from the sensor group 10. When the data D4 is acquired, the processing unit 23 calculates the next row of the first layer buffer.

[0030] At time T5, the data acquisition unit 21 acquires data D5 from the sensor group 10. When the data D5 is acquired, the processing unit 23 calculates the third row of the first layer buffer. Furthermore, at this point in time, processing of the convolution layer 232 (second layer) can be executed. The processing unit 23 calculates the first row of the second layer buffer. In this manner, the processing unit 23 sequentially processes the processes that can be executed after time T5.

[0031] In addition, similarly to the processing of the convolutional layer described above, the processing of the pooling layers 233 and 234 can also be executed. The processing unit 23 does not wait for all of the processing of the convolutional layers 231 and 232 to be completed, and also in the processing of the pooling layers 233 and 234, executes the processing that can be executed in order.

[0032] FIG. 7 is a diagram for explaining the processing of the fully connected layer. As an example, it is assumed that the input shown in FIG. 7 is input from the pooling layer 234 to the fully connected layer 235. In the input layer of the fully connected layer 235, the input is made one-dimensional using, for example, a flatten function. At this point (the point at which the input in FIG. 7 is received and made one-dimensional), the weight coefficients can be read out and the top row of the first layer of the intermediate layer can be processed. The weight coefficients are read out, for example, from the storage device 3. In this way, even in the fully connected layer 235, executable processes can be executed in order before all inputs are collected.

[0033] The processing unit 23 divides the currently executable processes into processing loads that can be completed within a predetermined time, and executes the divided processes in one task.

[0034] The output unit 25 outputs a classification result based on the results (probabilities) received from the output layer of the processing unit 23. For example, the output unit 25 outputs the highest probability among the probabilities received from the output layer as the classification result.

[0035] Fig. 8 is a diagram showing the processing time of each task in the comparative example. The horizontal axis of Fig. 8 shows the execution task number, and the vertical axis shows the processing load (processing time). The comparative example is a case where the conventional processing described in Fig. 3 is executed.

[0036] The tasks from 0 to N1 are the tasks immediately after the start of processing, and acquire the input data for executing the processing of the convolutional layer. Therefore, the processing load of the tasks from 0 to N1 is small. As the execution task number increases, that is, as the processing progresses, the amount of data to be processed increases and the processing load also increases. In the tasks after N3, there is a difference in processing load between the tasks with odd numbers and the tasks with even numbers. The processing load of the tasks with odd numbers is X1, and the processing load of the tasks with even numbers is y1 (<X1). The difference in processing load between the tasks with odd numbers and the tasks with even numbers can occur, for example, due to filter settings or pooling processing settings. Thus, the occurrence of a difference in processing load between tasks is not desirable from the viewpoint of system stabilization and the like.

[0037] On the other hand, in the present embodiment, since the processing is divided so that the processing load of task 1 becomes the target processing load, the processing load between tasks can be equalized.

[0038] As described above, in the information processing apparatus 1 according to the present embodiment, the processing unit 23 of the control apparatus 2 does not wait for all the input data required for the processing using the CNN to be prepared, but executes the processing when the input data for executing a certain task is prepared. As a result, the time required for the processing can be shortened compared to the case where the processing is started after all the data required for the processing using the CNN is prepared (executing the first task).

[0039] Further, the processing unit 23 divides the processing so that the processing load of task 1 becomes the target processing load. For example, if the processing is divided in units of layers or channels, variations in the processing load between tasks will occur. By dividing the processing based on the processing load instead of in units of layers or channels, the processing load between tasks can be equalized.

[0040] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is defined by the claims, not by the description of the embodiments described above, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0041] 1 information processing device, 2 control device, 3 storage device, 4 communication device, 5 first sensor, 6 second sensor, 7 third sensor, 8 fourth sensor, 9 fifth sensor, 10 sensor group, 21 data acquisition unit, 23 processing unit, 25 output unit, 231, 232 convolutional layer, 233, 234 pooling layer, 235 fully connected layer.

Claims

1. An information processing device using a convolutional neural network, A data acquisition unit for acquiring input data; a processing unit that processes the input data by the convolutional neural network; The processing unit includes: A convolution layer that performs convolution processing on the input data; A pooling layer that performs a pooling process on the output of the convolution layer; a fully connected layer that performs a fully connected process using the output of the pooling layer as an input; When an input for executing at least a part of the processes included in the convolution process, the pooling process, and the full connection process is obtained, the processing unit executes the part of the processes; The processing unit includes: Dividing the processing to be executed based on the processing load of the task executed at the determined execution cycle; An information processing device that executes the task for each divided process.

2. The information processing device described in claim 1, wherein the processing unit divides the processing to be executed so that the processing load becomes a target processing load.

Images