Calculation device and calculation method

Abstract

This computation device comprises: an importance-degree calculation unit that uses a weighting coefficient set for a channel of a multilayer neural network model which has channels made up of a plurality of neurons per each of an input layer, an intermediate layer, and an output layer to calculate a degree of importance of the channel; a target contraction rate setting unit that sets a target contraction rate in the entirety of the multilayer neural network model; a per-layer contraction rate calculation unit that calculates a contraction rate of each layer of the multilayer neural network model on the basis of the degree of importance and the target contraction rate; a contraction unit that contracts each layer to match the contraction rates calculated by the per-layer contraction rate calculation unit, and generates a post-contraction multilayer neural network model; and a retraining unit that retrains the post-contraction multilayer neural network model.

Description

[Technical Field]

[0001] The present invention relates to a computing device and a computing method. [Background technology]

[0002] In recent years, advances have been made in the application of machine learning to object recognition and behavior prediction in autonomous driving. Deep neural network (DNN) models are well known as machine learning techniques applied to object recognition and other tasks. Because DNN models are composed of many layers, they are used as an example of multi-layer neural networks.

[0003] In autonomous driving systems, DNN models are used to recognize objects around the vehicle (called "surrounding recognition"), perform automatic steering, and control the speed, thereby controlling the vehicle to its destination. Therefore, DNN models are used in the learning process, which learns the features of objects from images, and in the inference process, which extracts objects from images based on the results of the learning process.

[0004] Generally, when an autonomous driving system using a DNN model controls the autonomous driving of a vehicle, it first acquires images of the external world from a camera and converts them into a format that can be used by the DNN model. In the inference process, the converted images are used as input images, and objects are extracted from the input images using a DNN model that has undergone training in advance. The autonomous driving system then creates a surrounding map representing the vehicle's external environment from the object extraction results, creates a vehicle behavior plan based on the surrounding map, and controls the vehicle based on the behavior plan.

[0005] The surrounding area map may be, for example, a map based on an image of the area in front of the vehicle, which indicates the type of object shown in the image, such as whether the object is a vehicle or a person. However, the surrounding area map may be created using not only an image of the area in front of the vehicle, but also an image of the area behind the vehicle or an image of the side of the vehicle. Furthermore, when creating the surrounding area map, a combination of images of the surroundings captured by a camera and information obtained from sensors such as LiDAR (Light Detection and Ranging) and Radar may be used.

[0006] Patent Documents 1 and 2 explain various models using neural networks. Patent Document 1 describes a neural network including "a first group of binding information for determining an output value when an input value is given, and a second group of binding information that is a plurality of pieces of binding information whose influence on predetermined data given as an input value exceeds a predetermined value, and a third group of binding information that is created by a thinning unit by deleting at least one piece of binding information from the first group of binding information."

[0007] Patent Document 2 describes that "model compression is performed efficiently by determining scaling coefficients by learning using a cost function that takes into account the impact of deleting each channel / filter, and determining which channels / filters to delete based on at least the scaling coefficients and taking into account the impact of deleting the channels / filters." [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Publication No. 2020-42496 [Patent Document 2] Patent Publication No. 2021-108039 Summary of the Invention [Problem to be solved by the invention]

[0009] DNN models used in autonomous driving systems require a very large number of calculations because they repeatedly perform convolution operations consisting of multiplications and additions. In particular, autonomous driving requires that autonomous driving systems continually update their behavior plans within extremely short time frames, requiring high-speed calculations for object recognition using DNN models. Therefore, when implementing a DNN model in an on-board autonomous driving ECU (Electronic Control Unit), it is necessary to reduce the number of calculations. Therefore, simplifying the DNN model using reduction makes it possible to reduce the number of calculations. While there are various methods for reducing DNN models, this specification describes the use of pruning, a reduction method that thins out weight coefficients and neurons.

[0010] Previously, when a DNN model was reduced, a uniform reduction rate was determined for all layers. However, when all layers of a DNN model were reduced, layers that contribute to improving object recognition accuracy were also reduced, resulting in a significant decrease in recognition accuracy.

[0011] For this reason, methods for determining the reduction rate for each layer of a DNN model have been considered. In this method, neurons in all layers are first uniformly deleted to the extent that a deterioration in accuracy is unlikely to occur, and then a DNN model in which neurons in only one layer are significantly deleted is trained to determine the deterioration in recognition accuracy. Then, the layers from which neurons are significantly deleted are sequentially changed, and the deterioration in recognition accuracy is compared, after which only the layers that can suppress the deterioration in recognition accuracy are deleted. However, this process requires multiple training rounds, and the labor required to obtain a reduced DNN model is quite large.

[0012] When a DNN model is contracted using the technology shown in Patent Document 1, the recognition accuracy is significantly degraded. In addition, in the technology described in Patent Document 2, re-learning is repeated using a DNN model contracted based on contraction rates set in various patterns, which results in a significant increase in the amount of work required to contract the DNN model.

[0013] The present invention has been made in view of the above circumstances, and has as its object to reduce the number of steps required for contracting a multilayer neural network. [Means for solving the problem]

[0014] The arithmetic device according to the present invention is a multi-layer neural network model having channels each composed of a plurality of neurons in an input layer, an intermediate layer, and an output layer, each of which has a weight coefficient set for the channel. Based on the weighting coefficients of the subsequent channels and the importance of the subsequent channels, the previous channels connected to the subsequent channels are selected in order from the layer closest to the output layer. an importance calculation unit that calculates the importance of a channel; and a target reduction rate setting unit that sets a target reduction rate for the entire multilayer neural network model; Based on the importance of each channel in each layer, the target reduction rate of the entire multi-layer neural network model is calculated. Each layer the goal A layer reduction ratio calculation unit that calculates a reduction ratio, and a layer reduction ratio calculation unit that calculates a reduction ratio. the goal The system includes a reduction unit that reduces each layer according to the reduction rate to generate a reduced multilayer neural network model, and a re-learning unit that re-learns the reduced multilayer neural network model. [Effects of the Invention]

[0015] According to the present invention, it is possible to reduce the number of steps required to reduce a multilayer neural network. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a block diagram showing an example of the overall configuration of an autonomous driving system according to a first embodiment of the present invention. [Figure 2] 1 is a block diagram showing an example of the hardware configuration of a computer according to a first embodiment of the present invention. [Figure 3] 1 is a flowchart illustrating an example of processing of a DNN contraction device according to a first embodiment of the present invention. [Figure 4] FIG. 1 is a simplified diagram illustrating a trained DNN model according to a first embodiment of the present invention. [Figure 5] FIG. 1 is a diagram showing a basic form of a convolutional neural network according to a first embodiment of the present invention. [Figure 6]FIG. 1 is a diagram illustrating an example of a reduced DNN model according to the first embodiment of the present invention. [Figure 7] FIG. 10 is a block diagram illustrating a configuration example of an autonomous driving system including a DNN reduction device according to a second embodiment of the present invention. [Figure 8] 10 is a flowchart illustrating an example of processing of a DNN contraction device according to a second embodiment of the present invention. [Figure 9] FIG. 10 is a block diagram showing an example of the configuration of an autonomous driving system including a DNN reduction device according to a third embodiment of the present invention. [Figure 10] 10 is a flowchart illustrating an example of processing of a DNN contraction device according to a third embodiment of the present invention. [Figure 11] FIG. 10 is a block diagram illustrating a configuration example of an autonomous driving system including a DNN reduction device according to a fourth embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functions or configurations are designated by the same reference numerals, and redundant description will be omitted. In the following embodiments, an example will be described in which the present invention is applied to a computing device with which an in-vehicle ECU for vehicle control, for example, an advanced driver assistance system (ADAS) or autonomous driving (AD), can communicate.

[0018] [First embodiment] <Overall configuration of the autonomous driving system> First, a configuration example of an autonomous driving system 1 using a DNN reduction device 100 according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing an example of the overall configuration of an autonomous driving system according to the first embodiment.

[0019] The autonomous driving system 1 is composed of a DNN reduction device 100 and an autonomous driving ECU 300 mounted on a vehicle 500, and controls the surrounding recognition and automatic steering of the vehicle 500, and controls the autonomous driving of the vehicle 500 to the destination by automatic speed control. The vehicle 500 is also mounted with a camera 200 and an actuator 400. The autonomous driving ECU 300 controls the autonomous driving of the vehicle 500 using a reduced DNN model 40 that the DNN reduction device 100 has reduced from the trained DNN model 10.

[0020] Camera 200 is an example of a monocular camera or stereo camera that can capture images of the outside world of vehicle 500 using visible light and infrared rays. Camera 200 outputs images of the outside world to autonomous driving ECU 300 as outside world information.

[0021] The actuator 400 is a device that is driven under the control of the automatic driving ECU 300, and operates various parts such as the accelerator, brake, and steering.

[0022] First, an example of the internal configuration and operation of the autonomous driving ECU 300 will be described. The autonomous driving ECU 300 includes an external environment recognition unit 310, a risk prediction unit 320, and an action planning unit 330. The functional units of the autonomous driving ECU 300 shown in Fig. 1 are the main units required for autonomous driving of the vehicle 500, and explanations and illustrations of other functional units required for autonomous driving will be omitted.

[0023] The external environment recognition unit 310 recognizes objects in the external world based on external environment information acquired from the camera 200. The objects recognized by the external environment recognition unit 310 are output as object recognition results to the risk prediction unit 320. The external environment recognition unit 310 uses a reduced DNN model 40 output from the DNN re-learning unit 150, which will be described later, during the external world recognition process.

[0024] The risk prediction unit 320 predicts possible risks from the current external situation using the object recognition results input from the external environment recognition unit 310. The risks predicted by the risk prediction unit 320 are output to the action planning unit 330 as risk prediction results.

[0025] The behavior planning unit 330 generates a behavior plan including the traveling direction, traveling speed, etc. of the vehicle 500 based on the risk prediction result input from the risk prediction unit 320. The behavior plan generated by the behavior planning unit 330 is output to the actuator 400. The actuator 400 realizes autonomous driving of the vehicle 500 by driving each part of the vehicle 500 in accordance with the behavior plan.

[0026] Next, an example of the internal configuration of the DNN contracting device 100 will be described. It is assumed that the DNN reduction device 100 is provided on a cloud server installed at a location remote from the vehicle 500. The autonomous driving ECU 300 is configured to download the reduced DNN model 40 from the DNN reduction device 100 every time the autonomous driving ECU 300 starts up. By providing the DNN reduction device 100 on the cloud server, the load of reduction processing of the trained DNN model 10 and the like can be borne by the cloud server.

[0027] The DNN reduction device 100 is a device that performs reduction processing of a trained DNN model 10. The reduction processing of a trained DNN model 10 requires a large amount of calculation processing. This DNN reduction device 100 includes an importance calculation unit 110, a target reduction rate setting unit 120, a reduction condition setting unit 130, a DNN reduction unit 140, and a DNN re-learning unit 150. In the following diagrams, rectangles surrounded by solid lines represent various processing entities, and rectangles surrounded by dashed lines represent various data or information.

[0028] The importance calculation unit 110 and the target reduction rate setting unit 120 are connected to the layer reduction rate calculation unit 131 . The importance calculation unit (importance calculation unit 110) calculates the importance of each channel using a weighting coefficient 20 set for each channel of a multilayer neural network model (trained DNN model 10) having channels each composed of a plurality of neurons in an input layer, an intermediate layer, and an output layer. This multilayer neural network model is a deep neural network model and is used as the trained DNN model 10. For this purpose, the importance calculation unit 110 extracts the weighting coefficient 20 of each channel (hereinafter abbreviated as "ch") from the trained DNN model 10. Then, the importance calculation unit 110 calculates the importance of each ch using the weighting coefficient 20 of each ch. The importance is an index representing the influence of each ch of the trained DNN model 10 on the recognition accuracy. The importance of each ch calculated by the importance calculation unit 110 is output to the layer reduction rate calculation unit 131.

[0029] Here, ch in the trained DNN model 10 will be described with reference to Figures 4 and 5. FIG. 4 is a schematic diagram of a trained DNN model 10. The trained DNN model 10 is an example of a multi-layer neural network, and is composed of an input layer, multiple hidden layers, and an output layer. Each layer of the trained DNN model 10 is composed of a neural network including multiple neurons. A neuron in a hidden layer is connected to multiple neurons in the previous layer and multiple neurons in the subsequent layer.

[0030] FIG. 5 shows the basic form of a convolutional neural network. The input image data (containing the number "7") is divided into pieces of a specified size and input to the input layer. The value of a neuron in a given layer is convolved with the values ​​of multiple neurons in the previous layer using a filter, and the result is input to multiple convolutional layers for each filter. A pooling process is then performed on each convolutional layer to generate a pooling layer. Here, one cell in the pooling layer is called a neuron, and a set of neurons in the pooling layer is called a channel (ch). The output of the pooling layer is combined into a fully connected layer. The output layer, connected to the fully connected layer, outputs the probability of determining "0" to "9." The eighth black circle from the top of the output layer indicates that the determined number is most likely to be "7." Note that in the trained DNN model 10 shown in Figure 4, each intermediate layer has multiple pairs of convolutional layers and pooling layers.

[0031] Returning to FIG. 1 , the explanation will be continued. The target reduction rate setting unit (target reduction rate setting unit 120) sets a target reduction rate for the entire multilayer neural network model (trained DNN model 10). For example, the target reduction rate setting unit 120 sets the percentage of channels to be reduced for the entire trained DNN model 10, rather than for each layer of the trained DNN model 10. Therefore, the target reduction rate setting unit 120 sets the target reduction rate based on the target reduction rate 30 input in advance by the user. The target reduction rate set by the target reduction rate setting unit 120 is output to the each layer reduction rate calculation unit 131.

[0032] The contraction condition setting unit 130 sets conditions for contracting the DNN based on conditions such as the weighting coefficient 20 and the number of operations. The contraction condition setting unit 130 includes a layer contraction rate calculation unit 131.

[0033] The each layer reduction rate calculation unit (each layer reduction rate calculation unit 131) calculates the reduction rate of each layer of the multilayer neural network model (trained DNN model 10) based on the importance and the target reduction rate. This each layer reduction rate calculation unit 131 is connected to the DNN reduction unit 140. The each layer reduction rate calculation unit 131 calculates the reduction rate of each layer based on the importance calculated by the importance calculation unit 110 and the reduction rate of the entire trained DNN model 10 set by the target reduction rate setting unit 120 so that the trained DNN model 10 has the target reduction rate. For example, the each layer reduction rate calculation unit 131 calculates the reduction rate of each layer using the importance of each channel and the set target reduction rate as input. The reduction rate of each layer calculated by the each layer reduction rate calculation unit 131 is output to the DNN reduction unit 140.

[0034] The reduction unit (DNN reduction unit 140) reduces each layer in accordance with the reduction rate calculated by the each layer reduction rate calculation unit (each layer reduction rate calculation unit 131) to generate a reduced multilayer neural network model (reduced DNN model 40). This DNN reduction unit 140 is connected to the DNN re-learning unit 150. The trained DNN model 10 and the reduction rate of each layer calculated by the each layer reduction rate calculation unit 131 are input to the DNN reduction unit 140. Then, the DNN reduction unit 140 reduces the trained DNN model 10 in accordance with the reduction rate of each layer set by the each layer reduction rate calculation unit 131. For example, if 10% is set as the reduction rate of a certain layer of the trained DNN model 10, reduction is performed to remove 10% of the channels of this layer. The trained DNN model 10 contracted by the DNN contraction unit 140 is output to the DNN re-training unit 150 as a contracted DNN model 40.

[0035] The re-learning unit (DNN re-learning unit 150) re-learns the reduced multilayer neural network model (reduced DNN model 40). This DNN re-learning unit 150 is connected to the external environment recognition unit 310 of the autonomous driving ECU 300. The reduced DNN model 40 is input to the DNN re-learning unit 150 from the DNN re-learning unit 140. Here, the reduced DNN model 40 input to the DNN re-learning unit 150 is not shown. Then, the DNN re-learning unit 150 re-learns the input reduced DNN model 40. For example, the DNN re-learning unit 150 reads training data (not shown) to re-learn the reduced DNN model 40, and reads evaluation data (not shown) to evaluate the reduced DNN model 40. Then, if the DNN re-learning unit 150 can confirm that the object recognition accuracy of the reduced DNN model 40 is equal to or greater than the target recognition accuracy, it outputs the reduced DNN model 40 to the external environment recognition unit 310 of the autonomous driving ECU 300. In this way, the DNN re-learning unit 150 only needs to perform the process of relearning the reduced DNN model 40 at least once.

[0036] The DNN contraction device 100 may be mounted on the vehicle 500. In this case, even if the vehicle 500 is in an environment where it cannot connect to a wireless network (such as a tunnel or underground road), the contracted DNN model 40 generated by the DNN contraction device 100 and retrained is immediately input to the external environment recognition unit 310. Therefore, even if the vehicle 500 is in an environment where it cannot connect to a wireless network, the external environment recognition unit 310 can accurately recognize objects.

[0037] <Computer hardware configuration> Next, the hardware configuration of the computer 600 that constitutes the DNN contracting device 100 will be described. 2 is a block diagram showing an example of the hardware configuration of a calculator 600. The calculator 600 is an example of hardware used as a computer capable of operating as the DNN contraction device 100 according to this embodiment. The DNN contraction device 100 according to this embodiment realizes a DNN contraction calculation method in which the functional blocks shown in FIG. 1 cooperate with each other by the calculator 600 (computer) executing a program.

[0038] The computer 600 includes a CPU (Central Processing Unit) 610, a ROM (Read Only Memory) 620, and a RAM (Random Access Memory) 630, each connected to a bus 640. The computer 600 further includes a non-volatile storage 650 and a network interface 660.

[0039] The CPU 610 reads out the program code of the software that realizes each function according to this embodiment from the ROM 620, loads it into the RAM 630, and executes it. Variables, parameters, etc. that are generated during the calculation processing of the CPU 610 are temporarily written to the RAM 630, and these variables, parameters, etc. are read out by the CPU 610 as appropriate. However, an MPU (Micro Processing Unit) may be used instead of the CPU 610. The CPU 610 realizes the processing of each functional unit that the DNN contraction device 100 has.

[0040] The nonvolatile storage 650 may be, for example, a hard disk drive (HDD), a solid state drive (SSD), a flexible disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, or a nonvolatile memory. The nonvolatile storage 650 stores an operating system (OS), various parameters, and programs for operating the computer 600. The ROM 620 and the nonvolatile storage 650 store programs, data, and the like required for the CPU 610 to operate, and are used as an example of a computer-readable non-transitory storage medium that stores programs executed by the computer 600. The nonvolatile storage 650 stores the trained DNN model 10 and the reduced DNN model 40.

[0041] The network interface 660 uses, for example, a NIC (Network Interface Card) or the like, and can transmit and receive various data between devices via a LAN (Local Area Network), a dedicated line, etc. connected to the terminals of the NIC. The DNN reduction device 100 transmits the reduced DNN model 40 to the autonomous driving ECU 300 via the network interface 660.

[0042] <Operation example of DNN reduction device> Next, the operation of the DNN reduction device 100 will be described with reference to FIG. 3. FIG. 3 is a flowchart showing an example of the process of the DNN reduction device 100 according to the first embodiment.

[0043] As described above, the DNN reduction device 100 has a pre-trained learned DNN model 10. Then, the DNN reduction device 100 acquires the weight coefficients 20 of each ch from the learned DNN model 10.

[0044] First, the importance calculation unit 110 calculates the importance of each ch (S1). Here, the procedure for calculating the importance performed by the importance calculation unit 110 will be described with reference to FIG. 4. FIG. 4 is a diagram showing a simplified learned DNN model 10.

[0045] Each layer of the intermediate layer of the learned DNN model 10 shown in FIG. 4 contains five chs. Here, in FIG. 4, one ch in the intermediate layer of the learned DNN model 10 is shown as the region 501. Next, the procedure for the importance calculation unit 110 to calculate the importance of the ch shown in the region 501 will be described.

[0046] The importance of each channel represents the degree of influence that each channel has on recognition accuracy. It is known that, among the channels in the intermediate layers, the average importance of layers closer to the input layer is higher, while the average importance of layers closer to the output layer is lower. The more important a channel is, the less likely it is to be deleted in subsequent processing. In the past, layers closer to the output layer with lower importance had fewer calculations, so almost all layers were sometimes deleted.

[0047] Therefore, the importance calculation unit 110 calculates the importance so that channels with a large variance of the weighting coefficient 20 or channels with characteristics different from other channels are given a high importance. For example, a channel with an importance of 0.9 can be said to have a high variance. Furthermore, channels with characteristics different from other channels reflect features different from other channels, so if they are deleted in later processing, it may affect recognition accuracy. Therefore, the importance calculation unit 110 calculates the importance of channels with characteristics different from other channels to be high, making these channels less likely to be deleted in later processing.

[0048] The importance calculation unit (importance calculation unit 110) calculates the importance of the preceding channels connected to the succeeding channels, starting from the layer closest to the output layer, based on the importance of the succeeding channel and the weighting coefficient 20 of each succeeding channel. In the trained DNN model 10 shown in FIG. 4, the importance of the channels succeeding the region 501 (i.e., the channels on the right side of the region 501) is 0.6, 0.9, 0.4, 0.2, and 0.1. Therefore, assuming that the weighting coefficients 20 of each channel are w1, w2, w3, w4, and w5, the importance calculation unit 110 calculates the importance of the region 501 using the following formula (1):

[0049]

number

[0050] Normally, when the weighting coefficient 20 of each channel is used in calculations, the absolute value is not used because the weighting coefficient 20 can take either a positive or negative value. However, in equation (1), whether the weighting coefficient 20 is positive or negative, if the value becomes large, it will have a large impact on the calculation and the channel will not be deleted, so the absolute value is used.

[0051] Returning to FIG. 3 again, the explanation will be continued. After step S1, the target reduction rate setting unit 120 sets a target reduction rate 30 preset by the user (S2). The target reduction rate 30 is an index that indicates how much the number of operations of the trained DNN model 10 is reduced to become a DNN model (reduced DNN model 40), and is the target reduction rate for the entire trained DNN model 10.

[0052] Next, the each-layer reduction rate calculation unit (each-layer reduction rate calculation unit 131) calculates a target reduction rate for each layer relative to the overall target reduction rate of the multilayer neural network model (trained DNN model 10) based on the importance of each layer calculated for each channel. For example, the each-layer reduction rate calculation unit 131 calculates a target reduction rate for each layer based on the importance of each channel calculated by the importance calculation unit 110 and the target reduction rate 30 set by the target reduction rate setting unit 120 (S3). The target reduction rate for each layer is an index indicating how much of each channel in each layer of the trained DNN model 10 should be reduced, and this makes it possible to reduce each layer, compared to the conventional case where only the overall target reduction rate is used.

[0053] Next, the DNN reduction unit 140 reduces the trained DNN model 10 based on the input trained DNN model 10 and the target reduction rate for each layer calculated by the each layer reduction rate calculation unit 131. The DNN re-learning unit 150 sets the reduced trained DNN model 10 as the reduced DNN model 40 and re-learns the reduced DNN model 40 (S4).

[0054] Next, the DNN re-learning unit 150 outputs the re-learned reduced DNN model 40 to the autonomous driving ECU 300 of the vehicle 500 (S5). After processing step S5, the processing of the DNN reduction device 100 ends.

[0055] Here, the process of contracting the trained DNN model 10 shown in FIG. 4 will be described. 6 is a diagram showing an example of a reduced DNN model 40. Here, a reduction process will be described in the case where the target reduction rate 30 in the trained DNN model 10 is set to 40%.

[0056] For simplicity, it is assumed that the number of operations in each intermediate layer is the same. In this case, the DNN reduction unit 140 reduces the number of channels included in the intermediate layer in order of decreasing importance. For example, the DNN reduction unit 140 reduces 6 channels from the 15 channels in the intermediate layer of the trained DNN model 10 shown in FIG. 4 in order of decreasing importance, so that the target reduction rate 30 is 40%. As a result, the reduced DNN model 40 shown in FIG. 6 is created, in which the number of channels in the intermediate layer is 9. This reduced DNN model 40 is re-trained by the DNN re-training unit 150 and then output to the external environment recognition unit 310.

[0057] In the DNN contraction device 100 according to the first embodiment described above, the channels of the trained DNN model 10 are reduced in order of decreasing importance according to the target contraction rate 30 to create the reduced DNN model 40. The DNN contraction device 100 then determines the optimal contraction rate in advance by using the importance calculated from the weighting coefficient 20 of each channel in the importance analysis and the number of operations for each layer. Therefore, by contracting the trained DNN model 10 using the importance, the DNN contraction device 100 can suppress a decrease in the object recognition accuracy using the reduced DNN model 40. Furthermore, the DNN contraction device 100 only needs to retrain the reduced DNN model 40 at least once. This eliminates the need to repeatedly retrain the DNN model as in the past, thereby reducing the man-hours required to contract the trained DNN model 10.

[0058] Furthermore, the number of channels in the contracted DNN model 40 is reduced compared to the trained DNN model 10 before contraction. Therefore, the external environment recognition unit 310 can reduce the number of operations in the external environment recognition process using the contracted DNN model 40 and quickly obtain necessary information such as the external environment recognition result.

[0059] Furthermore, since the trained DNN model 10 is contracted according to the target contraction rate 30 set by the user, the trained DNN model 10 is contracted to the contraction rate intended by the user. In other words, the trained DNN model 10 is not contracted excessively, which reduces the recognition accuracy of the contracted DNN model 40, and conversely, it is not contracted at all, which increases the number of calculations by the autonomous driving ECU 300.

[0060] In the first embodiment, the external information is acquired from an image captured by the camera 200, but a sensor capable of acquiring the distance to an object or the type of object, such as a Lidar, Radar, or far-infrared camera, may also be used, and the means for acquiring the external information is not limited to the camera 200. Furthermore, a sensor capable of acquiring external information may be used alone, or a plurality of sensors may be used in combination.

[0061] Furthermore, the autonomous driving ECU 300 may store the external environment recognition result by the external environment recognition unit 310, the risk prediction result by the risk prediction unit 320, and the action plan by the action planning unit 330. The DNN contraction device 100 may acquire this information from the autonomous driving ECU 300, calculate a new DNN model, and use it as the trained DNN model 10.

[0062] [Second embodiment] The layer reduction rate calculation unit 131 of the DNN reduction device 100 according to the first embodiment shown in FIG. 1 calculates the reduction rate for each layer based on the importance of each channel. However, if the importance of each channel is used alone, the weight coefficients 20 of the later layers (layers close to the output layer) of the trained DNN model 10 tend to be calculated to be small, which reduces the importance of the channels in the later layers and makes the channels in the later layers more likely to be the target of reduction. However, since the later layers of the trained DNN model 10 originally have a small number of operations, it is expected that the effect of reduction will be difficult to obtain. Therefore, when the layer reduction rate calculation unit 131 calculates the reduction rate for each layer, the number of operations is also taken into consideration. As a result, it is possible to avoid only layers with a small number of operations being the target of reduction.

[0063] Therefore, a configuration example and an operation example of an autonomous driving system according to a second embodiment of the present invention will be described with reference to FIGS. 7 is a block diagram showing a configuration example of an autonomous driving system 1A including a DNN contraction device 100A according to the second embodiment. In FIG. 7, the same names and numbers are given to functional blocks that perform the same processes as in FIG. 1, and unless otherwise specified, the functional blocks have the same or similar functions and will not be described again.

[0064] An autonomous driving system 1A according to the second embodiment shown in FIG. 7 includes a DNN reduction device 100A and an autonomous driving ECU 300. The contraction condition setting unit 130A included in the DNN contraction device 100A newly includes a parameter setting unit 132 in addition to the each layer contraction rate calculation unit 131. The parameter setting unit 132 sets a parameter that contributes when the each layer contraction rate calculation unit 131 calculates the contraction rate of each layer. This parameter is also called a contribution parameter.

[0065] The parameter setting unit (parameter setting unit 132) sets the importance and the number of operations for each layer extracted from the multilayer neural network model (trained DNN model 10) as parameters. The parameter setting unit 132 receives the importance for each channel calculated by the importance calculation unit 110, as well as the number of operations (50) for each layer extracted from the trained DNN model 10. For example, in the example of the trained DNN model 10 shown in FIG. 4, the number of operations (50) for each layer is the number of connection lines for multiple channels in the next layer connected to one channel in a given layer. Since FIG. 4 shows a simplified example of the trained DNN model 10, five channels in the next layer (the middle layer of the middle layers) are connected to one channel in the layer on the input side of the intermediate layers. The number of operations between one channel in the layer on the input side of the intermediate layers and a channel in the next layer is "5." Since the number of operations between each channel in a given layer and a channel in the next layer is calculated, the total number of operations for the channels in each layer (50) is calculated as the number of operations for each layer.

[0066] Then, the parameter setting unit 132 sets parameters that determine the contribution rates of the importance and the number of operations depending on whether the recognition accuracy and calculation speed of the trained DNN model 10 achieve their respective targets. At this time, the parameter setting unit (parameter setting unit 132) sets the contribution rate of the importance calculated for each channel and the contribution rate of the number of operations 50 for each layer as parameters (contribution parameters) used by the each layer reduction rate calculation unit 131 when calculating the reduction rate. By setting the contribution rate of the importance calculated for each channel and the contribution rate of the number of operations 50 for each layer as parameters (contribution parameters), the trained DNN model 10 is reduced based on the importance and the number of operations 50 for each layer. Note that parameter setting may also be referred to as parameter adjustment. The contribution rate parameters set by the parameter setting unit 132 are input to the each layer reduction rate calculation unit 131.

[0067] The layer reduction rate calculation unit (layer reduction rate calculation unit 131) calculates the target reduction rate for each layer based on the parameter (contribution parameter) and the target reduction rate. For example, the layer reduction rate calculation unit 131 calculates the reduction rate for each layer based on the reduction rate, importance, number of operations, and contribution parameter of the entire trained DNN model 10 determined by the target reduction rate setting unit 120.

[0068] 8 is a flowchart showing an example of processing of the DNN contraction device 100 according to the second embodiment. In the processing shown in FIG. 8, the same processing as in FIG. 3 is assigned the same number, and unless otherwise specified, the description will be omitted assuming that it has the same or similar function.

[0069] 3, the DNN contracting device 100 has a trained DNN model 10 that has been trained in advance. The DNN contracting device 100 then acquires a weighting coefficient 20 for each channel from the trained DNN model 10. The importance calculation unit 110 then calculates the importance of each channel based on the weighting coefficient 20 for each channel (S1).

[0070] Next, the parameter setting unit 132 of the DNN contracting device 100 acquires the number of operations 50 for each layer in addition to the importance of each channel calculated from the weighting coefficient 20 for each channel according to the first embodiment (S11).

[0071] Next, the parameter setting unit 132 sets a contribution parameter for each channel when contracting the trained DNN model 10 based on the contribution of importance for each channel and the number of operations for each channel (S12). Then, the parameter setting unit 132 applies the set contribution parameter to the following equation (2) that calculates the contraction rate for each layer, and calculates the contraction rate for each layer (S13). For example, the parameter setting unit 132 sets the contribution rate x for importance to 0.5 and the contribution rate y for the number of operations to 0.5.

[0072]

number

[0073] In equation (2), C(L) represents the contraction rate of each layer. P(L) represents the importance of each layer. x represents the contribution of the importance. O(L) represents the number of operations in each layer (50). y represents the contribution of the number of operations. base_rate represents the target contraction rate (30).

[0074] In the above example, the importance contribution x and the number of operations contribution y are both set to 0.5, but different values ​​such as 0.3 for the importance contribution x and 0.6 for the number of operations contribution y are also possible. The initial values ​​of the importance contribution x and the number of operations contribution y can be determined in advance, for example, by the user. The processing after the reduction rate of each layer is calculated by the layer reduction rate calculation unit 131 in step S13 is the same as the processing in the DNN reduction device 100 according to the first embodiment.

[0075] In the autonomous driving system 1A according to the second embodiment described above, the importance calculation unit 110 sets contribution parameters for the importance and the number of operations using the importance of each channel analyzed using importance analysis and the number of operations 50 of each layer extracted from the trained DNN model 10. Then, the layer reduction rate calculation unit 131 calculates the reduction rate of each layer based on the contribution parameters. By using the contribution parameters for the number of operations to calculate the reduction rate, the layer reduction rate calculation unit 131 can avoid only layers with a small number of operations being targeted for reduction.

[0076] Furthermore, by using the importance contribution parameter in calculating the reduction rate, the layer reduction rate calculation unit 131 can avoid layers with high importance being the target of reduction. As a result, the DNN reduction unit 140 can reduce the trained DNN model 10 in accordance with the target reduction rate 30 by retaining channels with high importance in the trained DNN model 10 and reducing channels with a large number of operations. As a result, the autonomous driving ECU 300 can quickly execute the external environment recognition calculation process using the reduced DNN model 40, in which the overall number of operations has been significantly reduced. Furthermore, the autonomous driving ECU 300 can reliably recognize objects that should be recognized and control the driving of the vehicle 500 to avoid the objects.

[0077] [Third embodiment] When the trained DNN model 10 is contracted using the method according to the first embodiment (see FIG. 3) described above, the recognition accuracy of the contracted DNN model 40 may decrease. Similarly, when the trained DNN model 10 is contracted using the method according to the second embodiment (see FIG. 8) described above, the recognition accuracy of the contracted DNN model 40 may also decrease. Therefore, the DNN contraction device according to the third embodiment checks the recognition accuracy of the contracted DNN model 40 and adjusts the contraction rate based on the results of the confirmation of the recognition accuracy, thereby maintaining recognition accuracy that ensures safety.

[0078] Therefore, a configuration example and an operation example of an autonomous driving system according to a third embodiment of the present invention will be described with reference to FIGS. 9 and 10. FIG. 9 is a block diagram showing a configuration example of an autonomous driving system 1B including a DNN contraction device 100B according to the third embodiment. In FIG. 9, the blocks that perform the same processes as those in FIG. 7 are given the same names and numbers, and unless otherwise specified, they have the same or similar functions and will not be described again.

[0079] The autonomous driving system 1B according to the third embodiment is composed of a DNN contraction device 100B and an autonomous driving ECU 300. The DNN contraction device 100B is configured by adding a new FPS (Frames Per Second) checking unit 160 and a recognition accuracy checking unit 170 to the DNN contraction device 100A according to the second embodiment. In the following description, fps is also referred to as the calculation speed.

[0080] The DNN re-learning unit 150 is connected to the FPS checking unit 160. Therefore, the reduced DNN model 40 re-learned by the DNN re-learning unit 150 is output to the FPS checking unit 160.

[0081] The calculation speed confirmation unit (FPS confirmation unit 160) confirms the calculation speed from the re-learned, reduced multilayer neural network model (reduced DNN model 40). This FPS confirmation unit 160 is connected to the parameter setting unit 132 and the recognition accuracy confirmation unit 170. The FPS confirmation unit 160 measures the calculation speed of the reduced DNN model 40 input from the DNN re-learning unit 150 and confirms whether the calculation speed of the reduced DNN model 40 satisfies the target calculation speed. For example, the recognition accuracy confirmation unit 170 loads several tens of images with a known data volume into the reduced DNN model 40 and determines the number of images that the reduced DNN model 40 can process within a predetermined time as the calculation speed. The faster the calculation speed, the more images the reduced DNN model 40 can process per second. The confirmation result of the calculation speed of the reduced DNN model 40 by the FPS confirmation unit 160 is output to the parameter setting unit 132. In addition, the FPS confirmation unit 160 outputs the reduced DNN model 40 to the recognition accuracy confirmation unit 170.

[0082] The recognition accuracy confirmation unit (recognition accuracy confirmation unit 170) confirms the recognition accuracy of the re-learned, reduced multilayer neural network model (reduced DNN model 40). This recognition accuracy confirmation unit 170 is connected to the parameter setting unit 132 and the external environment recognition unit 310 of the autonomous driving ECU 300. The recognition accuracy confirmation unit 170 measures the recognition accuracy of the reduced DNN model 40 and confirms whether the recognition accuracy of the reduced DNN model 40 satisfies the target recognition accuracy. For example, the recognition accuracy confirmation unit 170 loads several tens of images in which objects are known in advance into the reduced DNN model 40, and determines the rate at which the reduced DNN model 40 correctly recognizes the objects as the recognition accuracy. The confirmation result of the recognition accuracy of the reduced DNN model 40 by the recognition accuracy confirmation unit 170 is output to the parameter setting unit 132. In addition, when the FPS confirmation unit 160 determines that the calculation speed of the reduced DNN model 40 satisfies the target calculation speed and the recognition accuracy confirmation unit 170 determines that the recognition accuracy of the reduced DNN model 40 satisfies the target recognition accuracy, the recognition accuracy confirmation unit 170 outputs the reduced DNN model 40 to the external environment recognition unit 310 of the autonomous driving ECU 300.

[0083] If the configuration of the reduced DNN model 40 is too simple, the calculation speed of the reduced DNN model 40 will be too fast, but the recognition accuracy may also be reduced. Therefore, the DNN reduction device 100B checks not only the calculation speed of the reduced DNN model 40 but also the recognition accuracy, thereby making it possible to confirm whether a reduced DNN model 40 with a fast calculation speed and high recognition accuracy has been obtained.

[0084] Furthermore, the calculation speed and recognition accuracy of the reduced DNN model 40 may vary depending on the device on which it is executed. For this reason, the FPS confirmation unit 160 and the recognition accuracy confirmation unit 170 confirm the calculation speed and recognition accuracy according to the type of CPU of the autonomous driving ECU 300 on which the reduced DNN model 40 is executed. For this reason, the DNN reduction device 100B can generate a reduced DNN model 40 that can achieve optimal calculation speed and recognition accuracy for each CPU of the autonomous driving ECU 300.

[0085] Then, the parameter setting unit (parameter setting unit 132) determines the contribution of the importance and the contribution of the number of operations 50 of each layer based on the comparison result between the confirmed operation speed and the target operation speed, and the comparison result between the confirmed recognition accuracy and the target recognition accuracy. In this way, the parameter setting unit 132 can set optimal parameters for generating the contracted DNN model 40 based on the comparison result of the operation speed and the comparison result of the recognition accuracy that are fed back.

[0086] 9, the processing is performed in the order of the FPS checking unit 160 and the recognition accuracy checking unit 170, but the FPS checking unit 160 and the recognition accuracy checking unit 170 may be processed in parallel. In the case of parallel processing, either the FPS checking unit 160 or the recognition accuracy checking unit 170 is configured to make the determination in step S22 shown in FIG. 10 (to be described later) and output the contracted DNN model 40 to the external environment recognition unit 310.

[0087] Next, the operation of the DNN contraction device 100B will be described. 10 is a flowchart showing an example of the processing of the DNN contraction device 100B according to the third embodiment. In addition, in FIG. 10, the same processing as in FIG. 8 is assigned the same number, and unless otherwise specified, the description will be omitted assuming that it has the same or similar function.

[0088] The processes in steps S1, S11 to S13, and S2 to S4 are the same as the processes performed by the DNN contracting device 100A according to the second embodiment. After the trained DNN model 10 is reduced and re-learned in step S4, the FPS confirmation unit 160 (an example of a calculation speed confirmation unit) measures the calculation speed (FPS) of the reduced DNN model 40 re-learned by the DNN re-learning unit 150. In addition, the recognition accuracy confirmation unit 170 measures the recognition accuracy from the DNN model re-learned by the DNN re-learning unit 150 (S21).

[0089] Next, the FPS confirmation unit 160 compares the measured calculation speed with a preset target calculation speed and determines whether the measured calculation speed has achieved the target calculation speed (target FPS). Also, the recognition accuracy confirmation unit 170 compares the measured recognition accuracy with a preset target recognition accuracy and determines whether the measured recognition accuracy has achieved the target recognition accuracy (S22).

[0090] If the calculation speed measured by the FPS checking unit 160 does not achieve the target calculation speed (NO in S22), the FPS checking unit 160 feeds back the result of the calculation speed comparison to the parameter setting unit 132. The result of the calculation speed comparison includes, for example, a numerical value of 70% of the target calculation speed. Also, if the recognition accuracy measured by the recognition accuracy checking unit 170 does not achieve the target recognition accuracy (NO in S22), the recognition accuracy checking unit 170 feeds back the result of the recognition accuracy comparison to the parameter setting unit 132. The result of the recognition accuracy comparison includes a numerical value of 80% of the target recognition accuracy.

[0091] Here, the recognition accuracy measured by the recognition accuracy confirmation unit 170 is calculated based on how accurate the reduced DNN model 40 is when, for example, sample images for which the correct answers are known are input to the reduced DNN model 40. For example, if 100 sample images are input to the reduced DNN model 40 and correct answers are obtained for 80 of the sample images, the recognition accuracy confirmation unit 170 measures the recognition accuracy of the reduced DNN model 40 to be 80%. The target accuracy is set based on the recognition accuracy required by the user. For example, if the target accuracy is 85%, the recognition accuracy of the reduced DNN model 40 is insufficient, and therefore it is necessary to re-reduce the trained DNN model 10 in processing after the parameter setting unit 132 and re-train the reduced DNN model 40. On the other hand, if the target accuracy is 75%, the recognition accuracy of the reduced DNN model 40 is sufficient, and therefore it is unnecessary to re-reduce the trained DNN model 10 and re-train the reduced DNN model 40.

[0092] In step S22, if the result is at least one of the following: the calculation speed measured by the FPS checking unit 160 does not achieve the target calculation speed; and the recognition accuracy measured by the recognition accuracy checking unit 170 does not achieve the target recognition accuracy (NO in S22), the comparison result is fed back to the parameter setting unit 132. The parameter setting unit 132 modifies the importance and contribution parameters of the number of calculations based on the comparison results fed back from the FPS checking unit 160 and the recognition accuracy checking unit 170 (S23).

[0093] In step S23, if the calculation speed does not achieve the target calculation speed, the parameter setting unit 132 sets the contribution parameter of importance to increase and the contribution parameter of the number of calculations to decrease. Also, if the recognition accuracy does not achieve the target recognition accuracy, the parameter setting unit 132 sets the contribution parameter of importance to decrease and the contribution parameter of the number of calculations to increase. Also, if the calculation speed does not achieve the target calculation speed and the recognition accuracy does not achieve the target recognition accuracy, the parameter setting unit 132 sets the contribution parameter of importance to increase and the contribution parameter of the number of calculations to increase.

[0094] As an example, assume that the calculation speed and recognition accuracy of a certain contracted DNN model 40 are measured. Here, the measured calculation speed is 200 fps, and the measured recognition accuracy is 60 pt (%). Furthermore, the target calculation speed is 220 fps, and the target recognition accuracy is 50 pt (%). At this time, the FPS confirmation unit 160 compares the measured calculation speed with the target calculation speed and determines that the measured calculation speed does not reach the target calculation speed. Furthermore, the recognition accuracy confirmation unit 170 compares the measured recognition accuracy with the target recognition accuracy and determines that the measured recognition accuracy meets the target recognition accuracy. Therefore, since the calculation speed has not reached the target, the parameter setting unit 132 modifies the parameters by increasing the importance contribution parameter from 0.5 to 0.6 and decreasing the number of operations contribution parameter from 0.5 to 0.4. Note that there is no restriction that the contributions of the importance and the number of operations must add up to 1.0.

[0095] Thereafter, the layer contraction rate calculation unit 131 again calculates the contraction rate of each layer of the DNN model, and the DNN contraction unit 140 contracts the DNN model based on the calculated contraction rate of each layer. Furthermore, the DNN re-learning unit 150 re-learns the DNN model. Then, again, the calculation speed and recognition accuracy are measured in steps S21 and S22, and the measured calculation speed and recognition accuracy are compared with the target calculation speed and target recognition accuracy.

[0096] In step S23, if the measured calculation speed achieves the target calculation speed and the measured recognition accuracy achieves the target recognition accuracy (YES in S23), the DNN re-learning unit 150 outputs the re-learned reduced DNN model 40 to the autonomous driving ECU 300 of the vehicle 500 (S5). After processing step S5, the processing of the DNN reduction device 100B ends.

[0097] In the autonomous driving system 1B according to the third embodiment described above, when the measured calculation speed satisfies the target calculation speed and the measured recognition accuracy satisfies the target recognition accuracy, the contracted DNN model 40 is output to the autonomous driving ECU 300. Therefore, the autonomous driving ECU 300 can recognize the outside world with sufficient calculation speed and recognition accuracy using the contracted DNN model 40, and no wasted processing occurs.

[0098] [Fourth embodiment] In the DNN contraction devices according to the first to third embodiments, the trained DNN model 10 is not contracted in consideration of the situation in which the vehicle 500 is placed or the situation around the vehicle 500. However, there has been a demand for generating a contracted DNN model 40 that matches the situation of the vehicle 500. For this reason, the DNN contraction device according to the fourth embodiment generates a contracted DNN model 40 that matches the situation of the vehicle 500.

[0099] Therefore, a configuration example and an operation example of an autonomous driving system according to a fourth embodiment of the present invention will be described with reference to FIG. 11 is a block diagram showing a configuration example of an autonomous driving system 1B including a DNN contraction device 100C according to the fourth embodiment. In FIG. 11, blocks that perform the same processing as those in FIG. 9 are given the same names and numbers, and unless otherwise specified, they have the same or similar functions and will not be described again.

[0100] The autonomous driving system 1C according to the fourth embodiment is configured with a DNN contraction device 100C, an autonomous driving ECU 300 of a vehicle 500, and a server 550.

[0101] For example, when the weather is fine, the image captured by the camera 200 shows an object at a long distance in front of the vehicle 500, and the external environment recognition unit 310 can recognize this object. However, when the weather is rainy, the image captured by the camera 200 shows an object at a short distance in front of the vehicle 500, which is difficult to distinguish from the background, and the external environment recognition unit 310 may not be able to recognize this object.

[0102] For this reason, when contracting the trained DNN model 10, the DNN contraction device 100C can obtain information on the status of the vehicle 500 (driving, stopped, etc.) or the status around the vehicle 500 (traffic jam, bad weather, etc.), and replace the trained DNN model 10 before contraction. This DNN contraction device 100C has a configuration in which a model receiving unit 180 and a driving status observation unit 190 are newly added to the DNN contraction device 100B according to the third embodiment.

[0103] The driving situation observation unit (driving situation observation unit 190) observes the driving situation of the vehicle using the reduced multilayer neural network model (reduced DNN model 40), and outputs the observed driving situation results to a server (server 550) that stores multiple multilayer neural network models (trained DNN models 10). This driving situation observation unit 190 is connected to the server 550, the external environment recognition unit 310, and the behavior planning unit 330 of the autonomous driving ECU 300. The driving situation observation unit 190 receives an action plan for the vehicle 500 from the behavior planning unit 330 of the autonomous driving ECU 300. Then, the driving situation observation unit 190 observes the driving situation of the vehicle 500 based on the received action plan. In addition, the driving situation observation unit 190 transmits driving situation observation results indicating the observed driving situation of the vehicle 500 to the server 550.

[0104] Furthermore, when the traveling condition observation unit 190 receives the reduced DNN model 40 re-learned by the DNN re-learning unit 150 via the FPS confirmation unit 160 and the recognition accuracy confirmation unit 170, it outputs the reduced DNN model 40 to the external environment recognition unit 310. When the reduced DNN model 40 reflecting the traveling condition of the vehicle 500 is created, the traveling condition observation unit 190 can also output the reduced DNN model 40 reflecting the traveling condition of the vehicle 500 to the external environment recognition unit 310.

[0105] The server 550 is, for example, a cloud server, and can transmit and receive various data to and from the DNN reduction device 100C via wireless or wired communication. The server 550 stores multiple DNN models in a storage device (not shown). The DNN models stored in the server 550 are prepared in advance to suit typical driving conditions of the vehicle 500.

[0106] Then, when the server 550 receives the driving situation observation result from the driving situation observation unit 190 included in the DNN contraction device 100C, the server 550 selects a DNN model to be transmitted to the model receiving unit 180 based on the received driving situation observation result. After that, the server 550 transmits the selected model to the model receiving unit 180.

[0107] The DNN model selected by the server 550 and transmitted to the model receiving unit 180 is one that has been trained in advance by a learning device (not shown). The server 550 selects a DNN model trained for each driving situation of the vehicle 500 based on the driving situation observation results received from the driving situation observing unit 190. For example, the range of the external environment that the external environment recognizing unit 310 should recognize may differ between when the vehicle 500 is driving and when the vehicle is stopped. Therefore, the server 550 selects a DNN model for when the vehicle 500 is stopped if the vehicle 500 is stopped, and selects a DNN model for when the vehicle 500 is driving if the vehicle 500 is driving, and transmits the selected DNN model to the model receiving unit 180.

[0108] The model receiving unit (model receiving unit 180) receives a multilayer neural network model (trained DNN model 10) suitable for the driving situation, selected from the server (server 550) based on the observation results of the driving situation. Then, the model receiving unit (model receiving unit 180) sets the multilayer neural network model (trained DNN model 10) received from the server (server 550) as the multilayer neural network model (trained DNN model 10) to be read by the importance calculation unit (importance calculation unit 110). Therefore, the contracted DNN model 40 is created using the trained DNN model 10 according to the driving situation of the vehicle 500.

[0109] In the autonomous driving system 1C according to the fourth embodiment described above, a DNN model according to the driving situation of the vehicle 500 is selected by the server 550 and is imported into the DNN reduction device 100C as the trained DNN model 10. The DNN reduction device 100C then performs predetermined processing on the imported trained DNN model 10 to create a reduced DNN model 40 and outputs the reduced DNN model 40 to the autonomous driving ECU 300. This allows the autonomous driving ECU 300 to accurately control the vehicle 500 using the reduced DNN model 40 created according to the driving situation of the vehicle 500.

[0110] Furthermore, the reduced DNN model 40 generated by the DNN model selected according to the driving situation of the vehicle 500 is used in the autonomous driving ECU 300. For example, the reduced DNN model 40 used while the vehicle 500 is driving requires a large number of calculations, whereas the reduced DNN model 40 used while the vehicle 500 is stopped requires a small number of calculations, which can reduce the power consumption of the autonomous driving ECU 300.

[0111] Furthermore, the traveling condition observing unit 190 may receive GPS (Global Positioning System) information received by the vehicle 500 together with the action plan from the action planning unit 330 of the autonomous driving ECU 300, and determine the current position of the vehicle 500. If the current position of the vehicle 500 is known, the traveling condition observing unit 190 can receive weather information and the like tailored to this current position, and therefore may transmit the weather information together with the traveling condition observation results to the server 550. In this case, the server 550 can select a DNN model tailored to the weather information.

[0112] As a modified example of the autonomous driving system 1C according to the fourth embodiment, the model receiving unit 180 may store a plurality of DNN models in advance, and may receive driving situation observation results from the driving situation observing unit 190. In such a configuration, the model receiving unit 180 may select a DNN model to be used as the trained DNN model 10 in the DNN contraction device 100C based on the driving situation observation results.

[0113] Furthermore, the driving condition observing unit 190 observes the driving condition of the vehicle 500 (driving or stopped) and the surrounding conditions of the vehicle 500 (presence or absence of traffic congestion, weather, etc.). For this reason, if the model receiving unit 180 is configured to select a DNN model, the driving condition observing unit 190 may directly feed back the driving condition to the model receiving unit 180 without going through the server 550. For example, the driving condition observing unit 190 can also observe the actual driving condition of the vehicle 500 using information obtained from the actuator 400, the sensor, the autonomous driving ECU 300, etc.

[0114] Furthermore, since the driving condition observing unit 190 observes the driving condition of the vehicle 500, when the vehicle 500 breaks down, information detected by each sensor of the vehicle 500 may be received from the autonomous driving ECU 300 as a driving condition observation result. In this case, a DNN model suited to the failure mode of the vehicle 500 is selected, and the DNN contracting device 100C generates the contracted DNN model 40 from the selected DNN model. Therefore, the autonomous driving ECU 300 can perform autonomous driving processing using the contracted DNN model 40 generated to suit the failure mode.

[0115] Furthermore, the server 500 can later determine whether the DNN model selected before the vehicle 500 broke down is good or bad, and can determine for itself or leave the determination to the user as to under what driving conditions the DNN model could have been appropriately selected. Then, the server 500 can select a DNN model that is less likely to cause a breakdown in the vehicle 500, and transmit the DNN model to the model receiving unit 180.

[0116] Furthermore, the DNN contraction device 100C may be mounted on the vehicle 500. In this case, the driving situation of the vehicle 500 is observed in real time by the driving situation observation unit 170. As a result, a DNN model suited to the current driving situation of the vehicle 500 is selected by the server 550, and the DNN contraction device 100C can generate the contracted DNN model 40 from this DNN model in real time. Therefore, the external environment recognition unit 310 receives the contracted DNN model 40 suited to the current driving situation of the vehicle 500 in real time, and the external environment recognition unit 310 can accurately recognize objects. As described above, if the model receiving unit 180 is configured to select a DNN model, the driving situation observation unit 190 may directly feed back the driving situation to the model receiving unit 180 without going through the server 550, and the model receiving unit 180 may select a DNN model.

[0117] The DNN contraction device according to each of the above-described embodiments has been described as an example of a calculation device for contracting a DNN. However, the calculation device according to the present invention may be applied to a device for contracting a multilayer neural network other than a DNN.

[0118] Furthermore, the present invention is not limited to the above-described embodiments, and it goes without saying that various other applications and modifications are possible without departing from the gist of the present invention as set forth in the claims. For example, the above-described embodiments have described the configuration of the device and system in detail and specifically in order to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of the embodiments described here with the configuration of other embodiments, and it is also possible to add the configuration of one embodiment to the configuration of another embodiment. Furthermore, it is also possible to add, delete, or replace part of the configuration of each embodiment with other configurations. In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and do not necessarily show all the control lines and information lines in the product. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0119] 1...Autonomous driving system, 10...Trained DNN model, 20...Weight coefficient for each channel, 30...Target reduction rate, 40...Reduced DNN model, 50...Number of operations for each layer, 100...DNN reduction device, 110...Importance calculation unit, 120...Target reduction rate setting unit, 130...Reduction condition setting unit, 131...Layer reduction rate calculation unit, 132...Parameter setting unit, 140...DNN reduction unit, 150...DNN re-learning unit, 160...FPS confirmation unit, 170...Recognition accuracy confirmation unit, 180...Model receiving unit, 190...Driving situation observation unit, 300...Autonomous driving ECU, 310...External environment recognition unit, 320...Risk prediction unit, 330...Action planning unit, 500...Vehicle

Claims

1. an importance calculation unit that calculates the importance of a preceding channel connected to a subsequent channel in order from the layer closest to the output layer based on the weight coefficient of the subsequent channel and the importance of the subsequent channel among weight coefficients set for the channels of a multilayer neural network model having channels each composed of a plurality of neurons in an input layer, an intermediate layer, and an output layer; a target reduction rate setting unit that sets a target reduction rate for the entire multilayer neural network model; a layer reduction rate calculation unit that calculates a target reduction rate for each layer with respect to the target reduction rate for the entire multilayer neural network model based on the importance in each layer of the importance calculated for each channel; a reduction unit that reduces each layer in accordance with the target reduction rate calculated by the each layer reduction rate calculation unit to generate a reduced multilayer neural network model; a re-learning unit that re-learns the multilayer neural network model after the contraction. Computing device.

2. a parameter setting unit that sets the importance and the number of operations of each layer extracted from the multilayer neural network model as parameters; The each layer reduction rate calculation unit calculates a target reduction rate for each layer based on the parameters and the target reduction rate for the entire multilayer neural network model. The computing device of claim 1 .

3. The parameter setting unit sets the contribution of the importance calculated for each channel and the contribution of the number of operations of each layer as the parameters. The computing device according to claim 2 .

4. a calculation speed confirmation unit that confirms a calculation speed from the re-trained reduced multilayer neural network model; a recognition accuracy confirmation unit that confirms the recognition accuracy of the retrained multilayer neural network model after contraction, The parameter setting unit determines the contribution of the importance and the contribution of the number of operations of each layer based on a comparison result between the confirmed operation speed and a target operation speed and a comparison result between the confirmed recognition accuracy and a target recognition accuracy. The computing device according to claim 3 .

5. a driving situation observation unit that observes a driving situation of a vehicle in which the reduced multilayer neural network model is used and outputs the observed results of the driving situation to a server that stores a plurality of multilayer neural network models; a model receiving unit that receives the multilayer neural network model suitable for the driving situation, selected from the server based on the observation result of the driving situation, and sets the multilayer neural network model received from the server as the multilayer neural network model that is read by the importance calculation unit. The computing device according to claim 4.

6. The multilayer neural network model is a deep neural network model. The computing device of claim 1 .

7. a process of calculating the importance of a preceding channel connected to a subsequent channel in order from the layer closest to the output layer based on the weight coefficient of the subsequent channel and the importance of the subsequent channel among weight coefficients set for the channels of a multilayer neural network model having channels each composed of a plurality of neurons in an input layer, an intermediate layer, and an output layer; A process of setting a target reduction rate for the entire multilayer neural network model; calculating a target reduction ratio for each layer relative to the target reduction ratio for the entire multilayer neural network model based on the importance in each layer of the importance calculated for each channel; a process of reducing each layer in accordance with the target reduction ratio to generate a reduced multilayer neural network model; and a process of re-learning the multilayer neural network model after the contraction. Calculation method.

Images