Neural network compression device and method

Abstract

When assuming the compression of a large-scale Deep Neural Network for autonomous driving, the recognition accuracy of the compressed Neural Network (NN) model is reduced due to the large number of harmful or unnecessary learning images (ineffective learning images), and the compression design period is prolonged, which is a problem. The learning image selection unit (B100) calculates the influence value on the inference, and uses the influence value to add indexes to the learning images required and unnecessary learning images in the NN compression design to generate an indexed learning image set (1004-1), and the NN is compressed in the neural network compression unit (B200) that notifies the result via the memory (B300).

Description

Technical Field

[0001] The present invention relates to compression of a neural network. In this application specification, compression refers to reduction of the amount of computation. Background Art

[0002] There are many neurons (nerve cells) in the brain of an organism, and each neuron performs the operation of inputting signals from and outputting signals to other neurons. The Deep Neural Network (DNN) is an engineering model that imitates the behavior of the biological nerve cell network to realize this brain structure with a computer.

[0003] As an example of DNN, there is a convolutional neural network (CNN) that is effective for object recognition and behavior prediction. Figure 1 This shows an example of a CNN structure. CNN consists of an input layer, one or more intermediate layers, and a multi-layer convolution operation layer called an output layer. In the Nth convolution operation layer, the value output from the N-1th layer is used as input, and the result is output to the input of the N+1th layer by convolving the input value with a weight filter. At this time, the core coefficient (weight coefficient) of the weight filter is set to an appropriate value (learning) according to the application, thereby achieving high generalization performance.

[0004] In recent years, the development of technologies that realize autonomous driving and driving assistance by installing CNN on the vehicle-mounted ECU (Electronic Control Unit) has been accelerating. The amount of computing required to realize large-scale CNN for autonomous driving is more than 100TOPs (Tera Operations). On the other hand, the computing power of the processor that can be installed in the AD-ECU (Autonomous Driving-Electronic Control Unit) 1, which is a type of vehicle-mounted ECU, is around tens of TOPS (Tera Operation per Sec), making it difficult to achieve real-time processing. Therefore, Figure 2 As shown, the amount of computation of CNN needs to be reduced (compressed). However, it takes a long time to search for compression conditions that can both suppress the reduction in recognition accuracy due to compression and adapt to the calculation capabilities of the on-board processor. From the above point of view, for large-scale CNNs for autonomous driving, existing compression design methods have difficulty in balancing (1) suppressing the reduction in recognition accuracy during compression and (2) shortening the compression design period.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent Application Publication No. 2005-100121

[0008] Patent Document 1 discloses that the type of feature quantity and recognition conditions used in the recognition process are determined based on the desired balance between recognition accuracy and the load of computational effort. A recognition accuracy determination unit and a computational effort determination unit determine a recognition accuracy index value and a computational effort index value, respectively, to select a good recognizer. Summary of the invention

[0009] Problems to be solved by the invention

[0010] Here, Figure 5 Indicates the conventional compression design process. In addition, this process is performed by a conventional neural network compression device.

[0011] First, the learning data set 1001 is accepted as input, and the initial compression conditions consisting of the compression part and the compression rate for the pre-compression neural network model (pre-compression NN model) 1003 are determined (S3). Next, the initial compression conditions are adjusted to reflect the optimal compression conditions, and the compression conditions 1005 are calculated (S4). Then, the compression processing is performed using the compression conditions 1005 (S5). Based on this, a provisional compression neural network model (provisional compression NN model) 1006 is determined. Next, re-learning (adjustment) is performed using the provisional compression NN model and the learning data set 1001 (S7). Then, it is determined whether the re-learning (S6) has ended with the last learning data set 1001 and the inference data set 1002 (S7). As a result, if it has ended, the result of the re-learning is saved as the post-compression neural network model (post-compression NN model) 1007. If it has not ended, the optimal compression condition search process is performed on the result of the re-learning (S8). Then, the result of the optimal compression condition search process, that is, the optimal compression condition, is used in S4.

[0012] When performing such processing, the following issues arise. For example, in the vehicle sensor data and the learning data set, many images that induce misrecognition are included in the inference using the compressed NN model 1007. These images are represented, for example, by images containing slight noise that is unrecognizable due to accidental or malicious third-party attacks that induce misrecognition, or images that lack generalization and map limited special environments. In previous CNN compression designs, since the learning images used for compression contain these harmful images (invalid learning images), (1) the recognition accuracy of the compressed DNN model is reduced, and (2) the compression design becomes long-term due to the large number of learning images.

[0013] Means of solving the problem

[0014] In order to solve the above-mentioned problems, in the present invention, the learning data set is classified into effective learning data that is needed or effective in the compression design and invalid learning data that is unnecessary or harmful, and based on the results of the classification, learning related to the data compression design using at least one of the effective learning data and the invalid learning data is performed.

[0015] More specifically, it is a neural network compression device that uses an input learning data set to compress a neural network, and the neural network compression device has: a learning image selection unit that calculates an influence value on an inference result, which is calculated using an inference data set and a neural network model for the learning data set, and classifies the learning data set into valid learning data required for the compression and invalid learning data not required for the compression according to the influence value, thereby generating an indexed learning data set; and a neural network compression unit that compresses the neural network model according to the indexed learning data set and the neural network model. In addition, the present invention also includes a compression method in the above-mentioned neural network compression device.

[0016] Effects of the Invention

[0017] According to the present invention, it is possible to perform repeated learning (relearning) that can improve recognition performance by reducing accuracy and learning time. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 It is a diagram showing the structure of a convolutional neural network (CNN).

[0019] Figure 2 It is a diagram showing the overall structure of an in-vehicle ECU (AD-ECU1) to which the embodiment of the present invention is applied.

[0020] Figure 3 This is a diagram for explaining the relationship between compression and recognition accuracy.

[0021] Figure 4 is a diagram for explaining the relationship between compression and compression design period.

[0022] Figure 5 This is a diagram showing a conventional compression design flow.

[0023] Figure 6 It is a diagram showing the compression design flow in each embodiment of the present invention.

[0024] Figure 7 This is a diagram for explaining the determination of initial compression conditions using the selection of learning images.

[0025] Figure 8 It is a diagram showing the effect of shortening the learning time in the embodiment of the present invention.

[0026] Fig. 9 It is a diagram showing the effect of improving recognition accuracy in the embodiment of the present invention.

[0027] Fig.10 This is an overall structural diagram of the NN compression device 121 of Example 1.

[0028] Fig.11 This is a diagram showing a schematic processing flow of Example 1.

[0029] Fig.12 1 is a diagram showing the configuration of the learning image selecting unit B100 according to the first embodiment.

[0030] Fig.13 This is a diagram showing the processing flow of the learning image selecting unit B100 according to the first embodiment.

[0031] Fig.14 This is a diagram showing the structure of the neural network compression unit B200 of Example 1.

[0032] Fig.15 This is a diagram showing the processing flow of the neural network compression unit B200 of Example 1.

[0033] Fig.16 This is a diagram showing the structure of the PCA / clustering unit B150 of Example 1.

[0034] Fig.17 This is a diagram showing the processing flow of the PCA / clustering unit B150 of the first embodiment.

[0035] Fig.18 This is a diagram showing the structure of the compression condition determination unit B210.

[0036] Fig.19 This is a diagram showing the processing flow of the compression condition determination unit B210.

[0037] Fig. 20 This is a diagram showing the configuration of the learning image selection unit B100 according to the second embodiment.

[0038] Fig.21 This is a diagram showing the processing flow of the learning image selection unit B100 according to the second embodiment.

[0039] Fig. 22 This is a diagram showing the configuration of the NN compression device 121 in the AD-ECU 1 of the third embodiment.

[0040] Fig.23 This is a diagram showing the structure of the NN compression device 121 in the AD-ECU 1 of the fourth embodiment.

[0041] Fig.24 This is a diagram showing the structure of the NN compression device 121 in the AD-ECU 1 of the fifth embodiment. DETAILED DESCRIPTION

[0042] Before describing each embodiment (1 to 5) of the present invention, Figure 6 The following describes the compression design process common to each embodiment of the present invention. In addition, this process is executed by the neural network compression device 212 described later. Figure 5 The common symbols of the symbols represent the same processing or data. Here, the same also includes completely different processing and data depending on the processing performed and the data used.

[0043] First, using the input learning data set 1001 and the inference data set 1002, the influence value of each data included in the learning data set 1001 on the inference result is estimated. Then, each learning data is classified as either valid learning data or invalid learning data, and an indexed learning data set 1004 (S1) is produced. As the generation of the indexed learning data set 1004, for example, an index is added. The assignment of the index is an assignment to at least one party. In a state where one party (for example, valid) is assigned, the learning data set that is not assigned an index can be judged as the other party (for example, invalid).

[0044] In addition, as another example, the classification result may be stored in a storage medium at an address corresponding to the result.

[0045] Next, effective learning data is selected based on the index of the indexed learning data set 1004 (S2). Then, the selected effective learning data is used to determine the initial compression condition for the pre-compression neural network model (pre-compression NN model) 1003 (S3). Here, the above-mentioned index can also be used in the initial compression condition determination (S3). Specifically, Figure 7 As shown, the compression priority of each neuron is determined based on the above index and activation state. The compression of each neuron is repeated until the compressed NN model 1007 satisfies the computational constraints set by the user in descending order of priority. Here, the activation of a neuron means that the neuron takes a non-zero value.

[0046] In summary, compression is performed to remove neurons with low effective learning images and high invalid learning images. Neurons with low effective learning images and invalid learning images are also removed. In addition, if the effective learning image is high, the compression of the neuron is shelved regardless of the activation frequency of the invalid learning image. And, it is preferred to compress the neuron according to the priority shown in this figure. As mentioned above, it is more appropriate to use a combination related to the activation frequency of neurons with effective learning images and invalid learning images for compression. In addition, the high and low here can be determined by comparison with a pre-set baseline value. And, in this figure, only the results of compression priorities 1 and 2 can be used.

[0047] Next, the initial compression condition is adjusted to reflect the optimal compression condition, and the compression condition 1005 is calculated (S4). Then, the compression process is performed using the compression condition 1005 (S5). Based on this, the provisional compression NN model 1006 is determined.

[0048] Next, relearning is performed using the provisional compressed NN model 1006 and the selected valid learning images (S6). Then, it is determined whether the relearning (S6) has been completed until the last learning data set 1001 and the inference data set 1002 (S7). As a result, if it has been completed, the result of the relearning is saved as a compressed neural network model (compressed NN model) 1007. If it has not been completed, the optimal compression condition search process is performed on the result of the relearning (S8). Then, the result of the optimal compression condition search process, that is, the optimal compression condition, is used in S4.

[0049] In the following embodiments, images are used as the learning data set, but other data can also be applied to each embodiment.

[0050] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0051] Example 1

[0052] use Fig.10 and Fig.11 The schematic structure and schematic processing flow of the first embodiment are described.

[0053] first, Fig.10The various components constituting the neural network compression device (NN compression device 121) of this embodiment are described in FIG. Fig.10 As shown, the neural network compression device has a learning image selection unit B100, a neural network compression unit (NN compression unit) B200 and a memory B300.

[0054] In addition, the neural network compression device can be installed on a computing device. For example, it can be implemented as an FPGA (field-programmable gate array) installed on an integrated circuit. In addition, each structure and submodule described below can be implemented as hardware or as software (program). The following is an explanation of the processing.

[0055] Next, use Fig.11 The general processing flow of the first embodiment is described.

[0056] First, in step S100, the learning image selection unit B100 receives the learning image set 1001-1, the inference image set 1002-1, and the pre-compression NN model 1003 from the memory B300. Then, the learning image selection unit B100 generates an indexed learning image set 1004-1 based on these data. This process is equivalent to Figure 6 Step S1.

[0057] In the following embodiments, the case where the object data for learning is an image is used as an example for explanation. Figure 6 A kind of learning data set 1001. In addition, the inference image set 1002-1 is Figure 6 A type of inference data set 1002. In addition, the learning image set 1001-1 and the inference image set 1002-1 are respectively composed of a plurality of learning images and inference images.

[0058] In step S200, the learning image selection unit B100 selects the effective learning image 1008 from the indexed learning image set 1004-1. This process is equivalent to Figure 6 Step S2.

[0059] In step S300, the NN compression unit B200 receives the indexed learning image set 1004-1 from which the effective learning images 1008 are selected and the pre-compression NN model 1003, and performs compression processing.

[0060] This corresponds to Figure 6 S3~S8.

[0061] Next, the detailed structure and detailed flow in Example 1 are described.

[0062] Fig.12 : is a diagram showing the structure of the learning image selection unit B100. The learning image selection unit B100 is composed of the following submodules. B110 is an influence value calculation unit B110 that calculates the influence value based on the learning image set 1001-1, the inference data set 1002 and the pre-compression NN model 1003. B120 is an unbiased variance calculation unit B120 that calculates the unbiased variance using the input from the influence value calculation unit B110. B130 is a selector B130 that selects a submodule that performs processing on input data. B140 is an averaging unit B140 that calculates the average value of input data. B150 is a PCA / clustering unit B150 that performs clustering processing on the input data. B160 is an index adding unit B160 that assigns or classifies indexes to learning images.

[0063] B170 and B180 are ROMs (Read Only Memory) B170 and B180 for storing data.

[0064] Through these submodules, the following processing is performed in the learning image selection unit B100. Fig.13 shown.

[0065] In S110, the influence value calculation unit B110 receives the pre-compression NN model 1003, the inference data set 1002, and the learning image set 1001-1. Then, the influence value calculation unit B110 calculates the influence value of each learning image included in the learning image set 1001-1 on the inference result of the inference image k, and outputs it as the influence value on the inference image k.

[0066] Next, in S120, the unbiased variance calculation unit B120 receives the influence value on the inference image k calculated in S110, and calculates the unbiased variance of the influence value relative to the difference in the inference image. In addition, in S130, the unbiased variance calculation unit B120 calculates the maximum value of the unbiased variance calculated in S120.

[0067] Next, in S140, the selector B130 selects the target submodule to which the influence value on the inference image k is transmitted based on the maximum value of the unbiased variance calculated in S130. Specifically, when the maximum value var of the unbiased variance is greater than an arbitrary threshold value th_var read from ROMB180, the PCA / clustering unit B150 is selected. On the other hand, when the maximum value var of the unbiased variance is less than an arbitrary threshold value th_var read from ROMB180, the averaging unit B140 is selected.

[0068] Next, in S150, the PCA / clustering unit (B150) applies principal component analysis and clustering processing to the influence value on the inference image k calculated in S110, thereby calculating the influence value on each inference image. The calculation of the influence value here is more suitable for performing on all inference images.

[0069] In S160, the averaging unit (B140) applies averaging processing to the influence values ​​on the inference image k calculated in S110, thereby calculating the influence values ​​on each inference image. The calculation of the influence values ​​here is more preferably performed on all the inference images.

[0070] Next, in S170, the index adding unit B160 classifies each learning image into a valid learning image 1008 or an invalid learning image 1009 according to the influence value calculated in S150 or S160. This classification can also be achieved by adding an index indicating whether it is a valid learning image 1008 or an invalid learning image 1009. Accordingly, the index adding unit B160 outputs it as an indexed learning image set 1004-1. Specifically, when the influence value of each of the above-mentioned inference images is greater than the threshold th_inf read from ROMB180, it is determined to be a valid learning image 1008, and conversely, when it is below th_inf, it is determined to be an invalid learning image 1009. Here, the invalid learning image 1009 refers to a learning image that lacks generalization and a learning image to which noise that induces erroneous judgment is added by a malicious third party. In addition, the valid learning image 1008 refers to an image that does not belong to the invalid learning image 1009 among the learning images.

[0071] Next, use Fig.14 and Fig.15 The structure and detailed processing flow of the NN compression unit B200 are described.

[0072] First, use Fig.14 The detailed structure of the NN compression unit B200 is described. The NN compression unit B200 performs Figure 6 Step S3 in the above. The NN compression unit B200 is also composed of the following submodules. B210 is a compression condition determination unit B210 that determines the compression condition based on the invalid learning image 1009 and the valid learning image 1008. B220 is a compression unit B220 that performs compression processing according to the compression condition. B203 is a re-learning unit B230 that inputs the valid learning image 1008 and performs re-learning using the provisional compression NN model 1006.

[0073] Through these submodules, the following processing is performed in the NN compression unit B200. Fig.15 shown.

[0074] In step S210, the compression condition determination unit B210 receives the indexed learning image set 1004-1 consisting of the invalid learning image 1009 and the valid learning image 1008 and the pre-compression NN model 1003. Then, the compression condition determination unit B210 uses these data to determine the compression conditions consisting of the compression part and the compression rate based on the compression priority of the neuron.

[0075] Next, in S220, the compression unit B220 compresses the NN model using the compression conditions determined in step S210. As a result, the compression unit B220 outputs a temporary compressed NN model.

[0076] Next, in step S230, the re-learning unit B230 re-learns the temporary compressed NN model outputted in S220 using the effective learning image 1008. As a result, the compressed NN model 1007 is outputted.

[0077] Next, use Fig.16 and Fig.17 , explaining the structure and detailed processing flow of the PCA / clustering unit B150.

[0078] First, use Fig.16 The submodules constituting the PCA / clustering unit B150 are described below. B151 is a PCA unit B151 that receives an influence value and outputs a cumulative contribution and a principal component score. B152 is a dimensionality compression unit B152 that uses the cumulative contribution, the principal component score, and the lower limit (threshold) of the contribution, i.e., th_pca, to calculate the principal component feature value. B153 is a clustering unit B153 that receives the principal component feature value and calculates the centroid of the principal component feature value as the influence value for each inference image through clustering processing. ROMB154 is a ROMB154 that stores th_pca.

[0079] Next, use Fig.17 The detailed processing flow of the PCA / clustering unit B150 will be described.

[0080] First, in step S151 , the PCA unit B151 receives the influence value on the inference image k, and outputs the cumulative contribution and the principal component score.

[0081] Next, in step S152, the dimension compression unit B152 receives the cumulative contribution and principal component score output in step S151 and the lower limit value th_pca of the cumulative contribution read from ROM B154. Then, the dimension compression unit B152 performs dimension compression within a range where the cumulative contribution is not less than the lower limit value th_pca, and outputs the principal component feature value.

[0082] Next, in step S153, the clustering unit B153 clusters the principal component feature values. At this time, the centroid of each cluster is output as the influence value on each inference image. The influence value on each inference image may also be the influence value on all inference images.

[0083] Next, use Fig.18 and Fig.19 The structure and detailed processing flow of the compression condition determination unit B210 constituting the NN compression unit B200 will be described.

[0084] first, Fig.18 Indicates the submodules constituting the compression condition determination unit B210. B211 is a neural network forward propagation unit B211 that receives the pre-compression NN model 1003, the invalid learning image 1009 and the valid learning image 1008, and outputs the hidden layer activation data. B212 is a compression condition calculation unit B212 that inputs the hidden layer activation data and each threshold value (th_), and outputs the compression condition 1005.

[0085] B213 is a ROM B213 for storing various threshold values.

[0086] then, Fig.19 The following shows the detailed processing flow of the compression condition determination unit B210.

[0087] In step S211, the neural network forward propagation unit B211 receives the NN model 1003 before compression, the effective learning image 1008, and the invalid learning image 1009. Then, the neural network forward propagation unit B211 forward propagates the effective learning image 1008 and the invalid learning image 1009 to the NN respectively. As a result, the neural network forward propagation unit B211 outputs the hidden layer activation data act_val when the effective learning image is propagated and the hidden layer activation data act_inval when the invalid learning image is propagated. Here, the hidden layer activation data represents the frequency of the neurons constituting the hidden layer taking non-zero values.

[0088] Next, in step S212 and step S213, the compression condition calculation unit B212 determines the relationship between the hidden layer activation data and the threshold. Specifically, in step S212, when act_val is greater than an arbitrary threshold th_val, the process proceeds to S213, and when act_val is less than an arbitrary threshold th_val, it is set to don't care (end processing). In addition, in step S213, when act_inval is less than an arbitrary threshold th_inval, the compression condition calculation unit B212 determines it as a first compression priority neuron. On the contrary, when act_inval is greater than an arbitrary threshold th_inval, it is determined to be a second compression priority neuron. The processing order of these steps S212 and S213 is not limited. That is, step S213 may be processed first, or the two steps may be processed in parallel.

[0089] Next, in step S214, the compression condition calculation unit B212 receives the first compression priority neuron and the second compression priority neuron, and then sets the neurons as compression targets in descending order of priority of each neuron, and outputs a compression condition consisting of a compression location and a compression rate.

[0090] Example 2

[0091] Next, the second embodiment will be described. The second embodiment is different from the first embodiment in that the threshold th_inf input to the index adding unit B160 of the learning image selecting unit B100 is automatically determined.

[0092] Fig. 20 FIG. 2 shows the structure of the learning image selection unit B100 of the second embodiment. In this structure, instead of the first embodiment ( Fig.12 ) ROMB180 is provided with an inference unit B181 and a threshold determination unit B182. Fig.21 FIG. 4 shows the processing flow of the learning image selection unit B100 of the second embodiment. In this processing flow, the second embodiment ( Fig.13 ) Step S180 is added. Since the other structures and processing procedures are the same as those of Example 1, only the above differences are described.

[0093] In step S180, the inference unit B181 receives the inference data set 1002 and the pre-compression NN model 1003. Then, in the inference unit B181, inference is performed on these data and the accuracy of the inference result is output. Then, the threshold determination unit B182 determines the value of the threshold th_inf according to the accuracy of the inference result. The threshold th_inf is used for index addition in step S170.

[0094] Example 3

[0095] Next, the third embodiment is described. The third embodiment is an example in which the above embodiments are applied to AD-ECU 1. In the third embodiment, the compression of NN is adjusted online in AD-ECU 1. In addition, AD-ECU 1 is an ECU for automatic driving or driving assistance (also referred to as automatic driving ECU for short).

[0096] Figure 2 Represents the overall structure of AD-ECU1. AD-ECU1 is a type of vehicle-mounted ECU installed on a mobile body such as a car, and performs information processing for automatic driving and driving assistance. Specifically, external information is obtained from sensors such as camera 21, LiDAR22 (Light Detection and Ranging, Laser Imaging Detection and Ranging), Radar23, etc. In addition, for external information, the information detected by LiDAR22 and Radar23 is fused as external information. Then, the external information is stored in the storage unit 11. In addition, the learned NN and external information stored in the storage unit 11 are used together with NN inference (neural network inference) and DNN inference to perform compression in the NN compression device 121. Then, an action plan representing the control content is output to the control system that controls the actuator, engine, etc., to achieve automatic driving and driving assistance.

[0097] Fig. 22 The device configuration of the NN compression device 121 in the AD-ECU 1 is shown. Fig.10 The NN compression device 121, the learning image selection unit B100, the NN compression unit B200, and the memory B300 shown in the figure are common in structure. In addition, there is also an action plan inference unit X100 (output device) that outputs the action plan (inference result) to the control system. Other structures have the same Fig.10 The same structure. In the present NN compression device 121, the action plan (inference result) is output to the control system using the information obtained from each sensor type, i.e., the inference image set 1002-1, the condition of the engine of the mobile body, etc., and the information input via the communication I / F. At this time, the compression processing and relearning processing described in Examples 1 and 2 are performed. In this way, the NN can be appropriately adjusted according to the driving environment, and the reduction in recognition accuracy of the compressed NN model 1007 caused by changes in the driving environment can be suppressed.

[0098] Furthermore, by applying each embodiment to autonomous driving and driving assistance, it is expected that the accuracy of object recognition of pedestrians, cars, etc. can be improved, and the security against network attacks by third parties using learning images that induce misperception can be strengthened.

[0099] Example 4

[0100] Next, the fourth embodiment will be described. The fourth embodiment is similar to the third embodiment, and the functions of each embodiment are applied to the AD-ECU 1. In the fourth embodiment, fine adjustment of the NN is performed in the AD-ECU 1.

[0101] Fig.23 The device structure of the NN compression device 121 of Example 4 is shown. In this structure, compared with Example 3, the adjusted NN model is calculated in the re-learning unit B230 using the pre-adjustment NN model before fine adjustment and the indexed learning image set 1004-1 (the effective learning images therein). As a result, micro-corrections can be made according to the driving environment of the mobile body. Therefore, in automatic driving and driving assistance, it can be expected to improve the accuracy of target recognition of pedestrians, cars, etc. and strengthen the security against network attacks against third parties using learning images that induce misrecognition.

[0102] Example 5

[0103] Next, the fifth embodiment will be described. The fifth embodiment is an example of using the present invention to adjust the compression of the NN by specializing a specific "target of interest" in the AD-ECU. Fig.24 The device structure of the NN compression device 121 of Example 5 is shown. In this structure, information related to the "target of interest" is also input into the learning image selection unit B100 for selection processing. In automatic driving and driving assistance, people, other cars, bicycles, etc. are used as targets of interest. By distinguishing these targets of interest from other images, it is expected to improve the target recognition accuracy of pedestrians, cars, etc. and strengthen the security against network attacks against third parties using learning images that induce misrecognition.

[0104] According to the above embodiments, the following effects are achieved. Figure 8 and Fig. 9 Respectively represent the evaluation of recognition accuracy and learning time when learning the recognition problem in ResNet110. By applying the embodiments of the present invention, the learning time to achieve the same degree of recognition accuracy is reduced to 1 / 10. And by removing invalid images using the present invention, the recognition accuracy is improved by 4.2 pt. when learning with the same number of repetitions. In addition, (1) erroneous recognition of the compressed NN model 1007 caused by the presence of invalid learning images can be suppressed. In addition, since invalid learning images can be removed from the learning data set used for re-learning, (2) the prolonged compression design period caused by the large number of learning images can also be suppressed at the same time.

[0105] In addition, according to an embodiment of the present invention, Figure 3and Figure 4 As shown, compared with the prior arts 1 and 2, the recognition accuracy (suppression of recognition degradation) and the shortening of the design period can be achieved.

[0106] In addition, this technology can be extended not only to deep learning, but also to classical machine learning classifiers such as SVM (Support Vector Machine).

[0107] Explanation of symbols

[0108] Learning image selection unit...B100, neural network compression unit...B200, memory...B300, influence value calculation unit...B110, index attachment unit...B160, unbiased variance calculation unit...B120, selector...B130, averaging unit...B140, PCA / clustering unit...B150, compression unit...B220, re-learning unit...B230, compression condition determination unit...B210, PCA unit...B151, dimensionality compression unit...B152, clustering unit...B153, neural network forward propagation unit...B211, ROM...B170, B180, B154, B213, B190, compression condition calculation unit...B212, inference unit...B181, action plan inference unit...X100.

Claims

1. A neural network compression device, which compresses a neural network using an input learning data set, wherein the neural network compression device is characterized in that: have: a learning image selection unit that calculates an influence value on an inference result, the inference result being calculated using the inference data set and a neural network model for the learning data set, and classifies the learning data set into valid learning data required for the compression and invalid learning data not required for the compression according to the influence value, thereby generating an indexed learning data set; and A neural network compression unit compresses the neural network model based on the indexed learning data set and the neural network model.

2. The neural network compression device according to claim 1, characterized in that: The learning image selecting unit assigns an index indicating the effective learning data and the invalid learning data to at least one of the effective learning data and the invalid learning data, thereby generating the indexed learning data set.

3. The neural network compression device according to claim 2, characterized in that: The learning image selecting unit assigns an index indicating the valid learning data and the invalid learning data to both the valid learning data and the invalid learning data, thereby generating the indexed learning data set.

4. The neural network compression device according to claim 1, characterized in that: The learning image selecting unit generates the indexed learning data set by storing the valid learning data and the invalid learning data in storage media at different addresses.

5. The neural network compression device according to claim 1, characterized in that: The neural network compression unit performs re-learning of the compressed neural network using the effective learning data.

6. The neural network compression device according to claim 1, characterized in that: The neural network compression unit determines whether to remove the neuron using a combination of activation frequencies of the effective learning data and the ineffective learning data for the neuron constituting the neural network, and performs the compression.

7. The neural network compression device according to claim 6, characterized in that: Removal of neurons whose activation frequencies indicate that the effective learning data is low and the invalid learning data is high is performed with priority over removal of neurons whose activation frequencies indicate that the effective learning data is low and the invalid learning data is low.

8. An ECU for automatic driving, which is provided in a mobile body, wherein the ECU for automatic driving has: A storage device for storing external information obtained from the sensor and the neural network; The neural network compression device of claim 1, which compresses the neural network stored in the storage device; as well as An output device uses the compressed neural network to output the action plan of the moving body to the control system of the moving body.

9. A neural network compression method, which uses a neural network compression device, the neural network compression device uses an input learning data set to compress the neural network, the neural network compression method is characterized in that: calculating an influence value on an inference result calculated using an inference data set and a neural network model for the learning data set, According to the influence value, the learning data set is classified into valid learning data required for compression and invalid learning data not required for compression, and an indexed learning data set is generated. The neural network model is compressed based on the indexed learning data set and the neural network model.

10. The neural network compression method according to claim 9, characterized in that: An index indicating the effective learning data and the invalid learning data is assigned to at least one of the effective learning data and the invalid learning data, thereby generating the indexed learning data set.

11. The neural network compression method according to claim 10, characterized in that: An index indicating the effective learning data and the invalid learning data is assigned to both of the effective learning data and the invalid learning data, thereby generating the indexed learning data set.

12. The neural network compression method according to claim 9, characterized in that: The indexed learning data set is generated by storing the valid learning data and the invalid learning data in storage media at different addresses.

13. The neural network compression method according to claim 9, characterized in that: The compressed neural network is re-learned using the effective learning data.

14. The neural network compression method according to claim 9, characterized in that: The compression is performed by determining whether to remove the neuron or not, using a combination of activation frequencies of the effective learning data and the ineffective learning data for the neuron constituting the neural network.

15. The neural network compression method according to claim 14, characterized in that: Removal of neurons whose activation frequencies indicate that the effective learning data is low and the invalid learning data is high is performed with priority over removal of neurons whose activation frequencies indicate that the effective learning data is low and the invalid learning data is low.

Images