Information processing apparatus, information processing method, and program

By integrating similar filters in neural networks, the weight of the model is reduced without compromising prediction accuracy, addressing the issue of impaired feature extraction from filter pruning.

JP2024170070APending Publication Date: 2024-12-06NEC SOLUTION INNOVATORS LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
JP2023087029
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Filter pruning in neural networks can impair feature extraction characteristics, leading to a decrease in prediction accuracy.

Method used

Integrate similar filters in a learning model to reduce the weight of the model while maintaining prediction accuracy by selecting and combining filters with similar parameters.

Benefits of technology

Reduces the weight of the learning model while preserving prediction accuracy and minimizing the computational load for relearning.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024170070000001_ABST
    Figure 2024170070000001_ABST
Patent Text Reader

Abstract

To solve the problem that prediction accuracy is degraded by reduction in weight of a learning model.SOLUTION: An information processing apparatus of the present invention comprises: a learning model unit having a plurality of filters each of which weights each of a plurality of elements of learning data; a selection unit which selects a first filter and a second filter similar to each other from among the plurality of filters on the basis of respective parameters of the plurality of filters; and a generation unit which generates an integrated filter into which the first filter and the second filter are integrated..SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] As a method for reducing the weight of neural networks, a technique for pruning the filters of a learning model has been proposed. In general, a pruned neural network is retrained to obtain a neural network with high prediction accuracy and a lightweight network structure.

[0003] Patent Documents 1 and 2 disclose methods for re-learning a pruned neural network. Patent Document 3 discloses criteria for selecting filters to be pruned. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Jonathan Frankle and Michael Carbin The lottery ticket hypothesis: Finding sparse, trainable neural networks In International Conference on Learning Representations, 2019 [Non-Patent Document 2] Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, Trevor Darrell Rethinking the Value of Network Pruning In International Conference on Learning Representations, 2019 [Non-Patent Document 3] Zhongzhan Huang, Wenqi Shao, Xinjiang Wang, Liang Lin, Ping Luo Rethinking the Pruning Criteria for Convolutional Neural Network NeurIPS 2021 Conference Program Chairs Summary of the Invention [Problem to be solved by the invention]

[0005] However, filter pruning may impair the feature extraction properties of the learning model, which may result in a decrease in the prediction accuracy of the learning model.

[0006] An object of the present invention is to reduce the weight of a learning model while maintaining prediction accuracy. [Means for solving the problem]

[0007] According to one aspect of the present invention, there is provided an information processing device comprising: a learning model unit having a plurality of filters that weight each of a plurality of elements of learning data; a selection unit that selects a first filter and a second filter that are similar to each other from the plurality of filters based on parameters of each of the plurality of filters; and a generation unit that generates an integrated filter that integrates the first filter and the second filter.

[0008] According to another aspect of the present invention, there is provided an information processing method comprising the steps of: selecting a first filter and a second filter that are similar to each other from a plurality of filters based on parameters of each of the plurality of filters that weight each of a plurality of elements of training data; and generating an integrated filter by integrating the first filter and the second filter.

[0009] According to another aspect of the present invention, there is provided a program for causing a computer to execute the steps of selecting a first filter and a second filter that are similar to each other from a plurality of filters based on parameters of each of the plurality of filters that weight each of a plurality of elements of learning data, and generating an integrated filter by integrating the first filter and the second filter. Effect of the Invention

[0010] According to the present invention, it is possible to reduce the weight of a learning model while maintaining prediction accuracy. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram of an information processing system according to a first embodiment. [Diagram 2] 1 is a block diagram showing a hardware configuration of an information processing device according to a first embodiment. [Diagram 3] 2 is a block diagram showing a hardware configuration of a device to be used in the first embodiment. FIG. [Figure 4] FIG. 2 is a conceptual diagram of a learning model of the information processing device according to the first embodiment. [Diagram 5] FIG. 2 is a conceptual diagram showing an example of convolution processing of the information processing device according to the first embodiment. [Figure 6] 4 is a flowchart of a filter integration process according to the first embodiment. [Figure 7A] FIG. 2 is a schematic diagram showing an example of filter selection in the information processing device according to the first embodiment. [Figure 7B] FIG. 2 is a schematic diagram showing an example of filter selection in the information processing device according to the first embodiment. [Figure 7C] FIG. 2 is a schematic diagram showing an example of filter selection in the information processing device according to the first embodiment. [Figure 7D] FIG. 2 is a schematic diagram showing an example of filter selection in the information processing device according to the first embodiment. [Figure 8A]3 is a schematic diagram showing an example of filter integration in the information processing device according to the first embodiment. FIG. [Figure 8B] 4 is a schematic diagram showing an example of filter integration in the information processing device according to the first embodiment. FIG. [Figure 8C] 3 is a schematic diagram showing an example of filter integration in the information processing device according to the first embodiment. FIG. [Figure 9A] FIG. 2 is a neural network diagram of a learning model of the information processing device according to the first embodiment. [Figure 9B] FIG. 2 is a neural network diagram of a learning model of the information processing device according to the first embodiment. [Figure 9C] FIG. 2 is a neural network diagram of a learning model of the information processing device according to the first embodiment. [Figure 10] FIG. 11 is a functional block diagram of an information processing device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, exemplary embodiments of the present disclosure will be described with reference to the drawings. In the drawings, similar or corresponding elements are denoted by the same reference numerals, and descriptions thereof may be omitted or simplified.

[0013] [First embodiment] 1 is a block diagram of an information processing system according to this embodiment. The information processing system includes an information processing device 1, a utilization device 2, and a network 3.

[0014] The information processing device 1 generates a learning model using AI (Artificial Intelligence). The learning model according to the present embodiment may be a Convolution Neural Network (CNN), but may be any learning model including convolution processing. The information processing device 1 may be a computer such as a workstation.

[0015] The utilization device 2 generates output data corresponding to the input data by using the learning model generated by the information processing device 1. For example, the utilization device 2 may have functions such as image recognition, image identification, and voice recognition. The utilization device 2 may be a computer, a tablet PC, or a smartphone.

[0016] The information processing device 1 and the utilization device 2 may be connected to each other via a network 3. The network 3 may be a Wide Area Network (WAN) such as the Internet, or a Local Area Network (LAN).

[0017] Although the information processing device 1 and the utilization device 2 are illustrated as separate devices in FIG. 1, the information processing device 1 and the utilization device 2 may be an integrated device.

[0018] 2 is a block diagram showing a hardware configuration of the information processing device 1. The information processing device 1 includes a CPU (Central Processing Unit) 101, a RAM (Random Access Memory) 102, a ROM (Read Only Memory) 103, a storage device 104, a communication I / F (Interface) 105, an input / output device 106, and a display device 107. The components of the information processing device 1 are connected to each other via a bus, wiring, a drive device, etc. (not shown).

[0019] The CPU 101 is a processor that performs a predetermined calculation according to a program stored in the ROM 103, the storage device 104, etc., and also has a function of controlling each part of the information processing device 1. The CPU 101 loads the program stored in the ROM 103, the storage device 104, etc., into the RAM 102 and executes it. The RAM 102 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of the CPU 101. The ROM 103 is composed of a non-volatile storage medium, and stores necessary information such as the operation program of the CPU 101. The storage device 104 is composed of a non-volatile storage medium such as a flash memory or a hard disk, and stores input data used to construct a learning model. The learning model is constructed in the storage device 104.

[0020] The communication I / F 105 is a communication interface based on standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), and is a module for communicating with the utilization device 2.

[0021] The input / output device 106 includes a user interface used by the user to operate the information processing device 1, a speaker that issues warning sounds, etc. The input / output device 106 can be, for example, a keyboard, a mouse, a trackball, a touch panel, a pen tablet, a button, etc.

[0022] The display device 107 is a display unit such as a liquid crystal display or an OLED (Organic Light Emitting Diode) display, and is used to display information, a GUI (Graphical User Interface) for operation input, etc. The input / output device 106 and the display device 107 may be integrally formed as a touch panel.

[0023] 2, the components constituting the information processing device 1 are illustrated as an integrated device, but some of these functions may be provided by external devices. For example, the storage device 104, the input / output device 106, and the display device 107 may be external devices separate from the CPU 101, etc.

[0024] 3 is a block diagram showing a hardware configuration of the utilization device 2. The utilization device 2 includes a CPU 201, a RAM 202, a ROM 203, a storage device 204, a communication I / F 205, an input / output device 206, and a display device 207. The components of the utilization device 2 are connected to each other via a bus, wiring, a drive device, etc. (not shown).

[0025] CPU 201 is a processor that performs predetermined calculations according to programs stored in ROM 203, storage device 204, etc., and also has the function of controlling each part of utilization device 2. CPU 201 loads programs stored in ROM 203, storage device 204, etc., into RAM 202 and executes them. RAM 202 is composed of a volatile storage medium, and provides a temporary memory area necessary for the operation of CPU 201. ROM 203 is composed of a non-volatile storage medium, and stores necessary information such as the operating program of CPU 201. Storage device 204 is composed of a non-volatile storage medium such as a flash memory, a hard disk, etc.

[0026] The communication I / F 205 is a communication interface based on standards such as Bluetooth (registered trademark) and Wi-Fi (registered trademark), and is a module for communicating with the information processing device 1.

[0027] The input / output device 206 includes a user interface used by the user to operate the utilization device 2, a speaker that issues warning sounds, etc. The input / output device 206 may be, for example, a keyboard, a mouse, a trackball, a touch panel, a pen tablet, a button, etc.

[0028] The display device 207 is a display unit such as a liquid crystal display or an OLED (Organic Light Emitting Diode) display, and is used to display information, a GUI (Graphical User Interface) for operation input, etc. The input / output device 206 and the display device 207 may be integrally formed as a touch panel.

[0029] 3, the components constituting the utilization device 2 are illustrated as an integrated device, but some of these functions may be provided by external devices. For example, the storage device 204, the input / output device 206, and the display device 207 may be external devices separate from the CPU 201, etc.

[0030] 4 is a conceptual diagram of a learning model of the information processing device 1. The learning model includes a convolutional layer 11 and a convolutional layer 12. The learning model also includes an input layer 41, an intermediate layer 42, and an output layer 43. The number of convolutional layers is not limited to two, and may be any number. The learning model may also include a pooling layer and a fully connected layer after the convolutional layer 11 or after the convolutional layer 12.

[0031] Input data is provided to the input layer 41. The input data may be a one-dimensional time series signal or two-dimensional image data, or may be higher-dimensional data. The input data may be a set of data values ​​(channels). Although one channel is set in the input layer 41 in FIG. 4, multiple channels may be set.

[0032] The convolutional layer 11 receives the channel ch1 of the input layer 41. The convolutional layer 11 performs a convolution process on the channel ch1 of the input layer 41, and outputs the channel ch1 and the channel ch2 to the intermediate layer 42. The convolutional layer 12 receives the channel ch1 and the channel ch2 of the intermediate layer 42. The convolutional layer 12 performs a convolution process on the channel ch1 and the channel ch2 of the intermediate layer 42, and outputs the channel ch1, the channel ch2, and the channel ch3 to the output layer 43. Note that the number of channels output by the convolutional layer 11 and the convolutional layer 12 can be selected arbitrarily.

[0033] Fig. 5 is a conceptual diagram showing an example of convolution processing of the information processing device 1. Fig. 5 shows details of the convolution processing in the convolution layer 12. Note that the convolution processing in the convolution layer 11 may be the same processing as that in the convolution layer 12.

[0034] The channels ch1 and ch2 in the intermediate layer 42 are represented as a matrix of 3 rows and 3 columns. The values ​​in each matrix represent the feature amounts included in the channels ch1 and ch2, respectively. Note that the matrices of the channels ch1 and ch2 are not limited to 3 rows and 3 columns.

[0035] The convolution layer 12 includes filters F11, F12, F21, F22, F31, and F32. The number of filters in the convolution layer 12 is determined by the product of the number of channels in the intermediate layer 42 and the number of channels in the output layer 43. In FIG. 5, the number of channels in the intermediate layer 42 is 2, and the number of channels in the output layer 43 is 3, so that the convolution layer 12 includes six filters. Each filter is represented as a 3-row, 3-column matrix of learning parameters (weighting coefficients) of the learning model. The size of the matrix of each filter is equal to or smaller than the size of the channels in the intermediate layer, and can be selected arbitrarily.

[0036] The convolution layer 12 generates a channel ch1 of the output layer 43 from the channel ch1 and the channel ch2 of the intermediate layer 42 using a filter F11 and a filter F12. The filter F11 is used for the convolution process of the channel ch1 of the intermediate layer 42. The convolution process may be a process of calculating the sum of multiplications of corresponding values ​​in the matrix of the channel ch1 and the matrix of the filter F11. In FIG. 5, the result of the convolution process of the channel ch1 and the filter F11 is 0.1×1+0.3×7+0.5×13+0.7×19+0.9×25+1.1×31+1.3×37+1.5×43+1.7×49=274.5. The filter F12 is used for the convolution process of the channel ch2 of the intermediate layer 42. In Fig. 5, the result of the convolution process of the channel ch2 and the filter F12 is 0.2 x 4 + 0.4 x 10 + 0.6 x 16 + 0.8 x 22 + 1.0 x 28 + 1.2 x 34 + 1.4 x 40 + 1.6 x 46 + 1.8 x 52 = 324.0. The convolution layer 12 adds the result of the convolution process of the channel ch1 and the filter F11 to the result of the convolution process of the channel ch2 and the filter F12 to generate the channel ch1 of the output layer 43. In Fig. 5, the channel ch1 of the output layer 43 is 274.5 + 324.0 = 598.5.

[0037] The convolution layer 12 generates a channel ch2 of the output layer 43 from the channels ch1 and ch2 of the intermediate layer 42 using a filter F21 and a filter F22. The filter F21 is used for the convolution process of the channel ch1 of the intermediate layer 42. In FIG. 5, the result of the convolution process of the channel ch1 and the filter F21 is 0.1×2+0.3×8+0.5×14+0.7×20+0.9×26+1.1×32+1.3×38+1.5×44+1.7×50=282.6. The filter F22 is used for the convolution process of the channel ch2 of the intermediate layer 42. In Fig. 5, the result of the convolution process of the channel ch2 and the filter F22 is 0.2 x 5 + 0.4 x 11 + 0.6 x 17 + 0.8 x 23 + 1.0 x 29 + 1.2 x 35 + 1.4 x 41 + 1.6 x 47 + 1.8 x 53 = 333.0. The convolution layer 12 adds the result of the convolution process of the channel ch1 and the filter F21 to the result of the convolution process of the channel ch2 and the filter F22 to generate the channel ch2 of the output layer 43. In Fig. 5, the channel ch2 of the output layer 43 is 282.6 + 333.0 = 615.6.

[0038] The convolution layer 12 generates a channel ch3 of the output layer 43 from the channels ch1 and ch2 of the intermediate layer 42 using a filter F31 and a filter F32. The filter F31 is used for the convolution process of the channel ch1 of the intermediate layer 42. In FIG. 5, the result of the convolution process of the channel ch1 and the filter F31 is 0.1×3+0.3×9+0.5×15+0.7×21+0.9×27+1.1×33+1.3×39+1.5×45+1.7×51=290.7. The filter F32 is used for the convolution process of the channel ch2 of the intermediate layer 42. In Fig. 5, the result of the convolution process of channel ch2 and filter F32 is 0.2 x 6 + 0.4 x 12 + 0.6 x 18 + 0.8 x 24 + 1.0 x 30 + 1.2 x 36 + 1.4 x 42 + 1.6 x 48 + 1.8 x 54 = 342.0. The convolution layer 12 adds the result of the convolution process of channel ch1 and filter F31 to the result of the convolution process of channel ch2 and filter F32 to generate channel ch3 of the output layer 43. In Fig. 5, the channel ch3 of the output layer 43 is 290.7 + 342.0 = 632.7.

[0039] The channels ch1, ch2, and ch3 in the output layer 43 are represented as a 1-row, 1-column matrix. The convolution layer 12 extracts features of the channels ch1 and ch2 in the intermediate layer 42, and outputs the channels ch1, ch2, and ch3 with reduced data volume to the output layer 43.

[0040] 6 is a flowchart of a filter integration process according to this embodiment, and shows the filter integration process in a learning model of the information processing device 1. The information processing device 1 according to this embodiment can reduce the number of filters in the convolution layer and reduce the weight of the learning model by integrating filters.

[0041] First, the information processing device 1 selects a convolution layer for performing integration processing based on the parameters of the convolution layer (step S101). The parameters of the convolution layer can be the number of filters included in the convolution layer or the number of the convolution layer when counted from the input layer 41. The information processing device 1 may select a convolution layer having the most filters among a plurality of convolution layers in the learning model. This allows filters to be integrated from many filters, and more appropriate filter integration can be performed. The information processing device 1 may select a convolution layer having a predetermined number or more of filters among a plurality of convolution layers in the learning model. This allows the learning model to be efficiently lightened.

[0042] In addition, when there are multiple convolution layers each having a predetermined number of filters or more, the convolution layer closest to the input layer 41 may be selected. This makes it possible to more efficiently reduce the weight of the learning model.

[0043] Next, the information processing device 1 calculates filter parameters for the convolutional layers to be integrated (step S102). The method of calculating the filter parameters is not particularly limited. For example, the filter parameters may be the sum (norm) of weighting coefficients, the standard deviation of the norm, or the like. The norm may be the absolute sum (first-order norm) of the weighting coefficients, or the square root of the sum of squares (second-order norm) of the weighting coefficients. The norm indicates the importance of feature extraction by the filter. That is, a filter having a large norm has a high importance. The standard deviation of the norm indicates the amount of learning in the filter. That is, a filter having a small standard deviation has a high importance. The information processing device 1 may create a list of filters based on the filter parameters. The list of filters is used to determine the priority order of the filters to be integrated.

[0044] The information processing device 1 selects a first filter and a second filter similar to each other based on the parameters of the filters (step S103). The method of selecting the filters is not particularly limited. For example, a filter having the largest sum of weighting coefficients may be selected as the first filter and the second filter. By selecting a filter having a large sum of weighting coefficients, it is possible to reduce the weight of the learning model while maintaining the characteristics of feature extraction by convolution processing. In addition, a second filter may be selected based on the weighting coefficient of the first filter. For example, the second filter may be selected so that the difference in norm between the first filter and the second filter is less than a predetermined number. The number of filters selected for filter integration may be three or more. By selecting an integrated filter from many filters, the number of filters can be reduced more efficiently.

[0045] The information processing device 1 generates an integrated filter by integrating the first filter and the second filter (step S104). The weighting coefficient of the integrated filter is calculated based on the weighting coefficient of the first filter and the weighting coefficient of the second filter. For example, the weighting coefficient of the integrated filter may be an average of the weighting coefficient of the first filter and the weighting coefficient of the second filter.

[0046] Next, the information processing device 1 determines whether the filter reduction rate is equal to or greater than a target value (step S105). The filter reduction rate may be the number of filters reduced by integration relative to the number of filters before integration. If the filter reduction rate is less than the target value (NO in step S105), the information processing device 1 returns to step S101 and selects a convolution layer for performing integration processing. If the filter reduction rate is equal to or greater than the target value (YES in step S105), the information processing device 1 ends the filter integration processing.

[0047] Fig. 7 is a schematic diagram showing an example of filter selection in the information processing device 1 according to the first embodiment. In Fig. 7, eight filters of a convolution layer are shown when the number of input channels is two and the number of output channels is four. "Input ch1" and "input ch2" in Fig. 7(a) indicate filters to be multiplied to channels ch1 and ch2, respectively. "Output ch1", "output ch2", "output ch3", and "output ch4" indicate filters used to output channels ch1, ch2, ch3, and ch4.

[0048] Figure 7(b) shows the norms of each filter. "Mean" is the mean of the norms. "Standard deviation" is the standard deviation of the norms.

[0049] 7(c) shows a list of filters. The filters are arranged in descending order of "average" value. "Rank" is the rank of "average". The information processing device 1 may select the filter of channel ch4, which has the first "rank", as the first filter.

[0050] 7(d) shows the filters in ascending order of difference from the standard deviation of the first filter. The information processing device 1 may select the filter of channel ch1 whose standard deviation is closest to that of the first filter as the second filter. By selecting filters whose standard deviation difference is small, it is possible to select filters that are similar to each other.

[0051] FIG. 8 is a schematic diagram showing an example of filter integration in the information processing device 1 according to the first embodiment. FIG. 8(a) shows a learning model before filter integration. FIG. 8(a) shows the nth convolution layer and the (n+1)th convolution layer. The number of input channels of the nth convolution layer is 1, and the number of output channels is 2. The nth convolution layer has two filters. The (n+1)th convolution layer is the next convolution layer after the nth convolution layer. The number of input channels of the (n+1)th convolution layer is 2, and the number of output channels is 3. The (n+1)th convolution layer has six filters. In FIG. 8, the filter process is expressed as multiplication of values ​​corresponding to each other in the channel matrix and the filter matrix.

[0052] Figure 8(b) shows the change in the structure of the learning model when two filters in the nth convolution layer are integrated. The integration of filters is the average of the weighting coefficients of the filters. Note that the integration of filters may be a countable average based on the ratio of the average norms of the filters.

[0053] The integration of filters in the nth convolutional layer reduces the number of filters in the nth convolutional layer from 2 to 1. This reduces the number of channels in the output of the nth convolutional layer to 1. Here, the number of channels in the output of the nth convolutional layer is equal to the number of channels in the input of the (n+1)th convolutional layer. That is, in response to the integration of filters in the nth convolutional layer, the integration of filters is performed in the (n+1)th convolutional layer. This reduces the number of channels in the input of the (n+1)th convolutional layer from 2 to 1, and reduces the number of filters in the (n+1)th convolutional layer from 6 to 3.

[0054] FIG. 8(c) shows the learning model after the integration of filters. In FIG. 8(c), the number of filters is reduced, thereby making the learning model lighter. In this embodiment, by integrating a plurality of filters to generate one integrated filter, the relationship between the input and the output in the convolution process can be maintained. This makes it possible to reduce the weight of the learning model while maintaining the prediction accuracy. In addition, by reducing the weight of the learning model by integrating filters, the amount of calculation required for re-learning the learning model can be reduced.

[0055] Fig. 9 is a neural network diagram of a learning model of the information processing device 1 according to this embodiment. In Fig. 9(a), the learning model includes a neural network including an input layer 91, an intermediate layer 92, and an output layer 93. A neuron 951 in the input layer 91, a neuron 952 in the intermediate layer 92, and a neuron 953 in the output layer 93 are each connected by a synapse 96. The neuron 951, the neuron 952, and the neuron 953 each correspond to a channel. Each synapse 96 corresponds to a weighting coefficient of a filter.

[0056] FIG. 9(b) shows a filter integration process of the learning model in the information processing device 1 according to the present embodiment. In FIG. 9(b), the neurons 952 in the intermediate layer 92 are integrated to generate a new neuron 952. The integration of the neurons 952 reduces the number of neurons 952 in the intermediate layer 92 and the number of synapses connected to the neurons 952, thereby reducing the weight of the learning model. In addition, the neuron 952 obtained by the integration has the properties of the neuron 952 used in the integration. In other words, the relationship between the intermediate layer 92 and the output layer 93 does not change due to the integration of the neurons 952. This allows the characteristics of the feature extraction of the learning model to be maintained.

[0057] 9(c) shows pruning of a learning model in an information processing device according to a conventional example. In pruning of the learning model, neurons 952 and synapses 96 in the intermediate layer 92 that are determined to be unnecessary are deleted. In pruning of the learning model, the structure of the learning model is changed by pruning the neurons 952 and synapses 96. That is, the relationship between the intermediate layer 92 and the output layer 93 is changed by pruning. This may impair the prediction accuracy of the learning model.

[0058] As described above, the information processing device 1 according to the present embodiment can reduce the weight of the learning model while maintaining the prediction accuracy by integrating filters similar to each other and generating an integrated filter. In addition, the amount of calculation required for re-learning the learning model can be reduced by generating the integrated filter.

[0059] [Second embodiment] 10 is a functional block diagram of an information processing device 5 according to the second embodiment. The information processing device 5 includes a learning model unit 51, a selection unit 52, and a generation unit 53. The learning model unit 51 has a plurality of filters that weight each of a plurality of elements of learning data. The selection unit 52 selects a first filter and a second filter that are similar to each other from the plurality of filters based on the parameters of each of the plurality of filters. The generation unit 53 generates an integrated filter that integrates the first filter and the second filter.

[0060] According to the present embodiment, an information processing device 5 capable of reducing the weight of a learning model while maintaining prediction accuracy is provided.

[0061] [Modified embodiment] The present invention is not limited to the above-described embodiments, and can be modified as appropriate without departing from the spirit of the present invention. For example, an example in which a part of the configuration of any of the embodiments is added to another embodiment, or an example in which a part of the configuration of any of the embodiments is replaced with another embodiment, is also an embodiment of the present invention.

[0062] The scope of each embodiment also includes a processing method in which a program that operates the configuration of the embodiment to realize the functions of the above-mentioned embodiment is recorded on a storage medium, the program recorded on the storage medium is read as code, and executed on a computer. That is, a computer-readable storage medium is also included in the scope of each embodiment. In addition, not only the storage medium on which the above-mentioned program is recorded, but also the program itself is included in each embodiment. In addition, one or more components included in the above-mentioned embodiments may be circuits such as ASICs and FPGAs configured to realize the functions of each component.

[0063] Examples of the storage medium that can be used include a floppy disk, a hard disk, an optical disk, a magneto-optical disk, a CD (Compact Disk)-ROM, a magnetic tape, a non-volatile memory card, and a ROM. In addition, the scope of each embodiment does not include those that execute processing by themselves recorded on the storage medium, but also those that execute processing by operating on an OS (Operating System) in cooperation with other software and functions of an expansion board.

[0064] The services realized by the functions of the above-described embodiments can also be provided to users in the form of SaaS (Software as a Service).

[0065] It should be noted that the above-mentioned embodiments are merely examples of the implementation of the present invention, and the technical scope of the present invention should not be interpreted as being limited by these embodiments. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features.

[0066] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0067] (Appendix 1) a learning model unit having a plurality of filters that weight each of a plurality of elements of the learning data; a selection unit that selects a first filter and a second filter that are similar to each other from the plurality of filters based on parameters of each of the plurality of filters; a generation unit that generates an integrated filter by integrating the first filter and the second filter; An information processing device comprising:

[0068] (Appendix 2) The learning model unit includes a plurality of layers each having a plurality of filters; The selection unit further selects one layer from the plurality of layers, the selection unit selects the first filter and the second filter from the plurality of filters included in the one layer. 2. An information processing device according to claim 1.

[0069] (Appendix 3) the selection unit selects, from among the plurality of layers, a layer having a predetermined number or more of filters as the one layer; 3. An information processing device according to claim 2.

[0070] (Appendix 4) The generation unit generates the integrated filter in another layer different from the one layer in response to the generation of the integrated filter in the one layer. 3. An information processing device according to claim 2.

[0071] (Appendix 5) The other layer is a layer next to the one layer. 5. The information processing device according to claim 4.

[0072] (Appendix 6) The filter has a plurality of weighting coefficients; The parameters of the filter are calculated based on the plurality of weighting coefficients. 2. An information processing device according to claim 1.

[0073] (Appendix 7) the selection unit selects the first filter based on a sum of the plurality of weighting coefficients of the filter; The selection unit selects the second filter based on the plurality of weighting coefficients of the first filter. 7. The information processing device according to claim 6.

[0074] (Appendix 8) the weighting coefficients of the integrated filter are calculated based on the weighting coefficients of the first filter and the weighting coefficients of the second filter; 7. The information processing device according to claim 6.

[0075] (Appendix 9) selecting a first filter and a second filter that are similar to each other from among a plurality of filters based on parameters of each of the plurality of filters that weight each of a plurality of elements of the training data; generating an integrated filter by integrating the first filter and the second filter; An information processing method comprising:

[0076] (Appendix 10) On the computer, selecting a first filter and a second filter that are similar to each other from among a plurality of filters based on parameters of each of the plurality of filters that weight each of a plurality of elements of the training data; generating an integrated filter by integrating the first filter and the second filter; A program for executing the above. [Explanation of symbols]

[0077] 1, 5 Information processing device 2 Equipment used 3. Network 11, 12 Convolutional Layer 41, 91 Input layer 42, 92 Middle class 43, 93 Output layer 51 Learning Model Section 52 Selection section 53 Generation part 96 Synapse 101, 201 CPU 102, 202 RAM 103, 203 ROM 104, 204 Storage device 105, 205 Communication I / F 106, 206 I / O devices 107, 207 Display device 951, 952, 953 Neurons ch1, ch2, ch3, ch4 channels F11, F12, F21, F22, F31, F32 Filters

Claims

1. a learning model unit having a plurality of filters that weight each of a plurality of elements of the learning data; a selection unit that selects a first filter and a second filter that are similar to each other from the plurality of filters based on parameters of each of the plurality of filters; a generation unit that generates an integrated filter by integrating the first filter and the second filter; An information processing device comprising:

2. The learning model unit includes a plurality of layers each having a plurality of filters; The selection unit further selects one layer from the plurality of layers, the selection unit selects the first filter and the second filter from the plurality of filters included in the one layer. The information processing device according to claim 1 .

3. the selection unit selects, from among the plurality of layers, a layer having a predetermined number or more of filters as the one layer; The information processing device according to claim 2 .

4. The generation unit generates the integrated filter in another layer different from the one layer in response to the generation of the integrated filter in the one layer. The information processing device according to claim 2 .

5. The other layer is a layer next to the one layer. The information processing device according to claim 4.

6. The filter has a plurality of weighting coefficients; The parameters of the filter are calculated based on the plurality of weighting coefficients. The information processing device according to claim 1 .

7. the selection unit selects the first filter based on a sum of the plurality of weighting coefficients of the filter; the selection unit selects the second filter based on the plurality of weighting coefficients of the first filter. The information processing device according to claim 6.

8. the weighting coefficients of the integrated filter are calculated based on the weighting coefficients of the first filter and the weighting coefficients of the second filter; The information processing device according to claim 6.

9. selecting a first filter and a second filter that are similar to each other from among a plurality of filters based on parameters of each of the plurality of filters that weight each of a plurality of elements of the training data; generating an integrated filter by integrating the first filter and the second filter; An information processing method comprising:

10. On the computer, selecting a first filter and a second filter that are similar to each other from among a plurality of filters based on parameters of each of the plurality of filters that weight each of a plurality of elements of the training data; generating an integrated filter by integrating the first filter and the second filter; A program for executing the above.