Transplantation support device, transplantation support method, and transplantation support program

Abstract

To provide a technique enabling facilitation of implantation of a CNN framework.SOLUTION: A transplant support device includes: a model acquisition part for acquiring a CNN model; a missing model generation part for generating a plurality of missing CNN models in which layers are sequentially deleted from the last layer of a plurality of layers of the CNN model; a test pattern acquisition part for acquiring a test input pattern of the CNN model; and an inference part for generating a plurality of test output patterns by inference using at least the plurality of missing CNN models and the test input pattern.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a transplantation assistance device, a transplantation assistance method, and a transplantation assistance program. [Background technology]

[0002] In recent years, a porting support device has been proposed that supports porting of a framework from a processing system in which the framework exists to a processing system in which the framework does not exist. For example, Patent Document 1 proposes a technology that automatically corrects build errors by analyzing source code from compiler build errors and correcting the source code based on past correction history.

[0003] Furthermore, for example, Non-Patent Document 1 proposes a CNN framework that learns a CNN model including the configuration and weights of a Convolution Neural Network (CNN) and performs inference using the learned CNN model. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2015-26365 A [Non-patent literature]

[0005] [Non-Patent Document 1] internet<URL:https: / / www.tensorflow.org> Summary of the Invention [Problem to be solved by the invention]

[0006] In the CNN framework, inference using a trained CNN model can be performed only in a processing system that the CNN framework runs on. Therefore, in order to port to a processing system that does not have the CNN framework, it is necessary to identify multiple layers, such as the convolution layer, included in the CNN configuration and the weights used in those layers.

[0007] However, it is difficult to infer the output patterns of each of the multiple layers included in the CNN model from the input patterns to the CNN model and the output patterns from the CNN model. Therefore, it is difficult to specify the multiple layers and weights included in the CNN model, and as a result, there is a problem that it is difficult to port the CNN framework.

[0008] Therefore, the present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a technology that can facilitate the porting of a CNN framework. [Means for solving the problem]

[0009] The transplantation support device according to the present disclosure includes multiple layers of a convolution neural network (CNN) and weighting factors associated with the layers. fruit a missing model generation unit that generates a plurality of missing CNN models in which layers are deleted in order from the last layer of the plurality of layers of the CNN model; a test pattern acquisition unit that acquires a test input pattern for the CNN model; and an inference unit that generates a plurality of test output patterns by inference using at least the plurality of missing CNN models and the test input pattern. Effect of the Invention

[0010] According to the present disclosure, a plurality of missing CNN models are generated by sequentially removing layers from the last layer of a plurality of layers of a CNN model, and a plurality of test output patterns are generated by inference using the plurality of missing CNN models and a test input pattern. With such a configuration, it is possible to facilitate the porting of a CNN framework. [Brief description of the drawings]

[0011] [Figure 1] 1 is a block diagram showing a configuration of a transplantation assistance device according to a first embodiment. [Diagram 2] FIG. 2 is a diagram illustrating an example of a CNN model according to the first embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a missing CNN model according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a test input pattern according to the first embodiment; [Diagram 5] FIG. 4 is a diagram showing an example of a test output pattern according to the first embodiment. [Figure 6] 4 is a flowchart showing an operation of a CNN layer reducing unit according to the first embodiment. [Figure 7] FIG. 11 is a block diagram showing a configuration of a transplantation assistance device according to a second embodiment. [Figure 8] FIG. 11 is a diagram showing an example of a skeleton template according to the second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a skeleton code according to the second embodiment. [Figure 10] 13 is a flowchart showing the operation of a skeleton generating unit according to the second embodiment. [Figure 11] 13 is a flowchart showing the operation of a skeleton generating unit according to the second embodiment. [Figure 12] FIG. 11 is a block diagram showing the configuration of a transplantation assistance device according to embodiment 3. [Figure 13] FIG. 13 is a diagram showing an example of a skeleton code with a layer name according to the third embodiment. [Figure 14] 13 is a flowchart showing the operation of a layer-named skeleton generating unit according to the third embodiment. [Figure 15] 13 is a flowchart showing the operation of a layer-named skeleton generating unit according to the third embodiment. [Figure 16] FIG. 13 is a block diagram showing a hardware configuration of a transplantation assistance device according to another modified example. [Figure 17] FIG. 13 is a block diagram showing a hardware configuration of a transplantation assistance device according to another modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] <Embodiment 1> Fig. 1 is a block diagram showing a configuration of a transplantation support device according to the present embodiment 1. The transplantation support device shown in Fig. 1 includes a CNN input unit 1 which is a model acquisition unit, a CNN layer reduction unit 2 which is a missing model generation unit, a CNN storage unit 3, a test pattern input unit 4 which is a test pattern acquisition unit, a CNN inference unit 5 which is an inference unit, and a test pattern storage unit 6.

[0013] The input of the CNN layer reduction unit 2 is connected to the output of the CNN input unit 1 and the output of the CNN storage unit 3. The input of the CNN storage unit 3 is connected to the output of the CNN layer reduction unit 2. The input of the CNN inference unit 5 is connected to the output of the CNN storage unit 3 and the output of the test pattern input unit 4. The input of the test pattern storage unit 6 is connected to the output of the CNN inference unit 5.

[0014] The CNN input unit 1 acquires a CNN model trained in a CNN framework such as a source.

[0015] FIG. 2 is a diagram showing an example of a CNN model 7. The CNN model 7 includes a CNN configuration 8 and CNN weights 9. The CNN configuration 8 includes multiple CNN layers 10 that describe what processing is to be performed. In the following description, the multiple CNN layers 10 may be referred to simply as "multiple layers 10," or the CNN layer 10 may be referred to simply as "layer 10."

[0016] For example, the CNN configuration 8 in Fig. 2 includes a convolution layer → a ReLU layer → a max pooling layer → ... → a convolution layer, which are arranged in this order, as multiple CNN layers 10. The CNN configuration 8 is described in a hierarchical data structure such as JSON, XML, and HDF.

[0017] At least one layer 10 is accompanied by coefficients and parameters used in the calculation of the layer 10, etc. as CNN weights 9. For example, in the convolution layer, which is the first layer 10 in FIG. 2, numerical values ​​of parameters necessary for executing convolution, such as kernel_size, kernel, and bias, are accompanied by CNN weights 9. The CNN weights 9 are described, for example, in a matrix data structure such as TSV or CSV, or in a hierarchical data structure such as JSON, XML, or HDF.

[0018] 1 generates a plurality of missing CNN models by deleting layers 10 in order from the last layer 10 of the plurality of layers 10 of the CNN model 7 acquired by the CNN input unit 1. An example of such a CNN layer reduction unit 2 will be described below.

[0019] First, the CNN layer reduction unit 2 generates a missing CNN model by deleting the last layer 10 of the multiple layers 10 of the CNN model 7 acquired by the CNN input unit 1. The CNN layer reduction unit 2 accumulates the generated missing CNN model in the CNN accumulation unit 3.

[0020] Then, the CNN layer reduction unit 2 generates a new missing CNN model by deleting the last layer 10 of the multiple layers 10 of the missing CNN model stored in the CNN storage unit 3, and stores the new missing CNN model in the CNN storage unit 3. The CNN layer reduction unit 2 repeats this operation until the number of layers 10 of the missing CNN model becomes one. As a result, the CNN layer reduction unit 2 generates missing CNN models in a number that is one less than the number of the multiple layers 10.

[0021] In the example of FIG. 2, the CNN layer reduction unit 2 generates the missing CNN model 11a in FIG. 3 by deleting the Convolution layer, which is the last layer 10 of the CNN model 7. In addition, the CNN layer reduction unit 2 generates the missing CNN model 11b by deleting the Max Unpooling layer, which is the last layer 10 of the missing CNN model 11a. By repeating such processing, the CNN layer reduction unit 2 finally generates the missing CNN model 11n including the Convolution layer, which is the first layer of the CNN model 7. Hereinafter, for convenience of explanation, each of the missing CNN models 11a to 11n may be referred to as the missing CNN model 11.

[0022] The CNN storage unit 3 stores a plurality of missing CNN models 11 generated by the CNN layer reduction unit 2.

[0023] The test pattern input unit 4 acquires input data such as a test input pattern for the CNN model 7. The test input pattern 15 for the CNN model 7 is a test input pattern that is used for inference using the CNN model 7 and is also used for inference using the missing CNN model 11.

[0024] Fig. 4 is a diagram showing an example of a test input pattern 15. In the example of Fig. 4, the test input pattern 15 is data of a two-dimensional numerical matrix, and is described in CSV. The number of dimensions and size of the test input pattern 15 differ depending on the top layer 10 included in the CNN model 7 and the weights 9 of the CNN.

[0025] The CNN inference unit 5 generates a plurality of test output patterns by inference using a plurality of missing CNN models 11 stored in the CNN storage unit 3 and a test input pattern 15 acquired by the test pattern input unit 4. In the present embodiment 1, the CNN inference unit 5 generates a plurality of test output patterns by inference in which the test input pattern 15 is applied to the plurality of missing CNN models 11. The CNN inference unit 5 is configured, for example, with a CNN framework.

[0026] Fig. 5 is a diagram showing an example of the test output pattern 16. In the example of Fig. 5, the test output pattern 16 is data of a two-dimensional numerical matrix, and is described in CSV. The test output pattern 16 for the same test input pattern 15 differs for each missing CNN model 11, and the number of dimensions and size of the test output pattern 16 differ depending on the layer 10 included in the missing CNN model 11 and the CNN weights 9 thereof.

[0027] The test pattern storage unit 6 stores a plurality of test output patterns 16 generated by the CNN inference unit 5 .

[0028] Next, a detailed description will be given of the operation of the CNN layer reducing unit 2. FIG. 6 is a flowchart showing the operation of generating the missing CNN model 11 by the CNN layer reducing unit 2.

[0029] First, in step S1, the CNN layer reduction unit 2 determines whether or not there is a missing CNN model 11 in the CNN storage unit 3. If it is determined that there is no missing CNN model 11 in the CNN storage unit 3, the process proceeds to step S2, and if it is determined that there is a missing CNN model 11 in the CNN storage unit 3, the process proceeds to step S3.

[0030] In step S2, the CNN layer reduction unit 2 acquires the CNN model 7 as a processing target CNN model from the CNN input unit 1. After that, the process proceeds to step S4.

[0031] In step S3, the CNN layer reduction unit 2 acquires the last stored missing CNN model 11 from the CNN storage unit 3 as the CNN model to be processed. After that, the process proceeds to step S4.

[0032] In step S4, the CNN layer reduction unit 2 determines whether the CNN model to be processed includes two or more layers 10. If it is determined that the CNN model to be processed includes two or more layers 10, the process proceeds to step S5, and if it is not determined that the CNN model to be processed includes two or more layers 10, the operation in FIG. 6 ends.

[0033] In step S5, the CNN layer reduction unit 2 deletes the last layer 10 from the processing target CNN model to generate a missing CNN model 11. After that, the process proceeds to step S6.

[0034] In step S6, the CNN layer reduction unit 2 stores the missing CNN model 11 generated in step S5 in the CNN storage unit 3. After that, the process returns to step S1.

[0035] <Summary of the first embodiment> According to the transplantation support device of the first embodiment as described above, a plurality of missing CNN models 11 are generated by deleting layers 10 in order from the last layer 10 of the plurality of layers 10 of the CNN model 7, and a plurality of test output patterns 16 are generated by inference using the plurality of missing CNN models 11 and the test input pattern 15. According to such a configuration, the plurality of layers 10 and weights 9 of the CNN model 7 can be identified to some extent from the plurality of test output patterns 16 that roughly correspond to the output patterns of the plurality of layers 10 of the CNN model 7, which makes it possible to facilitate transplantation of the CNN framework.

[0036] <Variation 1> In the first embodiment, the CNN layer reduction unit 2 generates the multiple missing CNN models 11 by deleting the layers 10 in order from the last layer 10 of the multiple layers 10 of the CNN model 7 acquired by the CNN input unit 1. However, the method of generating the multiple missing CNN models 11 is not limited to this, as long as the multiple missing CNN models 11 are obtained by deleting the layers 10 in order from the last layer 10 of the multiple layers 10 of the CNN model 7. For example, the CNN layer reduction unit 2 may generate the multiple missing CNN models 11 by adding layers 10 in order from the first layer 10 of the multiple layers 10 of the CNN model 7 acquired by the CNN input unit 1.

[0037] <Variation 2> In the first embodiment, the CNN inference unit 5 generates a plurality of test output patterns by inference using a plurality of missing CNN models 11 and the test input pattern 15, but this is not limited to the above. For example, the CNN inference unit 5 may generate a plurality of test output patterns by inference using a plurality of missing CNN models 11 and the test input pattern 15, and inference using the CNN model 7 and the test input pattern 15.

[0038] According to such a configuration, the multiple layers 10 and weights 9 of the CNN model 7 can be identified from the multiple test output patterns 16 corresponding to the output patterns of the multiple layers 10 of the CNN model 7, which makes it easier to port the CNN framework. In the second embodiment described below, the CNN inference unit 5 generates multiple test output patterns by inference using the CNN model 7, the multiple missing CNN models 11, and the test input pattern 15.

[0039] <Embodiment 2> 7 is a block diagram showing the configuration of a transplantation assistance device according to the present embodiment 2. In the following, among the components according to the present embodiment 2, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components will be mainly described.

[0040] As described below, according to the transplantation support device of the second embodiment, by analyzing the dimensions and sizes of the multiple test output patterns 16, it is possible to generate skeleton codes for multiple target processing systems as multiple skeletons corresponding to each of the multiple layers 10.

[0041] The configuration of the transplantation support device in Fig. 7 is the same as that in Fig. 1, except that a skeleton template input unit 17, which is a template acquisition unit, and a skeleton generation unit 18 are added to the configuration in Fig. 1. The input of the skeleton generation unit 18 is connected to the output of the test pattern storage unit 6 and the output of the skeleton template input unit 17.

[0042] The skeleton template input unit 17 acquires a fragment of skeleton code of the processing system to which the code is to be ported as a skeleton template.

[0043] FIG. 8 is a diagram showing an example of a skeleton template 19. The skeleton template 19 in FIG. 8 includes a function skeleton 20, a dimension start string 21, and a dimension end string 22. In the example of FIG. 8, the skeleton template 19 is a template in the C language, and the function skeleton 20 includes a void type return value, in indicating an input and out indicating an output, both of which are float types. The function name and the dimensions and sizes of the arguments indicating the input and output are substituted with the strings *NAME*, *IN_DIM*, and *OUT_DIM*, respectively. The dimension start string 21 is a character to be added before the values ​​of these strings, and in the example of FIG. 8, it is "[". The dimension end string 22 is a character to be added after the values ​​of these strings, and in the example of FIG. 8, it is "[".

[0044] The skeleton generation unit 18 acquires the test output pattern 16 from the test pattern storage unit 6 and analyzes the number of dimensions and size. Then, based on the analysis result and the skeleton template 19 acquired by the skeleton template input unit 17, the skeleton generation unit 18 generates skeleton codes of multiple destination processing systems corresponding to the multiple layers 10 of the CNN. The generated skeleton codes are written in a skeleton file (not shown) or the like. In the following description, the skeleton code of the destination processing system may be referred to simply as "skeleton code". The skeleton codes are functions and methods of the destination processing system in the multiple layers 10 of the CNN model 7, and no specific processing is implemented.

[0045] Fig. 9 is a diagram showing an example of a skeleton code 23. Fig. 9 shows the skeleton code 23 generated from the skeleton template 19 of Fig. 8. In the example of Fig. 9, *NAME* in Fig. 8 is replaced with layer_1, *IN_DIM* is replaced with

[0128] , and *OUT_DIM* is replaced with

[0128]

[64] . The skeleton code 23 in the example of Fig. 9 corresponds to the first layer 10 of Fig. 2, the input dimension is one dimension and the size is 128, and the output dimension is two dimensions and the size is 128 x 64.

[0046] 10 and 11 are flow charts showing the operation of generating the skeleton code 23 by the skeleton generating unit 18. FIG.

[0047] First, in step S11, the skeleton generating unit 18 deletes the skeleton file, after which the process proceeds to step S12.

[0048] In step S12, the skeleton generating unit 18 assigns 0 to the "number of layers." After that, the process proceeds to step S13.

[0049] In step S13, the skeleton generating unit 18 judges whether or not there is a test input pattern 15 and a test output pattern 16 corresponding to the "number of layers" in the test pattern input unit 4 and the test pattern storage unit 6. For example, the skeleton generating unit 18 makes the judgment in step S13 based on whether or not the "number of layers" is smaller than the number of layers acquired from the test input pattern 15. If it is judged that there is a test input pattern 15 and a test output pattern 16 corresponding to the "number of layers", the process proceeds to step S14, and if it is not judged that there is a test input pattern 15 and a test output pattern 16 corresponding to the "number of layers", the operation of FIG. 10 ends.

[0050] In step S14, the skeleton generating unit 18 acquires the test input pattern 15 and the test output pattern 16 corresponding to the "number of layers" from the test pattern input unit 4 and the test pattern storage unit 6. After that, the process proceeds to step S15.

[0051] In step S15, the skeleton generating unit 18 duplicates the function skeleton 20 and replaces *NAME* of the duplicated function skeleton with "layer_1 + number of layers." For example, if the "number of layers" is 0, *NAME* of the duplicated function skeleton is replaced with "layer_1." Then, the process proceeds to step S16.

[0052] In step S16, the skeleton generating unit 18 obtains the number of input dimensions from the test input pattern 15. After that, the process proceeds to step S17.

[0053] In step S17, the skeleton generating unit 18 assigns 0 to the "input dimension." After that, the process proceeds to step S18.

[0054] In step S18, the skeleton generating unit 18 initializes the "input dimension character string." After that, the process proceeds to step S19.

[0055] In step S19, the skeleton generating unit 18 determines whether or not the "input dimension" is smaller than the number of dimensions of the input acquired in step S16. If the "input dimension" is smaller than the number of dimensions of the input acquired in step S16, the process proceeds to step S20, and if the "input dimension" is equal to or larger than the number of dimensions of the input acquired in step S16, the process proceeds to step S23 in FIG.

[0056] In step S20, the skeleton generating unit 18 obtains the size of the input dimensions from the test input pattern 15. After that, the process proceeds to step S21.

[0057] In step S21, the skeleton generating unit 18 adds "dimension start string 21 + input dimension size acquired in step S20 + dimension end string 22" to the "input dimension string". For example, in the example of Fig. 8, if the dimension size acquired in step S20 is 128,

[0128] is added to the "input dimension string". Then, the process proceeds to step S22.

[0058] In step S22, the skeleton generating unit 18 assigns the “input dimension”+1 to the “input dimension”, that is, increments the “input dimension”. After that, the process returns to step S19.

[0059] 11, the skeleton generating unit 18 replaces *IN_DIM* in the duplicated function skeleton with the "input dimension string." After that, the process proceeds to step S24.

[0060] In step S24, the skeleton generating unit 18 obtains the number of output dimensions from the test output pattern 16. After that, the process proceeds to step S25.

[0061] In step S25, the skeleton generating unit 18 assigns 0 to the "output dimension." After that, the process proceeds to step S26.

[0062] In step S26, the skeleton generating unit 18 initializes the "output dimension character string." After that, the process proceeds to step S27.

[0063] In step S27, the skeleton generation unit 18 determines whether or not the "output dimension" is smaller than the number of dimensions of the output acquired in step S24. If the "output dimension" is smaller than the number of dimensions of the output acquired in step S24, the process proceeds to step S28, and if the "output dimension" is equal to or larger than the number of dimensions of the output acquired in step S24, the process proceeds to step S31.

[0064] In step S28, the skeleton generating unit 18 obtains the size of the output dimension from the test output pattern 16. After that, the process proceeds to step S29.

[0065] In step S29, the skeleton generating unit 18 adds "dimension start string 21 + size of the output dimension acquired in step S28 + dimension end string 22" to the "output dimension string".

[0066] In step S30, the skeleton generating unit 18 assigns the "output dimension" + 1 to the "output dimension", that is, increments the "output dimension". After that, the process returns to step S27.

[0067] In step S31, the skeleton generating unit 18 replaces *OUT_DIM* in the duplicated function skeleton with the “output dimension string.” After that, the process proceeds to step S32.

[0068] In step S32, the skeleton generating unit 18 adds the replaced function skeleton to the end of the skeleton file, after which the process proceeds to step S33.

[0069] In step S33, the skeleton generating unit 18 assigns "number of layers"+1 to "number of layers", that is, increments "number of layers". After that, the process returns to step S13.

[0070] <Summary of the second embodiment> According to the transplantation support device of the second embodiment as described above, a plurality of skeleton codes 23 corresponding to the plurality of layers 10 of the CNN model 7 are generated based on a plurality of test output patterns 16 and a skeleton template 19. Such a configuration can facilitate the transplantation of the CNN framework.

[0071] <Embodiment 3> 12 is a block diagram showing the configuration of a transplantation assistance device according to the present embodiment 3. In the following, among the components according to the present embodiment 3, components that are the same as or similar to the components described above are given the same or similar reference numerals, and different components are mainly described.

[0072] In the second embodiment, the skeleton codes of the destination processing system for the multiple layers 10 of the CNN are given consecutive numbers such as layer_1, layer_2, .... In contrast, as described below, the transplantation support device according to the third embodiment makes it possible to generate layer-named skeleton codes of the multiple destination processing systems as multiple skeletons each having the name of the multiple layers 10.

[0073] The configuration of the transplantation support device in FIG. 12 is the same as the configuration in FIG. 7 except that a CNN layer name extraction unit 25, which is a layer name extraction unit, is added, and the test pattern storage unit 6 and the skeleton generation unit 18 are changed to a layer-named test pattern storage unit 26 and a layer-named skeleton generation unit 27.

[0074] The input of the CNN layer name extraction unit 25 is connected to the output of the CNN input unit 1. The input of the layer named test pattern storage unit 26 is connected to the output of the CNN layer name extraction unit 25 and the output of the CNN inference unit 5. The input of the layer named skeleton generation unit 27 is connected to the output of the layer named test pattern storage unit 26 and the output of the skeleton template input unit 17.

[0075] The CNN layer name extraction unit 25 extracts the names of a plurality of layers 10 from the CNN model 7 acquired by the CNN input unit 1.

[0076] The layer-named test pattern storage unit 26 stores the test output patterns 16 generated by the CNN inference unit 5 and the names of the multiple layers 10 extracted by the CNN layer name extraction unit 25.

[0077] The layer-named skeleton generator 27 generates a plurality of layer-named skeleton codes based on the test output pattern 16 and the names of the layers 10 stored in the layer-named test pattern storage unit 26 and the skeleton template 19 acquired by the skeleton template input unit 17. In other words, the layer-named skeleton generator 27 corresponds to the skeleton generator 18 to which a function of giving a layer name is added, and the plurality of layer-named skeleton codes correspond to a plurality of skeleton codes 23 to which the names of the layers 10 are respectively added. The layer-named skeleton codes are written in a skeleton file (not shown) or the like. The layer-named skeleton codes are functions or methods of a destination processing system in the layers 10 of the CNN model 7, similar to the skeleton codes 23, and no specific processing is implemented.

[0078] FIG. 13 is a diagram showing an example of a layer named skeleton code 28. FIG. 13 is a layer named skeleton code 28 generated from the skeleton template 19 of FIG. 8. In the example of FIG. 13, *NAME* in FIG. 8 is replaced with Convolution_1 indicating the name of the first layer 10 of FIG. 2, *IN_DIM* is replaced with

[0128] , and *OUT_DIM* is replaced with

[0128]

[64] . The layer named skeleton code 28 in the example of FIG. 13 corresponds to the first layer 10 of FIG. 2, the input dimension is one dimension and the size is 128, and the output dimension is two dimensions and the size is 128×64.

[0079] Figures 14 and 15 are flowcharts showing the operation of generating the layer-named skeleton code 28 by the layer-named skeleton generating unit 27. The flowcharts in Figures 14 and 15 are similar to those in Figures 10 and 11, except that steps S14 and S15 in the flowcharts are replaced with steps S14a and S15a. Therefore, only the processing in steps S14a and S15a will be described below.

[0080] In step S14a, the skeleton generating unit 18 acquires the test input pattern 15, the test output pattern 16, and the name of the layer 10 corresponding to the "number of layers" from the test pattern input unit 4 and the test pattern storage unit 6. After that, the process proceeds to step S15a.

[0081] In step S15a, the skeleton generating unit 18 duplicates the function skeleton 20 and replaces *NAME* of the duplicated function skeleton with "name of layer 10 acquired in step S14_1+number of layers". For example, when an operation is performed on the CNN model 7 in FIG. 2 and the "number of layers" is 0, *NAME* of the duplicated function skeleton is replaced with "Convolution_1". Then, the process proceeds to step S16.

[0082] <Summary of the Third Embodiment> According to the transplantation support device of the third embodiment as described above, it is possible to generate a plurality of layer-named skeleton codes, that is, a plurality of skeleton codes 23 each having the name of a plurality of layers 10. This makes it possible to avoid implementation errors due to mix-up of transplantation destination function skeletons, and thus facilitates the transplantation of the CNN framework.

[0083] <Other Modifications> The CNN input unit 1, the CNN layer reduction unit 2, the test pattern input unit 4, and the CNN inference unit 5 in FIG. 1 described above are hereinafter referred to as the "CNN input unit 1, etc." The CNN input unit 1, etc. are realized by a processing circuit 81 shown in FIG. 16. That is, the processing circuit 81 includes a CNN input unit 1 that acquires a CNN model including a plurality of layers of CNN and weights associated with the layers, a CNN layer reduction unit 2 that generates a plurality of missing CNN models in which layers are deleted in order from the last layer of the plurality of layers of the CNN model, a test pattern input unit 4 that acquires a test input pattern for the CNN model, and a CNN inference unit 5 that generates a plurality of test output patterns by inference using at least a plurality of missing CNN models and the test input pattern. The processing circuit 81 may be applied with dedicated hardware, or a processor that executes a program stored in a memory. The processor may be, for example, a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a DSP (Digital Signal Processor), or the like.

[0084] When the processing circuit 81 is a dedicated hardware, the processing circuit 81 corresponds to, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of the CNN input unit 1 and the like may be realized by a circuit in which the processing circuits are distributed, or the functions of each unit may be realized by a single processing circuit.

[0085] When the processing circuit 81 is a processor, the functions of the CNN input unit 1 and the like are realized by a combination with software and the like. The software and the like includes, for example, software, firmware, or software and firmware. The software and the like are written as a program and stored in a memory. As shown in FIG. 17, the processor 82 applied to the processing circuit 81 realizes the functions of each unit by reading and executing a program stored in the memory 83. That is, the transplantation support device includes a memory 83 for storing a program that, when executed by the processing circuit 81, results in the execution of the steps of: acquiring a CNN model including multiple layers of CNN and weights associated with the layers; generating multiple missing CNN models in which layers are deleted in order from the last layer of the multiple layers of the CNN model; acquiring a test input pattern for the CNN model; and generating multiple test output patterns by inference using at least the multiple missing CNN models and the test input pattern. In other words, this program can be said to cause a computer to execute the procedure or method of the CNN input unit 1 and the like. Here, the memory 83 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), an EEPROM (Electrically Erasable Programmable Read Only Memory), an HDD (Hard Disk Drive), a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD (Digital Versatile Disc), a drive device for these, or any storage medium to be used in the future.

[0086] The above describes a configuration in which each function of the CNN input unit 1, etc. is realized either by hardware or software, etc. However, the present invention is not limited to this, and a configuration in which a part of the CNN input unit 1, etc. is realized by dedicated hardware and another part is realized by software, etc. For example, the functions of the CNN input unit 1 can be realized by a processing circuit 81, an interface, a receiver, etc. as dedicated hardware, and the functions of the other units can be realized by the processing circuit 81 as a processor 82 reading and executing a program stored in a memory 83.

[0087] As described above, the processing circuitry 81 can realize each of the above-mentioned functions by hardware, software, or a combination of these.

[0088] The transplantation support device described above can also be applied to a transplantation support system that is constructed as a system by appropriately combining a communication terminal and a server. In this case, each function or each component of the transplantation support device described above may be distributed among the devices that construct the system, or may be concentrated in one of the devices.

[0089] It should be noted that the embodiments and modifications may be freely combined, and the embodiments and modifications may be modified or omitted as appropriate. [Explanation of symbols]

[0090] 1 CNN input section, 2 CNN layer reduction section, 4 test pattern input section, 5 CNN inference section, 7 CNN model, 9 weights, 10 layers, 11 missing CNN model, 15 test input pattern, 16 test output pattern, 17 skeleton template input section, 18 skeleton generation section, 19 skeleton template, 23 skeleton code, 26 test pattern storage section with layer names, 27 skeleton generation section with layer names, 28 skeleton code with layer names.

Claims

1. A model acquisition unit that acquires a CNN model including a plurality of layers of a convolution neural network (CNN) and weights associated with the layers; A missing model generation unit that generates a plurality of missing CNN models in which layers are deleted in order from the last layer of the plurality of layers of the CNN model; A test pattern acquisition unit that acquires a test input pattern for the CNN model; an inference unit that generates a plurality of test output patterns by inference using at least the plurality of missing CNN models and the test input pattern; A transplantation assistance device comprising:

2. The transplantation assistance device according to claim 1, The inference unit is A transplantation support device that generates the plurality of test output patterns by inference using the CNN model, the plurality of missing CNN models, and the test input pattern.

3. The transplantation assistance device according to claim 2, a template acquisition unit that acquires a fragment of a skeleton code of a processing system to which the skeleton code is to be ported as a skeleton template; a skeleton generating unit that generates a plurality of skeletons corresponding to the plurality of layers, based on the plurality of test output patterns and the skeleton template; The transplantation assistance device further comprises:

4. The transplantation assistance device according to claim 3, A layer name extraction unit that extracts names of the plurality of layers from the CNN model, The skeleton generation unit A transplantation support device that generates the plurality of skeletons, each of which is assigned the names of the plurality of layers, based on the skeleton template, the plurality of test output patterns, and the names of the plurality of layers.

5. A model acquisition unit acquires a CNN model including a plurality of layers of a convolution neural network (CNN) and weights associated with the layers; A missing model generation unit generates a plurality of missing CNN models in which layers are deleted in order from the last layer of the plurality of layers of the CNN model; A test pattern acquisition unit acquires a test input pattern for the CNN model; a reasoning unit generating a plurality of test output patterns by reasoning using at least the plurality of missing CNN models and the test input pattern.

6. A means for acquiring a CNN model including a plurality of layers of a convolution neural network (CNN) and weights associated with the layers; A means for generating a plurality of missing CNN models in which layers are deleted in sequence from the last layer of the plurality of layers of the CNN model; A means for acquiring a test input pattern for the CNN model; A means for generating a plurality of test output patterns by inference using at least the plurality of missing CNN models and the test input patterns. A transplantation assistance program that makes your computer function.

Images