Model training device, model training method, and program

The model training device and method optimize CNN models by calculating and estimating FPN blocks for object detection, addressing the accuracy-speed trade-off and enhancing performance in real-time applications.

JP7593532B2Active Publication Date: 2024-12-03NEC CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2024518938
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-10-07
Publication Date
2024-12-03
Estimated Expiration
2041-10-07

AI Technical Summary

Technical Problem

There is a trade-off between accuracy and execution speed in Convolutional Neural Network (CNN) models due to the stacking of Feature Pyramid Network (FPN) blocks, which affects real-time applications.

Method used

A model training device and method that calculates the number of FPN blocks and estimates a parameter value based on the number of blocks required for object detection, using a design space reducer (DSR) to survey FPN block requirements and determine optimal candidate values for training the CNN model.

Benefits of technology

Resolves the trade-off between accuracy and execution speed by minimizing the number of FPN blocks required, resulting in a CNN model with high performance in speed and minimal loss in accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007593532000001
    Figure 0007593532000001
  • Figure 0007593532000002
    Figure 0007593532000002
  • Figure 0007593532000003
    Figure 0007593532000003
Patent Text Reader

Abstract

The objective of the present disclosure is to provide a model training apparatus, a model training method, and a non-transitory computer-readable medium that can resolve the trade-off between the accuracy and execution speed of a CNN model. In one aspect, the model training apparatus (100) includes an estimation unit (101) configured to calculate, for an object in a dataset previously used to train a convolutional neural network (CNN) model, the number of feature pyramid network (FPN) blocks in the CNN model required to detect the object, and estimate a parameter value of the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object, and a CNN model training unit (102) configured to train the CNN model by using the estimated parameter value.
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present disclosure relates to a model training apparatus, a model training method, and a non-transitory computer-readable medium. [Background technology]

[0002] In the era of machine learning, the Convolutional Neural Network (CNN) model is one of the most prominent solutions for visual analysis tasks. The ability to achieve high accuracy on large public datasets is a key reason for the success of CNN models.

[0003] As examples of related technologies, Patent Document 1 (PTL1) discloses a neural network optimization method for obtaining an optimal solution of a neural network, and PTL2 discloses a computing device that uses a neural network to improve recognition accuracy in a deep neural network (DNN). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-105950 [Patent Document 2] JP 2020-123270 A Summary of the Invention [Problem to be solved by the invention]

[0005] The Feature Pyramid Network (FPN) block is a feature extractor block responsible for extracting features of objects of various scales and complexities. Stacking several numbers of FPN blocks is beneficial to obtain high accuracy. However, in real-time applications, it is beneficial to stack fewer numbers of FPN blocks to obtain high speed. This presents a trade-off problem.

[0006] An objective of the present disclosure is to provide a model training apparatus, a model training method, and a non-transitory computer-readable medium that can resolve the trade-off between the accuracy and execution speed of a CNN model. [Means for solving the problem]

[0007] According to a first aspect of the present disclosure, there is a model training apparatus including: an estimation means for calculating, for an object in a dataset previously used to train a convolutional neural network (CNN) model, a number of feature pyramid network (FPN) blocks in the CNN model required to detect the object; and estimating a parameter value for the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object; and a CNN model training means for training the CNN model by using the estimated parameter value.

[0008] According to a second aspect of the present disclosure, there is a model training method that includes: for an object in a dataset previously used to train a convolutional neural network (CNN) model, calculating a number of feature pyramid network (FPN) blocks in the CNN model required to detect the object; estimating a parameter value for the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object; and training the CNN model by using the estimated parameter value.

[0009] According to a third aspect of the present disclosure, a non-transitory computer-readable medium is provided that stores a program for causing a computer to: calculate, for an object in a dataset previously used to train a Convolutional Neural Network (CNN) model, a number of Feature Pyramid Network (FPN) blocks in the CNN model required to detect the object; estimate a parameter value for the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object; and train the CNN model by using the estimated parameter value. Effect of the Invention

[0010] According to the present disclosure, it is possible to provide a model training apparatus, a model training method, and a non-transitory computer-readable medium that can resolve the trade-off between the accuracy and execution speed of a CNN model. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram showing an example of a block diagram of a model training device according to a first embodiment. [Diagram 2] 4 is a flowchart showing an example of a method of the model training device according to the first embodiment. [Diagram 3] FIG. 13 is a block diagram illustrating an example of a CNN model training system according to a second embodiment. [Figure 4] FIG. 13 is a diagram illustrating an example of a block diagram of a CNN model according to a second embodiment. [Diagram 5] FIG. 11 is a diagram illustrating an example of an FPN block according to the second embodiment. [Figure 6] FIG. 13 illustrates an example of a block diagram of a Design Space Reducer (DSR) unit according to a second embodiment. [Figure 7A] FIG. 13 is a diagram illustrating an example of a block diagram of a CNN model in which FPN_1 is disabled according to the second embodiment. [Figure 7B] FIG. 13 is a diagram illustrating an example of a block diagram of a CNN model in which FPN_n is disabled according to the second embodiment. [Figure 8] FIG. 11 is a diagram showing an example of a survey table according to the second embodiment. [Figure 9] FIG. 11 is a diagram illustrating an example of a cardinality table according to the second embodiment. [Figure 10] 11 is a flowchart showing an example of an overall process of the CNN model training system according to the second embodiment. [Figure 11A] 13 is a flow chart illustrating an example of a process of a survey table generator unit according to the second embodiment. [Figure 11B] 13 is a flow chart illustrating an example of a process of a survey table generator unit according to the second embodiment. [Figure 12] 11 is a flow chart illustrating an example of a process of an Optimal Candidate Estimation (OCE) unit according to a second embodiment. [Figure 13] FIG. 1 is a block diagram illustrating an example of an information processing device according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] (Summary of Related Technology) Prior to describing the embodiments of the present disclosure, an overview of the related art will be provided.

[0013] Designing a highly accurate CNN model for a given dataset requires both expertise and a large amount of time. A particular CNN model tends to have several configurable parameters and several candidates for each configurable parameter, thus requiring a very large design space in this case. However, performing an exhaustive search within such a large design space is computationally very expensive.

[0014] The Multi-Level Feature Pyramid Network (MLFPN) block is one of the most important blocks of a CNN model. The MLFPN block is formed by stacking multi-level feature pyramid network (FPN) blocks. However, as explained above, there is a trade-off between the accuracy and execution speed of a CNN model.

[0015] Also, since the configurable parameters can have any non-negative integer value, we need to explore several candidates in the design space to optimize the tradeoff. The number of stacked FPN blocks is one of the important configurable parameters of the MLFPN block (and CNN model). Therefore, exploring a large design space for the number of stacked FPN blocks with a tradeoff between accuracy and execution speed for a given dataset can be called an optimization problem.

[0016] The naive solution exhaustively enumerates all candidates in the design space, but requires a large computation time. Another solution is retry-based optimization, where a small number of candidates are tried first, and then the next candidate is selected for the trail based on the knowledge acquired from the trail. However, the time required to explore only a small number of candidates is also large.

[0017] In view of this related art, one of the objectives of the present disclosure is to reduce the time to search the space for the number of FPN block parameters. One specific solution is to use a Design Space Reducer (DSR) block to perform a survey of FPN block requirements by individual objects in a dataset, and based on the survey, an estimate of the number of FPN block parameters is made.

[0018] (First embodiment) First, with reference to FIG. 1, a model training device 100 according to a first embodiment of the present disclosure will be described.

[0019] 1, the model training device 100 includes an estimation unit 101 and a convolutional neural network (CNN) model training unit 102. The model training device 100 includes, for example, one or more computers or machines having an information processing device. As an example, at least one of the components in the model training device 100 may be mounted on a computer as a combination of one or more memories and one or more processors.

[0020] The estimation unit 101 calculates, for an object in a dataset previously used to train the CNN model, the number of FPN blocks in the CNN model required to detect the object. Further, the estimation unit 101 estimates a parameter value for the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object.

[0021] A dataset can include images and labels. The images contain the objects to be analyzed by the CNN model, and the labels indicate ground truth information. In other words, the labels are the predictions that the trained CNN model is intended to generate as output.

[0022] The multiple objects for which the estimation unit 101 calculates the number of FPN blocks may be all or a portion of the objects in the dataset.

[0023] The FPN blocks can be stacked in, but not limited to, an MLFPN structure, a Single Shot Multibox Detector (SSD) style feature pyramid structure, or an FPN style feature pyramid structure. In the second embodiment, an example of a CNN model structure including MLFPN blocks is described.

[0024] The CNN model training unit 102 trains the CNN model by using the estimated parameter values.

[0025] Next, an example of the operation of this embodiment will be described with reference to the flowchart of FIG.

[0026] First, the estimation unit 101 calculates the number of feature pyramid network (FPN) blocks in the convolutional neural network (CNN) model required to detect the object for an object in a dataset previously used to train the CNN model (step S11 in FIG. 2). Next, the estimation unit 101 estimates a parameter value of the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object (step S12 in FIG. 2). Finally, the CNN model training unit 102 trains the CNN model by using the estimated parameter value (step S13 in FIG. 2).

[0027] By estimating parameter values ​​based on the number of FPN blocks required for the estimation unit 101 to detect an object, the CNN model training unit 102 can train the CNN model so that the number of FPN blocks is the minimum required. Therefore, a model training apparatus and method capable of resolving the trade-off between the accuracy and execution speed of the CNN model can be provided.

[0028] (Second embodiment) Next, a second embodiment of the present disclosure will be described below with reference to the accompanying drawings. The second embodiment shows a specific example of the first embodiment, but the specific example of the first embodiment is not limited thereto.

[0029] FIG. 3 shows a CNN model training system (hereinafter referred to as a model training system) according to a second embodiment. As shown in FIG. 3, the model training system 200 includes a CNN model training unit 210 (hereinafter referred to as a model training unit), a trained CNN model 220 (hereinafter referred to as a CNN model), a design space reducer (DSR) unit 230, an optimized CNN model 240, etc. For example, the CNN model training system 200 may also include a memory for storing configurations, conditions, and thresholds used in the processing described later. In addition, the model training system 200 may receive a training dataset TD as an input and output the CNN model 220 and the optimized CNN model 240.

[0030] The model training system 200 can be realized as a system mounted on an information processing device. The model training system 200 may also include other units for calculation. In this embodiment, the model training system 200 can be applied to image recognition, but the application is not limited thereto.

[0031] The training dataset TD includes images along with the labels of each image. The descriptions of the images and labels are as described in the first embodiment.

[0032] The model training unit 210 takes the training dataset TD as input, constructs an original CNN model based on the training dataset TD and a predefined configuration, performs training of the original CNN model, and outputs the training result, i.e., the CNN model 220, which is used as the input of the DSR unit 230.

[0033] Furthermore, after this first training, the model training unit 210 takes the training data set TD and the optimal candidate value (the number of FPN blocks of the CNN model) as input, and sets the optimal candidate value as a defined configuration, not a predefined configuration, of the value of the number of FPN blocks. The optimal candidate value is a non-negative integer. In this way, the model training unit 210 uses the new optimal candidate value to build a new CNN model, and then trains it by using the training data set TD.

[0034] In this procedure, the model training unit 210 may set the initial weights of the new CNN model to those of the CNN model 220. The model training unit 210 can then fine-tune the weights of the new CNN model by training it using the training dataset TD.

[0035] Thus, the model training unit 210 outputs the optimized CNN model 240, which has relatively high performance in terms of speed compared to the CNN model 220 with minimal loss of accuracy. The method of generating the optimal candidate value information will be described in detail later.

[0036] In Fig. 4, an example of a block diagram of a CNN model 220 is shown. As shown in Fig. 4, the CNN model 220 includes several vital blocks, specifically, a backbone block 221, a direct sum block 222, a multi-level feature pyramid network (MLFPN) block 223, and a predictor block 224. The CNN model 220 takes an image IM of a training dataset TD as input, processes it, and outputs an output OU based on the image IM.

[0037] The architecture of each block in the CNN model 220 is designed with the goal of achieving high accuracy for a given data set, and for real-time applications, the architecture in the CNN model 220 is additionally designed with the goal of achieving a desired execution speed.

[0038] The backbone block 221 functions as a backbone convolution architecture, and the direct sum block 222 integrates the data input from the backbone block 221. The MLFPN block 223 processes the data input from the direct sum block 222 and outputs the processed data to the predictor block 224. The predictor block 224 generates a prediction result for the detection of the image IM based on the processed data, and outputs the prediction as an output OU.

[0039] The MLFPN block 223 is one of the key components in the CNN model 220. As shown in Fig. 4, the MLFPN block 223 is known as a "multi-level feature pyramid network block" because it is formed by stacking multiple levels of FPN blocks 225. In Fig. 4, there are n FPN blocks, FPN_1, FPN_2, ..., FPN_n.

[0040] The FPN block is a feature extractor block responsible for extracting features of objects of various scales and complexities. A sample example of the FPN block 225 is shown in FIG. 5, but is not limited thereto. FIG. 5 shows the architecture of the FPN block, including layers L1, L2, L3, ..., L4, L5, ..., L6, and L7. Each of the layers L1-L4 in the bottom-up step generates feature maps of different scales, and each of the layers L5-L7 in the top-down step outputs a prediction based on the feature map from the corresponding layer in the bottom-up step. For example, layer L5 outputs a prediction P1 using the feature map from layer L4, layer L6 outputs a prediction P2 using the feature map from layer L3, and layer L7 outputs a prediction P3 using the feature map from layer L2. The architecture of one FPN block 225 can be optimized by an expert, and then such optimized FPN blocks are stacked to form the MLFPN block 223.

[0041] Stacking a large number of FPN blocks 225 is beneficial to obtain high accuracy. However, increasing the level of FPN blocks in the MLFPN block 223 increases the computational complexity of the CNN model, which increases the execution time of the CNN model. The large execution time is not acceptable for real-time applications. Also, stacking one or a few levels of FPN blocks in the CNN model to enjoy lower computational complexity and lower execution time (faster) may result in low accuracy. Therefore, there is a trade-off problem, and the number of FPN blocks in the MLFPN block 223 is an important parameter of the CNN model, which is called the FPN count value in this embodiment. It is an optimization problem to determine the optimal value of the FPN count value considering the trade-off between accuracy and computational complexity or execution time for a given real-time application and dataset. This second embodiment can solve this problem.

[0042] The DSR unit 230 receives the training data set TD and the CNN model 220 as input, performs analysis of the CNN model 220, and estimates and outputs an optimal candidate value (the number of FPN blocks of the CNN model). As described above, the model training unit 210 outputs an optimized CNN model 240 based on the optimal candidate value.

[0043] Figure 6 shows the details of the DSR unit 230. As shown in Figure 6, the DSR unit 230 includes a survey table generator unit 231 and an optimal candidate estimation (OCE) unit 232. The details of both units will now be described.

[0044] The survey table generator unit 231 takes the training dataset TD and the CNN model 220 as input, and generates a survey table as output. First, the survey table generator unit 231 performs a survey of the number of FPN blocks in the CNN model 220 required to detect the object according to each object in the training dataset TD.

[0045] Specifically, in the survey procedure, when the survey table generator unit 231 inputs each object in the training dataset TD into the CNN model 220, the survey table generator unit 231 sets the weight of any one particular FPN block 225 in the MLFPN block 223 as 0. In other words, the survey table generator unit 231 disables one particular FPN block 225.

[0046] 7A shows an example of the above-mentioned situation, in which the survey table generator unit 231 sets the weight of FPN_1 as 0, so that FPN_1 is masked. In FIG. 7A, FPN_1 is invalid, while FPN_2, ..., FPN_n are valid for processing image detection using the training dataset TD. In this state, the survey table generator unit 231 enumerates all images in the trained dataset TD and inputs all images into the CNN model 220.

[0047] By masking FPN_1 (a particular FPN block), some of the objects in the image may be detected and other objects may not be detected. The survey table generator unit 231 may determine whether each object in the image can be detected or not, and this information is recorded in the form of a survey table as "detected" or "not detected."

[0048] The survey table generator unit 231 sequentially performs the same determination of FPN_1 for other FPNs (FPN_2, ..., FPN_n). In other words, in one step of the sequence, the survey table generator unit 231 disables any one particular FPN block for processing of image detection, inputs all images into the CNN model 220, and determines whether each object in the image can be detected. FIG. 7B shows an example of the last step of the sequence, in which the survey table generator unit 231 sets the weight of FPN_n as 0, and thus FPN_n is masked. This survey method can be called "weight zeroing-based survey", which is the way in which the survey table generator unit 231 sequentially infers all images in the trained dataset TD.

[0049] Fig. 8 shows an example of a survey table. In Fig. 8, column C0 shows a list of all objects in all images, column C1 shows a list of detection results for each object when FPN_1 is invalid, column C2 shows a list of detection results for each object when FPN_2 is invalid, and column Cn shows a list of detection results for each object when FPN_n is invalid. Detection results for each object from FPN_3 to FPN_n-1 are also included in the survey table as columns C3 to Cn-1, but are not shown in Fig. 8.

[0050] The survey table generator unit 231 records the information in columns C1 to Cn, and after verifying that all FPN blocks have been covered, counts the number of detection results of FPN blocks in each object that could not be detected.

[0051] For example, for image1_object1, the survey table generator unit 231 refers to the results for all FPN blocks in the same row and counts the number determined to be "not detected." Here, since the determined number is 4, the survey table generator unit 231 determines the requirement number to be 4. This requirement number means the number of FPN blocks in the CNN model 220 required to detect image1_object1. Similar to image1_object1, the survey table generator unit 231 determines the requirement number for image1_object2 to be 2, and the requirement number for image1_object3 to be 3, and so on.

[0052] "Detected" in FIG. 8 basically means that the corresponding FPN block is not important for detecting the corresponding object. Even if the corresponding FPN block is masked, the corresponding object is still detected, so the corresponding FPN block is not necessary to detect the object. However, "Not Detected" in FIG. 8 means that the corresponding FPN block is necessary for detecting the corresponding object, because masking the corresponding FPN block leads to non-detection of the object. Thus, the count in each row represents the number of important FPN blocks for each object. In the example of FIG. 8, the number of important FPN blocks for image 1_object 1 is 4, the number of important FPN blocks for image 1_object 2 is 2, and the number of important FPN blocks for image 1_object 3 is 3. The table thus formed is called a survey table, since it is like conducting a requirement survey of the number of FPN blocks per object in the training dataset TD.

[0053] In this way, for each row in the survey table, the survey table generator unit 231 counts the number of “not found” entries in the same respective row, and records the count value (number of requirements) in the requirement column Cn+1, respectively, as shown in Fig. 8. After completely generating the survey table, the survey table generator unit 231 outputs the survey table.

[0054] The OCE unit 232 takes the survey table as input and performs an analysis of the survey table to estimate the best candidate value for the number of FPN blocks used to generate the optimized CNN model 240.

[0055] First, the OCE unit 232 lists all unique entries (representation values) in the requirement column Cn+1 of the survey table in ascending order. Each entry in the requirement column Cn+1 is recorded in the form of a new table (hereinafter referred to as a cardinality table) and can be called a candidate value. Then, the OCE unit 232 calculates the occurrence rate (percentage) of each candidate value based on the data of the candidate value and records it in the table. In other words, the OCE unit 232 calculates the distribution of the occurrence rate. After that, the OCE unit 232 calculates the cardinality of each candidate value and records it in the table. The cardinality (probability) of each candidate value is the cumulative sum of the occurrence rates below each candidate value.

[0056] Fig. 9 shows an example of a cardinality table. In Fig. 9, a candidate column C10 shows candidate values ​​(unique entries) listed in ascending order, an occurrence column C11 shows the occurrence rate (percentage) of each candidate value, and a cardinality column C12 shows the cardinality of each candidate value. The OCE unit 232 generates the cardinality table by a log procedure.

[0057] Each entry (value) of the cardinality column C12 represents the probability of detecting an object when the candidate value corresponding to each entry (i.e., the candidate value listed on the same line as each entry) is the number of FPN blocks required to detect the object. For example, FIG. 9 shows that when the number of FPN blocks in the CNN model 220 is one, the object can be detected at a rate of 43%, and when the number of FPN blocks is two, the object can be detected at a rate 25% higher than when the number of FPN blocks is one. Furthermore, when the number of FPN blocks is three, the object can be detected at a rate 4% higher than when the number of FPN blocks is two, and when the number of FPN blocks is four, the object can be detected at a rate 13% higher than when the number of FPN blocks is three. Also, when the number of FPN blocks is five, the object can be detected at a rate 3% higher than when the number of FPN blocks is four, and when the number of FPN blocks is n, the object can be detected at a rate 1% higher than when the number of FPN blocks is n-1.

[0058] Based on the specific data, the OCE unit 232 calculates the cardinality of each candidate value as follows: The OCE unit 232 calculates the cardinality of the candidate value "1" as the same number as the cardinality of the candidate value "1", i.e., 43%. Then, the OCE unit 232 calculates the cardinality of the candidate value "2" as the cumulative sum of the cardinalities of the candidate values ​​"1" and "2", i.e., 68% (43% + 25%). Then, the OCE unit 232 calculates the cardinality of the candidate value "3" as the cumulative sum of the cardinalities of the candidate values ​​"1", "2", and "3", i.e., 72% (43% + 25% + 4%). Furthermore, the OCE unit 232 calculates the cardinality of candidate value “4” as the cumulative sum of the cardinalities of candidate values ​​“1” to “4”, i.e., 85% (43%+25%+4%+13%), and calculates the cardinality of candidate value “5” as the cumulative sum of the cardinalities of candidate values ​​“1” to “5”, i.e., 88% (43%+25%+4%+13%+3%).

[0059] After calculating the cardinality of all the candidate values, the OCE unit 232 selects one candidate value whose cardinality value is less than and closest to a predefined threshold according to a predefined condition. The predefined threshold can be set based on experience with experimental results. Then, the OCE unit 232 outputs the one candidate value as the optimal candidate value (parameter value) for the number of FPN blocks in the CNN model 220.

[0060] In the example of FIG. 9, since the predefined threshold is 86% and the cardinality of the candidate value "4" satisfies the above conditions, the OCE unit 232 selects "4" as the optimal candidate value.

[0061] After these procedures, the model training unit 210 sets the optimal candidate value input from the OCE unit 232 as the defined configuration, and generates an optimized CNN model 240 based on the optimal candidate value. Thus, the optimized CNN model 240 has the same number of FPN blocks stacked into the MLFPN block as indicated by the optimal candidate value. The optimal candidate value not only ensures that the accuracy of object detection is close to a predefined value and a certain degree of accuracy is guaranteed, but also ensures that the number of blocks in the optimized CNN model 240 is minimized. Therefore, the optimized CNN model 240 has a relatively high performance in terms of speed compared to the CNN model 220 with the minimum loss of accuracy.

[0062] Next, an example of the overall process of the CNN model training system 200 will be described with reference to the flowchart of Fig. 10. Details of each step have been described above.

[0063] First, the model training unit 210 constructs an original CNN model based on the training dataset TD and a predefined configuration, and trains the original CNN model to output the CNN model 220 (step S21 in FIG. 10). Then, the DSR unit 230 conducts a survey in the survey table generator unit 231 (step S22 in FIG. 10), and estimates the optimal candidate value in the OCE unit 232 (step S23 in FIG. 10). Finally, the model training unit 210 constructs a new CNN model using the new optimal candidate value, and then trains the optimized CNN model 240 (step S24 in FIG. 10).

[0064] Further, an example of the process of the survey table generator unit 231 will be described with reference to the flowcharts of Figures 11A and 11B. In other words, the details of the process of step S12 are given below.

[0065] First, the survey table generator unit 231 sets the weight of the Xth ("X" is any natural number from 1 to n) FPN block in the CNN model 220 as 0 (step S31 in FIG. 11A). Then, the survey table generator unit 231 inputs each object in the training data set TD into the CNN model 220 and surveys whether each object in the image can be accurately detected. That is, the survey table generator unit 231 surveys the Xth FPN block and records this information in the survey table (step S32 in FIG. 11A).

[0066] After that, the survey table generator unit 231 checks whether all FPN blocks have been surveyed (step S33 in FIG. 11A). If not (No in step S33), the survey table generator unit 231 changes X to X+1 (step S34 in FIG. 11A) and repeats the procedure of steps S31 to S33. In other words, the survey table generator unit 231 surveys the X+1th FPN block and records this information in the survey table. This iteration ends when all FPN blocks have been surveyed (Yes in step S33).

[0067] When all FPN blocks have been surveyed, in other words, when the information in columns C1 to Cn has already been surveyed, the survey table generator unit 231 calculates the number of requirements for each object in the training data set TD (step S35 in FIG. 11B). Finally, the survey table generator unit 231 records and outputs the information of the number of requirements in the survey table (step S36 in FIG. 11B).

[0068] Next, an example of the overall process of the OCE unit 232 will be described with reference to the flowchart of Fig. 12. Details of each step are as described above.

[0069] First, the OCE unit 232 organizes the information of the candidate values ​​and calculates the occurrence rates of all the candidate values ​​based on the data of the candidate values ​​(step S41 in FIG. 12). Then, the OCE unit 232 calculates the cardinality of all the candidate values ​​based on the occurrence rates (step S42 in FIG. 12). After that, the OCE unit 232 selects one optimal candidate value according to a predefined condition (step S43 in FIG. 12).

[0070] As described above, one of the challenges in the related art is the optimization problem. In this embodiment, the DSR unit 230 calculates, for each object in the training dataset TD, the number of FPN blocks in the CNN model 220 required to detect the object, and estimates an optimal candidate value (parameter value) of the number of FPN blocks in the CNN model 220 based on the calculated number of FPN blocks for each object. Then, the CNN model training unit 210 trains the optimized CNN 240 model by using the optimal candidate value. For the same reason as described in the first embodiment, the design space of the parameter value of the number of FPN blocks can be reduced, and the optimization problem can be solved.

[0071] Furthermore, the survey table generator unit 231 in the DSR unit 230 may determine whether each object is detected by the CNN model 220 by inputting the training data set TD to the CNN model 220 with one of the FPN blocks disabled, and calculate the number of FPN blocks for each object by performing the determination for every FPN block individually. This method can simplify the calculation.

[0072] Furthermore, the OCE unit 232 in the DSR unit 230 may calculate the distribution of the occurrence rate of the number of FPN blocks for the object, and estimate the optimal candidate value. Specifically, the OCE unit 232 may calculate the probability of each object being detected when the number of FPN blocks is equal to or less than the optimal candidate value based on the distribution, and estimate the parameter value. According to this method, it is possible to perform a highly accurate calculation that achieves the above-mentioned effects, and it is possible to obtain the optimal candidate value in a shorter time.

[0073] Furthermore, the FPN block in the CNN model 220 may constitute an MLFPN block in the CNN model, and the CNN model training unit may adjust the MLFPN block in the optimized CNN model 240 as indicated by the optimal candidate value. Therefore, the method is effective for a CNN model that includes an MLFPN block.

[0074] It should be noted that in the description of this disclosure, elements referred to by the singular forms "a," "an," and "the" may refer to a single element as well as a plural element, unless the context clearly dictates otherwise.

[0075] It should be noted that the present disclosure is not limited to the above-described embodiment, and can be appropriately modified without departing from the spirit and scope of the present invention. For example, the target for calculation is not limited to an image.

[0076] In the second embodiment, all objects in the training dataset TD are subject to calculations by the DSR unit 230, but not all objects (especially important objects) in the training dataset TD are necessarily subject to calculations by the DSR unit 230.

[0077] The order in which the survey table generator unit 231 masks the FPN blocks is not limited to the method described in the second embodiment.

[0078] In a second embodiment, the OCE unit 232 may adopt another predefined condition that the cardinality value of the selected candidate value exceeds a predefined threshold and is closest to the predefined threshold. Alternatively, the predefined condition may be that the cardinality value of the selected candidate value is closest to the predefined threshold. The predefined threshold may be set based on experience with experimental results.

[0079] As yet another example, the OCE unit 232 may calculate a distribution of the occurrence rates of the candidate values ​​and estimate an optimal candidate value based on the distribution. For example, the optimal candidate value may be selected that is closest to a value that has a certain deviation (a predefined threshold) from the mean value of the distribution. It should be noted that the optimal candidate value may be smaller or larger than the value that has a certain deviation from the mean value, or may be the value that is closest to the value that has a certain deviation from the mean value.

[0080] (Hardware configuration) Next, with reference to FIG. 13, a configuration example of the information processing device described in the above-mentioned embodiments will be described below.

[0081] 13 is a block diagram showing an example of the configuration of an information processing device 90. As shown in FIG.

[0082] The network interface 91 is used for communication with other network node devices forming a communication system. For example, the network interface 91 can receive a training data set TD.

[0083] The processor 92 loads software (computer programs) from the memory 93 and executes the loaded software to implement the processes described with reference to the drawings in the above-mentioned embodiments. The processor 92 may be, for example, a microprocessor, an MPU (microprocessing unit), or a CPU (central processing unit). The processor 92 may include multiple processors. In this case, each of the processors executes one or more programs including a group of instructions to cause the computer to implement the algorithms described above with reference to the drawings.

[0084] The memory 93 may be formed by a volatile memory or a non-volatile memory, or may be formed by a combination of a volatile memory and a non-volatile memory. The memory 93 may include a storage device located remotely from the processor 92. In this case, the processor 92 may access the memory 93 through an I / O interface (not shown).

[0085] 13, the memory 93 is used to store software modules. The processor 92 can perform the processes described in the above embodiments by reading the software modules from the memory 93 and executing the read software modules. The memory 93 can also store the predefined threshold values ​​described above.

[0086] As described above, each of the configurations in the above-described embodiments may be configured by a combination of hardware and software (computer programs). However, each of the configurations may be configured by one piece of hardware or software, or may be configured by multiple pieces of hardware or software.

[0087] The program includes instructions (or software codes) that, when loaded into a computer, cause the computer to perform one or more of the functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or tangible storage medium. By way of example and not limitation, the non-transitory computer-readable medium or tangible storage medium may include random access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disk (DVD), Blu-ray disk (®) or other optical disk storage device, magnetic cassette, magnetic tape, magnetic disk storage device, or other magnetic storage device. The program may be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, the transitory computer-readable medium or communication medium may include electrical, optical, acoustic, or other form of propagated signal.

[0088] It should be noted that the present disclosure is not limited to the above-described embodiments, and can be appropriately modified without departing from the spirit and scope of the present disclosure. [Explanation of symbols]

[0089] 100 Model Training Device 101 Estimated Units 102 Convolutional Neural Network (CNN) model training units 200 CNN model training system 210 Convolutional Neural Network (CNN) model training units 220 pre-trained CNN models 221 Backbone Block 222 Direct Sum Block 223 Multi-Level Feature Pyramid Network (MLFPN) Block 224 Predictor Block 225 FPN Block 230 Design Space Reducer (DSR) Unit 231 Survey Table Generator Unit 232 Optimal Candidate Estimation (OCE) Unit 240 Optimized CNN Models

Claims

1. an estimation means for calculating, for an object in a dataset previously used to train a convolutional neural network (CNN) model, a number of feature pyramid network (FPN) blocks in the CNN model required to detect the object, and estimating a parameter value for the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object; a CNN model training means for training the CNN model by using the estimated parameter values; A model training device comprising:

2. The estimation means further determines whether the object is detected by the CNN model by inputting the data set to the CNN model with one of the FPN blocks disabled, and calculates the number of the FPN blocks for the object by performing the determination individually for all the FPN blocks.

2. The model training apparatus of claim 1.

3. The estimation means further calculates a distribution of occurrence rates of the number of FPN blocks for the object and estimates the parameter value.

3. A model training device according to claim 1 or 2.

4. The estimation means further calculates a probability that the object is detected when the number of the FPN blocks is equal to or less than the parameter value based on the distribution, and estimates the parameter value.

4. The model training apparatus of claim 3.

5. The FPN block in the CNN model constitutes a multi-level feature pyramid network (MLFPN) block in the CNN model; The CNN model training means adjusts the MLFPN block in the CNN model as indicated by the estimated parameter values. A model training device according to any one of claims 1 to 4.

6. For an object in a dataset previously used to train a convolutional neural network (CNN) model, calculating the number of feature pyramid network (FPN) blocks in the CNN model required to detect the object; estimating a parameter value for the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object; training the CNN model by using the estimated parameter values; A method for training a model, comprising:

7. On the computer, For an object in a dataset previously used to train a convolutional neural network (CNN) model, calculating the number of feature pyramid network (FPN) blocks in the CNN model required to detect the object; estimating a parameter value for the number of FPN blocks in the CNN model based on the calculated number of FPN blocks for the object; training the CNN model by using the estimated parameter values; A program for executing.

Citation Information

Patent Citations

  • Arithmetic unit

    JP2020123270A

  • Neural network optimizing method, neural network optimizing device and program

    JP2021105950A