Information processing device, estimation method, learning method, program, and recording medium

Abstract

To reduce the calculation cost in a learning model and improve the accuracy of inference when performing learning or inference by using the learning model.SOLUTION: An information processing apparatus has: image acquisition means that acquires a plurality of narrow area images that are partial images of a target image and are images including ranges of the target image different from each other, and a wide area image that is an image in which the amount of information on the target image is reduced and is an image including a wider range of the target image than those of the plurality of narrow area images; and information acquisition means that acquires information correlated to the target image from a learning model to which the plurality of narrow area images and the wide area image are input.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an estimation method, a learning method, and a program. , recording media Regarding. [Background technology]

[0002] A pre-trained model is used to predict a given result based on an arbitrary image.

[0003] Patent document 1 proposes a technology in which, when predicting a predetermined result based on an entire image, a global judgment is made using the entire image, and then a detailed judgment is made based on the global judgment result and a partial image cut out from the entire image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2020-187693 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in Patent Document 1, when the entire image has a high resolution, the entire image may contain a lot of information that is unnecessary for learning and inference. Therefore, if the entire image is used for learning and inference as is, there is a risk that a large calculation cost will be incurred in the learning model and the inference accuracy will be reduced.

[0006] Therefore, an object of the present invention is to reduce the calculation cost of a learning model and improve the prediction accuracy when learning or prediction is performed using a learning model. [Means for solving the problem]

[0007] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. Target image represents a part of a plurality of narrow-band images and an image having a reduced amount of information compared to the target image, and the plurality of narrow-area images including a part of the than the target image Nohiro A wide range represent image acquisition means for acquiring a wide-area image; an information acquisition means for acquiring information correlated with the target image from a learning model to which the plurality of narrow-area images and the wide-area image are input; The information processing device is characterized by having:

[0008] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. Target image represents a part of a plurality of narrow-area images and an image in which the amount of information of the target image is reduced, and the plurality of narrow-area images including a part of the than the target image Nohiro A wide range represent image acquisition means for acquiring a wide-area image; a learning means for learning a learning model based on the plurality of narrow-area images, the wide-area image, and information correlated with the target image; The information processing device is characterized by having:

[0009] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. Target image represents a part of a plurality of narrow-band images and an image having a reduced amount of information compared to the target image, and the plurality of narrow-area images including a part of the than the target image Nohiro A wide range represent an image acquisition step of acquiring a wide-area image; an information acquisition step of acquiring information correlated with the target image from a learning model to which the plurality of narrow-area images and the wide-area image are input; A propellant characterized by having measurement It is a method.

[0010] One aspect of the present invention is a method for producing a medicament for the treatment of a pulmonary arthritis. Target image represents a part ofa plurality of narrow-area images and an image in which the amount of information of the target image is reduced, and the plurality of narrow-area images including a part of the than the target image Nohiro A wide range represent an image acquisition step of acquiring a wide-area image; a learning step of learning a learning model based on the plurality of narrow-area images, the wide-area image, and information correlated with the target image; The learning method is characterized by having the following. [Effects of the Invention]

[0011] According to the present invention, when learning or inference is performed using a learning model, it is possible to reduce the calculation cost of the learning model and improve the accuracy of inference. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a configuration diagram of an information processing device according to a first embodiment. [Figure 2] FIG. 2 is a diagram illustrating the internal configuration of an image processing unit according to the first embodiment. [Figure 3] FIG. 2 is a diagram illustrating generation of a divided image according to the first embodiment. [Figure 4] 3A to 3C are diagrams illustrating generation of a compressed image according to the first embodiment. [Figure 5] FIG. 2 is a diagram illustrating the internal configuration of an information acquisition unit according to the first embodiment. [Figure 6] FIG. 4 is a diagram illustrating a learning phase according to the first embodiment. [Figure 7] FIG. 1 is a diagram illustrating a CNN according to a first embodiment. [Figure 8] FIG. 10 is a diagram illustrating an estimation phase according to the first embodiment. [Figure 9] FIG. 10 is a diagram illustrating a learning phase according to the third embodiment. [Figure 10] FIG. 10 is a diagram illustrating a learning phase according to a comparative example. [Figure 11] FIG. 10 is a diagram illustrating an estimation phase according to a comparative example. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments for realizing the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all combinations of features described in the present embodiments are necessarily essential to the present invention. Furthermore, the following Examples 1 to 3 can be arbitrarily combined as long as no contradiction occurs.

[0014] <Embodiment 1> 1 is a configuration diagram of an information processing device 1 according to embodiment 1. The information processing device 1 trains a learning model in a learning phase, and infers information correlated with (corresponding to) an input image (target image) in an estimation phase. Therefore, the information processing device 1 can be regarded as a learning device in the learning phase and as an estimation device in the estimation phase.

[0015] The information that the information processing device 1 infers from the input image may be any information. The information to be inferred may be information about the future of the object shown in the image, information about the past of the object shown in the image, characteristics of the object shown in the image (physical properties, attributes, etc.), etc. In learning for inference, a teacher image and highly accurate teacher information corresponding to that teacher image may be input as a pair, and the correlation between the image and the information may be learned. Note that the input image may be used to model an object. It may be a design image, an RGB image of an object, a distance image in which each pixel indicates the distance to the object, or a temperature image in which each pixel indicates the temperature of the object.

[0016] For example, the information processing device 1 may estimate the production cost (manufacturing cost) of a product from an image that shows the shape of the product (manufacturing object). The information processing device 1 may estimate the completion time or construction period from an image that shows the progress of a construction site or product manufacturing. The information processing device 1 may estimate a person's age from a photograph of their face. The information processing device 1 may estimate information related to life, such as the incidence rate of disease, survival rate, and life expectancy, from images obtained by diagnostic imaging such as CT scans.

[0017] The information processing device 1 may also estimate the calories and expiration date of food from an image of the food. The information processing device 1 may also estimate the degree of deterioration of an object, such as a building or furniture, from an image of the object. The degree of deterioration may be, for example, the remaining useful life of the object, or a value where a new item is set to 0% and a broken item is set to 100%. The information processing device 1 may also estimate the price of an item, such as a car or a house, from an image of the item. In this case, the price may be the selling price (purchase price), the resale price, or the like. The information processing device 1 may also estimate the catch from an image of a tidal current. The information processing device 1 may also estimate weather-related information, such as the amount of precipitation or the time when a typhoon will be closest to the object, from a weather image. The information processing device 1 may also estimate the type and condition (body length and weight) of an organism from an image of the organism. The estimated condition of the organism may be any of the present, past, and future states. The information processing device 1 may also estimate the proportions of ingredients in a product from a scanned image of the product. The information processing device 1 may estimate the year and month when a photographed image of a person or a landscape was taken.

[0018] The information processing device 1 may estimate future prices from an image of a graph (chart) showing past price fluctuations, and in this case the price may be the price of any of movable property, real estate, securities, or commodities (products or services). The information processing device 1 may also estimate the condition of a living thing such as a horse or a person from an image of the living thing, and may estimate short-term or long-term future prospects based on the condition. For example, the short-term future prospects of a racehorse may be the winning percentage and predicted time in its next race, and the long-term future prospects of a racehorse may be the winning percentage and number of wins in the following year's races.

[0019] The information processing device 1 may estimate the distance from a camera to a subject such as a living thing or a building, based on an RGB image of the subject captured by the camera. The information processing device 1 may also estimate the condition of a space (the concentration of each liquid or gas in the space, or the humidity and temperature in the space) based on an image of the space. The information processing device 1 may also estimate information about living things inhabiting a specific place, such as a forest or a river, based on an image of the place (for example, the population of a specific living thing such as deer or bear, or the average age of trees).

[0020] The information processing device 1 includes an image acquisition unit 10, an image processing unit 11, an information acquisition unit 12, an output unit 13, and an adjustment unit 14.

[0021] The image acquisition unit 10 acquires an input image (target image) obtained by capturing an arbitrary subject. The input image may be a photograph taken by a digital camera, or an image captured from a 3D-CAD model. For example, the input image may be an image of a three-dimensional (3D) object viewed from a single direction, or a multi-viewpoint image of a 3D object viewed from multiple viewpoints (multi-viewpoints). The input image may be in any of a variety of formats, such as JPEG or PNG. The image acquisition unit 10 stores the input image in a specific file storage location. The input image is stored in

[0022] The image processing unit 11 performs image processing on the input image acquired by the image acquisition unit 10, and outputs the processed image to the information acquisition unit 12. The image processing unit 11 performs image processing to acquire a plurality of divided images (narrow area images) that are images of parts of the input image, and to acquire one compressed image (wide area image) in which the amount of information of the input image is reduced.

[0023] The information acquiring unit 12 acquires information (output data) output from the learning model by inputting the image processed by the image processing unit 11 to the learning model. If the learning model has already been trained, the information acquiring unit 12 can acquire an inference result corresponding to (correlated with) the input image from the learning model as output data. Details of the learning model will be described later with reference to FIGS. 5 and 6.

[0024] The output unit 13 outputs the output data acquired by the information acquisition unit 12. Therefore, when the information acquisition unit 12 acquires an inference result as output data, the output unit 13 outputs the inference result.

[0025] During the learning phase, the adjustment unit 14 adjusts (trains) the learning model based on the output data acquired by the information acquisition unit 12 and the correct answer data (correct answer information that the information acquisition unit 12 should acquire). The correct answer data is information associated with the input image and prepared in advance. The correct answer data does not need to be accurate information; it need only be information correlated with the input image and plausible information. For example, correct answer data indicating the height of a building associated with an input image showing a 10.5-meter-tall building may be a value indicating the exact height of 10.5 meters, or a value indicating an approximate height of 10 meters. Note that the adjustment unit 14 is a component that operates during the learning phase and does not operate during the estimation phase. Therefore, during the learning phase, the learning model can be trained based on multiple segmented images, compressed images, and correct answer data by passing through the information acquisition unit 12, the output unit 13, and the adjustment unit 14. Therefore, during the learning phase, the information acquisition unit 12, the output unit 13, and the adjustment unit 14 can be collectively referred to as a learning unit.

[0026] (Image processing unit) The internal configuration of the image processing unit 11 will be described in detail with reference to Fig. 2. The image processing unit 11 has an input unit 21, an image dividing unit 22, a divided image output unit 23, an information amount reduction unit 24, and a compressed image output unit 25.

[0027] The input unit 21 acquires the input image stored by the image acquisition unit 10 and expands it in memory. The input unit 21 may perform any processing (resolution adjustment, cropping, or color tone adjustment) on the input image as necessary. In this specification, resolution refers to the screen resolution (total number of pixels) which represents the product of the number of vertical pixels and the number of horizontal pixels. Therefore, reducing the resolution of an image is the same as reducing the number of pixels of the image.

[0028] The image dividing unit 22 divides the input image that the input unit 21 has expanded in memory. For example, as shown in FIG. 3, the image dividing unit 22 divides the input image 31 into four parts to generate four divided images 32. The image dividing unit 22 generates the divided images 32 so that the resolution of the divided images 32 matches the resolution defined in the divided image input unit 51 of the information acquiring unit 12. In this way, when the information acquiring unit 12 uses the multiple divided images obtained by dividing the input image, it becomes possible to capture detailed features of the input image.

[0029] Furthermore, if the image dividing unit 22 can detect a characteristic portion in the input image 31, it may partially cut out only the characteristic portion instead of dividing the input image 31. In this case, the image dividing unit 22 sets the resolution of each of the cut-out images (narrow area images) to match the resolution defined by the divided image input unit 51. The cut-out images may include a portion of the input image 31 that is not in common with each other (different ranges). The cut-out images may or may not include a common range with each other (the same range).

[0030] The input image expanded in memory by the input unit 21 can be used by the image division unit 22 and the information amount reduction unit 24 without interference from each other's processing. In other words, even if the image division unit 22 divides the input image to generate divided images, the information amount reduction unit 24 can use the undivided input image. Furthermore, even if the information amount reduction unit 24 reduces the amount of information of the input image to generate a compressed image, the image division unit 22 can use the input image without the amount of information reduced.

[0031] The segmented image output unit 23 stores the multiple segmented images generated by the image segmentation unit 22 in a specific file storage location, thereby enabling the information acquisition unit 12 to use the multiple segmented images. Here, in order to train the learning model, the information acquisition unit 12 may input images used in previous training into the learning model again to obtain output data from the learning model. Therefore, if the segmented image output unit 23 stores the multiple segmented images in a specific file storage location, it is not necessary to regenerate the segmented images each time the learning model is trained. However, this is not limited to this, and the segmented image output unit 23 may generate the segmented images required for the current training each time the learning model is trained and output the segmented images to the information acquisition unit 12.

[0032] The information amount reduction unit 24 generates a compressed image by reducing (compressing) the amount of information of the input image expanded in memory by the input unit 21. For example, as shown in FIG. 4, the information amount reduction unit 24 generates a compressed image 41 by reducing the resolution of the input image 31 (by reducing the input image 31). Alternatively, as shown in FIG. 4, the information amount reduction unit 24 may generate a compressed image 42 by grayscaling or binarizing the input image 31, which is an RGB color image, to reduce the amount of color information (color information). The information amount reduction unit 24 may generate a compressed image by reducing the resolution of the input image and then further reducing the color information. Furthermore, the information amount reduction unit 24 may generate a compressed image by trimming (extracting) a portion of the input image. In this case, the compressed image may include a wider range of the input image than each of the multiple divided images. The compressed image may include a range of the input image that is not shared (does not overlap) among the multiple divided images (a range that does not overlap with each other in each of the multiple divided images). The compressed image may also include the entire range of the input image that is included in the multiple divided regions.

[0033] In this way, if the information acquisition unit 12 uses a compressed image in which the amount of information of the input image has been reduced, it becomes possible to capture the overall characteristics of the input image. Also, because the amount of information of the compressed image is less than the amount of information of the input image, the calculation cost of the information acquisition unit 12 in the learning phase and the estimation phase can be reduced compared to when the input image is used as is. Therefore, the calculation cost of the information processing device 1 in learning and estimation can be reduced.

[0034] The compressed image output unit 25 stores in a specific file storage location the compressed image whose amount of information has been reduced by the information amount reduction unit 24. If the resolution defined in the compressed image input unit 53 of the information acquisition unit 12 does not match the resolution of the compressed image, the compressed image output unit 25 adjusts the resolution of the compressed image so that the two resolutions match, and then stores the compressed image in the specific file storage location.

[0035] Furthermore, the resolution (number of pixels) defined in the divided image input unit 51 and the resolution defined in the compressed image input unit 53 may be the same or different. The resolution of the divided image and the resolution of the compressed image may be the same or different. For example, if the resolution of the compressed image is smaller than the resolution of the divided image, the cost of the convolution process described below for the compressed image can be reduced. Note that the resolution defined in the divided image input unit 51 and the resolution defined in the compressed image input unit 53 depend on the amount of information that can be input to the learning model, and therefore can be changed as desired by changing the design of the learning model (CNN).

[0036] Note that, if the image acquisition unit 10 is capable of acquiring a plurality of divided images and a compressed image instead of an input image, the information processing device 1 does not need to have the image processing unit 11. In this case, the image acquisition unit 10 outputs the plurality of divided images and the compressed image to the information acquisition unit 12.

[0037] (About the information acquisition unit) The internal configuration of the information acquisition unit 12 will be described in detail with reference to Fig. 5. The information acquisition unit 12 has a learning model 50, a segmented image input unit 51, and a compressed image input unit 53. The learning model 50 has a segmented image convolution unit 52, a compressed image convolution unit 54, a data connection unit 55, a fully connected layer 56, and an output unit 57.

[0038] The divided image input unit 51 acquires a plurality of divided images from a specific file storage location and expands the plurality of divided images in memory. The plurality of divided images expanded by the divided image input unit 51 are represented as a multidimensional array.

[0039] The segmented image convolution unit 52 performs convolution processing on the multidimensional array information of each segmented image and outputs the result of the convolution processing as array information. The convolution processing is performed for each segmented image. In the convolution processing, a unique filter is applied to each of multiple convolution layers to gradually capture the features of the input data. Here, when the weights of the filters in the convolution layers are adjusted, the array information output from the segmented image convolution unit 52 changes. Therefore, the adjustment unit 14 can adjust the output data acquired by the information acquisition unit 12 from the learning model (i.e., training the learning model 50) by adjusting the weights of the filters in the convolution layers. In addition, an activation function, a dropout layer, or a pooling layer may be inserted between multiple convolution layers to extract more detailed features or prevent overlearning.

[0040] In the learning phase, the segmented image convolution unit 52 may use a technique (transfer learning) that uses a model that has already been pre-trained using a large number of images. By using transfer learning, the segmented image convolution unit 52 can shorten the learning time and improve the prediction accuracy even when there is a small amount of data used for learning (training data). The pre-trained model may be VGG16, Inception V3, ResNet-50, or the like, but any model may be used.

[0041] The compressed image input unit 53 acquires a compressed image from a specific file storage location and expands the compressed image in memory. The compressed image is represented as a multidimensional array.

[0042] The compressed image convolution unit 54 performs convolution processing on the multidimensional array information of the compressed image in the same manner as the divided image convolution unit 52, and outputs the result of the convolution processing as array information. In addition, in the learning phase, the compressed image convolution unit 54 may also use transfer learning.

[0043] The data concatenation unit 55 concatenates the array information output by the divided image convolution unit 52 and the array information output by the compressed image convolution unit 54 to generate one piece of array information. One piece of array information is information that handles multiple values ​​of the same data type as a group of variables (one set of information). The divided image convolution unit 52 performs convolution processing on each divided image, so Array information is output for each image. Therefore, if the data linking unit 55 simply combines these array information, the influence of global features that should be obtained from the compressed image will be weakened. For this reason, the data linking unit 55 may combine the array information of multiple divided images into one array information using a pooling layer or the like, and then link the combined array information with the array information output by the compressed image convolution unit 54. This makes it possible to adjust the balance in data size between the array information corresponding to the divided images and the array information corresponding to the compressed image.

[0044] The fully connected layer 56 further summarizes the features indicated by the sequence information generated by the data linking unit 55. The fully connected layer 56 is a layer in which all neurons in the input layer and all neurons in the output layer are connected. In the fully connected layer 56, multiple layers are stacked, making it possible to extract more complex features from the input sequence information.

[0045] The output unit 57 aggregates the sequence information acquired from the fully connected layer 56 so that output data with a predetermined number of elements is output. The number of elements of the output data is, for example, 1 when it is desired to output (guess) one numerical value, and 1000 when it is desired to classify into one of 1000 classes.

[0046] [Example 1] Example 1 will be described below as a specific example of Embodiment 1. In Example 1, an information processing device 1 will be described that estimates the processing time of die pieces that make up a plastic injection molding die from an image (design image) of the die (die 3D shape) modeled using 3D-CAD.

[0047] (Learning phase) First, a learning phase (learning method) in which the learning model 50 of the information processing device 1 is trained will be described with reference to Fig. 6. In Fig. 6, a flow of training the learning model 50 will be described by taking up some of the components of the information processing device 1 described above.

[0048] The input image 601 is an image (target image) that captures the entire mold when viewed from the +Z direction in the absolute coordinate system of the 3D-CAD. The resolution of the input image 601 is set to a resolution that allows the shape of the mold to be confirmed in the input image 601. Specifically, the resolution of the input image 601 is set to 448×448.

[0049] The image dividing unit 22 and the information amount reducing unit 24 perform image processing on the input image 601 to generate an image that can be used in the learning model 50.

[0050] The image dividing unit 22 divides the input image 601 to generate a plurality of divided images 604. Since the resolution of the image input to the learning model 50 is assumed to be 224×224, the image dividing unit 22 divides the input image 601, which has a resolution of 448×448, into four divided images 604, each having a resolution of 224×224.

[0051] The information amount reduction unit 24 generates a compressed image 609 by reducing the amount of information of the input image 601. Here, the information amount reduction unit 24 reduces the amount of information of the input image 601 by lowering the resolution of the input image 601 from 448×448 to 224×224.

[0052] In the information acquisition unit 12, the four divided images 604 and the compressed image 609 are input to the learning model 50. The learning model 50 includes a divided image convolution unit 52, a compressed image convolution unit 54, a data connection unit 55, a fully connected layer 56, and an output unit 57.

[0053] The divided image convolution unit 52 extracts features from each of the four divided images 604. The image convolution unit 52 has four convolutional neural networks (CNNs) 606. Fig. 7 shows an example of the convolutional neural network 606. In the convolutional neural network 606, a plurality of convolutional layers and a plurality of pooling layers are connected in series, thereby extracting features from an image input to the convolutional neural network 606.

[0054] Each of the convolutional neural networks 606 receives as input one divided image 604 generated by the image dividing unit 22. At this time, the divided image 604 is an image with a resolution of 224 x 224, and has RGB color information. Therefore, the convolutional neural network 606 receives as input array information (224, 224, 3) as the divided image 604. Then, the convolutional neural network 606 performs a convolution process, and outputs one-dimensional array information with 6,400 elements.

[0055] The compressed image convolution unit 54 has a convolutional neural network (CNN) 610. A compressed image 609 whose amount of information has been reduced by the information amount reduction unit 24 is input to the convolutional neural network 610. The compressed image 609 has a resolution of 224 × 224 and has RGB color information. Therefore, the compressed image 609 is input to the convolutional neural network 610 as array information of (224, 224, 3). Then, in the convolutional neural network 610, convolution processing and the like are performed in the same manner as the convolutional neural network 606 (see FIG. 7), and one-dimensional array information having 6,400 elements is output.

[0056] The convolutional neural networks 606 and 610 may use transfer learning using VGG16. VGG16 uses a model that has been trained using a large-scale image set called ImageNet, so it is possible to improve the prediction accuracy in the prediction phase even when there is a small amount of training data.

[0057] The data connection unit 55 includes a pooling layer 607 , a fully connected layer 608 , a fully connected layer 611 , and a concatenation layer 612 .

[0058] The pooling layer 607 combines the outputs (array information) from the four convolutional neural networks 606 into one. The pooling layer 607 performs, for example, max pooling (a process of obtaining the maximum value for each pooling region). This results in one-dimensional array information with 6400 elements, emphasizing the features of the (6400, 4) array information. Note that instead of max pooling, average pooling (a process of obtaining the average value for each pooling region) or L2 pooling (a process of obtaining the root mean square for each pooling region) may be performed. In this embodiment, the amount of array information output from the pooling layer 607 is the same as the amount of array information output from one convolutional neural network 606. However, it is sufficient that the amount of array information output from the pooling layer 607 is less than the total amount of array information output from the four convolutional neural networks 606. In this case, for example, the amount of array information output from the pooling layer 607 is equal to or greater than the amount of array information output from one convolutional neural network 606. The amount of information in the array information can be considered to be the number of elements in the array.

[0059] The fully connected layer 608 and the fully connected layer 611 further summarize the features of the input one-dimensional array information. The number of elements of the array information output by each of the fully connected layer 608 and the fully connected layer 611 may be changed depending on the importance of the features in the input image 601. When estimating the processing time from the image of the mold according to the first embodiment, both the detailed features obtained from the four divided images 604 and the global features obtained from the single compressed image 609 are important (similarly The fully connected layer 608 and the fully connected layer 611 output the same number of elements of the array information (the amount of information in the array information). Specifically, the fully connected layer 608 combines the array information so that the number of elements of the array information representing the features of the four divided images 604 is reduced from 6400 to 1024. The fully connected layer 611 combines the array information so that the number of elements of the array information representing the features of one compressed image 609 is reduced from 6400 to 1024.

[0060] The link layer 612 serially links the one-dimensional array information (first information) output by the fully connected layer 608 and the one-dimensional array information (second information) output by the fully connected layer 611, and outputs the linked array information. In Example 1, since one-dimensional array information with 1024 elements is output from each of the fully connected layers 608 and 611, the link layer 612 outputs one-dimensional array information with 2048 elements.

[0061] The fully connected layer 56 further summarizes the features indicated by the sequence information output by the concatenation layer 612.

[0062] The output unit 57 changes the number of elements of the array information output from the fully connected layer 56 to match the information estimated by the information processing device 1 in the estimation phase. In the first embodiment, the information processing device 1 estimates the processing time in the estimation phase, so the output unit 57 outputs array information with one element. This allows the information acquisition unit 12 to acquire output data corresponding to the input image 601 from the learning model 50 to which the four divided images 604 and one compressed image 609 have been input.

[0063] The adjustment unit 14 trains the learning model 50 based on the output data output from the learning model 50 (output unit 57) and information on the correct answer (correct answer data; teacher data) to be output from the learning model 50. Specifically, the adjustment unit 14 adjusts the filter weights of the convolutional neural networks 606, 610 (convolutional layers) so as to minimize the value of a loss function that represents the degree of error between the output data and the correct answer data for multiple input images. If the filter weights of the convolutional neural networks 606, 610 are adjusted, the output data output from the learning model 50 changes, and therefore the value of the loss function can be updated. In this way, by adjusting the filter weights of the convolutional neural networks 606, 610, the adjustment unit 14 can turn the learning model 50 into a trained model for which training has been completed. Note that the mean absolute percentage error, which will be described later with reference to Equation 1, can be used as the loss function. Alternatively, the square sum error or cross entropy error may be used as the loss function.

[0064] In the first embodiment, the user prepares in advance approximately 1,000 sets of learning data, each set consisting of an input image representing a mold and actual information (ground truth data) on the processing time corresponding to the input image. The adjustment unit 14 then performs the above-described learning on a workstation equipped with a GPU. During the learning process, the user may also adjust the number of learning epochs and the batch size so that highly accurate predictions can be made in the prediction phase.

[0065] (guessing phase) Next, with reference to FIG. 8, a process of estimating the processing time (estimation phase; estimation method) will be described using the trained model 804 trained by the learning model 50 in the learning phase.

[0066] An input image 801 is an input image of a mold obtained by capturing the 3D shape data of the mold from the +Z direction of the absolute coordinate system. The color information of the input image 801 is held in RGB. The resolution of the input image 801 is 448 x 448.

[0067] The image dividing unit 22 divides the input image 801 into four parts to generate four divided images 802 with a resolution of 224×224. It becomes easier to obtain detailed features from the four divided images 802.

[0068] The information amount reduction unit 24 reduces the resolution of the input image 801 to generate a compressed image 803. Specifically, the information amount reduction unit 24 reduces the input image 801 to generate the compressed image 803 so that the resolution becomes 224 × 224. This makes it easier for the information acquisition unit 12 to obtain global features of the input image 801 from the compressed image 803.

[0069] The information acquisition unit 12 has a trained model 804. The trained model 804 is a model trained by the training model 50 in the training phase. The information acquisition unit 12 receives four divided images 802 and one compressed image 803 as input to the trained model 804, and outputs the machining time of the mold, which is an estimation result (estimated value). The estimation result may be written to a log file or may be displayed on the screen of the display unit of the information processing device 1.

[0070] As an index for evaluating the accuracy of the estimation result (estimation performance of the information processing device 1), the mean absolute percentage error relative to the actual value of the processing time (correct data) can be used. The lower the mean absolute percentage error, the more accurate the estimation. Equation 1 shows a method for calculating the mean absolute percentage error M. In Equation 1, n is the number of input images, and t is a value (identifier) ​​that identifies the input image. At is the measured value corresponding to the identifier t, and Ft is the estimation result (estimated value) corresponding to the identifier t. In Example 1, a mean absolute percentage error M of 32.4% was obtained.

number

[0071] [Example 2] As Example 2, which is a specific example of Embodiment 1, an example will be described in which a compressed image in which color information of an input image is reduced is used instead of a compressed image (wide-area image) in which the resolution of the input image (target image) is reduced in the information processing device 1 according to Example 1. Note that, hereinafter, only the differences between Example 2 and Example 1 will be described, and descriptions of the same contents as Example 1 will be omitted.

[0072] In the second embodiment, in the learning phase shown in FIG. 6 , the information amount reduction unit 24 generates a compressed image 609 in which color information is reduced from an input image 601. In the second embodiment, the input image 601 has a resolution of 448 × 448 and is formed in RGB format. Since the input image 601 is an image generated from mold 3D shape data, various colors are added to the image to allow recognition of tolerance information and various attributes. However, color can be considered unnecessary information for obtaining global shape features of the input image 601. Therefore, the information amount reduction unit 24 grayscales the input image 601 to generate a compressed image 609 in which color information is reduced. This makes it possible to obtain features more efficiently from the compressed image 609.

[0073] Moreover, unlike in the first embodiment, a compressed image 609 with a resolution of 448 × 448 is input to the convolutional neural network 610 (learning model 50). Specifically, since the compressed image 609 has a resolution of 448 × 448 and is a grayscale image, the convolutional neural network 610 receives array information of (448, 448, 1) as the compressed image 609.

[0074] Other processes in the learning phase in the second embodiment are the same as those described in the first embodiment. Similarly, in the estimation phase in the second embodiment, the information amount reduction unit 24 generates a compressed image 803 by reducing the color information of the input image 801, and the compressed image 803 is input to the trained model 804. This is the only difference from the first embodiment.

[0075] In Example 2, the mean absolute percentage error, which is an index for evaluating the accuracy of the estimation results, was 37.8%.

[0076] [Example 3] As Example 3, which is a specific example of Embodiment 1, an example different from Example 1 will be described, in which a compressed image (wide-area image) obtained by reducing the resolution of an input image is used in the information processing device 1 according to Example 1. Note that, hereinafter, only the differences between Example 3 and Example 1 will be described, and descriptions of the same contents as Example 1 will be omitted.

[0077] First, the learning phase, which is a phase in which the learning model 900 of the information processing device 1 is trained, will be described with reference to Fig. 9. The third embodiment differs from the first embodiment only in that a data linking unit 901 of the learning model 900 exists instead of the data linking unit 55 of the learning model 50 according to the first embodiment. Therefore, only the data linking unit 901 will be described in detail below, and descriptions of the other components will be omitted.

[0078] Unlike the data connection unit 55 according to the first embodiment, the data connection unit 901 does not include a pooling layer 607 and fully connected layers 608 and 611. The array information output from the four convolutional neural networks 606 and the convolutional neural network 610 is all connected in the connection layer 612. Therefore, the array information output from the connection layer 612 contains more features of the divided image 604 than the features of the compressed image 609. Therefore, when detailed features of the input image 601 should be particularly emphasized in estimation, the use of the third embodiment improves the accuracy of estimation compared to the first embodiment. Furthermore, the pooling layer 607 and fully connected layers 608 and 611 can be excluded from the learning model, thereby reducing the calculation cost using the learning model.

[0079] On the other hand, in the estimation phase, the divided image 802 and the compressed image 803 are input to the learning model 900 learned in the learning phase, as in Example 1 (see Figure 8), and the processing time estimated from the learning model 900 is obtained.

[0080] In Example 3, the mean absolute percentage error, which is an index for evaluating the accuracy of the estimation results, was 34.6%.

[0081] (Comparative Example) 10 and 11, an evaluation of the accuracy of the estimation result when the amount of information of the input image is not reduced will be described as an example for comparison with Examples 1 to 3. In Fig. 10 and Fig. 11, components with the same numbers as those in Fig. 6 and Fig. 8 are the same as those in Example 1, and therefore detailed description thereof will be omitted.

[0082] FIG. 10 is a diagram illustrating the learning phase when the amount of information in the input image 601 is not reduced.

[0083] This comparative example does not include the information amount reduction unit 24. Therefore, the amount of information of the input image 601 is not reduced, and the input image 601 is input as array information of (448, 448, 3) to the convolutional neural network (CNN) 1001 in the learning model 50. Then, the convolutional neural network 1001 performs a convolution process on the input image 601, and outputs one-dimensional array information with 6,400 elements.

[0084] Therefore, the convolutional neural network 1001 receives an input image 601 that contains more information than the compressed image 609 that is input to the convolutional neural network 610 according to Examples 1 to 3. As a result, in this comparative example, the calculation cost for performing convolution in the convolutional neural network increases compared to Examples 1 to 3.

[0085] 11, in the estimation phase, the information processing device 1 determines an estimation result using a trained model 1101 that has trained the learning model 50 in the learning phase. In the estimation phase, four divided images 802 obtained by dividing an input image 801 and the input image 801 with no information amount reduction are input to the trained model 1101. Therefore, in the estimation phase according to this comparative example, the calculation cost for performing convolution using a convolutional neural network is similarly higher than in Examples 1 to 3.

[0086] In this comparative example, a mean absolute percentage error, which is an index for evaluating the accuracy of the estimation result, of 40.6 was obtained. On the other hand, in Example 1, the mean absolute percentage error was 32.4%, and in Example 2, the mean absolute percentage error was 37.8%. Furthermore, in Example 3, the mean absolute percentage error was 34.6%. Therefore, it was confirmed that the accuracy of the estimation result can be improved by using an image in which the amount of information of the input image is reduced.

[0087] As described above, according to the first embodiment (Examples 1 to 3), in an information processing device that performs learning or inference using a learning model, it is possible to reduce the calculation cost of the learning model and improve the inference accuracy.

[0088] In the above embodiments and examples, examples of machine learning using a convolutional neural network have been described, but any machine learning may be performed using a support vector machine (SVM), linear regression, etc. Furthermore, in the above, learning and inference are performed by a single information processing device, but the device that performs learning and the device that performs inference may be separate devices.

[0089] Furthermore, reducing the amount of information in the input image is not limited to reducing the resolution or reducing the color information. For example, the amount of information may be reduced by reducing the number of bits of the pixel value of each pixel in the input image.

[0090] Note that each functional unit in each of the above embodiments may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of multiple functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Furthermore, each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, an apparatus may have a processor and a memory (storage medium) in which a control program is stored. Then, the functions of at least some of the functional units of the apparatus may be realized by the processor reading and executing the control program from the memory.

[0091] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0092] 1: information processing device, 11: image processing unit, 12: information acquisition unit, 50: Learning model

Claims

1. an image acquisition means for acquiring a plurality of narrow band images representing a portion of a target image and a wide band image which is an image with a reduced amount of information compared to the target image, includes a portion of the plurality of narrow band images, and represents a wider range of the target image than the plurality of narrow band images; an information acquisition means for acquiring information correlated with the target image from a learning model to which the plurality of narrow-area images and the wide-area image are input; An information processing device comprising:

2. an image acquisition means for acquiring a plurality of narrow band images representing a portion of a target image, and a wide band image which is an image in which the amount of information of the target image has been reduced and which includes a portion of the plurality of narrow band images and represents a wider range of the target image than the plurality of narrow band images; a learning means for learning a learning model based on the plurality of narrow-area images, the wide-area image, and information correlated with the target image; An information processing device comprising:

3. The method further includes an information acquisition means for acquiring information correlated with the target image from a learning model to which the plurality of narrow-area images and the wide-area image are input, the learning means uses information correlated with the target image acquired by the information acquisition means to train the learning model.

3. The information processing apparatus according to claim 2, wherein:

4. The wide-area image is an image obtained by reducing the number of pixels of the target image.

4. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

5. The wide-area image is an image obtained by reducing color information of the target image.

5. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

6. The wide-area image is an image obtained by reducing the number of bits of pixel values ​​of the target image.

6. The information processing device according to claim 1, wherein the information processing device is a computer.

7. The wide-area image is a target image in which the plurality of narrow-area images are not common to each other. Contains ranges, 7. The information processing apparatus according to claim 1, wherein the information processing apparatus is a computer.

8. In the learning model, processing is performed based on information obtained by linking first information acquired from the plurality of narrow-angle images and second information acquired from the wide-angle image.

8. The information processing device according to claim 1, wherein the information processing device is a computer.

9. the first information is information obtained by at least combining a plurality of pieces of information acquired by performing at least a convolution process on each of the plurality of narrow-area images into one piece of information, the second information is information obtained by performing at least a convolution process on the wide-area image; 9. The information processing apparatus according to claim 8,

10. an amount of information of the first information is less than a total amount of information of the plurality of information before being combined into the single information; 10. The information processing apparatus according to claim 9,

11. an amount of information of the first information is equal to or greater than an amount of information of each of the plurality of pieces of information before being combined into the single piece of information; 11. The information processing device according to claim 9, wherein:

12. The amount of information of the first information is the same as the amount of information of the second information.

12. The information processing device according to claim 9, wherein the information processing device is a computer.

13. the number of pixels of each of the plurality of narrow area images is different from the number of pixels of the wide area image; 13. The information processing device according to claim 1, wherein the information processing device is a computer.

14. the target image is a design image of a manufacturing target object, The information correlated with the target image is a processing time of the target object or a manufacturing cost of the target object.

14. The information processing device according to claim 1,

15. The manufacturing object is a mold.

15. The information processing apparatus according to claim 14,

16. The wide-area image is input to the learning model in association with the plurality of narrow-area images.

16. The information processing device according to claim 1,

17. The target image is an image obtained by image diagnosis, The information correlated with the target image is information related to life.

13. The information processing device according to claim 1, wherein the information processing device is a computer.

18. the target image is an image of a living organism, The information correlated with the target image is information about the type, state, or prospects of the living thing.

13. The information processing device according to claim 1, wherein the information processing device is a computer.

19. the target image is an image of a building or furniture, the information correlated with the target image is the degree of deterioration of the building or the furniture; 13. The information processing device according to claim 1, wherein the information processing device is a computer.

20. the target image is an image of a car or a house, The information correlated with the target image is the price of the car or the house.

13. The information processing device according to claim 1, wherein the information processing device is a computer.

21. the target image is a tide current image, The information correlated with the target image is the catch amount.

13. The information processing device according to claim 1, wherein the information processing device is a computer.

22. the target image is a weather image, The information correlated with the target image is information about weather.

13. The information processing device according to claim 1, wherein the information processing device is a computer.

23. the target image is a graph showing past price fluctuations, The information correlated with the target image is a future price.

13. The information processing device according to claim 1, wherein the information processing device is a computer.

24. an image acquisition step of acquiring a plurality of narrow band images representing a portion of a target image and a wide band image which is an image with a reduced amount of information compared to the target image, includes a portion of the plurality of narrow band images, and represents a wider range of the target image than the plurality of narrow band images; an information acquisition step of acquiring information correlated with the target image from a learning model to which the plurality of narrow-area images and the wide-area image are input; The method for estimating the above-mentioned

25. an image acquisition step of acquiring a plurality of narrow band images representing a portion of a target image and a wide band image which is an image in which the amount of information of the target image has been reduced and which includes a portion of the plurality of narrow band images and represents a wider range of the target image than the plurality of narrow band images; a learning step of learning a learning model based on the plurality of narrow-area images, the wide-area image, and information correlated with the target image; A learning method comprising:

26. A program for causing a computer to execute each step of the estimation method according to claim 24 or the learning method according to claim 25.

27. A computer-readable recording medium storing the program according to claim 26.

Images