Quantitative evaluation device, quantitative evaluation program, and learning model generation program

Abstract

To quantitatively evaluate an appearance degree of a casting surface of a cast iron pipe.SOLUTION: A quantitative evaluation device used for inspecting quality of a casting surface of a cast iron pipe comprises: storage means (42) which stores a learning model (M2) generated by machine-learning image data of a plurality of casting surfaces classified into plural stages in advance as teacher data; image acquisition means (41) which acquires image data of the casting surface of the cast iron pipe being the inspection object; and evaluation determination means (43) which inputs the image data acquired by the image acquisition means to the learning model, evaluates and determines a level or a numeric value based on output from the learning model as an appearance degree of the casting surface of the cast iron pipe being the inspection object.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a quantitative evaluation device, a quantitative evaluation program, and a learning model generation program used for inspecting the quality of the casting skin of cast iron pipes. In particular, the present invention relates to a quantitative evaluation device and program for quantitatively evaluating the appearance degree of the casting skin using a learning model, and a learning model generation program for generating a learning model.

Background Art

[0002] One of the manufacturing methods of cast iron pipes is the mold centrifugal casting method. As shown in, for example, Japanese Patent Application Laid-Open No. 2002-153962 (Patent Document 1), in the mold centrifugal casting method, while rotating a cylindrical mold around a horizontal axis, molten metal is supplied to the inside of the mold, and a pipe (cast iron pipe) is manufactured in a state where centrifugal force is applied to the molten metal. This mold centrifugal casting method is a casting method that water-cools the mold (cylindrical mold), unlike the air-cooled sand mold centrifugal casting method. In casting methods such as the mold centrifugal casting method, pinholes (including blowholes) may occur in the cast iron pipe.

[0003] When a large number of pinholes occur, they are treated as appearance defects. Therefore, before shipment, a pinhole inspection of the casting skin is performed.

[0004] <000​​​​​​​​​​​​​​​​​​ [Patent Document 3] Japanese Patent Publication No. 2019-2788 [Patent Document 4] WO2020 / 137151 publication [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] While the quality of castings, based on the degree of pinhole occurrence, is generally inspected visually by humans, it is difficult to judge at a glance, and the inspection results may be influenced by the surrounding environment (such as the degree of lighting) and the human sense. Therefore, it has been difficult to quantitatively evaluate the appearance of the casting surface of cast iron pipes.

[0007] The inspection methods disclosed in the above-mentioned Patent Documents 2 to 4 are simply methods for determining the presence or absence of defects such as scratches, and therefore cannot be applied to inspecting the quality of the casting surface of cast iron pipes. In other words, methods for determining the presence or absence of defects cannot quantitatively evaluate the appearance of the casting surface.

[0008] The present invention was made to solve the above-mentioned problems, and its objective is to provide a quantitative evaluation device and program capable of quantitatively evaluating the appearance of the casting surface of a cast iron pipe. Another objective is to provide a learning model generation program that generates a learning model for quantitatively evaluating the appearance of the casting surface of a cast iron pipe. [Means for solving the problem]

[0009] A quantitative evaluation device according to a certain aspect of this invention is a quantitative evaluation device used for inspecting the quality of the casting surface of a cast iron pipe, and comprises: a storage means for storing a learning model generated by machine learning using a plurality of image data of casting surfaces that have been pre-classified into multiple stages as training data; an image acquisition means for acquiring image data of the casting surface of a cast iron pipe to be inspected; and an evaluation determination means for inputting the image data acquired by the image acquisition means into the learning model and evaluating and determining a level or numerical value based on the output from the learning model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

[0010] Preferably, the evaluation determination means evaluates and determines the degree of appearance of the casting surface by selecting a level with more stages than the number of classes in the training data, or a numerical value within a continuous, stepless range.

[0011] Preferably, the training data for the learning model includes, as augmentation data, trimmed image data obtained by removing the edges of the original image data of the casting surface.

[0012] The cast iron pipes being inspected are typically manufactured using the die centrifugal casting method. In this case, it is desirable that the training data for the learning model include, as augmented data, rotational image data obtained by rotating the original image data of the casting surface within an angular range of less than 5°.

[0013] A quantitative evaluation apparatus according to another aspect of this invention includes: a storage means for storing a learning model generated by machine learning using image data that has been preprocessed, including adaptive histogram flattening and blurring by a bilateral filter, on image data of multiple casting surfaces that have been pre-classified into multiple stages; an image acquisition means for acquiring image data of the casting surface of a cast iron pipe to be inspected; an adjustment means for performing input image adjustment processing, including preprocessing, on the image data acquired by the image acquisition means; and an evaluation determination means for inputting the image data that has undergone input image adjustment processing into the learning model and evaluating and determining a level or numerical value based on the output from the learning model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

[0014] A quantitative evaluation device according to yet another aspect of this invention includes a storage means for storing regression models and defective product classification models generated by machine learning using multiple image data of casting surfaces that have been pre-classified into multiple stages as training data; an image acquisition means for acquiring image data of the casting surface of a cast iron pipe to be inspected; and an evaluation determination means for inputting the image data acquired by the image acquisition means into the regression model and the defective product classification model, and evaluating and determining a numerical value obtained by weighting the output value from the regression model and the output value from the defective product classification model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

[0015] A quantitative evaluation program according to another aspect of this invention is a quantitative evaluation program used for inspecting the quality of the casting surface of a cast iron pipe, and causes a computer to perform the following steps: read a learning model generated by machine learning using multiple image data of casting surfaces that have been pre-classified into multiple stages as training data; acquire image data of the casting surface of the cast iron pipe to be inspected; input the acquired image data into the learning model, and evaluate and determine a level or numerical value based on the output from the learning model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

[0016] A learning model generation program according to another aspect of this invention is a learning model generation program for generating a learning model for inspecting the quality of the casting surface of a cast iron pipe, and causes a computer to perform the following steps: inputting raw image data of multiple casting surfaces that have been pre-classified into multiple stages; generating augmented data based on the input raw image data; and performing machine learning for each class using at least a portion of the image data of the multiple casting surfaces, including the raw image data and augmented data, as training data.

[0017] A learning model generation program according to another aspect of the present invention causes a computer to execute steps of: inputting a plurality of original image data of casting surfaces that are classified in multiple stages in advance; generating teacher data by performing preprocessing on the input original image data, the preprocessing including adaptive histogram equalization and blurring processing using a bilateral filter; and performing machine learning on the generated teacher data for each class.

Effect of the Invention

[0018] According to the present invention, it is possible to quantitatively evaluate the appearance degree of the casting surface of a cast iron pipe.

Brief Description of the Drawings

[0019] [Figure 1] It is a diagram schematically showing an overview of the quantitative evaluation system 1 in each embodiment of the present invention. [Figure 2] It is a diagram schematically showing a method for evaluating the appearance degree of a casting surface in each embodiment of the present invention. [Figure 3] (A) is a functional block diagram showing the functional configuration of the learning device in each embodiment of the present invention, and (B) is a functional block diagram showing the functional configuration of the quantitative evaluation device in each embodiment of the present invention. [Figure 4] It is a flowchart showing a method for generating a learning model in Embodiment 1 of the present invention. [Figure 5] It is a schematic diagram for explaining step S1 in FIG. 4. [Figure 6] It is a flowchart showing a method for quantitatively evaluating the appearance degree of a casting surface in each embodiment of the present invention. [Figure 7] It is a flowchart showing a method for generating a learning model in Embodiment 2 of the present invention. [Figure 8] (A) and (B) are diagrams showing expansion patterns of original image data in Embodiment 2 of the present invention. [[ID=​​ [Figure 10] This figure shows an overview of ensemble learning performed by the learning device according to Embodiment 3 of the present invention. [Figure 11] (A) is a flowchart showing the input image tuning process in Embodiment 3 of the present invention, and (B) is a graph showing the relationship between the type of preprocessing and the error. [Figure 12] (A) and (B) are graphs showing the results of verifying the weights of defective product classes in the classification model. [Figure 13] (A) and (B) are graphs showing the results of verifying the weights in ensemble learning. [Figure 14] This graph shows the evaluation results of the ensemble learning model in Embodiment 3 of the present invention. [Figure 15] The results of verifying whether or not data augmentation occurred during training and inference in Embodiment 3 of the present invention are shown. [Figure 16] This is a diagram illustrating a ductile cast iron pipe manufactured by the die centrifugal casting method. [Modes for carrying out the invention]

[0020] Embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and their descriptions will not be repeated.

[0021] <Embodiment 1> (About the overview) First, an overview of the quantitative evaluation system 1 in this embodiment will be described with reference to Figures 1 and 2. Figure 1 is a schematic diagram showing an overview of the quantitative evaluation system 1, and Figure 2 is a schematic diagram showing a method for evaluating the appearance of the casting surface 11 of a cast iron pipe 10.

[0022] The quantitative evaluation system 1 quantitatively evaluates the appearance of the surface 11 of the cast iron pipe 10 to be inspected by performing machine learning on image data of the surface of a large number of cast iron pipes in advance. The cast iron pipe 10 is typically a ductile cast iron pipe manufactured by die centrifugal casting.

[0023] The characteristics of the cast iron pipe will be explained with reference to Figure 16. In the die centrifugal casting method, as shown in Figure 16(A), molten ductile iron C is supplied into the cylindrical mold 101 via a trough 105 while the mold 101 is rotated by rollers 102. The mold 101 is provided with a cooling water passage 103, and by circulating cooling water through the cooling water passage 103 during centrifugal casting, the molten metal and the mold 101 are cooled, and the cast iron pipe (ductile cast iron pipe) 10 is formed.

[0024] Because fine irregularities (irregularities due to peening) are formed on the inner circumferential surface of the mold 101, the surface of the cast iron pipe 10 manufactured by the mold centrifugal casting method shows numerous annular irregularities 110, as shown in Figure 16(B).

[0025] In this type of casting method, there is no escape route for the gas contained in the molten metal, so pinholes P, as shown in Figure 16(B), are likely to occur on the surface of the cast iron pipe 10.

[0026] Quantitative evaluation system 1 uses deep learning to perform machine learning on image data of casting surfaces, which have been pre-classified into five levels, as training data during the learning phase. Specifically, it uses a Convolutional Neural Network (CNN) to perform machine learning on a large amount of image data of casting surfaces to construct a learning model M1.

[0027] The appearance of the cast surface of the cast iron pipe 10 is classified into five grades according to the amount (number) of pinholes P per unit area. Figure 16(C) shows the appearance of the cast surface 11 for each grade. A lower grade indicates a better evaluation.

[0028] A CNN is a neural network consisting of multiple layers of networks that perform image feature extraction and pooling (noise reduction). This allows for quantitative evaluation of the appearance of the target cast iron pipe 10 by inputting image data of the cast surface 11 of the pipe to be inspected into a learning model (trained model) M2 during the inference phase.

[0029] Furthermore, the quantitative evaluation system 1 according to this embodiment is constructed to output, in the inference phase, a level with more stages than the number of classes in the training data (5 classes), or a numerical value within a continuous, stepless range, as the result of analyzing the image data of the target casting surface 11. Specifically, as shown in Figure 2, the image data of the target casting surface 11 is solved as a regression problem through a CNN as the learning model M2, and the calculation result is evaluated and determined as the degree of appearance of the casting surface 11. Therefore, in the inference phase, by inputting the image data of the casting surface 11 into the learning model M2, it is possible to output evaluation values ​​with finer criteria than the original 5 classes (for example, numerical values ​​from 1.0 to 5.0) as the degree of appearance of the casting surface 11.

[0030] Various processes in the learning phase are performed by the learning device, and various processes in the inference phase are performed by the quantitative evaluation device. The learning device and quantitative evaluation device can be implemented using a computer (information processing terminal) equipped with a processor such as a CPU (Central Processing Unit) or GPU (Graphics Processing Unit), and non-volatile memory or storage device. Although the learning device and quantitative evaluation device are typically implemented by separate computers, they may also be implemented by a common computer.

[0031] (Functional configuration of the learning device) Figure 3(A) is a functional block diagram showing the functional configuration of the learning device 3.

[0032] The learning device 3 includes an input unit 31 that inputs (acquires) image data of the casting surface of multiple cast iron pipes that have been pre-classified into multiple stages, and a raw image data storage unit 32 that stores the multiple image data input to the input unit 31 (hereinafter referred to as "raw image data") for each class. It is desirable that all images of the raw image data input from the input unit 31 are taken under the same conditions (direction of light source, resolution, color, etc.) and classified by the same person.

[0033] The learning device 3 further comprises a training data generation unit 33 that classifies the original image data stored in the original image data storage unit 32 according to predetermined rules and generates training data and test data; a training data storage unit 34 that stores the training data and test data generated by the training data generation unit 33 for each class; a learning processing unit 35 that uses a CNN to machine learn the training data (at least a part of the original image data) stored in the training data storage unit 34 for each class and generates a learning model M1; and a model storage unit 36 ​​that stores the learning model M1 generated by the learning processing unit 35.

[0034] The learning device 3 may further include an evaluation unit 37 that evaluates the general-purpose performance of the learning model M1 using test data stored in the training data storage unit 34.

[0035] The functions of the training data generation unit 33, the learning processing unit 35, and the evaluation unit 37 are realized by the processor executing software. The original image data storage unit 32 and the model storage unit 36 ​​are composed of non-volatile storage devices. The training data storage unit 34 is composed of non-volatile or volatile storage devices. The input unit 31 may be, for example, a communication interface capable of sending and receiving data from an external device, or a drive device that reads and writes data from a removable recording medium.

[0036] (Functional configuration of quantitative evaluation device) Figure 3(B) is a functional block diagram showing the functional configuration of the quantitative evaluation device 4.

[0037] The quantitative evaluation device 4 includes an input unit 41 that inputs (acquires) image data of the casting surface 11 of the cast iron pipe 10 to be inspected, a model storage unit 42 that stores a learning model M2 in advance, an evaluation determination unit 43 that inputs the image data input to the input unit 41 to the learning model M2 and evaluates and determines a level or numerical value based on the output from the learning model M2 as the degree of appearance of the casting surface 11 of the target, and an output unit 44 that outputs the evaluation result from the evaluation determination unit 43.

[0038] The learning model M2 stored in the model memory unit 42 corresponds to the learning model M1 constructed in the learning device 3, that is, the learning model M1 generated by machine learning using multiple image data of casting surfaces that have been pre-classified into multiple stages as training data.

[0039] The evaluation determination unit 43 evaluates and determines the appearance degree of the cast surface 11 as the result of analyzing the image data input to the learning model M2, selecting a level with more stages than the original number of classes (5 stages) (for example, 10 stages), or a numerical value within a continuous, stepless range (for example, 1.0 to 5.0).

[0040] The functions of the evaluation determination unit 43 are realized by the processor executing software. The model storage unit 42 is typically composed of a non-volatile storage device. The input unit 41 may be, for example, a communication interface capable of sending and receiving data from an external device, or a drive device that reads and writes data from a removable recording medium. Alternatively, it may be an input interface that directly receives image data from an imaging device (not shown). The output unit 44 is composed of, for example, a display. Alternatively, the output unit 44 may also be composed of a communication interface or a drive device, similar to the input unit 41.

[0041] (Regarding learning methods) Referring to the flowchart in Figure 4, the learning method executed by the learning device 3 according to this embodiment, that is, the method for generating the learning model M1, will be explained. The process shown in Figure 4 is realized by the processor of the learning device 3 executing a program that has been pre-stored in memory.

[0042] When the learning device 3 acquires the original image data via the input unit 31 (step S1), it stores the acquired original image data in the original image data storage unit 32. Each original image data is pre-classified into one of classes 1 to 5, and each original image data is stored in the original image data storage unit 32 in association with the class to which it belongs. As an example, the original image data stored in the original image data storage unit 32 is a three-color (RGB) image data obtained by compressing a 2048 x 1536 JPG file into a 256 x 192 PNG file. Note that the image data input from the input unit 31 may itself be compressed image data, or the learning device 3 may have a function to compress the image data input from the input unit 31.

[0043] Next, the teacher data generation unit 33 of the learning device 3 distributes the numerous raw image data stored in the raw image data storage unit 32 into training data, validation data, and test data according to predetermined rules (step S3). Specifically, as shown in Figure 5, first, a predetermined number (for example, 10 images) of raw image data are extracted from each class, and the extracted raw image data is saved in a folder corresponding to the class number in the test folder F3. Next, the remaining raw image data is classified into training data and validation data in an 8:2 ratio, and the classified raw image data is saved in folders corresponding to the class numbers in each folder F1 and F2, similar to the test data. The test folder F3, training folder F1, and validation folder F2 are stored in the teacher data storage unit 34.

[0044] Next, the learning processing unit 35 performs machine learning on training data, including the training data and validation data, using a CNN to generate a learning model M1 (step S5). Specifically, it uses, for example, Sony(R)'s Neural Network Console as the CNN framework and constructs the network architecture using the CNN structure search function. During this process, training is performed on the training data to learn parameters such as weights and biases, and validation is performed on the validation data, followed by a process of adjusting hyperparameters using Bayesian optimization.

[0045] Once the learning model M1 is generated, it is stored in the model storage unit 36. The evaluation unit 37 then evaluates the general-purpose performance of the learning model M1 using test data (step S7). The evaluation of the general-purpose performance of the learning model M1 may be performed according to the evaluation method of the evaluation determination unit 43 of the quantitative evaluation device 4. The learning model M1 may be reconstructed according to the evaluation results by the evaluation unit 37.

[0046] (Examples of learning methods) An example of the learning method described above will now be explained. In this example, 181 images from class 1, 240 images from class 2, 172 images from class 3, 86 images from class 4, and 75 images from class 5, for a total of 754 images, were used as the training dataset. In this case, all 754 original image data are distributed into 50 test data (10 images per class), 564 training data, and 140 validation data (step S3 in Figure 4).

[0047] In machine learning using training and validation data, the Neural Network Console described above was used to perform a one-week exploration using a Gaussian process, starting with a modified LeNet architecture with multiple layers (step S5 in Figure 4). As a result of the exploration, the architecture with the highest accuracy was adopted as the network (architecture) for the learning model M1. This architecture corresponds to the embodiment described in Embodiment 2 below.

[0048] The general performance of the learning model M1 generated in this embodiment was evaluated using 10 test data for each class (a total of 50) (step S7 in FIG. 4). FIG. 9(A) shows the evaluation results of the learning model M1 in this embodiment. The graph on the left side of the page shows the confusion matrix, where the vertical axis represents the true value (actual class) and the horizontal axis represents the predicted value (decision value). Here, similar to the evaluation method by the evaluation determination unit 43 of the quantitative evaluation device 4, the result of regression calculation (final output value) of the output of the CNN as the learning model M1 was used as the predicted value. When the predicted value is "Y", it is determined that Y is classified as 1 when Y≤1.5, Y = 2 when 1.5 < Y≤2.5, Y = 3 when 2.5 < Y≤3.5, Y = 4 when 3.5 < Y≤4.5, and Y = 5 when Y>4.5.

[0049] The graph on the right side of the page shows the MAE (mean absolute error) and RMSE (root mean square error) calculated based on the confusion matrix on the left, represented for each class and in total. From the graph in FIG. 9(A), although there is room for improvement in the learning method, by performing machine learning on a large number of original image data of cast surfaces that have been pre-classified using a CNN, a certain effect was confirmed that it is possible to predict the appearance level of the cast surface 11 of the inspection target.

[0050] (Quantitative evaluation method) Referring to the flowchart of FIG. 6, the quantitative evaluation method executed by the quantitative evaluation device 4 will be described. The process shown in FIG. 6 is realized by the processor of the quantitative evaluation device 4 executing a program stored in the memory in advance. Also, at the start of this process, it is assumed that the learning model M2 is read from the model storage unit 42 and expanded in the internal memory. The learning model M2 may be downloaded from a server or the like (not shown) at the start of this process.

[0051] When the quantitative evaluation device 4 acquires the image data of the cast surface 11 of the cast iron pipe 10 to be inspected, that is, the image data of the cast surface 11 with an unknown appearance level, via the input unit 41 (step S11), the evaluation determination unit 43 inputs the acquired image data into the learning model M2 (step S13).

[0052] The evaluation decision unit 43 solves the output from the learning model M2 as a regression problem and calculates a numerical value within the range of 1.0 to 5.0 (for example, "3.2", "3.27", etc.) (step S15).

[0053] Once the processing by the evaluation determination unit 43 is complete, the output unit 44 outputs the calculation result from step S15 as the appearance level of the casting surface 11 (step S17).

[0054] Thus, in this embodiment, the appearance quality of the cast surface 11 of the cast iron pipe 10 to be inspected is automatically evaluated and output, making it possible to accurately determine the quality of the cast surface 11. Furthermore, since a finer numerical value than the original class number is output as an index representing the appearance quality of the cast surface 11, the quality of the cast surface 11 can be determined with even greater accuracy.

[0055] <Embodiment 2> This embodiment differs from Embodiment 1 in that, in order to increase the amount of training data during the learning phase, the learning device 3 generates augmented data based on the original image data. In the following description, only the differences from Embodiment 1 will be explained in detail.

[0056] Figure 7 is a flowchart showing the method for generating the learning model M1 in this embodiment. In Figure 7, the same steps as in Embodiment 1 are numbered as in Figure 4.

[0057] Referring to Figure 7, in this embodiment, the learning device 3 acquires original image data via the input unit 31 (step S1), the training data generation unit 33 sorts the original image data (step S3), and then executes a process to generate augmented data (step S24). In this embodiment, the training data generation unit 33 generates augmented data by inverting, rotating, and cropping each original image data for each data classification.

[0058] Referring to Figure 8(A), the expansion pattern of the original image data (Expansion Pattern 1) will be explained. Similar to the embodiment of Embodiment 1, the 754-image dataset is assumed to be classified into training (564 images), validation (140 images), and test (50 images).

[0059] The original image data is inverted in three directions (vertical, horizontal, and vertical-horizontal), generating three types of inverted image data from a single original image.

[0060] The original image data is rotated, for example, in the left and right directions within an angle range of less than 5°, generating at least two types of rotated image data from a single original image data. In this embodiment, four types of rotated image data were generated by rotating the original image data by 1 degree, 2 degrees, -1 degree, and -2 degrees, respectively. By reducing the rotation angle in this way, the influence of the numerous annular ridges and depressions 110 appearing on the casting surface of the cast iron pipe 10, as shown in Figure 16(B), can be reduced. In other words, since the numerous annular ridges and depressions 110 appear as vertical line patterns at regular intervals in the captured image, reducing the rotation angle in this way reduces the possibility of misidentifying the depressions in the vertical line patterns of the rotated image data as pinholes P.

[0061] The cropping of the original image data is done within a range of less than 5% for each side (top, bottom, left, and right). This process generates at least four types of cropped image data from a single original image data. In this embodiment, 4 × 3 types of trim images were obtained by removing 1%, 2%, and 3% of the edges of the original image data in each direction. Trimmed image data was generated. The edges of the image of the cast iron pipe 10 (especially the upper and lower edges) correspond to the ends of the arcuate surface, and are therefore more likely to be darker than the central part, making it difficult to capture the pinhole P. For this reason, as in this embodiment, by generating trimmed image data with the edges of the original image data removed, it is possible to generate a large amount of enhanced data while maintaining the image quality.

[0062] In this embodiment, as shown in Figure 8(A), inversion, rotation, and trimming were performed in this order to further expand the extended data from the previous step. In this case, there are 13,000 test images, 146,460 training images, and 36,400 verification images.

[0063] Referring again to Figure 7, the learning processing unit 35 of this embodiment performs machine learning on the image data that has been sorted from all the original image data and augmented data as training data and validation data, and generates a learning model M1 (step S25).

[0064] The evaluation results of the general-purpose performance of the learning model M1 generated in this way (step S27) are shown in Figure 9(B). The evaluation of the learning model M1 was performed using 2,600 test images for each class (13,000 images in total). The graph on the left of the page shows the confusion matrix, with the vertical axis representing the true value (actual class) and the horizontal axis representing the predicted value (decision value). The graph on the right of the page shows the MAE (mean absolute error) and RMSE (root mean squared error) calculated based on the confusion matrix on the left, for each class and overall.

[0065] Compared to the MAE and RMSE in Embodiment 1 shown in Figure 9(A), it can be seen that the accuracy is improved in this embodiment, which has an increased amount of training data.

[0066] In this embodiment, augmented data was also generated for the test images, but it is also possible to augment only the training images and validation images, without augmenting the test images. The evaluation results of the general-purpose performance corresponding to this augmentation pattern (Augmentation Pattern 2) are shown in Figure 9(C). The evaluation of the learning model M1 in this embodiment was performed using 10 test data for each class (50 images in total), similar to Embodiment 1. In this embodiment as well, it can be seen that the accuracy has improved when compared with the MAE and RMSE in Embodiment 1 shown in Figure 9(A).

[0067] In this embodiment, trimmed image data and rotated image data are generated in order to increase the training data during the learning phase. However, for example, trimmed image data may also be generated for the purpose of improving prediction accuracy during the inference phase. That is, the image data input to the learning model M2 in the quantitative evaluation device 4 may be trimmed image data.

[0068] <Embodiment 3> In this embodiment, considering the purpose of casting surface inspection (to eliminate defective products), we will describe an example in which the learning device performs machine learning using two models: a regression-based learning model (hereinafter referred to as the "regression model") and a defective product classification model (hereinafter abbreviated as the "classification model"). In this embodiment, performing machine learning using two models, a regression model and a classification model, is called "ensemble learning."

[0069] (About the overview) Figure 10 is a diagram illustrating the overview of ensemble learning performed by the learning device 3 according to this embodiment. As shown in Figure 10, the learning device 3 performs a process to tune (adjust) the original image (pinhole image) to an image suitable for learning (step S31), and then performs ensemble learning (step S32). The output value of the ensemble learning is output as the grade determination result of the casting surface 11 (step S33).

[0070] In ensemble learning, the output value from the regression model M11 (e.g., 4.2) and the output value from the classification model M12 (e.g., 4) are combined to produce the final estimated value. In this embodiment, the process of combining the output values ​​from the regression model M11 and the classification model M12 is called "stacking." The output value obtained through stacking is a continuous value (e.g., 4.1).

[0071] (Regarding the tuning process) A specific example of the input image tuning process will be explained with reference to Figure 11(A). The input image tuning process is performed by the training data generation unit 33 shown in Figure 3(A).

[0072] The learning device 3 first performs oversampling of the input images to resolve class imbalance (step S41). Specifically, it randomly duplicates the training data for each class and performs oversampling so that each class has the same number of images (for example, 1,000 images).

[0073] Next, the resolution of all images is adjusted (resized) (step S42). This is because if the original images of the casting surface 11 are input to the CNN (each model M11, M12) at their original resolution, the high-frequency spatial frequency components contained in the images may become noise, potentially reducing accuracy. The resolution of the images to be input to the CNN can be determined by verification from among several size candidates. For example, if the resolution of the original image is 2736×1824, the input image resolution is set to 798×532.

[0074] After adjusting the resolution, the learning device 3 performs preprocessing on each image (step S43). Preprocessing includes "adaptive histogram equalization" and "pseudocolor conversion," and preferably further includes "blurring with a bilateral filter." In this case, it is desirable to perform the preprocessing in the order of histogram equalization, blurring, and color conversion.

[0075] "Adaptive histogram equalization" is a process that improves the contrast of background areas while limiting the contrast. "Pseudocolor conversion" is a process that converts a grayscale image into a pseudocolor image, for example, a 3-column array, and applies a color map such as JET. "Bilateral filtering" is a type of smoothing filter that adds color distance weighting to a Gaussian filter. By using this filter for blurring, it is possible to blur an image while preserving high-contrast areas.

[0076] Figure 11(B) is a graph showing the relationship between the type of preprocessing of the input image and the error (RMSE), where "adaptive histogram equalization" is denoted as "CLAHE," "pseudocolor conversion" as "JET," and "bilateral filter blurring" as "BF." From this graph, it can be seen that the error is smaller when preprocessed images are trained than when unprocessed images are trained. Furthermore, it can be seen that in addition to "adaptive histogram equalization" and "pseudocolor conversion," it is effective to perform "bilateral filter blurring" at least once as preprocessing. This is because blurring makes the vertical line patterns that appear on the casting surface 11 of the cast iron pipe 10 less noticeable. In other words, blurring reduces the possibility of mistakenly learning the recesses of the vertical line patterns on the casting surface 11 as pinholes P. It is even more desirable to perform blurring multiple times, between two and four times.

[0077] Furthermore, in terms of making the vertical line patterns appearing on the cast surface 11 of the cast iron pipe 10 less noticeable, it may be possible to improve accuracy (without performing "pseudo-color conversion") by using image data that has undergone preprocessing including "adaptive histogram equalization" and "blurring by bilateral filtering" as the training data for the learning models (regression model M11, classification model M12).

[0078] After performing the preprocessing described above, the training data (at least the instructional data) is augmented (step S44), similar to Embodiment 2. In Embodiment 2, augmented data was generated by flipping, rotating, and cropping the image data, but in this embodiment, augmented data is generated by randomly performing rotation and shearing of 2 degrees or less (less than 5 degrees), vertical flipping, horizontal flipping, scaling of 95% to 105%, and vertical and horizontal movement of 2% or less. Furthermore, the pixel values ​​are normalized to be in the range of 0 to 1. Note that this data augmentation is performed the same number of times for each classification (class).

[0079] The tuning process described above generates a large amount of training data common to both the regression model M11 and the classification model M12.

[0080] (Regarding ensemble learning) In ensemble learning, the training data that has undergone the tuning process described above is input to both the regression model M11 and the classification model M12. The regression model M11 corresponds to the learning model M1 described in Embodiments 1 and 2, and is a program that learns to output a continuous value indicating the degree of appearance of the cast surface 11 through regression calculation using a CNN. The classification model M12 is a program that learns to output a class (grade) of the cast surface 11 through CNN output. The architecture of the classification model M12 is the same as that of the regression model.

[0081] The architectures of the regression model M11 and the classification model M12 are preferably the same. In this embodiment, Keras with TensorFlow as the backend can be used for the CNN implementation. For training, models that are fine-tuned versions of existing architectures such as VGG and DenseNet can be used.

[0082] For the architectures used in these models M11 and M12, it is desirable to select an architecture that yields high accuracy after comparing and verifying various types of CNN architectures. Furthermore, it is desirable to adopt a model with a high accuracy range after comparing and verifying the range of convolutional layers in which parameters are fixed through fine tuning using the selected architecture. As an example, a model can be used in which the parameters from the input layer to the 481st layer of "DenseNet201" are fixed, and the layers from the 481st layer onward are fine-tuned. The model used in this embodiment is a transfer learning model that has been pre-trained on 3-channel color images from ImageNet.

[0083] The purpose of using the classification model M12 here is to improve the estimation accuracy of the defective product class (class 5). Therefore, the training data may be weighted to reflect the defective product class. The weighted cross-entropy of the loss function used in the classification model M12 is shown in equation (1) below.

[0084]

number

[0085] Here, P i Q is the true probability distribution. i is the estimated probability distribution, C is the number of classification classes, N is the number of data points, W is the number of data points. c This represents the weight of class C. The weight of the defective class is W5, and the weights of the other classes, W1, W2, W3, and W4, are all set to "1". That is, when W5=1, the normal cross-entropy state is achieved.

[0086] Figure 12(A) shows the relationship between the defective product class weight W5 and the recall rate of the defective product class in the classification model M12. Figure 12(B) shows the relationship between the defective product class weight W5 and the error (RMSE). In Figure 12(A), the recall rate was "1" when the weight was "6" or greater. This indicates that all defective products in the verification data were detected. In Figure 12(B), among the errors for weights of "6" or greater, the error rate was best when the weight was "8". It is considered effective to select the weight that minimizes the error while maintaining the best recall rate. In the example shown in Figure 12, "8" can be adopted as the defective product class weight W5. According to the verification results in Figure 12, the defective product class weight W5 should be set between 4 and 8.

[0087] When stacking the output values ​​of regression model M11 and classification model M12, it is desirable to use a weighted average. Below are the estimated values ​​y obtained by ensemble learning. ~ Equation (2) that defines it is shown below.

[0088]

number

[0089] y r is the estimated value by the regression model M11, y c is the estimated value by the classification model M12, and w is the weight (0.1 to 1.0). When w=0, y c When the coefficient of w=0, only the estimate from the regression model M11 is obtained. When w=1, the regression coefficient is also 0, and only the estimate from the classification model M12 is obtained. Thus, the ensemble learning weight w represents the weight for the regression model M11.

[0090] It is desirable to determine the ensemble learning weights w by comparing and verifying within the range of 0.0 to 1.0. The comparison results of the ensemble learning weights w in the verification data are shown in Figures 13(A) and (B). The verification results shown in Figures 13(A) and (B) represent the accuracy when the ratio of the classification model M12 is changed in a model case where the regression model M11 and classification model M12 have undergone the tuning process described above, and the weight W5 of the bad class in the classification model M12 is set to "8".

[0091] As shown in Figure 13(B), the error relative to the true value (RMSE) worsens as the weight w increases, but as shown in Figure 13(A), the recall is 1 when the weight w is 0.6 or greater. In this case as well, assuming the best recall, it is considered effective to select the weight that minimizes the error within that range. In the example shown in Figure 13, "0.6" can be adopted as the ensemble learning weight W5. Furthermore, according to the verification results in Figure 13, the ensemble learning weight W5 should be set within the range of 0.2 to 0.6.

[0092] Figure 14 shows the evaluation results (evaluation results using test data) of the ensemble learning model with weight W5 in the above model case. In this evaluation result, the error from the true value was 0.571 and the recall rate was 1.0.

[0093] Based on the above, the ensemble learning method of this embodiment enables grade determination using continuous values ​​while achieving a 100% recall rate for defective products. Therefore, the ensemble learning model of this embodiment is a useful model for inspecting the casting surface 11 of cast iron pipes 10. In other words, by storing the regression model M11 and classification model M12 generated by the learning device 3 in the model storage unit 42 of the quantitative evaluation device 4, the degree of appearance of the casting surface 11 of the cast iron pipe 10 to be inspected can be accurately evaluated and determined during the inference phase.

[0094] (Regarding quantitative evaluation in the inference phase) In the inference phase, the quantitative evaluation device 4 primarily performs the following processing. Specifically, the quantitative evaluation device 4 inputs the image data received by the input unit 41 into the regression model M11 and the classification model M12, and the evaluation decision unit 43 outputs a weighted average of the output values ​​from the regression model M11 and the output values ​​from the classification model M12 as an index value indicating the appearance degree of the cast surface 11.

[0095] In the inference phase, as in the learning phase, it is desirable to perform tuning processing (input image adjustment processing), including resizing and preprocessing, on the original image data before inputting it to the regression model M11 and classification model M12. Thus, it is desirable that the quantitative evaluation device 4 has the function of performing at least preprocessing on the image data acquired by the input unit 41. Furthermore, the tuning processing performed by the quantitative evaluation device 4 may also include the data augmentation processing described above.

[0096] Figure 15 shows the results of verifying whether data augmentation was performed during training and inference. The graphs in Figure 15 labeled "No Data Augmentation," "Data Augmentation with Inversion Only," and "Data Augmentation" correspond to the cases in the training phase where no data augmentation was performed, data augmentation with inversion only was performed, and data augmentation was performed by random rotation, inversion, scaling, and translation, respectively. For each of these cases, the error (RMSE) is shown for no test time augmentation (TTA) during inference, data augmentation with only inversion performed four times, and data augmentation with 20 random rotations, inversions, scaling, and translations.

[0097] When data augmentation is performed during inference, multiple image data are input to the regression model M11 and the classification model M12 for each subject being examined. Figure 15 shows the verification results when the regression model M11 averages the inference results and the classification model M12 performs a majority vote.

[0098] These verification results show that accuracy improves with data augmentation during training, and that the case where data is augmented by randomly rotating, flipping, scaling, and translating is the most accurate during both training and inference. Therefore, it is desirable that the tuning process (input image adjustment process) performed by the quantitative evaluation device 4 also includes data augmentation processing that randomly rotates, flips, scales, and translates.

[0099] In this embodiment, we have described an example in which tuning processing, including preprocessing, is performed on the original image data in ensemble learning using both the regression model M11 and the classification model M12. However, even when using a regression model alone, as in Embodiments 1 and 2, the tuning processing described in this embodiment may also be performed. This makes it possible to improve the estimation accuracy of the regression model.

[0100] <Variation> In the embodiments described above, cast iron pipes manufactured by die centrifugal casting were used as the subject of the explanation. However, cast iron pipes manufactured by other centrifugal casting methods, such as sand mold centrifugal casting, can also be used. In addition, because sand mold centrifugal casting has an air escape route in the sand mold, cast iron pipes manufactured by sand mold centrifugal casting have a lower frequency of pinholes P than cast iron pipes manufactured by die centrifugal casting.

[0101] Furthermore, while the above embodiments describe examples in which the appearance degree of the cast surface 11 to be inspected is output as a continuous numerical value, it is also possible to output a class (any of classes 1 to 5) at the same stage as the training data. In other words, the classification model M12 of Embodiment 3 may be used alone (preferably in combination with the input image tuning process).

[0102] Furthermore, while the above embodiments use a CNN as the learning model, the method is not limited to CNNs, as long as it is suitable for image analysis.

[0103] Furthermore, the learning method executed by the learning device 3 in each embodiment can also be provided as a program. Similarly, the evaluation method executed by the quantitative evaluation device 4 can also be provided as a program. Such programs can be provided by recording them on optical media such as CD-ROMs (Compact Disc-ROMs) or computer-readable non-transitory recording media such as memory cards. Programs can also be provided via download over a network.

[0104] The program according to the present invention may call necessary program modules from among the program modules provided as part of a computer's operating system (OS) in a predetermined sequence and at predetermined timings to execute processing. In that case, the program itself does not contain the above modules and processes in cooperation with the OS. Such a program that does not contain modules may also be included in the program according to the present invention.

[0105] Furthermore, the program according to the present invention may be provided as part of another program. In that case, the program itself does not contain modules included in the other program, and processing is executed in cooperation with the other program. Such a program incorporated into another program may also be included in the program according to the present invention.

[0106] The embodiments disclosed herein should be considered in all respects to be illustrative and not restrictive. The scope of the present invention is indicated by the claims rather than by the foregoing description, and all modifications within the meaning and scope equivalent to the claims are intended to be included. [Explanation of Symbols]

[0107] 1 Quantitative evaluation system, 3 Learning device, 4 Quantitative evaluation device, 10 Cast iron pipe, 11 Cast surface, 31, 41 Input unit, 32 Original image data storage unit, 33 Training data generation unit, 34 Training data storage unit, 35 Learning processing unit, 36, 42 Model storage unit, 37 Evaluation unit, 43 Evaluation decision unit, 44 Output unit, M1, M2 Learning model, M11 Regression model, M12 Defective product classification model, P Pinhole.

Claims

1. A quantitative evaluation device used for inspecting the quality of the casting surface of cast iron pipes, A memory means for storing a learning model generated by machine learning using image data of multiple casting surfaces that have been pre-classified into multiple stages as training data, An image acquisition means for acquiring image data of the casting surface of a cast iron pipe to be inspected, A quantitative evaluation device comprising: an image acquisition means that inputs image data acquired by the image acquisition means into the learning model, and an evaluation determination means that evaluates and determines a level or numerical value based on the output from the model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

2. The quantitative evaluation apparatus according to claim 1, wherein the evaluation determination means evaluates and determines a level greater than the number of classes in the training data, or a numerical value within a continuous, stepless range, as the degree of appearance of the casting surface.

3. The quantitative evaluation apparatus according to claim 1, wherein the training data of the learning model includes, as augmented data, trimmed image data obtained by removing the edges of the original image data of the casting surface.

4. The cast iron pipes subject to inspection were manufactured by the die centrifugal casting method. The quantitative evaluation apparatus according to claim 1, wherein the training data of the learning model includes, as augmented data, rotated image data obtained by rotating the original image data of the casting surface within an angular range of less than 5°.

5. A quantitative evaluation device used for inspecting the quality of the casting surface of cast iron pipes, A storage means for storing a learning model generated by machine learning using image data that has been preprocessed, including adaptive histogram equalization and blurring by bilateral filtering, on image data of multiple casting surfaces that have been pre-classified into multiple stages, as training data, and An image acquisition means for acquiring image data of the casting surface of a cast iron pipe to be inspected, An adjustment means that performs input image adjustment processing, including the preprocessing, on the image data acquired by the image acquisition means, A quantitative evaluation device comprising: an evaluation determination means that inputs image data subjected to the input image adjustment process into the learning model, and evaluates and determines a level or numerical value based on the output from the learning model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

6. The training data of the learning model includes augmented data that has been randomly rotated, flipped, scaled, and translated. The quantitative evaluation apparatus according to claim 5, wherein the input image adjustment process includes data augmentation processing that randomly performs rotation, inversion, scaling, and movement.

7. A quantitative evaluation device used for inspecting the quality of the casting surface of cast iron pipes, A storage means for storing regression models and defective product classification models generated by machine learning using image data of multiple casting surfaces that have been pre-classified into multiple stages, as training data, An image acquisition means for acquiring image data of the casting surface of a cast iron pipe to be inspected, A quantitative evaluation device comprising: an image acquisition means that inputs image data acquired by the image acquisition means into the regression model and the defective product classification model; and an evaluation determination means that evaluates and determines a weighted average of the output values ​​from the regression model and the output values ​​from the defective product classification model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

8. A quantitative evaluation program used for inspecting the quality of the casting surface of cast iron pipes, The process involves reading a learning model, generated by machine learning using image data of multiple casting surfaces that have been pre-classified into multiple stages, from the memory unit, and The steps include: acquiring image data of the casting surface of the cast iron pipe to be inspected, A quantitative evaluation program that causes a computer to perform the steps of inputting acquired image data into the learning model, and evaluating and determining a level or numerical value based on the output from the learning model as the degree of appearance of the casting surface of the cast iron pipe to be inspected.

9. A learning model generation program for generating a learning model for inspecting the quality of the casting surface of cast iron pipes, The process involves inputting raw image data of multiple casting surfaces that have been pre-classified into multiple stages, and The steps include generating extended data based on the input original image data, A learning model generation program that causes a computer to perform a step of machine learning for each class, using at least a portion of the image data of multiple casting surfaces, including the original image data and the extended data, as training data.

10. A learning model generation program for generating a learning model for inspecting the quality of the casting surface of cast iron pipes, The process involves inputting raw image data of multiple casting surfaces that have been pre-classified into multiple stages, and The process involves generating training data by preprocessing the input original image data, including adaptive histogram equalization and blurring using a bilateral filter. A learning model generation program that has a computer perform the steps of machine learning on the generated training data for each class.

Images