Ductility estimation device, estimation model generation device, ductility estimation method, estimation model generation method, and program

Abstract

To predict a value related to the ductility of a ferrous material correctly.SOLUTION: A ductility estimation device comprises: an image acquisition unit which acquires a plurality of images of a ferrous material photographed so as to have different magnifications; a feature amount extraction unit which extracts feature amounts from the plurality of images; an estimation unit which estimates a value related to the ductility of the ferrous material by inputting the feature amounts to an estimation model that has learned so as to output the value related to the ductility of the ferrous material with the feature amounts of the images expressing the ferrous material photographed at the different magnifications as input; and an estimation result output unit which outputs the estimation result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a ductility estimation device, an estimation model generation device, a ductility estimation method, an estimation model generation method, and a program. [Background technology]

[0002] Steel materials are widely used to make industrial products. Among them, high-tensile steel materials with high tensile strength are widely used, mainly in automobiles. It is important to control the properties of high-tensile steel materials, especially their ductility, which indicates the degree of processing that is possible. For example, Patent Document 1 proposes a technology for predicting tensile strength using image analysis and machine learning as a method for predicting the properties of steel materials. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6747391 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the properties of steel materials, such as ductility, are difficult to predict because they are affected by various factors. An object of the present invention is to provide a ductility estimation device, an estimation model generation device, a ductility estimation method, an estimation model generation method, and a program that solve the above-mentioned problems. [Means for solving the problem]

[0005] According to a first aspect of the present invention, a ductility estimation device includes an image acquisition unit that acquires multiple images of a steel material taken at different magnifications, a feature extraction unit that extracts features from the multiple images, an estimation unit that estimates a value related to the ductility of the steel material by inputting the features of images representing the steel material taken at different magnifications into an estimation model that has been trained to output a value related to the ductility of the steel material, and an estimation result output unit that outputs the result of the estimation. [Effects of the Invention]

[0006] According to the present invention, it is possible to more accurately predict values relating to the ductility of steel materials. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an estimation system 1. [Figure 2] FIG. 2 is a diagram illustrating a configuration of an estimation model generating device 13. [Figure 3] 10 is a flowchart showing a method for generating an estimation model by the estimation model generating device 13. [Figure 4] FIG. 2 is a diagram showing the configuration of a ductility estimation device 14. [Figure 5] 4 is a flowchart showing the operation of the ductility estimation device 14. [Figure 6] 1 is an example of an image taken by an electron microscope in this experiment. [Figure 7] FIG. 10 is a diagram showing an estimation model used by the ductility estimation device 14 and the prediction accuracy of the stretch flange value output. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. First Embodiment FIG. 1 is a diagram showing the configuration of an estimation system 1. As shown in FIG. The estimation system 1 includes an electron microscope 10, a ductility data output device 11, a data set output device 12, an estimation model generation device 13, and a ductility estimation device 14.

[0009] The electron microscope 10 captures images of a steel material sample and outputs them to a dataset output device 12 or a ductility estimation device 14. The electron microscope 10 is, for example, a scanning electron microscope or a transmission electron microscope. The ductility data output device 11 acquires values related to the ductility of the steel material and outputs them to the dataset output device 12. The dataset output device 12 combines the acquired sample images with the values related to the ductility of the steel material to generate a dataset and outputs it to the estimation model generation device 13. The estimation model generation device 13 generates an estimation model used by the ductility estimation device 14 for estimation. The ductility estimation device 14 estimates the values related to the ductility of the steel material based on the images acquired from the electron microscope 10. In the following explanation, it is assumed that the ductility estimation device 14 estimates a value related to stretch flangeability, which is a representative value related to ductility.

[0010] The ductility data output device 11 accepts input of values indicating the measured stretch flangeability, generates data on the stretch flangeability, and outputs it to the data set output device 12. The stretch flangeability is evaluated, for example, by the following hole expansion test. The person performing the hole expansion test (hereinafter referred to as the tester) first punches out the steel material with a 10 mm diameter hole punch so that the clearance during punching is 12.5%. Next, the tester presses out the punched sample with a conical punch with an apex angle of 60 degrees until the sample breaks, and measures the hole diameter of the sample at the time of break. The tester divides the hole diameter of the sample at the time of break by the hole diameter before pressing out, and determines this value as the hole expansion test value, and this value is a value indicating the measured stretch flangeability (hereinafter referred to as the stretch flange measurement value). The ductility data output device 11 acquires data on stretch flangeability from an external source. For example, the ductility data output device 11 acquires values input via a keyboard or the like.

[0011] The data set output device 12 generates a data set consisting of a plurality of combinations of sample images and stretch flange measurement values for the sample. One combination includes a plurality of sample images taken at different magnifications that are input to the data set output device 12 by the electron microscope 10, and stretch flange measurement values that are input to the data set output device 12 by the ductility data output device 11.

[0012] <<Estimation model generation device>> 2 is a diagram showing the configuration of the estimation model generation device 13. The estimation model generation device 13 includes a data acquisition unit 131, a feature extraction model storage unit 132, a feature extraction unit 133, an estimation model creation unit 134, and an estimation model output unit 135. The data acquisition unit 131 acquires a data set from the data set output device 12 . The feature extraction model storage unit 132 stores a feature extraction model used by the feature extraction unit 133. The feature extraction model is a model that receives an image as input and outputs features of the image. For example, the feature extraction model may be a part of an existing image recognition model such as inception-v3 (https: / / tfhub.dev / google / imagenet / inception_v3 / feature_vector / ) to reduce the learning burden. In other words, of an image recognition model having an input layer, an intermediate layer, and an output layer, only the input layer and the intermediate layer may be extracted and used as the feature extraction model. Note that the feature extraction model according to other embodiments may be a model trained by an autoconverter or the like using a dataset, for example.

[0013] The feature extraction unit 133 extracts features of an image by inputting the image generated from multiple images at different magnifications included in the dataset into the feature extraction model. Specifically, if the dataset includes three grayscale images at different magnifications corresponding to one stretch flange measurement value, the feature extraction unit may generate a color image by assigning each image to an R layer, a G layer, and a B layer, respectively, and input the generated color image into the feature extraction model to extract features. In other words, the feature extraction unit can extract features by inputting grayscale images at different magnifications into the R layer, G layer, and B layer of the feature extraction model. The extracted features may include all of the features possessed by each of the three images. Furthermore, the feature extracted by the feature extraction unit 133 does not have to be the calculation result that is finally output by the feature extraction model. For example, the feature extraction unit may use a calculation result prior to the final calculation result as the feature.

[0014] The estimation model creation unit 134 creates an estimation model based on the feature values extracted by the feature value extraction unit 133 and the stretch flange measurement values included in the dataset. For example, the estimation model creation unit 134 creates multiple decision trees from the dataset and the extracted feature values using a technique called bagging or boosting, and creates an estimation model that obtains a solution from the multiple decision trees. Note that the estimation model according to other embodiments may be another machine learning model such as a neural network model. The estimation model output unit 135 outputs the estimation model created by the estimation model creation unit 134.

[0015] <<Operation of the estimation model generation device>> FIG. 3 is a flowchart showing a method for generating an estimation model by the estimation model generating device 13. First, the data acquisition unit 131 acquires a dataset from the dataset output device 12 (step S101). The feature extraction unit 133 generates one color image from three grayscale images at different magnifications included in the dataset (step S102). Specifically, the color image is generated by assigning an X-magnified image to the R layer, a Y-magnified image to the G layer, and a Z-magnified image to the B layer. The feature extraction unit 133 inputs the color image into a feature extraction model stored in the feature extraction model storage unit 132, thereby obtaining features of the image from the feature extraction model (step S103). Thereafter, the estimation model creation unit 134 creates an estimation model based on the features and the stretch flange measurement values (step S104). Thereafter, the estimation model output unit 135 outputs the estimation model created by the estimation model creation unit 134 to the ductility estimation device 14 (step S105).

[0016] Ductility estimation device FIG. 4 is a diagram showing the configuration of the ductility estimation device 14. The ductility estimation device 14 includes an image acquisition unit 141 , a feature extraction model storage unit 142 , a feature extraction unit 143 , an estimation model storage unit 144 , an estimation unit 145 , and an estimation result output unit 146 . The image acquisition unit 141 acquires a plurality of images taken at different magnifications from the electron microscope 10. The feature extraction model storage unit 142 stores the feature extraction model used by the feature extraction unit. The feature extraction unit 143 extracts the feature of the image by inputting the image acquired by the image acquisition unit into the feature extraction model. The feature extraction model storage unit 142 and the feature extraction unit 143 are similar to the feature extraction model storage unit 132 and the feature extraction unit 133 provided in the estimation model generating device 13 .

[0017] The estimation model storage unit 144 stores the estimation model input from the estimation model generation device 13 to the ductility estimation device 14 . The estimation unit 145 inputs the feature amounts of the image extracted by the feature amount extraction unit 143 into an estimation model, thereby estimating the stretch flange value of the steel material of the image. The estimation result output unit 146 outputs the estimated stretch flangeability to a display or the like.

[0018] <<Operation of the ductility estimation device>> FIG. 5 is a flowchart showing the operation of the ductility estimation device 14. The image acquisition unit 141 acquires multiple images of the steel material from the electron microscope 10 (step S201). The multiple images acquired by the image acquisition unit 141 are grayscale images. Thereafter, the feature extraction unit 143 generates one color image from three grayscale images at different magnifications among the images acquired by the image acquisition unit (step S202). The feature extraction unit 143 inputs the color image into a feature extraction model stored in the feature extraction model storage unit 142, thereby obtaining feature quantities of the image from the feature extraction model (step S203). The estimation unit 145 estimates the stretch flangeability by inputting the feature quantities into an estimation model stored in the estimation model storage unit 144 (step S204). The estimation result output unit 146 outputs the estimation result (step S205).

[0019] As described above, the ductility estimation device 14 includes an image acquisition unit 141 that acquires multiple images of the steel material taken at different magnifications, a feature extraction unit 143 that extracts features from the multiple images, an estimation unit 145 that estimates a value related to the ductility of the steel material by inputting the features into an estimation model that has been trained to use the feature values of the images representing the steel material taken at different magnifications as input and output a value related to the ductility of the steel material, and an estimation result output unit 146 that outputs the estimation result. As will be described later, by acquiring multiple images of the steel material and using the feature values extracted from the multiple images, the ductility estimation device 14 can reflect the features contained in the multiple images in the feature values, thereby enabling more accurate estimation of stretch flangeability.

[0020] <experiment> The evaluation experiment of the ductility estimation device will be described below. The object of measurement in this experiment was a TS 780MPa-class hot-rolled high-tensile strength steel plate. The chemical composition of this steel, excluding iron (Fe), is as follows: C 0.03-0.07%, Si 0-0.5%, Mn 1.2-1.8%, P 0.008-0.012%, S 0.0007-0.0016%, and Ti 0.05-0.15%. The stretch flange value was a hole expansion test value measured by the hole expansion test. The measurement method was as described above. The images were taken using a scanning electron microscope, a type of electron microscope, of the metal structure at a position 1 / 4 of the plate thickness in the cross section of the steel plate. Magnifications were set at 1000x, 2500x, and 4000x, and three images were taken for one sample.

[0021] <Example of image> Figure 6 shows an example of an image taken with an electron microscope in this experiment. The images shown in Figure 6 are, from left to right, images of the sample taken at 1000x, 2500x, and 4000x magnifications, respectively. The multiple images taken with the electron microscope preferably include images taken at magnifications of 500x to 1200x, 1500x to 3000x, and 3500x to 10000x. Magnifications of 500x to 1200x are effective for analyzing the volume fraction of the matrix phase of a sample such as ferrite. Magnifications of 1500x to 3000x are effective for analyzing the distribution of second phases in a sample such as martensite. Magnifications of 3500x to 10000x are effective for analyzing the grain size and shape of the crystals in the sample.

[0022] For example, in the image shown in Figure 6, soft structures such as ferrite are shown in black, and hard structures such as martensite are shown in white. From an image magnified at 1000x, the ratio of white to black regions can be obtained, i.e., information about the average soft structure ratio in steel materials. In an image magnified at 2500x, the white regions are more clearly visible, allowing information about the grain size and distribution of the hard structure crystals to be obtained. In an image magnified at 4000x, the black regions are more clearly visible, allowing information about the grain size of the soft structure crystals to be obtained. It is known that the proportion of soft structure, the grain size and distribution of the hard structure, and the grain size of the soft structure crystals each affect stretch flangeability. For example, it is known that a higher proportion of soft structure (ferrite) improves stretch flangeability.

[0023] Seventy-nine types of hot-rolled steel sheets with different chemical composition ratios within the aforementioned range and manufactured under different hot rolling conditions were used as test materials. The hot rolling conditions included the rolling start temperature (temperature before the final stand of the finishing mill) in the final stand of the finishing mill ranging from 820°C to 980°C, and the coiling temperature ranging from 400°C to 650°C. Hole expansion test values and three images with different magnifications were acquired from each test material, and these were combined to form one dataset. In other words, one dataset consists of one hole expansion test value and three images with different magnifications. A dataset was created from each test material, for a total of 79 datasets. These 79 datasets were then used to generate two different estimation models. One model (estimation model A) was generated based on the hole expansion test values included in each dataset and the three images corresponding to each hole expansion test value. The three images were images of the test material taken at magnifications of 1000x, 2500x, and 4000x. Then, an arbitrary area smaller than the image was specified from the image, and the specified area was used as a new image (hereinafter referred to as a small image). Five small images were generated for each image. In other words, five small images taken at different magnifications correspond to one hole expansion test value. The number of small images is the product of the number of data sets, the number of images included in one data set, and the number of areas specified in one image, which is 1185 (= 79 × 3 × 5). Estimation model A is an estimation model according to the first embodiment. More specifically, estimation model A was generated from a combination of features extracted by inputting one image each of hole expansion test values and three small images at different magnifications corresponding to the hole expansion test values into the R layer, G layer, and B layer of the feature extraction model.

[0024] The other model, estimation model B, was generated using the same 79 datasets as estimation model A. However, only the 4000x magnification images were used from the 1000x, 2500x, and 4000x images included in each dataset, and 15 small images were obtained from each image. Therefore, the number of small images generated was 1185 (= 79 × 1 × 15), the same as estimation model A. This resulted in the same number of images input when constructing estimation model A and estimation model B. Note that when generating estimation model A, one small image with a different magnification was input to the R layer, G layer, and B layer of the feature extraction model, for a total of three small images, but when generating estimation model B, the same small image with the same magnification (4000x) was input to all layers. Therefore, the number of training cycles was 1185 when generating estimation model B, compared to 395 when generating estimation model A, resulting in more training cycles when generating estimation model B. Estimation model B is an estimation model related to a comparative example.

[0025] The two estimation models were then verified using eight similarly created datasets, using eight types of hot-rolled steel sheets manufactured under different hot-rolling conditions and with different chemical composition ratios within the aforementioned range of chemical composition ratios as test materials. Estimation Model A used feature values extracted by inputting 1000x, 2500x, and 4000x magnification images into the R, G, and B layers, respectively, while Estimation Model B used images taken at 4000x magnification.

[0026] 7 is a diagram showing the estimation model used by the ductility estimation device 14 and the prediction accuracy of the stretch flange value output. The prediction accuracy is a value indicating the reliability of the estimation model used, and the closer it is to 1, the better the estimation model is. It can be seen that estimated model A, created from three images taken at different magnifications, has higher prediction accuracy and is a better estimated model than estimated model B, created from one image taken at one magnification. This is because the image used to generate estimated model A contains information on the ratio of hard and soft tissue, as well as the grain size and distribution of hard tissue, and therefore contains information on different scales of the steel material.

[0027] Other Embodiments One embodiment of the present invention has been described in detail above with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes and the like are possible within the scope that does not deviate from the gist of the present invention. For example, in the above-described embodiment, the ductility estimation device 14 acquires images from the electron microscope 10, but this is not limited to this. For example, the ductility estimation device 14 may acquire images from a database in which images taken by the electron microscope 10 are stored.

[0028] Furthermore, the values estimated by the ductility estimation device 14 are not limited to the hole expansion test values of the stretch flangeability. For example, values related to deep drawing characteristics, stretching characteristics, and bending characteristics, which are values related to ductility, can also be estimated. This is because the deep drawing characteristics, stretching characteristics, and bending characteristics, which are values related to ductility, are affected by factors of various scales, just like the stretch flangeability. The value indicating deep drawing properties can be measured by the following method. First, the front and back surfaces of the sample are machined to prepare a circular test piece for cylindrical drawing. The drawing ratio at which the test piece can be drawn without breaking in cylindrical drawing is taken as the limiting drawing ratio, and this value is taken as the value indicating deep drawing properties. The value indicating the stretching property can be measured by the following method. First, the front and back surfaces of the sample are machined to prepare a rectangular test piece. Then, the test piece is completely restrained with a beaded die and stretched with a ball-nosed punch until it breaks. The maximum forming height at which it breaks is taken as the value indicating the stretching property. The value indicating the bending properties can be measured using the following method. First, the front and back surfaces of the sample are machined to create a rectangular test piece. Then, using a V-shaped block with an apex angle of 90° and a die, the test piece is pressed into blocks of various radii and bent. The bending ridge of the bent test piece is observed, and the smallest radius at which no cracks are found is taken as the limiting bending radius, and this value is taken as the value indicating the bending properties. In the estimation system 1, by using the deep drawing property, the stretching property, or the bending property instead of the stretch flange value, it is possible to similarly estimate the value of each property. Furthermore, the magnification of the image of the test material used to estimate the deep drawing property, the stretching property, or the bending property may be the same as the magnification of the image of the test material used to estimate the stretch flange value. [Explanation of symbols]

[0029] 1. Estimation System 10. Electron microscope 11 Ductility data output device 12 Dataset output device 13 Estimation model generation device 14 Ductility estimation device 131 Data Acquisition Unit 132 Feature extraction model memory unit 133 Feature Extraction Unit 134 Estimation Model Creation Department 135 Estimation model output section 141 Image acquisition unit 142 Feature extraction model memory unit 143 Feature Extraction Unit 144 Estimation model memory unit 145 Estimation Department 146 Estimation result output section

Claims

1. an image acquisition unit that acquires a plurality of images of the steel material taken at different magnifications; a feature extraction unit that extracts feature amounts from the plurality of images; an estimation unit that estimates a value related to the ductility of the steel material by inputting feature quantities of images representing the steel material photographed at different magnifications into an estimation model that has been trained to output a value related to the ductility of the steel material using the feature quantities as input; an estimation result output unit that outputs the result of the estimation; Equipped with the plurality of images of the steel material are grayscale images of the steel material taken at three different magnifications; the feature extraction unit generates one color image by allocating the grayscale images of the steel material photographed at the three different magnifications to an R layer, a G layer, and a B layer, respectively, inputs the generated color image to a feature extraction model that takes an image as input and outputs feature amounts of the input image, and extracts the output from the feature extraction model as feature amounts of the color image; Ductility estimation device.

2. The ductility is a stretch flangeability. The ductility estimation device according to claim 1 .

3. The plurality of images are an image of the steel material taken at a magnification of 500 times or more and 1200 times or less; An image of the steel material taken at a magnification of 1500 times or more and 3000 times or less; and An image of the steel material taken at a magnification of 3,500 times or more and 10,000 times or less The ductility estimation device according to claim 1 or 2, comprising:

4. The feature extraction model is a model trained using a dataset containing grayscale images of steel materials taken at three different magnifications and corresponding values related to the ductility of the steel materials. The ductility estimation device according to any one of claims 1 to 3.

5. The feature extraction unit extracts intermediate calculation results obtained in the calculation process of the trained model as the feature. The ductility estimation device according to claim 4.

6. a data acquisition unit that acquires a plurality of images of the steel material taken at different magnifications and a value related to the ductility of the steel material; a feature extraction unit that extracts feature amounts from the plurality of images; an estimation model creation unit that creates an estimation model for determining a value related to the ductility of a steel material from the feature amounts of an image of the steel material, using a learning dataset in which feature amounts of images representing the steel material photographed at different magnifications are used as input samples and values related to the ductility of the steel material are used as output samples; an estimation model output unit that outputs the estimation model; Equipped with the plurality of images of the steel material are grayscale images of the steel material taken at three different magnifications; the feature extraction unit generates one color image by allocating the grayscale images of the steel material photographed at the three different magnifications to an R layer, a G layer, and a B layer, respectively, inputs the generated color image to a feature extraction model that takes an image as input and outputs feature amounts of the input image, and extracts the output from the feature extraction model as feature amounts of the color image; Estimation model generator.

7. The estimation model is an estimation model output by the estimation model generation device according to claim 6. The ductility estimation device according to any one of claims 1 to 5.

8. an image acquisition step of acquiring a plurality of images of the steel material taken at different magnifications; a feature extraction step of extracting feature amounts from the plurality of images; an estimation step of estimating a value related to the ductility of the steel material by inputting feature quantities of images representing the steel material photographed at different magnifications into an estimation model that has been trained to output a value related to the ductility of the steel material using the feature quantities as input; an estimation result output step of outputting a result of the estimation; and the plurality of images of the steel material are grayscale images of the steel material taken at three different magnifications; In the feature extraction step, one color image is generated by allocating the grayscale images of the steel material photographed at the three different magnifications to an R layer, a G layer, and a B layer, respectively, and the generated color image is input to a feature extraction model that takes an image as input and outputs feature amounts of the input image, and the output from the feature extraction model is extracted as the feature amount of the color image. Ductility estimation method.

9. a data acquisition step of acquiring a plurality of images of the steel material taken at different magnifications and a value related to the ductility of the steel material; a feature extraction step of extracting feature amounts from the plurality of images; an estimation model step of creating an estimation model for determining a value related to the ductility of a steel material from the feature amounts of the image of the steel material using a learning dataset in which feature amounts of images representing the steel material photographed at different magnifications are used as input samples and values related to the ductility of the steel material are used as output samples; and the plurality of images of the steel material are grayscale images of the steel material taken at three different magnifications; In the feature extraction step, one color image is generated by allocating the grayscale images of the steel material photographed at the three different magnifications to an R layer, a G layer, and a B layer, respectively, and the generated color image is input to a feature extraction model that takes an image as input and outputs feature amounts of the input image, and the output from the feature extraction model is extracted as the feature amount of the color image. Estimation model generation method.

10. A program for causing a computer to execute the method according to claim 8.

11. A program for causing a computer to execute the method according to claim 9.

Images