Fracture surface evaluation system, fracture surface evaluation method, and fracture surface evaluation program

Abstract

To efficiently estimate the cause of fracture at the fracture surface. [Solution] The fracture surface evaluation system acquires a target image that captures at least a portion of the target fracture surface of the target article, receives a first image that captures the first fracture surface of the first article, and inputs the target image to a first trained model that estimates which of a predetermined number of types of fractures the cause of the first fracture surface is, and estimates the cause of the target fracture surface. If it is estimated that the cause of the target fracture surface is potential fatigue fracture, it acquires a small region image that captures a small region which is a part of the captured range of the target image, receives a second image that captures a small region of the second fracture surface of the second article, and inputs the small region image to a second trained model that determines whether or not striations exist on the small region, and executes a determination process to determine whether or not striations exist on the small region, and if it is determined that striations exist on the small region, it estimates that the cause of the target fracture surface is fatigue fracture.

Description

Technical Field

[0001] One aspect of the present disclosure relates to a broken surface evaluation system, a broken surface evaluation method, and a broken surface evaluation program.

Background Art

[0002] Techniques for evaluating the broken surface of an article are known. For example, Patent Document 1 discloses a starting point estimation unit that estimates the position of a starting point on the broken surface of a substance included in an input image based on the input image including the broken surface of the substance and a pre-learned starting point estimation model, and a destruction mode estimation unit that estimates the destruction mode of the broken surface of the substance included in the input image based on the input image and a pre-learned destruction mode estimation model.

[0003] Patent Document 2 describes an apparatus including a model generation unit that generates a broken surface feature region determination model for determining a feature region of a broken surface image by machine learning using, as learning data, a plurality of data associated with a broken surface image of a material and information representing the position of a feature region in the broken surface image, and an output unit that outputs the broken surface feature region determination model.

[0004] Patent Document 3 describes a broken surface analysis system characterized by estimating mechanical numerical values acting at the time of broken surface formation from the step interval of the broken surface unevenness of the fatigue broken surface of a structure.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0006] There is a need to efficiently estimate the cause of fracture at the fracture surface. [Means for solving the problem]

[0007] A fracture surface evaluation system relating to one aspect of this disclosure comprises at least one processor. The at least one processor acquires a target image that captures at least a portion of the target fracture surface, which is the fracture surface of a target article, and inputs the target image to a first trained model that receives a first image capturing the first fracture surface of a first article and estimates which of a predetermined plurality of types of fracture the cause of the failure of the first fracture surface is, thereby estimating the cause of the failure of the target fracture surface. If the predetermined plurality of types of fracture include potential fatigue fracture and one or more other types of fracture, and it is estimated that the cause of the failure of the target fracture surface is potential fatigue fracture, the processor acquires a small region image that captures a small region which is part of the captured range of the target image, and inputs the small region image to a second trained model that receives a second image capturing a portion of the second fracture surface of a second article and determines whether or not striations exist on the portion of the region, thereby executing a determination process to determine whether or not striations exist on the small region, and if it is determined that striations exist on the small region, it is estimated that the cause of the failure of the target fracture surface is fatigue fracture.

[0008] In this regard, first, the target image and the first trained model are used to estimate which of several types of fracture, including potential fatigue fracture (the possibility of fatigue fracture), the cause of fracture on the target fracture surface falls under. If the cause of fracture is estimated to be potential fatigue fracture, a small-region image showing a part (small area) of the target image and the second trained model are used to determine whether or not striations exist on the small region. If striations exist, the cause of fracture on the target fracture surface is determined to be fatigue fracture. In this way, fatigue fracture can be estimated efficiently by estimating it through a two-stage process that includes striation determination. [Effects of the Invention]

[0009] According to one aspect of this disclosure, the cause of fracture of the fracture surface can be efficiently estimated. [Brief explanation of the drawing]

[0010] [Figure 1] This figure shows an example of the functional configuration of a fracture surface evaluation system. [Figure 2] This diagram illustrates the two-stage estimation process performed by the estimation system. [Figure 3] This diagram illustrates the two-stage estimation process performed by the estimation system. [Figure 4] This figure shows an example of the computer hardware configuration used in a fracture surface evaluation system. [Figure 5] This flowchart shows an example of how the second learning system works. [Figure 6] This figure shows an example of an actual striation image. [Figure 7] This is a flowchart illustrating an example of how the estimation system works. [Figure 8] This flowchart shows the details of the second stage of estimation. [Figure 9] This figure shows an example of a result image corresponding to the target image. [Figure 10] This figure shows an example of a result image corresponding to an observed image. [Figure 11] This diagram shows the functional configuration of the fracture surface evaluation system related to modified shapes. [Figure 12] This diagram shows the functional configuration of the fracture surface evaluation system related to modified shapes. [Figure 13] This diagram shows the functional configuration of the fracture surface evaluation system related to modified shapes. [Modes for carrying out the invention]

[0011] The following describes various examples in this disclosure in detail with reference to the attached drawings. In the description of the drawings, identical or equivalent elements are denoted by the same reference numeral, and redundant descriptions are omitted.

[0012] [System Overview] The fracture surface evaluation system according to the present disclosure is a computer system for evaluating a target fracture surface that is a fracture surface of a target article. As at least part of the evaluation, the fracture surface evaluation system determines whether striations are present on a small area that is part of the target fracture surface by means of a machine learning model. Striations found in fatigue fractures are microscopic phenomena confirmed in areas on the order of several micrometers square. Although striations can be searched for using an image obtained by photographing the target fracture surface with an electron microscope, manual search requires a lot of time and labor. Also, it is difficult to accurately identify striations with an object detection algorithm. The fracture surface evaluation system processes an image depicting a small area of the target fracture surface by means of a machine learning model and automatically determines whether striations are present on the small area. Therefore, it becomes possible to easily search for striations. Also, it becomes possible to accurately determine striations. Striations serve as evidence that the cause of fracture of the target fracture surface is fatigue fracture. Therefore, by automating the determination regarding striations, it is also possible to automate the estimation of whether the cause of the fracture is fatigue fracture. Thus, the fracture surface evaluation system can improve the efficiency of evaluating the target fracture surface.

[0013] The target article refers to an article that is the object of processing by the fracture surface evaluation system. For example, the target article can be any article made of metal or resin.

[0014] The image processed by the fracture surface evaluation system is, for example, an electron image. The electron image may be a secondary electron image, a reflected electron image (backscattered electron image), or an Auger electron image.

[0015] Machine learning is a method of autonomously discovering laws or rules by iteratively learning based on given information. A machine learning model is a computational model used for machine learning. In this disclosure, a machine learning model generated by machine learning using training data, that is, a machine learning model generated in the learning phase, is also referred to as a "trained model." In the operational phase (estimation phase), input data for which the correct answer is unknown is processed by the trained model, and estimation or judgment is performed on the input data.

[0016] [System Configuration] Figure 1 shows the functional configuration of a fracture surface evaluation system 1 according to one example. The fracture surface evaluation system 1 estimates the cause of fracture of a target fracture surface in two stages. In one example, the fracture surface evaluation system 1 comprises a first learning system 10 that generates a first trained model 41 used for the first stage of estimation, a second learning system 20 that generates a second trained model 42 used for the second stage of estimation, and an estimation system 30 that evaluates the target fracture surface using the first trained model and the second trained model 42. The first learning system 10 and the second learning system 20 correspond to the learning phase, and the estimation system 30 corresponds to the operation phase. The first learning system 10 and the second learning system 20 are connected to a learning database 2 via a communication network. The learning database 2 is a device that stores training data used for machine learning. The learning database 2 may be a component of the fracture surface evaluation system 1 or may be located outside of the fracture surface evaluation system 1. The communication network may be constructed by the internet, an intranet, or a combination thereof. The communication network may be constructed by a wired network, a wireless network, or a combination thereof.

[0017] Figures 2 and 3 are diagrams illustrating the two-stage estimation process performed by the estimation system 30. In this example, the estimation system 30 acquires an observation image 200 that captures the target fracture surface 90 as the observation area. The estimation system 30 performs a segmentation process on the observation image 200 to acquire multiple target images 210 from the observation image 200. Each target image captures at least a portion of the target fracture surface. For each of the multiple target images 210, the estimation system 30 estimates which of a predetermined number of fracture types caused the fracture of the target fracture surface 90. This is the first stage of estimation. In the following explanation, it is assumed that the "predetermined number of fracture types" are three types: "brittle fracture," "ductile fracture," and "other types of fracture." "Other types of fracture" means fracture that is neither brittle fracture nor ductile fracture. If the fracture cause is estimated to be "other types of fracture," it is possible that the final fracture cause will be estimated to be fatigue fracture. In other words, "other types of fracture" can be rephrased as "potential fatigue fracture." In the first stage of estimation, please note that the estimated cause of destruction may not necessarily be the same across multiple target images 210 generated from a single observation image 200.

[0018] The estimation system 30 performs a second-stage estimation on the target image 210, which is estimated to be a potential fatigue fracture (other type of fracture). The estimation system 30 performs region segmentation on the target image 210 to obtain a plurality of sub-region images 220 from the target image 210. A sub-region image is an image that captures a small region that is part of the target fracture surface. For each of the plurality of sub-region images 220, the estimation system 30 performs a determination process to determine whether or not striations are present. Based on the determination process for each of the plurality of sub-region images 220, the estimation system 30 generates striation information indicating the location of striations within the captured area of ​​the target image. In this disclosure, the captured area of ​​the target image is also referred to as the "target area". Based on the striation information, the estimation system 30 estimates whether or not the cause of fracture of the target fracture surface 90 (the cause of fracture in the target area) is fatigue fracture. Based on the striation information, the estimation system 30 may generate a result image 230 representing the distribution of striations.

[0019] Brittle fracture and ductile fracture can be clearly identified from images that capture a relatively wide area. On the other hand, fatigue fracture cannot be identified without carefully examining images that capture a relatively narrow area. The magnification required to identify fatigue fracture depends on the material, stress, etc., but it is typically identified at a magnification of 3 times or more compared to brittle fracture and ductile fracture. For example, brittle fracture and ductile fracture can be identified at around 1000x magnification, while fatigue fracture can be identified at around 3000-5000x magnification. The estimation system 30 can efficiently estimate the cause of fracture at the fracture surface by performing the above two-stage estimation.

[0020] Returning to Figure 1, in one example, the first learning system 10 includes a first learning unit 11 as a functional component. The first learning unit 11 is a functional module that generates a first trained model 41 by machine learning, which is used in the first stage of estimating the cause of fracture of a target fracture surface. The first trained model 41 is a computational model that receives an image (first image) of the fracture surface of an article (first fracture surface of the first article) and estimates which of a predetermined number of fracture types the cause of fracture of the fracture surface (first fracture surface) is. The first trained model 41 is configured using a neural network model, such as a deep learning model. The first learning unit 11 may also generate the first trained model 41 so that it can estimate two or more fractures as the cause of fracture of a fracture surface from a single image.

[0021] In one example, the second learning system 20 includes a data expansion unit 21, a filtering unit 22, and a second learning unit 23 as functional components.

[0022] The data augmentation unit 21 is a functional module that augments the training data used to generate the second trained model 42, which is used in the second stage of estimating the cause of fracture of the target fracture surface. Training data augmentation refers to the process of increasing the number of data records in the training data. The data augmentation unit 21 comprises a model generation unit 21a, a pseudo-image generation unit 21b, and a transformation unit 21c. The model generation unit 21a is a functional module that generates an image generation model, which is a computational model for generating pseudo-images. The pseudo-image generation unit 21b is a functional module that generates pseudo-images as additional training images using the image generation model. The transformation unit 21c is a functional module that generates additional training images using geometric transformations.

[0023] The filtering unit 22 is a functional module that performs filtering on each training image in the training data to emphasize a predetermined frequency range for each training image.

[0024] The second learning unit 23 is a functional module that generates a second trained model 42 by machine learning using training data that has undergone data augmentation and filtering. The machine learning performed by the second learning unit 23 is different from the machine learning performed by the first learning unit 11. The second trained model 42 is a computational model that receives an image (second image) showing a portion of the fracture surface of an article (second fracture surface of the second article) and determines whether or not striations exist on the portion of the image. The second trained model 42 is constructed using a neural network model, such as a deep learning model.

[0025] In one example, the estimation system 30 includes a first image splitting unit 31, a first estimation unit 32, a second image splitting unit 33, a filtering unit 34, a second estimation unit 35, and a result output unit 36 ​​as functional components.

[0026] The first image division unit 31 is a functional module that performs a division process on an observation image that captures at least a portion of the target fracture surface as an observation area, and acquires multiple target images from the observation image.

[0027] The first estimation unit 32 is a functional module that inputs each of a plurality of target images into the first trained model 41 and estimates the cause of fracture of the target fracture surface for each of the plurality of target images.

[0028] The second image segmentation unit 33 is a functional module that performs region segmentation on a target image in which the cause of fracture of the target fracture surface is estimated to be potential fatigue fracture, and acquires multiple sub-region images from the target image.

[0029] The filtering unit 34 is a functional module that performs filtering on each of the multiple small region images to emphasize a predetermined frequency range for each small region image.

[0030] The second estimation unit 35 is a functional module that estimates whether or not the cause of failure in the target region is fatigue failure. The second estimation unit 35 includes a determination unit 35a that determines whether or not striations exist using a second trained model 42. The determination unit 35a is a functional module that inputs each of a plurality of sub-region images to the second trained model 42 and determines whether or not striations exist on each of the plurality of sub-regions.

[0031] The result output unit 36 ​​is a functional module that outputs the results of a two-stage estimation. The results indicate at least the cause of fracture of the target fracture surface.

[0032] Figure 4 shows an example of the hardware configuration of the computer 100 that constitutes the fracture surface evaluation system 1. For example, the computer 100 comprises a processor 101, a main memory unit 102, an auxiliary memory unit 103, a communication control unit 104, an input device 105, and an output device 106. The processor 101 executes the operating system and application programs. The main memory unit 102 is composed of, for example, ROM and RAM. The auxiliary memory unit 103 is composed of, for example, a hard disk or flash memory, and generally stores a larger amount of data than the main memory unit 102. The communication control unit 104 is composed of, for example, a network card or a wireless communication module. The input device 105 is composed of, for example, a keyboard, mouse, touch panel, etc. The output device 106 is composed of, for example, a monitor and a speaker.

[0033] Each functional module of the fracture surface evaluation system 1 is realized by a fracture surface evaluation program 110 pre-stored in the auxiliary storage unit 103. Each functional module is realized by loading the fracture surface evaluation program 110 onto the processor 101 or the main memory unit 102 and having the processor 101 execute the fracture surface evaluation program 110. The processor 101 operates the communication control unit 104, the input device 105, or the output device 106 according to the fracture surface evaluation program 110, and reads and writes data to the main memory unit 102 or the auxiliary storage unit 103.

[0034] The fracture surface evaluation program 110 may be provided on a non-temporary recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, the fracture surface evaluation program 110 may be provided via a communication network as a data signal superimposed on a carrier wave. The fracture surface evaluation program 110 may be divided into programs that implement each functional module of the first learning system 10, programs that implement each functional module of the second learning system 20, and programs that implement each functional module of the estimation system 30. In this case, each of the divided programs may be provided independently.

[0035] The fracture surface evaluation system 1 may consist of one computer 100 or multiple computers 100. When multiple computers 100 are used, these computers 100 are connected via a communication network such as the Internet or an intranet to logically construct a single fracture surface evaluation system 1. The fracture surface evaluation system 1 may also be constructed by combining multiple types of computers.

[0036] [System operation] The following describes the operation of the fracture surface evaluation system 1, as well as the fracture surface evaluation method related to this disclosure.

[0037] (Generation of the first pre-trained model) The first learning unit 11 accesses the learning database 2 and reads training data (first training data) from the learning database 2 to generate the first trained model 41. Each data record of the first training data represents a combination of a target image (training image) prepared in advance for machine learning and an annotation (label) indicating the cause of fracture in the area covered by the target image. In one example, each target image is obtained by photographing the fracture surface with an electron microscope such as a scanning electron microscope (SEM). As described above, "brittle fracture," "ductile fracture," or "other types of fracture" are set as annotations. The annotations are used as the ground truth in machine learning.

[0038] The first learning unit 11 generates a first trained model 41 by performing machine learning using the first training data. The machine learning model for generating the first trained model 41 is a computational model that receives an image (first image) of the fracture surface of an article (first fracture surface of the first article) and estimates which of a predetermined number of fracture types caused the fracture of the fracture surface (first fracture surface).

[0039] In one example, the first learning unit 11 uses a pre-prepared neural network model as a machine learning model to generate the first trained model 41. The first learning unit 11 inputs the target image (training image) indicated by the data records of the first training data into its machine learning model. The first learning unit 11 then obtains the cause of fracture estimated by the machine learning model. That is, the first learning unit 11 inputs the target image into the machine learning model and estimates the cause of fracture of the target fracture surface. The first learning unit 11 updates the parameter set in the machine learning model by backpropagation based on the error between the estimated cause of fracture and the annotation (ground truth) indicated by the data records. The first learning unit 11 repeats this process using multiple data records of the second training data to generate the first trained model 41. The first learning unit 11 stores the generated first trained model 41 in a predetermined memory unit. This first trained model 41 is then used by the first estimation unit 32. The first pre-trained model 41 can also be described as a pre-trained model that estimates the cause of failure, in other words, a failure cause estimation model.

[0040] (Generating the second pre-trained model) The operation of the second learning system 20 will be explained with reference to Figure 5. Figure 5 is a flowchart showing an example of the operation of the second learning system 20 as processing flow S1.

[0041] In step S11, the data augmentation unit 21 reads training data (second training data) from the training database 2 to generate the second trained model 42. Each data record in the second training data represents a combination of a small region image (training image) prepared in advance for machine learning and an annotation (label) indicating whether or not striations exist in the area (i.e., small region) of the small region image. In one example, individual small region images are obtained by photographing the fracture surface with an electron microscope such as a SEM. The annotation is used as the ground truth in machine learning.

[0042] In one example, a set of small region images (training images) is classified into one or more images that depict striations of n cycles or more, and one or more images that do not depict striations of n cycles or more. The threshold n is an integer of 2 or greater. In this disclosure, small region images that are annotated to indicate the presence of striations and depict striations of n cycles or more are called "actual striation images." Also, small region images that are annotated to indicate the absence of striations and do not depict striations of n cycles or more are called "actual non-striation images." The data extension unit 21 acquires one or more actual striation images and one or more actual non-striation images as multiple small region images.

[0043] Figure 6 shows an example of an actual striation image. In this figure, the same small-region image 220 is shown side by side, and in the small-region image 220 on the right, several striations are highlighted with dashed lines. As highlighted by the dashed lines, the small-region image 220 shown in Figure 6 captures at least 6 cycles of striations. For the small-region image 220, it can be said that multiple striations exist across almost the entire small region. In one example, the threshold n is set so that it can be determined whether or not striations exist over a wide area of ​​the small region. For example, the threshold n is set to 4. The small-region image 220 shown in Figure 6 is annotated to indicate the presence of striations.

[0044] The actual set of non-striation images may include images that do not depict striations, such as brittle fractures or images depicting brittle fractures, or images depicting striations of less than n cycles.

[0045] Returning to Figure 5, in step S12, the model generation unit 21a generates an image generation model by performing machine learning using one or more actual striation images. In one example, the model generation unit 21a generates an image generation model based on a pre-prepared generative adversarial network (GAN) model structure. A GAN is a technique that improves data generation capabilities by having a generator that generates new data compete with a discriminator that identifies whether the generated data is real or fake. The model generation unit 21a trains the generator by inputting one or more actual striation images into its model structure. Through this machine learning, an image generation model that generates pseudo-striation images is obtained. The model generation unit 21a may also generate an image generation model using a method other than GAN, such as a variational autoencoder (VAE) or a diffusion model. The machine learning performed by the model generation unit 21a is different from the machine learning performed by the first learning unit 11 and the machine learning performed by the second learning unit 23.

[0046] In step S13, the pseudo-image generation unit 21b generates one or more pseudo-striation images using the generated image generation model and adds these one or more pseudo-striation images to the second training data as training images. The pseudo-image generation unit 21b causes the image generation model to generate one or more pseudo-striation images. The pseudo-image generation unit 21b adds annotations to each of the one or more pseudo-striation images to indicate the presence of striations and generates data records that show the combination of the pseudo-striation image and the annotation. The pseudo-image generation unit 21b adds the generated one or more data records to the second training data. As a result, the number of data records in the second training data increases.

[0047] In step S14, the transformation unit 21c expands the second training data through graphic transformation. For example, the transformation unit 21c performs at least one of various graphic transformations, such as rotation, flipping, scaling, brightness change, affine transformation, and homography transformation, on each of the multiple small region images (multiple original small region images) of the second training data, to generate one or more new small region images as training images from the original small region images. Then, for each of the multiple original small region images, the transformation unit 21c assigns the same annotation as the original small region image to each new small region image generated from the original small region image, and generates one or more data records that show the combination of the new image and the annotation. The transformation unit 21c adds the generated multiple data records to the second training data. As a result, the number of data records in the second training data increases further.

[0048] In step S15, the filtering unit 22 performs filtering on each of the multiple small region images (training images) of the second training data to emphasize a predetermined frequency range for each small region image. For example, the filtering unit 22 removes high frequencies (noise) and emphasizes low frequencies for each small region image through its filtering. In one example, the filtering unit 22 performs this filtering using a Gaussian filter. Using a Gaussian filter can reduce the false positive rate. In one example, reducing the false positive rate is effective from the standpoint of efficiently estimating the cause of fracture of the fracture surface. Also, since high frequencies are removed by a single linear transformation in a Gaussian filter, the processing load on the processor 101 can be reduced. In other words, a Gaussian filter is advantageous in that it can improve processing performance. As another example, the filtering unit 22 may perform this filtering using one of the following filters: a bilateral filter, unsharp masking, guided filter, anisotropic diffusion filter, non-local mean filter, morphological filter, Laplacian pyramid, and wavelet transform-based filter.

[0049] In step S16, the second learning unit 23 generates a second trained model 42 by performing machine learning using the second training data processed as described above. The machine learning model for generating the second trained model 42 is a computational model that receives an image (second image) of a portion of the fracture surface of an article (second fracture surface of the second article) and determines whether or not striations exist on that portion. The second image is an image with a predetermined frequency range emphasized. That is, the second learning unit 23 receives an image (second image) of a portion of the captured second fracture surface with a predetermined frequency range emphasized and generates the second trained model 42 by machine learning that determines whether or not striations exist on that portion.

[0050] In one example, the second learning unit 23 uses a pre-prepared neural network model as a machine learning model to generate the second trained model 42. The second learning unit 23 inputs a small region image (training image) indicated by the data record of the second training data into its machine learning model. The second learning unit 23 then obtains a determination result indicating whether or not striations exist, which is estimated by the machine learning model. That is, the second learning unit 23 inputs a small region image into the machine learning model and performs a determination process to determine whether or not striations exist on the small region. The second learning unit 23 updates the parameter set in the machine learning model by backpropagation based on the error between the obtained determination result and the annotation (ground truth) indicated by the data record. The second learning unit 23 repeats this process using multiple data records of the second training data to generate the second trained model 42.

[0051] In one example, the second pre-trained model 42 is generated to output the result of determining whether or not striations exist on a small region as the probability that striations may exist. For example, the second pre-trained model 42 outputs a vector {Pa,Pb} whose components are the probability Pa that striations may exist and the probability Pb ​​that striations do not exist. Both probabilities Pa and Pb are represented by continuous values ​​between 0 and 1. For example, if {Pa,Pb} = {0.88,0.12}, it can be determined that there is a high probability that striations exist on the small region.

[0052] The processed second training data includes one or more actual striation images and one or more actual non-striation images. Therefore, in one example, the second learning unit 23 acquires these images as multiple sub-region images and performs machine learning using these sub-region images to generate a second trained model 42 from a neural network model.

[0053] The processed second training data includes one or more pseudo-striation images. Therefore, in one example, the second learning unit 23 performs machine learning using multiple small-region images that further include one or more pseudo-striation images to generate a second trained model 42.

[0054] Each subregion image of the processed second training data is an image with a predetermined frequency range emphasized. Therefore, in one example, the second learning unit 23 inputs the subregion image with the predetermined frequency range emphasized into a machine learning model to determine whether or not striations exist on the subregion. The machine learning model receives the subregion image with the predetermined frequency range emphasized and determines whether or not striations exist on the subregion.

[0055] In one example, the second learning unit 23 may generate a second trained model 42 using cross-validation, a method for evaluating the performance of a machine learning model. In this case, the second learning unit 23 divides the second training data into multiple groups, selects one of these groups as validation data, and selects the remaining groups as training data in the narrow sense. Each of the divided groups is also called a "fold." A combination of validation data and training data in the narrow sense is also called a "split." The second learning unit 23 performs the division process such that the ratio of the number of images between striation images and non-striation images is the same or approximately the same among the multiple groups (folds), and the ratio of the number of images between multiple types of non-striation images (images of brittle fracture, images of ductile fracture, and other images) is also the same or approximately the same.

[0056] The second learning unit 23 generates a provisional second trained model by performing machine learning using the training data in the narrow sense, and evaluates the provisional second trained model using the validation data. The second learning unit 23 performs the generation and evaluation of the provisional second trained model while changing the group (fold) used as the validation data. The second learning unit 23 calculates evaluation metrics for the provisional second trained model for each split and obtains statistical values ​​of multiple evaluation metrics obtained from multiple splits. The second learning unit 23 obtains a machine learning model with hyperparameters adjusted based on these statistical values, and performs machine learning on that machine learning model using the entire second training data to generate a second trained model 42.

[0057] The second learning unit 23 stores the generated second trained model 42 in a predetermined memory unit. This second trained model 42 is used by the second estimation unit 35 (determination unit 35a). The second trained model 42 can also be described as a trained model that estimates whether or not striations exist, i.e., a striation determination model.

[0058] (Estimated cause of destruction) The operation of the estimation system 30 will be explained with reference to Figure 7. Figure 7 is a flowchart showing an example of the operation of the estimation system 30 as the processing flow S2.

[0059] In step S21, the first image segmentation unit 31 acquires an observation image that captures at least a portion of the target fracture surface as the observation area. The observation image is obtained by photographing the fracture surface with an electron microscope such as a scanning electron microscope (SEM). The first image segmentation unit 31 may read an observation image from a predetermined storage unit based on user instructions, or it may receive an observation image transmitted from an electron microscope or another computer.

[0060] In step S22, the first image division unit 31 performs a division process on the observed image to obtain multiple target images from the observed image. In one example, the first image division unit 31 analyzes the observed image to identify the target fracture surface as an observation region, and divides the observation region into a grid to obtain multiple target images. Each target image shows a part of the target fracture surface as a target region. Each target region does not overlap with any adjacent target regions.

[0061] In step S23, the first estimation unit 32 selects one of the multiple target images.

[0062] In step S24, the first estimation unit 32 inputs the selected target image into the first trained model 41 to estimate the cause of fracture of the target fracture surface (target region). Step S24 corresponds to the first stage of estimation.

[0063] As shown in step S25, the subsequent processing changes depending on the estimated cause of failure. If the cause of failure is estimated to be "potential fatigue failure," the process proceeds to step S26. On the other hand, if the cause of failure is estimated to be a type of failure other than "potential fatigue failure," that is, if the cause of failure is estimated to be brittle failure or ductile failure, the process skips step S26 and proceeds to step S27. As another example, if the cause of failure is estimated to include both "potential fatigue failure" and a type of failure other than "potential fatigue failure," the process proceeds to step S26. That is, if the cause of failure is estimated to include "potential fatigue failure," the process proceeds to step S26.

[0064] In step S26, the second image segmentation unit 33, the filtering unit 34, and the second estimation unit 35 (determination unit 35a) work together to estimate whether the cause of failure in the selected target image is fatigue failure. Step S26 corresponds to the second stage of estimation. This estimation process (step S26) will be explained in detail with reference to Figure 8. Figure 8 is a flowchart showing the details of the second stage of estimation.

[0065] In step S261, the second image division unit 33 generates multiple sub-region images from the selected target image. The second image division unit 33 performs region division, dividing the subject area (i.e., the target region) of the target image into multiple sub-regions, and generates multiple sub-region images corresponding to these sub-regions. In one example, the second image division unit 33 performs region division such that each of the multiple sub-regions partially overlaps with at least one adjacent sub-region. Figure 2 shows such region division. In another example, the second image division unit 33 divides the target image in a grid pattern to generate multiple sub-region images. In this case, each of the multiple sub-regions does not overlap with any adjacent sub-regions.

[0066] In step S262, the filtering unit 34 performs filtering on each of the multiple small region images to emphasize a predetermined frequency range for each small region image. For example, the filtering unit 34 removes high frequencies (noise) and emphasizes low frequencies for each small region image by using the same type of filter (e.g., a Gaussian filter) as the filtering unit 22.

[0067] In step S263, the determination unit 35a selects one from a plurality of small region images.

[0068] In step S264, the determination unit 35a inputs the selected small region image to the second trained model 42 to determine whether or not striations exist on the small region indicated by the small region image. That is, the determination unit 35a performs a determination process. As described above, the second trained model 42 may express the determination result of whether or not striations exist on the small region as the probability Pa that striations may exist. The determination unit 35a may obtain the probability Pa, which is represented by a continuous value between 0 and 1, as the determination result. Alternatively, the determination unit 35a may determine that there is a high possibility of striations existing if the probability Pa is greater than 0.5 (i.e., consider Pa=1), and determine that there is a low possibility of striations existing if the probability is 0.5 or less (i.e., consider Pa=0).

[0069] As shown in step S265, the determination unit 35a performs a determination process for each of the multiple sub-region images. If there are unprocessed sub-region images (NO in step S265), the process returns to step S263. The determination unit 35a selects the next sub-region image (step S263) and performs a determination process for that sub-region image (step S264). If all sub-region images have been processed (YES in step S265), the process proceeds to step S266.

[0070] As shown in steps S262 to S265, the determination unit 35a inputs each of the multiple small region images generated from the target image and with a predetermined frequency range emphasized into the second trained model 42, and performs a determination process for each of the multiple small regions.

[0071] In step S266, the second estimation unit 35 generates striation information indicating the location of striations within the captured area of ​​the target image (i.e., the target area) based on the determination process for each of the multiple sub-regions. Striation information indicates where striations exist or are likely to exist within the target area. In one example, the second estimation unit 35 generates the distribution of the probability Pa within the target area as striation information, based on the probability Pa of possible presence of striations in each of the multiple sub-regions. As shown in Figure 2, if the second image division unit 33 performs region division such that each sub-region partially overlaps with at least one adjacent sub-region, there are parts of the target area where the probability Pa is calculated multiple times. The second estimation unit 35 may calculate a statistical value (e.g., the mean) of the multiple Pa values ​​calculated in that part as the final probability for that part. The second estimation unit 35 can represent the final probability as a continuous value within a predetermined numerical range (e.g., from 0 to 1).

[0072] In step S267, the second estimation unit 35 estimates, based on the striation information, whether or not the cause of fracture of the target fracture surface (target region) is fatigue fracture. For example, the second estimation unit 35 estimates that the cause of fracture of the target fracture surface is fatigue fracture if the striation information satisfies predetermined criteria. For example, the second estimation unit 35 estimates that the cause of fracture of the target fracture surface is fatigue fracture if the distribution of the probability that striations may exist satisfies predetermined criteria. The second estimation unit 35 may also estimate that the cause of fracture of the target fracture surface is fatigue fracture if the proportion of the area in the target region where the probability that striations may exist exceeds a threshold Tp (Tp≧0.5) is greater than or equal to a threshold Tr, and estimate that the cause of fracture of the target fracture surface is not fatigue fracture if the proportion of such area is less than the threshold Tr.

[0073] Return to Figure 7. As shown in step S27, the estimation system 30 estimates the cause of fracture for all target images. If there are unprocessed target images (NO in step S27), the process returns to step S23. The first estimation unit 32 selects the next target image (step S23) and inputs that target image into the first trained model 41 to estimate the cause of fracture of the target fracture surface (next target region) (step S24). If the cause of fracture is estimated to be "potential fatigue fracture", the second image segmentation unit 33, the filtering unit 34, and the second estimation unit 35 (determination unit 35a) work together to estimate whether the cause of fracture for the next target image is fatigue fracture (step S26). If all target images have been processed (YES in step S27), the process proceeds to step S28.

[0074] As shown in steps S23 to S27, the estimation system 30 inputs each of the multiple target images into the first trained model 41 and estimates the cause of fracture of the target fracture surface (target region) for each of the multiple target images. If the cause of fracture is estimated to be latent fatigue fracture, the estimation system 30 performs a determination process using the second trained model 42 to estimate whether or not the cause of fracture of the target fracture surface (target region) is fatigue fracture.

[0075] In step S28, the result output unit 36 ​​outputs the estimation results. The result output unit 36 ​​may output at least one of the estimation results for each of the multiple target images and the estimation results for the entire observed image. The result output unit 36 ​​may output the various estimation results in various representation formats, for example, by at least one of the following methods: text, images (computer graphics), and sound.

[0076] In one example, the result output unit 36 ​​generates a heatmap as an estimation result for each of the one or more target images in which the cause of failure is estimated to be potential fatigue failure, representing the distribution of probabilities that striations may exist in the target region shown in the target image. The result output unit 36 ​​generates a heatmap as an estimation result for each of the one or more target images in which the distribution of probabilities that striations may exist in the target region shown in the target image. This heatmap expresses the magnitude of the probability by the type of color, density, or transmittance. The result output unit 36 ​​may set the color of the heatmap for each of the multiple sub-regions generated from the target region based on the probability in that sub-region, and generate a heatmap based on the color in each of the multiple sub-regions. The color setting may be at least one of the type of color, density, and transmittance. If the second estimation unit 35 represents the probability by a continuous value in a predetermined numerical range (for example, a range from 0 to 1), the result output unit 36 ​​generates a heatmap that represents the distribution of the continuous value by a continuous change in color. The continuous color change may not only be a change in pigment, but may also be a continuous change in, for example, color density or transmittance. The result output unit 36 ​​may generate a heat map as an estimation result for each of the one or more target images in which the cause of failure is estimated to be potential fatigue failure, representing the distribution of probability vectors {Pa, Pb} in the target region shown in the target image.

[0077] In one example, the result output unit 36 ​​may generate a result image representing the striation distribution by superimposing a heatmap onto each of the one or more target images whose failure cause is estimated to be potential fatigue failure. Figure 9 shows an example of such a result image. In this example, the heatmap is schematically represented by shading. The result image 230 shown in this example is obtained by superimposing a heatmap 240 onto a single target image 210.

[0078] Figure 10 shows an example of a result image corresponding to an observed image. In this example, the result output unit 36 ​​superimposes a representation for distinguishing the fracture cause of each target region onto the observed image 200 to generate a result image 250. The result image 250 shows that the fracture cause in the upper part 91 of the target fracture surface 90 is estimated to be fatigue fracture, and the fracture cause in the lower part 92 of the target fracture surface 90 is estimated to be brittle fracture.

[0079] The result output unit 36 ​​may display the estimation results on a display device, transmit them to another computer such as a user terminal, or store them in a predetermined storage device. The user can then refer to the estimation results to efficiently perform various tasks such as further analysis of the fracture surface, investigation of the cause of fracture, and preparation of a fracture report.

[0080] [Differentiation] The technology relating to this disclosure has been described in detail above based on various examples. However, this disclosure is not limited to the examples given above. The technology relating to this disclosure can be modified in various ways without departing from its essence.

[0081] The fracture surface evaluation system described herein may be connected to an electron microscope such as a scanning electron microscope (SEM) to acquire various images (electron images) from the electron microscope. Alternatively, the fracture surface evaluation system may be incorporated into an electron microscope system.

[0082] In the example above, the fracture surface evaluation system 1 comprises a first learning system 10 and a second learning system 20 that perform the learning phase, and an estimation system 30 that performs the operation phase. However, the fracture surface evaluation system may be configured to perform either the learning phase or the operation phase, or both. The trained model generated by the fracture surface evaluation system performing the learning phase can be ported to other computers. The fracture surface evaluation system performing the operation phase may incorporate a trained model generated by other computers.

[0083] The various configurations of the fracture surface evaluation system will be explained with reference to Figures 11 to 13. Figures 11 to 13 all show the functional configuration of the fracture surface evaluation system according to modified examples.

[0084] The fracture surface evaluation system 1A shown in Figure 11 corresponds to the second learning system 20 described above, which generates the second trained model 42. Similar to the fracture surface evaluation system 1, the fracture surface evaluation system 1A also (A) acquires a small region image that captures a small area that is a part of the target fracture surface, which is the fracture surface of the target article, and (B) receives an image that captures a part of the fracture surface of the article. The small region image is input to a machine learning model that determines whether or not striations exist on the part of the fracture surface, and the system performs a determination process to determine whether or not striations exist on the small region. As shown in this modified example, the fracture surface evaluation system that executes the learning phase may not have the functionality corresponding to the first learning system 10, but may have the functionality corresponding to the second learning system 20.

[0085] The fracture surface evaluation system 1B shown in Figure 12 comprises the same second image segmentation unit 33, filtering unit 34, second estimation unit 35, and result output unit 36 ​​as the estimation system 30 described above. Similar to the estimation system 30, the second estimation unit 35 includes a determination unit 35a that performs determination processing using a second trained model 42. In this modified example, the second image segmentation unit 33 acquires a target image from an external source. The second image segmentation unit 33 may read a target image from a predetermined storage unit based on user instructions, or it may receive a target image transmitted from an electron microscope or another computer. Similar to fracture surface evaluation system 1, fracture surface evaluation system 1B also (A) acquires a target image that captures at least a portion of the target fracture surface, which is the fracture surface of the target article; (B) performs region segmentation to divide the captured area of ​​the target image into multiple sub-regions, and generates multiple sub-region images corresponding to the multiple sub-regions; (C) receives an image that captures a portion of the fracture surface of the article and inputs each of the multiple sub-region images into a trained model that determines whether or not striations exist in the portion of the area, and performs a determination process for each of the multiple sub-regions to determine whether or not striations exist in the sub-region; and (D) generates striation information indicating the location of striations in the captured area based on the determination process for each of the multiple sub-regions.

[0086] The fracture surface evaluation system 1C shown in Figure 13 comprises an image acquisition unit 39, a filtering unit 34, a second estimation unit 35, and a result output unit 36, the same as those in the estimation system 30 described above. The second estimation unit 35 includes a determination unit 35a that performs a determination process using a second trained model 42. The image acquisition unit 39 is a functional module that acquires small-region images from an external source. The image acquisition unit 39 may read small-region images from a predetermined storage unit based on user instructions, or it may receive small-region images transmitted from an electron microscope or another computer. Similar to the fracture surface evaluation system 1, the fracture surface evaluation system 1C also (A) acquires a small-region image that shows a small region which is a part of the target fracture surface of the target article, and (B) receives an image that shows a part of the fracture surface of the article. The small-region image is input to a machine learning model that determines whether or not striations exist on the part of the fracture surface, and the system performs a determination process that determines whether or not striations exist on the small region.

[0087] As with fracture surface evaluation systems 1B and 1C, the fracture surface evaluation system according to this disclosure may perform the second stage of estimation in fracture surface evaluation system 1 without performing the first stage of estimation in fracture surface evaluation system 1.

[0088] The fracture surface evaluation system does not need to have at least some of the functions for extending the training data. For example, the fracture surface evaluation system does not need to have a function for generating pseudo-striation images (model generation unit 21a and pseudo-image generation unit 21b), nor does it need to have a function for extending the training data by geometric transformation (transformation unit 21c).

[0089] The fracture surface evaluation system may acquire a small-region image that captures a specific part of the target image's coverage area, input this single small-region image into a second trained model to perform a determination process, and if it determines that striations exist on the small-region, it may estimate that the cause of fracture of the target fracture surface is fatigue fracture. In this example, the fracture surface evaluation system does not need to generate striation information.

[0090] The fracture surface evaluation system does not need to have a function to perform filtering on small area images. The fracture surface evaluation system performing the learning phase does not need to have a function equivalent to the filtering unit 22. The fracture surface evaluation system performing the operation phase does not need to have a function equivalent to the filtering unit 34.

[0091] In the above example, brittle fracture and ductile fracture were listed as types of fracture other than fatigue fracture, but the fracture surface evaluation system may estimate other types of fracture as causes of fracture, in addition to or instead of these two types.

[0092] In this disclosure, the expression "at least one processor executes the first process, the second process, ... the nth process," or a corresponding expression, is a concept that includes cases where the entity executing the n processes from the first process to the nth process changes midway through. In other words, this expression is a concept that includes both cases where all n processes are executed by the same processor and cases where the processor changes at an arbitrary rate for the n processes.

[0093] The processing steps for a method executed by at least one processor are not limited to the examples above. For example, some of the steps described above may be omitted, or each step may be performed in a different order. Also, any two or more of the steps described above may be combined, or some of the steps may be modified or deleted. Alternatively, other steps may be performed in addition to each of the steps described above.

[0094] When comparing the relative magnitudes of two numbers in a computer system or within a computer, either the two criteria "greater than or equal to" and "greater than" may be used, or either the two criteria "less than or equal to" and "less than" may be used.

[0095] [First Addendum] As can be seen from the various examples above, this disclosure includes the following aspects: (Note A1) Equipped with at least one processor, The at least one processor, A small region image is obtained that captures a small area that is a part of the fracture surface of the target object. The machine learning model receives an image of a portion of the fracture surface of an article and determines whether or not striations exist in the portion of the article. The machine learning model receives the small region image and performs a determination process to determine whether or not striations exist in the small region. Fracture surface evaluation system. (Appendix A2) The machine learning model is a neural network model used to generate a trained model by machine learning that receives an image of a portion of a region and determines whether or not striations exist in that portion of the region. Fracture surface evaluation system as described in Appendix A1. (Note A3) The at least one processor, Multiple small region images are obtained, each containing one or more actual striation images that are annotated to indicate the presence of striations and depict striations of n cycles or more, and one or more actual non-striation images that are annotated to indicate the absence of striations and do not depict the aforementioned striations of n cycles or more. The machine learning described above is performed using the plurality of small region images to generate the trained model from the neural network model. Fracture surface evaluation system as described in Appendix A2. (Note A4) The at least one processor, Generate one or more pseudo-striation images, The machine learning is performed using the plurality of small region images, which further include the one or more pseudo-striation images, to generate the trained model. Fracture surface evaluation system as described in Appendix A3. (Note A5) The at least one processor, An image generation model is generated by another machine learning method using the one or more actual striation images mentioned above. The image generation model generates one or more pseudo-striation images. Fracture surface evaluation system as described in Appendix A4. (Note A6) The machine learning model is a trained model that receives the image showing a portion of the region and determines whether or not the striations exist in that portion of the region. Fracture surface evaluation system as described in Appendix A1. (Note A7) A fracture surface evaluation method performed by a fracture surface evaluation system comprising at least one processor, The steps include: obtaining a small region image that captures a small area that is part of the fracture surface of the target item; The process involves receiving an image of a portion of the fracture surface of an article and inputting the small region image into a machine learning model that determines whether or not striations exist in the portion of the fracture surface, and then performing a determination process to determine whether or not striations exist in the small region. A fracture surface evaluation method including the following. (Note A8) The steps include: obtaining a small region image that captures a small area that is part of the fracture surface of the target item; The process involves receiving an image of a portion of the fracture surface of an article and inputting the small region image into a machine learning model that determines whether or not striations exist in the portion of the fracture surface, and then performing a determination process to determine whether or not striations exist in the small region. A fracture surface evaluation program that is executed by a computer.

[0096] According to appendices A1, A7, and A8, the presence or absence of striations on a small region that is part of the target fracture surface can be determined by inputting a small region image of that region into a machine learning model. By introducing a machine learning model in this way, striations can be easily searched for.

[0097] According to Appendix A2, by using a neural network model, it is possible to generate a trained model that determines striations from images based on criteria similar to human experience, sensibilities, or knowledge.

[0098] According to Appendix A3, machine learning is performed using actual striation images and actual non-striation images classified by focusing on the number of striation cycles, thus enabling the generation of a trained model that can more accurately determine striations.

[0099] According to Appendix A4, since pseudo-striation images are generated, the number of striation images, which would otherwise be time-consuming and labor-intensive to collect, can be easily increased. Furthermore, since the training data used for machine learning is increased by these pseudo-images, it is possible to generate a trained model with higher estimation accuracy.

[0100] According to Appendix A5, an image generation model is generated by another machine learning process using one or more actual striation images, and this image generation model generates pseudo-striation images. By dynamically generating image generation models in this way, it is possible to increase the number of pseudo-striation images that resemble the collected actual striation images.

[0101] According to Appendix 6, the determination of whether or not striations exist on a small region that is part of the target fracture surface can be obtained by inputting a small region image that captures the small region into a trained model. By introducing a trained model in this way, striations can be easily searched for in images where it is unknown whether or not striations exist.

[0102] [Second Addendum] As can be seen from the various examples above, this disclosure also includes the following aspects: (Note B1) Equipped with at least one processor, The at least one processor, A target image is obtained that captures at least a portion of the target fracture surface, which is the fracture surface of the target item. A first image of a first fracture surface of a first article is received and the target image is input to a first trained model that estimates which of a predetermined plurality of fractures caused the fracture of the first fracture surface, thereby estimating the cause of fracture of the target fracture surface, where the predetermined plurality of fractures include potential fatigue fracture and one or more other types of fracture. When it is estimated that the cause of fracture of the target fracture surface is the potential fatigue fracture, a small-region image is obtained that captures a small area which is part of the area to be captured in the target image. The system receives a second image showing a portion of the second fracture surface of the second article and inputs the small region image to a second trained model that determines whether or not striations exist in the portion of the area, and performs a determination process to determine whether or not striations exist in the small region. If it is determined that striations exist on the aforementioned small region, it is presumed that the cause of fracture of the target fracture surface is fatigue fracture. Fracture surface evaluation system. (Note B2) The at least one processor, Region division is performed to divide the subject area into a plurality of sub-regions, and a plurality of sub-region images corresponding to the plurality of sub-regions are generated from the target image. Each of the multiple sub-region images is input to the second trained model, and the determination process is executed for each of the multiple sub-regions. Based on the determination process for each of the plurality of sub-regions, striation information indicating the position of the striations within the subject area is generated. If the striation information meets predetermined criteria, it is presumed that the cause of fracture of the target fracture surface is fatigue fracture. Fracture surface evaluation system as described in Appendix B1. (Note B3) The at least one processor performs the region partitioning such that each of the multiple subregions partially overlaps with at least one adjacent subregion. Fracture surface evaluation system as described in Appendix B2. (Note B4) The second trained model outputs the result of determining whether or not the striation exists as the probability that the striation may exist. The at least one processor generates the distribution of the probabilities in the subject area as striation information based on the probabilities in each of the plurality of sub-regions, If the distribution of the aforementioned probabilities satisfies the predetermined criteria, it is estimated that the cause of failure of the target fracture surface is fatigue failure. Fracture surface evaluation system as described in Appendix B2 or B3. (Note B5) The at least one processor, An observation image is obtained that captures at least a portion of the target fracture surface as the observation area. A segmentation process is performed on the observation image to obtain multiple target images from the observation image. Each of the above-mentioned multiple target images is input into the first trained model to estimate the cause of fracture of the target fracture surface for each of the above-mentioned multiple target images. The fracture surface evaluation system described in any one of the appendices B1 to B4. (Note B6) A fracture surface evaluation method performed by a fracture surface evaluation system comprising at least one processor, A step of obtaining a target image that shows at least a portion of the target fracture surface, which is the fracture surface of the target item, A step of receiving a first image of a first fracture surface of a first article and inputting the target image into a first trained model that estimates which of a predetermined plurality of types of fracture the cause of the fracture of the first fracture surface is, wherein the predetermined plurality of types of fracture include potential fatigue fracture and one or more other types of fracture, When it is estimated that the cause of fracture of the target fracture surface is the potential fatigue fracture, the step is to acquire a small-region image that captures a small region which is part of the area to be captured in the target image, The steps include: receiving a second image showing a portion of the second fracture surface of a second article and inputting the small region image to a second trained model that determines whether or not striations exist on the portion of the model, and performing a determination process to determine whether or not striations exist on the small region; If it is determined that striations exist on the small region, the step of estimating that the cause of fracture of the target fracture surface is fatigue fracture, A fracture surface evaluation method including the following. (Note B7) A step of obtaining a target image that shows at least a portion of the target fracture surface, which is the fracture surface of the target item, A step of receiving a first image of a first fracture surface of a first article and inputting the target image into a first trained model that estimates which of a predetermined plurality of types of fracture the cause of the fracture of the first fracture surface is, wherein the predetermined plurality of types of fracture include potential fatigue fracture and one or more other types of fracture, When it is estimated that the cause of fracture of the target fracture surface is the potential fatigue fracture, the step is to acquire a small-region image that captures a small region which is part of the area to be captured in the target image, The steps include: receiving a second image showing a portion of the second fracture surface of a second article and inputting the small region image to a second trained model that determines whether or not striations exist on the portion of the model, and performing a determination process to determine whether or not striations exist on the small region; If it is determined that striations exist on the small region, the step of estimating that the cause of fracture of the target fracture surface is fatigue fracture, A fracture surface evaluation program that is executed by a computer.

[0103] According to appendices B1, B6, and B7, first, the target image and the first trained model are used to estimate which of several types of fracture, including potential fatigue fracture (the possibility of fatigue fracture), the cause of fracture on the target fracture surface falls under. If the cause of fracture is estimated to be potential fatigue fracture, a small-region image showing a part (small region) of the target image and the second trained model are used to determine whether or not striations exist on the small region. If striations exist, the cause of fracture on the target fracture surface is determined to be fatigue fracture. In this way, fatigue fracture can be efficiently estimated by estimating it through a two-stage process that includes striation determination.

[0104] According to Appendix B2, the subject area of ​​the target image is divided into multiple sub-regions, the presence or absence of striations is determined for each sub-region, and striation information indicating the location of striations within the subject area is generated. By utilizing the striation information shown for the entire subject area, it becomes possible to improve the accuracy of fatigue fracture estimation.

[0105] According to Appendix B3, since each sub-region is generated with partial overlap, at least a portion of the subject area of ​​the target image is judged at least twice for the presence or absence of striations. This method allows for the accurate identification of areas with a high probability of striations being present. As a result, it becomes possible to further improve the accuracy of fatigue fracture estimations.

[0106] According to Appendix B4, a distribution of probabilities indicating the presence of striations within the captured area of ​​the target image can be obtained, thus providing more detailed information about striations in that area. By using this detailed information, it becomes possible to further improve the accuracy of fatigue fracture estimations.

[0107] According to Appendix B5, the observed image is divided into multiple target images, and the cause of fracture is estimated for each individual target image. This mechanism allows for detailed estimation of the cause of fracture that occurred at a single fracture surface, even when multiple types of fracture causes are involved in that fracture surface.

[0108] [Third Addendum] As can be seen from the various examples above, this disclosure also includes the following aspects: (Note C1) Equipped with at least one processor, The at least one processor, A target image is obtained that captures at least a portion of the target fracture surface, which is the fracture surface of the target item. The area of ​​the target image is divided into multiple sub-regions by performing region segmentation, and multiple sub-region images corresponding to the multiple sub-regions are generated. The system receives an image of a portion of the fracture surface of an article and uses a trained model to determine whether or not striations exist in that portion of the article. Each of the multiple small region images is input to this trained model, and a determination process is performed for each of the multiple small regions to determine whether or not striations exist in that region. Based on the determination process for each of the plurality of sub-regions, striation information indicating the position of the striations within the subject area is generated. Fracture surface evaluation system. (Note C2) The at least one processor performs the region partitioning such that each of the multiple subregions partially overlaps with at least one adjacent subregion. Fracture surface evaluation system as described in Appendix C1. (Note C3) The trained model outputs the result of determining whether or not the striation exists as the probability that the striation may exist. The at least one processor generates the distribution of probabilities in the subject area as striation information based on the probabilities in each of the plurality of sub-regions. Fracture surface evaluation system as described in Appendix C1 or C2. (Note C4) The at least one processor generates a heatmap representing the distribution of the probabilities. Fracture surface evaluation system as described in Appendix C3. (Note C5) The at least one processor, The probability is represented by a continuous value within a predetermined numerical range. The heatmap is generated, which represents the distribution of the continuous values ​​by a continuous change in color. Fracture surface evaluation system as described in Appendix C4. (Appendix C6) The at least one processor generates a result image by superimposing the heatmap onto the target image. Fracture surface evaluation system as described in Appendix C4 or C5. (Note C7) The at least one processor, For each of the aforementioned sub-regions, the color density or transmittance of the heatmap in that sub-region is set based on the probability in that sub-region. The heat map is generated based on the density or transmittance of the color in each of the plurality of subregions. The fracture surface evaluation system described in any one of the appendices C4 to C6. (Note C8) A fracture surface evaluation method performed by a fracture surface evaluation system comprising at least one processor, A step of obtaining a target image that shows at least a portion of the target fracture surface, which is the fracture surface of the target item, The steps include: performing region segmentation to divide the subject area of ​​the target image into multiple sub-regions, and generating multiple sub-region images corresponding to the multiple sub-regions; The steps include: receiving an image of a portion of the fracture surface of an article and inputting each of the multiple small region images into a trained model that determines whether or not striations exist in the portion of the fracture surface, and performing a determination process for each of the multiple small regions to determine whether or not striations exist in the small region; A step of generating striation information indicating the position of the striations within the subject area based on the determination process for each of the plurality of sub-regions, A fracture surface evaluation method including the following. (Note C9) A step of obtaining a target image that shows at least a portion of the target fracture surface, which is the fracture surface of the target item, The steps include: performing region segmentation to divide the subject area of ​​the target image into multiple sub-regions, and generating multiple sub-region images corresponding to the multiple sub-regions; The steps include: receiving an image of a portion of the fracture surface of an article and inputting each of the multiple small region images into a trained model that determines whether or not striations exist in the portion of the fracture surface, and performing a determination process for each of the multiple small regions to determine whether or not striations exist in the small region; A step of generating striation information indicating the position of the striations within the subject area based on the determination process for each of the plurality of sub-regions, A fracture surface evaluation program that is executed by a computer.

[0109] According to appendices C1, C8, and C9, the subject area of ​​the target image is divided into multiple sub-regions, the presence or absence of striations is determined for each sub-region, and striation information indicating the location of striations within the subject area is generated. By referring to this striation information, striations can be easily searched across the subject area.

[0110] According to Appendix C2, since each sub-region is generated in a partially overlapping manner, at least a portion of the subject area of ​​the target image is judged at least twice for the presence or absence of striations. This method allows for the accurate identification of areas with a high probability of striations being present. As a result, the location of striations can be identified with greater accuracy.

[0111] According to Appendix C3, a distribution of probabilities indicating the presence of striations within the subject area of ​​the target image can be obtained, thus providing more detailed information about striations within that area. By using this detailed information, the location of striations can be identified with greater accuracy.

[0112] According to Appendix C4, the distribution of probabilities of the existence of striations is represented by a heatmap, making it easy to visually understand the location of striations to the user.

[0113] According to Appendix C5, a heatmap is generated that represents the probability distribution expressed as a continuous value using continuous color changes, thus allowing for a detailed and easy-to-understand representation of the probability of striations existing.

[0114] According to Appendix C6, the resulting image, created by overlaying a heatmap onto the target image, makes it easy to visually show the user where striations are located on the fracture surface.

[0115] According to Appendix C7, the color density or transmittance of the heatmap is set based on the probability that striations may exist, thus allowing for a detailed representation of the likelihood of striations being present.

[0116] [Fourth Addendum] As can be seen from the various examples above, this disclosure also includes the following aspects: (Note D1) Equipped with at least one processor, The at least one processor, A small region image is obtained that captures a small area that is a part of the fracture surface of the target object. A filter is applied to the small region image to enhance a predetermined frequency range of the small region image. The system receives an image of a portion of the fracture surface of an article and uses a trained model to determine whether or not striations exist in that portion of the article. The trained model then inputs the small region image with the predetermined frequency range emphasized and performs a determination process to determine whether or not striations exist in that small region. Fracture surface evaluation system. (Note D2) The at least one processor performs the filtering process using a Gaussian filter. Fracture surface evaluation system as described in Appendix D1. (Note D3) The trained model receives the image in which the predetermined frequency range of the captured portion of the region is emphasized, and determines whether or not striations exist in the portion of the region, generated by machine learning. Fracture surface evaluation system as described in Appendix D1 or D2. (Note D4) The at least one processor, A target image is obtained that captures at least a portion of the target fracture surface. Region segmentation is performed to divide the subject area of ​​the target image into a plurality of sub-regions, and a plurality of sub-region images corresponding to the plurality of sub-regions are generated from the target image. The filtering process is performed on each of the plurality of small region images to emphasize the predetermined frequency range of each of the plurality of small region images. Each of the multiple small region images with the predetermined frequency range emphasized is input to the trained model, and the determination process is executed for each of the multiple small regions. Based on the determination for each of the plurality of sub-regions, striation information indicating the position of the striations within the subject area is generated. The fracture surface evaluation system described in any one of the appendices D1 to D3. (Note D5) The at least one processor performs the region partitioning such that each of the multiple subregions partially overlaps with at least one adjacent subregion. Fracture surface evaluation system as described in Appendix D4. (Note D6) A fracture surface evaluation method performed by a fracture surface evaluation system comprising at least one processor, The steps include: obtaining a small region image that captures a small area that is part of the fracture surface of the target item; The steps include: performing a filter process on the small region image to emphasize a predetermined frequency range of the small region image; The steps include: receiving an image of a portion of the fracture surface of an article and inputting the small region image with a predetermined frequency range emphasized into a trained model that determines whether or not striations exist in the portion of the article, and performing a determination process to determine whether or not striations exist in the small region; A fracture surface evaluation method including the following. (Note D7) The steps include: obtaining a small region image that captures a small area that is part of the fracture surface of the target item; The steps include: performing a filter process on the small region image to emphasize a predetermined frequency range of the small region image; The steps include: receiving an image of a portion of the fracture surface of an article and inputting the small region image with a predetermined frequency range emphasized into a trained model that determines whether or not striations exist in the portion of the article, and performing a determination process to determine whether or not striations exist in the small region; A fracture surface evaluation program that is executed by a computer.

[0117] According to appendices D1, D6, and D7, a specific frequency range of a small-region image, which is a part of the target fracture surface, is enhanced by filtering, and then this small-region image is input into a trained model. The trained model then determines whether or not striations exist on that small region. By introducing filtering and a machine learning model in this way, striations can be searched for easily and accurately.

[0118] According to Appendix D2, using a Gaussian filter can reduce the false positive rate and the processing load on the processor performing the filtering (i.e., improve processing performance). In other words, it can improve the performance of the striation determination process.

[0119] According to Appendix D3, a trained model is generated by machine learning using images with a specified frequency range emphasized. Since images with a specified frequency range emphasized are used even in the so-called training phase, the judgment process for small-region images to which filtering has been applied can be performed with greater accuracy.

[0120] According to Appendix D4, the subject area of ​​the target image is divided into multiple sub-regions, the presence or absence of striations is determined for each sub-region, and striation information indicating the location of the striations within the subject area is generated. By referring to this striation information, striations can be easily searched across the subject area.

[0121] According to Appendix D5, since each sub-region is generated with partial overlap, at least a portion of the subject area of ​​the target image is judged at least twice for the presence or absence of striations. This method allows for accurate identification of areas with a high probability of striations being present. As a result, the location of striations can be identified with greater accuracy. [Explanation of symbols]

[0122] 1, 1A, 1B, 1C... Fracture surface evaluation system, 2... Learning database, 10... First learning system, 11... First learning unit, 20... Second learning system, 21... Data expansion unit, 21a... Model generation unit, 21b... Pseudo-image generation unit, 21c... Transformation unit, 22... Filtering unit, 23... Second learning unit, 30... Estimation system, 31... First image segmentation unit, 32... First estimation unit, 33... Second image segmentation unit, 34... Filtering unit, 35... Second estimation unit, 35a... Judgment unit, 36... Result output unit, 39... Image acquisition unit, 41... First trained model, 42... Second trained model, 90... Target fracture surface, 110... Fracture surface evaluation program, 200... Observation image, 210... Target image, 220... Small region image, 230... Result image, 240... Heatmap, 250... Result image.

Claims

1. Equipped with at least one processor, The at least one processor, A target image is obtained that captures at least a portion of the target fracture surface, which is the fracture surface of the target item. A first image of a first fracture surface of a first article is received and the target image is input to a first trained model that estimates which of a predetermined plurality of fractures the cause of the fracture of the first fracture surface is, thereby estimating the cause of the fracture of the target fracture surface, where the predetermined plurality of fractures include potential fatigue fracture and one or more other types of fracture. When it is estimated that the cause of fracture of the target fracture surface is the potential fatigue fracture, a small-region image is obtained that captures a small area which is part of the area to be captured in the target image. The second trained model receives a second image showing a portion of the second fracture surface of the second article and determines whether or not striations exist in the portion of the model. The trained model receives the small region image and performs a determination process to determine whether or not striations exist in the small region. If it is determined that striations exist on the aforementioned small region, it is presumed that the cause of fracture of the target fracture surface is fatigue fracture. Fracture surface evaluation system.

2. The at least one processor, Region division is performed to divide the subject area into a plurality of sub-regions, and a plurality of sub-region images corresponding to the plurality of sub-regions are generated from the target image. Each of the above-mentioned sub-region images is input to the second trained model, and the judgment process is executed for each of the above-mentioned sub-regions. Based on the determination process for each of the plurality of sub-regions, striation information indicating the position of the striations within the subject area is generated. If the striation information meets predetermined criteria, it is presumed that the cause of fracture of the target fracture surface is fatigue fracture. The fracture surface evaluation system according to claim 1.

3. The at least one processor performs the region partitioning such that each of the multiple subregions partially overlaps with at least one adjacent subregion. The fracture surface evaluation system according to claim 2.

4. The second trained model outputs the result of determining whether or not the striation exists as the probability that the striation may exist. The at least one processor generates the distribution of the probabilities in the subject area as striation information based on the probabilities in each of the plurality of sub-regions, If the distribution of the aforementioned probabilities satisfies the predetermined criteria, it is estimated that the cause of failure of the target fracture surface is fatigue failure. The fracture surface evaluation system according to claim 2 or 3.

5. The at least one processor, An observation image is obtained that captures at least a portion of the target fracture surface as the observation area. A segmentation process is performed on the observation image to obtain multiple target images from the observation image. Each of the above-mentioned multiple target images is input into the first trained model to estimate the cause of fracture of the target fracture surface for each of the above-mentioned multiple target images. A fracture surface evaluation system according to any one of claims 1 to 3.

6. A fracture surface evaluation method performed by a fracture surface evaluation system comprising at least one processor, A step of obtaining a target image that shows at least a portion of the target fracture surface, which is the fracture surface of the target item, A step of receiving a first image of a first fracture surface of a first article and inputting the target image into a first trained model that estimates which of a predetermined plurality of types of fracture the cause of the fracture of the first fracture surface is, wherein the predetermined plurality of types of fracture include potential fatigue fracture and one or more other types of fracture, When it is estimated that the cause of fracture of the target fracture surface is the potential fatigue fracture, the step is to acquire a small-region image that captures a small region which is part of the area to be captured in the target image, The steps include: receiving a second image showing a portion of the second fracture surface of a second article and inputting the small region image to a second trained model that determines whether or not striations exist on the portion of the model, and performing a determination process to determine whether or not striations exist on the small region; If it is determined that striations exist on the small region, the step of estimating that the cause of fracture of the target fracture surface is fatigue fracture, A fracture surface evaluation method including the following.

7. A step of obtaining a target image that shows at least a portion of the target fracture surface, which is the fracture surface of the target item, A step of receiving a first image of a first fracture surface of a first article and inputting the target image into a first trained model that estimates which of a predetermined plurality of types of fracture the cause of the fracture of the first fracture surface is, wherein the predetermined plurality of types of fracture include potential fatigue fracture and one or more other types of fracture, When it is estimated that the cause of fracture of the target fracture surface is the potential fatigue fracture, the step is to acquire a small-region image that captures a small region which is part of the area to be captured in the target image, The steps include: receiving a second image showing a portion of the second fracture surface of a second article and inputting the small region image to a second trained model that determines whether or not striations exist on the portion of the model, and performing a determination process to determine whether or not striations exist on the small region; If it is determined that striations exist on the small region, the step of estimating that the cause of fracture of the target fracture surface is fatigue fracture, A fracture surface evaluation program that is executed by a computer.

Images