Information processing apparatus, determination method, and recording medium

Abstract

The present application can perform high-precision determination even for images that are likely to be misjudged. An information processing device (1) includes a classification section (105) that acquires an output value by inputting an inspection image to a classification model that is generated by learning in such a manner that the mutual distances of feature amounts extracted from a noise-free image group in a feature space are reduced, and a determination section (102) that determines the presence or absence of a defect by applying a method for a noise-free image group or a method for a noisy image group in accordance with the output value.

Description

Technical Field

[0001] This invention relates to an information processing device for image-based judgment, etc. Background Technology

[0002] Images are widely used to determine various criteria. For example, Patent Document 1 discloses an ultrasonic flaw detection method using a phased array TOFD (Time Of Flight Diffraction) method. In this ultrasonic flaw detection method, a flaw detection image is displayed, which is generated based on the diffraction waves of an ultrasonic beam emitted from a phased array flaw detection element and focused onto a stainless steel weld. This allows for the detection of welding defects generated inside the stainless steel weld.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Publication No. 2014-48169 Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] The technology in Patent Document 1 has the following problem: the human and time costs required for visually confirming flaw inspection images to detect welding defects are high. As a solution to this problem, one approach is to automatically determine the presence or absence of welding defects by analyzing the flaw inspection images using a computer.

[0008] However, sometimes noise with an appearance similar to the echo of a welding defect may appear in the flaw detection image. In the case of automatic judgment, this noise may be misidentified as a welding defect. Such misjudgments are not limited to flaw detection images; they can also occur in any image that may reflect a signal similar in appearance to the object being inspected. Furthermore, when performing object detection on an inspection image, it is difficult to correctly detect the object from such images.

[0009] As described above, various automatic image determination processes suffer from the following problem: when the object image being determined is prone to misdetermination, the determination accuracy decreases. One objective of this invention is to provide an information processing apparatus, etc., capable of performing high-precision determinations even for images prone to misdetermination.

[0010] (II) Technical Solution

[0011] To address the aforementioned problems, an information processing apparatus according to one aspect of the present invention includes: an acquisition unit that acquires output values ​​obtained by inputting an object image into a classification model, the classification model being generated by learning in such a way that when multiple feature quantities extracted from a first image group having common features are embedded into a feature space, the distance between the feature quantities is reduced; and a determination unit that, based on the output values, applies a first method for the first image group or a second method for a second image group consisting of images not belonging to the first image group to determine predetermined determination matters related to the object image.

[0012] In addition, to address the aforementioned problems, one aspect of the present invention provides a determination method executed by an information processing device, comprising: an acquisition step of acquiring an output value obtained by inputting an object image into a classification model, the classification model being generated by learning in such a way that when multiple feature quantities extracted from a first image group having common features are embedded into a feature space, the distance between the feature quantities is reduced; and a determination step of applying a first method for the first image group or a second method for a second image group consisting of images not belonging to the first image group to determine predetermined determination matters related to the object image based on the output value.

[0013] (III) Beneficial Effects

[0014] According to one aspect of the present invention, high-precision determination can be achieved even for images prone to misjudgment. Attached Figure Description

[0015] Figure 1 This is a block diagram illustrating an example of the structure of the main parts of the information processing apparatus according to Embodiment 1 of the present invention.

[0016] Figure 2 This is a diagram showing an overview of an inspection system that includes the aforementioned information processing device.

[0017] Figure 3 This is a diagram showing a summary of the inspection performed using the aforementioned information processing device.

[0018] Figure 4 This diagram illustrates a structural example of the determination unit provided in the aforementioned information processing apparatus, and an example of a determination method that uses the determination unit to determine whether or not a defect exists.

[0019] Figure 5 This is a diagram illustrating an example of embedding features extracted from multiple inspection images using a classification model into a feature space.

[0020] Figure 6 This diagram illustrates an example of an inspection method using the aforementioned information processing device.

[0021] Figure 7 This is a block diagram illustrating an example of the main component structure of the information processing apparatus according to Embodiment 2 of the present invention.

[0022] Figure 8 This diagram illustrates an example of an inspection method using the aforementioned information processing device.

[0023] Figure 9 This is a block diagram illustrating an example of the main component structure of the information processing apparatus according to Embodiment 3 of the present invention.

[0024] Figure 10 This diagram illustrates an example of an inspection method using the aforementioned information processing device.

[0025] Figure 11 This is a block diagram illustrating an example of the structure of the main parts of the information processing apparatus according to Embodiment 4 of the present invention.

[0026] Figure 12 This diagram illustrates an example of an inspection method using the aforementioned information processing device. Detailed Implementation

[0027] (Implementation Method 1)

[0028] (System Overview)

[0029] based on Figure 2 An overview of an inspection system according to an embodiment of the present invention will be described. Figure 2 This is a diagram showing an outline of the inspection system 100. The inspection system 100 is a system that inspects an object for defects based on an image of the object being inspected, and it includes an information processing device 1 and an ultrasonic flaw detector 7.

[0030] In this embodiment, an example of using the inspection system 100 to inspect the tube-end welds of a heat exchanger for defects will be described. The tube-end weld refers to the portion formed by welding together multiple metal tubes constituting the heat exchanger and a metal tube sheet that bundles these tubes. A defect in the tube-end weld refers to a defect that creates voids within the weld. Furthermore, the tubes and tube sheet can be made of non-ferrous metals such as aluminum, or resin. Additionally, the inspection system 100 can also be used to inspect for defects in the welds (root welds) between tube seats and tubes in boiler equipment used in, for example, waste incineration facilities. Of course, the inspection area is not limited to the weld, and the inspection object is not limited to the heat exchanger.

[0031] During the inspection, such as Figure 2As shown, a probe coated with a contact medium is inserted into the pipe end. Ultrasonic waves are propagated from the inner wall of the pipe towards the welded portion at the pipe end through this probe, and the echo of the ultrasonic waves is measured. When a defect occurs, creating a void within the welded portion of the pipe end, the echo from that void can be measured, thus enabling defect detection. Furthermore, regarding the contact medium and the coating method, anything that allows for the acquisition of an ultrasonic image is acceptable. For example, water can be used as the contact medium. When water is used as the contact medium, water can be supplied to the area around the probe using a pump.

[0032] For example, in Figure 2 In the magnified view of the probe's periphery shown in the lower left, the ultrasonic wave, indicated by arrow L3, propagates towards the gapless portion within the welded section of the pipe end. Therefore, the echo of the ultrasonic wave indicated by arrow L3 is not measured. On the other hand, the ultrasonic wave, indicated by arrow L2, propagates towards the gapless portion within the welded section of the pipe end; therefore, the echo of the ultrasonic wave reflected from this gap is measured.

[0033] Furthermore, since ultrasonic waves are also reflected at the periphery of the welded portion at the pipe end, the echoes of the ultrasonic waves propagating towards the periphery are also measured. For example, the ultrasonic wave, indicated by arrow L1, propagates towards the pipe end side closer to the welded portion than to the welded portion, and therefore does not reach the welded portion, but is reflected at the pipe surface on the pipe end side of the welded portion. Therefore, the echo from the pipe surface is measured using the ultrasonic wave indicated by arrow L1. Additionally, the ultrasonic wave, indicated by arrow L4, is reflected at the pipe surface on the deeper side of the welded portion, and this echo is also measured.

[0034] Since the welded portion at the pipe end exists throughout the 360-degree radius around the pipe, the probe is rotated in units of a specified angle (e.g., 1 degree) and repeated measurements are performed. The data representing the probe measurement results is then transmitted to the ultrasonic flaw detection device 7. For example, the probe can be an array probe composed of multiple array elements. If it is an array probe, the welded portion at the pipe end, which has a width in the pipe's extension direction, can be efficiently inspected by configuring the array elements to be arranged in the same direction as the pipe's extension. Furthermore, the aforementioned array probe can be a matrix array probe with multiple array elements arranged in both the longitudinal and transverse directions.

[0035] The ultrasonic flaw detection device 7 uses data representing probe measurement results to generate an ultrasonic image that visualizes the echoes of ultrasonic waves propagating towards the pipe and the welded portion at the pipe end. Figure 2 The image shown is an example of an ultrasonic image 111 generated by the ultrasonic flaw detection device 7. Alternatively, the information processing device 1 may also generate the ultrasonic image 111. In this case, the ultrasonic flaw detection device 7 sends data representing the probe measurement results to the information processing device 1.

[0036] In the ultrasonic image 111, the measured echo intensity is represented as the pixel value of each pixel. In addition, the image area of ​​the ultrasonic image 111 can be divided into: the pipe region ar1 corresponding to the pipe, the welding region ar2 corresponding to the welded part at the pipe end, and the peripheral echo regions ar3 and ar4 from which echoes from around the welded part at the pipe end appear.

[0037] As described above, the ultrasonic waves propagating from the probe in the direction indicated by arrow L1 are reflected at the pipe surface on the pipe end side of the pipe end weld. Additionally, the ultrasonic waves are also reflected on the inner surface of the pipe, and these reflections occur repeatedly. Therefore, repeating echoes a1 to a4 appear in the peripheral echo region ar3 along arrow L1 in ultrasonic image 111. Furthermore, the ultrasonic waves propagating from the probe in the direction indicated by arrow L4 are also repeatedly reflected on the outer and inner surfaces of the pipe. Therefore, repeating echoes a6 to a9 appear in the peripheral echo region ar4 along arrow L4 in ultrasonic image 111. These echoes appearing in the peripheral echo regions ar3 and ar4 are also referred to as bottom surface echoes.

[0038] For ultrasonic waves propagating from the probe in the direction indicated by arrow L3, no echo appears in the area along arrow L3 in ultrasonic image 111 because there is no reflection. On the other hand, for ultrasonic waves propagating from the probe in the direction indicated by arrow L2, an echo a5 appears in the area along arrow L2 in ultrasonic image 111 because reflection occurs at the gap, i.e., the defect, within the welded portion of the tube end.

[0039] The information processing device 1 analyzes the ultrasonic image 111 to check for defects in the welded portion at the pipe end, as detailed later. Furthermore, the information processing device 1 can also determine the type of defect. For example, if a defect is determined to exist, the information processing device 1 can determine which of the following is a known defect in the welded portion at the pipe end: poor initial penetration, poor fusion between weld beads, undercut, or porosity.

[0040] As described above, the inspection system 100 includes: an ultrasonic flaw detection device 7 that generates an ultrasonic image 111 of the pipe end weld; and an information processing device 1 that analyzes the ultrasonic image 111 and checks for defects on the pipe end weld. Furthermore, the information processing device 1 acquires the output value obtained by inputting the inspection image generated from the ultrasonic image 111 into a classification model, and applies a first method for noise-free images or a second method for noisy images based on the output value to determine whether a defect exists. Details will be described later. The classification model is generated by learning such that when multiple features extracted from a group of noise-free images are embedded into a feature space, the distance between the features is reduced. Therefore, even when the ultrasonic image 111 contains echoes from defective areas and noise with varied appearances, the presence or absence of defects can be determined with high accuracy.

[0041] (Structure of an information processing device)

[0042] based on Figure 1 The structure of information processing device 1 will be described. Figure 1 This is a block diagram illustrating an example of the main structural components of the information processing device 1. For example... Figure 1 As shown, the information processing device 1 includes: a control unit 10, which controls all parts of the information processing device 1; and a storage unit 11, which stores various data used by the information processing device 1. Additionally, the information processing device 1 includes: an input unit 12, which accepts input operations to the information processing device 1; and an output unit 13, which outputs data from the information processing device 1.

[0043] The control unit 10 includes: an inspection image generation unit 101, a determination unit 102A, a determination unit 102B, a determination unit 102C, a reliability determination unit 103, a comprehensive determination unit (determination unit) 104, and a classification unit (acquisition unit) 105. Additionally, the storage unit 11 stores the ultrasonic image 111 and the inspection result data 112. Furthermore, when it is not necessary to distinguish between determination units 102A, 102B, and 102C, only determination unit 102 will be referred to below.

[0044] The inspection image generation unit 101 cuts out the inspection target area from the ultrasonic image 111 and generates an inspection image for determining whether the inspection target has defects. The method for generating the inspection image will be described later.

[0045] The determination unit 102, together with the comprehensive determination unit (determination unit) 104, determines the prescribed determination items based on the object image. In this embodiment, an example will be described where the inspection image generated by the inspection image generation unit 101 is the aforementioned object image, and whether or not there are welding defects at the tube end weld of the heat exchanger reflected in the inspection image is the aforementioned prescribed determination item. Hereinafter, welding defects will sometimes be referred to simply as defects.

[0046] Furthermore, the definition of a "defect" as the object of inspection can be predetermined based on the purpose of the inspection. For example, in the quality inspection of the welded joints at the tube ends of a manufactured heat exchanger, a "defect" can be defined as an echo appearing in the inspection image caused by internal voids or unacceptable dents on the surface of the welded joint. Such dents are, for example, caused by burn-through. In other words, the presence or absence of a defect means the existence of a part that differs from a normal product (abnormal part). Additionally, in the field of non-destructive testing, abnormal parts detected using ultrasonic waveforms or ultrasonic images are generally referred to as "damage." This "damage" is also included in the scope of the aforementioned "defect." Furthermore, the aforementioned "defect" also includes defects or cracks.

[0047] Determination units 102A, 102B, and 102C all determine the presence or absence of defects based on the inspection image generated by the inspection image generation unit 101, but their determination methods differ as will be explained below.

[0048] The determination unit 102A determines whether a defect exists based on an output value, which is obtained by inputting an inspection image into a learned model generated through machine learning. More specifically, the determination unit 102A uses a generated image to determine whether a defect exists, which is generated by inputting an inspection image into a learned model, i.e., a generative model, generated through machine learning. Furthermore, the determination unit 102B determines the inspection target area in the inspection image by analyzing the pixel values ​​of each pixel, and determines whether a defect exists based on the determined pixel values ​​of the inspection target area.

[0049] Furthermore, similar to determination unit 102A, determination unit 102C determines the presence or absence of defects based on an output value, which is obtained by inputting an inspection image into a learned model generated through machine learning. More specifically, determination unit 102C determines the presence or absence of defects based on an output value obtained by inputting an inspection image into a determination model, which performs machine learning in such a way that it outputs the presence or absence of defects based on the input inspection image. Details of the determinations performed by determination units 102A to 102C and the various models used will be explained later.

[0050] The reliability determination unit 103 determines the reliability index, which represents the probability, for each determination result of determination units 102A to 102C. Specifically, the reliability determination unit 103 determines the reliability of determination unit 102A when determining the inspection image based on the output value obtained by the reliability prediction model used when the inspection image is used to derive the determination result from determination unit 102A.

[0051] The reliability prediction model used by the decision unit 102A can be generated by learning from teacher data, which is teacher data that associates the test image as correct data with whether the decision result made by the decision unit 102A based on the test image is correct. The test image can be generated simply from the ultrasonic image 111, which is known to have or not have defects.

[0052] If the inspection image 111A is input into the reliability prediction model generated in this way, the output is a value from 0 to 1, representing the probability that the judgment result of the judgment unit 102A is correct when using the inspection image 111A. Therefore, the reliability judgment unit 103 can use the output value of the reliability prediction model as the reliability of the judgment result of the judgment unit 102A. In addition, the reliability prediction model used by the judgment unit 102B and the reliability prediction model used by the judgment unit 102C can be generated in the same way. Furthermore, the reliability judgment unit 103 uses the reliability prediction model used by the judgment unit 102B to determine the reliability of the judgment result of the judgment unit 102B, and determines the reliability of the judgment result of the judgment unit 102C by using the reliability prediction model used by the judgment unit 102C.

[0053] The comprehensive judgment unit 104 uses the judgment results of judgment units 102A to 102C and the reliability judgment unit 103 to determine whether a defect exists. Therefore, it is possible to obtain a judgment result that appropriately considers the judgment results of judgment units 102A to 102C based on the reliability corresponding to the inspection image. The judgment method performed by the comprehensive judgment unit 104 will be described in detail later.

[0054] The classification unit 105 classifies the inspected images using a prescribed classification model. Based on... Figure 5 To elaborate, the classification model is generated through learning in such a way that when multiple features extracted from a first group of images with common characteristics are embedded into the feature space, the distance between the features is reduced. These common characteristics do not contain noise. The classification unit 105 obtains the output value obtained by inputting the inspection image into the classification model.

[0055] Furthermore, the determination unit 102 determines whether there is a defect by applying a first method for a first image group or a second method for a second image group consisting of images that do not belong to the first image group, based on the output value obtained by the classification unit 105.

[0056] Specifically, the output value acquired by the classification unit 105 indicates whether the examined image is a noisy image or a noise-free image. Furthermore, if the output value indicates a noise-free image, the first method for examining noise-free images is applied. Conversely, if the output value indicates a noisy image, the second method for examining noisy images is applied.

[0057] Specifically, the first method described above is a method for determining whether or not a defect exists by combining the determination results of determination units 102A to 102C and the reliability determination results determined by reliability determination unit 103. On the other hand, the second method described above is a method for determining whether or not a defect exists by determination unit 102B.

[0058] As described above, the ultrasonic image 111 is an image obtained by imaging the echo of the ultrasonic wave propagating toward the object being inspected, and it is generated by the ultrasonic flaw detection device 7.

[0059] Inspection result data 112 represents the results of a defect inspection performed by the information processing device 1. Inspection result data 112 records the presence or absence of defects in the ultrasonic images 111 stored in the storage unit 11. Furthermore, if the type of defect is determined, the determination result of the defect type can also be stored as inspection result data 112.

[0060] As described above, the information processing apparatus 1 includes: a classification unit 105 that acquires an output value obtained by inputting an inspection image into a classification model, the classification model being generated by learning in such a way that when multiple feature quantities extracted from a noiseless image group (a first image group having common features) are embedded into a feature space, the distance between the feature quantities is reduced; and a determination unit 102 that, based on the output value, applies a first method for the first image group or a second method for a noisy image group (a second image group consisting of images not belonging to the first image group) to determine whether there is a defect (a predetermined determination item related to the inspection image).

[0061] The classification model described above is generated by learning in a way that reduces the distance between features when embedding them into the feature space. Therefore, even if the image being examined is a noisy image prone to misclassification, inputting it into the classification model will yield an output value indicating whether the features of the examined image are close to those of the noise-free first image group.

[0062] That is, if the feature values ​​of the examined image are close to those of the first group of noise-free images, the examined image is more likely to be noise-free. Conversely, if the feature values ​​of the examined image deviate from those of the first group of noise-free images, the examined image is more likely to contain noise. Generally, irregular noise is difficult to collect sufficient teacher data due to its diverse shapes, making it difficult to determine the presence or absence of noise using a learned model generated through machine learning. However, if the above output values ​​are used, it is also possible to determine whether the examined image contains noise.

[0063] Furthermore, using the above structure, the determination of the matter is performed by applying either a first method for images without noise or a second method for images with noise, based on the output value described above. Therefore, an appropriate method corresponding to the characteristics of the inspection image can be applied, enabling high-precision determination even for inspection images prone to misjudgment.

[0064] (Summary of the inspection)

[0065] based on Figure 3 Here is an overview of the inspection performed using the information processing device 1. Figure 3 This is a diagram showing an outline of the inspection performed using information processing device 1. Furthermore, in Figure 3 The diagram shows the process after the ultrasonic image 111 generated by the ultrasonic flaw detector 7 is stored in the storage unit 11 of the information processing device 1.

[0066] First, the inspection image generation unit 101 extracts the inspection target region from the ultrasound image 111 to generate an inspection image 111A. When extracting the inspection target region, an extraction model constructed using machine learning can be used. The extraction model can be constructed using any learning model suitable for extracting regions from an image. For example, the inspection image generation unit 101 can construct an extraction model using YOLO (You Only Look Once), which offers excellent extraction accuracy and processing speed.

[0067] The aforementioned inspection area is the region sandwiched between two peripheral echo regions ar3 and ar4, which are repeatedly observed by echoes from the peripheral portion of the inspection object. For example... Figure 2As shown, in the ultrasonic image 111, a predetermined echo (echoes a1 to a4 and a6 to a9) caused by the shape of the periphery of the inspected part is repeatedly observed. Therefore, the region corresponding to the inspected part in the ultrasonic image 111 can be determined based on the position of the peripheral echo regions ar3 and ar4 where such echoes repeatedly appear. Furthermore, the occurrence of predetermined echoes at the periphery of the inspected part is not limited to the ultrasonic image 111 of the pipe end weld. Therefore, it can be applied to other inspections besides the pipe end weld for structures where the area surrounded by the peripheral echo region is extracted as the inspected area.

[0068] Next, the inspection image 111A is classified using the classification unit 105. Furthermore, for the inspection image 111A classified as noisy by the classification unit 105, the presence or absence of defects is determined using the second method as described above. Specifically, as follows... Figure 3 As shown, for the inspection image 111A classified as noisy, the determination unit 102B determines whether there is a defect through numerical analysis. This result is then appended to the inspection result data 112. Additionally, the determination unit 102B can also cause the output unit 13 to output the determination result.

[0069] On the other hand, for the inspection image 111A, which is classified as noise-free by the classification unit 105, the presence or absence of defects is determined using the first method. Specifically, firstly, the presence or absence of defects is determined based on the inspection image 111A using the determination unit 102A, determination unit 102B, and determination unit 102C. The determination criteria will be explained in detail later.

[0070] Next, the reliability of each determination result of determination units 102A, 102B, and 102C is determined using the reliability determination unit 103. Specifically, the reliability of the determination result of determination unit 102A is determined based on the following output value, which is the output value obtained by inputting the inspection image 111A into the reliability prediction model used by determination unit 102A. Similarly, the reliability of the determination result of determination unit 102B is determined based on the following output value, which is the output value obtained by inputting the inspection image 111A into the reliability prediction model used by determination unit 102B. Furthermore, the reliability of the determination result of determination unit 102C is determined based on the following output value, which is the output value obtained by inputting the inspection image 111A into the reliability prediction model used by determination unit 102C.

[0071] Furthermore, the comprehensive judgment unit 104 uses the judgment results of judgment units 102A, 102B, and 102C, and the reliability judgment unit 103's determination of reliability based on these judgment results to comprehensively determine whether there is a defect, and outputs the comprehensive judgment result. This result is appended to the inspection result data 112. Additionally, the comprehensive judgment unit 104 can also cause the output unit 13 to output the comprehensive judgment result.

[0072] In the comprehensive judgment, the judgment result of the judgment unit 102 can be represented numerically, and the reliability determined by the reliability judgment unit 103 is used as a weight. For example, when the judgment units 102A, 102B, and 102C determine that there is a defect, "1" is output as the judgment result, and when it is determined that there is no defect, "-1" is output as the judgment result. In addition, the reliability judgment unit 103 is set to output a reliability value ranging from 0 to 1 (the closer to 1, the higher the reliability).

[0073] In this case, the comprehensive determination unit 104 can calculate a total value obtained by adding the values ​​of "1" or "-1" output by determination units 102A, 102B, and 102C to the reliability values ​​output by the reliability determination unit 103. Furthermore, the comprehensive determination unit 104 can determine whether there is a defect based on whether the calculated total value is greater than a predetermined threshold.

[0074] For example, the threshold is set to "0", which is the middle value between "1" representing defects and "-1" representing no defects. Furthermore, the output values ​​of the determination unit 102A, determination unit 102B, and determination unit 102C are set to "1", "-1", and "1", respectively, and their reliability is set to "0.87", "0.51", and "0.95", respectively.

[0075] In this case, the comprehensive determination unit 104 performs the calculation 1×0.87+(-1)×0.51+1×0.95. The result of this calculation is 1.31. Since this value is greater than "0" which is the threshold, the comprehensive determination result performed by the comprehensive determination unit 104 is defective.

[0076] (The determination performed by the determination unit 102A)

[0077] As described above, the determination unit 102A uses a generated image, produced by inputting the inspection image into a generation model, to determine whether a defect exists. This generation model is constructed to generate a new image with the same features as the input image through machine learning, which uses an image of a defect-free inspection object as training data. Furthermore, the term "features" refers to any information obtained from the image, such as the distribution or variance of pixel values ​​in the image.

[0078] The aforementioned generative model is constructed using machine learning by employing images of defect-free objects to be inspected as training data. Therefore, when an image of a defect-free object is input into this generative model as the inspection image, there is a high probability that the generated image will output a new image with the same features as that inspection image.

[0079] On the other hand, when an image of a defective object to be inspected is input into the generative model as the inspection image, even if a defect of any shape and size is reflected at any position in the inspection image, the generated image is more likely to have features different from the inspection image.

[0080] Thus, there will be differences in whether the object image input to the generative model is accurately restored between the generated image generated from the inspection image that reflects the defects and the generated image generated from the inspection image that does not reflect the defects.

[0081] Therefore, the information processing device 1 can accurately determine whether there are defects with variable positions, sizes and shapes. The information processing device 1 takes into account the determination results of the determination unit 102A to make a comprehensive determination. The determination unit 102A uses the generated image generated by the above-mentioned generation model to determine whether there are defects.

[0082] The following is based on Figure 4 The details of the determination performed using the determination unit 102A will be explained below. Figure 4 Examples of the configuration of the determination unit 102A and examples of methods for determining the presence or absence of defects using the determination unit 102A are shown. Figure 4 As shown, the determination unit 102A includes: an inspection image acquisition unit 1021, a restoration image generation unit 1022, and a defect presence / absence determination unit 1023.

[0083] The inspection image acquisition unit 1021 acquires the inspection image. As described above, the information processing apparatus 1 includes an inspection image generation unit 101, therefore the inspection image acquisition unit 1021 acquires the inspection image generated by the inspection image generation unit 101. Furthermore, the inspection image may also be generated by other devices. In this case, the inspection image acquisition unit 1021 acquires the inspection image generated by other devices.

[0084] The restored image generation unit 1022 generates a new image with the same features as the input inspection image by inputting the inspection image acquired by the inspection image acquisition unit 1021 into the generative model. Hereinafter, the image generated by the restored image generation unit 1022 will be referred to as the restored image. The generative model used to generate the restored image is also called an autoencoder, and can be constructed through machine learning using images of defect-free inspection objects as training data, as detailed later. Furthermore, the generative model can be a model obtained by improving or modifying an autoencoder. For example, an incremental autoencoder can be applied as the generative model.

[0085] The defect presence / absence determination unit 1023 uses the restored image generated by the restored image generation unit 1022 to determine whether the object to be inspected has a defect. Specifically, when the variance of the difference values ​​of each pixel between the inspection image and the restored image exceeds a predetermined threshold, the defect presence / absence determination unit 1023 determines that the object to be inspected has a defect.

[0086] In the method for determining the presence or absence of defects using the determination unit 102A with the above structure, firstly, the inspection image acquisition unit 1021 acquires the inspection image 111A. Then, the inspection image acquisition unit 1021 sends the acquired inspection image 111A to the restoration image generation unit 1022. As described above, the inspection image 111A is generated by the inspection image generation unit 101 based on the ultrasonic image 111.

[0087] Next, the restored image generation unit 1022 inputs the inspection image 111A into the generation model and generates a restored image 111B based on its output values. Furthermore, the inspection image acquisition unit 1021 removes the peripheral echo region from the inspection image 111A to generate a removed image 111C, and removes the peripheral echo region from the restored image 111B to generate a removed (restored) image 111D. Moreover, if the object being inspected is the same, the position and size of the peripheral echo region reflected in the inspection image 111A are approximately constant. Therefore, the inspection image acquisition unit 1021 can remove a predetermined area in the inspection image 111A as a peripheral echo region. Additionally, the inspection image acquisition unit 1021 can analyze the inspection image 111A to detect the peripheral echo region and remove it based on the detection result.

[0088] By removing the peripheral echo region in the above manner, the defect presence / absence determination unit 1023 uses the remaining image region after removing the peripheral echo region from the image region of the restored image 111B as the object to determine whether a defect exists. Therefore, the presence / absence of defects can be determined without being affected by echoes from the peripheral region, thus improving the accuracy of defect presence / absence determination.

[0089] Next, the defect presence / absence determination unit 1023 determines whether a defect exists. Specifically, the defect presence / absence determination unit 1023 first calculates the difference between the removed image 111C and the removed image (restoration) 111D on a pixel-by-pixel basis. Next, the defect presence / absence determination unit 1023 calculates the variance of the calculated difference. Furthermore, the defect presence / absence determination unit 1023 determines whether a defect exists based on whether the calculated variance value exceeds a predetermined threshold.

[0090] Here, the difference value calculated for the pixel that reflects the echo caused by the defect is a larger value compared to the difference values ​​calculated for other pixels. Therefore, the variance of the difference values ​​calculated for the removed image 111C and the removed image (restoration) 111D based on the inspection image 111A that reflects the echo caused by the defect becomes larger.

[0091] On the other hand, for the removed image 111C and the removed image (restored) 111D based on the inspection image 111A which does not reflect the echo caused by the defect, the variance of the difference value is relatively reduced. This is because, in the case where the echo caused by the defect is not reflected, there may be areas where the pixel value is relatively large due to the influence of noise, etc., but the possibility of areas with extremely large pixel values ​​is low.

[0092] Thus, when the object being inspected has a defect, the variance of the difference value increases, which is a characteristic phenomenon. Therefore, if the following structure is adopted, it is possible to properly determine whether there is a defect, that is, when the variance of the above difference value exceeds a predetermined threshold, the defect determination unit 1023 determines that there is a defect.

[0093] Furthermore, the timing for removing the peripheral echo region is not limited to the examples described above. For example, a difference image of the examination image 111A and the restored image 111B can be generated, and the peripheral echo region can be removed from the difference image.

[0094] (The determination performed by the determination unit 102B)

[0095] As described above, the determination unit 102B determines the inspection object part in the inspection image by analyzing the pixel values ​​of the image of the inspection object, i.e., the inspection image, and determines whether there is a defect based on the pixel values ​​of the determined inspection object part.

[0096] In existing image-based inspections, inspectors visually determine the areas of interest in the image and verify whether these areas show existing damage or defects such as gaps not present in the design. This type of visual inspection requires automation based on considerations of labor-saving and accuracy stabilization.

[0097] The determination unit 102B determines the inspection target area by analyzing the pixel values ​​of the image, and determines whether there is a defect based on the pixel values ​​of the determined inspection target area. Therefore, the visual inspection described above can be automated. Furthermore, the information processing device 1 makes a determination by comprehensively considering the determination results of the determination unit 102B and the determination results of other determination units 102 for inspection images classified as noise-free, thus enabling high-precision determination of whether there is a defect. In addition, for inspection images classified as noisy, the information processing device 1 can avoid mistaking noise for defects and accurately determine whether there is a defect by analyzing the pixel values.

[0098] The following describes in more detail the processing (numerical analysis) performed by the determination unit 102B. First, the determination unit 102B will determine the two peripheral echo regions (where the echo from the peripheral portion of the inspected object region repeatedly appears in the inspection image) that are observed in the inspection image. Figure 2 The area sandwiched between the peripheral echo regions ar3 and ar4 in the example is determined as the inspection target area. Furthermore, the determination unit 102B determines whether there is a defect based on whether the determined inspection target area contains an area composed of pixel values ​​above a threshold (also called a defect area).

[0099] The determination unit 102B can first generate a binarized image by binarizing the inspection image 111A using a predetermined threshold when detecting the peripheral echo region and the defect region. Then, the determination unit 102B detects the peripheral echo region based on the binarized image. For example, in... Figure 3 The inspection image 111A shown contains echoes a1, a2, a6, and a7. If the determination unit 102B binarizes the inspection image 111A using a threshold that can distinguish these echoes from noise components, it can detect these echoes based on the binarized image. Furthermore, the determination unit 102B can detect the ends of these detected echoes and determine the regions surrounded by these ends as the inspection target areas.

[0100] More specifically, the determination unit 102B determines the right end of echo a1 or a2 as the left end of the inspection target area, and the left end of echo a6 or a7 as the right end of the inspection target area. Their ends are the boundaries between the peripheral echo regions ar3 and ar4 and the inspection target area. Similarly, the determination unit 102B determines the upper end of echo a1 or a6 as the upper end of the inspection target area, and the lower end of echo a2 or a7 as the lower end of the inspection target area.

[0101] In addition, such as Figure 2 As shown in the ultrasonic image 111, since the echo caused by the defect appears above the echoes a1 and a6, the determination unit 102B can set the upper end of the inspection target area above the position of the upper end of the echoes a1 or a6.

[0102] Furthermore, the determination unit 102B analyzes the inspection target area determined in the binarized image and can determine whether an echo caused by a defect is reflected. For example, when there is a continuous region consisting of a predetermined number or more pixels in the inspection target area, the determination unit 102B can determine that an echo caused by a defect is reflected at the location where the continuous region exists.

[0103] Furthermore, the numerical analysis described above is just one example, and the content of numerical analysis is not limited to the example above. For example, when there is a substantial difference in the variance of the pixel values ​​of the inspected object area between the defective and non-defective cases, the determination unit 102B can determine whether there is a defect based on the variance value.

[0104] Furthermore, for example, the determination unit 102B can determine the presence or absence of defects through numerical analysis based on the simulation results of an ultrasonic beam simulator. The ultrasonic beam simulator outputs the height of the reflected echo when an artificial damage is placed at any location on the test subject for flaw detection. Therefore, the determination unit 102B can determine the presence or absence of defects and their location by comparing the height of the reflected echo corresponding to artificial damage at various locations output by the ultrasonic beam simulator with the reflected echo in the inspection image.

[0105] (The determination performed by the determination unit 102C)

[0106] As described above, the determination unit 102C determines whether a defect exists based on the output value obtained by inputting the inspection image into the determination model. This determination model is constructed, for example, by using machine learning with teacher data generated using ultrasound images 111 of defective inspection objects and ultrasound images 111 of defect-free inspection objects.

[0107] The aforementioned decision model can be constructed using any learning model suitable for image classification. For example, it can be constructed using convolutional neural networks, which have excellent image classification accuracy.

[0108] (Regarding classification models)

[0109] based on Figure 5 The classification model used by the classification unit 105 in classifying inspected images will be explained. Figure 5 This is a diagram representing an example of embedding features extracted from multiple inspection images using the aforementioned classification model in the feature space.

[0110] The classification model is generated by learning in a manner that reduces the distance between features extracted from a noise-free image group (first image group) within an image set of the object to be inspected, when embedded into the feature space. More specifically, the classification model is generated by learning in a manner that reduces the distance between features extracted from both the noise-free / defective image group and the noise-free / defect-free image group. That is, the classification model classifies the inspected images into two categories: noise-free / defective and noise-free / defect-free.

[0111] Figure 5 The feature space shown is a two-dimensional feature space with the horizontal axis set to x and the vertical axis set to y. Additionally, in Figure 5 The image also shows a portion of the inspection images from which the feature quantities were extracted (inspection images 111A1 to 111A5). Figure 5 The inspection images shown are 111A1 and 111A2, which are noise-free / defect-free images. On the other hand, inspection images 111A3 and 111A4 are noisy images, which show noise in regions AR1 and AR2. In addition, inspection image 111A5 is a noise-free / defect-free image that shows the defective echo a10 but not the noise.

[0112] As shown in the figure, when feature quantities are embedded in the feature space, the feature quantities of inspection images belonging to the same category are depicted in positions close to each other. These feature quantities are extracted from each inspection image using the classification model generated by learning as described above.

[0113] Specifically, the feature quantities of inspection images 111A1 and 111A2, which are noise-free and defect-free, converge approximately to a circle C1 with radius r1 centered at point P1. Similarly, the feature quantities of inspection images 111A5, which are noise-free and defective, converge approximately to a circle C2 with radius r2 centered at point P2.

[0114] On the other hand, the feature quantities of noisy inspection images such as 111A3 and 111A4 are depicted at positions far from circles C1 and C2. Therefore, it can be seen that noisy and noiseless inspection images can be identified by using a model that classifies inspection images into two categories: noiseless / defective and noiseless / defectless.

[0115] For example, if the feature values ​​obtained by inputting the inspection image into the classification model are plotted within circle C1, the classification unit 105 can classify the inspection image as defect-free. Furthermore, if the feature values ​​obtained by inputting the inspection image into the classification model are plotted within circle C2, the classification unit 105 can classify the inspection image as defective. And, if the feature values ​​obtained by inputting the inspection image into the classification model are plotted in a location not included by circles C1 and C2, the classification unit 105 can classify the inspection image as noisy.

[0116] Furthermore, the radius r1 of circle C1 and the radius r2 of circle C2 can be the same or different. For example, the radii r1 and r2 can be set to appropriate values. Alternatively, for example, the radius can be set to the distance from the center of the feature plot of the teacher data to the plot furthest from that center. Additionally, for example, the radius can be set to twice the standard deviation (σ) of the feature plot of the teacher data.

[0117] Alternatively, numerical values ​​from 0 to 1 can be used to represent the location of the feature quantities depicted in the inspected image. For example, the location of point P1 can be set to (0, 0), the location of point P2 can be set to (0, 1), and the depiction of each feature quantity of the inspected image can be projected onto the straight line L1 connecting points P1 and P2.

[0118] In this case, if the feature quantity is drawn on the line L1 within the range from point p11 to point p12, the inspection image can be determined to be noise-free / defect-free. Furthermore, point p11 is the intersection point of circle C1 and line L1 on the side closer to circle C2. Additionally, point p12 is the intersection point of circle C1 and line L1 on the side farther from circle C2.

[0119] Similarly, if feature quantities are drawn on the line L1 within the range from point p21 to point p22, the inspection image can be determined as noise-free / defective. Furthermore, point p21 is the intersection point of circle C2 and line L1 on the side closer to circle C1. Additionally, point p22 is the intersection point of circle C2 and line L1 on the side farther from circle C1.

[0120] Furthermore, if feature quantities are depicted within the range from point p11 to point p21, the inspected image can be determined to be noisy.

[0121] Furthermore, the value depicted outside point P1 on line L1 (opposite to the direction of circle C2) can be considered 0, and the value depicted outside point P2 on line L1 (opposite to the direction of circle C1) can be considered 1. Additionally, the value depicted inside circle C1 can be considered 0, and the value depicted inside circle C2 can also be considered 1. In this case, inspection images with a depicted value of 0 are classified as defect-free, inspection images with a depicted value of 1 are classified as defective, and inspection images with depicted values ​​other than 0 and 1 are classified as noisy.

[0122] Furthermore, even when using the following classification model, it is equally possible to classify inspection images as noisy or noise-free, wherein the classification model has only learned inspection images that are noise-free / defect-free or only learned inspection images that are noise-free / defect-free.

[0123] As described above, when multiple features extracted from a group of noise-free images are embedded in the feature space, the classification unit 105 can classify the examined image into noisy and noise-free categories using an output value generated by learning a classification model in a manner that reduces the distance between the features. Furthermore, the classification model can be designed to output a value representing the classification result (e.g., the reliability of each category) or to output the feature values. The reliability is a numerical value of 0 to 1 representing the probability of the classification result.

[0124] The classification model described above can be generated, for example, through deep metric learning. Deep metric learning is a method that learns features in a way that decreases the distance Sn between features of the same class and increases the distance Sp between features of different classes. During learning, the distance between features can be represented using Euclidean distance or other similar methods, or it can be represented by angles.

[0125] Furthermore, the inventors of this invention attempted to classify noisy and noise-free inspection images using a convolutional neural network classification model, but this method proved difficult. Therefore, to distinguish between noisy and noise-free inspection images, it is crucial to use a classification model learned in the following manner.

[0126] That is, when a feature is embedded in the feature space, the distance between the features is reduced.

[0127] (Processing flow during inspection)

[0128] based on Figure 6 This explains the process of handling (judgment method) during the inspection. Figure 6This diagram illustrates an example of the inspection method using information processing device 1. Furthermore, it is set that: in Figure 6 The start time of processing will be determined by... Figure 2 The ultrasonic image 111 generated by the method described for detecting flaws in the welded part of the pipe end and its periphery is stored in the storage unit 11, and the inspection image generation unit 101 has generated an inspection image based on the ultrasonic image 111.

[0129] In S11, the classification unit 105 acquires the inspection image generated by the inspection image generation unit 101. Next, in S12 (acquisition step), the classification unit 105 inputs the inspection image acquired in S11 into the aforementioned classification model and acquires the output value of the classification model. Furthermore, in S13, the classification unit 105 determines, based on the output value acquired in S12, whether the inspection image acquired in S11 is a noisy inspection image or a noise-free inspection image.

[0130] When the inspection image is determined to be noisy by S13 (yes in S13), the process proceeds to S17. In S17 (determination step), the second method for noisy inspection images, namely the determination unit 102B that performs numerical analysis on the pixel values ​​of the inspection image, is used to determine whether the inspection image has defects, and the determination result is recorded in the inspection result data 112.

[0131] On the other hand, when it is determined by S13 that the inspection image is noise-free (no in S13), the process proceeds to S14. Furthermore, in S14 to S16 (determination steps), the first method for noise-free inspection images, namely the determination units 102A, 102C, etc., which determine whether there is a defect by using a learned model, is used to determine whether the inspection image has a defect.

[0132] Specifically, in S14, determination units 102A, 102B, and 102C are used to determine whether a defect exists. Furthermore, in the following S15, a reliability determination unit 103 determines the reliability of the determination results of each of the determination units 102A, 102B, and 102C. Moreover, the processing in S15 can be performed before or in parallel with S14.

[0133] Furthermore, in S16, the comprehensive judgment unit 104 uses the judgment results from S14 and the reliability determined in S15 to determine whether there is a defect. Specifically, the comprehensive judgment unit 104 uses a value obtained by adding a weight corresponding to the reliability to the values ​​representing the judgment results of judgment units 102A to 102C and then summing them to determine whether there is a defect. In addition, the comprehensive judgment unit 104 appends this judgment result to the inspection result data 112.

[0134] For example, the judgment result of the judgment units 102A to 102C can be -1 (no defect) or 1.

[0135] The reliability is represented by a numerical value (if it is defective). In this case, if the reliability is calculated using a value between 0 and 1, then the reliability value can be directly used as a weight and multiplied by the judgment result.

[0136] For a specific example, the determination result of determination unit 102A is defective, the determination result of determination unit 102B is no defective, and the determination result of determination unit 102C is defective. Furthermore, the reliability of the determination results of determination units 102A to 102C are 0.87, 0.51, and 0.95, respectively. In this case, the comprehensive determination unit 104 performs a calculation of 1 × 0.87 + (-1) × 0.51.

[0137] The operation of adding 1 × 0.95 yields the value 1.31.

[0138] Furthermore, the comprehensive determination unit 104 can compare this value with a predetermined threshold. If the calculated value is greater than the threshold, it is determined to be defective. When "-1" represents no defect and "1" represents defective, the threshold only needs to be set to the midpoint of these values, i.e., "0". In this case, since 1.31 > 0, the final determination result of the comprehensive determination unit 104 is defective.

[0139] As described above, the determination method in this embodiment is executed by the information processing device 1 and includes: an acquisition step (S12) of acquiring an output value obtained by inputting an inspection image into a classification model, wherein the classification model is generated by learning in such a way that when multiple feature quantities extracted from a noiseless image group (a first image group with common features) are embedded into a feature space, the distance between these feature quantities is reduced; and a determination step (S14 to S16 when the first method is applied, and S17 when the second method is applied) of applying the first method for a noiseless inspection image or the second method for a noisy inspection image (a second image group consisting of images that do not belong to the first image group) based on the output value to determine whether there is a defect (a predetermined determination item related to the inspection image).

[0140] Therefore, it can also make high-precision judgments on images that are prone to misjudgment.

[0141] Furthermore, the noise-free inspection image is the image that the determination units 102A and 102C determine to be valid, based on the output value obtained by inputting the inspection image into a learned model generated through machine learning. Therefore, as described above... Figure 6As in the example, it is preferable that the first method includes at least the process of making a decision using the learned model, and the second method includes at least the process of performing the numerical analysis described above.

[0142] For noise-free inspection images, since they do not contain parts whose appearance is similar to defects in the inspected object, the judgments made using a learned model generated through machine learning are effective. Therefore, by incorporating a process that uses a learned model generated through machine learning to make judgments for noise-free inspection images, high-accuracy judgment results can be expected.

[0143] Here, because noise is irregular and its appearance resembles the defects of the object being inspected, there are cases where the judgments made using a learned model generated through machine learning are ineffective for noisy inspection images. However, even for such inspection images, appropriate judgments can sometimes be made through numerical analysis.

[0144] Therefore, based on the above structure for determining whether a noisy inspection image has defects using a second method incorporating numerical analysis, an appropriate determination result can be expected even if the determination of the inspection image using the learned model is ineffective. That is, according to the above structure, an appropriate determination can be made regardless of whether the determination of the inspection image using the learned model is effective.

[0145] Furthermore, in the first method, it is sufficient to include at least one decision process that uses a learned model generated through machine learning. For example, the first method may include only one of the decision processes performed by decision units 102A and 102C. In the second method, in addition to the decision process performed by decision unit 102B, decision processes performed by other methods such as decision units 102A and 102C may also be included. However, in this case, it is preferable to give a greater weight to the decision result of decision unit 102B than to the decision results of other methods.

[0146] (Implementation Method 2)

[0147] Another embodiment of the present invention will be described below. Furthermore, for ease of explanation, components having the same function as those described in the above embodiments will be labeled with the same reference numerals without repetition of their descriptions. The same applies to Embodiment 3 and subsequent embodiments.

[0148] (Device Structure)

[0149] based on Figure 7 The structure of the information processing apparatus 1A of this embodiment will be described. Figure 7 This is a block diagram illustrating an example of the main component structure of the information processing device 1A. The information processing device 1A and... Figure 1 Compared to the information processing device 1 shown, the difference is that it does not have a classification unit 105, but has a determination unit 102X and a determination method determination unit (acquisition unit) 106.

[0150] The determination unit 102X uses the classification model described in Embodiment 1 to determine whether or not a defect exists. More specifically, the determination unit 102X determines whether or not a defect exists based on the output value obtained by inputting the inspection image into the classification model.

[0151] For example, the decision unit 102X can use a classification model as follows, which is based on... Figure 5 The model is generated by learning from examples, that is, reducing the distance between features extracted from the noise-free / defective image set, and further reducing the distance between features extracted from the noise-free / defect-free image set. The determination unit 102X determines whether the inspection image is a noise-free / defect-free inspection image or a noise-free / defective inspection image based on the output value obtained by inputting the inspection image into the classification model.

[0152] The determination method determining unit 106 acquires the output value of the classification model used by the determination unit 102X for the aforementioned determination. Furthermore, if the output value indicates that it belongs to either "no noise / no defect" or "no noise / defective," the determination method determining unit 106 determines that there is no noise in the inspection image and determines a first method for applying a noise-free inspection image. On the other hand, if the output value indicates that it does not belong to either "no noise / no defect" or "no noise / defective," the determination method determining unit 106 determines that there is noise in the inspection image and determines a second method for applying a noisy inspection image.

[0153] (Processing flow)

[0154] based on Figure 8 This describes the process flow (determination method) executed by the information processing device 1A. Figure 8 This diagram illustrates an example of an inspection method using the information processing device 1A. Furthermore, it is set that: in Figure 8 At the start of the processing, the ultrasonic image 111 is stored in the storage unit 11, and the inspection image generation unit 101 has generated an inspection image based on the ultrasonic image 111.

[0155] In S21, all the determination units 102, namely determination units 102A, 102B, 102C, and 102X, acquire the inspection image generated by the inspection image generation unit 101. Furthermore, in S22, all the determination units 102 that acquired the inspection image in S21 use the inspection image to determine whether a defect exists.

[0156] In S23 (acquisition step), the determination method determining unit 106 acquires the output value obtained by the determination unit 102X inputting the inspection image into the classification model in S22. Furthermore, the determination method determining unit 106 determines, based on the acquired output value, whether the inspection image acquired in S21 is a noisy inspection image or a noise-free inspection image.

[0157] When the inspection image is determined to be noisy in S23 (yes in S23), the determination method determination unit 106 instructs the determination unit 102B to perform the determination, and the process proceeds to S26. Furthermore, in S26 (the determination step), the second method for noisy inspection images, namely, the determination unit 102B which performs numerical analysis on the pixel values ​​of the inspection image, is used to determine whether the inspection image has defects, and the determination result is appended to the inspection result data 112.

[0158] Furthermore, since the determination of the determination unit 102B has already been executed in S22, if the determination is yes in S23, the determination result of the determination unit 102B in S22 can be added to the inspection result data 112 as the final determination result, instead of the processing of S26.

[0159] When the inspection image is determined to be noise-free in S23 (which is not the case in S23), the determination method determination unit 106 instructs the reliability determination unit 103 and the comprehensive determination unit 104 to perform the determination, and the process proceeds to S24. Furthermore, in S24 to S25 (determination steps), the presence or absence of defects in the inspection image is determined using the first method for noise-free inspection images, which is a method that integrates the determination results of multiple methods in S22 to make a final determination.

[0160] Specifically, in S24, the reliability determination unit 103 determines the reliability of the determination results of each of the determination units 102A, 102B, 102C, and 102X. The method for determining the reliability of the determination results of the determination units 102A, 102B, and 102C is as described in Embodiment 1. As for the reliability of the determination result of the determination unit 102X, similarly to the reliability prediction model used by the determination unit 102A described in Embodiment 1, it is sufficient to generate the reliability prediction model used by the determination unit 102X in advance and use the prediction model for determination.

[0161] Furthermore, in S25, the comprehensive judgment unit 104 uses the judgment results of S22 and the reliability determined by S24 to determine whether there is a defect. The comprehensive judgment unit 104 then appends this judgment result to the inspection result data 112.

[0162] A noise-free inspection image is an image deemed valid by the determination units 102A and 102C, the determination being based on the output value obtained by inputting the inspection image into a learned model generated through machine learning. Therefore, when the first method is configured to combine the defect determination results from multiple methods to make a final determination, it is preferable to include a method that uses a learned model generated through machine learning for determination. Furthermore, it is preferable to also include a method that determines the defect based on the output value of a classification model. Additionally, in this case, it is preferable that the second method is a method that determines the defect by numerically analyzing the pixel values ​​of the inspection image.

[0163] Based on the above structure, the first method for determining noise-free inspection images is as follows: In addition to using multiple methods for determination, this first method integrates the results of each determination to make a final determination. Among the multiple methods is a method where determination units 102A and 102C make determinations using a learned model. Regarding noise-free inspection images, since the determination using the learned model is effective, high-precision determination is possible. Furthermore, this determination also considers the determination result of determination unit 102X, which determines the determination item based on the output value of the classification model; therefore, further improvement in determination accuracy can be expected.

[0164] Furthermore, based on the above structure, the determination method for noisy inspection images, namely the second method, is a method of determining the image by numerically analyzing the pixel values ​​of the inspection image using the determination unit 102B. In noisy inspection images, sometimes the determination using a learned model is ineffective; even in such cases, numerical analysis can sometimes enable appropriate determination.

[0165] Therefore, based on the above structure, appropriate judgments can be made regardless of whether the judgments of the learned model used to examine the image are valid.

[0166] (Implementation Method 3)

[0167] (Device Structure)

[0168] based on Figure 9 The structure of the information processing apparatus 1B in this embodiment will be described. Figure 9 This is a block diagram showing an example of the main component structure of the information processing apparatus 1B. The information processing apparatus 1B includes: an image generation unit 101, a determination unit 102B, a determination unit 102Y, and a determination method determination unit (acquisition unit) 106.

[0169] Similar to the determination unit 102X in Embodiment 2, the determination unit 102Y uses a classification model to determine whether a defect exists. More specifically, the determination unit 102Y determines whether a defect exists based on the output value obtained by inputting the inspection image into the classification model.

[0170] For example, a classification model can also be used, which is based on, as follows: Figure 5 The model is generated by learning from examples, that is, reducing the distance between features extracted from the noise-free / defective image set, and further reducing the distance between features extracted from the noise-free / defect-free image set. The determination unit 102Y determines whether the inspection image is a noise-free / defect-free inspection image or a noise-free / defective inspection image based on the output value obtained by inputting the inspection image into the classification model.

[0171] (Processing flow)

[0172] based on Figure 10 This describes the process (determination method) executed by the information processing device 1B. Figure 10 This diagram illustrates an example of an inspection method using the information processing device 1B. Furthermore, it is set that: in Figure 10 At the start of the processing, the ultrasonic image 111 is stored in the storage unit 11, and the inspection image generation unit 101 has generated an inspection image based on the ultrasonic image 111.

[0173] In step S31, the determination unit 102Y acquires the inspection image generated by the inspection image generation unit 101. Then, in step S32 (determination step), the determination unit 102Y uses the inspection image acquired in step S31 to determine whether or not a defect exists.

[0174] In S33 (acquisition step), the determination method determination unit 106 acquires the output value obtained by the determination unit 102Y in S32 by inputting the inspection image into the classification model, and determines whether the inspection image acquired in S31 is a noisy inspection image or a noise-free inspection image based on the output value.

[0175] When the inspection image is determined to be noisy in S33 (yes in S33), the determination method determination unit 106 instructs the determination unit 102B to perform the determination, and the process proceeds to S35. Furthermore, in S35 (the determination step), the second method for noisy inspection images, namely, the determination unit 102B which performs numerical analysis on the pixel values ​​of the inspection image, is used to determine whether the inspection image has defects, and the determination result is appended to the inspection result data 112.

[0176] On the other hand, when the inspection image is determined to be noise-free in S33 (which is not the case in S33), the determination method determination unit 106 proceeds to the processing in S34. Furthermore, in S34, the determination method determination unit 106 appends the determination result of S32 as the final determination result to the inspection result data 112.

[0177] In this embodiment, similar to embodiments 1 and 2, it is determined whether there is a defect in the inspected object reflected in the inspection image, i.e., whether there is an abnormal part. When the defective part is referred to as an abnormal part, since noise is similar in appearance to the abnormal part, it can be said that the images in the noisy image group contain images of suspected abnormal parts that resemble the appearance of the abnormal part. Conversely, it can be said that the inspection images in the noise-free image group do not contain images of suspected abnormal parts.

[0178] Furthermore, the output value of the classification model used by the determination unit 102Y indicates whether the image being inspected belongs to a noisy image group, whether it belongs to a noise-free image group and contains anomalies, or whether it belongs to a noise-free image group and does not contain anomalies. In this case, as in the example above, the first method may include a process of determining whether anomalies exist in the object being inspected based on the aforementioned output value of the classification model. Furthermore, the second method may include a process of determining whether anomalies exist in the object being inspected by performing numerical analysis on the pixel values ​​of the inspected image.

[0179] Based on the above structure, the first method is applied to the inspection image contained in the noise-free image group, and the presence or absence of abnormal parts is determined based on the output value of the classification model. As described above, the classification model is generated by learning in a manner that reduces the distance between feature values ​​extracted from the noise-free / defective image group, and also reduces the distance between feature values ​​extracted from the noise-free / defect-free image group. Therefore, by using the classification model, it is possible to determine with high accuracy whether the inspection image belongs to the noise-free / defective image group.

[0180] However, it is also considered that even using the above classification model, it is difficult to identify suspected abnormalities and abnormal parts. Therefore, according to the above structure, for inspection images whose output values ​​of the classification model represent noisy images (second image group), that is, inspection images containing suspected abnormalities, the determination unit 102B determines whether there are abnormal parts in the inspected object by performing numerical analysis on the pixel values ​​of the inspection image. Thus, even for inspection images containing suspected abnormalities that are difficult to identify as abnormal parts, it is possible to determine whether there are abnormal parts with high accuracy.

[0181] Therefore, based on the above structure, appropriate determination can be made for both inspection images containing suspected abnormal areas and inspection images not containing suspected abnormal areas. Of course, the first method may also include determination processing by determination unit 102B, and determination processing by determination units 102A, 102C, etc., as described in Embodiment 1. Similarly, the second method may include determination processing by determination unit 102Y, and determination processing by determination units 102A, 102C, etc., as described in Embodiment 1, in addition to determination processing by determination unit 102B.

[0182] Furthermore, in S32, the determination unit 102Y can use the following classification model to determine the inspected images and classify them into four categories: no noise / defective, no noise / no defective, noisy / defective, and a total of noisy / no defective. This classification model can be generated by learning in a way that reduces the distance between features extracted from the noisy / defective image group and also reduces the distance between features extracted from the noisy / no defective image group.

[0183] In this case, in S35, the determination result of noisy / defective or noisy / defect-free from determination unit 102Y and the determination result of defect-free from determination unit 102B can both be used as the final determination result. Alternatively, these determination results can be combined to determine the final determination result. For example, multiple determination results can be combined based on reliability, similar to embodiments 1 and 2. However, in this case, it is preferable to make the weight of the determination result for determination unit 102B greater than that for determination unit 102Y.

[0184] (Implementation Method 4)

[0185] (Device Structure)

[0186] based on Figure 11 The structure of the information processing apparatus 1C of this embodiment will be described. Figure 11 This is a block diagram showing an example of the main component structure of the information processing apparatus 1C. The information processing apparatus 1C includes: an image generation unit 101, a determination unit 102A to 102C, a reliability determination unit 103, a comprehensive determination unit 104, a weight setting unit (acquisition unit) 107, and a comprehensive weight determination unit 108.

[0187] The weight setting unit 107 obtains the output value obtained by inputting the object image into the classification model, which is generated by learning in such a way that when multiple feature quantities extracted from a noiseless image group (a first image group with common features) are embedded into the feature space, the distance between the feature quantities is reduced.

[0188] Furthermore, the weight setting unit 107 sets the weight for each determination result when integrating the determination results of determination units 102A to 102C based on the acquired output values. Specifically, when applying the first method for inspecting noiseless images, the weight setting unit 107 sets the weight for the determination results of determination units 102A and 102C, which use a learned model generated through machine learning, to be greater than the weight for the determination result of determination unit 102B, which uses a numerical analysis method. On the other hand, when applying the second method for inspecting noisy images, the weight setting unit 107 sets the weight for the determination result of determination unit 102B to be greater than the weight for the determination results of determination units 102A and 102C.

[0189] Furthermore, the specific method for determining the weight values ​​only needs to be predetermined. For example, using the following classification model, which classifies weights by using weights such as... Figure 5 The example is generated by learning, that is, reducing the distance between features extracted from noise-free / defective image sets, and further reducing the distance between features extracted from noise-free / defect-free image sets.

[0190] In this case, the weight setting unit 107 can convert the coordinate values ​​of the feature quantities extracted from the inspection image in the feature space into weight values ​​of 0 to 1 using a prescribed mathematical formula. In this case, the weight value calculation order is as follows: (1) Calculate the distance from the position of the feature quantity extracted from the inspection image in the feature space to the center point of the noise-free / defect-free class, i.e., point P1. (2) Similarly to (1) above, calculate the distance from the position of the feature quantity extracted from the inspection image to the center point of the noise-free / defective class, i.e., point P2. (3) Substitute the shorter of the calculated distances into the prescribed mathematical formula to calculate the weight value.

[0191] The aforementioned mathematical formula is a function of the distance and weight value as variables, satisfying the condition that the shorter the distance, the greater the weight value of the judgment results of determination units 102A and 102C. Furthermore, when a distance shorter than radius r1 or r2 is substituted into this mathematical formula, a weight value equal to or greater than the weight value of the judgment result of determination unit 102B is calculated for the judgment results of determination units 102A and 102C. Moreover, "equal" also includes the same value. On the other hand, when a distance longer than radius r1 or r2 is substituted into this mathematical formula, a weight value smaller than the weight value of the judgment result of determination unit 102B is calculated for the judgment results of determination units 102A and 102C.

[0192] Furthermore, the weight setting unit 107 can determine the weight values ​​using the same method as the reliability determination unit 103 for determining reliability. In this case, the weight setting unit 107 uses the reliability prediction model used by the determination unit 102X described in Embodiment 2 to calculate the reliability of the output value of the classification model. Furthermore, the weight setting unit 107 can be set to a value such that the higher the calculated reliability, the larger the weight value for the determination results of the determination units 102A and 102C.

[0193] Therefore, similar to the images with high classification success rates in the aforementioned classification model, for inspection images where the determination results of determination units 102A and 102C are more likely to be appropriate, the weight values ​​for the determination results of determination units 102A and 102C increase. On the other hand, unlike the aforementioned images, for inspection images where the determination results of determination units 102A and 102C are more likely to be inappropriate, the weight value for the determination result of determination unit 102B increases.

[0194] Furthermore, for example, when the output value of the classification model represents the reliability of the examined image as a noise-free image, the weight setting unit 107 can set the weight for each determination result to a predetermined value, which corresponds to whether the reliability is above or below a predetermined threshold. For example, when the reliability is 0.8 or higher, the weight setting unit 107 can set the weights of determination units 102A and 102C to 0.4 and the weight of determination unit 102B to 0.2. In this case, when the reliability is less than 0.8, the weight setting unit 107 can set the weights of determination units 102A and 102C to 0.2 and the weight of determination unit 102B to 0.6.

[0195] The comprehensive weight determination unit 108 uses the weights set by the weight setting unit 107 and the reliability determined by the reliability determination unit 103 to calculate the comprehensive weight when combining the determination results of the determination units 102A to 102C. The comprehensive weight only needs to reflect the following two factors: the weights set by the weight setting unit 107 and the reliability determined by the reliability determination unit 103. For example, the comprehensive weight determination unit 108 can set the arithmetic mean of the weights set by the weight setting unit 107 and the reliability determined by the reliability determination unit 103 as the comprehensive weight.

[0196] (Processing flow)

[0197] based on Figure 12 This describes the process (determination method) executed by the information processing device 1C. Figure 12 This diagram illustrates an example of an inspection method using the information processing device 1C. Furthermore, it is set that: in Figure 12At the start of the processing, the ultrasonic image 111 is stored in the storage unit 11, and the inspection image generation unit 101 has generated an inspection image based on the ultrasonic image 111.

[0198] In S41, all the determination units 102, namely determination units 102A, 102B, and 102C, acquire the inspection image generated by the inspection image generation unit 101. Additionally, the weight setting unit 107 and the reliability determination unit 103 also acquire the inspection image. Furthermore, in S42, all the determination units 102 that acquired the inspection image in S41 use the inspection image to determine whether a defect exists.

[0199] In step S43 (acquisition step), the weight setting unit 107 inputs the inspection image acquired in step S41 into the classification model and acquires its output value. Then, in step S44, the weight setting unit 107 calculates the weights corresponding to the output values ​​acquired in step S43.

[0200] Specifically, when the output value obtained through S43 indicates that the inspected image is a noise-free image, the weight setting unit 107 assigns a greater weight to the determination results of determination units 102A and 102C than to the determination result of determination unit 102B. On the other hand, when the output value obtained through S43 indicates that the inspected image is a noisy image, the weight setting unit 107 assigns a greater weight to the determination result of determination unit 102B than to the determination results of determination units 102A and 102C.

[0201] In S45, the reliability determination unit 103 determines the reliability of the determination results of determination units 102A, 102B, and 102C. Furthermore, the processing in S45 can be executed before S42 to S44, or it can be executed in parallel with any of the processing in S42 to S44.

[0202] In S46, the comprehensive weight determination unit 108 calculates the comprehensive weight using the weights calculated in S44 and the reliability calculated in S45. For example, the weights of the determination units 102A to 102C are set to 0.2, 0.7, and 0.1, respectively, and the reliability is determined to be 0.3, 0.4, and 0.3, respectively. In this case, the comprehensive weight determination unit 108 can calculate the comprehensive weights of the determination units 102A to 102C to be 0.25, 0.55, and 0.2, respectively.

[0203] In S47, the comprehensive determination unit 104 uses the determination results from S42 and the comprehensive weight calculated in S46 to determine whether there is a defect. Furthermore, the determination using the comprehensive weight is the same as the determination using reliability as described in Embodiments 1 and 2. The comprehensive determination unit 104 then appends this determination result to the inspection result data 112.

[0204] A noise-free inspection image is an image deemed valid by the determination units 102A and 102C, the determination being based on the output value obtained by inputting the inspection image into a learned model generated through machine learning. Therefore, when the first method is configured to combine the defect determination results from multiple methods to make a final determination, it is preferable to include a method that uses a learned model generated through machine learning for determination. Furthermore, the method may also include a method that performs determination by numerically analyzing the pixel values ​​of the inspection image.

[0205] In this case, when applying the first method for inspecting noiseless images, the weight setting unit 107 preferably assigns the same or greater weight to the judgment results of the judgment units 102A and 102C using the learned model as to the judgment results of the judgment unit 102B performing numerical analysis. Essentially, the weight setting unit 107 simply needs to pre-set the weights for each judgment result to be equal and calculate the final judgment result based on the judgment reliability of the reliability judgment unit 103. On the other hand, when applying the second method for inspecting noisy images, the weight setting unit 107 preferably assigns the greater weight to the judgment results of the judgment unit 102B performing numerical analysis than to the judgment results of the judgment units 102A and 102C using the learned model.

[0206] Based on the above structure, when applying the first method for judging noise-free inspection images, the weight of the judgment result for the method using a learned model generated through machine learning is made larger or the same as the weight of the judgment result for the method using numerical analysis. Regarding noise-free inspection images, since the judgment using the learned model generated through machine learning is effective, high-precision judgment can be achieved.

[0207] Furthermore, according to the above structure, when applying the second method for judging noisy inspection images, the weight of the judgment result for the numerical analysis method is increased compared to the weight of the judgment result for the method using the learned model. For noisy inspection images, there are cases where the judgment using the learned model is ineffective; however, even in such cases, appropriate judgments can sometimes be made using numerical analysis. Therefore, the above structure increases the likelihood of obtaining appropriate judgment results.

[0208] Therefore, based on the above structure, appropriate judgments can be made regardless of whether the judgments of the learned model used to examine the image are valid.

[0209] Furthermore, as described above, the information processing apparatus 1C includes a reliability determination unit 103, which determines the reliability of each determination unit 102 based on the inspection image. The comprehensive determination unit 104 performs a determination using the determination results of each determination unit 102, the reliability determined by the reliability determination unit 103, and the weights set by the weight setting unit 107. With this structure, each determination result can be appropriately considered based on the inspection image, and a final determination result can be derived.

[0210] (Defect type determination)

[0211] In the above embodiments, examples of determining the presence or absence of defects have been described. It is also possible to configure the system to determine the type of defect in addition to determining whether a defect exists, or to replace the determination of whether a defect exists. For example, in Embodiment 1, an inspection image determined to be defective can be input into a type determination model for determining the type of defect, and the type of defect can be determined based on its output value. The type determination model can be constructed by using images reflecting known types of defects as teacher data for machine learning. Alternatively, the type can be determined by image analysis or the like, instead of using the type determination model. Alternatively, the determination unit 102 can also be configured to use a type determination model for determination.

[0212] In Embodiment 2, the determination units 102A to 102C may also use a type determination model for determination. In this case, the classification model used by the determination unit 102X can be any model that classifies defects based on their presence or absence, as well as the type of defect, in addition to the presence or absence of noise. During the learning of this model, the distance between feature quantities can be represented by Euclidean distance or by angle. Similarly, in Embodiment 3, the determination unit 102Y can determine the type of defect.

[0213] (Application Example)

[0214] In the above embodiments, an example of determining whether there are defects in the welded part of the pipe end based on the ultrasonic image 111 has been described. However, the setting of the determination item is arbitrary, as long as the object image used in the determination is also any image corresponding to the determination item, and is not limited to the example of the above embodiments.

[0215] For example, the information processing device 1 can also be applied to the inspection of an object to determine whether it has defects (also known as abnormalities) in a radiographic test (RT). In this case, the image generated by the abnormality is detected based on image data obtained using electronic equipment such as an imaging plate instead of radiographic photography. Thus, the information processing devices 1, 1A, 1B, and 1C can be applied to various non-destructive inspections using various data. Furthermore, in addition to non-destructive inspections, the information processing devices 1, 1A, 1B, and 1C can also be applied to object detection based on still or moving images, and to the classification of inspected objects.

[0216] (Variation Example 1)

[0217] In the above embodiments, an example of using the output value obtained by inputting the inspection image into the reliability prediction model as reliability has been described. However, the reliability can be derived based on the data used by the determination unit 102 in the determination, and is not limited to this example.

[0218] For example, when the determination unit 102B uses a binarized image (obtained by binarizing the inspection image) to determine whether a defect exists, the reliability prediction model used by the determination unit 102B can be a model that takes the binarized image as input data. On the other hand, if the determination unit 102C directly uses the inspection image to determine whether a defect exists, the reliability prediction model used by the determination unit 102C can be a model that takes the inspection image as input data. In this way, the data input to the reliability prediction models used by each determination unit 102 does not need to be completely identical.

[0219] Furthermore, in Embodiment 1, an example using three decision units 102 was described, but there can also be two or more decision units 102. Also, in Embodiment 1, the decision methods of the three decision units 102 are different, but the decision methods of the decision units 102 can also be the same. For decision units 102 with the same decision method, it is sufficient that the threshold used in that decision or the teacher data used to construct the learned model in that decision is different. Similarly, in Embodiments 2 and 4, it is sufficient that the total number of decision units 102 used is two or more.

[0220] Furthermore, the entity performing each process described in the above embodiments can be appropriately changed. For example, it can also be performed by other information processing devices: Figure 6The flowchart shows all or part of S12 (classification based on noise presence or absence), S14 (determination by each determination unit 102), S15 (reliability determination), and S16 (comprehensive determination). Similarly, the processing performed by determination units 102A to 102C can also be performed by other information processing devices, either part or all of it. In addition, in these cases, there can be one or more other information processing devices.

[0221] Thus, the function of information processing device 1 can be implemented through various system architectures. Furthermore, in the case of constructing a system containing multiple information processing devices, some of the information processing devices can be configured in the cloud. That is to say, the function of information processing device 1 can also be implemented using one or more information processing devices that perform information processing online. The same applies to information processing devices 1A, 1B, and 1C.

[0222] (Variation Example 2)

[0223] The learned models described in the above embodiments can also be constructed using pseudo-data or synthetic data that approximates the inspection images, instead of the actual inspection images. The pseudo-data or synthetic data can be generated, for example, using a generative model built through machine learning, or by manually synthesizing images. Furthermore, when constructing the learned model, these data can be augmented to improve decision-making performance.

[0224] (Software-based implementation example)

[0225] The functions of the information processing devices 1, 1A, 1B, and 1C (hereinafter referred to as "devices") are implemented through a program that enables the computer to function as the device. That is, the program is a determination program that enables the computer to function as each control module of the device (especially each part included in the control unit 10).

[0226] In this case, the aforementioned device includes a computer as hardware for executing the aforementioned program, the computer having at least one control device (e.g., a processor) and at least one storage device (e.g., a memory). By executing the aforementioned program using the control device and the storage device, the functions described in the above embodiments are realized.

[0227] The program described above is not temporary and can be stored in one or more computer-readable storage media. The device may or may not have such a storage medium. In the latter case, the program can be supplied to the device via any wired or wireless transmission medium.

[0228] Furthermore, some or all of the functions of the aforementioned control modules can also be implemented using logic circuits. For example, integrated circuits that form the logic circuits that enable the functions of the aforementioned control modules are also included within the scope of this invention. In addition, the functions of the aforementioned control modules can also be implemented using, for example, a quantum computer.

[0229] This invention is not limited to the embodiments described above, and various modifications can be made within the scope of the claims. Embodiments obtained by appropriately combining the technical solutions disclosed in different embodiments are also included in the technical scope of this invention.

[0230] Explanation of reference numerals in the attached figures

[0231] 1, 1A, 1B, 1C - Information processing device; 102 (102A, 102B, 102C, 102X, 102Y) - Judgment unit; 103 - Reliability judgment unit; 104 - Comprehensive judgment unit (judgment unit); 105 - Classification unit (acquisition unit); 106 - Judgment method determination unit (acquisition unit); 107 - Weight setting unit (acquisition unit).

Claims

1. An information processing device, comprising: The acquisition unit acquires the output value obtained by inputting an object image into a classification model, the classification model being generated through learning in such a way that when multiple features extracted from a first group of images with common features are embedded into a feature space, the distance between the features is reduced; and The determination unit, based on the output value, applies a first method for the first image group or a second method for a second image group consisting of images not belonging to the first image group to determine prescribed determination items related to the object image. The specified criterion is whether there are any abnormal parts in the objects reflected in the object image. The first image group contains images of the object that do not contain suspected abnormal parts that look similar to the abnormal parts.

2. The information processing device according to claim 1, characterized in that, The first image group is a valid image group based on the output value, which is obtained by inputting the object image into a learned model generated through machine learning. The first method includes at least the process of using the learned model to determine the decision item. The second method includes at least a process of determining the determination item by numerically analyzing the pixel values ​​of the object image.

3. The information processing device according to claim 1, characterized in that, The first image group is a valid image group based on the output value, which is obtained by inputting the object image into a learned model generated through machine learning. In addition to using multiple methods to determine the aforementioned matters, the first method also combines the results of each determination to make a final determination. Among the plurality of methods, at least one is included: A method for determining the decision items using the learned model; as well as A method for determining the decision item based on the output value of the classification model. The second method determines the judgment item by performing numerical analysis on the pixel values ​​of the object image.

4. An information processing device, comprising: The acquisition unit acquires the output value obtained by inputting an object image into a classification model, the classification model being generated through learning in such a way that when multiple features extracted from a first group of images with common features are embedded into a feature space, the distance between the features is reduced; and The determination unit, based on the output value, applies a first method for the first image group or a second method for a second image group consisting of images not belonging to the first image group to determine prescribed determination items related to the object image. The specified criterion is whether there are any abnormal parts in the objects reflected in the object image. The first image group contains images of the object that do not include suspected abnormal parts that resemble the abnormal part in appearance. The second image group contains images of the object that include the suspected abnormal area. The output value of the classification model indicates that the object image belongs to the second image group, or belongs to the first image group and contains the abnormal part, or belongs to the first image group and does not contain the abnormal part. The first method includes at least a process for determining whether there are abnormal parts in the object based on the output value. The second method includes at least a process of determining whether there are abnormal parts in the object by numerically analyzing the pixel values ​​of the object image.

5. An information processing device, comprising: The acquisition unit acquires the output value obtained by inputting an object image into a classification model, the classification model being generated through learning in such a way that when multiple features extracted from a first group of images with common features are embedded into a feature space, the distance between the features is reduced; and The determination unit, based on the output value, applies a first method for the first image group or a second method for a second image group consisting of images not belonging to the first image group to determine prescribed determination items related to the object image. The first image group is a valid image group based on the output value, which is obtained by inputting the object image into a learned model generated through machine learning. In addition to using multiple methods to determine the matter separately, the determination unit also combines the results of each determination to make a final determination on the matter. Among the multiple methods, there are: a method for determining the decision item using the learned model, and a method for determining the decision item by numerically analyzing the pixel values ​​of the object image. The information processing device includes a weight setting unit, which sets the weights for each determination result when combining the various determination results. Regarding the weight setting unit... When applying the first method, the weight of the judgment result for the method using the learned model is equal to or greater than the weight of the judgment result for the method performing numerical analysis. When applying the second method, the weight of the judgment result for the method of numerical analysis is made greater than the weight of the judgment result for the method of using the learned model.

6. The information processing apparatus according to claim 5, characterized in that, The system includes a reliability determination unit, which processes the plurality of methods as follows: Based on the object image, it determines an index representing the probability of each determination result, i.e., reliability. The determination unit uses the determination results, the reliability determined by the reliability determination unit, and the weight set by the weight setting unit to determine the determination item.

7. A determination method, executed by an information processing device, comprising: The acquisition step involves acquiring the output value obtained by inputting the object image into a classification model, wherein the classification model is generated by learning in such a way that when multiple features extracted from a first group of images with common features are embedded into a feature space, the distance between the features is reduced; and The determination step involves applying a first method for the first image group or a second method for a second image group consisting of images not belonging to the first image group, based on the output value, to determine prescribed determination items related to the object image. The specified criterion is whether there are any abnormal parts in the objects reflected in the object image. The first image group contains images of the object that do not contain suspected abnormal parts that look similar to the abnormal parts.

8. A recording medium, a computer-readable recording medium, the recording medium recording a determination program for enabling a computer to function as the information processing apparatus of claim 1, the determination program being used to enable the computer to function as the acquisition unit and the determination unit.

Images