Determination device, determination method, and determination program

Abstract

Provided is a determination device that determines the presence or absence of a defective portion from a monochrome image obtained by imaging a surface of a determination target using a learned model on the assumption that a color image is input.SOLUTION: A determination device includes an image conversion unit that performs conversion of multiplying a monochrome image obtained by imaging a determination target object in accordance with the number of channels of an input of a learned model used to determine the presence or absence of a defective portion of the determination target object, and a determination unit that inputs the monochrome image after conversion by the image conversion unit to the learned model and determines the presence or absence of the defective portion of the determination target object on the basis of an output from the learned model.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a determination device, a determination method, and a determination program. [Background technology]

[0002] Patent Document 1 discloses a technique for detecting surface defects of an inspection object from an image of the surface of the inspection object using a convolutional neural network. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2023 / 282043 Summary of the Invention [Problem to be solved by the invention]

[0004] When capturing a color image of the surface of an object to be inspected for defects, false colors occur near edges where there are sudden changes in contrast, and the image becomes out of focus due to the effects of color interpolation. Color images are not suitable for capturing the contours or features of minute defects of around 0.5mm to 1mm, pattern defects, etc. Therefore, monochrome images are used when using images to detect surface defects on an object to be inspected.

[0005] However, it is not possible to input monochrome images into a trained model that requires input of images with multiple channel directions, such as color images.

[0006] The present disclosure has been made in consideration of the above points, and aims to provide a determination device, a determination method, and a determination program that determine the presence or absence of defective parts from a monochrome image of the surface of the object to be determined using a trained model that extracts features from a color image. [Means for solving the problem]

[0007] A determination device according to a first aspect of the present disclosure includes an image conversion unit that performs a conversion by multiplying a monochrome image of an object to be determined to match the number of input channels of a trained model used to determine whether or not the object to be determined has a defective portion, and a determination unit that inputs the monochrome image converted by the image conversion unit into the trained model and determines whether or not the object to be determined has a defective portion based on the output from the trained model.

[0008] A determination device according to a second aspect of the present disclosure is the determination device according to the first aspect, wherein the image conversion unit multiplies the monochrome image after masking an area other than a determination target area.

[0009] A determination device according to a third aspect of the present disclosure is the determination device according to the first or second aspect, wherein the object to be determined is a casting or an object manufactured using the Gigacast method.

[0010] A determination method according to a fourth aspect of the present disclosure includes a processor that performs a conversion of a monochrome image of an object to be determined by multiplying the image to match the number of input channels of a trained model used to determine whether or not the object to be determined has a defective portion, inputs the converted monochrome image to the trained model, and performs a process of determining whether or not the object to be determined has a defective portion based on the output from the trained model.

[0011] A determination program according to a fifth aspect of the present disclosure causes a computer to perform a process of converting a monochrome image of an object to be determined by multiplying the image to match the number of input channels of a trained model used to determine whether or not the object has a defective portion, inputting the converted monochrome image to the trained model, and determining whether or not the object has a defective portion based on the output from the trained model. [Effects of the Invention]

[0012] According to the present disclosure, by multiplying a monochrome image to match the number of channels of a trained model that extracts features from a color image, it is possible to provide a determination device, a determination method, and a determination program that determine the presence or absence of defects from a monochrome image of the surface of the object to be determined using the trained model. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a diagram illustrating an overview of a determination device according to an embodiment of the disclosed technology. [Figure 2] FIG. 2 is a block diagram showing a hardware configuration of the determination device. [Figure 3] FIG. 2 is a block diagram illustrating an example of a functional configuration of a determination device. [Figure 4] FIG. 10 is a diagram illustrating an outline of monochrome image conversion by an image conversion unit. [Figure 5] 10 is a graph showing an evaluation of the determination result by the determination unit. [Figure 6] 10 is a flowchart showing the flow of a determination process performed by the determination device. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of the present disclosure will be described below with reference to the drawings. The same reference numerals are used throughout the drawings to designate identical or equivalent components and parts. The dimensional proportions of the drawings are exaggerated for illustrative purposes and may differ from the actual proportions.

[0015] FIG. 1 is a diagram showing an overview of a determination device according to this embodiment. The determination device 10 according to this embodiment is a device that determines whether or not an object to be determined has a defective portion using a monochrome image 1 in which the object to be determined is captured. The object to be determined is captured by an inspection device (not shown) that inspects the object to be determined. The monochrome image 1 is used as the image in which the object to be determined is captured because it does not produce false colors and is clearly focused compared to a color image.

[0016] The determination device 10 according to this embodiment uses a trained model trained on image data to determine whether or not an object to be determined has a defect. The trained model is a model trained on a teacher-labeled image database such as ImageNet. Because such an image database is composed of color images, a trained model trained on an image database also assumes that the input is a color image. A color image has multiple channel directions, such as R (red), G (green), and B (blue). Therefore, because a trained model trained on an image database assumes the input of an image with multiple channel directions, a monochrome image 1 with only one channel direction cannot be input to a trained model trained on an image database. In other words, a trained model trained on an image database cannot be used as a feature extractor from a monochrome image 1.

[0017] Furthermore, when generating a trained model by training monochrome images, it is necessary to prepare a large dataset, and training a large dataset requires a huge computational cost, detailed knowledge of hyperparameters, and a huge amount of time for training. Therefore, in order to determine the presence or absence of defects in an object to be determined from monochrome image 1, it is desirable to use an existing trained model that has been trained using an existing dataset.

[0018] Therefore, the determination device 10 according to this embodiment performs a conversion process on the monochrome image 1 in which the object to be determined is captured, multiplying the image to match the number of input channels of the trained model, inputs the converted monochrome image 1 to the trained model, and determines the presence or absence of a defect in the object to be determined using the output from the trained model. By performing the conversion process to match the number of input channels of the trained model in this way, the determination device 10 according to this embodiment can determine the presence or absence of a defect in the object to be determined from the monochrome image 1 using an existing trained model that has been trained using an existing dataset.

[0019] FIG. 2 is a block diagram showing the hardware configuration of the determination device 10. As shown in FIG.

[0020] 2, the determination device 10 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0021] The CPU 11 is a central processing unit that executes various programs and controls each component. That is, the CPU 11 reads a program from the ROM 12 or storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls the above components and performs various arithmetic processing in accordance with the program recorded in the ROM 12 or storage 14. In this embodiment, the ROM 12 or storage 14 stores a determination program that determines whether or not a defect exists in the object to be determined from the monochrome image 1.

[0022] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory, and stores various programs including the operating system and various data.

[0023] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0024] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may also function as the input unit 15 by adopting a touch panel system.

[0025] The communication interface 17 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0026] When executing the above-mentioned determination program, the determination device 10 uses the above-mentioned hardware resources to realize various functions. The functional configuration realized by the determination device 10 will be described.

[0027] FIG. 3 is a block diagram showing an example of the functional configuration of the determination device 10. As shown in FIG.

[0028] As shown in FIG. 3, the determination device 10 has, as its functional components, an image conversion unit 101 and a determination unit 102. Each functional component is realized by the CPU 11 reading and executing a determination program stored in the ROM 12 or the storage 14. Also as shown in FIG. 3, the determination device 10 has a trained model 110. The trained model 110 is, for example, a pre-trained convolutional neural network. The trained model 110 is recorded, for example, in the storage 14. Note that while FIG. 3 shows a case where the trained model 110 is included inside the determination device 10, the trained model that performs determination may be included in a device different from the determination device 10.

[0029] The image conversion unit 101 performs conversion by multiplying the monochrome image 1, which is an image of the object to be determined, to match the number of input channels of the trained model 110 that the determination unit 102 uses to determine whether or not there is a defect in the object to be determined. The object to be determined is, for example, a casting, an object manufactured using Gigacast technology, etc. If the number of input channels of the trained model 110 is three, R (red), G (green), and B (blue), the image conversion unit 101 performs conversion by tripling the monochrome image 1. The trained model 110 that the determination unit 102 uses for determination is a model that has been trained in advance using a large amount of color image data so that it can determine whether or not there is a defect in the object to be determined.

[0030] FIG. 4 is a diagram illustrating an overview of monochrome image conversion by the image conversion unit 101. The trained model 110 is premised on the input of, for example, a three-channel image of 224 pixels by 224 pixels. Therefore, the image conversion unit 101 converts a one-channel monochrome image 1 into a monochrome image 1' by tripling it so that it can be input to the trained model 110. In other words, the image conversion unit 101 performs a process of converting the one-channel monochrome image 1 into a three-channel monochrome image 1'. By converting the one-channel monochrome image 1 into a three-channel monochrome image 1', it becomes possible to input the monochrome image 1' into the trained model 110.

[0031] In the present disclosure, a trained model for a CMYK image with four input channels may be used, instead of a trained model trained on an RGB image. In this case, the image conversion unit 101 multiplies the monochrome image by four. Alternatively, the image conversion unit 101 may multiply the monochrome image by N, depending on the number of channels N of the image when the model was trained.

[0032] The determination unit 102 determines whether or not the object to be determined has a defective portion using the monochrome image 1' converted by the image conversion unit 101. Specifically, the determination unit 102 inputs the monochrome image 1' converted from the monochrome image 1 by the image conversion unit 101 into the trained model 110, and determines whether or not the object to be determined has a defective portion based on the output obtained from the trained model 110. The defective portion determined by the determination unit 102 is, for example, a defective portion that occurred during manufacturing, such as a chip, a hole, wrinkles caused by hot water, water residue, a dent, a chipped edge, discoloration, hot water running, burn-in, galling, or foreign matter contamination.

[0033] Fig. 5 is a graph showing an evaluation of the determination result by the determination unit 102. The vertical axis of the graph shown in Fig. 5 is a receiver operating characteristic (ROC) curve, with the true positive rate (TPR) on the vertical axis and the false positive rate (FPR) on the horizontal axis. The graph in Fig. 5 shows what kind of determination the determination unit 102 obtains from the trained model 110 when the threshold for determining NG is changed, in a case where the determination result is a binary value of OK (no defect) or NG (defect).

[0034] As shown in FIG. 5, when the threshold value is less than a predetermined value A, there is a tendency for overdetection, whereby an object is judged as NG even when it is actually OK, whereas when the threshold value is equal to or greater than A, there is a tendency for overdetection, whereby an object is judged as OK even when it is actually NG. Therefore, in this case, by setting the threshold value to A, the determination unit 102 can determine the presence or absence of a defective portion using the trained model 110, which is based on the input of a color image, even when the monochrome image 1' converted from the monochrome image 1 is input to the trained model 110. It goes without saying that the threshold value A can vary depending on the dataset used to train the trained model 110, and can be set appropriately depending on the results of the training.

[0035] The determination device 10 according to this embodiment has such a configuration, and by performing a conversion in which the image conversion unit 101 multiplies the monochrome image 1 to match the number of input channels of the trained model 110, it becomes possible to make a determination using the trained model 110 that has been trained in advance using a large amount of color image data.

[0036] The monochrome image 1 capturing the object to be determined may contain regions that are not subject to determination. For such an image, inputting the regions that are not subject to determination into the trained model 110 is a wasteful determination process.

[0037] Therefore, the image conversion unit 101 may perform a multiplication process after masking the areas other than the determination target area in the monochrome image 1 obtained by capturing an image of the object to be determined. When masking the areas other than the determination target area in the monochrome image 1, the image conversion unit 101 may mask the areas not to be determined using a predetermined pattern, or may detect edge portions of the monochrome image 1 to distinguish between the determination target area and the non-determination area, and then mask the non-determination area.

[0038] The masking pattern may be generated from a master image prepared in advance. The image conversion unit 101 may simply superimpose the pattern onto the monochrome image 1. However, the master image and the monochrome image 1 capturing the target object may have different imaging angles, imaging ranges, etc. Therefore, the image conversion unit 101 may scan a scanning range set in any number and size within the monochrome image 1 to search for a location matching the master image, perform affine transformation on the pattern to match that location, and superimpose the pattern transformed by the affine transformation onto the monochrome image 1, thereby masking the non-target area. Alternatively, the image conversion unit 101 may determine the masking target using semantic segmentation, which labels each pixel of the image. Semantic segmentation enables masking without the need to generate a mask pattern in advance.

[0039] Next, the operation of the determination device 10 will be described.

[0040] 6 is a flowchart showing the flow of the determination process by the determination device 10. The CPU 11 reads out a determination program from the ROM 12 or the storage 14, loads it into the RAM 13, and executes it, thereby performing the determination process.

[0041] In step S101, the CPU 11 acquires a monochrome image 1 of an object to be determined from an inspection device.

[0042] Following step S101, in step S102, CPU 11 performs a conversion to multiply monochrome image 1, which is an image of the object to be determined, in accordance with the number of input channels of trained model 110 used to determine whether or not the object to be determined has a defective portion. If the number of input channels of trained model 110 is three, R (red), G (green), and B (blue), CPU 11 performs a conversion to triple monochrome image 1.

[0043] Following step S102, in step S103, CPU 11 inputs the converted monochrome image 1 into trained model 110, and determines whether or not the object to be determined has a defective portion based on the output obtained from trained model 110.

[0044] By executing such processing, the judgment device 10 of this embodiment performs a conversion in which the image conversion unit 101 multiplies the monochrome image 1 to match the number of input channels of the trained model 110, thereby making it possible to make judgments using the trained model 110 that has been trained in advance using a large amount of color image data.

[0045] Although the embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. The above-described embodiments are illustrative and do not limit the technical scope of the present disclosure. It is clear that a person skilled in the art of the present disclosure can conceive of various modifications or alterations within the scope of the technical idea described in the claims, and it is understood that these modifications or alterations also naturally fall within the technical scope of the present disclosure.

[0046] Furthermore, the effects described in the above embodiments are explanatory or exemplary and are not limited to those described in the above embodiments. In other words, the technology according to the present disclosure may achieve other effects that are obvious to a person skilled in the art of the present disclosure from the description in the above embodiments, in addition to or instead of the effects described in the above embodiments.

[0047] In the above embodiments, the determination process executed by the CPU after reading the software (program) may be executed by various processors other than the CPU, such as a graphics processing unit (GPU). Examples of such processors include programmable logic devices (PLDs) whose circuit configuration can be changed after fabrication, such as field-programmable gate arrays (FPGAs), and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to execute specific processes. The determination process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0048] In addition, in each of the above embodiments, the determination processing program is described as being pre-stored (installed) in a ROM or storage, but this is not limiting. The program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network. [Explanation of symbols]

[0049] 10 Judgment device 101 Image conversion unit 102 Judgment section

Claims

1. an image conversion unit that performs conversion by multiplying a monochrome image of an object to be determined in accordance with the number of input channels of a trained model used to determine the presence or absence of a defective portion of the object to be determined; a determination unit that inputs the monochrome image converted by the image conversion unit into the trained model and determines whether or not there is a defective portion in the object to be determined based on an output from the trained model; and A determination device comprising:

2. The determination device according to claim 1 , wherein the image conversion unit multiplies the monochrome image after masking an area other than a determination target area.

3. The determination device according to claim 1 , wherein the object to be determined is a casting or an object manufactured using a gigacast method.

4. The processor: performing a conversion in which a monochrome image of an object to be determined is multiplied in accordance with the number of input channels of a trained model used to determine the presence or absence of a defective portion of the object to be determined; The converted monochrome image is input to the trained model, and the presence or absence of a defect in the object to be determined is determined based on the output from the trained model. A method of determining whether to perform processing.

5. On the computer, performing a conversion in which a monochrome image of an object to be determined is multiplied in accordance with the number of input channels of a trained model used to determine the presence or absence of a defective portion of the object to be determined; The converted monochrome image is input to the trained model, and the presence or absence of a defect in the object to be determined is determined based on the output from the trained model. A judgment program that executes processing.

Images