System and Program

Abstract

To support more appropriate image analysis of an object by performing learning for improving the visibility of only an attention region according to the purpose of image analysis of the object.SOLUTION: There is provided a system having one or more processors and one or more memory resources. The memory resource stores: an ROI (Region Of Interest) emphasis engine; a learning phase execution program; and an image processing phase execution program. The processor generates an ROI emphasis learning image in which only the ROI corresponding to an attention region of a processing image obtained by imaging the image processing target is emphasized by using the learning image obtained by imaging the learning target by executing the learning phase execution program, and performs learning for optimizing an internal parameter of the ROI emphasis engine so as to generate the ROI emphasis learning image when the learning image is input.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a system Muo and programs. [Background technology]

[0002] For example, in the fields of machinery, materials, food, biology, medicine, etc., observation, appearance inspection, measurement, etc. are performed by capturing an image of an object and analyzing the acquired image. Since it is desirable to use an image with high visibility when analyzing an image, methods have been proposed to improve visibility by improving the imaging system, such as capturing an image of an object with high resolution, or by image processing.

[0003] In addition, in recent years, the performance of machine learning has improved dramatically due to the proposal of a deep network model, and a method for improving the visibility of an image based on machine learning has been proposed. For example, Patent Document 1 describes, with respect to an image estimation method, that "in image estimation of a system having a memory unit and a calculation unit, the memory unit captures a first image of a first region of a first sample and a second image of the first region with a microscope and stores them. The calculation unit estimates an estimation processing parameter based on the first image and the second image, acquires a third image captured using a first imaging condition for a desired region of the first sample or the second sample, and estimates a fourth image of the desired region based on the third image and the estimation processing parameter. When estimating the estimation processing parameter, the calculation unit obtains the difference between the first image, the estimation image during learning, and the second image as an error, and compares the error with a preset threshold value to determine an opportunity to set the estimation processing parameter during learning as the estimation processing parameter." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-113769 A Summary of the Invention [Problem to be solved by the invention]

[0005] Patent Document 1 discloses a method for learning parameters for estimating a high-quality image from a degraded image by using pairs of degraded and high-quality images as learning data. In this learning method, parameters for estimating a high-quality image from the entire degraded image are learned. On the other hand, depending on the purpose of image analysis, there are cases where it is desired to improve the visibility of only a part of an attention area of ​​an image.

[0006] For example, when observing an object, it may be better to improve the visibility of only a specific part of the object or a specific position in the image to obtain an image suitable for observation. When performing an external inspection of an object, it is required to improve the visibility of only the defective part in order to prevent non-defective parts other than the defective part from being erroneously recognized as defective. When measuring a specific part of an object, it may be effective to improve the visibility of only the outline of the part to be measured. However, the technology described in Patent Document 1 does not take into consideration improving the visibility of only the region of interest.

[0007] The present invention has been made in consideration of the above-mentioned problems, and aims to support more appropriate image analysis of objects by performing learning to improve the visibility of only the area of ​​interest depending on the purpose of the image analysis of the object. [Means for solving the problem]

[0008] The present application includes a plurality of means for solving at least a part of the above problems, examples of which are as follows: A system according to one aspect of the present invention for solving the above problems is a system having one or more processors and one or more memory resources, the memory resources storing an ROI emphasis engine, a learning phase execution program, and an image processing phase execution program, the processor executing the learning phase execution program to generate an ROI emphasis training image in which only an ROI (Region Of Interest) corresponding to an attention region of a processing image in which an image of a target for image processing is captured is emphasized, using a training image in which an object for training is captured, and to perform learning to optimize internal parameters of the ROI emphasis engine so that the ROI emphasis training image is generated when the training image is input. Effect of the Invention

[0009] According to the present invention, by performing learning to improve the visibility of only the attention area according to the purpose of the image analysis of the object, it is possible to support more appropriate image analysis of the object. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 illustrates an example of a schematic configuration of a processor system. [Diagram 2] 5A to 5C are diagrams illustrating an example of a processing sequence of a learning phase and an image processing phase according to the first embodiment. [Diagram 3] 11 is a diagram for explaining a method for specifying an ROI using design data of an object. FIG. [Figure 4] FIG. 13 is a diagram showing an example of a GUI for specifying an ROI, a type of image enhancement processing, and a degree of image enhancement. [Diagram 5] FIG. 13 is a diagram showing an example of a processing sequence for performing machine learning on two ROI emphasis engines. [Figure 6] FIG. 13 is a diagram showing an example of a GUI for specifying an ROI, a type of image enhancement processing, and a degree of image enhancement. [Figure 7]FIG. 11 is a diagram showing an example of a processing sequence of a learning phase and an image processing phase according to the second embodiment. [Figure 8] 11A and 11B are diagrams for explaining a method of designating an ROI based on a difference from a reference image. [Figure 9] FIG. 13 is a diagram showing an example of a processing sequence of a learning phase and an image processing phase according to the third embodiment. [Figure 10] 13 is a diagram for explaining a method of designating an ROI based on an area in which simulated defects are combined; FIG. [Figure 11] 13 is a diagram showing a processing sequence for determining pass / fail of an inspection object using a processed image and a comparison image. FIG. [Figure 12] FIG. 13 is a diagram showing an example of a processing sequence for performing machine learning on two ROI emphasis engines. [Figure 13] 13 is a diagram showing a processing sequence for determining pass / fail of an inspection object using a processed image and a comparison image. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[0012] First Embodiment The system (processor system) of this embodiment is a system that performs image processing in which only a specified region of interest, ROI (Region Of Interest), is highlighted in a processed image of an object (subject), and outputs an ROI-emphasized image in which only the visibility of the ROI is improved.

[0013] This system uses training images to generate ROI-emphasized training images that emphasize only the ROI corresponding to the region of interest of a processed image that captures the target of image processing, and performs machine learning on the ROI emphasis engine so that the training images are used as input and an ROI-emphasized training image is output.

[0014] In addition, this system inputs a processed image of an object into an ROI emphasis engine, and outputs an ROI emphasis processed image in which only the ROI in the processed image is emphasised.

[0015] As a result, the present system can obtain an image in which only the ROI that is suitable for the purpose is highlighted.

[0016] The learning phase in which machine learning of the ROI emphasis engine is performed, and the image processing phase in which the ROI emphasis processed image is output will be described in detail later.

[0017] As mentioned above, ROI is the area that the user focuses on in image analysis, and it can be any area depending on the purpose of the image analysis. Below is an example of ROI. *(A1): Area that contains the pattern to be detected, such as defects (foreign objects, scratches, etc.) *(A2): Area that contains the part or structure you want to recognize, such as the shape contour (edge) *(A3): A specific area to focus on, such as surface texture *(A4): Dark areas (dark areas) or areas with low contrast due to shading, materials, structure, etc. It should be noted that the ROI is not limited to (A1) to (A4), and any region can be set as the ROI depending on the purpose of the image analysis or the user's designation.

[0018] In addition, in the learning phase in which machine learning of the ROI emphasis engine is performed, it is necessary to specify the ROI in order to generate an ROI-emphasized learning image in which only the ROIs exemplified in (A1) to (A4) are emphasized and their visibility is improved. On the other hand, since the ROI differs depending on the user's analysis purpose, it is necessary to specify to the ROI emphasis engine 31 which region should be emphasized. Therefore, an example of a method for specifying the ROI is shown below. *(B1) User sets using GUI (Graphical User Interface) etc. *(B2) Take the difference using the reference image and set based on the difference value *(B3) Set based on the divided area obtained by image segmentation *(B4) Acquire correspondence by matching the design data and image of the object, and set it based on the area set on the design data. *(B5) Set based on the area to which image processing is applied The method of specifying an ROI is not limited to (B1) to (B5), and various methods including automatic and manual specification are possible.

[0019] In addition, (B1), (B2), (B4) and (B5) will be described in detail in the following embodiments. The designation method of (B3) is, for example, a method of designating an ROI for each divided region when an object shown in a learning image can be divided into multiple parts. The designation method of (B3) also includes, for example, a method of dividing a learning image into multiple equal parts and designating each divided region as an ROI.

[0020] The configuration of this system and the details of the processing according to this embodiment will be described below with reference to FIGS.

[0021] <Configuration of the Processor System (This System) 100> 1 is a diagram showing an example of a schematic configuration of a processor system 100. As shown in the figure, the system 100 is connected to an imaging device 10 so as to be able to communicate with each other, for example, via a communication cable or a predetermined communication network (for example, the Internet, a LAN (Local Area Network) or a WAN (Wide Area Network)).

[0022] <<Details of the imaging device 10>> The imaging device 10 is an imaging device 10 capable of capturing digital images or digital videos of the surface or interior of an object (subject). Specifically, the imaging device 10 is a CCD (Charge Coupled Device) camera, an optical microscope, a charged particle microscope, an ultrasonic inspection device, an X-ray inspection device, or the like. The imaging device 10 captures an image of an object and outputs (or transmits) the captured image to the processor system 100. Note that a plurality of imaging devices 10 may be connected to the system 100.

[0023] <<Details of the Processor System 100>> The processor system 100 executes the processes in the learning phase and the image processing phase by the processor 20 reading various programs stored in the memory resources.

[0024] The processor system 100 is a computer such as a personal computer, a tablet terminal, a smartphone, a server computer, or a cloud server, and is a system including at least one of these computers.

[0025] Specifically, the processor system 100 includes a processor 20, a memory resource 30, an NI (Network Interface Device) 40, and a UI (User Interface Device) 50.

[0026] The processor 20 is an arithmetic device that reads various programs stored in the memory resource 30 and executes processing corresponding to each program. Examples of the processor 20 include a microprocessor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field Programmable Gate Array), or other semiconductor devices capable of performing calculations.

[0027] The memory resource 30 is a storage device that stores various information. Specifically, the memory resource 30 is a non-volatile or volatile storage medium such as a RAM (Random Access Memory) or a ROM (Read Only Memory). The memory resource may be a rewritable storage medium such as a flash memory, a hard disk, or an SSD (Solid State Drive), or a USB (Universal Serial Bus) memory, a memory card, or a hard disk.

[0028] The NI 40 is a communication device that performs information communication with an external device. The NI 40 performs information communication with an external device (e.g., the imaging device 10) via a predetermined communication network such as a LAN or the Internet. Unless otherwise specified below, information communication between the processor system 100 and the imaging device 10 is assumed to be performed via the NI 40.

[0029] The UI 50 is an input device that outputs instructions from a user (operator) to the processor system 100, and an output device that outputs information generated by the processor system 100. Examples of the input device include a keyboard, a touch panel, a pointing device such as a mouse, and a voice input device such as a microphone.

[0030] Moreover, the output device may be, for example, a display, a printer, a voice synthesizer, etc. Unless otherwise specified below, it is assumed that user operations on the processor system 100 (for example, inputting and outputting information, and issuing instructions to execute processing, etc.) are performed via the UI 50.

[0031] Furthermore, each component, function, processing means, etc. of the present system 100 may be implemented in hardware by designing part or all of them, for example, by means of an integrated circuit. Also, the present system 100 can implement part or all of each function by software, or can be implemented by the cooperation of software and hardware. Also, the present system 100 may use hardware having fixed circuits, or may use hardware in which at least a part of the circuits can be changed.

[0032] Also, the present system 100 can be realized by a user (operator) implementing part or all of the functions and processes realized by each program.

[0033] Note that each DB (database) in the memory resource 30 described below may be a file or other data structure other than a database as long as it is an area capable of storing data.

[0034] <<ROI Emphasis Engine 31>> The ROI emphasis engine 31 is a deep neural network typified by, for example, a CNN (Convolutional Neural Network). Note that the ROI emphasis engine 31 is not limited to a machine learning-based deep neural network, and for example, a rule-based engine can also be used.

[0035] The ROI emphasis engine 31 performs machine learning in the learning phase. Specifically, when a learning image is input, the ROI emphasis engine 31 optimizes internal parameters so as to output an ROI emphasis learning image obtained by performing image enhancement processing (for example, contrast enhancement processing, histogram equalization processing, and contour enhancement processing, etc.) only on the designated ROI of the learning image.

[0036] Also, in the image processing phase, when an imaging image of an object (hereinafter sometimes referred to as a processing image) is input, the ROI emphasis engine 31 generates an ROI emphasis processing image in which only the ROI is emphasized, and outputs this.

[0037] In addition, when a neural network is used for the ROI emphasis engine 31, the internal parameters of the engine include, for example, the network structure of the neural network, the activation function, hyperparameters such as the learning rate and the learning termination condition, and model parameters such as the weights (coupling coefficients) and biases between the nodes of the network.

[0038] Furthermore, when a rule-based engine is used as the ROI emphasis engine 31, the internal parameters of the engine include image processing parameters such as filter coefficients for various image processes and decision thresholds.

[0039] In addition, the ROI emphasis engine 31 may use a machine learning engine and a rule-based engine in combination.

[0040] <<Learning image DB32>> The training image DB 32 is a database that stores training images used in machine learning in the training phase. The training image DB 32 may store training images in advance, or may store training images captured by the imaging device 10 in the training phase.

[0041] <<Processed Image DB33>> The processed image DB 33 is a database that stores processed images of the object captured by the imaging device 10 during execution of the image processing phase.

[0042] <<Other information DB34>> The other information DB 34 is a database that stores various information used in the learning phase and the image processing phase. For example, the other information DB 34 stores design data of the object used in the ROI designation method (B4). The other information DB 34 also stores information on the pseudo-failures used in the ROI designation method (B5).

[0043] <<GUI Execution Program 35>> The GUI execution program 35 is a program that generates predetermined screen information to be output to the UI 50 (in this case, the display), and accepts input of information and execution instructions for processing from the user via the UI 50 (in this case, a pointing device such as a keyboard or mouse). Specifically, the GUI execution program 35 generates screen information for accepting specification of the ROI and the like, and outputs it to the display. Also, the GUI execution program 35 accepts specification of the ROI and input of other information from the user via the UI 50.

[0044] <<Learning Phase Execution Program 36>> The learning phase execution program 36 is a program that executes various processes in the learning phase. Specifically, the learning phase execution program 36 acquires learning images from the learning image DB 32, and generates ROI-enhanced learning images in which only the ROI is subjected to image enhancement processing. Also, the learning phase execution program 36 inputs the learning images to the ROI enhancement engine 31, and performs machine learning of the ROI enhancement engine 31 so that ROI-enhanced learning images are output.

[0045] <<Image Processing Phase Execution Program 37>> The image processing phase execution program 37 is a program that executes various processes related to the image processing phase. Specifically, the image processing phase execution program 37 acquires a processed image of the object from the processed image DB 33, and inputs this to the ROI enhancement engine 31 to acquire an ROI-enhanced processed image in which only the ROI is enhanced.

[0046] The details of the processor system 100 have been described above.

[0047] <Details of the Learning Phase and Image Processing Phase> FIG. 2 is a diagram showing an example of the processing sequence of the learning phase and the image processing phase.

[0048] In the learning phase, machine learning is performed by the ROI emphasis engine 31. The learning phase is started at a predetermined timing, for example, when the user instructs the processor system 100 to execute the learning phase via the UI 50.

[0049] When the learning phase starts, the processor system 100 executes the learning phase execution program 36. The learning phase execution program 36 acquires a learning image 120 obtained by capturing an image of the learning object 110 from the learning image DB 32 (step S10). Note that the learning phase execution program 36 may output an instruction to capture the learning image 120 of the learning object 110 to the imaging device 10 via the NI 40, and acquire the learning image 120 of the object captured by the imaging device 10 from the learning image DB 32.

[0050] Next, the learning phase execution program 36 executes machine learning of the ROI emphasis engine 31 (step S20). Specifically, the learning phase execution program 36 specifies an ROI (step S21).

[0051] Here, a method for specifying an ROI using design data of an object given by CAD (Computer-Aided Design) or the like (a method for specifying an ROI corresponding to the above-mentioned B4) will be described.

[0052] 3 is a diagram for explaining a method for specifying an ROI using design data of an object. This method involves matching a learning image 120 with design data 160, and specifying an area 162 where the learning image 120 is matched with a set area on the design data 160 as an ROI 121.

[0053] Specifically, the learning phase execution program 36 acquires design data of the object from, for example, the other information DB 34. Note that a dashed line 161 in the illustrated design data indicates the contour line of the design shape of the object.

[0054] Furthermore, the learning phase execution program 36 matches the learning image 120 onto the design data 160 based on the respective feature points of the design data and the learning image 120. Then, the learning phase execution program 36 determines the area on the design data 160 to which the learning image 120 has been matched (in FIG. 3, the area 162 to which the learning image 120 has been matched) as the ROI 121, and specifies the area 162 as the ROI 121.

[0055] Next, the learning phase execution program 36 generates an ROI-emphasized learning image based on the specified ROI 121 (step S022). Specifically, the learning phase execution program 36 performs image enhancement processing such as contrast enhancement processing on the specified ROI 121 to generate an ROI-emphasized learning image in which only the ROI 121 is emphasized. In the example shown in FIG. 3, a set region 163 in which only a region 162 in which the learning image 120 is matched on the design data 160 is emphasized is generated as the ROI-emphasized learning image. The set region 163 may be set by the user via the UI 50, or may be set according to a predetermined rule.

[0056] Next, the learning phase execution program 36 performs machine learning of the ROI emphasis engine 31 using the learning image 120 and the ROI emphasis training image (step S023). Specifically, the learning phase execution program 36 performs machine learning to optimize the internal parameters of the ROI emphasis engine 31 so that the generated ROI emphasis training image is output when the learning image 120 is input.

[0057] In the learning phase, the processes of steps S10 to S23 are repeatedly performed using a plurality of (for example, 10 to 100) learning images, thereby performing machine learning of the ROI emphasis engine 31. Also, in the ROI emphasis engine 31 described in the embodiment described later, the machine learning is performed by performing the learning phase multiple times.

[0058] Next, the image processing phase will be described. In the image processing phase, a ROI-emphasized processed image in which only the ROI in the processed image 140 corresponding to the ROI 121 is emphasized is output using the processed image 140 obtained by capturing an image of the object 130. The image processing phase is started at a predetermined timing, for example, when the user outputs an instruction to execute the image processing phase to the processor system 100 via the UI 50.

[0059] When the image processing phase is started, the processor system 100 executes the image processing phase execution program 37. The image processing phase execution program 37 acquires a processed image of the object 130 (in the example of FIG. 3, an image of the object 130 corresponding to the set area 163 on the design data) from the processed image DB (step S30). Note that the image processing phase execution program 37 may output an instruction to the imaging device 10 via the NI 40 to capture the processed image of the object 130, and acquire the processed image of the object captured by the imaging device 10 from the processed image DB 33.

[0060] Next, the image processing phase execution program 37 uses the ROI emphasis engine 31 to obtain an ROI emphasis processed image in which only the ROI is emphasised (step S40). Specifically, when the processed image 140 is input by the image processing phase execution program 37, the ROI emphasis engine 31 identifies the ROI corresponding to the ROI learned in the learning phase (in the example of FIG. 3, region 121) from the processed image (step S41).

[0061] Furthermore, the ROI emphasis engine 31 performs image processing to emphasize only the identified ROI, and generates an ROI emphasis processed image 150 (step S42). Furthermore, the image processing phase execution program 37 acquires the ROI emphasis processed image 150 output from the ROI emphasis engine 31. Note that in the example shown in Fig. 3, the ROI emphasis processed image 150 is output in which only the ROI corresponding to the ROI 121 in the region 162 is emphasized in the image (processed image) of the object 130 corresponding to the set region 163 on the design data.

[0062] In this way, according to the present system 100, by performing image processing so that only the region of interest (ROI) is highlighted according to the purpose of the image analysis of the object, it is possible to obtain an ROI highlighted image in which only the visibility of the ROI is improved. As a result, the image analysis of the object can be performed more appropriately.

[0063] In particular, when a matching method using design data is used to specify the ROI, the learning image can be automatically matched onto the design data, so that machine learning of the ROI emphasis engine 31 using multiple learning images can be performed efficiently and easily.

[0064] Furthermore, when a processed image is input to the ROI emphasis engine 31 that has performed such machine learning, the ROI is identified with high accuracy from the processed image without being easily affected by deviations in the imaging range or imaging position of the object, and an ROI emphasis processed image in which only the ROI is emphasised is generated.

[0065] Next, regarding the designation of the ROI in step S21, a method in which the user designates the ROI in the learning image, the type of image enhancement processing, and the degree of image enhancement using a GUI (a method of designating the ROI corresponding to B1 above) will be described.

[0066] As mentioned above, there are various targets for ROI, so it is required to be able to specify the ROI with a high degree of freedom depending on the purpose. Also, contrast enhancement, histogram equalization, and edge enhancement are given as examples of types of image enhancement, but it is desirable to be able to specify the type of these with a high degree of freedom depending on the purpose. Furthermore, depending on the purpose of image analysis, a weak degree of image enhancement is sometimes better, and a strong degree is sometimes better, so it is desirable to be able to specify the degree with a high degree of freedom.

[0067] Therefore, in the ROI specification method described below, the user specifies the ROI, the type of image enhancement processing, and the degree of image enhancement via a GUI, making it possible to obtain an ROI-enhanced image that matches the purpose of image analysis of the target object.

[0068] 4 is a diagram showing an example of a GUI 170 for specifying an ROI, a type of image enhancement processing, and a degree of image enhancement. As shown in the figure, the GUI 170 displays an image ID selection button 171 for selecting a learning image, an area 172 for displaying the selected learning image, an area 173 for displaying an ROI specified by a user via the GUI, an area 174 for displaying an ROI-emphasized learning image in which only the ROI is emphasized, an area 175 for specifying a type of image enhancement processing, and an area 176 for specifying a degree of image enhancement.

[0069] To display such a GUI, the processor 20 loads the GUI execution program 35. Then, the GUI execution program 35 generates a predetermined GUI 170 shown in Fig. 4 and outputs it to the UI 50, which is a display.

[0070] Furthermore, the GUI execution program 35 accepts user designation of the ROI, the type of image enhancement processing, and the degree of image enhancement via the GUI 170 displayed on the display. The user selects, for example, the image ID of a learning image using the UI 50, which is a pointing device such as a keyboard or a mouse. When the GUI execution program 35 acquires input information indicating an image ID, it acquires the learning image of the corresponding ID from the learning image DB 32 and displays it in the area 172 of the GUI 170 (step S10).

[0071] In addition, when the user selects a portion 177 to be designated as an ROI in the learning image displayed on the GUI 170 using the UI 50 (in the example of FIG. 4, the user traces the portion 177 or surrounds it with a frame such as a rectangle), the GUI execution program 35 displays the designated ROI 177 in the area 173 based on the input information (step S21). In the example of FIG. 4, the white pixel portion of the area 173 indicates the ROI designated by the user.

[0072] Furthermore, when the user specifies the type and degree of image enhancement displayed on the GUI 170 using the UI 50, the learning phase execution program 36 acquires the input information via the GUI execution program 35, and generates an ROI-emphasized learning image by performing the specified type of image enhancement at the specified degree of image enhancement for the ROI 177 (step S22). Furthermore, the GUI execution program 35 displays the ROI-emphasized learning image generated by the learning phase execution program 36 in the GUI area 174.

[0073] The learning phase execution program 36 performs machine learning of the ROI emphasis engine 31 using the ROI emphasis learning image thus generated (step S23).

[0074] Furthermore, in the image processing phase, when a processed image is input to the ROI emphasis engine 31 that has undergone such machine learning, an ROI emphasis processed image in which a specified degree of image emphasis and a specified type of image emphasis processing have been performed only for the ROI specified in the learning phase is output (steps S30, S40). That is, the ROI emphasis processed image output through image processing by the ROI emphasis engine becomes, for example, something like the ROI emphasis learning image displayed in area 174 in FIG.

[0075] The other processes in the learning phase and image processing phase are the same as those described above, and therefore detailed description thereof will be omitted.

[0076] In this way, according to the present system 100, it is possible to specify the ROI, the type of image enhancement processing, and the degree of image enhancement with a high degree of freedom according to the purpose of image analysis using the GUI. This makes it possible to obtain an ROI-enhanced image according to the purpose of image analysis of the target object.

[0077] Next, a case will be described in which machine learning is performed on a plurality of ROI enhancement engines 31 and mutually different ROI enhancement processed images are output using the plurality of ROI enhancement engines 31. In the learning phase, the present system 100 performs machine learning on the plurality of ROI enhancement engines 31 so as to output ROI enhancement training images in which at least one of the ROI, the type of image enhancement processing, or the degree of image enhancement is different. Also, in the present system 100, in the image processing phase, the plurality of ROI enhancement engines 31 output ROI enhancement processed images in which at least one of the ROI, the type of image enhancement processing, or the degree of image enhancement is different.

[0078] As described above, it may be desirable to specify the ROI, the type of image enhancement processing, and the degree of image enhancement according to the purpose in the machine learning of the ROI enhancement engine 31. Also, in the image analysis of the object, it may be desirable to use an image to which multiple types of ROIs and types of image enhancement processing are applied, rather than a single ROI or type of image enhancement processing.

[0079] Therefore, the present system 100 performs machine learning of multiple ROI enhancement engines 31, and uses the multiple ROI enhancement engines 31 to obtain multiple types of ROI enhancement processed images that differ in at least one of the following: ROI, type of image enhancement processing, or degree of image enhancement.

[0080] 5 is a diagram showing an example of a process sequence for performing machine learning on two ROI emphasis engines E1 and E2 in the learning phase. The learning phase execution program 36 accepts, via the GUI execution program 35, a specification of an ROI, a type of image emphasis processing, and a degree of image emphasis from a user for a learning image 120 acquired from the learning image DB 32 (step S21). In addition, the learning phase execution program 36 generates an ROI emphasis training image using input information about the ROI etc. acquired via the GUI execution program 35 (step S22), and performs machine learning on the ROI emphasis engines E1 and E2 using the training image 120 and the ROI emphasis training images 181 and 184 (step S23).

[0081] 6 is a diagram showing an example of a GUI 190 for specifying an ROI, a type of image enhancement processing, and a degree of image enhancement. As shown in the figure, an upper section 191 of the GUI is an area corresponding to an ROI enhancement engine E1, and a lower section 192 is an area corresponding to an ROI enhancement engine E2. In addition, an add button 193 is displayed in the GUI 190, which is pressed when an ROI enhancement engine that performs machine learning is to be further added. Note that the basic configuration of the GUI 190 shown in FIG. 6 is the same as that of the GUI 170 in FIG. 4, and therefore a detailed description thereof will be omitted.

[0082] 6, the GUI execution program 35 receives a specification of an ROI, a type of image enhancement processing, and a degree of image enhancement from a user (step S21). Specifically, the GUI execution program 35 acquires input information for specifying the portion 180 of the learning image as an ROI, "contour" as the type of image enhancement processing, and "strong" as the degree of image enhancement, for the ROI enhancement engine E1.

[0083] The GUI execution program 35 also acquires input information that specifies the portion 183 of the learning image as the ROI for the ROI enhancement engine E2, specifies "contrast" as the type of image enhancement processing, and specifies "strong" as the degree of image enhancement.

[0084] Based on the input information, the GUI execution program 35 also displays the designated ROI in the area 172 corresponding to each of the ROI emphasis engines E1 and E2.

[0085] The learning phase execution program 36 also generates ROI-emphasized learning images 181, 184 corresponding to the ROI emphasis engines E1, E2 by performing the designated type of image emphasis processing at the designated degree of image emphasis for each ROI (step S22). The GUI execution program 35 also displays the ROI-emphasized learning images 181, 184 generated by the learning phase execution program 36 in the areas 174 of the GUI 190 corresponding to the ROI emphasis engines E1, E2, respectively.

[0086] The learning phase execution program 36 performs machine learning of the ROI emphasis engines E1 and E2 using the ROI emphasis training images 181 and 184 generated in this manner (step S23). Specifically, when the training image 120 is input, the learning phase execution program 36 performs machine learning of the ROI emphasis engine E1 so as to generate the generated ROI emphasis training image 181. Similarly, when the training image 120 is input, the learning phase execution program 36 performs machine learning of the ROI emphasis engine E2 so as to generate the generated ROI emphasis training image 184.

[0087] In the image processing phase, when a processed image is input to the ROI enhancement engine E1, an ROI enhancement processed image is output in which the specified degree of image enhancement (in this case, "strong") and the specified type of image enhancement processing (in this case, "contour enhancement processing") have been performed only for the ROI specified in the learning phase.

[0088] Furthermore, when the processed image is input to the ROI enhancement engine E2, an ROI enhancement processed image is output in which the specified degree of image enhancement (in this case, "strong") and the specified type of image enhancement processing (in this case, "contrast enhancement processing") have been performed only for the ROI specified in the learning phase.

[0089] In this way, according to the present system 100, it is possible to apply multiple types of ROI, types of image enhancement processing, and degrees of image enhancement to obtain multiple types of ROI-enhanced images, thereby making it possible to perform more appropriate image analysis of the object.

[0090] Second Embodiment Next, a second embodiment will be described. In the learning phase, the system 100 according to this embodiment generates a difference image using a learning good product image and a learning defective product image, and specifies an ROI based on the difference image. In addition, the system 100 generates an ROI-emphasized learning image in which the specified ROI is emphasized, and performs machine learning of the ROI emphasis engine 31 so that, when a learning defective product image is input, an ROI-emphasized learning image is output.

[0091] In addition, in the image processing phase, the system according to this embodiment generates a comparison image in which the ROI is emphasized by inputting the processed image to the ROI emphasis engine 31, and judges whether the object is good or defective (defective / failure determination) by comparing the processed image with the comparison image.

[0092] Conventionally, quality judgement was mostly done by visual inspection of appearance using images. However, with the increasing demand for mass production and quality improvement, the inspection cost and the burden on inspectors are increasing. In addition, sensory inspection based on human senses requires a particularly high level of experience and skill, and there are also issues with subjectivity and reproducibility, such as different evaluation values ​​depending on the inspector and different results each time the inspection is performed.

[0093] There is a strong demand for automation of inspection to address issues such as the cost, skill, and dependency on individuals in such inspections. Therefore, for example, if a defective portion is designated as an ROI by the method described in the first embodiment, an image in which the defect is emphasized can be obtained, making inspection easy.

[0094] On the other hand, when a user specifies defects using a GUI in the learning phase, this places a heavy burden on the user. In particular, when a machine learning engine is used as the ROI emphasis engine 31, a large number of learning images are generally required, and specifying ROIs for defects in all of these learning images using a GUI would be a heavy time burden.

[0095] Therefore, the system of this embodiment provides a method for automatically specifying an ROI (a method for specifying an ROI corresponding to B2 described above) using a difference image calculated using a reference image, which is a training image of a good product, and a training image of a defective product.

[0096] The same objects and processes as those in the first embodiment are denoted by the same reference numerals and detailed description thereof will be omitted.

[0097] <Details of the learning phase and image processing phase> FIG. 7 is a diagram showing an example of a processing sequence of the learning phase and the image processing phase according to this embodiment.

[0098] When the learning phase is started, the learning phase execution program 36 acquires a learning good product image 203 and a learning defective product image 204 obtained by capturing images of the learning good product 200 and the learning defective product 201, respectively, from the learning image DB 32 (step S50).

[0099] The learning phase execution program 36 may output an instruction to the imaging device 10 via the NI 40 to capture images of the learning good product 200 and the learning defective product 201, and may acquire the learning good product image 203 and the learning defective product image 204 captured by the imaging device 10 from the learning image DB 32.

[0100] Next, the learning phase execution program 36 performs machine learning of the ROI emphasis engine 31 using the learning good product image 203 and the learning defective product image 204 (step S60). Specifically, the learning phase execution program 36 sets the learning good product image 203 as a reference image, and specifies an ROI that is a part in the learning defective product image 204 that is likely to be defective.

[0101] Here, regarding the designation of ROI, a method of taking a difference using a reference image and designating ROI based on the difference value (ROI designation method corresponding to B2 above) will be described.

[0102] 8 is a diagram for explaining a method for designating an ROI based on the difference from a reference image. This method designates an area with a large difference value from a learning good product image as an ROI with a high possibility of being defective.

[0103] Specifically, the learning phase execution program 36 aligns the learning good product image 203 with the learning defective product image 204, and generates a difference image 214 between the learning good product image 203 and the learning defective product image 204 using the learning good product image 203 as a reference (step S61). The learning phase execution program 36 also designates an area (part 215 in FIG. 8) in which the pixel value of the difference image 214, i.e., the difference value, is greater than a preset threshold value, as the ROI 216.

[0104] Returning to Fig. 7, the explanation will be given. Next, the learning phase execution program 36 generates an ROI-emphasized learning image based on the specified ROI 216 (step S63). Specifically, the learning phase execution program 36 performs image enhancement processing, such as contrast enhancement processing, on the specified ROI 216 in the learning defective product image 204 to generate an ROI-emphasized learning image in which only the ROI 216 is emphasized.

[0105] Next, the learning phase execution program 36 performs machine learning of the ROI emphasis engine 31 (step S64). Specifically, the learning phase execution program 36 performs machine learning to optimize the internal parameters of the ROI emphasis engine 31 so that a generated ROI emphasis training image is output when the training defective product image 204 is input.

[0106] Next, the image processing phase will be described. In the image processing phase, a comparison image (ROI emphasis processed image) 212 is generated (estimated) from a processed image 211 obtained by capturing an image of an object 210, thereby generating an image in which a region (ROI) that is likely to be defective is emphasized. In the image processing phase, the processed image 211 is compared with the comparison image 212 to determine whether the object is a good product or a defective product.

[0107] When the image processing phase is started, the image processing phase execution program 37 acquires a processed image 211 obtained by capturing an image of an object 210 to be inspected from the processed image DB 33 (step S70).

[0108] Next, the image processing phase execution program 37 uses the ROI emphasis engine 31 to obtain a comparison image 212, which is an ROI emphasis processed image in which only the ROI is emphasized (step S80). Specifically, when the processed image 211 is input to the ROI emphasis engine 31 by the image processing phase execution program 37, the ROI emphasis engine 31 identifies the ROI in the processed image (step S81).

[0109] Furthermore, the ROI emphasis engine 31 performs image processing to emphasize only the specified ROI, thereby generating a comparison image 212 which is an ROI emphasis processed image (step S82), and outputs this.

[0110] Next, the image processing phase execution program 37 uses the processed image 211 and the comparison image 212 to compare both images to determine whether the inspection object 210 is a good or bad item (good or bad determination) (step S90). Specifically, the image processing phase execution program 37 generates a difference image between the processed image 211 and the comparison image 212, and determines that the object is bad if there is a location where the pixel value of the difference image is greater than a preset threshold value.

[0111] In addition, if the image processing phase execution program 37 determines that the inspection object 210 is defective, it may perform processing such as outputting a processed image (defective product image) 211 to a specified external device via, for example, the NI 40, and prompting an inspector to confirm the defective product image (step S100).

[0112] In this way, according to the present system 100, it is possible to automatically specify an ROI from a defective image based on the difference from a reference image, and to efficiently execute machine learning of the ROI emphasis engine. Also, according to the present system 100, it is possible to judge whether an object to be inspected is good or bad based on the difference between a processed image and a comparison image in which the ROI is emphasized. As a result, it is possible to realize the automation of inspection, addressing issues such as inspection cost, skill, and personal dependency.

[0113] <Third embodiment> Next, a third embodiment will be described. In the learning phase, the system according to this embodiment generates a pseudo-failure image by combining a pseudo-failure with a learning good product image, and designates the area where the pseudo-failure is combined as an ROI. In addition, the system 100 generates an ROI-emphasized learning image in which the ROI of the pseudo-failure image is emphasized, and performs machine learning of the ROI emphasis engine 31 so that the ROI-emphasized learning image is output when the pseudo-failure image is input.

[0114] In addition, in the image processing phase, the system according to this embodiment generates a comparison image in which the ROI is emphasized by inputting the processed image into the ROI emphasis engine 31, and judges whether the object is good or defective (defective / failure determination) by comparing and inspecting the processed image and the comparison image.

[0115] In the system according to the second embodiment, machine learning of the ROI emphasis engine 31 is performed using learning defective product images, but there is a problem in that collecting images of defective products as objects requires a large cost.

[0116] Therefore, it is desirable to perform machine learning of the ROI emphasis engine 31 using only images of good products, and to be able to determine whether an object is good or defective. A neural network learning method is known in which, using images of good products, an image of a good product is output when an image of a defective product is input. However, with this method, even if an image of a good product can be correctly output for defects in which the luminance value of a defective part such as a scratch or uneven color is close to that of a good product, a problem occurs in that it is difficult to accurately detect the defect in a comparative inspection between an image including the defect and an image of a good product because the difference value is small.

[0117] Therefore, the system according to this embodiment provides a method for designating an ROI based on an area that is created by combining pseudo-defects, such as scratches and color unevenness, whose luminance values ​​are close to those of a good product, using image processing (a method for designating an ROI corresponding to B5 described above).

[0118] Note that the same objects and processes as those in the above-described embodiment are denoted by the same reference numerals and detailed description thereof will be omitted.

[0119] <Details of the learning phase and image processing phase> FIG. 9 is a diagram showing an example of a processing sequence of the learning phase and the image processing phase according to this embodiment.

[0120] When the learning phase is started, the learning phase execution program 36 acquires a learning good-product image 203 obtained by capturing an image of the learning good product 200 from the learning image DB 32 .

[0121] Next, the learning phase execution program 36 performs machine learning of the ROI emphasis engine 31 (step S120). Specifically, the learning phase execution program 36 combines the pseudo-defect with the learning good product image 203 (step S121). More specifically, the learning phase execution program 36 combines the pseudo-defect with the learning good product image 203, the brightness value of the defective part such as a scratch or color unevenness being close to that of the good product.

[0122] Next, the learning phase execution program 36 designates the region where the pseudo defects are combined as an ROI (step S122).

[0123] Here, regarding the designation of an ROI, we will explain a method of designating an ROI based on an area that is created by combining pseudo-defects, such as scratches and color unevenness, whose luminance values ​​are close to those of a non-defective product, using image processing (a method of designating an ROI that corresponds to B5 described above).

[0124] 10 is a diagram for explaining a method for designating an ROI based on an area where a pseudo defect is synthesized. In this method, an area where a pseudo defect is synthesized is designated as an ROI.

[0125] Specifically, the learning phase execution program 36 synthesizes a pseudo defect 224, in which the luminance value of a defect portion such as a scratch or color unevenness is close to that of a good product, at a predetermined position on the learning good product image 203 (step S121), to generate a pseudo defect image 225. In addition, the learning phase execution program 36 designates the region where the pseudo defect 224 is synthesized as an ROI 226 (step S122).

[0126] Returning to Fig. 9, the explanation will be given. Next, the learning phase execution program 36 generates an ROI-emphasized learning image based on the specified ROI 226 (step S123). Specifically, the learning phase execution program 36 performs image enhancement processing such as contrast enhancement processing on the specified ROI 226, i.e., the portion of the synthesized pseudo defect 224, in the pseudo defect image 225, to generate an ROI-emphasized learning image in which only the ROI 226 is emphasized.

[0127] Next, the learning phase execution program 36 performs machine learning of the ROI emphasis engine 31 (step S124). Specifically, the learning phase execution program 36 performs machine learning to optimize the internal parameters of the ROI emphasis engine 31 so that, when the pseudo defect image 225 is input, a generated ROI emphasis learning image is output.

[0128] Next, the image processing phase will be described. In the image processing phase, a comparison image (ROI emphasis processed image) is generated (estimated) from the processed image, and the processed image and the comparison image are compared (inspected) to determine whether the object is a good or bad product. Note that steps S130, S140 to S142, and S160 are the same processes as steps S70, S80 to S82, and S100 according to the second embodiment, and therefore detailed descriptions thereof will be omitted.

[0129] In step S150, the image processing phase execution program 37 uses the processed image 221 and the comparison image 222 to compare the two images, thereby determining (distinguishing) whether the inspection target 220 is a non-defective or defective product.

[0130] Here, a pass / fail judgment of an inspection object 220 using a processed image 221 and a comparison image 222 will be described with reference to FIG.

[0131] 11 is a diagram showing a processing sequence for determining pass / fail of an inspection object 220 using a processed image 221 and a comparison image 222. As shown in the figure, the processed image 221 acquired in step S130 includes a portion 227 indicative of a relatively large foreign object or defect, and a portion 228 indicative of a defect such as a scratch or color unevenness whose brightness value is close to that of a good product.

[0132] The image processing phase execution program 37 generates a comparison image 222, which is an ROI emphasis processed image, by using such a processed image 221. Specifically, the image processing phase execution program 37 inputs such a processed image 221 to the ROI emphasis engine 31 to obtain an ROI emphasis processed image output from the ROI emphasis engine 31, and sets this as the comparison image 222.

[0133] Since the ROI emphasis engine 31 performs machine learning by designating a defective region having a brightness value close to that of a non-defective product as an ROI, when a processed image 221 is input to the ROI emphasis engine 31, an ROI emphasis processed image (comparison image 222) is output in which only defective region 228, such as scratches and color unevenness, whose brightness value is close to that of a non-defective product, is emphasized. On the other hand, for a relatively large defective region 227, such as a foreign object or defect, an image like the learning non-defective product image 203 is output, so that the comparison image 222 does not show a portion showing such a foreign object or the like.

[0134] Next, the image processing phase execution program 37 uses the processed image 221 and the comparison image 222 to compare both images, thereby determining (distinguishing) whether the inspection target 220 is a good product or a defective product. Specifically, the image processing phase execution program 37 generates a difference image 229 between the processed image 221 and the comparison image 222 (step S151). The image processing phase execution program 37 also performs a binarization process on the difference image based on a preset threshold value (step S152), thereby generating a binarized image 230.

[0135] As shown in the figure, the difference image 229 is generated based on the difference between the processed image 221 and the comparison image 222, and therefore includes a portion 227 indicating a foreign body or defect, and an ROI 228 that emphasizes a defective portion such as a scratch or color unevenness whose luminance value is close to that of a non-defective product. When a binarization process is performed on such a difference image 229, an area where the pixel value is higher than the threshold value (in the example of FIG. 11, the area corresponding to the portion 227 indicating a foreign body or the like and the ROI 228) is shown in white, and other areas where the pixel value is lower than the threshold value are shown in black.

[0136] The image processing phase execution program 37 refers to the binarized image 230, and when it detects a portion larger than a preset threshold value, that is, a portion shown in white, it determines that the object is defective.

[0137] In this way, according to the present system 100, by performing machine learning of the ROI emphasis engine using a defective product image that combines pseudo-defects with brightness values ​​of defective parts such as scratches and color unevenness close to those of a good product, when an image including scratches or color unevenness is input to the ROI emphasis engine, an ROI emphasis processed image in which the scratch or color unevenness area is emphasized is output. This makes it possible for the present system 100 to detect defects with brightness values ​​close to those of a good product by inspection.

[0138] In addition, the ROI emphasis engine is trained to make good parts that do not have pseudo defects synthesized by the machine learning of the ROI emphasis engine the same as good images, so when an image containing a relatively large defect such as a foreign body or defect is input to the ROI emphasis engine during inspection, an image that looks like a good product without defects is output as an ROI emphasis processed image for areas that contain foreign bodies, defects, etc. Meanwhile, the present system 100 can similarly detect relatively large defects such as foreign bodies and defects by generating a difference image using the processed image and a comparison image and performing binarization processing on the difference image.

[0139] <Fourth embodiment> Next, a fourth embodiment will be described. In the learning phase, the system 100 according to this embodiment performs machine learning on the multiple ROI emphasis engines 31 so as to designate an area where different pseudo-failures are combined as an ROI, and output ROI-emphasized learning images with different types of image enhancement processing and degrees of image enhancement. Specifically, the present system 100 generates multiple pseudo-failure images where different pseudo-failures are combined with a learning good image, and designates an area where the pseudo-failures are combined as an ROI. The present system 100 also generates multiple ROI-emphasized learning images where the ROI of each pseudo-failure image is enhanced with different types of image enhancement processing and degrees of image enhancement. The present system 100 also performs machine learning on the multiple ROI emphasis engines 31 so that when a pseudo-failure image corresponding to each ROI emphasis engine 31 is input, each corresponding ROI-emphasized learning image is output.

[0140] In addition, in the image processing phase, the system 100 of this embodiment inputs a processed image to multiple ROI emphasis engines 31, which then output a comparison image in which the ROI has been emphasised, and determines whether the object is a pass / fail product (determines whether it is pass / fail) by comparing the processed image with the multiple comparison images output from each ROI emphasis engine 31.

[0141] In the third embodiment, a method has been described in which a single ROI, a type of image enhancement processing, and a degree of image enhancement are applied to a pseudo-fault image in the learning phase to generate an ROI-emphasized training image, and the pseudo-fault image and the ROI-emphasized training image are used to train the ROI enhancement engine 31. On the other hand, depending on the type of ROI, the accuracy of the inspection may be improved by using an ROI-emphasized processed image that is enhanced not by a single ROI but by different types of image enhancement processing and degrees of image enhancement.

[0142] Therefore, the system of this embodiment performs machine learning on multiple ROI emphasis engines 31 so as to output ROI emphasis training images that differ from one another in terms of ROI, type of image emphasis processing, and degree of image emphasis, and during inspection, uses multiple types of ROI emphasis processed images output by the multiple ROI emphasis engines 31 to more accurately determine whether an object is good or bad.

[0143] Note that the same objects and processes as those in the above-described embodiment are denoted by the same reference numerals and detailed description thereof will be omitted.

[0144] <Details of the learning phase and image processing phase> 12 is a diagram showing an example of a process sequence for performing machine learning on two ROI emphasis engines E3 and E4 in the learning phase. The learning phase execution program 36 acquires a learning good-quality image 203 from the learning image DB 32 (step S110), and synthesizes a first simulated defect 231 with the learning good-quality image (step S121) to generate a first simulated defect image 232. The learning phase execution program 36 also designates the region where the first simulated defect 231 is synthesized as the ROI 233 (step S122).

[0145] Furthermore, the learning phase execution program 36 generates a first ROI-emphasized learning image 234 based on the specified ROI 233 (step S123). Specifically, the learning phase execution program 36 generates the first ROI-emphasized learning image 234 in which the ROI 233 is emphasized by performing image processing according to the first type and degree of image emphasis.

[0146] Next, the learning phase execution program 36 performs machine learning of the ROI emphasis engine E3 so that the first simulated fault image 232 synthesized with the first simulated fault 231 is input and a first ROI emphasis learning image 234 is output (step S124).

[0147] The learning phase execution program 36 also performs machine learning of the ROI emphasis engine E2 in a similar manner. Specifically, the learning phase execution program 36 acquires a learning good-quality image 203 from the learning image DB 32 (step S110), and synthesizes a second simulated defect 235 different from the first simulated defect with the learning good-quality image (step S121) to generate a second simulated defect image 236. The learning phase execution program 36 also designates the region where the second simulated defect 235 is synthesized as the ROI 237 (step S122).

[0148] Furthermore, the learning phase execution program 36 generates a second ROI-emphasized training image 238 based on the specified ROI 237 (step S123). Specifically, the learning phase execution program 36 generates the second ROI-emphasized training image 238 in which the ROI 237 is emphasized by performing image processing with a second type and degree of image enhancement that are different from the first type and degree of image enhancement.

[0149] Next, the learning phase execution program 36 performs machine learning of the ROI emphasis engine E4 so that the second simulated fault image 236 synthesized with the second simulated fault 235 is input and a second ROI emphasis learning image 238 is output (step S124).

[0150] The type and degree of the first image enhancement, and the type and degree of the second image enhancement may be designated by a user using, for example, the GUI shown in FIG. 6, or may be preset.

[0151] Next, the image processing phase will be described. In the image processing phase, for each of the ROI emphasis engines E3 and E4, a comparison image (ROI emphasis processed image) 222 is generated (estimated) from the processed image 221, and the processed image 221 is compared with the comparison image 222 to determine whether the inspection target 220 is a non-defective or defective product. This process is the same as the image processing phase of the third embodiment.

[0152] Here, with reference to FIG. 13, a description will be given of pass / fail judgment of an inspection object 220 using a processed image 221 and a comparison image 222. FIG.

[0153] 13 is a diagram showing a processing sequence for determining whether an inspection target 220 is good or bad using a processed image 221 and a comparison image 222. As shown in the figure, the processed image 221 acquired in step S130 includes a first defect 240 and a second defect 241. The image processing phase execution program 37 inputs the processed image 221 to an ROI emphasis engine E3 to acquire an ROI emphasis processed image in which only the ROI is emphasised (step S140), and sets this as a comparison image 222m.

[0154] Furthermore, the image processing phase execution program 37 judges the quality of the object 220 using the processed image 221. Specifically, the image processing phase execution program 37 generates a difference image 229m between the processed image 221 and the comparison image 222m (step S151), and performs a binarization process on the difference image based on a preset threshold value (step S152) to generate a binarized image 230m.

[0155] The image processing phase execution program 37 generates a binarized image 230n using the ROI emphasis engine E4 by performing processing similar to that using the ROI emphasis engine E3.

[0156] Then, the image processing phase execution program 37 refers to the binarized images 230m, 230n, and when it detects a portion in at least one of the binarized images that is larger than a preset threshold value, that is, a portion shown in white, it determines that the object 220 is defective. Note that the process of step S160 is the same as that described above, and therefore a detailed description thereof will be omitted.

[0157] In this way, according to the present system 100, it is possible to inspect for defects, etc., using a plurality of ROI emphasis processed images generated by different types and degrees of image emphasis. Therefore, according to the present system 100, it is possible to generate an appropriate ROI emphasis processed image according to the type of ROI, and it is possible to improve the accuracy of the inspection.

[0158] Note that the above-described embodiment includes both cases where the same operator performs the learning phase and the image processing phase using the processor system 100, and cases where the operator performing only the learning phase is different from the operator performing only the image processing phase (i.e., the phase in which image processing is performed using the ROI emphasis engine 31 after machine learning).

[0159] Furthermore, the present invention is not limited to the above-mentioned embodiment and modifications, and includes various modifications within the scope of the same technical idea. For example, the above-mentioned examples are described in detail to explain the present invention in an easily understandable manner, and the present invention is not necessarily limited to those having all of the described configurations.

[0160] In addition, it is possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. For example, the ROI designation method in each embodiment may be used in a different embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0161] In addition, in the above explanation, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. [Explanation of symbols]

[0162] 100 Processor system, 20 Processor, 30 Memory resource, 31 ROI emphasis engine, 32 Learning image DB, 33 Processed image DB, 34 Other information DB, 35 GUI execution program, 36 Learning phase execution program, 37 Image processing phase execution program, 40 NI (Network Interface Device), 50 UI (User Interface Device), 10 Imaging device

Claims

1. 1. A system having one or more processors and one or more memory resources, comprising: The memory resource is storing an ROI emphasis engine, a learning phase execution program, and a GUI execution program; The processor executes the learning phase execution program, A training image obtained by capturing an object for training is used to generate an ROI-emphasized training image in which only an ROI (Region Of Interest) corresponding to a region of interest in a processing image obtained by capturing an object for image processing is emphasized; performing learning to optimize internal parameters of the ROI enhancement engine so that the ROI enhancement training image is generated when the training image is input; The processor executes the GUI execution program, In a learning phase in which the ROI enhancement engine is trained, screen information is output for receiving designation of the ROI in the training image, and a type of image enhancement processing and a degree of image enhancement to be performed on the ROI; The processor executes a learning phase execution program, A specified type of image enhancement process is performed on the specified ROI to a specified degree of image enhancement, thereby generating the ROI-enhanced training image. A system characterized by:

2. 2. The system of claim 1, The memory resource further stores an image processing phase execution program; The processor executes the image processing phase execution program, The processed image is input to the ROI emphasis engine, and an ROI emphasis processed image in which only the ROI is emphasised is obtained, which is output from the ROI emphasis engine. A system characterized by:

3. 2. The system of claim 1, The processor executes the learning phase execution program, The learning is performed on the plurality of ROI enhancement engines so as to output the ROI enhancement training images that are different in at least one of the ROI, the type of the image enhancement process, and the degree of image enhancement. A system characterized by:

4. 1. A system having one or more processors and one or more memory resources, comprising: The memory resource is storing an ROI emphasis engine, a learning phase execution program, and an image processing phase execution program; The processor executes the learning phase execution program, A training image obtained by capturing an object for training is used to generate an ROI-emphasized training image in which only an ROI (Region Of Interest) corresponding to a region of interest in a processing image obtained by capturing an object for image processing is emphasized; performing learning to optimize internal parameters of the ROI enhancement engine so that the ROI enhancement training image is generated when the training image is input; generating an ROI-emphasized training image in which the ROI designated based on a difference image between a training good-product image obtained by capturing an image of a training good product and a training defective product image obtained by capturing an image of a training defective product is emphasized; The processor executes the image processing phase execution program, The processed image is compared with a comparison image, which is an ROI-emphasized image in which only the ROI is emphasized and is obtained by inputting the processed image into the ROI emphasis engine, to thereby determine whether the object is a good product or a defective product. A system characterized by:

5. 1. A system having one or more processors and one or more memory resources, comprising: The memory resource is storing an ROI emphasis engine, a learning phase execution program, and an image processing phase execution program; The processor executes the learning phase execution program, A training image obtained by capturing an object for training is used to generate an ROI-emphasized training image in which only an ROI (Region Of Interest) corresponding to a region of interest in a processing image obtained by capturing an object for image processing is emphasized; performing learning to optimize internal parameters of the ROI enhancement engine so that the ROI enhancement training image is generated when the training image is input; generating an ROI-emphasized training image in which a training image of a training good product is combined with a pseudo-defective product as the ROI; The processor executes the image processing phase execution program, The processed image is compared with a comparison image, which is an ROI-emphasized image in which only the ROI is emphasized and is obtained by inputting the processed image into the ROI emphasis engine, to thereby determine whether the object is a good product or a defective product. A system characterized by:

6. 6. The system of claim 5, The processor executes the image processing phase execution program, A binary image generated by performing a binary processing on a difference image between the processed image and the comparison image is used to determine whether the object is a non-defective product or a defective product. A system characterized by:

7. 6. The system of claim 5, The processor executes the learning phase execution program, A region in which the pseudo defects different from each other are combined is designated as the ROI, and the learning is performed on the plurality of ROI enhancement engines so as to output the ROI enhancement training images in which the types of image enhancement processing and the degrees of image enhancement performed on the ROI are different from each other. A system characterized by:

8. A program read from and executed by one or more processors and one or more memory resources in a system having the processors, the program comprising: A learning phase execution program executed by the processor includes: A training image obtained by capturing an object for training is used to generate an ROI-emphasized training image in which only an ROI (Region Of Interest) corresponding to a region of interest in a processing image obtained by capturing an object for image processing is emphasized; performing learning to optimize internal parameters of an ROI enhancement engine so that the ROI-emphasized training image is generated when the training image is input; The GUI execution program executed by the processor includes: In a learning phase in which the ROI enhancement engine is trained, screen information is output for receiving designation of the ROI in the training image, and a type of image enhancement processing and a degree of image enhancement to be performed on the ROI; The learning phase execution program executed by the processor includes: A specified type of image enhancement process is performed on the specified ROI to a specified degree of image enhancement, thereby generating the ROI-enhanced training image. A program characterized by:

9. The program according to claim 8, The image processing phase execution program executed by the processor includes: The processed image is input to the ROI emphasis engine, and an ROI emphasis processed image in which only the ROI is emphasised is obtained, which is output from the ROI emphasis engine. A program characterized by:

10. The program according to claim 8, A learning phase execution program executed by the processor includes: The learning is performed on the plurality of ROI enhancement engines so as to output the ROI enhancement training images that are different in at least one of the ROI, the type of the image enhancement process, and the degree of image enhancement. A program characterized by:

11. A program for a system having one or more processors and one or more memory resources, the program being read from the memory resource and executed by the processor, comprising: A learning phase execution program executed by the processor includes: A training image obtained by capturing an object for training is used to generate an ROI-emphasized training image in which only an ROI (Region Of Interest) corresponding to a region of interest in a processing image obtained by capturing an object for image processing is emphasized; performing learning to optimize internal parameters of an ROI enhancement engine so that the ROI-emphasized training image is generated when the training image is input; generating an ROI-emphasized training image in which the ROI designated based on a difference image between a training good-product image obtained by capturing an image of a training good product and a training defective product image obtained by capturing an image of a training defective product is emphasized; The image processing phase execution program executed by the processor includes: The processed image is compared with a comparison image, which is an ROI-emphasized image in which only the ROI is emphasized and is obtained by inputting the processed image into the ROI emphasis engine, to thereby determine whether the object is a good product or a defective product. A program characterized by:

12. A program for a system having one or more processors and one or more memory resources, the program being read from the memory resource and executed by the processor, comprising: A learning phase execution program executed by the processor includes: A training image obtained by capturing an object for training is used to generate an ROI-emphasized training image in which only an ROI (Region Of Interest) corresponding to a region of interest in a processing image obtained by capturing an object for image processing is emphasized; performing learning to optimize internal parameters of an ROI enhancement engine so that the ROI-emphasized training image is generated when the training image is input; generating an ROI-emphasized training image in which a training image of a training good product is combined with a pseudo-defective product as the ROI; The image processing phase execution program executed by the processor includes: The processed image is compared with a comparison image, which is an ROI-emphasized image in which only the ROI is emphasized and is obtained by inputting the processed image into the ROI emphasis engine, to thereby determine whether the object is a good product or a defective product. A program characterized by:

13. The program according to claim 12, The image processing phase execution program executed by the processor includes: A binary image generated by performing a binary processing on a difference image between the processed image and the comparison image is used to determine whether the object is a non-defective product or a defective product. A program characterized by:

14. The program according to claim 12, The learning phase execution program executed by the processor includes: A region in which the pseudo defects different from each other are combined is designated as the ROI, and the learning is performed on the plurality of ROI enhancement engines so as to output the ROI enhancement training images in which the types of image enhancement processing and the degrees of image enhancement performed on the ROI are different from each other. A program characterized by:

Images