Template image creating method, template image creating system, and program

Abstract

The problem addressed by this disclosure is to provide a template image creation method, a template image creation system, and a program, by which it is possible to create a highly accurate template image with little noise, even if the positions of a test object appearing in a plurality of candidate images are not aligned. The template image creation method creates a template image from a plurality of candidate images, each of which includes a target region in which a test object appears. The template creation method: uses pattern matching to perform position correction to align the positions of the respective target regions of the plurality of candidate images; and creates at least one template image by sequentially synthesizing the plurality of candidate images.

Description

[Technical field]

[0001] The present disclosure relates to a template image creating method, a template image creating system, and a program. [Background technology]

[0002] Conventionally, there is an object recognition device that recognizes an object by template matching. Templates used in such object recognition devices are created by a template creation device disclosed in, for example, Patent Document 1.

[0003] The template creation device acquires multiple templates from multiple images of different postures of an object, or from multiple images of multiple objects. The template creation device calculates the similarity of image features for a combination of two templates selected from the multiple templates, and performs a clustering process to divide the multiple templates into multiple groups based on the similarity. The template creation device performs an integration process for each of the multiple groups to integrate all templates in the group into one integrated template or a number of integrated templates less than the number of templates in the group, and generates a new template set consisting of multiple integrated templates corresponding to each of the multiple groups.

[0004] That is, a template creation device such as that in Patent Document 1 treats multiple acquired templates as multiple candidate images, and divides each of the multiple candidate images into multiple groups based on the similarity between the multiple candidate images. The template creation device performs an integration process for integrating all the candidate images in each of the multiple groups into an integrated template, and generates a new template set (template images) made up of multiple integrated templates corresponding to the multiple groups, respectively.

[0005] The above-mentioned template creation device is based on the premise that the positions of the objects to be inspected in the multiple candidate images are aligned. Therefore, if a template image is created using multiple candidate images in which the positions of the objects to be inspected are not aligned, the accuracy of the template image is low and the template image contains a lot of noise, making it difficult to use the template image for template matching. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2016-207147 A Summary of the Invention

[0007] An object of the present disclosure is to provide a template image creation method, a template image creation system, and a program capable of creating a high-precision template image with little noise even if the positions of objects to be inspected in multiple candidate images are not aligned.

[0008] A template image creation method according to one aspect of the present disclosure creates a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured. The template image creation method creates at least one template image by performing position correction for aligning the position of the target area of ​​each of the plurality of candidate images by pattern matching, and sequentially combining the plurality of candidate images. The template image creation method performs image processing including a combination step, a position correction step, and a combination step. The combination step performs combination processing for generating a set including two or more input images from a plurality of input images. The position correction step performs the position correction on the two or more input images for each set. The combination step creates an output image corresponding to the set by combining the two or more input images that have been subjected to the position correction. After performing the image processing using the plurality of candidate images as the plurality of input images, if there are a plurality of output images, the image processing using the plurality of output images as the plurality of input images is repeated until the output image by the image processing satisfies a predetermined condition. 。 A template image creation method according to another aspect of the present disclosure creates a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured. The template image creation method performs position correction to align the position of the target area of ​​each of the plurality of candidate images by pattern matching, and creates at least one template image by sequentially combining the plurality of candidate images. An order of combining the plurality of candidate images is set based on the mutual similarity of the plurality of candidate images.

[0009] A template image creation system according to an aspect of the present disclosure creates a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured. The template image creation system includes an image processing unit that performs position correction for aligning the position of the target area of ​​each of the plurality of candidate images by pattern matching, and creates at least one of the template images by sequentially combining the plurality of candidate images. The image processing unit performs image processing including a combination process, a position correction process, and a combination process. The combination process generates a set including two or more input images from a plurality of input images. The position correction process performs the position correction on the two or more input images for each set. The combination process creates an output image corresponding to the set by combining the two or more input images that have been subjected to the position correction. After performing the image processing using the plurality of candidate images as the plurality of input images, if there are a plurality of output images, the image processing using the plurality of output images as the plurality of input images is repeated until the output image by the image processing satisfies a predetermined condition. 。 A template image creation system according to another aspect of the present disclosure creates a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured. The template image creation system includes an image processing unit that performs position correction to align the position of the target area of ​​each of the plurality of candidate images by pattern matching, and creates at least one of the template images by sequentially synthesizing the plurality of candidate images. The image processing unit sets an order of synthesis of the plurality of candidate images based on the mutual similarity of the plurality of candidate images.

[0010] A program according to an aspect of the present disclosure causes a computer system to execute the above-described template image creation method. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 is a diagram for explaining template matching using a template image created by a template image creating method according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing a template image creating system for executing the above-mentioned template image creating method. [Diagram 3] FIG. 3 is a diagram showing the operation of the template image creation system. [Figure 4] FIG. 4 is a diagram showing a grayscale candidate image used in the above template image method. [Diagram 5] 5A to 5D are enlarged views of a portion of the above-mentioned gradation candidate image. [Figure 6] FIG. 6 is a diagram showing a binarization candidate image used in the above template image method. [Figure 7] 7A to 7D are enlarged views of a part of the binarization candidate image. [Figure 8] FIG. 8 is a flowchart showing the image processing method. [Figure 9] FIG. 9 is a schematic diagram showing the image processing method of the above embodiment. [Figure 10] FIG. 10 is a diagram showing a template image created by the image processing method of the above embodiment. [Figure 11] FIG. 11 is a flowchart showing an image processing method according to the first modified example of the above embodiment. [Figure 12] FIG. 12 is a flowchart showing an image processing method according to the second modified example of the above embodiment. [Figure 13] FIG. 13 is a flowchart showing an image processing method according to the fourth modified example of the above embodiment. [Figure 14] FIG. 14 is a schematic diagram showing the image processing method of the above embodiment. [Figure 15] FIG. 15 is a flowchart showing an image processing method according to the sixth modified example of the above embodiment. [Figure 16] FIG. 16 is a diagram showing the above-mentioned gray-scale candidate image. [Figure 17] FIG. 17 is a schematic diagram showing the image processing method of the above embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The following embodiments generally relate to a template image creation method, a template image creation system, and a program. More specifically, the following embodiments relate to a template image creation method, a template image creation system, and a program for creating a template image from a plurality of candidate images. Note that the embodiments described below are merely examples of the embodiments of the present disclosure. The present disclosure is not limited to the following embodiments, and various modifications are possible depending on the design, etc., as long as the effects of the present disclosure can be achieved.

[0013] (1) Template matching Template matching using image processing technology is used for the inspection of objects to be inspected and for pre-inspection processing. Inspections include, for example, mounting inspection to check whether a specific part on a printed circuit board is mounted in the position as designed, processing inspection to check whether a processed product is processed to the designed dimensions and shape, assembly inspection to check whether an assembly product is assembled as designed, and appearance inspection to check whether a specific part has features such as scratches or dirt. In template matching, a standard pattern that is a normal pattern (feature) of the structure of the object to be inspected is created in advance as a template image, and the template image is applied to the captured image of the object to be inspected to perform pattern matching.

[0014] In this embodiment, a MEMS (Micro Electro Mechanical System) is the object to be inspected, and the internal structure of the MEMS is inspected.

[0015] An inspection device that performs structural inspection of MEMS by template matching applies a rectangular template image Gt to a rectangular inspection image Ga shown in Fig. 1. The size of the template image Gt is smaller than the size of the inspection image Ga. Then, the inspection device calculates the similarity between the template image Gt and a part of the inspection image Ga that overlaps with the template image Gt each time the inspection device moves the template image Gt in the search range of the inspection image Ga by raster scanning or the like. Then, the inspection device can determine the position in the search range where the similarity is greatest as the detection position and perform structural inspection at the detection position.

[0016] In the above-mentioned template matching, the template image Gt is created based on a captured image of a non-defective or defective product. The template image Gt is required to accurately reflect the characteristics of a non-defective product and to have little noise.

[0017] Therefore, in this embodiment, the template image Gt is created by a template image creating method executed by the following template image creating system.

[0018] (2) Template image creation system (2.1) System Configuration The template image creation system 1 includes a computer system CS, a display unit 1d, and an operation unit 1e, as shown in Fig. 2. The computer system CS includes an image acquisition unit 1a, a storage unit 1b, and an image processing unit 1c.

[0019] In the computer system CS, a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) reads and executes a program for a vibration inspection method stored in a memory, thereby realizing some or all of the functions of the template image creation system 1. The computer system CS has a processor that operates according to a program as a main hardware configuration. The type of processor is not important as long as it can realize the function by executing the program. The processor is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or an LSI (large scale integration). Although IC and LSI are referred to here, the name may change depending on the degree of integration, and may be called a system LSI, VLSI (very large scale integration), or ULSI (ultra large scale integration). A field programmable gate array (FPGA), which is programmed after the manufacture of the LSI, or a reconfigurable logic device that can reconfigure the connection relationship inside the LSI or set up the circuit partition inside the LSI, can also be used for the same purpose. A plurality of electronic circuits may be integrated into one chip, or may be provided on multiple chips. The multiple chips may be integrated into one device, or may be provided in multiple devices.

[0020] Then, as shown in FIG. 3, the template image creation system 1 acquires a plurality of captured images of the same size as the template image Gt as a plurality of shading candidate images Gb. The shading candidate image Gb is a rectangular image that serves as a base when creating the template image Gt. FIG. 4 shows an example of the shading candidate image Gb. The shading candidate image Gb is a shading image captured by an infrared camera of the inside of a non-defective or defective MEMS, and a specific element that constitutes a part of the inside of the MEMS is captured. The shading candidate image Gb includes target areas Ra1 to Ra6 as areas (target areas Ra) in which a specific element that constitutes a part of the inside of the MEMS is captured. The target areas Ra1 to Ra6 are brighter than the surroundings of the target areas Ra1 to Ra6. The shading image is an image in which the shading value is set to, for example, 256 levels. In the shading image of this embodiment, a dark pixel has a low shading value, and a bright pixel has a high shading value. The shading image may be either a monochrome image or a color image.

[0021] The template image creation system 1 obtains a plurality of shading candidate images Gb and creates a template image Gt by performing image processing on the plurality of shading candidate images Gb. However, the positions of the target regions Ra1 to Ra6 in each of the plurality of shading candidate images Gb are not aligned with each other. For example, the position of the target region Ra4 relative to the rectangular range 9 existing at a predetermined coordinate in the inspection space may be shifted from each other as shown in FIGS. 5A to 5D. If the template image Gt is created using a plurality of shading candidate images Gb in which the positions of the target regions Ra1 to Ra6 are shifted from each other, the template image Gt is likely to include noise.

[0022] Furthermore, when the gradation candidate image Gb in Fig. 4 is subjected to binarization processing and edge detection processing, a binarized candidate image Gc is created by extracting edges of the target regions Ra1 to Ra4 of the gradation candidate image Gb, as shown in Fig. 6. Therefore, a template image Gt can also be created by performing image processing on a plurality of binarized candidate images Gc. However, similar to the above-mentioned gradation candidate image Gb, the positions of the target regions Ra1 to Ra6 in each of the plurality of binarized candidate images Gc are not aligned with each other. If the template image Gt is created using a plurality of binarized candidate images Gc in which the positions of the target regions Ra1 to Ra6 are shifted from each other, the template image Gt is likely to include noise.

[0023] Furthermore, in the binarization candidate image Gc, the edges of the target regions Ra1 to Ra4 are extracted, but there are missing edges and erroneous edge extractions. In other words, when the noise in the gradation candidate image Gb is large, the binarization candidate image Gc may contain noise that cannot be completely removed even by the binarization process and edge detection process. For example, as shown in Figures 7A to 7D, the target region Ra4 in the rectangular range 9 has missing edges and erroneous edge extractions. If a template image Gt is created using a plurality of binarization candidate images Gc in which there are missing edges and erroneous edge extractions in the target regions Ra1 to Ra6, the template image Gt is likely to contain noise.

[0024] Therefore, the template image creation system 1 creates a template image Gt from a plurality of gray-scale candidate images Gb according to the flowchart of FIG.

[0025] (2.2) How to create a template image FIG. 8 shows a template image creating method executed by the computer system CS of the template image creating system 1.

[0026] First, the image acquisition unit 1a acquires N+1 gray-scale candidate images Gb from an external database, a camera, a storage medium, etc. (acquisition step S1), where N is a positive integer.

[0027] The image processing unit 1c performs preprocessing on each of the N+1 gray-scale candidate images Gb (preprocessing step S2). The preprocessing in this embodiment includes binarization processing and edge detection processing. In this case, the image processing unit 1c performs binarization processing and edge detection processing on each of the N+1 gray-scale candidate images Gb to create N+1 binarized candidate images Gc, and stores data of the N+1 binarized candidate images Gc in the memory unit 1b. In other words, the memory unit 1b stores data of the N+1 binarized candidate images Gc. The preprocessing may include median filtering, Gaussian filtering, histogram equalization, normalization, standardization, or the like.

[0028] The storage unit 1b preferably has a rewritable memory such as a solid state drive (SSD), a hard disk drive (HDD), an electrically erasable programmable read only memory (EEPROM), a random access memory (RAM), or a flash memory.

[0029] Next, the image processing unit 1c performs a parameter setting process (parameter setting step S3). In the parameter setting step S3, parameters related to position correction are set. In the above-mentioned positional deviation of the shading candidate image Gb, the direction in which deviation is likely to occur varies for each inspection object. The directions in which deviation is likely to occur include the direction along the long side of the shading candidate image Gb, the direction along the short side of the shading candidate image Gb, and the rotation direction. The parameters may also include perspective correction, enlargement, and reduction of the image. Therefore, in the parameter setting step S3, the direction in which deviation of the shading candidate image Gb is likely to occur is set as a parameter for each inspection object.

[0030] Next, the image processing unit 1c sequentially extracts one binarization candidate image Gc from the N+1 binarization candidate images Gc (extraction step S4). Specifically, if the N+1 binarization candidate images Gc are Gc(1), Gc(2), Gc(3), ..., Gc(N+1), the image processing unit 1c extracts one binarization candidate image Gc in the order of binarization candidate images Gc(1), Gc(2), Gc(3), ... each time it performs the process of extraction step S4. Here, since this is the first extraction step S4, the image processing unit 1c extracts the binarization candidate image Gc(1). Since the binarization candidate image Gc extracted in extraction step S4 is only one, the binarization candidate image Gc(1), the image processing unit 1c does not execute the processes in the following steps S5, S6, and S8, and deletes the data of the binarization candidate image Gc(1) from the storage unit 1b (data deletion step S7).Then, the image processing unit 1c determines whether or not the synthesis of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has been completed (completion determination step S9).

[0031] Since synthesis of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has not been completed, the image processing unit 1c performs the process of extraction step S4 again. Here, since this is the second extraction step S4, the image processing unit 1c extracts the binarization candidate image Gc(2). Therefore, the image processing unit 1c extracts the binarization candidate images Gc(1) and Gc(2) by the first and second extraction step S4 processes.

[0032] Next, the image processing unit 1c performs position correction of the binarization candidate images Gc(1) and Gc(2) (position correction step S5). Specifically, the image processing unit 1c performs position correction by pattern matching to align the positions of the target regions Ra1 to Ra6 of the binarization candidate images Gc(1) and Gc(2) with each other. The pattern matching adjusts the positions of the binarization candidate images Gc(1) and Gc(2) on the inspection coordinates so that the similarity of the binarization candidate images Gc(1) and Gc(2) on the inspection coordinates is maximized. In other words, the image processing unit 1c adjusts the positions of the binarization candidate images Gc(1) and Gc(2) so that the positions of the target regions Ra1 to Ra6 of the binarization candidate images Gc(1) and Gc(2) overlap. At this time, the image processing unit 1c adjusts the positions of the binarization candidate images Gc(1) and Gc(2) only along the direction set in the parameters set in the parameter setting step S3. Therefore, the image processor 1c can limit the direction of position correction, thereby reducing the calculation cost required for position correction. Hereinafter, the position correction in the position correction step S5 is performed only along the direction set in the parameter.

[0033] Next, the image processing unit 1c creates a composite image Gd(1) by combining the binarized candidate images Gc(1) and Gc(2) that have been subjected to position correction (see FIG. 9) (combining step S6). In the combining step S6, the average value, weighted average value, median value, logical sum or logical product of the grayscale values ​​of each pixel of the two binarized candidate images is set as the grayscale value of each pixel of the composite image. In the composite image Gd(1), the target regions Ra1-Ra6 of the binarized candidate images Gc(1) and Gc(2) overlap each other, forming the target regions Ra1-Ra6 of the composite image Gd(1).

[0034] The image processing unit 1c deletes the data of the binarization candidate image Gc(2) from the storage unit 1b (data deleting step S7). The image processing unit 1c stores the data of the composite image Gd(1) in the storage unit 1b (data storing step S8). As a result, the storage unit 1b stores the data of the binarization candidate images Gc(3) to Gc(N+1) and the composite image Gd(1). The image processing unit 1c then determines whether or not the composition of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has been completed (completion determining step S9).

[0035] Since synthesis of all of the binarization candidate images Gc(1), Gc(2), Gc(3), ... has not been completed, the image processing unit 1c performs the process of extraction step S4 again. Here, since this is the third extraction step S4, the image processing unit 1c extracts the binarization candidate image Gc(3). Therefore, the image processing unit 1c extracts the binarization candidate images Gc(1) to Gc(3) by the first to third extraction step S4 processes.

[0036] Next, the image processing unit 1c performs position correction of the composite image Gd(1) and the binarization candidate image Gc(3) (position correction step S5). Specifically, the image processing unit 1c performs position correction to align the positions of the target regions Ra1 to Ra6 of the composite image Gd(1) and the binarization candidate image Gc(3) with each other by pattern matching.

[0037] Next, the image processing unit 1c creates a composite image Gd(2) by combining the position-corrected composite image Gd(1) and the binarization candidate image Gc(3) (see FIG. 9) (combining step S6). Here, the composite image Gd(1) is a composite image of the two binarization candidate images Gc(1) and Gc(2), and the binarization candidate image Gc(3) is a single binarization candidate image. Therefore, when the image processing unit 1c sets the average of the gradation values ​​of each pixel of the composite image Gd(1) and the binarization candidate image Gc(3) as the gradation value of each pixel of the composite image Gd(2), it is preferable that the image processing unit 1c performs a weighted average process on the gradation values ​​of each pixel of the composite image Gd(1) and the binarization candidate image Gc(3). In the composite image Gd(2), the target regions Ra1-Ra6 of the composite image Gd(1) and the binarization candidate image Gc(3) overlap with each other to form the target regions Ra1-Ra6 of the composite image Gd(2). It can also be said that the composite image Gd(2) is an image obtained by combining the binarization candidate images Gc(1)-Gc(3).

[0038] The image processing unit 1c deletes each data of the composite image Gd(1) and the binarization candidate image Gc(3) from the storage unit 1b (data deleting step S7). The image processing unit 1c stores the data of the composite image Gd(2) in the storage unit 1b (data storing step S8). As a result, the storage unit 1b stores each data of the binarization candidate images Gc(4) to Gc(N+1) and the composite image Gd(2). Then, the image processing unit 1c determines whether or not the composition of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has been completed (completion determining step S9).

[0039] Since compositing of all of the binarization candidate images Gc(1), Gc(2), Gc(3), ... has not been completed, the image processing unit 1c performs the process of extraction step S4 again. Thereafter, the image processing unit 1c creates composite images Gd(3) to Gd(N) by repeatedly performing the processes of extraction step S4 to completion determination step S9.

[0040] That is, an extraction step S4 extracts an M-th (M is a positive integer equal to or less than N) binarization candidate image Gc(M) from the N+1 binarization candidate images Gc(1) to Gc(N+1). A position correction step S5 aligns the positions of the target regions Ra1 to Ra6 of the M-th binarization candidate image Gc(M) and a composite image Gd(M-2) obtained by combining the 1st to M-1th binarization candidate images Gc(1) to Gc(M-1) by pattern matching. A combination step S6 combines the composite image Gd(M-2) with the M-th binarization candidate image Gc(M).

[0041] Then, the image processing unit 1c creates a composite image Gd(N) after performing the process of the extraction step S4 for the (N+1)th time. In this case, since the composition of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has been completed, the image processing unit 1c sets the composite image Gd(N) as the template image Gt (decision step S10) (see FIG. 9). That is, the image processing unit 1c sets the composite image Gd(N) created in the last composition step S6 as the template image Gt. The image processing unit 1c stores the data of the template image Gt in the storage unit 1b.

[0042] Fig. 10 shows an example of the template image Gt. Compared to the binarization candidate image Gc (see Fig. 6), the template image Gt has less edge chipping and erroneous edge extraction in the target regions Ra1 to Ra6, and the edges of the target regions Ra1 to Ra6 are clearer, making it a highly accurate template image. Furthermore, the template image Gt has less noise compared to the binarization candidate image Gc (see Fig. 6).

[0043] The computer system CS outputs data of the template image Gt to the display unit 1d. The display unit 1d is a liquid crystal display, an organic EL display, or the like, and displays the template image Gt. Therefore, the examiner can visually recognize the template image Gt used in the inspection by looking at the template image Gt displayed on the display unit 1d.

[0044] The operation unit 1e has a user interface function that accepts operations by the examiner. The operation unit 1e has at least one user interface such as a touch panel display, a keyboard, and a mouse. The examiner performs operations on the operation unit 1e to start up the computer system CS, input parameters related to position correction in the parameter setting step S3, and control the display of the display unit 1d.

[0045] As described above, the template image creation method of this embodiment creates a template image Gt using a plurality of binarized candidate images Gc in which the positions of the target regions Ra1 to Ra6 are shifted from one another. Specifically, the template image creation method of this embodiment creates a template image Gt by sequentially compositing a plurality of binarized candidate images Gc in which the positional shifts have been corrected by position correction using pattern matching. As a result, the template image creation method of this embodiment can create a high-precision template image Gt with little noise even if the positions of the objects to be inspected shown in the plurality of binarized candidate images Gc are not aligned. Here, "sequentially compositing a plurality of images" refers to sequentially repeating a process of compositing some of the plurality of images without compositing all of the plurality of images at once.

[0046] In this embodiment, the image obtained by performing binarization and edge detection on the shading candidate image Gb is the binarized candidate image Gc, and each of the shading candidate image Gb and the binarized candidate image Gc is a candidate image that includes information on the target regions Ra1 to Ra6. In other words, both the shading candidate image Gb and the binarized candidate image Gc can be considered to be candidate images according to the present disclosure.

[0047] (3) First Modification The template image creation method of the first modification sets whether or not multiple binarization candidate images Gc can be combined based on the mutual similarity of the multiple binarization candidate images Gc. Thus, by optimizing whether or not multiple binarization candidate images Gc can be combined, a highly accurate and low-noise template image Gt can be generated even if a binarization candidate image Gc with high noise is included.

[0048] The similarity can be calculated by using the degree of conformance of pattern matching, or by extracting features using deep learning, histograms of the gray values ​​of each pixel, histogram statistics, the results of blob detection, the edge length of the target region Ra, etc., and comparing the features of each candidate image. Methods for comparing features include Euclidean distance, isolation index, and Bray-Curtis index.

[0049] FIG. 11 is a flowchart showing a template image generating method according to the first modified example.

[0050] First, the computer system CS performs the acquisition step S1, the preprocessing step S2, and the parameter setting step S3 in the same manner as described above.

[0051] Then, the image processing unit 1c extracts one binarization candidate image Gc from the N+1 binarization candidate images Gc (Gc(1) to Gc(N+1)) as a first candidate image (extraction step S21). Here, the image processing unit 1c extracts the binarization candidate image Gc(1) as the first candidate image. Then, the image processing unit 1c resets the value of a counter (count value) provided in the computer system CS to 0 (reset step S22).

[0052] Next, the image processing unit 1c extracts one binarization candidate image Gc as a second candidate image from the N binarization candidate images Gc(2) to Gc(N+1) excluding the binarization candidate image Gc(1) (extraction step S23). Here, the image processing unit 1c extracts the binarization candidate image Gc(2) as the second candidate image.

[0053] Next, the image processing unit 1c calculates the degree of pattern matching compatibility between the binarization candidate images Gc(1) and Gc(2) (degree of matching calculation step S24). When the number of binarization candidate images Gc is large, the similarity of the binarization candidate images Gc may be calculated using a feature extraction method and a feature comparison method instead of calculating the degree of pattern matching compatibility. The comparison of features may be performed using Euclidean distance, isolation index, Bray-Curtis index, etc.

[0054] Next, the image processing unit 1c judges whether the compatibility of the binarized candidate images Gc(1) and Gc(2) is equal to or greater than a compatibility threshold (compatibility judgment step S25). That is, in the compatibility judgment step S25, the image processing unit 1c sets whether or not the binarized candidate images Gc(1) and Gc(2) can be combined based on the similarity between the binarized candidate images Gc(1) and Gc(2). Note that a predetermined value may be set as the compatibility threshold. Also, the compatibility threshold may be set based on the distribution of the compatibility of the template matching of the binarized candidate images Gc by selecting a plurality of binarized candidate images Gc, recording the compatibility of the selected plurality of binarized candidate images Gc, and repeating the recording of the compatibility a predetermined number of times, and then using the top 50% of the values ​​as the compatibility threshold.

[0055] Next, if the degree of compatibility is equal to or greater than the compatibility threshold, the image processing unit 1c performs position correction of the binarization candidate images Gc(1) and Gc(2) by pattern matching (position correction step S26). At this time, the image processing unit 1c adjusts the positions of the binarization candidate images Gc(1) and Gc(2) only along the direction set in the parameters set in the parameter setting step S3. Therefore, the image processing unit 1c can limit the direction of position correction, thereby suppressing the calculation cost required for position correction. Hereinafter, the position correction in the position correction step S26 is performed only along the direction set in the parameters.

[0056] Next, the image processing unit 1c creates a composite image Gd(1) by combining the binarized candidate images Gc(1) and Gc(2) that have been subjected to position correction (combining step S27). Then, the image processing unit 1c deletes each data of the combined binarized candidate images Gc(1) and Gc(2) from the storage unit 1b (data deleting step S28), and stores the data of the composite image Gd(1) in the storage unit 1b (data storing step S30). As a result, the storage unit 1b stores each data of the binarized candidate images Gc(3) to Gc(N+1) and the composite image Gd(1). Then, the image processing unit 1c sets the count value to 1 (counting step S31).

[0057] If the degree of compatibility is less than the compatibility threshold, the image processing unit 1c postpones the synthesis process using the binarized candidate image Gc(2), which is the second candidate image (postponement step S29). In this case, the storage unit 1b stores the data of each of the binarized candidate images Gc(2) to Gc(N+1).

[0058] Then, the image processing unit 1c judges whether extraction of all the binarization candidate images Gc(1) to Gc(N+1) has been completed (completion judgment step S32). Since extraction of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has not been completed, the image processing unit 1c performs the process of extraction step S23 again. In extraction step S23, the image processing unit 1c sets the composite image Gd(1) or the binarization candidate image Gc(1) as the first candidate image, and further extracts the binarization candidate image Gc(3) as the second candidate image.

[0059] Next, the image processing unit 1c calculates the degree of pattern matching between the first candidate image and the binarized candidate image Gc(3) (degree of pattern matching calculation step S24).

[0060] Next, the image processing unit 1c judges whether or not the degree of compatibility between the first candidate image and the binarized candidate image Gc(3) is equal to or greater than a compatibility threshold (compatibility judgment step S25). That is, in the compatibility judgment step S25, the image processing unit 1c determines whether or not the first candidate image and the binarized candidate image Gc(3) can be combined based on the similarity between the first candidate image and the binarized candidate image Gc(3).

[0061] Next, if the degree of compatibility is equal to or greater than the compatibility threshold, the image processing unit 1c performs position correction between the first candidate image and the binarized candidate image Gc(3) by pattern matching (position correction step S26). The image processing unit 1c creates a composite image Gd(2) by combining the first candidate image and the binarized candidate image Gc(3) that have been subjected to position correction (combining step S27). Then, the image processing unit 1c deletes each data of the combined first candidate image and the binarized candidate image Gc(3) from the storage unit 1b (data deletion step S28), and stores the data of the composite image Gd(2) in the storage unit 1b (data storage step S30). Then, the image processing unit 1c sets the count value to 1 (counting step S31).

[0062] If the degree of compatibility is less than the compatibility threshold, the image processing unit 1c postpones the synthesis process using the binarized candidate image Gc(3), which is the second candidate image (postponement step S29).

[0063] Then, the image processing unit 1c judges whether extraction of all the binarization candidate images Gc(1) to Gc(N+1) has been completed (completion judgment step S32). Since extraction of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has not been completed, the image processing unit 1c performs the process of extraction step S23 again. In extraction step S23, the image processing unit 1c sets the composite image Gd(2) or the binarization candidate image Gc(1) as the first candidate image, and further extracts the binarization candidate image Gc(4) as the second candidate image. Thereafter, the image processing unit 1c repeatedly performs the processes of extraction step S23 to completion judgment step S32.

[0064] Then, when extraction of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... is completed in the completion determination step S32, the image processing unit 1c determines whether the count value is 0 and whether there are any binarization candidate images Gc that have been postponed (end determination step S33).

[0065] The image processing unit 1c The binarization candidate image Gc is not stored in the memory unit 1b, and only one composite image Gd is stored. The count value is 0 If at least one of these two conditions is satisfied, the template image Gt is determined (determination step S34).

[0066] Specifically, in the end determination step S33, if no binarized candidate image Gc is stored in the memory unit 1b and only one composite image Gd is stored, then the synthesis of all the binarized candidate images Gc(1), Gc(2), Gc(3), ... is completed, and the image processing unit 1c sets the composite image Gd(N) as the template image Gt (decision step S34).

[0067] Furthermore, when the binarization candidate images Gc(1), Gc(2), Gc(3), ... are sequentially synthesized, if all the conformances calculated in the conformance calculation step S24 are less than the conformance threshold, the synthesized image Gd is not updated. In this case, it is unnecessary to return to the reset step S22 and perform the processes after the reset step S22 again. Therefore, the count value is used as a value for determining whether it is necessary to perform the processes after the reset step S22 again. If all the conformances calculated in the conformance calculation step S24 are less than the conformance threshold, the count value becomes 0. Therefore, if the count value is 0 in the end determination step S33, the image processing unit 1c determines that the synthesized image Gd at that time is the template image Gt, or determines that the generation of the template image Gt has failed (decision step S34).

[0068] Furthermore, in the end determination step S33, if the count value is 1 (if the count value is not 0), the composite image Gd has been generated, and it is expected that the noise contained in the composite image Gd is less than the noise contained in the candidate image. Therefore, if the count value is 1 and the binarization candidate image Gc is stored in the storage unit 1b, the image processing unit 1c determines that there is a binarization candidate image Gc that has been postponed, returns to the reset step S22, and performs the processes from the reset step S22 onwards again. Then, the image processing unit 1c sets the current composite image Gd as the first candidate image and the postponed binarization candidate image Gc as the second candidate image (extraction step S23), and performs the processes from the conformance calculation step S24 onwards.

[0069] Furthermore, when the image processing unit 1c repeats the judgment process of the termination judgment step S33 a predetermined maximum number of times, even if there is still a binarization candidate image Gc that has been postponed, the image processing unit 1c may select the current composite image Gd as the template image Gt (decision step S34).

[0070] Therefore, in the template image creation method of this modified example, binarization candidate images Gc that are significantly different from the other binarization candidate images Gc(1) to Gc(N+1) are excluded, so that a highly accurate and low-noise template image Gt can be generated.

[0071] (4) Second Modification The template image creation method of the second modification sets whether or not multiple binarization candidate images Gc can be combined based on the mutual similarity of the multiple binarization candidate images Gc. Thus, by optimizing whether or not multiple binarization candidate images Gc can be combined, a highly accurate and low-noise template image Gt can be generated even if a binarization candidate image Gc with high noise is included.

[0072] FIG. 12 is a flowchart showing a template image generating method according to the second modified example.

[0073] In the second modified example, the acquisition step S1, preprocessing step S2, parameter setting step S3, extraction step S21, extraction step S23, compatibility calculation step S24, and compatibility determination step S25 of the flowchart of the first modified example shown in FIG. 11 are performed.

[0074] The process after the conformity determination step S25 in the second modified example will be described below.

[0075] The image processing unit 1c obtains the degree of suitability of the pattern matching in the composite image Gd, and judges whether the degree of suitability is equal to or greater than a suitability threshold (suitability judgment step S25). That is, in the suitability judgment step S25, the image processing unit 1c determines whether or not the first candidate image and the second candidate image can be composited based on the similarity between the first candidate image and the second candidate image.

[0076] If the degree of compatibility is equal to or greater than the compatibility threshold, the image processing unit 1c performs the position correction step S26, the synthesis step S27, the data deletion step S28, and the data storage step S30, similarly to the first modified example.

[0077] If the degree of suitability is less than the suitability threshold, the image processing unit 1c deletes the data of the second candidate image from the storage unit 1b (data deleting step S41).

[0078] Then, the image processing unit 1c judges whether or not the extraction of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has been completed (completion judgment step S42). If the extraction of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has not been completed, the image processing unit 1c performs the process of extraction step S23 again. If the extraction of all the binarization candidate images Gc(1), Gc(2), Gc(3), ... has been completed, the image processing unit 1c sets the current composite image Gd as the template image Gt (decision step S43).

[0079] Therefore, the template image creation method of this modified example can generate a highly accurate and low-noise template image Gt even if the binarization candidate images Gc(1) to Gc(N+1) include a binarization candidate image Gc that is significantly different from the other binarization candidate images Gc.

[0080] (5) Third Modification In the template image creation method of the third modified example, it is preferable to set the order of synthesis for each of the multiple binarization candidate images Gc based on the mutual similarity between the multiple binarization candidate images Gc in the extraction step S4 of the flowchart in Fig. 8. Thus, the order of synthesis for the multiple binarization candidate images Gc is optimized, and a highly accurate and low-noise template image Gt can be generated even if a binarization candidate image Gc with high noise is included.

[0081] Specifically, the image processing unit 1c selects one of the N+1 binarization candidate images Gc as a reference image. Here, the binarization candidate image Gc(1) is selected as the reference image. Then, the image processing unit 1c calculates the similarity between each of the binarization candidate images Gc(2) to Gc(N+1) and the binarization candidate image Gc(1). The image processing unit 1c assigns a rank to each of the binarization candidate images Gc(2) to Gc(N+1) in order of increasing similarity. That is, the image processing unit 1c assigns a rank "1" to the binarization candidate image Gc(1), and assigns ranks "2", "3", ...., "N+1" to each of the binarization candidate images Gc(2) to Gc(N+1) in order of similarity to the binarization candidate image Gc(1). The smaller the rank number, the higher the rank. The image processing unit 1c extracts one binarization candidate image Gc from the N+1 binarization candidate images Gc in the order of rank "1", "2", "3", ..., "N+1" from the highest rank to the lowest rank from the N+1 binarization candidate images Gc each time the extraction step S4 is performed. If the binarization candidate images Gc(1) to Gc(N+1) include a binarization candidate image Gc that is significantly different from the other binarization candidate images Gc, the similarity will be low when the pattern matching of the position correction step S5 is performed using this binarization candidate image Gc. When the similarity falls below a predetermined threshold, the image processing unit 1c stops the subsequent processing and sets the current composite image Gd(M) as the template image Gt.

[0082] Furthermore, the image processing unit 1c may determine whether or not to perform synthesis using each binarization candidate image Gc based on each similarity of the binarization candidate image Gc. That is, the image processing unit 1c does not perform synthesis using a binarization candidate image Gc whose similarity is equal to or less than a threshold. That is, the image processing unit 1c does not use a binarization candidate image Gc that is significantly different from the other binarization candidate images Gc among the binarization candidate images Gc(1) to Gc(N+1) to create the template image Gt.

[0083] Therefore, the template image creation method of this modified example can generate a highly accurate and low-noise template image Gt even if the binarization candidate images Gc(1) to Gc(N+1) include a binarization candidate image Gc that is significantly different from the other binarization candidate images Gc.

[0084] (6) Fourth Modification The template image creation method of the fourth modified example performs image processing including a combination step, a position correction step, and a synthesis step. The combination step performs combination processing to generate a set including two or more input images from a plurality of input images. The position correction step performs position correction on the two or more input images for each set. The synthesis step creates an output image corresponding to the set by synthesizing the two or more input images that have been subjected to position correction. Then, image processing is performed using the plurality of candidate images as the plurality of input images. Thereafter, if there are a plurality of output images, image processing using the plurality of output images as the plurality of input images is repeated until the output image by the image processing satisfies a predetermined condition.

[0085] FIG. 13 is a flowchart showing a template image generating method according to the fourth modified example.

[0086] First, the computer system CS performs the acquisition step S1, the preprocessing step S2, and the parameter setting step S3 in the same manner as described above.

[0087] Then, the image processing unit 1c sets the multiple binarization candidate images Gc as multiple input images. In this modification, seven binarization candidate images Gc(1) to Gc(7) are used as the multiple binarization candidate images Gc (see FIG. 14). In a similarity derivation step S51, the image processing unit 1c calculates the mutual similarities of the binarization candidate images Gc(1) to Gc(7).

[0088] Next, in a combination step S52, the image processing unit 1c sequentially extracts two binarization candidate images Gc from the binarization candidate images Gc(1) to Gc(7) in descending order of similarity, and includes the two extracted binarization candidate images Gc in the same set. Specifically, in Fig. 14, the binarization candidate images Gc(1), Gc(2), the binarization candidate images Gc(3), Gc(4), the binarization candidate images Gc(5), and Gc(6) are each included in the same set.

[0089] Next, in a position correction step S53, the image processing unit 1c performs position correction on two binarization candidate images Gc belonging to the same group. Specifically, as shown in Fig. 14, the image processing unit 1c performs position correction on the binarization candidate images Gc(1) and Gc(2) belonging to the same group. The image processing unit 1c performs position correction on the binarization candidate images Gc(3) and Gc(4) belonging to the same group. The image processing unit 1c performs position correction on the binarization candidate images Gc(5) and Gc(6) belonging to the same group.

[0090] Next, in a synthesis step S54, the image processing unit 1c synthesizes the two binarized candidate images Gc that have been subjected to position correction to create a synthetic image Gd. Specifically, as shown in Fig. 14, the image processing unit 1c synthesizes the binarized candidate images Gc(1) and Gc(2) that have been subjected to position correction to create a synthetic image Gd(1). The image processing unit 1c synthesizes the binarized candidate images Gc(3) and Gc(4) that have been subjected to position correction to create a synthetic image Gd(2). The image processing unit 1c synthesizes the binarized candidate images Gc(5) and Gc(6) that have been subjected to position correction to create a synthetic image Gd(3).

[0091] Next, in a data storage step S55, the image processing unit 1c stores each data of the composite images Gd(1)-Gd(3) in the memory unit 1b and deletes each data of the binarization candidate images Gc(1)-Gc(6) from the memory unit 1b. In this case, the memory unit 1b stores the binarization candidate image Gc(7) and each data of the composite images Gd(1)-Gd(3) as data of the output image.

[0092] Next, in a completion determination step S56, the image processing unit 1c determines whether the synthesis process is complete. Specifically, if the number of output images stored in the storage unit 1b is two or more, the image processing unit 1c determines that the synthesis process is not complete. If the number of output images stored in the storage unit 1b is one, the image processing unit 1c determines that the synthesis process is complete. Here, the storage unit 1b stores the data of the binarization candidate image Gc(7) and the composite images Gd(1)-Gd(3), and since the number of output images is two or more, the image processing unit 1c determines that the synthesis process is not complete. If the image processing unit 1c determines that the synthesis process is not complete, it returns to the similarity derivation step S51 with the binarization candidate image Gc(7) and the composite images Gd(1)-Gd(3) stored in the storage unit 1b as input images.

[0093] Then, in a similarity derivation step S51, the image processing unit 1c obtains the mutual similarities between the binarization candidate image Gc(7) and the composite images Gd(1) to Gd(3). Next, in a combination step S52, the image processing unit 1c sequentially extracts two images from the binarization candidate image Gc(7) and the composite images Gd(1) to Gd(3) in descending order of similarity, and includes the two extracted binarization candidate images Gc in the same group. Specifically, in FIG. 14, the composite images Gd(1) and Gd(2) are in the same group. Next, in a position correction step S53, the image processing unit 1c performs position correction of the composite images Gd(1) and Gd(2) belonging to the same group. Next, in a combination step S54, the image processing unit 1c creates a composite image Gd(4) by combining the two composite images Gd(1) and Gd(2) that have been subjected to position correction.

[0094] Next, in a data storage step S55, the image processing unit 1c stores the data of the composite image Gd(4) in the memory unit 1b and deletes the data of the composite images Gd(1) and Gd(2) from the memory unit 1b. As a result, the memory unit 1b stores the binarization candidate image Gc(7) and the data of the composite images Gd(3) and Gd(4) as data of the output image.

[0095] Next, in a completion determination step S56, the image processing unit 1c determines whether the synthesis process is complete. Here, since the storage unit 1b stores the data of the binarization candidate image Gc(7) and the composite images Gd(3) and Gd(4), and the number of output images is two or more, the image processing unit 1c determines that the synthesis process is not complete. When the image processing unit 1c determines that the synthesis process is not complete, it returns to the similarity derivation step S51 with the binarization candidate image Gc(7) and the composite images Gd(3) and Gd(4) stored in the storage unit 1b as input images.

[0096] Then, in a similarity derivation step S51, the image processing unit 1c obtains the mutual similarity between the binarization candidate image Gc(7) and the composite images Gd(3) and Gd(4). Next, in a combination step S52, the image processing unit 1c sequentially extracts two images from the binarization candidate image Gc(7) and the composite images Gd(3) and Gd(4) in descending order of similarity, and includes the two extracted binarization candidate images Gc in the same set. Specifically, in FIG. 14, the composite images Gd(3) and Gd(4) are in the same set. Next, in a position correction step S53, the image processing unit 1c performs position correction of the composite images Gd(3) and Gd(4) belonging to the same set. Next, in a combination step S54, the image processing unit 1c creates a composite image Gd(5) by combining the two composite images Gd(3) and Gd(4) that have been subjected to position correction.

[0097] Next, in a data storage step S55, the image processing unit 1c stores the data of the composite image Gd(5) in the memory unit 1b and deletes the data of the composite images Gd(3) and Gd(4) from the memory unit 1b. As a result, the memory unit 1b stores the data of the binarization candidate image Gc(7) and the composite image Gd(5) as data of the output image.

[0098] Next, in a completion determination step S56, the image processing unit 1c determines whether the synthesis process is complete. Here, since the storage unit 1b stores the data of the binarization candidate image Gc(7) and the composite image Gd(5), and the number of output images is two or more, the image processing unit 1c determines that the synthesis process is not complete. When the image processing unit 1c determines that the synthesis process is not complete, it returns to the similarity derivation step S51 with the binarization candidate image Gc(7) and the composite image Gd(5) stored in the storage unit 1b as input images.

[0099] Then, in a similarity derivation step S51, the image processing unit 1c calculates the similarity between the binarization candidate image Gc(7) and the composite image Gd(5). Next, in a combination step S52, the image processing unit 1c includes the binarization candidate image Gc(7) and the composite image Gd(5) in the same group. Next, in a position correction step S53, the image processing unit 1c performs position correction on the binarization candidate image Gc(7) and the composite image Gd(5) that belong to the same group. Next, in a combination step S54, the image processing unit 1c creates the composite image Gd(6) by combining the binarization candidate image Gc(7) and the composite image Gd(5) that have been subjected to position correction.

[0100] Next, in a data storage step S55, the image processing unit 1c stores the data of the composite image Gd(6) in the memory unit 1b, and deletes the data of the binarization candidate image Gc(7) and the composite image Gd(5) from the memory unit 1b. As a result, the memory unit 1b stores the data of the composite image Gd(6) as data of the output image.

[0101] Next, in a completion determination step S56, the image processing unit 1c determines whether the synthesis process is complete. Here, since the storage unit 1b stores data of the synthetic image Gd(6) and the number of output images is one, the image processing unit 1c determines that the synthesis process is complete. When the image processing unit 1c determines that the synthesis process is complete, in a determination step S57, the synthetic image Gd(6) stored in the storage unit 1b is set as the template image Gt.

[0102] In this embodiment, the similarity deriving step S51 and the combination step S52 are performed each time using a composite image. However, all combinations of multiple images may be determined first using hierarchical cluster analysis or the like.

[0103] As described above, the template image creation method of this embodiment creates a template image Gt using a plurality of binarization candidate images Gc in which the positions of the target regions Ra1 to Ra6 are shifted from one another. Specifically, the template image creation method of this embodiment creates a template image Gt by combining a plurality of binarization candidate images Gc in which the positional shifts have been corrected by position correction using pattern matching. As a result, the template image creation method of this embodiment can create a high-precision template image Gt with little noise even if the positions of the objects to be inspected shown in the plurality of binarization candidate images Gc are not aligned.

[0104] (7) Fifth Modification In the template image creation method of the fifth modified example, if there are multiple output images in the above-mentioned fourth modified example, the mutual similarities of the multiple output images are calculated. Then, if all mutual similarities of the multiple output images are equal to or less than a similarity threshold, each of the multiple output images is set as a template image Gt. In this case, even if the characteristics of the target region Ra differ for each lot of the inspection object, the accuracy of template matching can be improved by creating multiple template images Gt corresponding to each characteristic.

[0105] For example, in FIG. 14, if the mutual similarity between the composite images Gd(1), Gd(2), and the binarization candidate image Gc(7) is equal to or greater than a predetermined similarity threshold, the process of the fourth modified example is continued. If the mutual similarity between the composite images Gd(1), Gd(2), and the binarization candidate image Gc(7) is less than a predetermined similarity threshold, each of the composite images Gd(1), Gd(2), and the binarization candidate image Gc(7) is set as a template image Gt. In this case, the template image creation system 1 creates three template images Gt as a template set. Also, one or two of the three template images may be set as a template set.

[0106] (8) Sixth Modification The template image creation method of the sixth modified example further includes a display step S61 and a selection step S62 shown in Fig. 15 in addition to the above-mentioned fourth modified example. The display step S61 displays a tree structure on the display unit 1d (see Fig. 2) in which a plurality of input images and output images of the image processing are each a node. The selection step S62 selects at least one of the plurality of input images and output images displayed on the display unit 1d as a template image.

[0107] The appearance of the object shown in the candidate image may vary significantly depending on the manufacturing lot, product type, material type, or manufacturing or inspection conditions of the object, etc. Examples of such appearance include the shape, pattern, size, and two-dimensional code printed on the surface of the object.

[0108] For example, the shading candidate image Gb(101) in FIG. 16 includes an area in which the object to be inspected is shown as a target area Ra101. The shading candidate image Gb(102) includes an area in which the object to be inspected is shown as a target area Ra102. The shading candidate image Gb(103) includes an area in which the object to be inspected is shown as a target area Ra103. Then, by combining the shading candidate images Gb(101), Gb(102), and Gb(103), a combined image Gd(100) including a target area Ra100 is created. The target area Ra100 is the area where the target areas Ra101, Ra102, and Ra103 overlap. However, since the target regions Ra101, Ra102, and Ra103 are significantly different from each other, the similarity between the target regions Ra100 and Ra101, the similarity between the target regions Ra100 and Ra102, and the similarity between the target regions Ra100 and Ra103 are all small. Therefore, the accuracy of the template image Gt created based on the composite image Gd(100) is low, and the template image Gt contains a lot of noise.

[0109] Therefore, the computer system CS executes a template image creation method similar to that of the fourth modified example, using as input images each of the shading candidate images Gb(1)-Gb(4), Gb(11), Gb(12), and Gb(21) showing the object to be inspected shown in Fig. 17. Here, the object to be inspected shown in each of the shading candidate images Gb(1)-Gb(4) is significantly different from the object to be inspected shown in each of the shading candidate images Gb(11) and Gb(12). Furthermore, the shading candidate image Gb(21) is a distorted image and is a defective image.

[0110] In this case, the computer system CS creates a set of grayscale candidate images Gb(1) and Gb(2), a set of grayscale candidate images Gb(3) and Gb(4), and a set of grayscale candidate images Gb(11) and Gb(12). The computer system CS aligns and combines the two grayscale candidate images Gb of each set to create a composite image of each set as an output image. Furthermore, the computer system CS creates a set of two composite images using multiple composite images as input images, and aligns and combines the composite images of each set to create a composite image of each set as an output image. The computer system CS repeats the above process using multiple composite images as input images, and combines the grayscale candidate image Gb(21) to finally create one composite image.

[0111] The display unit 1d displays a tree structure Q1 (see FIG. 17) in which each of the multiple input images and output images of the above-mentioned image processing is a node. The tree structure Q1 includes nodes P1 to P6 corresponding to each composite image.

[0112] Then, the inspector operates the operation unit 1e to select any one of the nodes in the tree structure Q1. For example, when the inspector selects node P2, the display unit 1d displays a composite image Gd(b) having relatively more noise. When the inspector selects node P4, the display unit 1d displays a composite image Gd(a) having relatively less noise. When the inspector selects node P6, the display unit 1d displays a composite image Gd(c) having very more noise. That is, the inspector can check each composite image by having each composite image displayed on the display unit 1d. Then, the inspector sets a high-precision, low-noise composite image (for example, composite image Gd(a)) as the template image Gt.

[0113] In this modified example, the inspector does not need to repeat trial and error of parameter tuning and result confirmation when selecting a template image Gt, and can select efficiently.

[0114] (9) Seventh Modification It is preferable that the grayscale candidate image Gb and the binarized candidate image Gc are images with a resolution of 1 μm / pix or less.

[0115] Furthermore, the grayscale candidate image Gb and the binarized candidate image Gc may be images in which the target region Ra is not clearly visible even at the limit of optical zoom.

[0116] Furthermore, the grayscale candidate image Gb and the binarized candidate image Gc may be images in which the characteristics of the object to be inspected cannot be clearly captured with the resolution of the imaging device.

[0117] Moreover, the grayscale candidate image Gb and the binarized candidate image Gc may be either an image of the surface of the object to be inspected or a transmission image of the inside of the object to be inspected.

[0118] The grayscale candidate image Gb and the binarized candidate image Gc may be images captured without optical zoom. In this case, the range that can be captured at one time is expanded, and the inspection speed can be improved. In addition, the range of the focal depth is expanded, and a candidate image with less blurring can be created.

[0119] Furthermore, the grayscale candidate image Gb and the binarized candidate image Gc may be images with gradation, in which case noise remaining in the template image Gt after edge detection can be suppressed.

[0120] In addition, the candidate image, composite image, and template image may be either a grayscale image or a binary image. In the composition of grayscale images, the average, median, weighted average, maximum, or minimum value of the grayscale values ​​of each pixel of the grayscale images is set as the grayscale value of each pixel of the composite image. In the composition of binary images, the logical sum or logical product of the grayscale values ​​of each pixel of the binary images is set as the grayscale value of each pixel of the composite image.

[0121] In addition, the template image may be an image obtained by performing median filtering, Gaussian filtering, histogram equalization, normalization, standardization, binarization, edge detection, or a combination of two or more of the above on the synthesized image.

[0122] Also, in the synthesis step, three or more images may be synthesized at once.

[0123] (10) Summary The template image creation method of the first aspect according to the above-mentioned embodiment creates a template image (Gt) from a plurality of candidate images (Gb, Gc) each including a target area (Ra) in which an object to be inspected is captured. The template image creation method performs position correction by pattern matching to align the positions of the target areas (Ra) of the plurality of candidate images (Gb, Gc), and creates at least one template image (Gt) by sequentially synthesizing the plurality of candidate images (Gb, Gc).

[0124] Therefore, the template image creating method can create a highly accurate template image (Gt) with little noise even if the positions of the objects to be inspected are not aligned in the multiple candidate images (Gb, Gc).

[0125] The template image creating method of the second aspect according to the above-mentioned embodiment preferably includes, in the first aspect, a parameter setting step (S3) of setting parameters relating to position correction.

[0126] Therefore, the template image creation method can limit the direction of position correction, thereby reducing the calculation cost required for position correction.

[0127] The template image creation method of the third aspect according to the above-mentioned embodiment preferably includes an extraction step (S4, S23), a position correction step (S5, S26), a synthesis step (S6, S27), and a determination step (S10, S34, S43) in the first or second aspect. The extraction step (S4, S23) sequentially extracts one candidate image (Gb) from a plurality of candidate images (Gb). The position correction step (S5, S26) performs position correction on all the candidate images (Gb) extracted in the extraction step (S4, S23) every time one candidate image (Gb) is extracted in the extraction step (S4, S23). The synthesis step (S6, S27) creates a synthesis image (Gd) by synthesizing all the candidate images (Gb) that have been subjected to position correction every time position correction is performed. The determination steps (S10, S34, S43) determine the composite image (Gd) created by the final synthesis step (S6, S27) among the multiple synthesis steps (S6, S27) as the template image (Gt).

[0128] Therefore, the template image creating method can create a highly accurate template image (Gt) with little noise even if the positions of the objects to be inspected are not aligned in the multiple candidate images (Gb, Gc).

[0129] In the template image creation method of the fourth aspect according to the above-mentioned embodiment, in the third aspect, when the extraction step (S4, S23) extracts the Mth (M is a positive integer) candidate image (Gb) from the multiple candidate images (Gb), the position correction step (S5, S26) preferably aligns the positions of the target regions (Ra) of the Mth candidate image (Gb) and the composite image (Gd) obtained by combining the 1st to M-1th candidate images (Gb) by pattern matching. The composition step (S6, S27) combines the composite image (Gd) with the Mth candidate image (Gc).

[0130] Therefore, the template image creating method can create a highly accurate template image (Gt) with little noise even if the positions of the objects to be inspected are not aligned in the multiple candidate images (Gb, Gc).

[0131] In the template image creation method of the fifth aspect of the above-mentioned embodiment, in any one of the first to fourth aspects, it is preferable to set at least one of whether or not to combine multiple candidate images (Gc) and the order of combination based on the mutual similarity of the multiple candidate images (Gc).

[0132] Therefore, the template image creation method optimizes at least one of whether or not multiple binary candidate images (Gc) can be combined and the order of combination, and can generate a high-precision, low-noise template image (Gt) even if it includes a binary candidate image (Gc) with high noise.

[0133] In the template image creation method of the sixth aspect according to the above-mentioned embodiment, in the first or second aspect, it is preferable to perform image processing including a combination step (S52), a position correction step (S53), and a synthesis step (S54). The combination step (S52) performs a combination process of generating a set including two or more input images from a plurality of input images. The position correction step (S53) performs position correction on the two or more input images for each set. The synthesis step (S54) creates an output image corresponding to the set by synthesizing the two or more input images that have been subjected to position correction. Then, after performing image processing using a plurality of candidate images (Gc) as a plurality of input images, if there are a plurality of output images, image processing using the plurality of output images as a plurality of input images is repeated until the output image by the image processing satisfies a predetermined condition.

[0134] Therefore, the template image creating method can create a highly accurate template image (Gt) with little noise even if the positions of the objects to be inspected are not aligned in the multiple candidate images (Gb, Gc).

[0135] In the template image creation method of the seventh aspect of the above-mentioned embodiment, in the sixth aspect, it is preferable to calculate the mutual similarity between each of a plurality of input images, sequentially extract two or more input images from the plurality of input images in order of highest similarity, and include the extracted two or more input images in the same set.

[0136] Therefore, the template image creation method optimizes the combination of multiple binary candidate images (Gc) and can generate a high-precision, low-noise template image (Gt) even if it includes a binary candidate image (Gc) with high noise.

[0137] In the template image creation method of the eighth aspect of the above-mentioned embodiment, in the sixth or seventh aspect, if there are multiple output images, it is preferable to calculate the mutual similarities between the multiple output images, and if all of the mutual similarities between the multiple output images are less than a similarity threshold, to set each of the multiple output images as a template image (Gt).

[0138] Therefore, even if the features of the target area (Ra) differ for each lot of inspection objects, the template image creation method can improve the accuracy of template matching by creating multiple template images (Gt) corresponding to each feature.

[0139] The template image creation method of the ninth aspect according to the above-mentioned embodiment, in any one of the sixth to eighth aspects, preferably further includes a display step (S61) and a selection step (S62). The display step (S61) displays on the display unit (1d) a tree structure (Q1) in which each of a plurality of input images and output images of the image processing is a node. The selection step (S62) selects at least one of the plurality of input images and output images displayed on the display unit (1d) as a template image (Gt).

[0140] Therefore, in the template image creating method, the examiner can efficiently select a template image (Gt) without having to repeat trial and error of parameter tuning and result confirmation.

[0141] The template image creation system (1) of the tenth aspect according to the above-mentioned embodiment creates a template image (Gt) from a plurality of candidate images (Gc) each including a target area (Ra) in which an object to be inspected is captured, and the template image creation system (1) includes an image processing unit (1c). The image processing unit (1c) performs position correction for aligning the positions of the target areas (Ra) of the plurality of candidate images (Gc) by pattern matching, and creates at least one template image (Gt) by sequentially synthesizing the plurality of candidate images (Gb).

[0142] Therefore, the template image creation system (1) can create a highly accurate template image (Gt) with little noise even if the positions of the objects to be inspected shown in the multiple candidate images (Gb, Gc) are not aligned.

[0143] It is preferable that the template image creation system (1) of the eleventh aspect according to the above-mentioned embodiment further includes an image acquisition unit (1a) for acquiring a plurality of candidate images (Gb, Gc) in the tenth aspect.

[0144] Therefore, the template image creation system (1) can obtain multiple candidate images from an external database, camera, storage medium, etc.

[0145] A program according to a twelfth aspect of the above-mentioned embodiment causes a computer system (CS) to execute the template image creating method according to any one of the first to ninth aspects.

[0146] Therefore, even if the positions of the objects to be inspected shown in the multiple candidate images (Gb, Gc) are not aligned, the program can create a highly accurate template image (Gt) with little noise. [Explanation of symbols]

[0147] S3 Parameter Setting Step S4, S23 Extraction steps S5, S26 Position correction steps S6, S27 synthesis steps S10, S34, S43 Decision steps S52 Combination Step S53 Position correction step S54 Synthesis Step S61 Display Step S62 Selection Step Ra target area Gb Grayscale candidate image (candidate image) Gc Binarization candidate image (candidate image) Gd composite image Gt template image Q1 Tree structure CS Computer System M positive integer 1 Template image creation system 1a Image acquisition section 1c Image processing section 1d display section

Claims

1. A template image creation method for creating a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured, comprising the steps of: performing position correction for aligning the positions of the target regions of the plurality of candidate images by pattern matching, and creating at least one of the template images by sequentially combining the plurality of candidate images; a combining step of performing a combining process for generating a set including two or more input images from the plurality of input images; a position correction step of performing the position correction on the two or more input images for each of the sets; a synthesis step of synthesizing the two or more input images that have been subjected to the position correction to generate an output image corresponding to the set; After performing the image processing using the plurality of candidate images as the plurality of input images, if there are a plurality of output images, the image processing using the plurality of output images as the plurality of input images is repeatedly performed until the output image by the image processing satisfies a predetermined condition. How to create a template image.

2. A degree of similarity between the plurality of input images is calculated, and the two or more input images are sequentially extracted from the plurality of input images in descending order of degree of similarity, and the two or more extracted input images are included in the same set. The template image creating method according to claim 1.

3. If the output image is a plurality of images, calculating a mutual similarity of each of the plurality of output images; If all of the mutual similarities of the plurality of output images are less than a similarity threshold, each of the plurality of output images is set as the template image.

3. The template image creating method according to claim 1 or 2.

4. a display step of displaying on a display unit the plurality of input images and the output image of the image processing in a tree structure having nodes corresponding to the plurality of input images and the output image; and selecting at least one of the plurality of input images and the output image displayed on the display unit as the template image.

4. The template image creating method according to claim 1.

5. A template image creation method for creating a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured, comprising the steps of: performing position correction for aligning the positions of the target regions of the plurality of candidate images by pattern matching, and creating at least one of the template images by sequentially combining the plurality of candidate images; A synthesis order of the plurality of candidate images is set based on the mutual similarity of the plurality of candidate images. How to create a template image.

6. An extraction step of sequentially extracting one candidate image from the plurality of candidate images; a position correction step of performing the position correction on all the candidate images extracted in the extraction step every time one of the candidate images is extracted in the extraction step; a synthesis step of synthesizing all the candidate images subjected to the position correction each time the position correction is performed to generate a synthetic image; and a step of determining that a composite image created by a final composite step among the multiple composite steps is to be the template image. The template image creating method according to claim 5.

7. When the extraction step extracts an Mth candidate image (M is a positive integer) from the plurality of candidate images, The position correction step aligns the positions of the target regions of the composite image obtained by combining the 1st to (M−1)th candidate images and the Mth candidate image by pattern matching; The synthesis step synthesizes the synthetic image and the M-th candidate image.

7. The template image creating method according to claim 6.

8. A template image creation system for creating a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured, comprising: an image processing unit that performs position correction for aligning the positions of the target regions of the plurality of candidate images by pattern matching, and creates at least one of the template images by sequentially synthesizing the plurality of candidate images; The image processing unit includes: A combination process for generating a set including two or more input images from the plurality of input images; a position correction process for performing the position correction on the two or more input images for each of the sets; a synthesis process for generating an output image corresponding to the set by synthesizing the two or more input images that have been subjected to the position correction; After performing the image processing using the plurality of candidate images as the plurality of input images, if there are a plurality of output images, the image processing using the plurality of output images as the plurality of input images is repeatedly performed until the output image by the image processing satisfies a predetermined condition. Template image creation system.

9. A template image creation system that creates a template image from a plurality of candidate images each including a target area in which an object to be inspected is captured, an image processing unit that performs position correction for aligning the positions of the target regions of the plurality of candidate images by pattern matching, and creates at least one of the template images by sequentially synthesizing the plurality of candidate images; The image processing unit sets an order of combining the plurality of candidate images based on the mutual similarity between the plurality of candidate images. Template image creation system.

10. Further comprising an image acquisition unit for acquiring the plurality of candidate images.

10. The template image creating system according to claim 8 or 9.

11. A program for causing a computer system to execute the template image creation method of any one of claims 1 to 7.

Images