Pattern model positioning method in image processing, image processing apparatus, image processing program and computer-readable recording medium

Abstract

A pattern-model positioning method in image processing for searching an image to be searched (OI) and for positioning an object to be searched that resembles a pre-registered image (RI) using a pattern model (PM) corresponding to the registered image (RI), positioning with a higher accuracy than an initially given position, the method comprising the steps of: Extracting an outline from the registered image (RI), Constructing a pattern model (PM) of the registered image (RI) in which a plurality of reference points are set on the extracted outline, and also a corresponding point search line having a predetermined length and passing through each reference point, as well as being substantially orthogonal to the outline, is also assigned to each reference point; Acquiring an image to be searched (OI) and also arranging the pattern model (PM) so that it is superimposed on the image to be searched (OI), based on an initial position contained in the image to be searched (OI) and corresponding to the registered image (RI); Finding a corresponding edge point on the image to be searched (OI), corresponding to a reference point of each corresponding point search line of the pattern model (PM) from each pixel in the image to be searched (OI) positioned superimposed on the corresponding point search line, when superimposing the pattern model (PM) on the initial position of the image to be searched (OI); and Taking a distance between each corresponding edge point and an outline containing the reference point corresponding to the corresponding edge point as an evaluation value, and performing fine positioning with a higher accuracy than the accuracy at the given initial position so that an absolute value of an accumulated value of the evaluation values ​​becomes minimum, wherein, in the case of a plurality of corresponding edge point candidates present on the corresponding point search line, the step of finding the corresponding edge point on the image to be searched regards as the corresponding edge point a point having: an edge strength that is greater than an edge strength threshold and shows the maximum, and an edge angle that is sufficiently close to an ideal edge angle and also closest to the reference point.

Description

BACKGROUND OF THE INVENTIONField of the invention

[0001] The present invention relates to a pattern model positioning method in image processing, an image processing apparatus, an image processing program and a computer-readable recording medium in searching an image to be searched and positioning a pattern model corresponding to a pre-registered image. Description of related prior art

[0002] An image processing device for processing an image captured by an image pickup element typically includes: an image pickup device for capturing an image processing object (hereinafter also referred to as a "work"); an image data storage device for storing data on the image captured by the image pickup device; and an image data processing device for processing the image data stored by the image data storage device. For example, in the image processing device in which the image pickup device is constructed of a CCD camera, luminance data (so-called multi-value data) such as 256 gray levels or 1024 gray levels is obtained based on each charge amount of a large number of charge-coupled devices constituting the image pickup surface, whereby a position, rotation angle, and the like of the work as an object to be searched can be found.Conventionally, as techniques for performing processing on image data to search for a searchable object in image processing, a difference search performed using a total value of absolute values ​​of pixel difference values ​​between images, a normalized correlation search performed using normalized correlation values ​​between images, and the like are known. In these searches, an object to be searched for is registered in advance as a template image, and based on the image, a search for the object to be searched is performed from an image to be searched. In this search processing, a major trend has conventionally been a region-based search using image data.However, a conventional area-based search based on image thickness or the like has the problem of being sensitive to a change in illumination during image acquisition and the like.

[0003] Meanwhile, a method for performing edge extraction processing on a registered image and a search image to perform a search based on edge information has also been provided. This method does not use the concentration value of pixels constituting image data, but uses edge data based on a change in the concentration value, and therefore, it is possible to obtain the advantage of being immune to fluctuations in illumination during image acquisition. In particular, in recent years, edge-based pattern search using an edge as a characteristic set has attracted attention due to its high robustness and is in practical use in industrial applications and the like.

[0004] As a technique for improving the processing speed of pattern search, a coarse-to-fine approach is known. First, a coarse search is performed using a low-resolution image (coarse image), and after specifying a coarse position, detailed positioning is performed using a high-resolution image (fine image), thereby improving the accuracy of position and pose. When performing edge-based search using coarse-to-fine template matching, a pyramid search is used, in which a search is performed using coarse data obtained by compressing (also called "thinning") original data to specify a raw position, and then a search is performed using detailed data. Fig. Figure 87 shows a concept of pyramid search. As shown in this drawing, a raw search (referred to as a "coarse search" or the like) is performed using a low-resolution image with a high reduction ratio to find a coarse position. Next, a search is performed in the vicinity of the same with an improved resolution and a medium reduction ratio, and finally, a fine search is performed on an image of an original size or an image with a reduction ratio close to the original size. As thus described, in typical pyramid search, a plurality of images with changed resolutions are prepared, and a schematic position is first detected using an image with the lowest resolution. In subsequent processing, a search range is narrowed to the vicinity of the previously detected position while gradually increasing the resolution.As a result, the accuracy of the detected position increases with each subsequent processing level, ultimately leading to the detection of a highly accurate position with a resolution equal to or closer to that of the original image. As a technique related to fine positioning for finding a position and posture with high accuracy using such coarse-to-fine template matching, an image processing device disclosed in Japanese Patent No. 3759983 is known.

[0005] This image processing apparatus includes: an image pickup device for picking up an image processing object; an image data storage device for storing data on the image captured by the image pickup device; a search template data storage device for storing data on a search template comprising a plurality of point pairs, each consisting of two points sandwiching an edge of the image processing object and located at a fixed distance from each other that is greater than a gate pitch of the image pickup devices of the image pickup device and that are preset according to a plurality of positions on the edge;and a determination device for setting the image processing object as an object to be searched that matches the search template of the search template data storage device when the point pairs are taken to be in a matching state in which one point of the pair is located within the edge of the image processing object and the other is located within the background, and the point pairs are in a number not less than a set number among a plurality of point pairs in a matching state, in a case where the search template is superimposed on a screen with image data of the image data storage device present therein, and in a difference state of an optical characteristic value of each point constituting the plurality of point pairs is greater than a set state;

[0006] According to this device, it is possible to see the presence or absence of the object to be searched for, and whether the image processing object is the object to be searched for or not, using the image data and the search template. These can be determined by calculating and comparing optical characteristic values ​​of two points constituting each of all point pairs, so that an image processing device capable of performing processing easily and quickly can be obtained.

[0007] In this fine positioning, as described above, one-dimensional edge processing is performed on a search line to find an edge position, and a difference between an ideal point and the corresponding point is found. Using this reference and geometric information inherent in a model (information such as that a model is rectangular, etc.), an offset and a rotation angle are found. Fig. 88 shows the relationship between a pattern detected during registration and a search line SLN, and on the other hand shows Fig. 89 a state immediately before fine positioning after a coarse search during movement. Fig. 90 shows an enlarged view of the lower left area of Fig. 89. As an edge kernel, (-1, -1, -1, -1, 1, 1, 1, 1), shown in Table 1, is used. In edge detection, an edge is found by applying this kernel to a pixel value obtained by interpolation on the search line SLN. Table 1 - - - - + + + +

[0008] However, high accuracy cannot be expected from this technique because whether the corresponding point exists on the search line or not is determined based only on the relationship between a magnitude of a difference value and its threshold value, and an edge direction and an angle component are not considered. Furthermore, this technique has had a drawback in that no processing can be performed in a case where a plurality of points exist as candidates of the corresponding point on the search line; for example, a double line cannot be processed. Moreover, with this method, although the positioning of a simple shape such as a rectangle and a circle can be realized, positioning of a general and general-purpose shape such as a more complicated shape and a character pattern cannot be performed.Furthermore, there has been a problem that accurate edge detection cannot be performed on a thin linear object due to the long kernel of edge detection. Namely, a difference along the line is calculated by the one-dimensional kernel in the longitudinal direction to detect an edge. In particular, the kernel needs to be made longer to improve noise resistance. Consequently, there has been a drawback that when a work itself is long and narrow, edge data becomes relatively weak to be considered a noise component, and therefore accurate edge detection cannot be expected.

[0009] Meanwhile, a fine positioning method by a least-squares method using pixels and segments is disclosed in IJ Kox, JW Cruscal, and D.A. Wallach, "Predicting and Estimating the Performance of a Subpixel Registration Algorithm," IEEE Trans. Pattern. Anal. Machine Intel., Vol. 12, No. 6, pp. 721-734 (August 1990). In this method, a pattern model is configured from segments. Furthermore, error detection processing is performed on a not-yet-input image to be searched to generate an edge image including imaginary points.The pattern model is superimposed on a position in the edge image determined by a coarse search, and a least-squares method is performed using a distance, as an error value, between a point and a straight line, which is each pixel in the image to be searched and a segment within the pattern model that is close to the pixel, to decide a fine position. In the example of . Fig. 91 shows the relationship between a pattern model configured from two segments and pixels (input pixels) in an image to be searched.

[0010] However, there has been a problem with this technique in that, since the corresponding relationship between the segment and the point depends only on the distance between them, a point may be calculated as the corresponding point itself in an inherently non-corresponding relationship with the segment in terms of the edge direction or the like. Furthermore, there has also been a problem in that, due to the need to detect imaginary points in the image to be searched, edge detection must be performed at each point in a high-resolution image as an object to be searched, and therefore, "round robin" processing is performed, resulting in an increase in processing time with the increase in the resolution of the image data.

[0011] Cox et al., IEEE Transaction on Pattern Analysis and Machine Intelligence, 1990, p. 721ff describes an approach to product assembly, particularly for component assembly on circuit boards, using image recognition and the registration of images at subpixel level.

[0012] US 2002 / 0 057 838 A1 is directed to a further method for object detection of objects in different positions relative to a detection camera. SUMMARY OF THE INVENTION

[0013] The present invention has been conceived to solve the conventional problems thus described. A primary object of the present invention is to provide a pattern model positioning method in image processing, an image processing apparatus, an image processing program, and a computer-readable recording medium that can perform fine positioning with higher accuracy and higher speed.

[0014] To achieve the above object, a first pattern model positioning method in image processing of, when searching, an image to be searched, and positioning an object to be searched that resembles a pre-registered image using a pattern model corresponding to the registered image, positioning with a higher accuracy than an initially given position, the method is capable of including the steps of: extracting an outline from the registered image; constructing a pattern model of the registered image in which a plurality of reference points are set on the extracted outline, and also a corresponding point search line having a predetermined length and passing through each reference point, as well as being substantially orthogonal to the outline, is also assigned to each reference point; acquiring an image to be searched and also arranging the pattern model,so that it is superimposed on the image to be searched, based on an initial position contained in the image to be searched and corresponding to the registered image; finding a corresponding edge point on the image to be searched, corresponding to a reference point of each corresponding point search line of the pattern model from each pixel in the image to be searched, positioned superimposed on the corresponding point search line, while superimposing the pattern model on the initial position of the image to be searched; and taking a distance between each corresponding edge point and an outline containing the reference point corresponding to the corresponding edge point as an evaluation value, and performing fine positioning with a higher accuracy than the accuracy at the given initial position so that an absolute value of an accumulated value of the evaluation values ​​becomes minimal. This makes it possible toUse data on the corresponding point search line as the pattern model for fine positioning. Specifically, by using not only the edge strength of the image but also its edge angle, it is possible to add a directional component to perform highly accurate positioning that is resistant to a noise component. Furthermore, by changing the length of the corresponding point search line, it is possible to obtain an advantage in that one is able to easily change the corresponding edge point search range, thus adjusting the positioning accuracy.

[0015] According to a second pattern model positioning method in image processing, the step of extracting the outline from the registered image may include the steps of: extracting a plurality of edge points from the registered image; linking adjacent edge points from the extracted plurality of edge points to create a continuous chain; and generating a segment by each approximating one or more chains by line and / or circular arc, and viewing the aggregation of the segments as an outline. This makes it possible to construct a pattern model consisting of segments approximated by line and / or circular arc, thereby realizing positioning with higher accuracy.

[0016] According to a third pattern model positioning method in image processing, the step of performing fine positioning may be performed to calculate an error value or a weight value in a least-squares method using the distance between each segment constituting the outline and a corresponding edge point thereof to find the accumulated value of the evaluation values. This makes it possible to find a position and a posture of the pattern model by solving simultaneous equations obtained by calculating a least-squares method with respect to the corresponding edge point of each reference point.

[0017] According to a fourth pattern model positioning method in image processing, the least squares method is used in a fine positioning step to apply an error function obtained by calculating a distance between the corresponding edge point and a circular arc to a circular arc segment and an error function obtained by calculating a distance between the corresponding edge point and a straight line to a line segment. This makes it possible to realize fine positioning by the effective least squares method without hindering the rotational movement of the circular arc segment.

[0018] According to a fifth pattern model positioning method in image processing, the step of generating the segment with respect to the chains repeats an operation of first attempting to approximate the chains by the line and then switching to approximation by the circular arc when an error of approximation of the line exceeds a predetermined approximation threshold to generate a sequence of segments.

[0019] According to a sixth pattern model positioning method in image processing, the segment may be configured from a cone curve, a spline curve and / or a Bezier curve or a combination thereof.

[0020] According to a seventh pattern model positioning method in image processing, a preset margin can be set at the edge of the segment when setting the reference point on the outline, and the reference point is set in an area excluding the margin area. Since the setting can be made so that the corresponding point search line is not provided at the end area and corner area, it is possible to eliminate an area that becomes unstable due to a large change in the edge angle, thus obtaining a stable pattern matching result.

[0021] According to an eighth pattern model positioning method in image processing, the initial position corresponding to the registered image included in the search image can be acquired in the step of arranging the pattern model to be superimposed on the search image by reducing the search image and performing pattern search on the search image after reduction. This makes it possible to perform a simple coarse search at high speed on a reduced search image.

[0022] According to a ninth pattern model positioning method in image processing, the corresponding point search line can be formed by setting it to have a predetermined length in a direction substantially orthogonal to the segment with the reference point set on the segment used as the center. This makes it possible to set the corresponding point search line to be a line substantially orthogonal to the segment with the reference point set at the center.

[0023] According to a tenth pattern model positioning method in image processing, the predetermined length of the corresponding point search line can be decided according to a relationship between a reduction ratio of the image to be searched as an object of a coarse search, which is a search performed with a predetermined accuracy on the entire area of ​​the image to be searched, and a reduction ratio of the image to be searched to be subjected to fine positioning. This makes it possible to maintain positioning accuracy by searching at least an area corresponding to the ratio of reduction ratios of the image to be searched. For example, it is possible to make the corresponding point search line have at least a length based on the number of pixels corresponding to the reduction ratio for reducing the image to be searched.

[0024] According to an eleventh pattern model positioning method in image processing, the pattern model is capable of including grid data about the corresponding point search line, and includes, as the grid data, at least a coordinate of the reference point, an angle of the corresponding point search line, and a length of the corresponding point search line.

[0025] According to a twelfth pattern model positioning method in image processing, in the case of a plurality of corresponding edge point candidates existing on the corresponding point search line, the step of finding the corresponding edge point on the image to be searched may regard as the corresponding edge point a point having: an edge strength greater than an edge strength threshold and showing the maximum, and an edge angle sufficiently close to an ideal edge angle and also closest to the reference point.

[0026] According to a thirteenth pattern model positioning method in image processing, the step of extracting segments from the chains repeats an operation of attempting line approximation and then switching to arc approximation when the line approximation error exceeds a predetermined approximation threshold, to extract a sequence of segments. This makes it possible to perform highly accurate alignment by combining line and arc approximation.

[0027] According to a fourteenth pattern model positioning method in image processing, the method is configured to perform at least one of movement in an X direction, movement in a Y direction, rotation, and enlargement / reduction, adjustable as a degree of freedom of the least squares method. This makes it possible to handle rotation, enlargement, reduction, and the like of the registered image, as well as parallel movements in the XY directions.

[0028] According to a fifteenth model positioning method in image processing, the method is designed to further adjust one aspect of the least squares degree of freedom. This makes it possible to handle transfiguration of the registered image and the like, as well as parallel movements in the XY directions.

[0029] According to a sixteenth pattern model positioning method in image processing, the fine positioning step can be repeatedly performed while shortening the corresponding point search line. This allows the fine positioning to be performed repeatedly, thus making it possible to realize positioning with higher accuracy.

[0030] According to a seventeenth pattern model positioning method in image processing, in the step of performing fine positioning, the least squares method is repeatedly applied to the corresponding point search line, and the length of the corresponding point search line is also gradually shortened according to the number of repetitions of the least squares method. Thus, the least squares method is repeatedly applied, and it is thus possible to easily realize positioning with higher accuracy.

[0031] According to an eighteenth pattern model positioning method in image processing, in the step of performing fine positioning, the length of the corresponding point search line may be set based on a reduction ratio in the coarse search, which is a search performed with a predetermined accuracy on the entire area of ​​the image to be searched, and a reduction ratio in the fine positioning.

[0032] According to a nineteenth pattern model positioning method in image processing, in the fine positioning step, the least squares method is repeatedly applied to the corresponding point search line, and an edge angle threshold is gradually lowered according to the number of repetitions of the least squares method. This repeatedly applies the least squares method, making it possible to easily realize positioning with higher accuracy.

[0033] According to a twentieth model positioning method in image processing, in the step of performing fine positioning, when the least squares method is repeatedly applied to the corresponding point search line, the least squares method is suspended at the time when the error value exceeds a preset error value threshold, and the result is regarded as the final result. This repeatedly applies the least squares method, making it possible to easily realize positioning with higher accuracy.

[0034] According to a twenty-first pattern model positioning method in image processing, in the step of performing fine positioning, the method may be configured such that the least squares method is applied to the corresponding point search line, the least squares method is suspended at the time when the error value exceeds the preset error value threshold, an approximation performed using the error value of the least squares method completed in the previous step is applied as the final result, and the error value of the final result is also output. This makes it possible to use the error value calculated by the least squares method as an indicator showing whether the missing condition is good or bad.

[0035] According to a twenty-second pattern model positioning method in image processing, in the step of constructing the pattern model, the method can be configured to make the presence or absence of the polarity of an edge direction adjustable. This makes it possible to change the edge angle processing method according to the presence or absence of the polarity of the edge direction. Furthermore, it is possible to apply the same setting to each segment.

[0036] According to a twenty-third pattern model positioning method in image processing, in the step of constructing the pattern model, the angle and the corresponding point search line of each reference point can change according to an affine transformation value.

[0037] According to a twenty-fourth pattern model positioning method in image processing, when extracting the edge point from the registered image, Sobel processing can be performed to calculate the edge angle and edge strength of each edge point.

[0038] According to a twenty-fifth pattern model positioning method in image processing, when generating a corresponding point search line when searching the corresponding point, a calculation may be performed using Bresenham's algorithm for generating straight line data.

[0039] According to a twenty-sixth pattern model positioning method in image processing, in the pattern model construction step, a predetermined interval at which the reference points are set on the segments can be a fixed width. This makes it possible to easily set the corresponding point search line.

[0040] According to a twenty-seventh pattern model positioning method in image processing, filtering can be performed in the pattern model construction step to select the segment where the reference point is set. This makes it possible to set only the reference point with respect to the selected segment for setting the corresponding point line, thereby improving the accuracy of pattern search.

[0041] According to a twenty-eighth pattern model positioning method in image processing, criteria for selecting the segment during filtering can be set based on a result of calculating an average edge strength of the edge point included in each segment and comparing the obtained average edge strength with a preset average edge strength threshold for filtering. This makes it possible to select a segment with a large average edge strength, thus expecting a pattern search based on an edge with high accuracy, thus improving search accuracy.

[0042] According to a twenty-ninth pattern model positioning method in image processing, criteria for segment selection during filtering can be set based on a comparison result of the length of each segment with a preset edge strength threshold for filtering. This makes it possible to select only long segments for setting reference points, thereby improving the accuracy of pattern search.

[0043] According to a thirtieth pattern model positioning method in image processing, in the pattern model construction step, the pattern model can be generated based on a fixed geometric shape. This makes it possible to generate a pattern model with fixed geometric graphics, such as circular, oval, triangular, and rectangular shapes, regarded as references, individually or in combination, thus simplifying the generation of pattern search and any subsequent processing.

[0044] According to a thirty-first pattern model positioning method in image processing, the edge image generated from the registered image may be subjected to thinning processing as non-maximum point suppression processing before edge coupling processing.

[0045] According to a thirty-second pattern model positioning method in image processing, the edge angle obtained from the image to be searched, the edge strength threshold, a position, data expressing the edge angle in units of vectors, and / or an edge vector are used as data for finding the corresponding edge point.

[0046] According to a thirty-third pattern model positioning method in image processing, a threshold of each edge angle difference in the repeated least squares method in selecting the corresponding edge point is set based on an angular resolution obtained in the last coarse search.

[0047] According to a thirty-fourth model positioning method in image processing, the length of the corresponding point search line can be changed with respect to each reference point. Thus, the length of the corresponding point search line is not fixed but is set under the influence of the reference point, thus achieving an improvement in accuracy.

[0048] According to a thirty-fifth pattern model positioning method in image processing, the Bresenham algorithm can be used to generate straight line data when scanning the corresponding point search line. Therefore, when the Bresenham algorithm is used as an algorithm for continuously drawing a straight line without overlap, it is possible to easily and accurately set the corresponding point search line.

[0049] According to a thirty-sixth pattern model positioning method in image processing, the coordinate of the reference point can be represented by a subpixel coordinate.

[0050] According to a thirty-seventh pattern model positioning method in image processing, the step of refining the coordinate of the corresponding edge point may include the steps of: finding the corresponding edge point on the corresponding point search line; selecting a pair point corresponding to the corresponding edge point such that the corresponding edge point and the pair point are substantially orthogonal to and framing the corresponding point search line; and finding corresponding subpixel coordinates of the corresponding edge point and the pair point to find an average coordinate of the subpixel coordinates and taking the obtained average coordinate as a real corresponding edge point coordinate. This suppresses the calculation of the corresponding edge point in a waveform, and thus makes it possible to obtain a stable calculation result.

[0051] According to a thirty-eighth pattern model positioning method in image processing, the step of refining the coordinate of the corresponding edge point may include the steps of: searching a preliminary corresponding edge point on the corresponding point search line; and finding a plurality of neighborhood edge points existing around the preliminary corresponding edge point to find subpixel coordinates of the preliminary corresponding edge point and the plurality of neighborhood edge points.

[0052] According to a thirty-ninth pattern model positioning method in image processing, in the step of performing fine positioning, the least squares method is applied to the corresponding point search line, and a rank is calculated as an evaluation value by a ratio between the number of corresponding points and the number of reference points in the final least squares method processing.

[0053] According to a fortieth pattern model positioning method in image processing, the rank is found by the following expression: S=∑i=1n1∑i=1m1=nm S Rank n number of corresponding points m number of reference points

[0054] According to a forty-first pattern model positioning method in image processing, in the step of performing fine positioning, the least squares method is applied to the corresponding point search line, and a rank is calculated as an evaluation value by a ratio of a total of weights each consisting of a difference between the edge angle of the corresponding edge point in the final processing of the least squares method and the edge angle of the reference point corresponding thereto to the total number of reference points.

[0055] According to a forty-second pattern model positioning method in image processing, the rank is found by the following expression: S=∑i=1nω(|θi−θip|)∑i=1m1 S Rank N number of corresponding points M Number of reference points ωx function which is 1 when x = 0 and monotonically decreases with increasing x θi edge angle corresponding point θip: Angle from corresponding point to corresponding reference point

[0056] A forty-third image processing apparatus for positioning with a higher accuracy than at an initially given position when searching from an image to be searched and positioning an object resembling a pre-registered image using a pattern model corresponding to the registered image, the apparatus being capable of including: an image input device for acquiring a registered image and an image to be searched; an outline extraction device for extracting an outline from the registered image acquired by the image input device; a chain generation device for extracting a plurality of edge points from the outline extracted by the outline extraction device, and coupling adjacent edge points from the extracted plurality of edge points,to generate a continuous chain; a segment generation device for generating a segment by each approximating one or more chains by means of a line and / or a circular arc; a pattern model construction device for setting a plurality of reference points on the segment generated by the segment generation device and also constructing a pattern model of the registered image to which corresponding point search lines of a fixed length are assigned, which pass through the corresponding reference points and are substantially orthogonal to the outlines; and a fine positioning device for detecting an initial position corresponding to the registered image included in the image to be searched, which has been acquired by the image input device, for arranging the pattern model so as to be superimposed on the image to be searched,to find each individual corresponding edge point on the image to be searched that corresponds to each segment constituting the pattern model, regard a relationship between each segment and the corresponding edge point as an evaluation value, and operate at a higher accuracy than an initially given position so that an accumulated value of the evaluation values ​​becomes minimum or maximum. This makes it possible to use data on the corresponding point search line as the pattern model for fine positioning. In particular, by using not only the edge strength of the image along its length but also its image angle, it is possible to add a directional component to thereby perform highly accurate positioning that is resistant to a noise component. Furthermore, by changing the length of the corresponding point search line, it is possible to gain an advantage in being able toeasily change a corresponding edge point search area.,

[0057] A forty-fourth image processing apparatus may further include: an image reduction device for reducing the image to be searched, which is acquired by the image input device, with a predetermined reduction ratio; an edge angle image generation device for calculating an edge angle image containing edge angle information with respect to all pixels constituting the image on the reduction ratio image to be searched reduced by the image reduction device; an edge angle bit image generation device for transforming each pixel of the edge angle image generated by the edge angle image generation device into an edge angle bit image expressed by an edge angle bit indicating an angle with a predetermined fixed width; an edge angle bit image reduction device for performing to generate an edge angle bit reduction image reduced from the edge angle bit image,an OR operation on the edge angle bit of each pixel included in an OR operation area determined according to a reduction ratio for reducing the edge angle bit image to generate an edge angle bit reduction image consisting of reduced edge angle bit data representing each OR operation area; and a coarse search device for performing a pattern search on a first edge angle bit reduction image generated by the edge angle bit image reduction device using, as a template, a pattern model for the first coarse search generated with a first reduction ratio with respect to a first reduction ratio image to be searched, which has been reduced by the image reduction device with the first reduction ratio, to find, with first accuracy, a first position and posture corresponding to the pattern model for the first coarse search,from the total area of ​​the first edge angle bit reduction image, and also for performing a pattern search on a second edge angle bit reduction image generated by the edge angle bit image reduction device, using, as a template, a pattern model for the second coarse search generated with a second reduction ratio not greater than the first reduction ratio and not less than a zero magnification with respect to a second reduction ratio image to be searched, reduced by the image reduction device into the second reduction ratio, to find, with a second accuracy higher than the first accuracy, a second position and posture corresponding to the pattern model for the second coarse search from a predetermined range of the second edge angle bit reduction image where the first position and posture are set as references,wherein the fine positioning device arranges a pattern model so as to be superimposed on a third reduction ratio image to be searched, which is obtained by appropriately reducing the image to be searched to a third reduction ratio that is not smaller than unmagnified and not larger than the second reduction ratio, using the second position and posture of the third reduction ratio image to be searched to find a corresponding edge point on the third reduction ratio image to be searched corresponding to an outline constituting the pattern model, regards a relationship between each outline and its corresponding edge point as an evaluation value, and performs fine positioning with a third accuracy higher than the second accuracy so that an accumulated value of the evaluation values ​​becomes minimum or maximum. Thereby, it is possiblepreserve the edge angle information even in a case where the image data is further reduced, so as to perform a high-accuracy search with reduced data size without reducing the search accuracy.

[0058] A forty-fifth image processing apparatus may further include a segment filter function for selecting a segment where a reference point is set when constructing the pattern model. This makes it possible to set the reference point only with respect to the selected segment, thereby improving the accuracy of pattern search.

[0059] A forty-sixth image processing program for positioning with a higher accuracy than an initially given position when searching from an image to be searched and positioning an object resembling a pre-registered object using a pattern model corresponding to the registered image, the program being capable of causing a computer to implement: an image input function for acquiring a registered image and an image to be searched; an outline extraction function for extracting an outline from the registered image acquired by the image input function; a chain generation function for extracting a plurality of edge points from the outline extracted by the outline extraction function and linking adjacent edge points from the extracted plurality of edge points,to generate a continuous chain; a segment generation function for generating a segment by respectively approximating one or more chains by means of a line and / or a circular arc; a pattern model construction function for setting a plurality of reference points on the segment generated by the segment generation function, and also for constructing a pattern model of the registered image assigned corresponding point search lines of a fixed length, passing through the corresponding reference points and being substantially orthogonal to the outlines; and a fine positioning function for detecting an initial position corresponding to the registered image contained in the image to be searched acquired by the image input function, arranging the pattern model so as to be superimposed on the image to be searched,To find each individual corresponding edge point on the image to be searched that corresponds to each segment constituting the pattern model, consider a relationship between each segment and the corresponding edge point as an evaluation value, and perform it with a higher accuracy than at an initially given position so that an accumulated value of the evaluation values ​​becomes minimum or maximum. This makes it possible to use data on the corresponding point search line as the pattern model for fine positioning. In particular, by using not only the edge strength of the image along its length but also its edge angle, it is possible to add a directional component to perform highly accurate positioning that is resistant to a noise component. Furthermore, by changing the length of the corresponding point search line, it is possible to obtain an advantage in thatto be able to easily change a corresponding edge point search area.,

[0060] According to a forty-seventh image processing program, the program is capable of causing a computer to further realize: an image reduction function for reducing the image to be searched, which has been acquired by the image input function, with a predetermined reduction ratio; an edge angle image generation function for calculating an edge angle image including edge angle information with respect to each pixel constituting the image in the reduction ratio image to be searched, reduced by the image reduction function; an edge angle bit image generation function for transforming each pixel of the edge angle image generated by the edge angle image generation function into an edge angle bit image expressed by an edge angle bit indicating an angle with a predefined fixed width; an edge angle bit image reduction function for performing,to produce an edge angle bit reduction image reduced from the edge angle bit image, an OR operation on an edge angle bit of each pixel included in an OR operation range determined by a reduction ratio for reducing the edge angle bit image to generate an edge angle bit reduction image made of reduced edge angle bit data representing each OR operation range; and a coarse search function for performing a pattern search on a first edge angle bit reduction image generated by the edge angle bit image reduction function using, as a template, a pattern model for the first coarse search generated with a first reduction ratio with respect to a first reduction ratio image to be searched, which has been reduced by the image reduction function with the first reduction ratio,to find, with first accuracy, a first position and posture corresponding to the pattern model for the first coarse search from the entire area of ​​the first edge angle bit reduction image, and also to perform a pattern search on a second edge angle bit reduction image generated by the edge angle bit image reduction function, using, as a template, a pattern model for a second coarse search generated with a second reduction ratio that is not greater than the first reduction ratio and not less than unmagnified with respect to a second reduction ratio image to be searched, reduced by the image reduction function to a second reduction ratio, to find, with second accuracy higher than the first accuracy, a second position and posture corresponding to the pattern model for the second coarse search from a predetermined area of ​​the second edge angle bit reduction image,where the first position and posture are set as references; wherein the fine positioning function arranges a pattern model to be superimposed on a third reduction ratio image to be searched, which is obtained by reducing, as appropriate, the image to be searched to a third reduction ratio that is not smaller than unmagnified and not larger than the second reduction ratio, using the second position and posture of the third reduction ratio image to be searched to find a corresponding edge point on the third reduction ratio image to be searched corresponding to an outline constituting the pattern model, wherein a relation between each outline and its corresponding edge point is regarded as an evaluation value, and performing fine positioning with third precision higher than the second precision,so that an accumulated value of the evaluation values ​​becomes minimal or maximal. This makes it possible to retain the edge angle information even when the image data is further reduced, thus performing high-precision search with a reduced data size without reducing the search accuracy.

[0061] Furthermore, a forty-eighth computer-readable recording medium stores the above program. The recording medium includes magnetic disks, optical disks, magneto-optical disks, semiconductor memories, and some other media capable of storing a program, such as a CD-ROM, a CD-R, a CD-RW, a flexible disk, a magnetic tape, an MO, a DVD-ROM, a DVD-RAM, a DVD-R, a DVD+R, a DVD-RW, a DVD+RW, Blu-ray, an HD, and a DVD (AOD). Furthermore, the program includes one distributed in a form by downloading via a network line such as the Internet, other than one supplied stored on the above recording medium.Furthermore, the recording medium includes equipment capable of recording a program, such as general-purpose equipment or dedicated equipment mounted in a state where the above program is executable in the form of software or firmware. Furthermore, each processing and function included in the program may be executed by program software executable on a computer, or the processing in each section may be realized by hardware such as a predetermined gate array (gate array, FPGA, ASIC), or in a mixed form of program software and a partial hardware module that realizes part of elements of hardware. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing an example of an image processing apparatus; Fig. 2A to 2H are schematic views each showing a scheme of an operation when registering a pattern model and during movement to perform a search using that pattern model. Fig. 3 is a flowchart showing a scheme of operation when registering the sample model; Fig. Fig. 4 is a flowchart showing a scheme of operation during movement to actually perform a search; Fig. 5 is an image view showing a user interface screen for setting a reduction ratio in a manual reduction ratio decision mode; Fig. 6A to 6C are schematic views each showing a scheme of an edge angle bit; Fig. Fig. 7 is a schematic view showing the state of generating and compressing an edge angle bit image; Fig. 8A to 8C are schematic views each showing an example of a sample model, wherein Fig. 8A shows a sample model for local search, Fig. 8B shows a sample model for large area search and Fig. 8C shows a sample model for fine positioning; Fig. 9 is a flowchart showing a scheme of a procedure for a search while moving; Fig. Fig. 10 is a schematic view showing a concept of search during movement; Fig. 11A to 11D are schematic views each showing a state in which the edge angle bit changes as a result of rotation of the pattern model; Fig. 12 is a flowchart showing a procedure for generating a pattern model upon registration; Fig. 13A to 13C are image views each showing a user interface screen for automatically adjusting search accuracy and search time according to their priorities; Fig. 14 is a flowchart showing a procedure for registering the pattern model for fine positioning; Fig. 15 is a flowchart showing a procedure for performing preprocessing on an image to be searched during movement; Fig. 16 is a pictorial view showing a sample model with corresponding point search lines set therein; Fig. 17 is a schematic view showing a state where the sample model of Fig. 16 has been thinned out; Fig. Fig. 18 is a flowchart showing a procedure for performing pattern search during movement; Fig. 19A is an image view showing an example of the image to be searched and Fig. 19B is an image view showing a reduced image to be searched, which is enlarged to the same magnification as that of the registered image of Fig. 6A has been reduced; Fig. Fig. 20 is a flowchart showing a procedure for pattern search during movement; Fig. 21A and Fig. 21B are schematic views each showing respective edge point search processing for finding a corresponding edge point; Fig. Fig. 22 is a schematic view for describing a wave phenomenon of an edge position; Fig. 23 is a schematic view showing an example of a certain registered image in which a corresponding point search line may be difficult to select; Fig. Fig. 24 is a schematic view showing a state where corresponding point search lines are automatically drawn on the pattern of Fig. 23 are set; Fig. 25 is a schematic view showing a result of filtering the corresponding point search lines; Fig. Fig. 26 is a flowchart showing a procedure for filtering processing corresponding point search lines; Fig. Fig. 27 is a pictorial view showing the state of setting corresponding point search lines having the same length from reference points; Fig. Fig. 28 is a pictorial view showing the state of setting corresponding point search lines having different lengths from the reference points; Fig. Fig. 29 is a schematic view for describing a procedure for finding a coordinate of a corresponding edge point; Fig. 30 is a schematic view showing an edge angle image consisting of four pixels “a” to “d”; Fig. 31 is a schematic view showing edge angle sections defining edge angle bits; Fig. 32 is a schematic view showing a pattern obtained by transforming the edge angle image of Fig. 30 shows the obtained edge angle bitmap; Fig. 33 is a schematic view showing a bitmap obtained by reducing the edge angle bit map of Fig. 32 shows the obtained edge angle bit reduction image; Fig. 34 is a schematic view for describing the state of reducing the original edge angle bit image in units of 2x2 pixels; Fig. 35 is a schematic view for describing the state of enlargement of the image after reduction processing of Fig. 34; Fig. 36 is a schematic view for describing the state of reducing the original edge angle bit image to one half; Fig. 37 is a schematic view for describing the state of reducing the original edge angle bit image to one-third; Fig. 38 is a conceptual view showing a sample model before parallel processing; Fig. 39 is a conceptual view showing the pattern model after parallel processing; Fig. 40A and Fig. 40B are concept views showing both data for registering the pattern model; Fig. 41 is a pictorial view showing an example of a pattern model as a partially notched circle expressed by a circular arc segment and a line segment; Fig. 42 is an image view showing a state where a coarse search using the pattern model of Fig. 41 has been performed on an input image to perform a certain level of positioning; Fig. 43 is a picture view showing a state in which fine positioning is performed from the state of Fig. 42 by applying a least squares method with a distance between a point and the circular arc being regarded as an error function; Fig. 44 is a picture view showing an example of performing the generation processing of a corresponding point search line on the pattern model of Fig. 41 shows; Fig. 45 is an image view showing a state where the coarse search is performed on an image to be searched using the pattern model of Fig. 44 has been carried out to overlay the sample model at a position and attitude determined by the search; Fig. 46 is a schematic view showing an edge vector (Ex, Ey) with an edge strength EM and an edge angle θE; Fig. 47 is a conceptual view showing a state in which the coarse search has been performed on the circular work to complete a certain level of positioning; Fig. 48 is a conceptual view showing a state where the fine positioning is Fig. 47 in an attempt to overlay the pattern model on the image to be searched; Fig. 49A to 49D are schematic views each for describing weight processing in the case of a plurality of corresponding edge point candidates existing; Fig. 50 is a flowchart showing a procedure for selecting a segment assumed in a positioning direction; Fig. 51A and Fig. 51B are schematic views, both for describing the state of updating a setting of an angle range in Fig. 50 serve; Fig. 52A and Fig. 52B are schematic views both showing an example of reduction processing using saturated addition, wherein Fig. 52A is a schematic view showing an edge angle image in which each pixel has an edge angle; and Fig. 52B is a schematic view showing edge angle sections expressing edge angle bit data of the respective pixels with eight bits; Fig. 53 is an image view showing an example of a binary image; Fig. 54 is a graphic that has a pixel value of Fig. 53 shows; Fig. 55 is a graph showing a change in the strength of an edge of a high-sharpness image; Fig. 56 is a graph showing a pixel value of an unclear image; Fig. 57 is a graph showing a change in the strength of an edge of a low-sharpness image; Fig. 58 is a schematic view showing a method of calculating a subpixel coordinate according to Japanese Unexamined Patent Publication No. H07-128017; Fig. 59 is a graph showing a relationship between the edge strength error and the subpixel position; Fig. 60 is a flowchart showing a procedure for setting an image data reduction ratio based on sharpness of edge points; Fig. 61 is a graph showing an edge model function; Fig. 62 is a flowchart showing an operation in registering to use the image data reduction ratio; Fig. 63 is a flowchart showing an operation during movement using the image data reduction ratio; Fig. 64 is a schematic view showing edge strengths of three adjacent points B, C and F; Fig. 65 is a schematic view showing a procedure for finding a coordinate of a corresponding edge point using the neighborhood edge points; Fig. 66 is a flowchart showing a procedure for finding a coordinate of the corresponding edge points of Fig. 65 shows; Fig. 67 is an image view showing the state of setting a pattern model in a registered image in which characters are displayed in frames; Fig. 68 is an image view showing a user interface screen for setting a pattern characteristic selection function for performing sorting in the order of length in an image processing program; Fig. 69 is an image view showing a segment selected as a sample model in a registered image, in which various letters and numbers are displayed in grid frames; Fig. 70 is an image view showing a state where the outline registration order is set to “descending order of length” in the user interface screen of Fig. 68 has been set; Fig. 71 is a picture view showing a state where the outline registration order is set to “ascending order of length” in the user interface screen of Fig. 70 has been set; Fig. 72 is an image view showing a state where a segment in the registered image of Fig. 69 to the setting condition of Fig. 71 has been selected; Fig. 73 is a flowchart showing a procedure for performing sorting in the order of segment length; Fig. 74 is a flowchart showing a procedure of for sorting in the order of chain length; Fig. 75 is an image view showing a user interface screen for setting a pattern characteristic selection function to filter a long outline in the image processing program; Fig. 76 is an image view showing a state where the upper limit of the outline length in the user interface screen of Fig. 75 has been set high; Fig. 77 is an image view showing a state where a segment in the registered image of Fig. 69 to the setting condition of Fig. 76 has been selected; Fig. 78 is an image view showing a state where the upper limit of the outline length in the user interface screen of Fig. 75 has been set low; Fig. 79 is an image view showing a state where a segment in the registered image of Fig. 69 to the setting condition of Fig. 78 has been selected; Fig. 80 is a flowchart showing a procedure for filtering a long segment; Fig. 81 is a flowchart showing a procedure for filtering a long chain; Fig. Fig. 82 is a flowchart showing a procedure for selecting a segment after sorting with the segment length; Fig. Fig. 83 is a flowchart showing a procedure for selecting a segment by filtering; Fig. 84 is a flowchart showing a selection procedure for a segment considering a direction of a normal thereto; Fig. Fig. 85 is a schematic view showing an example of performing fine positioning on a graphic having high symmetry; Fig. 86A and Fig. 86B are schematic views each showing the state of approximating an error function by a reverse Hess method; Fig. 87 is a schematic view showing a concept of a pyramid search; Fig. Fig. 88 is a conceptual view showing a relationship between a pattern desired to be detected and a search line upon registration; Fig. 89 is a conceptual view showing a state immediately before fine positioning after a coarse search during movement; Fig. 90 is an enlarged view showing the lower angle range of Fig. 89 shows; and Fig. 91 is a view showing a relationship between a pattern model configured of two segments and input pixels of an image to be searched. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0062] Hereinafter, an embodiment of the present invention will be described based on drawings. However, the embodiments shown below exemplify a pattern model positioning method in image processing, an image processing apparatus, an image processing program, and a computer-readable recording medium to concretely demonstrate a technical concept of the present invention, and the present invention does not specify its pattern model positioning method in image processing, its image processing apparatus, its image processing program, and its computer-readable recording medium below. Furthermore, the present specification by no means specifies an element shown in the claims as an element in the embodiment.Specifically, the size, material, shape, relative arrangement, and the like of a constituent component described in the embodiment are not intended to limit the scope of the present invention to those, but are illustrative examples unless a specific description is particularly given. Note that there are cases where a size, positional relationship, or the like of an element shown by each drawing may be emphasized for the sake of clarity of description. Furthermore, in the following description, the same name or symbol indicates the same element of a homogeneous element, and a detailed description will not be given repeatedly as necessary.Furthermore, with respect to the element constituting the present invention, a plurality of elements may be made of the same object and thus achieve the aspect of dividing one object, or on the other hand, a function of one object may be realized by being divided by a plurality of objects.

[0063] The connections of an image processing apparatus for use in an example of the present invention to a computer, a printer, an external storage device, and other peripheral equipment connected to the image processing apparatus and used for operation, control, display, and other processing are made electrically, magnetically, or optically for communication through serial connection such as IEEE1394, RS-232x, RS-422, or USB, parallel connection, or connection via a network such as 10BASE-T, 100BASE-TX, or 1000BASE-T. The connection is not limited to physical connection using a cable, but may be wireless connection using electromagnetic waves such as a radio LAN such as IEEE802.1x or Bluetooth (registered trademark), infrared rays, optical communication, or the like, or other connection.Furthermore, as a recording medium for data exchange, setting storage, and the like, a memory card, a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like can be used. Note that in the present specification, the image processing device is used not only in the sense of a device body for performing edge extraction, pattern matching, and the like, but also in the sense that includes an outline extraction system formed by combining this device body with peripheral equipment such as a computer and an external storage device.

[0064] Furthermore, in the present specification, the pattern model positioning method in image processing, the image processing apparatus, the image processing program, and the computer-readable recording medium are not limited to a system itself that performs edge extraction, measurement area setting, and edge point coupling, and an apparatus and method that operate and perform, in a hardware manner, input / output processing, display, calculation, communication, and other processing related to capturing and acquiring a plurality of images. An apparatus and method for realizing the processing in a software manner are also included within the scope of the present invention.

[0065] For example, an apparatus and a system where software, a program, a plug-in, an object, a library, an applet, a compiler, a module, a macro operating at a specific point, or the like is integrated into a general-purpose circuit or a computer to allow performance of edge extraction and edge point coupling itself or processing related thereto also correspond to the pattern model positioning method in image processing, the image processing apparatus, the image processing program, and the computer-readable recording medium according to the present invention.Further, in the present specification, the computer includes, in addition to a general-purpose or special-purpose electronic computer, a workstation, a terminal, a mobile-type electronic device, PDC, CDMA, W-CDMA, FOMA (trademark), GSM, IMT2000, a cellular phone such as a fourth-generation mobile phone, a PHS, a PDA, a pager, a smart phone, and other electronic devices.Furthermore, in the present specification, the program is not limited to a standalone one, but can also be used in the mode of functioning as part of a specific computer program, software, service or the like, in the mode of functioning as invoked as needed, in the mode of being provided as a service in an environment of an operating system (OS) or the like, in the mode of being operated as being present in the environment, in the mode of working in a background, or in a position as another support program. (Short flow of image processing)

[0066] Fig. 1 shows a block diagram of an image processing device 100. As shown in the Fig. 2A to 2H, this image processing apparatus 100 previously registers an image to be searched and generates a pattern model from this registered model, and in actual operation, the apparatus finds a position corresponding to the pattern model from an input image to be searched. An operation scheme in generating the pattern model is shown in a flowchart of Fig. 3. In the present embodiment, as shown in Fig. 2A, a user sets an area where a pattern model is generated with respect to a registered image RI to be searched, namely, a search window PW is set by a user (step S301 in Fig. 3). An image containing an area where this pattern window PW is set is reduced as appropriate, as shown in Fig. 2B (step S302 in the same drawing). Further, as in Fig. As shown in Figure 2C, a pattern model PM is generated from the reduced pattern window RPM as a search pattern corresponding to the registered image RI (step S303 in the same drawing). As described above, before the actual search operation, the image processing device first generates the pattern model PM corresponding to the registered image RI to be searched from the image to be searched.

[0067] Continue as in Fig. As shown in Figure 2D, the pattern model PM is reduced into a reduction ratio for a first edge angle bit image and a reduction ratio for a second edge angle bit image (described later) and used during movement (step S304 in the same drawing). Such generation of the reduced pattern model RPM may be performed beforehand upon registration, or may also be performed during each operation.

[0068] In the meantime, an operating scheme is presented in a flow chart of Fig. 4. During the movement, after an image OI to be searched has been input in step S401 ( Fig. 2E), this image OI to be searched is reduced into a reduced image ROI to be searched, appropriately in step S402 ( Fig. 2F). Next, in step S403, an edge angle bit image EB is generated from the reduced image ROI to be searched ( Fig. 2G: executed later). Furthermore, in step S404, an edge angle bit reduction image REB reduced from the edge angle bit image EB is generated ( Fig. 2H). In step S405, a pattern search is performed on the edge angle bit reduction image REB obtained as described, using the pattern model obtained upon registration. (Image processing device 100)

[0069] Next, a configuration of the image processing apparatus 100 will be described. The image processing apparatus 100 shown in the block diagram of Fig. 1 includes an image input device 1 for inputting an image, a display device 3 for displaying an image and various data, an operation device 2 for allowing the user to perform various operations, and an output device 5 constituting an output interface for outputting a result of image processing in an image processing device body 100A to the outside. The image input device 1 is composed of an image pickup device such as a CCD. An input image input from the image input device 1 is captured in the image processing device body 100A by an A / D conversion device. Further, the display device 3 displays an original image of the input image or an edge image obtained by performing image processing on the original image by, for example, edge image processing. (Image processing device body 100A)

[0070] The image processing device body 100A includes a storage device 4 for storing a variety of data and a calculation device 6 for performing a variety of calculations related to image processing. In addition to a registered image input from the image input device 1 through the A / D conversion device, the storage device 4 includes a pattern model generated with this registered image, which is regarded as a template image, an image to be searched, a work area for use in holding and storing temporary data generated when generating the pattern model from the registered image during searching, and the like. (Computer device 6)

[0071] The computer device 6 includes an outline extraction device 62 for performing edge detection on an image to extract an outline, a chain generation device 63 for generating a chain from the outline information, a segment generation device 68 for generating a segment from the chains, and various other devices. Examples of such devices include an image angle bit image generation device 69, an edge angle bit image reduction device 78, an image reduction device 77 for reducing an image to a predetermined magnification, a pattern model constitution device 70 for constituting a pattern model, a coarse search device 71 that performs coarse search, and a fine positioning device 76 for performing highly accurate positioning within an image to be searched using a pattern model of a registered image based on edge information (explained later).This computer device 6 registers a pattern model and performs a search in an image to be searched using the registered pattern model as described above. To perform such operations, the chain generation device 63, the segment generation device 68, and the pattern model constitution device 70 are used when registering the pattern model. Furthermore, the coarse search device 71 and the fine positioning device 76 are used in the actual operation. Moreover, the image reduction device 77, the outline extraction device 62, the chain generation device 63, and the like are used for both the operation during registration and the operation during movement, and the edge angle bit image generation device 69 and the edge angle bit image reduction device 78 are used during movement.It should be noted that in the present embodiment, although the chain generating device 63 is used only at registration, it can also perform the chaining on the image to be searched during movement. (Outline extraction device 62)

[0072] The outline extraction device 62 includes an edge angle / edge strength image generation device 60 for generating an edge angle image and an edge strength image as edge images, and a thinning device 61. Specifically, with respect to a multi-value image inputted through the image input device 1 (an original image of an original size or a reduced image reduced by an image reduction device 77 mentioned later), the edge angle / edge strength image generation device 60 generates edge strength images in each of the X and Y directions (an edge strength image component in the X direction and an edge strength image component in the Y direction) using a filter known for extracting an edge point, for example, by using a Sobel filter for both the X and Y directions, and also generates a two-dimensional edge angle image from these edge angle images in the X and Y directions.

[0073] Further, with respect to this edge image generated by the edge angle / edge strength image generating device 60, the thinning device 61 thins out the edge point, for example, by edge strength non-maximum point suppression processing.

[0074] Note that in this specification, edge strength is a numerical value that expresses a degree of whether a pixel is part of an edge (darkness to brightness). Edge strength is typically calculated from pixel values ​​of a target pixel and nine pixels located around it. Furthermore, edge angle indicates a direction of an edge in a pixel and is typically calculated from edge strengths in the X and Y directions, which are calculated by using the above Sobel filter or the like, respectively. (Chain generating device 63)

[0075] Meanwhile, the chain generation device 63 and the segment generation device 68 are used in generating a pattern in which a part of the registered image is regarded as a template image. The chain generation device 63 in the present example includes an edge chaining device and a chain filtering device 64. Specifically, the edge chaining device 64 generates a chain by linking a plurality of adjacent edge points among the plurality of edge points included in an edge image generated by the outline extraction device 62. Furthermore, the edge filtering device 66 performs filtering on a plurality of chain groups generated by the edge chaining device 64 using a plurality of characteristic amounts of chains. (Segment generating device 68)

[0076] Furthermore, the segment generation device 68 includes an edge chain segmentation device 65 and a segment selection device 67. The edge chain segmentation device 65 approximates each chain generated by the edge chaining device 64 and filtered by the chain filtering device 66 to generate a segment. The segment, in this case, is a line and / or a circular arc fitted using a least squares method. Furthermore, the segment selection device 67 performs filtering on the segment.

[0077] Furthermore, the segment selection device 67 may also include a pattern characteristic selection function. Namely, changing a selection criterion for a segment constituting a pattern model according to the characteristics of a pattern obtained from an object to be searched can realize more stable positioning. This will be explained later. (Model constituting device 70)

[0078] The pattern model constructing device 70 is a device for creating a pattern model to be stored in the above storage device. Specifically, each segment created by performing processing on a registered image by the above outline extraction device 62, chain generation device 63, and segment generation device 68, which perform processing on the registered image, is processed by the pattern model constructing device 70. The pattern model will be executed later. (Image reduction device 77)

[0079] Meanwhile, the image reduction device 77 is a device for reducing a registered image and an image to be searched. Its reduction ratio is automatically set by an automatic reduction ratio decision mode. Specifically, as described above, in the present embodiment, a region that a user desires to be a pattern model is set on the registered image. The reduction ratio in this case is automatically set based on the region set by the user. Namely, the reduction ratio is automatically set based on a size of the pattern window PW for specifying a pattern model. For example, in the case where the pattern window PW has a rectangular shape, the reduction ratio is determined based on a length of its shorter side.

[0080] Note that in this specification, "increasing a reduction ratio," "a reduction ratio is large," or "a reduction ratio is high" means increasing a reduction degree or increasing a compression ratio, and indicates, for example, reducing a reduced image with a reduction ratio of one-eighth to a reduced image with a reduction ratio of one-sixteenth. In contrast, "decreasing a reduction ratio," "a reduction ratio is small," or "a reduction ratio is low" means decreasing a reduction degree and, for example, bringing a reduced image to the unenlarged image side, and indicates, for example, changing a reduced image with a reduction device from one-sixteenth to a reduced image with a reduction ratio of one-eighth.Furthermore, a “reduction ratio not smaller than unmagnified image” does not mean enlarging the unmagnified image, but means an image with a reduction ratio larger than the unmagnified image, namely a reduction ratio of 1, or the unmagnified image of a reduction ratio of 1.

[0081] In addition to such an automatic reduction ratio decision mode, a manual reduction ratio decision mode in which the user selects a desired reduction ratio and sets it as a reduction ratio can also be used. In an example of a Fig. In the user interface shown in Figure 5, the user is asked to select a reduction ratio as a reduction ratio for a large search area from a plurality of reduction ratios, here alternatives of one-half, one-quarter, and one-eighth. The reduction ratio can be specified with any numerical value. Furthermore, an image showing how the original image is transformed by reducing the image with a reduction ratio selected in this case can be displayed in conjunction. In this way, a reduction ratio for local search (medium reduction ratio) and a fine positioning reduction ratio can be set differently than the reduction ratio for a large-area search. (Reduction ratio for a large area).

[0082] Furthermore, a reduction ratio for local search can also be automatically added. Specifically, if a ratio between the reduction ratio for wide area search and the reduction ratio for local search is greater than a preset value, a local search (additional local search) is set with an additional reduction ratio (reduction ratio for local search) between this reduction ratio for wide area search and the reduction ratio for local search automatically added (see Fig. 10). Furthermore, the additional local search can be repeated multiple times. In this way, the ratio between the reduction ratio for wide-area search and the reduction ratio for local search can be kept within the specified value to achieve a reduction in search speed.

[0083] The image reduction device 77 is used for a registered image that creates a pattern model and an image to be searched that is reduced at the same reduction ratio as a reduction ratio set in the registered image. Namely, the image to be searched is also reduced at the reduction ratio set in the registered image. (Edge angle bit image generating device 69)

[0084] The edge angle bit image generating device 69 generates an edge angle bit image using an edge angle image generated by the above edge angle / edge strength image generating device 60 of the computing device 6 with respect to a searched image and a reduced image obtained by reducing the searched image by the above image reducing device 77. Namely, the edge angle bit image generating device 69 is used to generate an edge angle bit image of a searched image during movement. In other words, it is not used in the registration of a pattern model.

[0085] More precisely, as in Fig. 6A to 6C, values ​​from 0 to 360 degrees are sectionalized as edge angles in units of 45 degrees in eight-bit data ( Fig. 6B). It is determined whether edge angles obtained corresponding to the pixels of the edge angle image are in bit positions assigned to the corresponding eight sectionalized areas or not ( Fig. 6A) and a flag of 1 is set at the detected bit position to transform the edge angle image into an edge angle bit image ( Fig. 6C). In the present embodiment, an edge angle bit image is generated by the edge angle bit image generating device 69 using an edge angle image generated by the edge angle / edge strength image generating device 60 of the calculating device 6 (later based on the Fig. 30, etc.). However, this method is not restrictive, and, for example, an edge angle bit image may be generated by the edge angle bit image generating device 69 using an edge angle image thinned by the thinning device 61 of the calculating device 6. (Edge angle bit image reduction device 78)

[0086] The edge angle bit image reducing device 78 is a means for reducing an edge angle bit image generated by the above edge angle bit image generating device 69. An edge angle bit reduction image is thereby obtained.

[0087] A reduction ratio of an edge angle bit reduction image in the present embodiment is set according to the size of the pattern window PW, by using which the user sets a region desired to be a pattern model to a registered image. Therefore, in the case of automatically setting a reduction ratio in the edge angle bit image reduction device 78, the size of the pattern window PW set to the registered image is reflected. However, this method is not restrictive, and needless to say, the reduction ratio in the edge angle bit image reduction device 78 can be set directly by checking a reduction ratio of the pattern model.

[0088] This situation is described by comparing it with a conventional method. When reducing an edge angle bit image by the conventional method, for example, when the reduction ratio is one-eighth of the original image size of the image to be searched, a region of 8×8 pixels, namely, an edge angle of a pixel representing 64 pixels or an edge angle of a pixel obtained by reducing an average value of 64 pixels, is considered as a central value of the region.In contrast, in the reduction in the edge angle bit image by the edge angle bit image reducing device 78 according to the present embodiment, for example, in the case where the reduction ratio is one-eighth of the original of the image to be searched, a state is held where a flag of 1 is set in a bit position of each pixel of 64 pixels in the 8×8 area, namely, in a bit position corresponding to each angle possessed by 64 pixels, as shown in FIG. Fig. 7 in the form of storing the bit positions as they are through an OR operation. This makes it possible to store information on the edge angle bit image in such a way that it is prevented from being corrupted even after image reduction, thus maintaining accuracy in a search by using a pattern model.

[0089] An OR operation area where the OR operation is performed is determined based on a reduction ratio for reducing the edge angle bit image. For example, in the case of generating an edge angle bit reduction image by reducing the edge angle bit image by one-eighth, an 8×8 area is the area of ​​the OR operation. Namely, the edge angle bit image is sectionalized into 8×8 OR operation areas, and in each OR operation area, a result of performing the OR operation on edge angle bits of all pixels included in the OR operation area is reduced edge angle bit data representing each OR operation area. The aggregation of reduced edge angle bit data in each OR operation area obtained in the above manner is an edge angle bit reduction image.

[0090] Furthermore, the OR operation means a summation operation, specifically a summation operation of bits. Moreover, when performing the OR operation of pixels, in addition to simply adding bits, a lower limit can be set on the number of pixels with edge angle bits. For example, in a case where the number of pixels each having an edge angle bit to be added does not meet a threshold of a predetermined number of pixels, that edge angle bit is ignored. Namely, since edge angle bits of bit pixels to be added may be extremely smaller than others, such as one to several pixels, noise, or an error, such pixels with low plausibility can be ignored. It is therefore possible to generate a highly reliable edge angle bit reduction image based only on angle bits assumed to be highly reliable. (Other OR operation: saturated addition)

[0091] Furthermore, an operation other than the above-mentioned sum operation can also be used as the OR operation. Saturated addition is described as an example. For example, when performing reduction processing or enlargement processing by the OR operation, the OR operation is performed on pixel data on an n×n edge angle bit image. The extension of this operation is to "perform saturated addition on a bit corresponding to each angle and determine a nearest central value from a saturated addition result." Saturated addition means addition processing in which an upper limit is previously set on an addition result. In the case where a regular addition result exceeds the upper limit, truncation is performed at the upper limit. For example, if the upper limit is 100, the upper limit is limited to 100 as follows: 10+89=99 11+89=100 12+89=100 10+100=100

[0092] Next, a specific example of reduction processing using saturated addition based on the Fig. 52A and Fig. 52B. The Fig. 52A is a schematic view showing an edge angle image in which each pixel has an edge angle and Fig. Fig. 52B is a schematic view showing an edge angle section for expressing the corresponding pixels constituting the edge angle image with 8-bit edge angle bit data. Note that edge angle sections obtained by sectionalizing an edge angle to express edge angle directions by edge angle bits are later converted by Fig. 31. In the example of Fig. 52B, the edge angle is sectionalized by offsetting it by 22.5 degrees from the horizontal or vertical direction. The edge angle sections are labeled E, SE, S, SW, W, NW, N, and NE, respectively, by a width of 45 degrees clockwise from exactly the right, and are labeled with angle bits of 0, 1, 2, 3, 4, 5, 6, and 7, respectively. A pixel representing nine pixels "a" to "i" that represent the edge angle image of Fig. 52A, obtained edge angle bit image, with edge angle bits according to the edge angle sections of Fig. 52B has edge angle bit data as follows: 76543210 a: 00001000 b: 10000000 c: 10000000 d: 00001000 e: 10000000 f: 10000000 g: 00001000 h: 00000100 i: 10000101

[0093] Then, in an edge angle bit image in a new reduced image, bits from "a" to "i" corresponding to respective angles are all subjected to saturated addition. The upper limit of saturated addition is set to three. For example, the reduced image angle bit data "e'" of position "e" can be calculated by: e′=a+b+c+d+e+f+g+h+i

[0094] A result of the calculation is as follows: 7766554433221100 e' 1100000011100001; Binary display e' 3 0 0 0 3 2 0 1; decimal display

[0095] The above calculation result exhibits characteristics of saturated addition processing. Namely, values ​​not less than three are all clipped at three. Such addition is called saturated addition, in which, if a sum result is not less than a clipping threshold as described above, clipping is performed at that value. This reduced edge angle bit data "e'" can be used as it is in search processing. For example, reduced edge angle bit data "e'" in which only values ​​not less than two are clipped to one and the other values ​​are clipped to zero can be expressed as follows to be used for search: 76543210 e'': 10001100; binary display (Sample model X for coarse search device)

[0096] The pattern model constituting device 70 generates two kinds of pattern models, which are a pattern model X for coarse search to be used in the later-mentioned coarse search device 71 and a pattern model Y for fine positioning to be used in fine positioning ( Fig. 8A to 8C). The pattern model X for coarse search is formed such that a reference point is set on each segment based on a preset condition, and an angle is set in a direction of a normal to the segment at each reference point and the orientation of an edge is set. (Sample model Y for fine positioning)

[0097] Meanwhile, with respect to the fine positioning pattern Y, each reference point is generated by setting, in addition to the information of the above coarse search pattern X, a parameter classified by a segment expressing each corresponding segment (e.g., a parameter capable of defining a segment composed of a line or a circular arc), and a line (hereinafter referred to as "respective point search line") that is a kind of normal to the segment extending in the edge direction and further having a predetermined length. The corresponding point search line extends forward and backward in the direction of the normal to the reference point set at the center. Note that, preferably, one or more reference points are set on each segment.In other words, it is not necessary to include a segment without reference points set to it in the pattern model. (Coarse search device 71)

[0098] A search during movement is performed by the coarse search device 71 and the fine positioning device 76 using a dedicated pattern model generated during registration. The coarse search device 71 is a device for performing a coarse search during movement. The coarse search can be performed multiple times with different reduction ratios, as well as only once. In the case of performing the coarse search multiple times, it is preferably performed based on more detailed data approximating the original size in such a way that a reduction ratio of a second coarse search is suppressed to be lower than a reduction ratio of a first coarse search, namely, a resolution is increased, or in some other way. Furthermore, the second coarse search is preferably performed while narrowing a scanning range based on a result of the first coarse search.A scheme of searches while moving is shown in a flowchart by . Fig. 9 and a schematic view of Fig. 10. In the present embodiment, as in step S901 of Fig. As shown in Figure 9, a first coarse search (large area search), which is performed using a second pattern model obtained by reduction at a large area reduction ratio, is performed on the entire area of ​​an image to be searched. Thereafter, a second coarse search (local search) is performed on a region of a "detection candidate" obtained by the first coarse search and an area at the edge thereof in an image to be searched with an average reduction ratio lower than that in the first coarse search. Further, in step S903, fine positioning is performed by the fine positioning device 76 using the pattern model Y.

[0099] It should be noted that the order of generating a pattern model via registration and the order of performing a search during motion are not necessarily consistent. For example, during registration, pattern models are generated sequentially from a pattern model with a lower reduction ratio (close to the original size), (first pattern model → second pattern model, etc.). This can greatly reduce conditions where minute information is lost during image reduction. In contrast, during motion, a search of pattern models is performed in reverse order, starting from a pattern model with a higher reduction ratio (lower resolution, higher compression ratio, etc.), (first coarse search → second coarse search). This allows for efficient searches using a coarse-to-fine approach.This results in the first coarse search being performed using the second pattern model during the movement and subsequently the second coarse search being performed using the first pattern model. (First rough search)

[0100] Next, a scheme of the coarse search performed by the coarse search device 71 will be described. First, in the first coarse search, a raw search is performed on the entire area of ​​the image to be searched using the registered pattern model to extract a raw position, namely, a detection candidate. On the edge angle bit reduction image of the image to be searched generated by the edge angle bit image reduction device 78, scanning is performed using a pattern model generated from the registered image with the same reduction device. Specifically, for example, to scan the entire area of ​​the edge angle bit image obtained by reducing the original size to one-eighth, a pattern model set in a position with a specific rotation angle is scanned from the upper left of the edge angle bit reduction image to a lower right direction.This specifies a detection candidate region similar to the pattern model within the entire range of the edge angle bit reduction image. The same scan is performed in the same way using all of a plurality of different rotation positions individually set on the pattern model. Namely, the scan is repeated multiple times at a changed rotation angle. Consequently, the entire detection candidate region similar to the pattern model is extracted as a matching candidate with respect to the pattern model. Fig. 11A to 11D all show a state where edge angle bits change as a result of rotating the pattern model. In this example, a change in the edge angle bit from (b) to (d) in a case (c) of rotating a pattern model (a) for a large-area search clockwise by 60 degrees. Furthermore, an evaluation value (rank value) representing similarity is calculated for each detection candidate, and candidates with a rank value higher than a certain threshold are extracted. In addition, each detection candidate has information about its position and pose, namely, an XY coordinate of the pattern model, an angle θ, and a rank. Note that the rank will be explained later. (Second rough search)

[0101] Next, the coarse search device 71 performs the second coarse search based on a result of the first coarse search. In the second coarse search, a smaller reduction ratio than the reduction ratio of the image to be searched first used in the above first coarse search, that is, a reduction ratio with a larger amount of information, is used. The detection candidates are narrowed down, for example, by using an edge angle bit reduction image with a reduction ratio of one-quarter and by using a pattern model with a reduction ratio of one-eighth only in the entire peripheral area of ​​the detection candidate similar to the pattern model extracted in the edge angle bit reduction image with the same reduction ratio used in the first coarse search.Since this second coarse search is a search performed using a portion of the edge angle bit reduction image, it can efficiently narrow down the detection candidates. As described above, efficient search can be achieved by performing the coarse search in multiple stages. Namely, after scanning the entire area to specify a raw position in the first coarse search, the second coarse search is performed in the specified area (detection candidate or "one likely to be the object") or in the vicinity of it in the image with a lower reduction ratio. Furthermore, the local search can be performed multiple times based on reduction ratios. (Fine positioning device 76)

[0102] As thus described, after a "detection candidate more likely to be the object" is found by performing coarse search on the detection candidate and the surroundings thereof by the coarse search device 71, fine positioning is performed by the fine positioning device 76. In fine positioning, a search is performed on an unreduced image of the original size or an image with a reduction ratio lower than that used in the coarse search and close to the original size. Note that in this case, the original size image is also treated as an image with a reduction ratio of one. Furthermore, the image to be searched is an edge angle bit image in the coarse search, while in fine positioning, it is the original image or an image reduced therefrom.

[0103] The fine positioning device 76 arranges a corresponding point search line of the pattern model Y for fine positioning so as to be superimposed on a search image in the detection candidate region, which was obtained on the search image and is similar to the pattern model. Specifically, this is a search device for performing a search using a pattern model with the same magnification as that of the search image in the detection candidate region, which was narrowed down by the searches using the edge angle bit reduction image and the pattern model with two different reduction ratios using the above coarse search device 71 and is similar to the pattern model.

[0104] Each of the pattern models used by the coarse search device 71 and the fine positioning device 76 above is not an image itself as data for use in image processing (hereinafter referred to as "image data"), but data formed by one-dimensionally listing at least an X coordinate position, a Y coordinate position, and an edge angle value corresponding to each edge point obtained from the image data. The "pattern model" is one consisting of such data, and using the pattern model, which is not image data but listed data, can aim to accelerate processing. Meanwhile, the image data is so-called image data, and if it has a data size of 640 × 480 bits, for example, a value must be held for each coordinate position such as each pixel in the entire area.In contrast, when using a pattern model, it can only be configured from a position and angle corresponding to an edge region of an edge image. Therefore, the amount of data required is relatively small, allowing for a reduction in the amount of processing required. Therefore, using a pattern model instead of image data enables processing at a higher speed.

[0105] It should be noted that in the present specification, the "registered image" is the original image to be searched. Meanwhile, the pattern model is the above-listed data suitable for searching for the same image as the registered image from the image to be searched. In the present embodiment, the pattern model includes information on the XY coordinates and the angle θ of each point constituting the pattern model. As described later, data constituting the pattern model in fine positioning includes information on a later-mentioned corresponding point search line superimposed on the information on the XY coordinates and the angle θ of each point.

[0106] Each in Fig. The element shown in FIG. 1 can also be implemented by integrating a plurality of elements, or dividing a function into individual elements. For example, the image processing device 77 for reducing an image can be integrated into the outline extraction device 62. Furthermore, while each function of the calculation device can be processed by a CPU, LSI, or the like, the functions can be resolved into, for example, a dedicated FPGA for performing preprocessing, a dedicated DSP for performing image processing, and the like. In this image processing device 100, image processing is performed in an image processing section constructed of a DSP or the like, and display of an image and a search result are processed by a CPU. As thus described, the resolution and processing of each function can aim to speed up processing.An element that performs each function can be set up as desired.

[0107] The image input device 1 acquires an input image captured and generated by external equipment and undergoes image processing. Input image data can be acquired from external equipment through I / O communication, and alternatively, the candidates can also be input in the form of a data file through a recording medium. Furthermore, the image processing device itself can be made to have a function of capturing an input image. In this case, the image input device 1 functions as an image capturing device and an image generating device.When a camera using a solid-state image pickup element such as a CCD or CMOS as an image pickup device is used and a work, such as an electronic component to be subjected to image processing, is photographed, there is a difference in the amount of reflected light between the light irradiated onto the work and the light irradiated onto the background, and thus a difference in the amount of electronic charge of the solid-state image pickup element occurs between an area corresponding to the work and an area corresponding to the background. Namely, since a difference in the luminance of the image occurs between the work and the background, this luminance difference can be detected as an outline or edge of the work. Note that outline data to the registered image can also be input as CAD data of the work or the like.In such a manner, data input from the image input device 1 is appropriately A / D converted and sent to an image processing device body portion.

[0108] The storage device 4 stores a parameter file containing a variety of data necessary for various transformations and calculations for template matching and the like, as well as data on a pattern model of a registered image, an edge image for an image to be searched, and the like. As thus described, in addition to storing set contents, the storage device 4 can also be used as an area for storing data as an input image. As such, a storage device 4 used may be a semiconductor element such as a DRAM or a flash memory, or a solid-state storage device such as a hard disk.

[0109] The operation device 2 is an input device for operating the image processing device 100. For example, in a case where the user operates a mouse 81 and a keyboard 82 to manually specify a processing area, the input device functions as a processing area specifying device 5. On the other hand, calculation may also be performed by the computer device 6 based on image processing on the image processing device 100 side to automatically specify the processing area.

[0110] The input device is connected to the image processing device 100 by a cable, a wireless connection, or a fixed connection. Examples of a typical input device include a variety of pointing devices such as a mouse, a keyboard, a sliding mat, a track point, a tablet, a joystick, a console, a jog dial, a digitizer, a light pen, a numeric keypad, a touch pad, and a pointing stick. Furthermore, in the mode of connecting a computer on which an outline extraction program is installed to the image processing device 100, or in the mode of viewing the computer where the outline extraction program is installed as an image processing device or an outline extraction device, the above devices can also be used in operations of the image processing device itself and its peripheral equipment, other than an operation of the outline extraction program.Furthermore, the user can directly touch the surface of the screen to perform input and operation by using a touch-sensitive screen or a touch-sensitive panel for a display itself that displays an interface screen, or the user can use a voice input device or other existing input devices, or can also use them simultaneously. In the example of . Fig. 1 the input device consists of pointing devices such as a mouse and the keyboard.

[0111] As the display device 3, a display such as an external liquid crystal monitor or a CRT monitor can be used. Furthermore, a display device and an operation device can be used simultaneously by using a display device of a type equipped with an input function such as a touch panel. The display device 3 can also be incorporated into the image processing device itself if it does not take the form of being externally connected.

[0112] The above configuration is exemplary, and for example, the image processing device itself may include a display device, an operation device, and the like, and each element may also be used simultaneously in another element or may be integrated into the computer device 6. An example will be described below in which an outline extraction program is installed in a general-purpose computer to perform edge coupling processing and outline extraction processing. (Detailed procedure for image processing)

[0113] This image processing device performs preprocessing (enlargement, reduction, smoothing, Sobel filtering, and the like) on a registered image (also referred to as a standard image, reference image, and the like) and an image to be searched, which have been acquired by the image input device 1, and thereafter extracts an edge as feature extraction. The device then performs edge-based pattern search during movement using a pattern model obtained from the registered image. In the present embodiment, as described above, a pattern model is preregistered ( Fig. 3) and in an actual operation, the processing is performed on the image to be searched ( Fig. 4). As described above, distributing processing during registration and processing during movement can speed up processing.

[0114] The operation during registration will be described more specifically. By processing the registered image by the above outline extraction device 62, an outline region of the registered image is extracted as an edge, and an edge image for the registered image expressed by aggregating points with a width of about one pixel is generated. The edge image for the registered image is temporarily stored in an edge image memory for the registered image of the storage device 4. Furthermore, a pattern model is generated from the edge image for the registered image by the segment generation device 68 and the pattern model constitution device 70. The pattern model to be used in the search is held in a pattern model memory of the storage device 4 and is called as needed. (Sample model generation upon registration)

[0115] Fig. 12 shows a flowchart of a procedure for generating the pattern model upon registration. To generate the pattern model, as described above, a main image of an image to be extracted from the image to be searched is specified as a registered image and temporarily stored in the storage device 4. Therefore, a pattern window is set to the registered image (step S1201). A reduction ratio is set with respect to this registered image (step S1202), and thereafter, the image is attempted to be reduced by the image reducing device 77 (step S1203). Further, an outline is extracted from the reduced image. Specifically, edge extraction processing and concatenation processing are performed (step S1204). Further, chains are coupled to generate a segment (step S1205).In such a manner as above, the outline extractor 62, the chain generator 63, and the segment generator 68 attempt to segment chain data with respect to the reduced image. The pattern model is then generated by the pattern model generator 70 (step S1206).

[0116] As described above, in the above embodiment, two types of models are generated as the pattern model, which are the pattern model X for coarse search and the pattern model Y for fine positioning. Furthermore, a data structure of the pattern model for coarse search is formed from a coordinate position in the X and Y directions and an edge angle of each edge point with respect to an arbitrarily set origin point. A data structure of the pattern model for fine positioning is formed from a coordinate position in the X and Y directions, an edge angle, and a later-mentioned corresponding point search line of each edge point with respect to an arbitrarily set origin point. (edge ​​angle)

[0117] The edge angle is one that shows the concentration gradient direction of the edge at the edge point. To show the edge angle, 0 to 360 degrees are expressed by 256 levels. (edge ​​angle)

[0118] Note that, regarding an edge strength of each edge point, in the present embodiment, the data of only the edge points with strength values ​​larger than a preset strength value are configured, and therefore, the strength values ​​are not retained as data. However, the present invention is not limited to this method, and for example, in the case of performing evaluation, weighting, and the like by the later-mentioned rank calculation based on a similarity of edge strength values ​​in the search algorithm, edge strength data may be retained as a value of the data structure of the pattern model. (Decision on the reduction ratio of the image)

[0119] Furthermore, when using a reduced image during registration, the selection of its reduction ratio is very important because the reduction ratio also exerts an effect during movement. An appropriate reduction ratio is set that can eliminate noise and reduce the search time while retaining characteristic points in the image necessary for search. For example, in the case of placing a high value on search accuracy at the expense of search time, a reduction ratio is set to a specific, relatively low reduction ratio to balance search accuracy and search time to a certain extent. Alternatively, an optimal reduction ratio determined by the user through trial and error can be used.In the present embodiment, an automatic mode for automatically deciding a reduction ratio and a manual mode for the user to specify a reduction ratio are switchable. In the automatic mode, the reduction ratio is set based on a length of a shorter side defining a rectangular area of ​​the pattern window PW. Furthermore, in the manual mode, if the user selects the optimal value while visually checking how the pattern model actually transforms based on the reduction ratio, it is possible to facilitate a setting operation as a sense-impression operation. In an example of a user interface of the . Fig. 13A to 13C illustrate how the reduction ratio can be selected using a drop-down list when the manual mode is selected, and how an image is transformed according to the selected reduction ratio. The higher the reduction ratio is made, the more significantly the image changes shape as its corner area becomes less sharp. Regarding such a change in the image, the user can select an appropriate reduction ratio according to his or her application or intended use.

[0120] Another example is a technique for automatically deciding the balance between search accuracy and search time. For example, search accuracy, search time, and both are presented as alternatives for topics that the user attaches great importance to. After the user makes a selection, appropriate adjustments are automatically made according to the selection. (Sample model X for coarse search)

[0121] The coarse search is a search for efficiently narrowing down—for a short period of time—an area where the same image as the registered image is likely to be present as a detection candidate from the image to be searched, before the fine positioning mentioned later. Therefore, the coarse search achieves the above purpose by using a reduced image compared to the original image to be searched or an originally registered image. More specifically, as in the Fig. In the pyramid search shown in Figure 87, the coarse search is performed using a reduced image obtained by reducing an original-size image. This coarse search uses an image with a high reduction ratio and a low resolution. Then, fine positioning is performed on a coarse position (detection candidate) obtained in the coarse search. This fine positioning uses an image with a lower reduction ratio and a higher resolution than those in the coarse search.

[0122] A procedure for generating the pattern model X for the coarse search will be described. In step S1201, a main image of an image to be detected from the image to be searched is temporarily stored as the registered image in the registration image memory of the storage device 4. More specifically, the user sets on the screen a position and a size of an area required as a registered image with respect to a captured image captured from the image capturing device and displayed on the display device, using the pattern window PW in a rectangular shape as shown in FIGS. Fig. 2A to 2H.

[0123] Next, a reduction ratio when reducing the image by the image reduction device 77 in this registered image is decided (step S1202). In the present embodiment, a reduction ratio for a large area of ​​the pattern model described later is decided according to the size of this pattern window PW, namely, the number of pixels included in the rectangle.

[0124] That is, in the case where the number of pixels in the registered image is relatively large, the reduction ratio is set relatively high because the loss of characteristic points within the image is small even with a high reduction ratio. On the other hand, in the case where the number of pixels in the registered image is relatively small, the reduction ratio is set rather low because the characteristic points within the image are more likely to be lost. It is preferable to set a reduction ratio such that the reduction results in the removal of noise within the image but does not result in the loss of the image's characteristic points as the optimal reduction ratio.

[0125] As another technique for deciding a reduction ratio, for example, the registered image is shifted in the X and Y directions by a predetermined amount with respect to the same registered image. When the autocorrelation changes moderately, the image can be determined as having a characteristic that a correlation value is not prone to change as long as a certain degree of fit is realized, and therefore the reduction ratio is set high. On the other hand, when the autocorrelation changes drastically, the image can be determined as having a characteristic in which the correlation value is prone to change, and therefore the reduction ratio is kept low. In this way, the reduction ratio can also be decided based on the autocorrelation.

[0126] Based on the reduction ratio set in such a manner as above, the registered image is reduced in the original size by the image reduction device 77 (step S1203). Specifically, in the image reduction device 77, the reduction in the image is not performed with a final reduction ratio set based on the size of the pattern window PW, but with a reduction ratio positioned at an intermediate reduction ratio between the original size of the image and the reduction ratio for a large area. Further, the edge is aimed to be segmented by the outline extraction device 62, the chain generation device 63, and the segment generation device 68 with respect to the image with the intermediate reduction ratio by the image reduction device 77 (step S1204, step S1205). Note that the intermediate reduction ratio corresponds to the reduction ratio in the second coarse search.

[0127] Reduction in the original-size image is followed by mapping and segmentation of the image, as performing processing in this order can reduce noise while preserving characteristic points retained in the original image, compared to the reverse order in which mapping and segmentation of the original-size image are followed by reduction in the obtained image. Furthermore, this method is also consistent with the purpose of coarse search, which is to extract the detection candidate region that resembles the pattern model. Therefore, the average reduction ratio can be set in a manual setting, where the user manually decides the ratio while observing the segment, so that noise can be reduced to a certain extent while maintaining characteristic points in the original-size image.Alternatively, the average reduction ratio can be adjusted by automatic adjustment, where the ratio is automatically adjusted according to the size of the pattern window PW. (Creation of first sample model)

[0128] Next, in step S1206, the pattern model constructing device 70 generates the first pattern model having the above average reduction ratio using the data segmented by the segment generating device 68. This first pattern model is used for the local search from the coarse search (step S902 of Fig. 9). The Fig. Figures 8A to 8C all show an example of each pattern. In the first sample model, as shown in Fig. 8A, the reference point on each segment present within edge image data is decided based on a predetermined condition, and an edge model point is defined at which an edge angle in the direction of the normal to the segment and in the orientation of the edge with respect to each reference point is set. (Creation of second sample model)

[0129] Furthermore, the pattern model constituting device 70 generates a second pattern model having the above reduction ratio for large areas by using the data segmented by the segment generating device 68 ( Fig. 8B). In this second pattern model for coarse search, in the same manner as the first pattern model, a reference point is decided on each segment present within edge image data based on a predetermined condition, and an edge angle is adjusted in the direction of the normal to the segment and in the orientation of the edge with respect to each reference point. These pattern models do not need to be individually generated according to reduction ratios; rather, it is possible to generate only one pattern model and then use it by magnification, or a reduction ratio according to a reduction ratio of an image to be searched during movement.

[0130] One difference between the first sample model and the second sample model is that in a case where the reference points set on each segment exist multiple times, as in the Fig. 8A and Fig. As shown in Figure 8B, the distance between reference points in the second pattern model is long when compared with the first pattern model. This is attributed to a difference in the reduction ratio between the two pattern models. The degree of difference in the distance between the reference points is controlled by a difference between the average reduction ratio and the large-area reduction ratio.

[0131] As in Fig. 9, the specific coarse search consists of a "large area search" (step S901) performed on the entire area of ​​the image to be searched using the second pattern model with the large area reduction ratio, and a "local search" (step S902) performed only on the candidate region of the detection candidate extracted by this "large area search" using the first pattern model with the medium reduction ratio.

[0132] In the description of the present embodiment, in the coarse search, a "large area search" using a pattern model with the large area reduction ratio and a "local search" using a pattern model with the medium reduction ratio are performed. Specifically, in the case of the following medium reduction ratio with respect to the original image size, the large area reduction ratio is set as follows: (1) In the case where it is not higher than 1 / √2 times, the reduction ratio for large areas is set to one half of the average reduction ratio; (2) in the case of 1 / √2 to a quarter time, the reduction ratio for large areas is set to one-third of the average reduction ratio; and (3) in the case of not lower than one quarter, the reduction ratio for large areas shall be set to one quarter of the average reduction ratio.

[0133] As described above, the reduction ratio for large areas is decided according to the medium reduction ratio, which strikes the balance between search efficiency and storage of characteristic points from the original image.

[0134] The local search may be performed multiple times. In the case where the average reduction ratio is significantly high relative to the original size, for example, in the case where the average reduction ratio is set to one-quarter of the original image size as in (3) above, the average reduction ratio becomes one-sixteenth of the original image size. In this case, since the ratio between the large-area reduction ratio and the average reduction ratio is as large as four times, it may take too long to perform the "local search" on the candidate region of the detection candidate extracted using the second pattern model with the large-area reduction ratio and its periphery.

[0135] Therefore, in the present embodiment, when the ratio between the average reduction ratio and the large area reduction ratio is greater than two times, an additional average reduction ratio is set to add one or more searches so that the ratio between adjacent reduction ratios does not become greater than two times. Namely, by performing two or more local searches, the reduction is aimed at in the time required for one local search. Such automatic addition of the average reduction ratio (reduction ratio for local search) can be performed, for example, by selecting a checkbox for an option to "automatically add reduction ratio for local search" in the Fig. 5 shown user interface screen.

[0136] As described above, a pattern model has a data structure composed of a coordinate position in the X and Y directions and an edge angle of each edge point with respect to an arbitrarily set origin point. Therefore, when the additional average reduction ratio is set, a pattern model corresponding to the additional average reduction ratio is generated from one segment, as in the cases of the above first pattern model and the second pattern model. The pattern model with the additional average reduction ratio differs from both the first pattern model and the second pattern model in that when reference points set on each segment are present in a plurality, the distance between reference points is long compared to the first pattern model and short compared to the second pattern model.Since the pattern model itself consists of data on coordinate positions and the like, even if the pattern model is reduced, a defect in information due to the reduction can be made extremely small compared to the information defect due to reduction in the image data. (Wide area search)

[0137] The operation in the search during movement is also described for convenience of description. As in step S901 of Fig. As shown in Figure 9, by performing the "wide area search" using the pattern model with the reduction ratio for a large area, the candidate area of ​​the detection candidate is extracted. Then, the "local search" is performed on the extracted candidate area of ​​the detection candidate using the second pattern model having the second highest ratio only compared to the reduction ratio for the large area. This narrows down the candidate area of ​​the detection candidate with higher accuracy based on the search result. Subsequently, the "local search" is performed on the narrowed candidate area of ​​the detection candidate using a pattern model with a set average reduction ratio in decreasing order of reduction ratio to repeat the step of narrowing down the candidate area of ​​the detection candidate. (Fine positioning reduction ratio) (Manual reduction ratio decision mode)

[0138] Furthermore, as described above, in the case of the manual reduction ratio decision mode where the user sets the reduction ratio using the user interface of Fig. 5, the user can select a medium reduction ratio, a reduction ratio for a large area, and a fine positioning reduction ratio used using the fine positioning device 76. However, such a setting can also be made in this case that when the ratio between the selected medium reduction ratio and the reduction ratio for a large area is greater than twice between these reduction ratios, another medium reduction ratio is automatically generated based on the above conditions.Furthermore, regarding the fine positioning reduction ratio, when using the fine positioning device 76, when the user is to select a candidate reduction ratio, a value to be selected is limited to the lowest value among the reduction ratios set for the coarse search or a value of a reduction ratio that is still lower than this (including the original size of the image). This can prevent a condition in which fine positioning is erroneously performed using coarser data with a higher reduction ratio than the pre-stage coarse search. (Sample model for fine positioning)

[0139] As in step S903 of Fig. As shown in Figure 9, the fine positioning search is used to perform fine positioning using a pattern model having a final average reduction ratio last used in the "local search" or a lower reduction ratio than that (including the original size of the image) on the candidate area of ​​the one or more detection candidates. The reduction ratio of the pattern model for use in fine positioning is preferably an unmagnified image, namely, the original size original image. (edge ​​sharpness)

[0140] As described above, fine positioning does not necessarily use the original size image to be searched, but can also use an image reduced with a reduction ratio (fine positioning reduction ratio) in a range not exceeding the final average reduction ratio last used in the pre-stage local search. This allows for preferential search results, especially when the image to be searched is blurred.

[0141] For example, it can be considered that the sharper the luminance data waveform of the original-size edge image, the higher the sharpness, and conversely, the smoother the waveform, the more blurred the image. Therefore, when the sharpness of the edge region is lower than a predetermined value, namely, when the edge is dispersed in its width direction by a width not smaller than a predetermined width, thus blurring the image, the reduction ratio is set to an appropriate fine positioning reduction ratio, and the image is reduced to this reduction ratio, so that the sharpness of this image can be reduced to increase the sharpness of the edge, and thus stable positioning accuracy can be obtained.

[0142] For example, in the case of a binary image where pixels suddenly change, as in Fig. 53, an outline, namely an edge of this image, a so-called step edge, where its pixel concentration, namely pixel value, changes stepwise, as in Fig. 54. Therefore, as shown in Fig. As shown in Figure 55, a boundary of a change in the strength of the edge tends to appear sharp, and accurate positioning can be expected. On the other hand, when the binary image is unclear, the boundary area changes smoothly, as in Fig. 56, which results in the change in the strength of the edge becoming a weakly wavy curve, as in Fig. 57. Thus, there has been a problem that even a slight fluctuation in the edge environment, such as a change in illumination or light quantity, affects the edge detection accuracy, thereby preventing stable edge detection and lowering the reliability of image processing such as image recognition. Accordingly, in the present embodiment, the image is reduced to an appropriate reduction ratio to improve edge sharpness. (Decision about the subpixel coordinate of the edge)

[0143] Specifically, fine positioning using edge information performs highly accurate positioning based on the pattern model generated from the position information of the edge generated from the registered image and the position information of the edge generated from the image to be searched. Therefore, the edge position information is extremely important.

[0144] Conventionally, a technique for deciding a subpixel edge position is known, as described in Japanese Unexamined Patent Publication No. H07-128017, U.S. Patent No. 6408109B1, and the like. In these methods, a subpixel coordinate is found using quadratic interpolation, using a total of three data: an edge strength of a target pixel and the edge strengths of two pixels positioned around it. Fig. 58 shows a diagram of a method for calculating a subpixel coordinate according to Japanese Unexamined Patent Publication No. H07-128017. In this drawing, EMc (edge ​​size center) is an edge strength value of an edge point as an edge point remaining after nonmaximum point suppression (nonmaxsupress) processing for thinning the edge. Further, EMf (edge ​​size forward) is an estimated edge strength value indicated by an arrow in an orientation of an edge angle of the target edge point. Furthermore, EMb (edge ​​size backward) is an estimated edge strength value indicated by a circle in an orientation of the edge angle of the target edge point. Further, a subscript of 1 in EMf1, EMbl is given to a characteristic amount of an edge point in the transverse direction, and a subscript of 2 in EMf2, EMb2 is given to a characteristic amount of an edge point in the diagonal direction.In the following example, an angle EAc formed by an edge angle with the horizontal is considered to be < 45 degrees. Other angles can be found by considering a symmetry. The following expressions are established. EMf=EMf1*(1−tanEAc)+EMf2*tanEAc EMb=EMb1*(1−tanEAc)+EMb2*tanEAc

[0145] Using three edge strength data of EMc, EMf and EMb in the above expressions, an offset amount of the subpixel position can be calculated by the following expression: x=(EMf−EMb) / (2(2EMc−EMf−EMb)) y=x*tanθ

[0146] As described above, the subpixel position can be calculated. Next, Fig. 55 and Fig. 57 a state of edge strength in focus and a state of edge strength in an unclear image. As can be seen from Fig. 55, a peak of edge strength that is in focus to a certain extent is sharp, and the edge position can be clearly determined. On the other hand, in the case of an unclear and smeared image, as in Fig. As shown in Figure 57, the vicinity of the maximum is quite flat, and an error in the edge strength has a large effect on the subpixel coordinates. To evaluate this effect, the case of EMf with a graduation error in EMf = EMb is considered. The relationship between the edge angle error and the subpixel position is shown in the following expression: X′=fd(1)−fd(−1)2(2fd(0)−fd(1)−fd(−1) y=x−a2(2c−x−a)

[0147] In the above function, a state “x = a + 1” is checked using the following expressions: δy=a−a2(2c−a−a)−a+1−a2(2c−a−1−a) δy=02(2c−2a)−12(2c−2a−1) In the above expression, if “X = ca”, δy=−12(2X−1)

[0148] When this is expressed by a graph, a graph like Fig. 59. As shown in this drawing, it is found that when the edge strength becomes flat in X → 0, namely, c → a, an effect exerted by the edge strength error on the edge position error becomes larger. The same tendency can be recognized in EMf ≠ EMb. Therefore, in the present embodiment, the sharpness of the edge point is calculated, and using this value, an edge data reduction ratio is calculated for performing the calculation of the subpixel position of the edge. Further, using image data reduced in the image data reduction ratio, edge extraction is performed. As thus described, reducing the image data can produce a smooth waveform as in Fig. 57 into a sharp signal shape as in Fig. 55 transform to stabilize an edge position so that the position accuracy can be improved.

[0149] A procedure for deciding the image data reduction ratio based on the sharpness of the edge point is presented based on a flowchart of Fig. 60. First, in step S6001, an edge image is generated from the unmagnified image. Next, the edge of the edge image is attempted to be thinned in step S6002, while the sharpness of each edge point in the edge image is calculated in step S6003. The edge thinning processing is performed for non-maximum point suppression processing before the edge coupling processing. Furthermore, in step S6004, an average value of the sharpness of the thinned target edge points is calculated. Then, in step S6005, the image data reduction ratio is decided based on this average sharpness. The image data reduction ratio is decided so that the positional accuracy of the edge points is maintained not lower than a predetermined accuracy. (edge ​​model function)

[0150] Next, an edge model function is considered. It is assumed that many of the extracted edges are stepwise edges, and they are assumed to be expressible by an edge model function shown below. "σs" of this edge model function is the sharpness of the edge point. An ideal form of the edge in this case is expressed as follows: edge(x)=Φ(x−dσs) Φ(x)=12π∫−∝xe−z22 dz

[0151] Fig. Figure 61 shows a graph of the above function. The ideal shape (profile) of the edge shown in this figure is plotted by the following expression (in the case of I = 2, I = 0, and σs = 0.6 along the X-axis): S(x;θ)=I Φ(x−lσs) (Pattern search procedure applied to image data reduction ratio)

[0152] Next, a specific procedure for pattern search in image processing using the image data reduction ratio is presented based on flowcharts of Fig. 62 and Fig. 63. In these drawings, Fig. 62 a business upon registration and Fig. 63 shows an operation during movement. (Operation during registration, applied at image data reduction ratio)

[0153] First, the operation is based on registration Fig. 62. In step S6201, the pattern model for the coarse search is generated. Next, in step S6202, edge extraction is performed using the unmagnified image to find a standard deviation σs with respect to each of the extracted edge points. In finding the standard deviation σs, it is assumed that the shape of the edge actually extracted from the image data fits the model of Fig. 61. To calculate the subpixel position of each edge at an edge point with an edge strength not less than a given edge strength threshold, the next expression is considered as an approximate expression of a quadratic derivative of a logarithmic edge strength using edge strengths EMb, EMc, EMf of three adjacent points B, C, F located in Fig. 64 are used. t=(ln(EMf)+ln(EMb)−2 ln(EMc))*(cos(EAc)2)

[0154] Using “t” in the above expression, the standard deviation σs can be calculated from the following expression: σs=1.8263*t(−0.35661)−1.07197

[0155] What is important is that the standard deviation σs is expressible by a single-valued function and the above expression has no particular meaning.

[0156] Meanwhile, in step S6203, the edges extracted from the image data are concatenated and segmented by the edge generator 63 and the segment generator 68 to generate a preliminary pattern model. Furthermore, in step S6204, an average value σa is calculated using only σs of each edge used in generating this preliminary pattern model. Next, in step S6205, using this average value (α), an image data reduction ratio "r" is found by the following expression. This is the fine positioning reduction ratio. r=σa0.6

[0157] In this example, a differential value of a logarithm of the edge strength is obtained, and the reduction ratio is calculated from this differential value. However, this example is not restrictive, and a variety of approximate values ​​regarding a differential value of the logarithm of the edge strength can also be used to calculate the image data reduction ratio. Furthermore, the appropriate value referred to herein includes an approximate value of the differential value of the edge strength.

[0158] As thus described, when the image data reduction ratio "r" is obtained, the registered image is reduced again according to the fine positioning reduction ratio in step S6206. Further, the pattern model for fine positioning is generated from the reduced registered image in step S6207. (Operation during motion applied at image data reduction ratio)

[0159] Next, an operation performed during the movement in response to the registration operation is calculated using the image data reduction ratio based on Fig. 63. First, in step S6301, using the fine positioning reduction ratio obtained above, the searched image is reduced into the searched reduced image for fine positioning. Meanwhile, in step S6302, the reduced image for coarse search is generated from the searched image, and in step S6303, the position and posture of the detection candidate are calculated using the pattern model for coarse search and the reduced image for coarse search. Finally, in step S6304, fine positioning is performed using the pattern model for fine positioning, the reduced searched image for fine positioning, and the position and posture of the detection candidate. (Pre-processing / post-processing of image reduction)

[0160] Even after image data reduction, the image data is reduced to retain edge position information as much as possible. Specifically, subsampling is performed after applying a low-pass filter corresponding to the image data reduction ratio. Edge extraction is performed on the image data after subsampling to obtain accurate edge position information.

[0161] In such a manner as above, it is possible to suppress degradation in accuracy that occurs due to ambiguity of the source image, such as blurring. Furthermore, reducing the image data to reduce the data volume can provide a secondary benefit by allowing subsequent processing to be performed at high speed and with a light load.

[0162] In the present embodiment, the fine positioning reduction ratio is automatically set to the reduction ratio whose upper limit is the final average reduction ratio (first reduction ratio) last used in the "local search." Furthermore, the pattern model for use in fine positioning is generated from the registered image reduced to have the same reduction ratio as the fine positioning reduction ratio. As thus described, the fine positioning reduction ratio is decided based on the sharpness of the edge of the original-size edge image in the range from the original size of the image to the final average reduction ratio last used in the local search.

[0163] As thus described, the fine positioning reduction ratio is set to keep the sharpness at a level not lower than a fixed level, and the reduction ratio can thus be used in the state of high sharpness, so that the stability of the positional accuracy can be ensured. (Operation when registering the pattern model)

[0164] Returning again to the description of the registration operation of the pattern model, a procedure for registering the pattern model for fine positioning is described based on a flowchart of Fig. 14. The pattern model for fine positioning is generated using the same image as the registered image described in the above description of the pattern model for coarse search. First, in step S1401, before processing by the image reduction device 77, the edge image of the original size with respect to the registered image is generated by the outline extraction device 62, and the sharpness of the edge is evaluated. Based on this, the optimal reduction ratio is decided in step S1402. As thus described, in step S1403, the image reduction device 77 attempts to reduce the image based on the optimal fine positioning reduction ratio decided with respect to the registered image.Further, with respect to the image reduced with the reduction ratio (including a reduction ratio of one) set by the image reduction device 77, the edge is segmented by the outline extraction device 62, the chain generation device 63, and the segment generation device 68 (step S1404, step S1405). Specifically, the reduced image undergoes edge extraction processing for extracting the edge and concatenation processing for generating the chain from the edge points, and further, segmentation for coupling the edges is performed. Next, the pattern model constitution device 70 generates the pattern model for fine positioning with the set fine positioning reduction ratio by using data segmented by the segment generation device 68 (step S1406).

[0165] Furthermore, like the above pattern model for coarse search, the pattern model for fine positioning decides the reference point on each of the segments present within the edge image data based on a preset condition (step S1407). Furthermore, the angle with respect to each reference point is set (step S1408). The angle is set in the direction of the normal to the segment as well as the orientation of the edge.Furthermore, the type of segment (type of segment such as a line or a circular arc) where the reference point is provided, a segment representative parameter (a parameter capable of defining the segment made of a line or a circular arc), an angle in the direction of the normal to the segment and in the orientation close to the edge angle, and line segment information having a preset length in the direction of the normal to the segment, namely, a corresponding point search line, are set (step S1409). (Line length of the corresponding point search line)

[0166] Regarding the line length of the corresponding point search line given with respect to each reference point, the same length is set for each reference point. The length is determined by the ratio between the final average reduction ratio used in the last local search and the fine positioning reduction ratio used in fine positioning. In other words, it is set so that the line length is large when the ratio between the final average reduction ratio and the fine positioning reduction ratio is large, and the line length is small when the ratio is small.

[0167] For example, if the final average reduction ratio used in the local search is one-quarter of the original image size and the fine positioning reduction ratio is unmagnified, the ratio between the final average reduction ratio and the fine positioning reduction ratio is four times, and therefore, one pixel in the local search corresponds to four pixels in the fine positioning. Therefore, the length of the line in the fine positioning pattern model is set to a length that covers four pixels each in the positive and negative directions of the edge from the reference point. However, since the length of this line affects the positioning accuracy and search time, covering the total number of corresponding pixels by the ratio of the reduction ratios is not necessarily required. For example, the line length of the corresponding point search line is set short according to the required processing time.On the other hand, the line length can be set to be no less than the corresponding number of pixels. For example, a margin can be set on the line length according to the ratio of the reduction ratios to achieve processing stability. (Change in line length of corresponding point search line)

[0168] Furthermore, the length of the corresponding point search line with respect to the reference point may not be set uniformly forward and backward, and may be changed to make a line length longer or shorter. This processing is performed by the pattern model constituent device 70 or the like. An example of a change in the length of the corresponding point search line is determined based on the Fig. 27 and Fig. 28. In these drawings, Fig. 27 the case where these are equal from the reference points and Fig. Figure 28 shows the lengths from the reference point as different. It should be noted that in these drawings, the corresponding point search lines generated in an inner rectangular area in these drawings have been filtered. As in Fig. As shown in Figure 27, if the lengths of the corresponding point search lines extending from the reference point in the forward / backward and right / left directions are made constant, the lines of the inner rectangular shape will overlap, which may cause erroneous determination. Therefore, if the corresponding point search lines are set not in the inward direction but only in the outward direction, as shown in Fig. 28, a more accurate search result with fewer erroneous identifications can be expected. (Setting intervals of corresponding point search line)

[0169] The corresponding point search line is set on the segment that excludes its end region. This is because such an end region is strongly affected by offset. Therefore, stable processing can be expected by setting the corresponding point search line while excluding the region strongly affected by the offset.

[0170] The intervals for and the number of corresponding point search lines for their setting are determined according to the required processing speed and pattern search accuracy. It is possible to maintain pattern search accuracy by setting at least one corresponding point search line on each line or circular arc constituting the segment. The simplest approach is to place a reference point at the center of the segment, and from this point, reference points are set at equal intervals on the segment. Furthermore, by thinning the reference point setting in an area with a smeared edge angle within the segment and thickening the reference points in a reliably detected area, accuracy can be improved.

[0171] Furthermore, it is preferable to assign at least one corresponding point search line to the center of the segment. This ensures the alignment of at least one corresponding point search line, even if it is short, with respect to the segment that constitutes part of the pattern model. (Preprocessing of the image to be searched during movement)

[0172] The operation of registering the pattern model, namely the generation of the pattern models for coarse search and fine positioning, was described above ( Fig. 3, Fig. 12). During movement, a search is performed using these patterns ( Fig. 4). In the search, predetermined preprocessing is performed on an image to be searched from the image pickup device. A procedure for performing preprocessing on the image to be searched in the search while moving is described based on the flowchart of Fig. 15 described.

[0173] First, in step S1501, based on the input image to be searched, the image reducing device 77 generates a reduced image using the average reduction ratio for the first pattern model for coarse search (first reduction ratio) used in the registered image at registration.

[0174] Meanwhile, in step S1502, the edge angle / edge strength image generating device 60 of the outline extracting device 62 generates an edge angle image and an edge strength image. Furthermore, the thinning device 61 generates a thinned edge angle image based on this edge angle image and this edge strength image.

[0175] Next, in step S1503, the edge angle bit image generator 69 is formed by the outline extractor 62, and based on the thinned edge strength image, an edge angle bit image corresponding to the average reduction ratio of the first coarse search pattern model is generated. Needless to say, the edge angle bit image thus generated is applied to the "local search" by using the first coarse search pattern model in the search operation.

[0176] Further, in step S1504, the edge angle bit image reducing means 78 generates an edge angle bit reduction image corresponding to the large area reduction ratio for the second pattern model for “large area search” based on the edge angle bit image generated by the edge angle bit image generating means 69.

[0177] It should be noted that, as described in the description of the setting of the average reduction ratio in coarse search, in the case of generating the additional pattern model based on the additional average reduction ratio between the firstly set average reduction ratio and the large area reduction ratio, also in this preprocessing, as an arbitrary step S1505, an edge angle bit reduction image corresponding to the average reduction ratio of the additional pattern model is generated by the edge angle bit image reducing means 78 based on the edge angle bit image generated by the edge angle bit image generating means 69.

[0178] In addition, in the above preprocessing during a movement, the processing on the image to be searched is performed in the reverse order to the order of the large area search and the local search as coarse search, as well as the fine positioning, which are performed during the movement (see Fig. 8A to 8C), but the order of generating the pattern models is not particularly limited, and it is understood that the pattern model for coarse search may be generated after generating the pattern model for fine positioning.

[0179] Meanwhile, during the movement, the coarse search is performed by using an image with a high reduction ratio, and the reduction ratio is gradually lowered to perform the fine search on an image of a size close to the original size.

[0180] As thus described, after completion of the preprocessing during the movement, the large area search and the local search are performed as a coarse search using the generated edge angle bit reduction image, edge angle bit image and the like, and after the coordinate of the detection candidate is found, the fine positioning is performed ( Fig. 9). (Details of each operation upon registration)

[0181] The diagrams of the operations during registration and during motion were described above. Next, the image processing operation during registration will be detailed. During registration, the edge angle / edge strength image generating device 60 applies the Sobel filter to the registered image and finds the edge strength and edge angle at each point constituting the registered image to calculate edge information including the edge strength, edge angle, and edge position. Thinning processing is performed based on the edge information to find an edge point. As a specific example of thinning processing, edge strength non-maximum point suppression processing can be used. The edge is thinned to have a line shape with a width of one pixel.

[0182] Note that the edge point can also be found using the accuracy of the subpixel position. For example, the subpixel position can be calculated using quadratic interpolation (see, for example, Japanese Unexamined Patent Publication No. H07-128017).

[0183] Further, the obtained edge points are coupled to create a continuous chain. The edge linking device 64 performs edge coupling processing of coupling adjacent edge points whose edge angles point in virtually the same direction to create a continuous line element (chain). The chain thus obtained also has an XY subpixel coordinate. Each chain is an aggregation of the edge points, and each of the individual chains is provided with a chain index as an identifier for distinguishing each chain.

[0184] Further, the chains are subjected to approximation by edge chain segmentation device 65 to generate a segment. The segment is found by fitting where the chains are approximated by a line and a circular arc using the least squares method. In the fitting, the approximation is first performed by the line, and if an error of the approximation by the line exceeds a predetermined threshold, the fitting is switched to the approximation by the circular arc. If the error does not decrease even when approximated by a circular arc, a result of the fitting by a line is used.In this way, the operation of sequentially performing the matching in combination of lines and arcs is repeated. When the error of the matching result exceeds the threshold, the data obtained up to that point, if sufficiently long, is considered to be a continuous line segment. Since the edge point is found at the subpixel position, the segment can also be obtained at a highly accurate position in the subpixel order.

[0185] The segment is generated by approximating the chains by line and circular arc. The line segment can be expressed by an expression representing a straight line (e.g., ax + by + c = 0), coordinates of end points, and the like. Meanwhile, the circular arc segment can be expressed by a coordinate of the center, a radius, a start angle, an end angle, and the like. For example, a center coordinate (xo, yo) of a circular arc and a radius ro in "(x - xo)2 + (y - yo)2 = ro2" express the circular arc segment. At each of the segments generated in this way, the reference points are set at predetermined intervals.

[0186] It should be noted that although the example of approximation using a line or a circular arc as the segment has been described, this is not limiting, and a cone curve, a spline curve (wedge), a Bezier curve, and the like can also be used as appropriate. Thus, with a fixed geometric shape such as a circle, an oval, a triangle, or a rectangle as a reference, a pattern model can be generated using these shapes individually or in combination, thereby facilitating the generation of pattern search and any subsequent processing. (Reduction of pattern model)

[0187] Furthermore, the pattern model is reduced during the movement during the search. This reduction ratio is a reduction ratio for reducing the image to be searched for the coarse search during the movement, which will be described later. Since this reduction processing is to be performed, intervals from the reference points are set as the model edge points of the pattern model, so as to prevent the reference points from indicating the same coordinate as a result of the reduction processing. Consequently, the pattern model changes from Fig. 16 to one like in Fig. 17. (Difference between pattern model for coarse search and pattern model for fine positioning)

[0188] The pattern model for coarse search and the pattern model for fine positioning are generated separately from the registered image of the original size (or its reduced image). In other words, the segment of the pattern model for coarse search is not generated from the segment of the pattern model for fine positioning, and the segments of the two models do not necessarily match. Furthermore, since the size of the pattern model differs between the pattern model for coarse search and the pattern model for fine positioning, a distance also exists between each reference point. The difference in distance obtained by transforming the distance from each reference point by unmagnification depends on the reduction ratio.

[0189] Furthermore, the coarse search pattern model provides the coordinates of the reference point and the orientation of the edge at the reference point (angle information). In other words, the angle is set so that it is close to the orientation of the edge in the normal direction to a segment that does not contain information about the length of a corresponding point search line. When performing a coarse search using this coarse search pattern model, the pattern model is placed on the image to be searched, and it is checked whether the edge exists at the positions of the reference points and whether the edge orientation matches the orientation of the pattern model.

[0190] On the other hand, in addition to the coordinate of the reference point and the orientation of the edge at the reference point in the pattern model for coarse search, the pattern model for fine positioning has a corresponding point search line passing through the reference point and having a predetermined length extending in a direction substantially orthogonal to the segment (i.e., one that defines the length of the normal to the segment) and the type of the segment (e.g., an attribute such as a line or a circular arc). This difference corresponds to the processing contents of each search. Namely, in fine positioning, an edge corresponding within the range of the corresponding point search line is searched. In this way, the pattern model for fine positioning functions as a corresponding edge point selector for selecting a corresponding edge point corresponding to the reference point.

[0191] It should be noted that during contour extraction, the segment does not necessarily need to be generated. The corresponding point search line can be set not from the segment, but directly from the chain. For example, in the case of setting three reference points with respect to a certain contour, three edge points forming the chain corresponding to the contour are set at equal intervals to set the corresponding point search lines in the respective normal directions. With this method, high-speed processing can be expected because no segment is generated, while accuracy suffers somewhat because the chain is not approximated by straight lines or circular arcs.In particular, the chain is formed by simply coupling the edge points and therefore it may have poor linearity, while the segment is fitted by the straight line or circular arc, so that a more accurate calculation result is obtained and the position accuracy is also stabilized. (Details of the coarse search while moving)

[0192] Next, an operation during the movement for actually searching the appropriate area from the image to be searched using the pattern model registered in such a manner as above will be described. First, details of a procedure for obtaining a rough position and posture in the rough search will be described based on a flowchart of Fig. 18. In the present embodiment, the coarse search is divided into the large-area search and the local search and is executed to find the detection candidate. (Step S1801 - Reduction in the image to be searched)

[0193] First, in step S1801, the image to be searched is reduced as the object to be searched in accordance with the reduction ratio of the registered image. For example, an image in Fig. 19A is reduced to the same magnification as that of the registered image to obtain a Fig. 19B to obtain the reduced image to be searched. First, the image is reduced to the average reduction ratio as the ratio for coarse search. In other words, the reduction ratio is not first reduced to the reduction ratio for a large area, which is a large reduction ratio, but is first reduced to the average reduction ratio, which is a small reduction ratio. (Step S1802 - Acquisition of edge angle image and edge strength image)

[0194] Next, in step S1802, the edge strength image and the edge angle image are separately obtained from the reduced image to be searched by an edge calculator. Sobel filtering or the like can be used as the edge calculation method.

[0195] The Sobel method is described. The Sobel method uses a 3x3 matrix as an operator (kernel). This method extracts a pixel value from the central point of a value obtained by multiplying pixel values ​​(e.g., brightness) by a coefficient related to peripheral points, with the target point set at the center and the multiplied values ​​added together. This method is a horizontal and vertical filter and has the property of being resistant to noise because it includes a smoothing operation. The kernel for use in the Sobel filter is shown below. |−101−202−101| |−1−2−1000121|

[0196] As a result, the edge strength image and the edge angle image of the image to be searched are obtained independently. (Step S1803 - Generation of edge angle bit image from image to be searched)

[0197] Further, in step S1803, the edge angle bit image is generated by the edge angle bit image generation device 69 from the edge angle image and the edge strength image. The edge angle bit image is image data expressing, as an angle bit, the edge angle of each point constituting the edge angle image. As a result, the edge angle bit image is obtained. Transformation from the edge angle image to the edge angle bit image will be described later. (Step S1804 - Reduction in edge angle bitmap)

[0198] Further, in step S1804, the obtained edge angle bit image is reduced by the edge angle bit image reducer 78. The reduction ratio is set to the reduction ratio for a large area in the case of generating the second pattern model for large-area search and to the medium reduction ratio in the case of generating the first pattern model for local search. As a result, the reduced image of the edge angle bit image is obtained. (Step S1805 - Execution of the large area search)

[0199] Next, the wide-area search is performed on the edge angle bit reduction image reduced in step S1804 using a pre-reduced pattern model. Specifically, the search is performed in the entire area while changing the angle of the pattern model so that the image is scanned from the upper left to the lower right. This extracts the detection candidate area. The position and posture of the detection candidate are extracted, for example, by the XY coordinate and the angle θ, respectively, or the like. The detection candidate is found by rank calculation. The reduced pattern model is moved by the coarse search device 71 in the degree of freedom of the search position and posture, and a rank is calculated in each position and posture. (ranking calculation)

[0200] The rank calculation in the large-area search is performed by comparing the edge angle images at corresponding edge points included in the pattern model and the edge angle bit image obtained by reducing the image to be searched in the large-area reduction ratio, to calculate a coincidence. When performing the search, the position and angle data of the reference point are changed in accordance with the position and posture of the rank in which the rank is to be calculated. Subsequently, the angle is transformed as in the edge angle bit image, and AND processing is performed on a pixel value of the edge angle bit image data after reduction. A value obtained by dividing a total value of the number of remaining bits by a maximum value of an expected total value is regarded as the coincidence calculated by the coarse search device 71.Furthermore, a plurality of bits may be assigned in the angular direction to add a concept of weighting. (Step S1806 - Execution of local search)

[0201] In addition, the local search is performed on the area of ​​the detection candidate found in the large-area search. In the local search, the pattern model for local search is used, with its reduction ratio made smaller than that of the pattern model for large-area search. Furthermore, the reduced image, which is reduced by the reduction ratio for local search, which is lower than the reduction ratio for large-area search, is also used as the edge angle bit image to be searched.

[0202] Furthermore, when performing the local search, not only the area of ​​the detection candidate found in the wide-area search is used as it is, but the local search can also be performed in the vicinity of the detection candidate, for example, at edge pixels such as 3x3 pixels and 5x5 pixels. Thus, a stable search result can be expected. (Extension processing)

[0203] In order to stabilize the rank calculation result, magnification processing can be performed even when performing the coarse search. There is usually a tendency that when the reduction ratio of the image to be searched decreases to improve accuracy, even a small positional shift will cause a large drop in rank. A rotation angle can be changed minutely to avoid a rapid change in rank, but in this case, the disadvantage of increasing the processing amount occurs. Therefore, considering the balance between reducing the processing amount and improving accuracy, the edge angle bit image as the image to be searched is only magnified by a predetermined amount. For example, the image is magnified in its XY direction by a predetermined number of pixels, such as 2x2 pixels obtained by doubling one pixel.It is therefore possible to suppress rapid fluctuations in the rank value due to small offset of the angle in order to obtain a stable rank.

[0204] In this way, the coarse position in the reduced image to be searched of the reduced pattern model is decided based on the calculated rank. Furthermore, the above step can be repeated as appropriate to improve the accuracy of the coarse position. Not only is the coarse search simply divided into two steps, namely, large-area search and local search, but the local search can also be divided multiple times, and a larger reduced image to be searched can be used by gradually lowering the reduction ratio of the image to be searched, thereby performing highly accurate positioning.

[0205] It should be noted that due to its wide scope and large processing volume, large-area search is typically performed only once. However, it can be performed multiple times depending on the required accuracy or tact time. Furthermore, regarding the search technique, a well-known search technique can be used, such as edge search, normalized correlation search, generalized Hough transform, or geometric hashing. (Details of fine positioning during movement)

[0206] After the coarse search is performed in such a manner as above and data on the position and posture of the detection candidate where the pattern model is present are found, the fine positioning is performed by the fine positioning device 76. Next, a specific procedure for the fine positioning will be described in detail based on a flowchart of Fig. 20.

[0207] First, in step S2001, the pattern model for fine positioning is superimposed on the search image based on the position and pose of the detection candidate found in the coarse search. It is preferable to use the original size search image and the pattern model for fine positioning, while the position and pose finally found in the coarse search are regarded as an initial position and pose. However, fine positioning may also be performed with a reduction ratio higher than the original size (a reduction ratio of one) and lower than the reduction ratio last used in the coarse search.

[0208] Furthermore, in step S2002, the point is found as the corresponding edge point along the corresponding point search line of the pattern model for fine positioning. As described above, the corresponding point search line has a predetermined length and extends in the direction of the normal to the segment, and a start point as one of both ends of the line segment is regarded as a search start point, and an end point is regarded as a search end point. First, edge calculation is performed along the corresponding point search line to obtain an edge vector. As the edge calculation technique described above, the Sobel filter can be used as needed. The edge vector, edge angle, edge strength, edge position, and the like of each point on the corresponding point search line obtained by this edge calculation are found.Note that the edge vector is one that expresses the edge strength and orientation by the vector, and can be expressed as (Ex, Ey). For example, as shown in . Fig. As shown in Figure 46, when the edge strength is EM and the edge angle is θE, these are expressed by: The edge angle θE = Atan(Ey / Ex); and the edge strength EM = √(Ex2 + Ey2). (Corresponding edge point search processing)

[0209] Furthermore, based on the information of edge vector, edge angle, edge position, and the like, the corresponding edge point corresponding to the segment including the reference point of the corresponding point search line is found. As an example of the method for deciding a corresponding edge point, the corresponding edge point can be decided at high speed using the above edge vector. As another method, the calculation can be performed using the edge strength and edge angle as described below, but in this case, Atan must be calculated as described below, which complicates the calculation. The following describes a procedure for obtaining a corresponding edge point using an edge strength and an edge angle.

[0210] First, the maximum point with an edge strength greater than a predefined edge strength threshold and at which the absolute value of the difference between the edge angle and the angle of the reference point is smaller than a predefined edge angle threshold is taken as a candidate corresponding edge point. Furthermore, the closest point to the reference point among the corresponding edge point candidates is finally considered the corresponding edge point.

[0211] Furthermore, the subpixel position of the edge of the corresponding edge point is found (step S2003). Using this position and the geometric data of the segment, the error value is obtained, and the least squares calculation is performed (step S2004) to obtain a fine position (step S2005). Examples of error values ​​in the case of the line segment include a distance between the corresponding edge point and the straight line, and examples of the error value in the case of the circular arc segment include an absolute value of the difference between the radius and a distance between the corresponding edge point and the central position.

[0212] As thus described, the error value or the weight value for use in the least squares calculation is calculated by the fine positioning device 76, and from the calculated value, simultaneous equations obtained by the least squares method are acquired. The least squares method is adjusted so that the segment assumes an ideal shape and the error of a plurality of corresponding points corresponding to the segment is minimized. Further, the simultaneous equations are solved to find a highly accurate position and posture. In this way, a corrected amount Δx of a position X, a corrected amount Δy of a position Y, a corrected amount Δθ of the angle θ, and a corrected amount Δs (s = scale) of a scale "s" are obtained.

[0213] In fine positioning, the reference point is superimposed on the image to be searched using the position and pose data obtained during coarse search. Then, edge calculation such as Sobel filtering is performed along the corresponding point search line to obtain the edge vector. Note that the edge vector is represented by a result of applying the Sobel filter and can be expressed as (Sx, Sy) or the like. Furthermore, the edge strength EM can be expressed as "EM = √(Sx2 + Sy2)", and the edge angle θE can be expressed as "θE = Atan (Sy / Sx)", or the like. Furthermore, from the edge angle vector, the edge strength and the position of the pixel at the corresponding edge point are obtained.From these edge vector, edge angle, edge thickness and position, the corresponding edge point corresponding to the segment containing the reference point of the corresponding edge point is found by the fine positioning device 76.

[0214] This condition is based on Fig. 21A. First, the pattern model PM shown by a thick solid line is superimposed by the fine positioning device 76 and placed in the position of the detection candidate of the image to be searched (edge ​​angle bit reduction image EABR, shown by a broken line) obtained in the coarse search. Then, along a corresponding point search line TL passing through the reference point KT set on the pattern model and almost vertical to the segment of the pattern model, a corresponding edge point TT corresponding to the reference point KT is found. Fig. 21A, the corresponding point search line TL is shown by a thin solid line. Note that the corresponding point search line TL is an imaginary line and is not actually drawn. The corresponding edge point TT becomes an intersection point of the corresponding point search line TL and the reduction image EABR. Using the corresponding edge point TT, the subpixel coordinate position can be obtained. Using this position and geometric data about the segment, fine positioning is performed by the fine positioning device 76.

[0215] Specifically, the relationship between the geometric data (in this case, a line) of the segment and the corresponding edge point is considered as an evaluation value. Rank calculation is performed to minimize or maximize the accumulated value of the evaluation values. Typically, a distance can be used as the evaluation value. With this distance considered as an error value, least squares calculation is performed to minimize the error value, thereby obtaining the fine position. The distance used can be a Euclidean distance between the segment and the corresponding edge point.Namely, when the segment is a line, the distance between the corresponding edge point and the straight line is used. When the segment is a circular arc, the absolute value of the difference between the radius and the distance between the corresponding edge point and the central position is used. Solving the simultaneous equations obtained by a least-squares solution can find a highly accurate position and posture. Furthermore, the evaluation value is not limited to the distance and can be an angle formed by the reference point and the corresponding edge point of the reference point.

[0216] Furthermore, Fig. 21B shows a state of corresponding edge point search processing in which the fine positioning device 76 finds the corresponding edge point. In this drawing, as in Fig. 21A, a broken line indicates the reduction image EABR of the image to be searched, a thick solid line indicates the pattern model PM, and a thin solid line indicates the corresponding point search line TL set to the reference point KT. In this case, a coordinate position XY is found by applying the Sobel filter in a region SR of 3x3 pixels of the reduction image EABR. The central coordinate in this calculation is found using Bresenham's algorithm for generating straight line data. In the example of Fig. 21B, a pixel B is extracted as the corresponding edge point with respect to a model edge point A.

[0217] The Fig. The method shown in Fig. 21B differs from Japanese Patent No. 3759983 described above in that a point appropriately selected on the segment where the corresponding edge point has been automatically extracted is regarded as the reference point. Specifically, in Japanese Patent No. 3759983, no arrangement method for a search line is defined. Furthermore, in deciding the corresponding edge point, data on the edge angle, the edge point, and the like are used in addition to the edge strength to improve the reliability of the corresponding edge point. In addition, the kernel used in the processing for obtaining an edge angle and an edge strength is made more compact to reduce a load in computer processing. Furthermore, the position of the corresponding edge point can be obtained with subpixel accuracy.In addition, the use of the least squares method can provide the advantage of being able to fit models in a variety of shapes.

[0218] In such a manner as above, it is possible to perform highly accurate, high-speed positioning during pattern search. In particular, with this method, by changing the line length of the corresponding point search line, the range for searching the corresponding edge point can be easily changed using the corresponding point search line, thus providing the advantage of being able to adjust the necessary stability. Namely, by gradually reducing the length of the corresponding point search line during repeated application of the least squares method, positioning with higher accuracy at higher speed can be easily realized.

[0219] In addition, since the pattern model is expressed by the segment, a corrected phenomenon of edge position as in Fig. 22 can be eliminated. Namely, in the case of performing fine positioning between points, a large positional offset may occur between the pairings with a point in a higher wave and the pairing with a point in a lower wave, but such an influence can be reduced.

[0220] Furthermore, instead of or in addition to this, the edge angle threshold can be changed each time the least squares method is repeated. Gradually reducing the edge angle threshold according to the repeated number of least squares methods also allows for more stable positioning.

[0221] It should be noted that when superimposing the pattern model on the image to be searched based on the initial position obtained in the coarse search or the fine positioning start position obtained in another coarse search, unprocessed data, so-called original image data, is used as the image to be searched, and a pattern model corresponding to the original image data is used as the pattern model.

[0222] This method can eliminate the need to transform all pixels in the original image data of the image to be searched into edge image data, thus aiming for processing acceleration. Especially for inline processing, where tact time is required, such high-speed, low-load processing is preferred. It should be understood that if the overall pre-extraction of edge data is more efficient, all points in the image to be searched can be transformed into edge image data to perform pattern search.

[0223] Furthermore, superimposing and arranging the entire pattern model on the image to be searched is not necessary; superimposing and arranging at least the corresponding point search line is sufficient. In particular, since the corresponding point search line is a straight line, it can be easily obtained by calculation. Therefore, in the present specification, the term "superimposing" does not necessarily mean actually superimposing an image, but is used to mean the processing for deciding the corresponding edge point according to the corresponding point search line. Furthermore, the phrase "superimposing and arranging" in this case is intended to describe that the corresponding position of each image can be easily acquired by superimposing it as described above, which is merely imaginary in the calculation, and logically, the process of actually superimposing data is not necessary.

[0224] According to this method, a highly accurate edge-based search can be realized compared to the conventional method. In the above technique of Japanese Patent No. 3759983, the direction and angular component of the edge are not considered, but only a predefined edge direction is considered, and therefore stability cannot be expected in a complicated shape. In contrast, in the technique according to the present embodiment, importance is given to the edge direction, thereby allowing an improvement in the reliability of the corresponding edge point. Furthermore, in the present embodiment, since a difference is calculated using a small filter of the kernel, such as a Sobel filter, the edge can be detected even when the work is long and narrow.As described, it is possible to realize a stable edge-based search that is adaptable to a search object having a complicated shape, compared to the technique of Japanese Patent No. 3759983.

[0225] Furthermore, when obtaining the position of the detection candidate in the image of the pattern model to be searched, namely the initial position in the local search, there is the advantage of performing a high-speed, low-load pattern search by reducing an image to be searched and performing the search. However, since some information may be lost due to the reduction, thereby causing a deterioration in accuracy, it is desirable to perform the reduction in a manner that preserves a certain amount of information (discussed later). Furthermore, in addition to the form for obtaining the initial position of the pattern model in the coarse search, the position can be manually specified by the user.

[0226] The point mentioned in the above example means the point constituting the image to be searched or the registered image, namely a pixel, but it should be understood that a plurality of pixels (e.g., four pixels) can be packaged together as a single point. Therefore, in this specification, the point means one pixel or a predetermined number of pixels.

[0227] Furthermore, the phrase "based on the reference point" is used not only to mean that edge detection is performed at the reference point, but also to mean that edge detection is performed in the vicinity of the reference point. For example, edge detection is performed within a designated area, such as a range of one to ten pixels around the reference point.

[0228] Furthermore, a segment refers to a continuous line with a finite length configured from a line and / or circular arc, and / or a combination thereof. Furthermore, cone curves, spline curves, Bezier curves, and the like can be combined in addition to lines and circular arcs. Furthermore, the corresponding point search line data includes the coordinates of the reference point and the angle and length of the corresponding point search line. (least squares method)

[0229] In the least squares method, a straight-line error function is fitted to the line segment. The straight-line error function is a least squares method with the distance between a point and a straight line considered as an error function. Furthermore, a circular arc error function is fitted to the circular arc segment. The circular arc error function is a least squares method with the distance between a point and a circular arc considered as an error function. This will be discussed later.

[0230] Examples of the problem of the least squares method include that even when a fairly different value is present, the accuracy deteriorates dramatically due to the influence of that point. Therefore, the present technique uses a weighted least squares method, which is made to have a weight value so as to lower the weight on such a point, to suppress this influence.

[0231] Furthermore, as a degree of freedom to be used in the least squares method, X-direction movement, Y-direction movement, rotation, magnification / reduction, skew, aspect, and the like can be used. Selecting these can correspond to rotation, magnification / reduction, distortion, and the like of the registered image, in addition to parallel movements in the XY directions. (Generalization of the error function of the least squares method)

[0232] The error function of the least squares method is generalized and developed. First, it is considered that an error function E(po, p1, ..., pn) can be decided by an affine parameter po, p1, ..., pn (e.g., p0 = x, p1 = y, etc.). It is assumed that the error function E(po, p1, ..., pn) is minimized by an optimal affine parameter poo, plo, ..., pno (o: optimized). At this time, the error function E(po, p1, ..., pn) is expressed by the following expression: E(p0,p1,…,pn)=∑iωiei(p0,p1,…,pn)2

[0233] The meanings in this expression are as follows. i: Index of a corresponding edge point ωi: Weight determined according to the positional relationship between the corresponding edge point and the model. For example, if the point-line distance between the corresponding edge point and the line is long, this parameter is defined as close to zero. e(p0, p1, ..., pn): Individual error functions determined by the geometric distance between the corresponding edge point and the model. This parameter is determined by the point-line distance between the corresponding edge point and the line segment, or the like. p0 to pn: Affine parameters of parallel x-movement amount, parallel y-movement amount, rotation angle, scale value, and the like.

[0234] To find the affine parameters poo, plo, ... pno that minimize the error function E(po, p1, ..., pn), offset amounts are found as follows from affine parameters p0t, plt, ... pnt that are expected to be sufficiently close to the affine parameters to be found and that were obtained during the coarse search or the last fine positioning: Δp0,Δp1,…,Δpn(pi0≈pu+Δpi)(t:trial) Δp0, Δp1, ..., Δpn are obtained by solving the following simultaneous equations: [∑iωi∂ei∂p0∂ei∂p0∑iωi∂ei∂p0∂ei∂p1…∑iωi∂ei∂p0∂ei∂p0∑iωi∂ei∂p1∂ei∂p0∑iωi∂ei∂p1∂ei∂p0∑iωi∂ei∂p1∂ei∂p1…∑iωi∂ei∂p1∂ei∂p0…∑i ωi∂ei∂pn∂ei∂p0∑iωi∂ei∂pn∂ei∂p1…∑iωi∂ei∂pn∂ei∂pn][Δp 0Δp0Δpn]=[−∑iωiei∂ei∂p0−∑iωiei∂ei∂p0…−∑iωiei∂ei∂pn]

[0235] As described above, using the image edge angle as well as its edge strength can add a directional component, allowing for stable positioning that is resistant to noise. In particular, by applying differential processing to the image data, it is possible to perform a stable search that is less subject to brightness fluctuations. (Corresponding point search line filter processing)

[0236] In particular, it is very likely that performing the search processing of the corresponding point search line on the registered image to select the corresponding point search line is difficult; such a position is preferably eliminated from the pattern search. For example, when considering the case where a registered image as shown in Fig. 23, when corresponding point search lines automatically set to this pattern are automatically set, they are displayed as shown in Fig. 24. As shown in this figure, the corresponding point search line is set not only on the peripheral outline, but in an area with a contrast difference located near the center of the interior. When edge matching is performed using such corresponding point search lines set near the center, areas with similar edge angles are generated in large numbers. Consequently, when pattern search is performed using these corresponding point search lines, the corresponding edge points are more likely to be blurred during corresponding edge point detection.

[0237] In the present embodiment, such a blurred corresponding point search line is eliminated to enable stable, high-accuracy search. Specifically, the corresponding point search line set in such an unfavorable area is filtered by a corresponding point search line filtering device, and if one with a similar edge strength and edge angle exists, that line is eliminated. Fig. 25 shows an example of a result of filtering the corresponding point search lines from Fig. 24.

[0238] An example of a procedure for performing corresponding point search line filtering processing on a candidate of the corresponding point search line in the above manner is explained based on a flowchart of Fig. 26. First, in step S2601, corresponding point search line generation processing is performed on the registered image to generate candidates for the corresponding point search line. Next, in step S2602, the candidates for the corresponding point search line are arranged with respect to the registered image. When arranged at a position where the corresponding point search line was generated, the candidates for the corresponding point search line can be detected in the vicinity of the medium of the candidates for the corresponding point search line.

[0239] Further, in step S2603, the search for candidates for the corresponding point search line is performed along the candidates for the corresponding point search line to count the number of candidates for each corresponding point search line. Further, in step S2604, the filtering processing is performed, and if the number of candidates for the corresponding point search line is two or more, it is decided that the corresponding point search line is likely to be smeared and is thus eliminated from the candidates for the corresponding point search line. In step S2605, the remaining corresponding point search lines are regarded as final corresponding point search lines. Through this processing, an uncertain corresponding point search line is eliminated, so that stable pattern search can be expected.

[0240] Note that when repeating the fine positioning step, since the line length of the corresponding point search line is reduced based on the repeated number of times, the corresponding point search line selected by the filtering processing of the corresponding point search lines once is recorded in advance, and this information can be used in the repetition step. Alternatively, when shortening the corresponding point search line, similar filtering processing of the corresponding point search lines is performed, and the corresponding point search line selected as the result of the processing can be recorded. (Corresponding point search line)

[0241] Furthermore, by changing the line length of the corresponding point search line, it is expected to improve the stability of fine positioning and accelerate the fine positioning. The line length of the corresponding point search line is determined based on the difference in the reduction ratio between the coarse search and the fine positioning. For example, if fine positioning is performed on the unmagnified image and the final coarse search is performed with a reduction ratio of one-quarter, the length is set to 8 pixels (2*4=8). (Chain filtering device 66)

[0242] In the above example, at least one reference point is set with respect to each segment. In the present embodiment, when generating the segment, chains constituting the segment are selected to construct a highly reliable segment in advance and set the reference point in each segment. Selection or elimination of such specific chains is performed by the chain filtering device 66 shown in the block diagram of Fig. 1. Examples of a reference for selecting the chain by the chain filtering device 66 include an average edge thickness and a chain length.

[0243] The chain filtering device 66 performs the selection of chains during registration and during movement. During registration, chains worthy of constituting the segment are extracted. Specifically, filtering is performed to eliminate a chain with a short length that does not meet a predetermined chain length threshold, and a chain with a low average edge strength that does not meet a predetermined edge strength threshold, because even if the segment is generated from these chains, the reliability of segment data is expected to be low.

[0244] Meanwhile, during the movement, since a short chain is very likely to be noise, whether to use the chain or not is selected based on the state of the image being searched. For example, the user sets a length threshold to eliminate a short chain. These are presented in detail below.

[0245] First, in the case of performing filtering based on the average edge strength, the chain filtering device 66 performs filtering by calculating an average edge strength of the edge point included with each chain and comparing the calculated value with a preset average edge strength threshold. Namely, a chain with a low average edge strength is eliminated, and only a chain with an average edge strength not less than a fixed strength is segmented. The reference point is set with respect to the obtained segment to generate the pattern model, so that the edge-based pattern search can be expected with high accuracy and the search accuracy can be improved.The average edge strength threshold can be set by the user to a level that includes an edge strength sufficient to identify the outline of the pattern model.

[0246] Furthermore, in the case of performing filtering based on the chain length, the chain filtering device 66 performs filtering by comparing each chain length with a preset chain length threshold. Namely, only a chain with a chain length not less than a fixed length is selected, and a chain shorter than this is eliminated, so that pattern search can be performed based on the stable edge to contribute to the improvement in accuracy.

[0247] Meanwhile, the segment selector 67 can also be used to perform filtering on a segment configured from chains. Examples of a reference for selecting the segment by the segment selector 67 include, similarly to the above-mentioned chain filtering device 66, an average edge strength, a segment length, whether or not an edge angle image with a similar edge angle exists in the neighborhood, and elimination of uneven distribution of identical edge angles. Furthermore, a short segment does not simply have to be eliminated uniformly; the method for filtering a segment can be changed according to the combination of the line and circular arc that constitute the segment.For example, in a case where a sufficiently long segment is extracted from the aggregation of segments and the combination of these is estimated to find that one or more circular arc segments are present, if one or more lines are also present, the shorter segments are considered unnecessary and are thus eliminated or deleted. Furthermore, in the case where each segment is the line, if three or more long segments are present, sufficient accuracy can be maintained even if the other shorter segments are eliminated. As thus described, the filtering of the segment selector 67 can be changed based on the combination constituting the segment, and an appropriate segment can be selected, so that the search can be expected to be more efficient.

[0248] It should be noted that the segment length in this case refers to the length of the straight line or curve from one end to the other of the line or circular arc of each segment constituting the pattern model. Furthermore, corresponding segment length thresholds may be individually provided for the line length and the circular arc length. The segment length threshold is set in accordance with the registered image, the required accuracy, and the like, or may be set based on the average segment length of the registered image or the image to be searched.

[0249] Furthermore, when filtering is performed based on whether or not a segment having a similar edge angle exists in the neighborhood, the segment selector 67 determines whether or not another segment having a similar edge angle exists in the neighborhood with respect to the edge angle of the edge point included in each segment, and eliminates such a segment if it exists. Namely, considering the possibility that a pattern search result is unstable due to the presence of the segment having a similar edge angle, such a segment is eliminated, and the reference point is then set, allowing for improvement in the stability of the pattern search result. (Preliminary corresponding edge point)

[0250] Next, a procedure for finding the coordinate of the corresponding edge point corresponding to the reference point in fine positioning is performed based on Fig. 29. A corresponding edge point is first found on the corresponding point search line TL, and a corresponding edge point is found as the other of a pair. An average coordinate of these two points is found, and the obtained coordinate is regarded as a real corresponding edge point coordinate.

[0251] Specifically, Fig. 29, the reference point KT is set in a part of a circular arc segment (the position indicated by a circle in the drawing), and the corresponding point search line TL passing through this point is extended from the top left to the bottom right. First, an edge strength of each point of the image to be searched along the corresponding point search line TL is checked. In the example of Fig. 29, the edge strength is found based on the reference point KT. The edge strength is checked at each of the four points "a", "b", "c", and "d" as vertices of a raster shape containing the reference point KT. A subpixel position of the point with the greatest edge strength is found as a preliminary corresponding edge point. Consequently, if "e" is calculated as a preliminary corresponding edge point as the subpixel position of point "a", a pair point corresponding to this preliminary corresponding edge point "e" is subsequently selected. The pair point is selected such that the preliminary corresponding edge point "e" and the pair point sandwich the reference point KT. In this case, it is assumed that "f" is selected as the pair point. Furthermore, an average coordinate of the preliminary corresponding edge point "e" and the pair point "f" is obtained.The obtained average coordinate is then treated as the real corresponding edge point coordinate.

[0252] In this way, the calculation of the corresponding edge point in the waveform can be suppressed to obtain a stable calculation result. Namely, when performing fine positioning between points, a large positional shift may occur between the pairs with a point in a higher wave and the pairs with a point in a lower wave, but such an influence can be reduced by the above technique. (Method for obtaining a coordinate of the corresponding edge point using a neighborhood edge point)

[0253] Furthermore, the method for finding a coordinate of a corresponding edge point in units of subpixels is not limited to the above, but another method may also be employed. For example, finding the coordinate can also be realized in the following manner. The corresponding edge point on the corresponding edge point search line is searched in units of pixels, and the obtained point is regarded as the preliminary corresponding edge point. A plurality of neighboring edge points around this preliminary corresponding edge point are found in units of pixels. The subpixel coordinates of the preliminary corresponding edge point and the plurality of neighboring edge points are found, and their average coordinate is then found.With this method, the position of the real corresponding edge point can be found using the preliminary corresponding edge point and the plurality of adjacent neighboring edge points, and therefore, the coordinate position of the corresponding edge point can be obtained with good accuracy in units of pixels. Furthermore, by not using the corresponding point search line but using a plurality of neighboring edge points appropriately selected around the preliminary corresponding edge point, the coordinate position of the corresponding edge point can be determined accurately and easily.

[0254] An edge point located on the same outline or segment around the preliminary corresponding edge point can be selected as a neighborhood edge point.

[0255] Furthermore, by using an edge angle of the preliminary corresponding edge point, a neighborhood edge point with a similar edge angle can be obtained. Preferably, with an edge angle direction of the preliminary corresponding edge point set at the center, edge points adjacent to the left and right are respectively selected as the neighborhood edge point. Furthermore, a distance from the preliminary corresponding edge point to the neighborhood edge point is desirably close, for example, within two pixels, preferably on the order of one pixel. This is because too large a distance causes deterioration of accuracy. The following is a procedure for obtaining the coordinate of the corresponding edge point using the neighborhood edge points based on the schematic view of Fig. 65 and a flow chart of Fig. 66 described.

[0256] First, in step S6601, the preliminary corresponding edge point on the corresponding point search line is searched in units of pixels. Fig. 65, a hatched white circle is the reference point KT, and a position corresponding to the segment forming the outline is found on the corresponding point search line TL passing through the reference point KT. The search is performed in units of pixels, namely at each intersection of grids in Fig. 65, which are considered as a reference, and in this case (x, y) = (2, 3) is selected in the pixel coordinate.

[0257] Next, in step S6602, the neighborhood edge point is selected in units of pixels around the provisional corresponding edge point in units of pixels. Since the edge angle of the provisional corresponding edge point has a vector in a direction superimposed on the corresponding point search line, edge points enclosing the corresponding point search line and located to the left and right thereof, as well as having edge strengths not smaller than a predetermined value, are respectively selected as the neighborhood edge points. Specifically, a first neighborhood edge point located on the right side and a second neighborhood edge point located on the left side are selected. In the example of Fig. 65, a pixel coordinate of the first neighborhood edge point (2, 2) and is a pixel coordinate of the second neighborhood edge point (1, 3). At this time, since the edge angles of the corresponding neighborhood edge points are edge points in the neighborhood of the preliminary corresponding edge point, it is highly likely that the neighborhood edge points have similar edge angles, and therefore, the edge angles do not need to be checked. Needless to say, it is possible to select those edge points as the neighborhood edge points after determining high similarity of the edge angles. Note that if there is no edge point with an edge strength not smaller than the predetermined value in the neighborhood position, no neighborhood edge point is selected.In this case, a real corresponding edge point mentioned later is calculated using only the obtained edge points (the preliminary corresponding edge point and another neighboring edge point). Furthermore, although a total of two neighboring edge points are selected on the right and left in this example, the number of neighboring edge points selected can also be one or not less than three. However, the number is preferably two, which takes into account the balance between accuracy and computational load.

[0258] Furthermore, in step S6603, the coordinate of the real corresponding edge point is calculated based on the preliminary corresponding edge point and the neighboring edge points. The coordinate position of the real corresponding edge point is set as the average coordinate of subpixel positions of the preliminary corresponding edge point and the neighboring edge points. In the example of Fig. 65, the corresponding edge point is decided from an average of three points: TP1 indicated by a black circle as the subpixel position of the preliminary corresponding edge point; TP2 indicated by a hatched circle as the subpixel position of the first neighboring edge point; and TP3 indicated by a cross-hatched circle as the subpixel position of the second neighboring edge point. The subpixel position of each edge point is calculated in advance from pixel values ​​around it. A known method can be applied to a method for calculating a subpixel position. For example, the subpixel position may be calculated from pixel values ​​of surrounding 3x3 pixels with each edge point set at the center thereof, or calculated using information on the adjacent edge point present in the edge angle direction, or by any other means.It should be noted that the timing for calculating the subpixel position of each edge point is not particularly limited, and the timing can be immediately after deciding the edge point in units of pixels or immediately before calculating the average coordinate.

[0259] In the manner described above, the corresponding edge point can be calculated from the average of the subpixel coordinates of the three edge points. With this method, the three edge points can be easily extracted from the preliminary corresponding edge point; it is possible to decide a corresponding edge point with high accuracy by using many edge points. For example, in the case where the number of reference points is ten, the number of corresponding edge points is normally ten. However, with the above method, the respective ten preliminary corresponding edge points can be added to points to the right and left of them, and thus the corresponding edge point can be calculated from 30 edge points, thereby improving the accuracy due to an averaging effect.In particular, compared with the above method of finding the average coordinate of two points that are a pair point and the preliminary corresponding edge point, this method is advantageous in terms of accuracy because the average coordinate is found from three points, which are the preliminary corresponding edge point and the two neighboring edge points that are added. Note that three points are obtained and averaged above, and the three points can be used individually in the fine positioning calculation. (Transformation of the edge angle image to the edge angle bit image)

[0260] Next, the transformation from the edge angle image to the edge angle bit image is performed based on the Fig. 30 to 33. When using the coarse-to-fine approach, it is not easy to adjust the reduced image in the first coarse search. This is because information on a characteristic amount necessary for the search may be lost due to image reduction. Particularly in the edge-based search, edge angle information is important for improving the accuracy of the search. Therefore, in the present embodiment, the edge angle bit image generating means 69 and the edge angle bit image reducing means 78 capable of maintaining the edge angle information even when the reduction ratio upon reduction of the image by the thinning means 61 is made high are provided to reduce a data amount while retaining a sufficient characteristic amount, thus attempting to speed up processing.

[0261] Next, a procedure for detecting edge angle information when reducing the image by the thinning device 61 and obtaining this information will be described. First, the edge strength of each edge point of the edge angle image is checked, and a bit corresponding to the orientation of the edge angle is set to one if the edge strength is greater than a set edge strength threshold, and set to zero if not. For example, consider a case where the edge angle bit image is generated by the edge angle bit image generation device 69 from the edge angle image of 2×2 pixels made up of the four pixels (edge ​​points) "a" to "d" shown in Fig. 13. Each of the pixels "a" to "d" has an edge strength greater than the threshold and also has an edge strength indicated by an arrow. This edge angle is expressed by eight types of edge angle bits, which are 0 to 7, corresponding to edge angle sections that represent the correspondence of the edge angle to the bit. (edge ​​angle bitmap)

[0262] When transforming this edge angle image into the edge angle bit image, the edge angle information is transformed into the edge angle bits. The edge angle bit is a code that sections the edge angle direction with respect to each predetermined angle. Fig. 31 shown edge angle section can be the same as that shown in Fig. 6 described above. In the example of Fig. 31, a concentration gradient direction is sectioned as the edge angle direction into eight sections of 45 degrees, and the edge angle bit is assigned to each of these sections. This example is not limiting. The sections may further be counterclockwise at 22.5 degrees from the position of Fig. 31 can be rotated to be eight sections offset by 22.5 degrees from the horizontal or vertical direction, and the corresponding edge angle sections can be marked clockwise with E, SE, S, SW, W, NW, N, NE from the right, each with a width of 45 degrees, and then edge angle bit flags 0, 1, 2, 3, 4, 5, 6, 7 can be added with respect to the corresponding edge angle sections (above Fig. 52B). Of course, sectioning is exemplary and, for example, the edge angle can be sectioned into sixteen sections, four sections, or can still be three sections or five sections.

[0263] From the above, if the Fig. 30 are transformed into a 2x2 edge angle bit image based on the edge angle sections of Fig. 31, the transformed image is shown as in Fig. 32. As thus described, the bits are set to edge angle sections corresponding to the respective edge angles with respect to the four edge points sectioned by the labels “a”, “b”, “c”, “d”.

[0264] As a method for acquiring an edge angle bit image, in addition to the technique of performing processing on only a region not smaller than a certain edge strength threshold as described above, there is also a method of performing thinning processing using an edge strength image and an edge angle image to acquire an edge angle bit image as described above using the edge angle image subjected to thinning processing to acquire an edge angle bit image with a certain degree of width. In the case of the technique of performing thinning processing, the processing time is relatively long compared with the above technique, but there is an advantage in facilitating noise elimination because the outline range of the object to be searched can be limited. (Reduction in the edge angle bitmap)

[0265] As described above, after expression using the edge angle bit image, the data is reduced so that the edge angle information is sufficiently preserved. Specifically, the data is synthesized by taking "OR" or a bit sum of each edge angle bit of each pixel with respect to each edge angle bit label. For example, in a case where the 2x2 data to be reduced is in the state of Fig. 32 are when reduced to 1 / 2 long × 1 / 2 side (= 1 / 4) to express the four pixels “a” to “d” by one pixel “a”, the reduced data as in Fig. 33. As shown in this drawing, the edge bit of each pixel is synthesized in the edge angle bit reduction image, where the edge angle bits of the pixels "a" to "d" are contracted, and the edge angle bits are arranged in columns corresponding to the edge angle bit labels 0 to 7. This processing for storing the edge angle information in the edge angle bit reduction image is performed by the edge angle bit image reduction device 78. Thereby, while the data amount is reduced, the edge angle information is retained even after the reduction, and it is thereby possible to sufficiently preserve characteristics for the search even if the reduction ratio increases by repeating the reduction.

[0266] The above compression processing can improve the conventional problematic situation where the search processing speed becomes insufficient when the reduction ratio is suppressed so as to preserve a sufficient amount of characteristics for the search. Even if the reduction ratio for performing edge detection is fixed to, for example, one-half, a sufficiently high-speed search can be performed by processing a reduction of the edge angle bit image at this reduction ratio. The reduction ratio for performing edge detection can be automatically set based on the size of the registered image and / or the characteristic data of the pattern model. Furthermore, the reduction ratio can be designed to be set independently by the user.

[0267] An example of the procedure for reducing the edge angle bit image to generate the edge angle bit reduction image is based on the Fig. 34 to 37. When generating the edge angle bit reduction image, attention should be paid to a segmentation problem. Simply performing reduction processing using subsampling processing may cause large variations in the rank calculation mentioned later due to a subtle offset of a processing start coordinate or a position in the input image of the object to be searched. As reduction methods that avoid this segmentation problem, the following two methods can be considered.

[0268] The first method is a method for performing expansion after reduction processing. This method is explained using an example of processing a reduction to 1 / n (n=2) using Fig. 34 and Fig. 35. First, the OR operation is performed on each rectangular area of Fig. 34 included n × n edge angle bit data is performed. A result of the operation is substituted as edge angle bit data representing the edge angle bit data of each of the above n × n regions. Performing this processing can reduce the image to 1 / n of the original image.

[0269] Since leaving this reduced image in this state can cause the segmentation problem to occur, an expansion is performed on this image. As shown in Fig. 35, the OR operation is performed on edge angle bit data in each of the m × m (in this example, m = 2) rectangular regions in the image after reduction, and its result is substituted as edge angle bit data representing each of the m × m regions. Image reduction does not occur in this processing. In this example, "m = 2," but it can be considered that "m" is increased according to expected variations in the shape and size of the object to be searched.

[0270] Another method is a method to take a rather wide range for the OR operation to be performed in the above reduction processing and not perform subsequent expansion. Using Fig. 36, this method is described, assuming the case of processing a reduction image 1 / n (n = 2). The OR operation is performed on edge angle bit data of (n + m) × (n + m) (n = 1, m = 1) stored in each rectangular area of Fig. 36 are included. A result of the operation is substituted as edge angle bit data representing each of the n×n edge angle bit data in the vicinity of the center of the above region. Performing this processing can reduce the image to 1 / n of the original image. Fig. 37 shows the case of n = 2 and m = 1 in the second method.

[0271] As in Fig. As shown in Figure 36, if a normal 2×2 reduction is repeated without expansion, a segmentation problem may arise. This is because a one-pixel change in the coordinates in the upper left corner of the search area causes a change in the pixel count during the reduction, and therefore, the ranking calculation deteriorates regardless of whether the registered image is identical to the image to be searched. In contrast, performing expansion has the advantage of not generating such a problem. (Edge angle bit transformation processing in angle boundary)

[0272] Furthermore, when transforming the edge angle bit, two bits corresponding to the two angle regions constituting an angle boundary are set when the edge angle is in the vicinity of the angle boundary, and therefore the effect of improving stability can be expected. For example, in the above edge angle bit image of Fig. 31, which is obtained by transforming the pixels “a” to “d” that are shown in Fig. 38, when the edge angle is in the vicinity of the boundary between E and SE, the edge angle bit may be set in section E, or the edge angle bit may be set in SE, depending on the noise. Such oscillation is expected to cause a non-essential effect on the calculation of coincidence. Therefore, when the edge angle is on the boundary, both edge angle sections sectionalizing the boundary are set to one. This can eliminate oscillation due to noise, and a stable calculation result of the coincidence can be expected. Specifically, when the edge angle is located within a predetermined width (e.g., 5.625 degrees) with the boundary of the edge angle sections set at the center, both edge angle bits pointing to the boundary are set to one.

[0273] Note that this edge angle bit transformation processing is performed at an angle boundary only when performing edge angle bit transformation processing on the object to be searched, and is not performed when performing edge angle bit transformation of the pattern model. This is because performing the same processing when performing edge angle transformation of the pattern also results in an unnatural change in the weight with respect to each edge point. (Edge angle adjacent processing)

[0274] Furthermore, although only one edge angle bit is set in the transformation from the edge angle image to the edge angle bit image in the above example, such edge angle bit adjacency processing can also be performed where a relevant edge angle section is set at the center, and an edge angle bit is also set in each of the adjacent edge angle sections. For example, one edge angle bit is assigned to the relevant edge angle section and to the right and left adjacent edge angle sections. Furthermore, such weighting is also possible that two edge angle bits are assigned to the relevant edge angle section, and one edge angle bit is assigned to each of the right and left adjacent edge angle sections.Furthermore, such weighting is also possible with an added fuzzy effect, where three bits are given if the edge angle of the pattern model and the edge angle of the image to be searched match sufficiently, one bit is given if the angles are slightly offset, and zero bit is given if the offset is large.

[0275] Even in such edge angle bit contiguity processing as described above, when the edge angle is in the vicinity of the boundary, the effect of oscillation due to noise may be considered. Therefore, in the case where the edge angle bit of the image to be searched is at the boundary of the edge angle sections, two bits are set in each of the adjacent edge angle sections with the boundary set at the center, thereby avoiding the effect of oscillation.

[0276] It should be noted that although eight bits are used as the angular resolution of the edge angle in the above example, this is not limiting and the transformation can also be performed by another, higher angular resolution, such as 16 bits or 32 bits. (parallelization)

[0277] Furthermore, by parallelizing a value obtained by transforming the edge angle of the pattern model into the edge angle bit, an acceleration of the search processing can be achieved. Fig. 38 and Fig. 39 show an example of parallelizing edge data of a field. Fig. 38 is a conceptual view of a sample model before parallelization and Fig. Figure 39 is a conceptual view of the pattern model after parallelization. As shown in these drawings, edge angle bit data on a reference point pattern model is laterally arranged multiple times for parallelization, thereby enabling processing acceleration. A general-purpose CPU constituting a computer section is capable of parallel processing of four to eight bits, thus enabling processing at four to eight times the speed. In this way, it is possible to accelerate coarse search through parallel processing. (Coarse search using edge angle bit reduction image)

[0278] The following describes a procedure for performing the coarse search using such reduction data. Regarding the edge angle bit reduction image, for example, if the size of the unenlarged image is 640×480 and edge extraction is performed at a size of 320×240, one-half of the above, the size of the edge angle bit image subjected to one-eighth compression processing is 40×30. The search on this edge angle bit reduction image is performed as follows. First, when registering the pattern model before the search processing, data as shown in Fig. 40A and Fig. 40B. As shown in each drawing, the pattern model is maintained as an array of edge data with position and angle information. Fig. 40A shows an example of the sample model and Fig. Figure 40B shows an example of the pattern model of the edge data. In Fig. 40B, the symbols X, Y denote the coordinate position of the edge, and the symbol θ denotes the angle of the edge. This pattern model is maintained as the array of edge data containing information about the coordinate position and angle, as shown in Fig. 40B shown.

[0279] Using the above data and the edge angle bitmap of the image to be searched, the position and pose of the pattern are repeatedly changed, and the coincidence is sequentially calculated with respect to each position and pose of the pattern. This calculation is performed as follows. First, an affine transformation value expressing a position and pose whose coincidences are to be checked is determined. This affine transformation value is also generated in light of the reduction scale of the pattern model and the reduction scale of the image to be searched. Using this affine transformation value, an edge position xi, yi, and an edge angle θi are transformed. The edge position after the transformation is Xi, Yi, and the edge angle after the transformation is ϕi (i is a subscript expressing an index of the edge).This coincidence is calculated by transforming the edge angle θi into bit data using the same method as for the image to be searched. A calculation expression for a coincidence S is expressed as follows: S=∑i(EABI(Xi,Yi)& AngleToBit(ϕi)!=0)∑i1 ImageEABI(x,y) The edge angle bit image of the X1, Y1 The position of the reference point after the affine transformation AngleToBit(θ) function for transforming edge angle data into bit data ϕi edge angle at the expected reference point after the affine transformation & AND processing ! If the left side is equal to the right side, then 0, otherwise 1 Sum of each reference point

[0280] As described above, the fine positioning device 76 compares the edge strengths and edge angles of the corresponding edges included in the image to be searched and the pattern model. A high coincidence S indicates a high probability of the pattern model being present in each position and pose. This improves the situation where the processing speed is insufficient when a search is performed as it is, and the reduction ratio is decided so that a sufficient amount of characteristic is retained to perform the search. Furthermore, since a sufficiently high-speed search can be performed even with a reduction ratio for performing edge extraction fixed at half, it is possible to obtain the advantage of not having to decide on the reduction ratio for performing edge extraction, which is typically complex. (Reduction step for image to be searched)

[0281] With such reduction processing, sufficient characteristics for searching can be retained even at a high reduction ratio. When performing edge extraction processing such as the Sobel filter on the image to be searched, a lot of noise is generated if the unmagnified image of the original size remains unchanged, which may be inappropriate for extracting characteristics for coarse searching. Therefore, in the present embodiment, the reduced image is generated in advance and then edge extraction processing is performed. Thereby, the averaging effect can be obtained by reducing the image data, which also contributes to noise reduction. The reduction ratio as a first reduction ratio (first reduction ratio) is set to one-half of the original size.At this size, a noise reduction effect can be obtained by averaging while sufficiently preserving characteristics necessary for the search. (Polarity of edge direction)

[0282] Furthermore, this method allows the presence or absence of a polarity for the edge direction to be set. This allows the method for processing the edge angle to be modified based on the polarity.

[0283] Conventionally, such a concept of polarity has not been considered, but only the concept of edge direction (angle) has been considered. Therefore, there has been a problem that, for example, the edge direction (angle) is treated as 0 to 180 degrees, resulting in the inability to distinguish vectors of different orientations, causing the occurrence of a false search. In contrast, the above method covers 0 to 360 degrees by considering the concept of polarity, so that a more accurate search can be realized.

[0284] If polarity is ignored, the edge angle bit transformation can be easily implemented on the search object side by simultaneously processing the reverse bit. Otherwise, a bit can be uniformly assigned not only to the orientation but also to the edge direction. For example, in a coarse search, eight edge angle bits are uniformly assigned to the edge direction as the edge resolution, so that a search result can be obtained where the importance is assigned not to the edge polarity, but to the edge direction resolution. (Fine positioning regarding the angle of rotation)

[0285] Next, a procedure for performing fine positioning on a circular arc segment by the least square method at a distance between a point and a circular arc, which is regarded as an error angle, based on the Fig. 41 to 45. Consider the case of performing fine positioning on the image to be searched using a pattern as the pattern model PM, where a partially notched compass is expressed by a circular arc segment and a line segment as shown by a thick line in Fig. 41. The sample model PM from Fig. 41 is configured as a circular arc segment and a line segment, and such a work shape has been generally applied to a wafer provided with an orientation-flat surface and the like. Fig. 42 shows a state where the coarse search has been performed on the image to be searched using this pattern model PM to perform a certain level of positioning with the pattern model PM located at a position of a detection candidate. Consider a state where the pattern model PM is almost superimposed on the edge point of the image to be searched, and only the line segment of the notched portion does not match. Note that, in these drawings, the thick line indicates the pattern model PM, and a thin line indicates the edge point (input edge point IE) of the image to be searched. Further, a target point of the input edge point IE is indicated by the dashed arrow, and a tangent line SL to this input edge point is indicated by a broken line.It should be noted that although an actual tangent line SL is shorter and in the state where it is superimposed on the thin line pattern model PM, the signal line is shown longer in these drawings for convenience of description.

[0286] When the fine positioning is performed from this state, it can be expected that the pattern model is relatively rotated so that the line segment parts are allowed to fit together, namely in Fig. 42 the error does not necessarily increase due to the rotation, even at the input edge point indicated by the arrow in Fig. 42 displayed position. The least squares method, in which not the distance between the point and the straight line but the distance between the point and the circular arc is regarded as the error function, is applied to the circular arc segment. Specifically, the least squares method is applied such that an absolute value, a difference between a radius of the circular pattern model and a radius of the edge point of the image with the center of the circular pattern model to be searched as the circular arc, is regarded as the error value. Namely, the error function of the circular arc segment used in the least squares method is a difference between an ideal radius including the center of the circular arc segment and a distance between the center of the circular arc and the corresponding edge point.The error function of the circular arc segment can be expressed by the following expression:. e=abs(Rideal−(x−xc)2+(y−yc)2) R ideal an ideal radius (x c ,y c ) Center coordinate of a circular arc (x,y) corresponding edge point e=abs((x−xc)2+(y−yc)2−R2ideal) R ideal an ideal radius (x c, y c ) Center coordinate of a circular arc (x,y) corresponding edge point

[0287] Consequently, as in Fig. 43, the distance between the input edge point of the arrow and the circular arc model does not change much. This means that a counterclockwise rotation of the pattern model PM, as in Fig. 43, as a solution can be generated to a sufficient extent. It is therefore possible to expect the achievement of high angle accuracy on the circular work in the fine positioning performed on a small number of cases. It should be noted that, as in the examples of Fig. 42 and Fig. 43 the circular area indicated by the dashed line is in practice superimposed on the thin line of the sample model, but in these drawings is slightly offset and shown for convenience of description.

[0288] Meanwhile, in the conventional least squares method, the distance between the input edge point and the tangent line was considered as the error function, and the pattern model was moved to shorten this distance. Therefore, it can be considered that the pattern model was not rotated in a correct rotation direction, and this therefore has the opposite effect of increasing positional displacement. For example, there have been cases where rotating the pattern model PM counterclockwise, as in Fig. 48, compared to the state of Fig. 47 cannot be expected. If the sample model PM as in Fig. 48 from the state of Fig. 47, where the relationship between the input edge point and the tangent line is formed at a position indicated by an arrow, the rotation is performed in a direction in which the input edge point detaches from the tangent line, resulting in such a rotation not being able to be generated as a solution. In contrast, in the present embodiment, since the above least squares method is applied with the distance between the point and the circular arc regarded as an error function, it is possible to expect the achievement of high angle accuracy on the circular work when fine positioning is performed several times. (Corresponding edge point generation processing)

[0289] Fig. 44 shows an example of further execution of generation processing of a corresponding point search line on the sample model of Fig. 41. As shown, a reference point is assigned at the center of each arc segment and line segment, and reference points are set from each of those points at fixed intervals, except for the vicinity of the end edges of the segments. Furthermore, corresponding point search lines are set from the inside to the outside in the direction of the normal to the segments of the corresponding reference points. Fig. 45 shows a state where the coarse search has been performed on the image to be searched using the pattern model, and the pattern model PM has been superimposed on the position and posture of the specified detection candidate.

[0290] In this state, edge extraction is performed along each corresponding point search line to search the corresponding edge point with respect to each segment. Since the corresponding edge point is a point where each corresponding point search line and the edge point (thin line) of the image to be searched intersect, for example, in Fig. 45, points indicated by × are the corresponding edge points. In the example of Fig. 45, Since the circular arc segments of the pattern model arranged at the position of the detection candidate, obtained as a result of the coarse search, and the input edge point almost match, the corresponding edge point of the circular arc segment and the circular arc segment almost match. Therefore, even if the circular work is relatively rotated, the error function of the circular arc does not increase, and therefore this degree of freedom does not inhibit rotation. Therefore, rotational motion can be expected as a solution.

[0291] On the other hand, many of the corresponding edge points of the line segment are not on the line segment. In the least squares method to be performed on the line segment, since the distance between the point and the straight line is considered the error function, it is estimated that a counterclockwise rotation will reduce the error function value overall. It is therefore found that, to a sufficient degree, a counterclockwise rotation can be expected to be the solution obtained by the least squares method. (Weight processing in the case of a plurality of corresponding edge point candidates)

[0292] Furthermore, when there are a plurality of corresponding edge point candidates corresponding to the reference point, the weighting can be performed on each corresponding edge point to improve the accuracy of fine positioning in a case where the corresponding edge points are ambivalently determined. This condition is determined based on the Fig. 49A to 49D. An example of performing fine positioning is considered which uses as a pattern model the pattern model PM which is a rectangle with two longitudinal lines drawn therein as shown by the thick lines in Fig. 49A. In this case, it is assumed that the segments SG1 and SG2, as indicated by thin lines in Fig. 49B, which can be obtained during the coarse search. In Fig. 49B, only the two segments SG1, SG2 located to the right and left of the work are considered. Corresponding point search lines TTL1, TTL2 of these segments are set in directions to the normals (edge ​​direction) to corresponding segments SG1, SG2 passing through the reference points KT1, KT2, as described above. When the corresponding point search lines TTL1, TTL2 are set at the corresponding reference points KT1, KT2, as indicated by the dashed lines with respect to the respective segments SG1, SG2, corresponding edge point candidates located respectively at the corresponding point search lines TTL1, TTL2 can be obtained. While only one corresponding edge point candidate TTA is obtained with respect to the right-side segment SG1, two corresponding edge point candidates TTB, TTC are obtained with respect to the left-side segment SG2.Since a corresponding edge point closest to the reference point is selected from the candidate corresponding edge points and set as the corresponding edge point, TTA and TTB both become the corresponding edge point. In the example of . Fig. 49B, since the majority of corresponding edge point candidates are related to the left-side segment SG2, it can be assumed that an ambiguity is involved. In this case, the issue is in which direction the segment, namely the pattern model, should be moved. In particular, in the case of Fig. 49B, when the overall segment, namely the pattern model, moves to the right as a result of selecting TTB as the corresponding edge point, it moves in the opposite direction to the desired fine positioning, which is not preferred.

[0293] Therefore, when performing the least squares calculation, each corresponding edge point is weighted. In the example of Fig. 49B, since only one corresponding edge point TTA exists on the corresponding point search line TTL1 of the reference point KT1 with respect to the right-side segment SG1, a weight of "1.0" is given. On the other hand, with respect to the left-side segment SG2, the corresponding edge point candidates TTB, TTC are located to the right and left of the reference point KT2 sandwiched therebetween on the corresponding point search line TTL2 of the reference point KT2. Therefore, the weight of the corresponding edge point TTB with respect to the reference point KT2 is set according to the distance from the reference point KT2 to each corresponding edge point candidate. As an example of an expression for determining the weight, as in Fig. 49D, when a distance between a first corresponding edge point candidate and the reference point is d1, and a distance between a second corresponding edge point candidate and the reference point is d2 (d1 ≤ d2), Weighting W=1−α(d1 / d2) where 0 < α < 1.

[0294] In the above expression, W = 1 in the case where the number of corresponding edge point candidates is one, and W is smaller in the case where the number is greater than one. As thus described, by increasing the weight in the case where the number of corresponding edge point candidates is one, only in the case of no smearing, movement in a more probable direction can be expected. Furthermore, in the case where the number of corresponding edge point candidates is more than one, while the nearest corresponding edge point candidate is regarded as the corresponding edge point, the positional relationship between the corresponding edge point and the corresponding edge point candidate is the above expression W = 1 - α (shorter distance) / (longer distance) when an ideal point is sandwiched, and "W = 1" when the ideal point is not sandwiched.After such a weighting operation and weighting, the segment movement direction is determined by the fine positioning device 76 included in the computer device 6 of . Fig. 1 as a weighting calculation device. (ranking calculation)

[0295] Furthermore, when performing fine positioning calculations using the least squares method, a rank indicating similarity can also be calculated. Namely, the least squares method is applied to the corresponding point search line, and the rank is calculated by the ratio between the number of corresponding points and the number of reference points in the final least squares processing. In a simplified manner, a value obtained by subtracting the number of corresponding points from the number of reference points is regarded as the similarity, and it can be found by the following expression: S=∑i=1n1∑i=1m1=nm S Rank n number of corresponding points m number of reference points

[0296] Furthermore, the similarity between the ideal edge angle of the reference point and the crank angle of the corresponding edge point can also be reflected in the rank. In this case, the ratio of the total weights obtained from the difference between the edge angle of the corresponding edge point in the final least-squares processing and the edge angle of the corresponding reference point, and the number of reference points, is calculated as the rank. Specifically, it can be calculated by the following expression: S=∑i=1nω(|θt−θip|)∑i=1m1 S Rank n number of corresponding points m number of reference points ω(x) function which is 1 when x=0 and decreases monotonically with increasing x θi edge angle corresponding point θip Angle of the reference point corresponding to the corresponding point (ideal angle of the reference point)

[0297] The ideal edge angle of the reference point is the direction of the normal to the line when the segment is a line, and is superimposed on the corresponding point search line. In this method, a setting is made such that the smaller the difference between the edge angle of each corresponding edge point and the ideal edge angle of the reference point corresponding to the corresponding edge point, the closer the weight approaches 1, and conversely, the larger the angle difference, the closer the weight approaches 0. For example, the weight is set to zero in the case where the angle difference is 0 to 18 degrees, to 0.1, ... in the case where the angle difference is 18 to 36 degrees, and to 1 in the case where the angle difference is 162 to 180 degrees. In this way, a weight is obtained with respect to each reference point, and finally obtained weights are averaged to calculate the rank.

[0298] In the example of Fig. 49B, a weight of 1 is given to the right-side corresponding edge point A, and a weight of 0.2 is given to the left-side corresponding edge point B. This results in an action to move the pattern model to the left side being performed on the right-side segment SG1, and an action to move the pattern model to the right side being performed on the right-side segment SG2. Therefore, taking this together, the weight intended for the movement to the left side is 1.0, and the weight intended for the movement to the right side is 0.9, and consequently, the pattern model moves to the left side to enter a state as in Fig. 49C. Similarly, the state of Fig. 49C, weighting is performed, and motion processing is repeated based on the weighting result to determine final fine positioning. As thus described, the direction in which the pattern model is to be moved during positioning is estimated for weighting according to the distance between the corresponding edge point candidate and the reference point, and then positioning is performed. Therefore, it is possible to move the pattern model in a relatively probable direction, thus expecting an improvement in positioning reliability and stability. (Method for selecting a segment considering the positioning direction)

[0299] Such a case as above can occur especially when many line segments are concentrated in a specific direction. In the example of Fig. 49B, the segments are present only in the longitudinal direction (Y-axis direction), and therefore, accurate positioning can be expected in the X-axis direction. On the other hand, since there is no line segment extending in the lateral direction (X-axis direction), the Y-axis direction cannot be defined, which makes positioning in this direction ambiguous. Therefore, when selecting segments that constitute the pattern model, segments in the orthogonal relationship, such as in the X-axis direction and the Y-axis direction, are deliberately selected to prevent the positioning directions from being concentrated in a specific direction, so that a stable positioning result can be expected. The following describes a method for selecting a segment in consideration of a positioning direction based on a flowchart of Fig. 50 described.

[0300] First, in step S4901, in a state where a plurality of segment candidates have been obtained, the segment candidates are sorted in order of length. Note that in the case of the circular arc segment, the length of the circular arc is regarded as the segment length.

[0301] Next, in step S4092, the longest segment candidate is selected as the segment and also set as a reference point segment. The normal direction to this reference segment is considered a reference angle. Note that when a circular arc segment is selected, the reference angle becomes ineffective. If the reference angle is ineffective, a conjugate segment is selected not by the angle but only by the length of each segment candidate.

[0302] Furthermore, in step S4903, the segment candidates are extracted as the conjugate segment with respect to the reference segment. Starting from the reference angle, it is searched whether the segment candidate contained within a predetermined angle range, in this case, a first angle range, exists or not. Fig. 51A shows an example of the first angle range. The segment candidate included within the range of ±45 degrees to the direction of the normal to the reference segment (90 degrees) set at the center, namely, the range from 45 to 135 degrees, a total of 90 degrees, is extracted.

[0303] In the example of Fig. 51B, segment candidates marked with "◯" thereon are extracted, and segment candidates marked with "×" thereon are eliminated. When the line segment is selected as the conjugate segment, the reference angle becomes the direction of the normal to that segment and comes into an effective state. If the segment is not a line but a circular arc segment, it is extracted unconditionally. This is because, in the case of the circular arc segment, an angle change is expected to be large and can therefore be useful information. Furthermore, in the case of a circular arc, the state of the reference angle remains unchanged.

[0304] When the segment candidates are extracted, the process proceeds to step S4901-1, the longest segment candidate from the extracted segment candidates is selected as the segment and is also set as the conjugate segment with respect to the reference segment. Further, in step S4905, it is determined whether the number of already selected segments has reached a predetermined number or not. If it has reached the predetermined number, the processing is terminated. If it has not reached the predetermined number, the process proceeds to step S4906, and a new reference segment is again set as the conjugate segment. After that, the process returns to step S4903 to repeat the processing.It should be noted that in the case where a circular arc segment is selected as the conjugate segment and the circular arc segment is regarded as the reference segment, the selection of the conjugate segment is not made according to the angle but only the length as described above when the reference angle is in the ineffective state, and the extraction of the conjugate segment is performed by performing the same processing as shown above when the reference angle is in the effective state.

[0305] Meanwhile, if there is no segment candidate included within the first angular range in step S4903, the process proceeds to step S4904 and searches for whether or not there is a segment candidate included within a second angular range that is extended from the first angular range in the same manner as above. In the example of Fig. 51A, the range of 40 to 140 degrees, which is extended by ± 5 degrees from the first angle range, is set as an example of the second angle range. When the segment candidate is found, the process jumps to step S4904-1, and in the same manner as above, the longest segment is selected and set as the conjugate segment.

[0306] If no segment candidate is found even within the second angular range, in step S4904-3, it is further searched in the same manner as above whether or not a segment candidate included within a third angular range that is further expanded than the second angular range exists. In the example of Fig. 51A, the range of 35 to 145 degrees, which is further expanded by ±5 degrees from the second angle range, is set as an example of the third angle range. When the segment candidate is found, the process jumps to step S4904-1 in the same manner as above, and the longest segment is selected and set as the conjugate segment. When no segment candidate is found, the process goes to step S4902, and the longest segment among the segment candidates is reselected as the reference segment. It should be noted that the numerical value of the angle range, the number of times the angle range is reset, and the like can be appropriately changed. For example, when no segment is found in step S4904-3, a search may be performed in a further expanded angle range.On the other hand, step S4904-3 may not be performed, and if no segment is found in step S4904-2, the process may immediately return to step S4902 to reset the reference segment.

[0307] As described above, because the operation of selecting the conjugate segment close to the orthogonal direction to the reference segment is repeated, and as a result, the segment with a dispersed angle is selected, the stability of positioning can be improved. The segment is selected in this manner by the segment selection device 67 of the segment generation device 68. (Pattern characteristic selection function)

[0308] Furthermore, a pattern characteristic selection function may also be provided, which is capable of changing selection criteria for the segment constituting the pattern model based on characteristics of the pattern obtained from the object to be searched. Specifically, it is particularly effective in a registered image, as shown in Fig. 67. In the registered image in Fig. 67, different letters and numbers are displayed in grid frames. If the pattern window PW is set to such an image, as in Fig. As shown in Figure 67, many SGW segments are placed in frame regions, while few SGM segments are placed in the letters within the frames in the generated pattern model. In such a pattern model, positioning is performed only in the frame regions, resulting in the letters within the frames being ignored or not given any importance, making it difficult to recognize the letters, and therefore positioning may fail due to positional offset in units of frames or the like.

[0309] This is caused by the selection criteria for the segment. Namely, a long segment has been preferentially selected from the conventional viewpoint that eliminating the noise component and assigning importance to a more clearly detected edge point leads to an improvement in positioning accuracy. This is because a short outline is considered to have a large number of noise components, and based on the assumption that a longer line segment, in contrast, extracts more accurate edge information, a setting has been made to automatically select the segment from long line segments. In other words, there has been no image processing method and the like to make a short line segment preferentially selectable. Therefore, in the example of Fig. 67, since edge detection tends to be relatively simple and clear in the frame area surrounded by straight lines, the segment generated in the frame area is more likely to be selected, resulting in positioning failure as described above. Especially in the coarse search, since it is a simple search and thus not all outline information about extracted edges, chain segments, and the like is used, but only a part of the outline is selected, if the preferentially selected outline does not contribute to accurate positioning, the problem of such erroneous selection occurs.

[0310] In contrast, in the present embodiment, a function capable of selecting the outline in ascending order of length from the shorter outline is set to obtain a suitable search result in the registered image. Furthermore, the elimination of the noise component is realized by setting a threshold and eliminating a line segment with a length not greater than the predetermined length. Consequently, while effectively eliminating the noise component, a highly reliable search result can be obtained. (Sorted in order of outline length)

[0311] Next, two methods for selecting a short outline are described. First, a method for sorting outlines in order of length and selecting a predetermined number of outlines starting from a short one is described based on a user interface screen of Fig. 68. Fig. 68 is an image view showing a user interface of an image processing program setting screen 200 of the pattern characteristic selection function for appropriately selecting an outline constituting a pattern model of a registered image. On this screen, the user can make corresponding settings for coarse search and fine positioning with respect to, respectively, as setting objects, a lower edge strength limit 82, a lower outline length limit 83, a number of selected outlines 84, and registration 85 of the order of outlines. Of these, the objects related to the pattern characteristic selection function are the lower outline length limit 83, the number of selected outlines 84, and registration 85 of the order of outlines.

[0312] A range for detecting the edge is defined by the upper edge strength limit 81 and the lower edge strength limit 82, and such a filter condition to eliminate an edge strength higher than the upper limit or lower than the lower limit is called. (Outline length lower limit adjustment device)

[0313] The outline length lower limit 83 functions as an outline length lower limit setting device for setting a lower limit for detection as the outline. Namely, an outline shorter than a lower limit defined by the outline length lower limit 83 is filtered by a length filter. This can eliminate noise appearing as the short outline. Furthermore, if this value is made adjustable by the user, the filtering strength can be appropriately adjusted based on a used environment and application of the filtering. Furthermore, the outline length lower limit can be a fixed value that depends on the environment. (Selection number decision device)

[0314] The number of selected outlines 84 functions as a selection number decision device for defining the number of selected outlines. Defining the upper limit of the number of outlines used as the pattern model can simplify the pattern model to reduce the processing amount, thereby aiming for a reduction in search time. Further, as the number of selections increases, the processing amount increases, but on the other hand, a highly accurate search result can be expected. Note that setting the number of selected outlines to a fixed value (e.g., 50) can aim for simplification of the setting operation as described above. (Selection order decision device)

[0315] The outline order register 85 functions as a selection order decision device capable of switching the order of selecting outlines between ascending order and descending order of outline length. Thus, according to an image as an object for image processing, an appropriate selection method of an ascending order or descending order of length is set, so that image processing can be performed with greater flexibility and higher accuracy.

[0316] By setting the above setting items, such an extremely short outline close to the noise is filtered out from the plurality of outlines, and the outlines are made sequentially selectable from the shorter ones to effectively eliminate the noise component and also appropriately select a segment exerting an influence on the positioning accuracy, so that an efficient pattern model can be constructed.

[0317] For example, an example of selecting a segment constituting a pattern model by setting the pattern window PW to a registered image where different letters and numbers are displayed in grid frames is considered, as in Fig. 69. In the case of the Fig. 70, because the outline order registration 85 is set to "descending order of length", namely, it is set to select the outlines from the longer to the shorter. This way, as shown in Fig. 69, many of the segments SGW in the frame areas are undesirably selected and few of the segments SGM in the areas of letters and numbers which are important for identification are selected and accuracy in positioning cannot be expected if this condition remains unchanged.

[0318] In contrast, as in Fig. 71, the setting in the registration 85 of the order of the outlines is changed to “ascending order of length”, namely the order is changed so that the outlines are selected sequentially from the shorter to the longer, whereby many of the segments SGM of the letters and numbers within the frame are now selected, as in Fig. 72, so that a pattern model containing outline information suitable for the registered image can be constructed.

[0319] It should be noted that although each setting object is individually positioned in the coarse search and the fine positioning in the example of Fig. 68 is adjustable, it can be made adjustable in both, or it can be constituted such that a specific object is defined by the image processing program or the image processing device side, and setting by the user is prohibited. Reducing the number of setting items to allow a user who is not particularly familiar with the operation to use the device in a simplified manner can improve operability.

[0320] Furthermore, regarding the outline, outline information such as a chain or the like can also be used besides the segment. For example, in the case of using the chains obtained by the chain generating device 63 as they are as outline information without approximating the chains to the line segment or circular arc segment, a technique for selecting the chains from the short chain as described above, a technique for establishing the selection order such that it is switchable between ascending order and descending order, or a technique for eliminating a chain that is shorter or longer than a predetermined threshold can be applied to the selection criteria of the chain, and this can also allow a similar action effect to be obtained. Next, a specific procedure for sorting the outlines in the order of outline length is described based on flowcharts of Fig. 73 and Fig. 74 described.

[0321] First, the case of performing the sort using a segment length based on Fig. 73. First, an outline is extracted. First, in step S7301, the registered image is Sobel-filtered to find an edge angle image and an edge strength image. Next, in step S7302, an edge point is thinned using the edge angle image and the edge strength image to find an outline point. Specifically, after the edge angle image and the edge strength image are generated by the edge angle / edge strength image generation device 60 of the outline extraction device 62, the edge point is thinned by the thinning device 61 using edge strength non-maximum point suppression processing. Further, in step S7303, a chain is generated by the chain generation device 63. Specifically, the edge chaining device 64 links adjacent edge points to generate a chain.Furthermore, the filtering is performed by the chain filtering device 66 with a plurality of characteristic amounts as appropriate.

[0322] Next, in step S7304, a segment is generated. The edge chain segmentation device 65 of the segment generation device 68 generates a segment obtained by approximating each chain by line and / or circular arc. Further, in step S7305, the short segment is filtered. As a contour length lower limit setting device, the segment selection device 67 eliminates a segment with a length not greater than a lower limit value to eliminate the noise component.

[0323] Finally, in step S7306, the segments are sorted by segment length to select the segments constituting a pattern model sequentially from the segments having a shorter length. The segment selection device 67 functions as an outline sorting device for sorting the outlines in descending order of length, sorts the segments in order of length, and further selects the segments in a number defined by the selection number decision device in a selection order defined by the selection order decision device in ascending order of segment lengths, namely sequentially from the shorter ones. Thereby, a pattern model is constructed which is suitable for the registered image as shown in Fig. 62 described above.

[0324] On the other hand, an example of using the chains as they are not is shown by the segment as the outline based on a flowchart of Fig. 74. A procedure for extracting the outline point to create the chain is the same as Fig. 73 above. Namely, in step S7401, the registered image is Sobel filtered to find an edge angle image and an edge strength image. Next, in step S7402, an edge point is thinned using the edge angle image and the edge strength image to find a contour point. Further, in step S7403, a chain is generated.

[0325] Then, in step S7404, the short chain is filtered. According to a lower limit set by the outline length lower limit setting means, the chain filtering means 66 of the chain generating means 63 eliminates the chain shorter than the lower outline length limit 83. Subsequently, in step S7405, the chains are sorted in order of chain length to select those chains that constitute a pattern model, sequentially from the chain with a shorter length. In this case too, the chain filtering function 66 functions as an outline sorting function, sorting the chains in order of length and further selecting the chains in a number defined by the selection number decision means in a selection order defined by the selection order decision means in ascending order of chain length. Thereby, a pattern model suitable for the registered image is generated as shown in Fig. 72 constructed.

[0326] With this method, the processing of performing a single-segment approximation is not required, and therefore, the processing can be simplified accordingly. On the other hand, since the chain is a body of coupled random line segments that are not approximated to a fixed geometric graph, such as a line or circular arc, each subsequent processing becomes complicated. The method selected is decided based on whether the registered image is a simple graph, the detection accuracy of the edge point, and the like. (Filter long outline)

[0327] It should be noted that although the sorting in the above example is performed in order of outline length, the pattern model can be constructed by eliminating the long outline without performing any sorting. Below, this procedure is demonstrated based on a user interface screen of Fig. 75 described. Fig. 75 is also an image view showing a user interface of an image processing program setting screen 300 for setting the pattern characteristic selection function for the registered image in the image processing program. On this screen, in addition to the upper edge strength limit 81, the lower edge strength limit 82, and the lower outline length limit 83, an upper outline length limit 86 is given. The upper edge strength limit 81, the lower edge strength limit 82, and the lower outline length limit 83 are similar to those shown in Fig. 68 described above and their detailed descriptions are not repeated here. (Outline length upper limit adjustment device)

[0328] The upper outline length limit 86 functions as an outline length upper limit setting device for setting an upper limit of an outline. Namely, the upper outline length limit 86 filters the longer outline than the upper limit defined by the upper outline length limit 86. This makes it possible to construct a pattern model by intentionally eliminating the long outline while leaving only the short outline, thus consistently obtaining an effect similar to the case of preferentially selecting the short outline.

[0329] It should be noted that in the case of Fig. 75 the selection number decision device is not provided, and contours in a number of a preset defined value are automatically selected. However, the selection number decision device may also be provided so as to allow the user to manually set the number of contours.

[0330] As described above, without sorting the outlines in order of their length, a pattern model can be constructed in which a short outline is preferentially selected, and a pattern search can be realized which is also carried out on the registered image as in Fig. 67 is effective. For example, in such setting conditions as in Fig. 76, the sample model many of the segments SGW that are selected in the frame areas, as in Fig. 77. However, by changing the upper outline length limit 86 from 100 to 20, as in Fig. 78, a sample model can be constructed as shown in Fig. 79, which includes many of the segments SGM of the letters or numbers within the frames.

[0331] A procedure for filtering a long outline is developed based on the flowcharts of Fig. 80 and Fig. 81. First, Fig. 80 illustrates the case where the segment is used as the outline. In this method, as in the above case of sorting the outlines by the outline length, outline points are extracted to generate a chain. Namely, in step S8001, the registered image is Sobel-filtered to find an edge angle image and an edge strength image. Next, in step S8002, an edge point is thinned using the edge angle image and the edge strength image to find an outline point. Further, in step S8003, a chain is generated, and subsequently, in step S8004, a segment is generated.

[0332] Finally, in step S8005, the long segment and the short segment are deleted. In addition to the segment selection device 67 eliminating the segment having a length not greater than the lower limit as the outline length lower limit setting device, the segment selection device 67 further serves as an outline length upper limit setting device to delete segments having a length greater than the outline length upper limit 86. Consequently, since a short segment can be selected after eliminating the noise component, a pattern model can be constructed that includes a segment suitable for positioning in such a case as Fig. 79 is effective. Although the number of values ​​to be selected is a fixed value as described above, the selection number decision device may be provided separately to allow the user to make a setting manually.

[0333] Next, an example of constructing a pattern model using chains instead of segments based on Fig. 81. Also in this case, a procedure from the extraction of the outline point to the creation of the chain is similar to that described above by Fig. 74. Namely, in step S8101, the registered image is Sobel filtered to find an edge angle image and an edge strength image. In step S8102, an edge point is thinned using the edge angle image and the edge strength image to find a contour point. In step S8103, a chain is generated.

[0334] Then, in step S8104, the long chain and the short chain are deleted. The chain filtering device 66 eliminates the chain having a length not greater than a lower outline length limit as the outline length lower limit setting device and also serves as the outline length upper limit setting device to delete chains longer than the outline length upper limit 86.

[0335] Consequently, since the short chain can be selected after eliminating the noise component, a pattern model can be constructed which includes a chain used to position the registered image in such a case as Fig. 79 is effective. Although the number of chains to be selected is a fixed value as described above, the selection number decision device may be provided separately to allow the user to make a setting manually. (Combination of segment selection functions taking segment direction into account)

[0336] The pattern characteristic selection function can be used simultaneously with the above segment selection function, considering the positioning direction or angle. Namely, as a technique for deciding which segment is selected to construct a pattern model after sorting the segments in order of length and filtering a long segment, it is possible to adopt a method for selecting a conjugate segment that is close to the orthogonal direction to a reference segment as a method represented by a flowchart of Fig. 50 and the like. Note that this method can be used to select the segment, but cannot be used to select the chain. This is because the chain is not adjusted by line and / or circular arc, and thus does not have an angle or direction, as is the case with the segment.

[0337] An example of considering the direction of the normal to the segment when selecting the segment after sorting or filtering is shown below based on flowcharts of Fig. 82 and Fig. 83. First, Fig. 82 an example of sorting the segments by segment length. In this case too, a procedure for extracting the outline point for generating the chain and the segment for filtering the short one from the obtained segments for sorting the segments is similar to the above procedure of Fig. 73. Namely, first, in step S8201, the registered image is Sobel-filtered to find an edge angle image and an edge strength image. Next, in step S8202, an edge point is thinned using the edge angle image and the edge strength image to find an outline point. Further, in step S8203, a chain is generated, and then, in step S8204, a segment is generated. Subsequently, in step S8205, the short segment is filtered, and then, in step S8206, segments after filtering are sorted by segment length and arranged in ascending order of length.

[0338] In this state, in step S8207, the segments are selected sequentially from the shorter ones while considering the direction of the normal of each segment. A specific procedure after step S8206 of Fig. 82 is in a flowchart of Fig. 84. This flowchart is almost similar to the flowchart of Fig. 50, but differs from it by: setting the sorting order not in descending order of segment length, but in ascending order from the shortest in step S8401; in step S8402, selecting the shortest segment as a segment candidate and considering it as a reference segment; and in step S8404-1, selecting the shortest segment as a segment candidate and considering it as a conjugate segment. Everything other than the above is similar to Fig. 50 and its detailed descriptions are therefore not repeated.

[0339] This procedure is particularly effectively performed on the registered image where the short segment is effective and substantially orthogonal segments are alternately selected to construct a pattern model where segments have been deliberately selected in such a relation that normal angles are orthogonal, whereby a result of stable positioning in longitudinal and lateral directions can be expected.

[0340] Similarly, an example of considering the direction of the normal to the segment in the method of filtering a segment with a long length is given based on a flowchart of Fig. 83. In this case too, a procedure from the extraction of the outline point to the creation of the chain and the segment to the filtering of the short and the long is similar to the above procedure of Fig. 80 and the like. Namely, in step S8301, the registered image is Sobel-filtered to find an edge angle image and an edge strength image. Next, in step S8302, an edge point is thinned using the edge angle image and the edge strength image to find an outline point. Further, in step S8303, a chain is generated, and then, in step S8304, a segment is generated. Subsequently, in step S8305, the short segment and the long segment are filtered. Since an operation of sorting the segments by segment length in step S8306 is required at any rate in this case, the above method of Fig. 82 can be considered more efficient from this point of view. The other effects are similar to those in Fig. 82 and as a result of selecting a conjugate segment close to the orthogonal direction, the segment with a dispersed angle is selected to be the pattern model, thereby allowing an improvement in the stability of positioning. (Improvement in the stability of the least squares method)

[0341] Next, a technique for improving the stability of the least squares method performed in the fine positioning step will be described. The least squares method is briefly categorized into a linear method and a non-linear method. In these methods, a solution can theoretically be obtained uniformly in the linear least squares method. On the other hand, in the non-linear least squares method, the approximation is typically performed up to a quadratic expression, and therefore the approximated value is not necessarily accurate. In some cases, in the fine positioning step, the position to be detected may be moved or rotated in a direction to make the accuracy lower than the position obtained in the coarse search. For example, in a case of performing fine positioning on a graphic with high symmetry, such as a circular shape, Fig. 85 (a center coordinate of an outer circle is subtly different from a center coordinate of an inner circle), since an error value hardly ever changes, even if each circle is rotated around its center as a rotation axis, the circles rotate in an opposite direction to the direction in which the circles should essentially rotate, or consequently, a large change in angle as well as a large offset by a parallel movement amount may occur.

[0342] In a typical solution method of the non-linear least squares method approximated by a quadratic expression, such a procedure is taken in which an approximate error function is generated by approximating an error function E(pi) up to a square number of a trial parameter in the neighborhood of a group “pi” of trace parameters as a variable least squares method, and using the approximate error function, such a group “pi” of the trial parameters is found to minimize the error function.

[0343] As a solution method for obtaining a solution with a small error value in the nonlinear least squares method as described above, the following reverse Hess method has been proposed.

[0344] This is a method in which, after calculating the approximate error function and calculating a group of the smallest experimental parameters from the error function, a group of experimental parameters is found at a next stage with higher overall accuracy. However, in case of finding a solution with a small error value using this reverse Hessian method, such a defect as below might occur. This is based on the Fig. 86A and Fig. 86B. In each of the Fig. 86A and Fig. 86B, a solid line shows the error function and a broken line shows an approximate error function obtained by approximating this error function. In each of the Fig. 86A and Fig. In FIG. 86B, the symbol P1 provided on a curve showing the error function denotes a position (x, y, θ) obtained in the coarse search described above. What is obtained as a quadratic function based on a value of the error function in the neighborhood of this P1 is the quadratic curve showing the approximate error indicated by the broken line.

[0345] Furthermore, the drawing in Fig. 86A, a case where the reverse Hess method acts appropriately, showing a case where, since a position having the smallest error value of the quadratic curve of the broken line indicating the approximate error function, namely P2, is close to a position P having the smallest error value of the error function indicated by the solid line, a more accurate position P2 having a smaller error value is found.

[0346] On the other hand, in the case of Fig. 86B, the drawing shows a case where the reverse Hess method operates inappropriately, showing a case where, since a position having the smallest error value of the quadratic curve of the broken line indicating the approximate error function, namely P2, is away from the position P having the smallest error value of the error function indicated by the solid line, an inaccurate position P2 having a small error value is found.

[0347] Of the above cases, in Fig. In the case shown in Fig. 86B, if a solution is intended to be found as it is in the reverse Hess method, a large offset of the experimental parameters may be taken as the solution as described above, which, as a result, causes the problem of deterioration in the accuracy of fine positioning. The present inventor has found a technique for providing a restriction on movement or rotation of a pattern model as a technique for suppressing the occurrence of the case that the reverse Hess method operates in an inappropriate manner, which is described in Fig. 86B. Namely, as a solution method for the least squares method based on the reverse Hess method, a new term is added to the error function. Specifically, in addition to the above term related to a distance (first error function), such a term (second error function Ed) is added to increase the error value due to offset from the experimental parameter. Thus, an excessively large probability is suppressed by the second error function Ed, so that a reasonable approximation can be expected, as shown in Fig.86A. As described above, a condition of convergence in an appropriate direction can be added to the fine positioning step to avoid rotation and dispersion in an unintended direction, thus improving positioning reliability. When the experimental parameter in the least squares method is Pi, the second error function is a function of "Pi - P0i." An example of simultaneous equations of the error function E(P) obtained as a result is shown in the next expression: E(P0,P1,…,Pn)=E0(P0,P1,…Pn)+Ed(P0,P1,…Pn)Ed(P0,P1,…Pn)=∑((Pi−P0i)σi)2

[0348] In the above expression, a total error function E(P) is expressed by a sum of a first error function Eo showing a distance between each segment and its corresponding edge point, and a second error function Ed obtained by considering an amount of change in the trial parameter in the first error function when making this change and calculating an accumulated value of the second error value.As thus described, in calculating the least squares method, since adding the second error function Ed besides the term which becomes the smallest value when fitting with an ideal position, resulting in searching for a solution in such a direction as to make both terms smaller, and suppressing an excessively large change in the trial parameter, it is possible to avoid such a condition as rotation or dispersion in a wrong direction in fine positioning, so as to obtain the advantage of stabilizing a process result. Industrial applicability

[0349] The pattern model positioning method in image processing, the image processing apparatus, an image processing program, and the computer-readable recording medium according to the present invention are preferably applicable to positional detection of a coordinate position, a rotation angle, and the like of a work, position measurement of an external diameter, an internal diameter, a width, and the like, recognition, identification, determination, testing, and the like in image processing to be used in the field of factory automation (FA). For example, the present invention can be used for positioning an electrode of an integrated circuit (IC) for bonding thereof, and the like.

Claims

[1] A pattern-model positioning method in image processing for searching an image to be searched (OI), and for positioning an object to be searched that resembles a pre-registered image (RI) using a pattern model (PM) corresponding to the registered image (RI), positioning with a higher accuracy than an initially given position, the method comprising the steps of: Extracting an outline from the registered image (RI), Constructing a pattern model (PM) of the registered image (RI) in which a plurality of reference points are set on the extracted outline, and also a corresponding point search line having a predetermined length and passing through each reference point, as well as being substantially orthogonal to the outline, is also assigned to each reference point; Acquiring an image to be searched (OI) and also arranging the pattern model (PM) so that it is superimposed on the image to be searched (OI) based on an initial position contained in the image to be searched (OI) and corresponding to the registered image (RI); Finding a corresponding edge point on the image to be searched (OI), corresponding to a reference point of each corresponding point search line of the pattern model (PM) from each pixel in the image to be searched (OI) positioned superimposed on the corresponding point search line, when superimposing the pattern model (PM) on the initial position of the image to be searched (OI); and Taking a distance between each corresponding edge point and an outline containing the reference point corresponding to the corresponding edge point as an evaluation value, and performing fine positioning with a higher accuracy than the accuracy at the given initial position so that an absolute value of an accumulated value of the evaluation values becomes minimal, wherein, in the case of a plurality of corresponding edge point candidates present on the corresponding point search line, the step of finding the corresponding edge point on the image to be searched regards as the corresponding edge point a point having: an edge strength that is greater than an edge strength threshold and shows the maximum, and an edge angle that is sufficiently close to an ideal edge angle and also closest to the reference point. [2] A pattern model positioning method in image processing according to claim 1, wherein the step of extracting the outline from the registered image includes the steps: Extracting a plurality of edge points from the registered image; coupling adjacent edge points from the extracted plurality of edge points to create a continuous chain; and Creating a segment by approximating one or more chains using a line and / or arc, and viewing the aggregation of the segments as an outline. [3] A pattern model positioning method in image processing according to claim 1, wherein the step of performing fine positioning is made to calculate an error value or a weight value in a least squares method by using the distance between each segment constituting the outline and a corresponding edge point thereof to find the accumulated value of the evaluation values. [4] A pattern model positioning method in image processing according to claim 3, wherein the least squares method in a fine positioning step is to apply an error function obtained by calculating a distance between the corresponding edge point and a circular arc to a circular arc segment and an error function obtained by calculating a distance between the corresponding edge point and a straight line to a line segment. [5] A pattern model positioning method in image processing according to claim 2, wherein the step of generating the segment with respect to the chains repeats an operation of first attempting to approximate the chains by the line and then switching to approximation by the circular arc when an error of approximation of the line exceeds a predetermined approximation threshold to generate a sequence of segments. [6] A pattern model positioning method in image processing according to claim 2, wherein the segment is configured from a cone curve, a spline curve and / or a Bezier curve or a combination thereof. [7] A pattern model positioning method in image processing according to claim 2, wherein a preset margin is set at the edge of the segment when setting the reference point on the outline, and the reference point is set in an area excluding the margin area. [8] A pattern model positioning method in image processing according to claim 1, wherein the initial position corresponding to the registered image included in the image to be searched in the step of arranging the pattern model so as to be superimposed on the image to be searched is detected by reducing the image to be searched and performing a pattern search on the image to be searched after the reduction. [9] A pattern model positioning method in image processing according to claim 2, wherein the corresponding point search line is formed by setting it to have a predetermined length in a direction substantially orthogonal to the segment with the reference point set on the segment used as a center. [10] A pattern model positioning method in image processing according to claim 2, wherein the predetermined length of the corresponding point search line is decided according to a relationship between a reduction ratio of the image to be searched as an object of a coarse search, which is a search performed with a predetermined accuracy on the entire area of the image to be searched, and a reduction ratio of the image to be searched to be subjected to fine positioning. [11] A pattern model positioning method in image processing according to claim 1, wherein the pattern model includes mesh data about the corresponding point search line, and includes as the mesh data at least a coordinate of the reference point, an angle of the corresponding point search line, and a length of the corresponding point search line. [12] A pattern model positioning method in image processing according to claim 2, wherein the step of extracting the segment from the chains repeats an operation of attempting approximation by the line and then switching to approximation by the circular arc when an error of the approximation by line exceeds a predetermined approximation threshold to extract a sequence of segments. [13] A pattern model positioning method in image processing according to claim 3, wherein the method is configured to perform at least one of movement in an X direction, movement in a Y direction, rotation, and enlargement / reduction adjustable as a degree of freedom of the least square method. [14] A pattern model positioning method in image processing according to claim 1, wherein the method is configured to further make an aspect adjustable as a degree of freedom of the least square method. [15] A pattern model positioning method in image processing according to claim 1, wherein the fine positioning step is repeatedly performed while shortening the corresponding point search line. [16] A pattern model positioning method in image processing according to claim 3, wherein in the step of performing the fine positioning, the least squares method is repeatedly applied to the corresponding point search line and also the length of the corresponding point search line is gradually shortened according to the number of repetitions of the least squares method. [17] A pattern model positioning method in image processing according to claim 3, wherein in the step of performing fine positioning, the length of the corresponding point search line is determined based on a reduction ratio in the coarse search, which is a search performed with a predetermined accuracy on the entire area of the image to be searched, and a reduction ratio in the fine positioning. [18] A pattern model positioning method in image processing according to claim 3, wherein in the step of performing fine positioning, the least squares method is repeatedly applied to the corresponding point search line, and an edge angle threshold is gradually lowered according to the number of repetitions of the least squares method. [19] A pattern model positioning method in image processing according to claim 3, wherein in the step of performing fine positioning when repeatedly applying the least squares method to the corresponding point search line, the least squares method is suspended at the time when the error value exceeds a preset error value threshold and the result is regarded as a final result. [20] A pattern model positioning method in image processing according to claim 3, wherein in the step of performing fine positioning, the method is constructed such that the least squares method is applied to the corresponding point search line, the least squares method is suspended at the time when the error value exceeds the preset error value threshold, an approximation performed using the error value of the least squares method completed in the previous step is applied as the final result, and the error value of the final result is also output. [21] A pattern model positioning method in image processing according to claim 1, wherein in the step of constructing the pattern model, the method is configured to make presence or absence of a polarity of an edge direction adjustable. [22] A pattern model positioning method in image processing according to claim 1, wherein in the step of constructing the pattern model, an angle and the corresponding point search line of each reference point change according to an affine transformation value. [23] A pattern model positioning method in image processing according to claim 1, wherein, in extracting the edge point from the registered image, Sobel processing is performed to calculate the edge angle and edge strength of each edge point. [24] A pattern model positioning method in image processing according to claim 23, wherein, in generating a corresponding point search line, a calculation is performed using Bresenham's algorithm for generating straight line data in searching the corresponding point. [25] A pattern model positioning method in image processing according to claim 1, wherein in the step of constructing the pattern model, a predetermined interval at which the reference points are set on the segments is a fixed width. [26] A pattern model positioning method in image processing according to claim 2, wherein, in the step of constructing the pattern model, filtering is performed to select the segment at which the reference point is set. [27] A pattern model positioning method in image processing according to claim 26, wherein criteria for selecting the segment in filtering are set based on a result of calculating an average edge strength of the edge point included in each segment and comparing the obtained average edge strength with a preset average edge strength threshold for filtering. [28] A pattern model positioning method in image processing according to claim 26, wherein criteria for selecting the segment in filtering are set based on a comparison result of a length of each segment with a preset edge strength threshold for filtering. [29] A pattern model positioning method in image processing according to claim 1, wherein in the step of constructing the pattern model, the pattern model is generated based on a fixed geometric shape. [30] A pattern model positioning method in image processing according to claim 1, wherein the edge image generated from the registered image is subjected to thinning processing as non-maximum point suppression processing before the edge coupling processing. [31] A pattern model positioning method in image processing according to claim 1, wherein the edge angle obtained from the image to be searched, the edge strength threshold, a position, data expressing the edge angle in units of vectors, and / or an edge vector is used as data for finding the corresponding edge point. [32] A pattern model positioning method in image processing according to claim 3, wherein a threshold value of each edge angle difference in the repeated least squares method in selecting the corresponding edge point is set based on an angular resolution obtained in the last coarse search. [33] A pattern model positioning method in image processing according to claim 1, wherein the length of the corresponding point search line is changed with respect to each reference point. [34] A pattern model positioning method in image processing according to claim 1, wherein the Bresenham algorithm is used to generate straight line data when scanning the corresponding point search line. [35] A pattern model positioning method in image processing according to claim 1, wherein the coordinate of the reference point is represented by a subpixel coordinate. [36] A pattern model positioning method in image processing according to claim 1, wherein the step of refining the coordinate of the corresponding edge point includes the steps of: Finding the corresponding edge point on the corresponding point search line; Selecting a pair point corresponding to the corresponding edge point such that the corresponding edge point and the pair point are substantially orthogonal to and framing the corresponding point search line; and Finding corresponding subpixel coordinates of the corresponding edge point and the pair point to find an average coordinate of the subpixel coordinates and taking the obtained average coordinate as a real corresponding edge point coordinate. [37] A pattern model positioning method in image processing according to claim 1, wherein the step of refining the coordinate of the corresponding edge point includes the steps of: Finding a preliminary corresponding edge point on the corresponding point search line; and Finding a plurality of neighborhood edge points existing around the preliminary corresponding edge point to find subpixel coordinates of the preliminary corresponding edge point and the plurality of neighborhood edge points. [38] A pattern model positioning method in image processing according to claim 3, wherein in the step of performing fine positioning, the least squares method is applied to the corresponding point search line, and a rank is calculated as an evaluation value by a ratio between the number of corresponding points and the number of reference points in the final least squares method processing. [39] A pattern model positioning method in image processing according to claim 38, wherein the rank is found by the following formula: S=∑i=1n1∑t=1m1=nm S Rank n number of corresponding points m number of reference points [40] A pattern model positioning method in image processing according to claim 3, wherein in the step of performing fine positioning, the least squares method is applied to the corresponding point search line, and a rank as an evaluation value is calculated by a ratio of a total of weights, all of which are calculated from a difference between the edge angle of the corresponding edge point in the final processing of the least squares method and the edge angle of the reference point corresponding thereto, to the total number of reference points. [41] A pattern model positioning method in image processing according to claim 40, wherein the rank is found by the following formula: S=∑i=1nω(|θi=θiP|)∑i=1m1 S Rank N Number of corresponding points M Number of reference points ωx function which is 1 when x = 0 and monotonically decreases with increasing x θi edge angle corresponding point θip Angle from corresponding point to corresponding reference point [42] An image processing device for positioning with a higher accuracy than at an initially given position when searching from an image to be searched (OI) and positioning an object resembling a pre-registered image (RI) using a pattern model corresponding to the registered image (RI), the device comprising: an image input device (1) for capturing a registered image (RI) and an image to be searched (OI); an outline extraction device (62) for extracting an outline from the registered image (RI) acquired by the image input device (1); a chain generating device (63) for extracting a plurality of edge points from the outline extracted by the outline extracting device (62), and coupling adjacent edge points from the extracted plurality of edge points to generate a continuous chain; a segment generating device (68) for generating a segment by approximating one or more chains by means of a line and / or a circular arc; a pattern model constructing device (70) for setting a plurality of reference points on the segment generated by the segment generating device (68) and also constructing a pattern model (PM) of the registered image (RI) to which corresponding point search lines of a fixed length are assigned, which pass through the corresponding reference points and are substantially orthogonal to the contours; and a fine positioning device (76) for detecting an initial position corresponding to the registered image (RI) included in the image to be searched (OI) acquired by the image input device (1), for arranging the pattern model (PM) so as to be superimposed on the image to be searched, in order to find each individual corresponding edge point on the image to be searched (OI) corresponding to each segment constituting the pattern model, considering a relationship between each segment and the corresponding edge point as an evaluation value and operating at a higher accuracy than an initially given position so that an accumulated value of the evaluation values becomes minimum or maximum, an image reduction device (77) for reducing the image to be searched (OI) captured by the image input device (1) with a predetermined reduction ratio; an edge angle image generating device (60) for calculating an edge angle image containing edge angle information with respect to all pixels constituting the image on the reduction ratio image to be searched reduced by the image reducing device; an edge angle bit image generating device (69) for transforming each pixel of the edge angle image generated by the edge angle image generating device (60) into an edge angle bit image (EB) expressed by an edge angle bit indicating an angle with a predetermined fixed width; an edge angle bit image reduction device (78) for performing, in order to generate an edge angle bit reduction image (REB) reduced from the edge angle bit image (EB), an OR operation on the edge angle bit of each pixel included in an OR operation area determined according to a reduction ratio for reducing the edge angle bit image, to generate an edge angle bit reduction image consisting of reduced edge angle bit data representing each OR operation area; and a coarse search device (71) for performing a pattern search on a first edge angle bit reduction image (REB) generated by the edge angle bit image reduction device (78) using, as a template, a pattern model (PM) for the first coarse search, which has been generated with a first reduction ratio with respect to a first reduction ratio image to be searched, which has been reduced by the image reduction device (77) with the first reduction ratio, in order to find with first accuracy a first position and posture corresponding to the pattern model (PM) for the first coarse search, from the total area of the first edge angle bit reduction image (REB), and also for performing a pattern search on a second edge angle bit reduction image (REB) generated by the edge angle bit image reduction device (78) using, as a template, a pattern model (PM) for the second coarse search, which has been generated with a second reduction ratio that is not greater than the first reduction ratio and not less than a zero magnification, with respect to a second reduction ratio image to be searched,reduced by the image reduction device (77) into the second, Reduction ratio to obtain a second position with a second accuracy that is higher than the first accuracy and to find a position corresponding to the pattern model (PM) for the second coarse search from a predetermined range of the second edge angle bit reduction image where the first position and posture are set as references, wherein the fine positioning device (76) arranges a pattern model so as to be superimposed on a third reduction ratio image to be searched, which is obtained by appropriately reducing the image (OI) to be searched into a third reduction ratio that is not smaller than unmagnified and not larger than the second reduction ratio, using the second position and posture of the third reduction ratio image to be searched, to find a corresponding edge point on the third reduction ratio image to be searched corresponding to an outline constituting the pattern model (PM),considers a relationship between each contour and its corresponding edge point as an evaluation value and, Performs fine positioning with a third accuracy higher than the second accuracy so that an accumulated value of the evaluation values becomes minimum or maximum. [43] An image processing apparatus according to claim 42, further comprising a segment filter function for selecting a segment at which a reference point is set in constructing the pattern model (PM). [44] An image processing program for positioning with a higher accuracy than an initially given position when searching from an image to be searched (OI) and positioning an object that resembles a pre-registered object using a pattern model (PM) corresponding to the registered image, the program causing a computer to realize: an image input function for capturing a registered image and an image to be searched; an outline extraction function for extracting an outline from the registered image acquired by the image input function; a chain generation function for extracting a plurality of edge points from the outline extracted by the outline extraction function and coupling adjacent edge points from the extracted plurality of edge points to generate a continuous chain; a segment generation function for generating a segment by approximating one or more chains using a line and / or a circular arc; a pattern model construction function for setting a plurality of reference points on the segment generated by the segment generation function, and also for constructing a pattern model of the registered image to which corresponding point search lines of a fixed length are assigned, which pass through the corresponding reference points and are substantially orthogonal to the outlines; and a fine positioning function for detecting an initial position corresponding to the registered image contained in the image to be searched, which has been acquired by the image input function, arranging the pattern model so as to be superimposed on the image to be searched, in order to find each individual corresponding edge point on the image to be searched, which corresponds to each segment constituting the pattern model, viewing a relationship between each segment and the corresponding edge point as an evaluation value and performing, with a higher accuracy than at an initially given position, such that an accumulated value of the evaluation values becomes minimum or maximum, an image reduction function for reducing the image to be searched, which has been acquired by the image input function, with a predetermined reduction ratio; an edge angle image generation function for calculating an edge angle image including edge angle information with respect to each pixel constituting the image in the reduction ratio image to be searched, reduced by the image reduction function; an edge angle bit image generating function for transforming each pixel of the edge angle image generated by the edge angle image generating function into an edge angle bit image expressed by an edge angle bit indicating an angle with a predefined fixed width; an edge angle bit image reduction function for performing, in order to produce an edge angle bit reduction image reduced from the edge angle bit image, an OR operation on an edge angle bit of each pixel included in an OR operation area determined by a reduction ratio for reducing the edge angle bit image to produce an edge angle bit reduction image made of reduced edge angle bit data representing each OR operation area; and a coarse search function for performing a pattern search on a first edge angle bit reduction image generated by the edge angle bit image reduction function using, as a template, a pattern model for the first coarse search having a first reduction ratio with respect to a first reduction ratio image was generated which was reduced by the image reduction function with the first reduction ratio to find with first accuracy a first position and posture corresponding to the pattern model for the first coarse search from the total area of the first edge angle bit reduction image, and also to perform a pattern search on a second edge angle bit reduction image generated by the edge angle bit image reduction function, using, as a template, a pattern model for a second coarse search, which was generated with a second reduction ratio which is not greater than the first reduction ratio and not less than unmagnified with respect to a second reduction ratio image to be searched, reduced by the image reduction function to a second reduction ratio to obtain with a second precision which is higher than the first precision, a second position and to find a position corresponding to the pattern model for the second coarse search from a predetermined range of the second edge angle bit reduction image where the first position and pose are set as references; where the fine positioning function arranges a pattern model to be superimposed on a third reduction ratio image to be searched, which is obtained by reducing, as appropriate, the image to be searched into a third reduction ratio not smaller than unmagnified and not larger than the second reduction ratio, using the second position and posture of the third reduction ratio image to be searched, to find a corresponding edge point on the third reduction ratio image to be searched corresponding to an outline constituting the pattern model, wherein a relation between each outline and its corresponding edge point is regarded as an evaluation value, and performing fine positioning with third precision higher than the second precision so that an accumulated value of the evaluation values becomes minimum or maximum. [45] A computer-readable recording medium having recorded thereon a program according to claim 44.

Images