Image Processing Apparatus, Recording Medium, and Image Processing Method

Abstract

The image processing device (100) comprises: a template analysis unit (140) which converts a pattern contained in a template into a plurality of conversion patterns, and generates a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively contain respective conversion patterns of the plurality of conversion patterns; and a search processing unit (160) which performs a first search for searching a searched object image using a plurality of first extended templates selected from the plurality of extended templates, performs a second search for searching a searched object image using a plurality of second extended templates selected from the plurality of extended templates based on the result of the first search, performs the first search using a plurality of conversion templates and templates selected from the plurality of extended templates by coarse division, and performs a second search with high precision using a plurality of extended templates that are close to one extended template that can be detected in the first search.

Description

Technical Field

[0001] The present invention relates to an image processing device, a recording medium, and an image processing method. Background Art

[0002] Template matching is a widely known method for detecting specific patterns from images. In template matching, a template representing the pattern to be detected is prepared in advance, and the pattern is compared with various parts of the search target image. In this way, the part of the search target image that is most similar to the pattern shown in the template is detected.

[0003] Template matching can detect parts of a pattern that is identical to the pattern shown in the template. However, if the part is rotated, scaled, or deformed due to the composition of the captured image, the part may differ from the pattern shown in the template in terms of angle, size, or shape, and thus cannot be detected.

[0004] Therefore, in the subject following method described in Patent Document 1, the template is rotated, scaled, or deformed to expand into multiple templates, and all the expanded templates are compared with each part of the search object image, thereby responding to the rotation, scaling, or deformation.

[0005] Prior art literature

[0006] Patent Literature

[0007] Patent Document 1: Japanese Patent No. 4843787 Summary of the Invention

[0008] Problems to be solved by the invention

[0009] In the prior art, template matching corresponding to rotation, scaling, or deformation can be performed. However, this requires a comparison operation between all multiple templates and the search object image, resulting in a very large amount of calculation.

[0010] Therefore, one or more aspects of the present invention aim to achieve template matching corresponding to rotation, scaling, deformation, etc. with a smaller amount of computation.

[0011] Means for solving problems

[0012] An image processing device according to one embodiment of the present invention is characterized in that the image processing device comprises: a template analyzing unit which converts a pattern contained in a template into a plurality of conversion patterns using a plurality of numerical values of different degrees of changing the pattern, thereby generating a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; and a search processing unit which performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image based on a result of the first search using a plurality of second extended templates selected from the plurality of extended templates, wherein the search processing unit compares the first extended templates selected from the plurality of extended templates with the second extended templates selected from the plurality of extended templates according to the result of the first search. The plurality of conversion templates corresponding to the plurality of numerical values selected by the first partitioning from a set in which the plurality of numerical values are arranged in ascending or descending order and the templates are set as the plurality of first extended templates. The search processing unit sets as the plurality of second extended templates the plurality of conversion templates corresponding to the plurality of numerical values selected from a prescribed range by a second partitioning finer than the first partitioning and including the following one numerical value, wherein the one numerical value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns can be detected in the first search, and the number of numerical values included in the prescribed range is less than a number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set.

[0013] A program according to one embodiment of the present invention is characterized in that the program causes a computer to function as: a template analyzing unit that converts a pattern included in a template into a plurality of conversion patterns using a plurality of numerical values that change the pattern to different degrees, thereby generating a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; and a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image based on a result of the first search using a plurality of second extended templates selected from the plurality of extended templates, wherein the search processing unit compares the first and second extended templates selected from the plurality of extended templates with the first and second extended templates selected from the plurality of extended templates. The plurality of conversion templates corresponding to the plurality of numerical values selected by the first partitioning from a set in which the plurality of numerical values are arranged in ascending or descending order, and the templates are set as the plurality of first extended templates. The search processing unit sets as the plurality of second extended templates the plurality of conversion templates corresponding to the plurality of numerical values selected from a prescribed range by a second partitioning finer than the first partitioning, which includes the following one numerical value: the one numerical value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns can be detected in the first search, and the number of numerical values included in the prescribed range is less than a number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set.

[0014] In an image processing method according to one aspect of the present invention, a pattern included in a template is converted into a plurality of conversion patterns using a plurality of numerical values having different degrees of change, a plurality of extended templates consisting of a plurality of conversion templates and the template are generated, the plurality of conversion templates respectively including each of the plurality of conversion patterns, a first search for a searched object image is performed using a plurality of first extended templates selected from the plurality of extended templates, and a second search for the searched object image is performed using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search, characterized in that the plurality of numerical values ​​selected from the plurality of extended templates arranged in ascending or descending order are compared with the plurality of numerical values ​​selected from the plurality of extended templates. A plurality of conversion templates corresponding to a plurality of numerical values selected from a set of values through a first division and the templates are set as the plurality of first extended templates, and a plurality of conversion templates corresponding to a plurality of numerical values selected from a prescribed range through a second division finer than the first division and including the following one numerical value are set as the plurality of second extended templates, wherein the one numerical value corresponds to one or more first extended templates that can detect the pattern and the plurality of conversion patterns in the first search, and the number of numerical values included in the prescribed range is less than a number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set.

[0015] Effects of the Invention

[0016] According to one or more aspects of the present invention, template matching corresponding to rotation, scaling, deformation, etc. can be achieved with a smaller amount of computation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is a block diagram schematically showing the configuration of the image processing apparatus according to the first and second embodiments.

[0018] Figure 2 This is a block diagram schematically showing the configuration of the template analysis unit in the first and second embodiments.

[0019] Figure 3 This is a block diagram schematically showing the configuration of a unique pixel selection unit in the first embodiment.

[0020] Figure 4 This is a block diagram schematically showing the structure of a selected pixel determination unit.

[0021] Figure 5 This is a block diagram schematically showing the structure of a search method determination unit.

[0022] Figure 6 This is a block diagram schematically showing the configuration of a search processing unit.

[0023] Figure 7 (A) and (B) are block diagrams showing hardware configuration examples.

[0024] Figure 8 : is a flowchart showing the expansion process in the template expansion unit.

[0025] Figure 9 (A) to (D) are schematic diagrams for explaining the expansion processing performed by the template expansion unit.

[0026] Figure 10 This is a flowchart showing the pixel selection process performed by the unique pixel selection unit in the first embodiment.

[0027] Figure 11 (A) to (C) are schematic diagrams for explaining pixel selection.

[0028] Figure 12 (A) to (D) are schematic diagrams for explaining corresponding selected pixels determined in a conversion template generated by rotating the template.

[0029] Figure 13 This is a flowchart showing the pixel selection process performed by the selected pixel determination unit.

[0030] Figure 14(A) and (B) are schematic diagrams for explaining the process of calculating the co-occurrence histogram.

[0031] Figure 15 : is a flowchart showing the search process of the search processing unit.

[0032] Figure 16 (A) and (B) are schematic diagrams illustrating the relationship between coarse search and fine search.

[0033] Figure 17 This is a block diagram schematically showing the configuration of a unique pixel selection unit in the second embodiment.

[0034] Figure 18 This is a flowchart showing the pixel selection process performed by the selected pixel determination unit in the second embodiment. DETAILED DESCRIPTION

[0035] Implementation Method 1

[0036] Figure 1 This is a block diagram schematically showing the configuration of the image processing apparatus 100 according to the first embodiment.

[0037] The image processing apparatus 100 includes an input unit 110 , an image acquisition unit 120 , a template acquisition unit 130 , a template analysis unit 140 , a search processing unit 160 , a search result processing unit 180 , and an output unit 190 .

[0038] The image processing apparatus 100 searches for a pattern identical to or similar to the pattern indicated by the template within the search target image indicated by the search target image data, and outputs the search result.

[0039] The input unit 110 receives input of a template, which is image data of an image to be detected as a pattern, that is, search target image data, and image data representing a pattern to be detected.

[0040] The image acquisition unit 120 acquires the search target image data via the input unit 110 . The acquired search target image data is supplied to the search processing unit 160 .

[0041] The template acquisition unit 130 acquires a template via the input unit 110 . The acquired template is provided to the template analysis unit 140 .

[0042] The template analysis unit 140 converts the pattern contained in the template provided by the template acquisition unit 130 into multiple conversion patterns using multiple numerical values that vary in degree, thereby generating multiple conversion templates each containing one of the multiple conversion patterns. The template analysis unit 140 then generates multiple extended templates consisting of the template and the multiple conversion templates.

[0043] For example, the template analysis unit 140 uses the template provided by the template acquisition unit 130 to rotate, scale, and transform the pattern shown in the template to generate multiple conversion templates, and provides the template provided by the template acquisition unit 130 and the conversion template as multiple expansion templates to the search processing unit 160.

[0044] Furthermore, the template analysis unit 140 generates information necessary for the search processing unit 160 to perform a search using the extended template, and also provides this information to the search processing unit 160 .

[0045] Figure 2 It is a block diagram schematically showing the configuration of the template analysis unit 140 .

[0046] The template analysis unit 140 includes a template expansion unit 141 , a unique pixel selection unit 142 , and a search method determination unit 150 .

[0047] The template expansion unit 141 copies and converts the single template provided by the template acquisition unit 130 into a template required for the search, namely, a conversion template, thereby generating multiple expanded templates consisting of the single template and the multiple conversion templates. Furthermore, the template expansion unit 141 generates conversion method information indicating the conversion method used when converting from the single template to the multiple conversion templates. The conversion method indicates the expansion method (e.g., rotation, scaling, or deformation) and the numerical values of the expansion method (e.g., angle, expansion / reduction ratio, and deformation ratio).

[0048] Then, the template extension unit 141 provides the plurality of extended templates and the conversion method information to the unique pixel selection unit 142 , the search method determination unit 150 , and the search processing unit 160 .

[0049] The unique pixel selection unit 142 selects pixels to be used for search for each of the plurality of extended templates provided by the template extension unit 141 and generates selected pixel information indicating the selected pixels. The selected pixel information is provided to the search method determination unit 150 and the search processing unit 160.

[0050] Furthermore, regarding the selected pixel information, information containing different information may be generated separately for the coarse search and the fine search.

[0051] Figure 3 This is a block diagram schematically showing the configuration of the unique pixel selection unit 142 .

[0052] The unique pixel selection unit 142 includes a selected pixel determination unit 143 , a coordinate conversion unit 148 , and a selected pixel calculation unit 149 .

[0053] The selected pixel determination unit 143 determines characteristic pixels in the pattern indicated by the template as selected pixels according to a predetermined rule.

[0054] Figure 4 This is a block diagram schematically showing the configuration of the selected pixel determination unit 143 .

[0055] The selected pixel determination unit 143 includes an inter-pixel relationship analysis unit 144 , a similarity index calculation unit 145 , a selected pixel number determination unit 146 , and a pixel selection unit 147 .

[0056] The pixel relationship analysis unit 144 analyzes the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template and outputs the analysis results. For example, the pixel relationship analysis unit 144 generates a histogram of the relationship between the pixel values of the plurality of pixels and provides histogram data representing the generated histogram to the selected pixel number determination unit 146 and the pixel selection unit 147.

[0057] The similarity index calculation unit 145 calculates the similarity between the multiple extended templates provided by the template extension unit 141 and outputs the calculation results. For example, the similarity index calculation unit 145 calculates the sum of the absolute differences between each pixel for each combination of two extended templates selected from the multiple extended templates as the similarity. The sum of the absolute differences between each pixel can be calculated using all pixels within the template, or the template can be divided into specific regions and the sum can be calculated for each segmented region. The similarity index calculation unit 145 then generates similarity index information representing the calculated similarity for each combination of the two selected extended templates and provides the generated similarity index information to the selected pixel number determination unit 146.

[0058] The selected pixel number determination unit 146 determines the number of pixels to be selected when searching for the pattern indicated by the template in the search target image, i.e., the number of selected pixels. For example, the selected pixel number determination unit 146 may determine the number of selected pixels based on a predetermined value, based on the shape of the histogram generated by the inter-pixel relationship analysis unit 144, based on the distribution of similarities indicated by the similarity index information generated by the similarity index calculation unit 145, or based on a combination of these. The determined number of selected pixels is then provided to the pixel selection unit 147.

[0059] Specifically, the selected pixel number determination unit 146 may determine a predetermined value itself as the selected pixel number, or may calculate the selected pixel number by applying a specific formula (for example, multiplying by a constant) to the predetermined value.

[0060] Alternatively, the selected pixel count determination unit 146 may determine the number of selected pixels as follows: the smaller the change in frequency of each level in the histogram generated by the inter-pixel relationship analysis unit 144, the smaller the number of selected pixels. Alternatively, the frequency change of each level may be determined by calculating the rate of change between adjacent levels and making a determination based on the maximum or average value of the rate of change. Specifically, the smaller the maximum rate of change, the smaller the number of selected pixels, and the smaller the average rate of change, the smaller the number of selected pixels.

[0061] Furthermore, the selected pixel number determination unit 146 may determine the number of selected pixels by forming a histogram of the similarity indicated by the similarity index information generated by the similarity index calculation unit 145. The smaller the change in frequency of each class, the smaller the number of selected pixels. The change in frequency of each class is the same as described above.

[0062] In addition, as described above, the pixel number determination unit 146 may determine the number of pixels to be selected by assigning predetermined weights to at least two values among a value determined based on a predetermined value, a value determined based on a histogram between pixels, and a value determined based on a histogram of similarities between extended templates, and averaging the values.

[0063] The pixel selection unit 147 determines the pixels to be selected in the template based on the histogram, which is the result of the analysis of the relationship between pixel values provided by the inter-pixel relationship analysis unit 144, and the number of selected pixels provided by the selected pixel number determination unit 146, and generates selected pixel information indicating the determined pixels. The selected pixel information is provided to the selected pixel calculation unit 149 and the search method determination unit 150.

[0064] return Figure 3 The coordinate conversion unit 148 calculates a coordinate conversion formula for performing coordinate conversion between a single coordinate included in the template and the coordinates corresponding to the single coordinate in each of the multiple conversion templates included in the multiple expanded templates, according to the conversion method indicated by the conversion method information provided by the template expansion unit 141. The coordinate conversion unit 148 then generates coordinate conversion formula information representing the calculated coordinate conversion formula for each of the multiple conversion templates, and provides the coordinate conversion formula information to the selected pixel calculation unit 149.

[0065] The selected pixel calculation unit 149 performs coordinate conversion on the coordinates of the selected pixel determined by the selected pixel determination unit 143 using the coordinate conversion formula indicated by the coordinate conversion formula information provided by the coordinate conversion unit 148. This conversion calculates the coordinates of the corresponding selected pixel, i.e., the pixel corresponding to the selected pixel, in each of the plurality of conversion templates. The selected pixel calculation unit 149 then generates corresponding selected pixel information indicating the coordinates of the corresponding selected pixel in each of the plurality of conversion templates and provides this corresponding selected pixel information to the search method determination unit 150 and the search processing unit 160.

[0066] return Figure 2 The search mode determination unit 150 determines the search mode in the search processing unit 160. The search processing unit 160 performs a coarse search using a plurality of extended templates roughly selected from the entire set of extended templates, and a fine search that precisely searches around one or more extended templates further selected from the plurality of extended templates used in the coarse search. Therefore, the search mode determination unit 150 determines the coarse search mode, which is the coarse search mode, and the fine search mode, which is the fine search mode.

[0067] Figure 5 This is a block diagram schematically showing the configuration of the search method determination unit 150 .

[0068] The search mode determination unit 150 includes a coarse search mode determination unit 151 and a fine search mode determination unit 152 .

[0069] The coarse search mode determination unit 151 determines a coarse search mode based on the conversion mode information and the selected pixel information. For example, if the expansion mode is rotation, the coarse search mode determination unit 151 determines an angle interval of 5° as the coarse search mode. Furthermore, if the expansion mode is scaling, the coarse search mode determination unit 151 determines an enlargement / reduction ratio interval of 10% as the coarse search mode. Furthermore, if the expansion mode is deformation, the coarse search mode determination unit 151 determines a deformation ratio interval of 10% as the coarse search mode. Furthermore, the coarse search mode determination unit 151 generates coarse search mode information indicating the coarse search mode determined for each expansion mode and provides this coarse search mode information to the fine search mode determination unit 152 and the search processing unit 160.

[0070] Here, the coarse search mode determination unit 151 refers to the conversion mode information and searches the range of all numerical values used in the conversion at intervals wider than those in the fine search mode described later. Therefore, the coarse search mode determination unit 151 determines the division of the coarse search mode at intervals wider than the intervals of the numerical values indicated by the conversion mode information. The intervals of the coarse search mode can be predetermined or determined in a predetermined manner (e.g., a multiple) based on the intervals of the numerical values in the conversion mode. Furthermore, the intervals of the coarse search mode can be determined based on the distribution of similarities indicated by the similarity index information generated by the similarity index calculation unit 145. For example, the coarse search mode determination unit 151 can determine the intervals of the numerical values in the coarse search mode by forming a histogram of the similarities indicated by the similarity index information generated by the similarity index calculation unit 145, with the intervals becoming wider as the frequency variation of each level decreases. Furthermore, the coarse search mode determination unit 151 can determine the intervals of the coarse search mode by making the intervals of the numerical values wider as the number of selected pixels decreases. In other words, the intervals of the numerical values do not need to be constant between values.

[0071] The fine search mode determination unit 152 determines a fine search mode based on the coarse search mode information, conversion mode information, and selected pixel information. For example, if the expansion mode is rotation, the fine search mode determination unit 152 determines an angle interval of 1° and a search range of -5° to +5° as the fine search mode. Furthermore, if the expansion mode is zoom, the fine search mode determination unit 152 determines an enlargement / reduction ratio interval of 10% and a search range of -20% to +20% as the fine search mode. Furthermore, if the expansion mode is deformation, the fine search mode determination unit 152 determines a deformation ratio interval of 10% and a search range of -20% to +20% as the fine search mode. Furthermore, the fine search mode determination unit 152 generates fine search mode information indicating the fine search mode determined for each expansion mode and provides this fine search mode information to the search processing unit 160.

[0072] Here, the fine search mode determination unit 152 refers to the coarse search mode information and determines the search range for the fine search mode so that it fills the gap indicated by the coarse search mode. For example, when searching in the coarse search mode using an extended template corresponding to a certain value, the search range is preferably set to include at least the average of the values adjacent to that value in the coarse search mode and the range between those values. However, the number of values included in the search range for the fine search mode must be less than the number obtained by subtracting the number of values corresponding to the coarse search from the number of values included in the set to reduce the amount of computation required.

[0073] Furthermore, the fine search mode determination unit 152 may determine the intervals in the fine search mode by referring to the conversion mode information. For example, the fine search mode determination unit 152 may determine the intervals in the fine search mode based on the intervals used to generate the conversion template from the template. This is not limited to this method when the similarity between adjacent templates is extremely high. The fine search mode determination unit 152 may increase the intervals as the number of selected pixels indicated by the selected pixel information decreases. As with the coarse search mode, the intervals in the fine search mode do not need to be fixed.

[0074] Furthermore, by including the position information of the target pattern included in the rough search result in the fine search method, the spatial search range in the search target image may include the position including the target pattern and be smaller than the entire search target image.

[0075] return Figure 1 The search processing unit 160 performs a first search, ie, a rough search, for searching the search target image using a plurality of extended templates, ie, a plurality of first extended templates selected from the plurality of extended templates.

[0076] Here, the search processing unit 160 sets the plurality of conversion templates and the original template corresponding to the plurality of numerical values selected by the first division from the set of the plurality of numerical values arranged in ascending or descending order as the plurality of first expanded templates.

[0077] Here, the first division is based on the coarse search method determined by the coarse search method determination unit 151. Thus, the first division may be predetermined, and the higher the similarity between the multiple extended templates, the larger the first division. Alternatively, the smaller the frequency change in a histogram of the relationship between the pixel values of multiple pixels in a predetermined positional relationship within the template, the larger the first division.

[0078] Furthermore, the search processing unit 160 performs a second search, namely, a fine search, for searching for the search target image using a plurality of second extended templates selected from the plurality of extended templates based on the result of the rough search.

[0079] Here, the search processing unit 160 sets as a plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range using a second division finer than the first division, and including a value corresponding to one or more first extended templates in which a pattern and a plurality of conversion patterns were detected in the coarse search. Furthermore, the number of values included in the predetermined range is less than the number obtained by subtracting the number of values corresponding to the first extended template from the number of values included in the predetermined range.

[0080] Here, the second division is based on the fine search method determined by the fine search method determination unit 152 .

[0081] Furthermore, in a coarse search, whether the searched image contains the pattern and any one or more of the multiple conversion patterns is detected. If the pattern and any one or more of the multiple conversion patterns are contained, the position of the target pattern and the relationship of the target pattern to the pattern are detected. In a fine search, the position of the target pattern, i.e., the position of the target pattern, is detected in the searched image, and the relationship of the target pattern to the pattern is detected. Furthermore, the relationship may be at least one of a rotation angle, an enlargement / reduction ratio, and a deformation ratio.

[0082] Furthermore, the search processing unit 160 performs a coarse search or a fine search by comparing the pixel values of one or more pixels selected from a target extended template, one of a plurality of extended templates used in the coarse search or the fine search, with the pixel values of pixels corresponding to the one or more pixels in the partial image. The partial image is cut out of the search target image at the same size as the target extended template. The one or more pixels are selected based on the selected pixel information generated by the unique pixel selection unit 142, and the number of pixels is smaller than the total number of pixels included in the target extended template, thereby reducing the processing load of the comparison operation.

[0083] Here, in the case where the object extension template is one of multiple conversion templates, a coordinate conversion formula is used to convert the coordinates of the pixels contained in the pattern into the coordinates of the pixels of the conversion pattern contained in the object extension template, and the coordinates of each of the one or more pixels to be selected are calculated based on the coordinates of the pixels selected in the template.

[0084] Alternatively, the higher the similarity between the multiple extended templates, the smaller the number of one or more pixels to be selected. Alternatively, the smaller the frequency variation in a histogram of the relationship between the pixel values of multiple pixels in a predetermined positional relationship within the template, the smaller the number of one or more pixels to be selected.

[0085] Figure 6 It is a block diagram schematically showing the configuration of the search processing unit 160 .

[0086] The search processing unit 160 includes a rough search execution unit 161 and a fine search execution unit 162 .

[0087] The coarse search execution unit 161 performs a coarse search on the target image indicated by the target image data provided by the image acquisition unit 120, using the extended template provided by the template analysis unit 140, in a coarse search mode indicated by the coarse search mode information provided by the template analysis unit 140. The coarse search execution unit 161 then provides the conversion mode of the extended template, among the plurality of extended templates, for which a corresponding pattern was found in the target image, as a result of the coarse search, i.e., a coarse search result, to the fine search execution unit 162.

[0088] The fine search execution unit 162 performs a fine search based on the rough search results, using the expanded template provided by the template analysis unit 140, for the search object image indicated by the search object image data provided by the image acquisition unit 120, using the fine search method indicated by the fine search method information provided by the template analysis unit 140.

[0089] For example, the fine search execution unit 162 determines the expansion method and its value based on the conversion method included in the coarse search results, and determines the range for executing the fine search, i.e., the search execution range, based on the search range of the fine search method indicated by the fine search method information. For example, if the expansion method included in the coarse search results is rotation and its value is 12°, and the search range of the fine search method indicated by the fine search method information is -5° to +5°, the fine search execution unit 162 determines 7° to 17° as the search execution range. The fine search execution unit 162 then searches the target image using each of the multiple expansion templates converted using the values within the search execution range of 7° to 17°.

[0090] Then, the fine search execution unit 162 provides the search results of the fine search, i.e., the fine search results, as the final search results to the search result processing unit 180. The fine search results include the position where the corresponding pattern was found in the searched image, the value of the expansion method of the template for the pattern found in the searched image, and the like.

[0091] The search result processing unit 180 performs a process of outputting the search result provided from the search processing unit 160 in a predetermined format.

[0092] The output unit 190 performs output according to an instruction from the search result processing unit 180. For example, the search result is output in the form of an image or voice.

[0093] For example, Figure 7As shown in (A), part or all of the image acquisition unit 120, template acquisition unit 130, template analysis unit 140, search processing unit 160, and search result processing unit 180 described above can be composed of a memory 10 and a processor 11 such as a CPU (Central Processing Unit) that executes programs stored in the memory 10. Such programs can be provided via a network or recorded on a recording medium. In other words, such programs can be provided as program products, for example. In this case, the image processing device 100 can be implemented as a so-called computer.

[0094] In addition, for example Figure 7 As shown in (B), part or all of the image acquisition unit 120, the template acquisition unit 130, the template analysis unit 140, the search processing unit 160 and the search result processing unit 180 can also be composed of a processing circuit 12 such as a single circuit, a composite circuit, a processor that operates using a program, a parallel processor that operates using a program, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0095] As described above, the image acquisition unit 120 , the template acquisition unit 130 , the template analysis unit 140 , the search processing unit 160 , and the search result processing unit 180 can be implemented by a processing circuit network.

[0096] In addition, the input unit 110 can be realized by an interface such as a communication interface.

[0097] Furthermore, the output unit 190 can be realized by an output device such as a display or a speaker.

[0098] Next, the processing of the image processing apparatus 100 according to the first embodiment will be described with reference to the drawings.

[0099] Figure 8 3 is a flowchart showing the expansion process in the template expansion unit 141 .

[0100] First, the template expansion unit 141 determines the expansion method to be implemented (S10). For example, the expansion method to be implemented can be determined in advance or inputted via the input unit 110 or an input device such as a keyboard or mouse (not shown) when the search process is executed by the image processing device 100.

[0101] Next, the template expansion unit 141 selects one expansion method from the determined expansion methods ( S11 ).

[0102] Next, the template expansion unit 141 generates a conversion template from the template according to the selected expansion method (S12). The template and the conversion template constitute a plurality of expansion templates.

[0103] Next, the template expansion unit 141 determines whether all the expansion methods determined in step S10 have been selected (S13). If all expansion methods have been selected (S13: Yes), the process ends. If any expansion methods have not been selected (S13: No), the process returns to step S11. In step S11, one of the expansion methods that has not been selected is selected.

[0104] Figure 9 (A) to (D) are schematic diagrams for explaining the expansion process performed by the template expansion unit 141.

[0105] Here, the case where the expansion method is rotation is taken as an example for description.

[0106] Generally, a rotated pattern cannot be found in template matching, and therefore it is necessary to rotate the template in advance to perform the search process.

[0107] Here, it is set to be input Figure 9 Template 1#1 shown in (A).

[0108] Even if it is input into template 1#1, it cannot be found in Figure 9 The pattern of portion 3#1 in search target image 2#1 shown in (D) is because the position of the same pattern is detected by comparing template 1#1 with the image of portion 3#1 cut out of search target image 2#1 with the same size as template 1#1.

[0109] On the other hand, Figure 9 By searching using a template obtained by rotating template 1#1, as in converted template 1#2 shown in (B), the pattern included in portion 3#1 can be found.

[0110] It is unknown how many degrees the pattern corresponding to the pattern shown in the template 1#1 in the search target image 2#1 exists in. Therefore, the template expansion unit 141 performs the following operations, for example: Figure 9 (B) shows the conversion template 1#2 and Figure 9 As in the conversion template 1#3 shown in (C), a conversion template rotated to multiple angles is generated in advance.

[0111] At this time, if the expansion method specifies that the rotation interval is 1°, 359 conversion templates are generated for each rotation angle for 360°, in addition to the original template 1#1. The template expansion unit 141 then combines these 359 conversion templates with the original template 1#1 and provides the 360 expanded templates to the unique pixel selection unit 142, the search method determination unit 150, and the search processing unit 160.

[0112] In addition, although rotation is used as an example for explanation here, the expansion process is also performed in the expansion method such as enlargement and reduction or deformation. In addition, a plurality of expansion methods such as rotation and enlargement and reduction may be combined.

[0113] Figure 10 : is a flowchart showing the pixel selection process performed by the unique pixel selection unit 142 .

[0114] First, the selected pixel determination unit 143 included in the unique pixel selection unit 142 determines the selected pixels in the original template ( S20 ).

[0115] Next, the coordinate conversion unit 148 selects one conversion template from the plurality of conversion templates ( S21 ).

[0116] Next, the coordinate conversion unit 148 determines a coordinate conversion formula between the coordinates included in the original template and the coordinates included in the selected conversion template based on the conversion method of the selected conversion template ( S22 ).

[0117] Next, the selected pixel calculation unit 149 performs coordinate conversion on the selected pixels determined based on the original template using the coordinate conversion formula determined by the coordinate conversion unit 148, thereby calculating the coordinates of the corresponding selected pixels (S23). The selected pixel calculation unit 149 then generates corresponding selected pixel information indicating the calculated corresponding selected pixels for each of the plurality of conversion templates, and provides the corresponding selected pixel information to the search method determination unit 150 and the search processing unit 160.

[0118] Next, the coordinate conversion unit 148 determines whether all conversion templates have been selected (S24). If all conversion templates have been selected (S24: Yes), the process ends. If any conversion templates remain that have not been selected (S24: No), the process returns to step S21. In step S21, one conversion template that has not been selected is selected.

[0119] Figure 11 (A) to (C) are schematic diagrams for explaining pixel selection.

[0120] Here, the selection of pixels determined based on the template will be described.

[0121] The selected pixel determination unit 143 is based on, for example, Figure 11 The template 1#4 shown in (A) determines Figure 11 The selected pixels P1#1 to P11#1 are shown in (B).

[0122] Then, if Figure 11 As shown in (C), search processing unit 160 cuts out portion 3#2 of the same size as template 1#4 from search target image 2#2. Then, within portion 3#2, search processing unit 160 compares the pixel values of selected pixels P1#1 to P11#1 with the pixel values of pixels P1#2 to P11#2 located at the same coordinates as selected pixels P1#1 to P11#1, thereby detecting pattern identity.

[0123] Figure 12 (A) to (D) are schematic diagrams for explaining corresponding selected pixels determined in a conversion template generated by rotating the template.

[0124] exist Figure 12 (A) shows template 1#5 and selected pixels P1#3 to P11#3 determined based on template 1#5.

[0125] Figure 12 (B) shows a conversion template 1#6 generated by rotating template 1#5 45° clockwise.

[0126] Figure 12 (C) shows the corresponding selected pixels P1#4 to P11#4 calculated according to the conversion method of conversion template 1#6.

[0127] Figure 12 (D) is a diagram in which the corresponding selected pixels P1#4 to P11#4 are overlapped with the conversion template 1#6.

[0128] like Figure 12 As shown in (C), similar to conversion template 1#6, the corresponding selected pixels P1#4 to P11#4 can also be calculated by rotating the selected pixels P1#3 to P11#3 by 45 degrees clockwise.

[0129] Here, when the expansion method is rotation and the template is rotated counterclockwise by an angle θ to generate a conversion template, the coordinate conversion formula calculated by the coordinate conversion unit 148 is as shown in the following formula (1).

[0130]

[0131] In formula (1), (x, y) is the coordinate before coordinate transformation, and is the coordinate in the template. (x', y') is the coordinate after coordinate transformation, and is the coordinate in the transformed template. w is the width of the template, and h is the height of the template.

[0132] Figure 13 3 is a flowchart showing the pixel selection process performed by the selected pixel determination unit 143 .

[0133] First, the pixel relationship analysis unit 144 determines the inter-pixel relationships for pixels within the template (S30). The inter-pixel relationships serve as an indicator for calculating the specificity of each pixel within the template. Specificity indicates that the pixel is a characteristic pixel that meaningfully represents the template pattern. Based on the definition of specificity, for example, pixels at pattern boundaries or corners have high specificity.

[0134] Determining the relationship between pixels can be done, for example, based on co-occurrence probability. This is obtained by dividing each frequency in the co-occurrence histogram by the number of pixels. A co-occurrence histogram calculates the frequency of occurrence of pairs of pixel values p and q, consisting of two pixels in a predetermined positional relationship, across the entire image. The pixel values p and q are then arranged in a two-dimensional matrix with vertical and horizontal indices.

[0135] The frequency h2(p, q) in the index (p, q) of the co-occurrence histogram can be calculated by the following equations (2) and (3).

[0136]

[0137]

[0138] Among them, v P is the position vector of pixel P, v Q is the position vector of pixel Q, f(v) is the pixel value of the pixel located at position vector v, and d is the difference vector of the predetermined position vector. In other words, the positional relationship between pixels P and Q can be said to be always fixed due to the predetermined fixed value vector d.

[0139] In addition, as a general image, if it is an image with a bit depth of 8 bits, the possible pixel values are 0 to 255, so the possible ranges of p and q are also 0 to 255, and the co-occurrence histogram is represented by a square matrix with 256 rows and 256 columns.

[0140] Moreover, v P and v Q ∈ R₁₂₁₂ is the position vector of pixel P and pixel Q. Therefore, the range of the elements of each position vector has 0 as the lower limit and the vertical and horizontal dimensions of the template as the upper limit.

[0141] Figure 14 (A) and (B) are schematic diagrams for explaining the process of calculating the co-occurrence histogram.

[0142] Here, in order to simplify the description, the pixel values are set to 5 gray levels from 0 to 4, as shown in Figure 14 As shown in (A), template 1#7 is set to 49 pixels in 7 rows and 7 columns, and the difference vector d = (1, 0). In other words, the pixel pair (P, Q) is two pixels adjacent in the horizontal direction.

[0143] Figure 14 (B) is a table showing the frequencies of the co-occurrence histogram.

[0144] For example, according to Figure 14 (B), as shown in column 5#1, Figure 14 There are 19 pixel pairs in which the pixel values of the horizontally adjacent pixels are (0, 0), such as the pixel pair 4#1 in (A) in template 1#7.

[0145] Similarly, as shown in column 5#2, there are two pixel pairs having pixel values of (1, 2) such as pixel pair 4#2.

[0146] Here, a pixel pair consisting of two pixels is used as an example, but the same situation applies even when three or more pixel pairs are used in a predetermined positional relationship. For example, when using three pixel pairs in a predetermined positional relationship, the co-occurrence histogram is represented by a three-dimensional matrix. In this embodiment, there is no limit on the number of pixels constituting a pixel pair, and any co-occurrence histogram is applicable.

[0147] Alternatively, the inter-pixel relationship analysis unit 144 may generate multiple different co-occurrence histograms based on multiple different predetermined positional relationships and generate a new histogram by weighted averaging them. In this case, the pixel number determination unit 146 and the pixel selection unit 147 may use this new histogram.

[0148] return Figure 13 Next, the similarity index calculation unit 145 calculates the similarity between the multiple extended templates (S31). For example, when the extension method is rotation, in a flat area with a small rotation angle or no change in pixel value, the change in pixel value caused by rotation is small, and the similarity between the two templates becomes high.

[0149] Next, the selected pixel number determination unit 146 determines the number of selected pixels using the histogram generated by the inter-pixel relationship analysis unit 144 and the similarity calculated by the similarity index calculation unit 145 (S32). Here, the selected pixel number determination unit 146 assigns predetermined weights to the values determined based on the histogram and the values determined based on the similarity, and averages these weights to determine the number of selected pixels. Alternatively, the selected pixel number determination unit 146 may use other methods to determine the number of selected pixels.

[0150] Next, the pixel selection unit 147 determines the pixels to be selected in the template based on the histogram, which is the result of the analysis of the pixel value relationships provided by the inter-pixel relationship analysis unit 144, and the number of selected pixels provided by the selected pixel number determination unit 146 (S33). For example, the pixel selection unit 147 can select pixels in ascending order of co-occurrence probability, excluding 0. A low co-occurrence probability is because the probability of such a pair of pixel values existing in the template is low. In other words, this is because pixel pairs with low co-occurrence probabilities become characteristic pixel pairs.

[0151] Figure 15 3 is a flowchart showing the search process of the search processing unit 160 .

[0152] First, the rough search execution unit 161 selects one expansion method by referring to the rough search method information provided from the search method determination unit 150 ( S40 ).

[0153] Next, the rough search execution unit 161 determines the interval of the rough search in the one expansion mode selected in step S40 according to the rough search mode indicated by the rough search mode information supplied from the search mode determination unit 150 ( S41 ).

[0154] Next, the coarse search execution unit 161 executes a coarse search using the expansion method selected in step S40 at the intervals determined in step S41 (S42). If, as a result of the coarse search, a pattern included in the expansion template is detected in the search target image, the coarse search execution unit 161 provides a numerical value corresponding to the expansion template (e.g., an angle, an expansion / reduction ratio, or a deformation ratio) to the fine search execution unit 162.

[0155] Next, the fine search execution unit 162 determines a search execution range for the fine search based on the results of the coarse search from the coarse search execution unit 161, in accordance with the fine search mode indicated by the fine search mode information provided by the search mode determination unit 150 (S43). Alternatively, if the coarse search execution unit 161 cannot detect the pattern included in the expanded template in the search target image, the fine search execution unit 162 may determine the entire range in the expanded mode as the search execution range. Furthermore, the entire range in the expanded mode may be determined based on the coarse search mode information.

[0156] Next, the fine search execution unit 162 executes a fine search using the corresponding expansion template in the search execution range determined in step S43 ( S44 ).

[0157] The coarse search execution unit 161 then determines whether all expansion modes have been selected (S45). If all expansion modes have been selected (S45: Yes), the process ends. On the other hand, if any expansion modes remain that have not been selected (S45: No), the process returns to step S40, and the coarse search execution unit 161 selects one of the expansion modes that has not been selected.

[0158] Figure 16 (A) and (B) are schematic diagrams illustrating the relationship between coarse search and fine search.

[0159] Here, the case where the expansion method is rotation is taken as an example for description.

[0160] To perform template matching corresponding to rotation, it is necessary to use multiple rotated templates and perform template matching at each angle. For example, if matching is performed every 1°, 360 template matchings are performed.

[0161] Regarding the coarse search and the fine search, first, the coarse search is used to search from a rough perspective, and then the fine search is used to search in detail. Figure 16 In the example shown, Figure 16 As shown in (A), a rough search is performed at intervals of 45°. If the result of the rough search is that the pattern included in the extended template at the position of 45° is roughly consistent with the pattern included in the searched object image, as shown in FIG. Figure 16 As shown in (B), a fine search is performed every 1° within the 45° search range. Here, if the search range is a 50° range centered at the 45° position, 50 fine searches are performed within the 20° to 70° range. In this case, the coarse search is performed eight times, resulting in a total of 58 template matches. This significantly reduces the number of template matches compared to 360 times when the search range is 360°.

[0162] Furthermore, during the search, each extended template is compared with a portion of the search target image cut out of the same size as the template, i.e., a partial image. For example, the sum of the absolute differences between pixel values at the same position is calculated. However, in this embodiment, the comparison operation is performed only on the pixels selected by the unique pixel selection unit 142 and output by the template analysis unit 140. This significantly reduces the amount of computation compared to comparing all pixels.

[0163] Implementation Method 2

[0164] like Figure 1 As shown, the image processing apparatus 200 according to the second embodiment includes an input unit 110 , an image acquisition unit 120 , a template acquisition unit 130 , a template analysis unit 240 , a search processing unit 160 , a search result processing unit 180 , and an output unit 190 .

[0165] The input unit 110, image acquisition unit 120, template acquisition unit 130, search processing unit 160, search result processing unit 180 and output unit 190 of the image processing device 200 of embodiment 2 are the same as the input unit 110, image acquisition unit 120, template acquisition unit 130, search processing unit 160, search result processing unit 180 and output unit 190 of the image processing device 100 of embodiment 1.

[0166] like Figure 2 As shown, the template analysis unit 240 in the second embodiment includes a template expansion unit 141 , a unique pixel selection unit 242 , and a search method determination unit 150 .

[0167] The template expansion unit 141 and the search method determination unit 150 of the template analysis unit 240 in the second embodiment are the same as the template expansion unit 141 and the search method determination unit 150 of the template analysis unit 140 in the first embodiment.

[0168] Figure 17 This is a block diagram schematically showing the configuration of the unique pixel selection unit 242 in the second embodiment.

[0169] The unique pixel selection unit 242 includes a selected pixel determination unit 243. The unique pixel selection unit 242 in the second embodiment does not include the coordinate conversion unit 148 and the selected pixel calculation unit 149 in the unique pixel selection unit 142 in the first embodiment.

[0170] The selected pixel determination unit 243 determines characteristic pixels in the patterns represented by the plurality of extended templates as selected pixels in accordance with a predetermined rule.

[0171] Specifically, when the target extension template is a template, one or more pixels to be selected are selected from the template. On the other hand, when the target extension template is one of a plurality of conversion templates, one or more pixels to be selected are selected from the target extension template.

[0172] Figure 18 3 is a flowchart showing the pixel selection process performed by the selected pixel determination unit 243 .

[0173] First, the selected pixel determination unit 243 selects one extended template from a plurality of extended templates ( S50 ).

[0174] Next, the selected pixel determination unit 243 determines pixels to be selected based on the selected extended template ( S51 ).

[0175] Next, the selected pixel determination unit 243 determines whether all the extended templates have been selected (S52). If all the extended templates have been selected (S52: Yes), the process ends. If any extended templates remain that have not been selected (S52: No), the process returns to step S50, and the selected pixel determination unit 243 selects an extended template that has not been selected.

[0176] While the selected pixel determination unit 143 in Embodiment 1 selects pixels for the original template, the selected pixel determination unit 243 in Embodiment 2 selects pixels for each of the plurality of extended templates.

[0177] In the first and second embodiments described above, the pixel selection unit 147 determines the pixels to be selected in the template based on the histogram, which is the result of the analysis of the relationship between pixel values provided by the inter-pixel relationship analysis unit 144, and the number of selected pixels provided by the selected pixel number determination unit 146. In other words, the pixels to be selected in the template are selected in descending order of the pixels having the lowest co-occurrence probability calculated based on the histogram. However, the first and second embodiments are not limited to this example.

[0178] For example, the similarity index calculation unit 145 calculates the absolute difference between each pixel in a combination of two extended templates selected from the plurality of extended templates, among all combinations selected from the plurality of extended templates, and sums the calculated absolute differences for each pixel to calculate the similarity of each pixel, i.e., pixel similarity. The similarity index calculation unit 145 then provides pixel similarity information indicating the calculated pixel similarity to the pixel selection unit 147.

[0179] Then, the pixel selection unit 147 may determine the pixels to be selected in the template in order from pixels having lower pixel similarity, such that pixels having higher pixel similarity are less likely to be selected.

[0180] Alternatively, a value calculated by summing up the absolute differences of pixels included in a certain region may be used as the similarity of pixels included in the certain region.

[0181] Label Description

[0182] 100, 200: image processing device; 110: input unit; 120: image acquisition unit; 130: template acquisition unit; 140: template analysis unit; 141: template expansion unit; 142, 242: unique pixel selection unit; 143, 243: selected pixel determination unit; 144: inter-pixel relationship analysis unit; 145: similarity index calculation unit; 146: selected pixel number determination unit; 147: pixel selection unit; 148: coordinate conversion unit; 149: selected pixel calculation unit; 150: search method determination unit; 151: coarse search method determination unit; 152: fine search method determination unit; 160: search processing unit; 161: coarse search execution unit; 162: fine search execution unit; 180: search result processing unit; 190: output unit.

Claims

1. An image processing device, characterized in that The image processing device comprises: a template analysis unit that converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values that change the pattern to different degrees, and generates a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively include each of the plurality of conversion patterns; and a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the larger the first division.

2. The image processing device according to claim 1, wherein The first division is predetermined.

3. The image processing device according to claim 1, wherein The higher the similarity of the multiple extended templates, the larger the first division.

4. The image processing device according to any one of claims 1 to 3, wherein The prescribed range for determining the second extended template is predetermined.

5. The image processing device according to any one of claims 1 to 3, wherein The template analysis unit analyzes the plurality of extended templates and determines the predetermined range for determining the second extended template for each numerical value corresponding to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search.

6. The image processing device according to any one of claims 1 to 3, wherein: The second division is predetermined.

7. The image processing device according to any one of claims 1 to 3, wherein In the first search, it is detected whether the searched object image contains the pattern and any one of the plurality of conversion patterns. In the second search, a position of a target pattern that is any one of the pattern and the plurality of conversion patterns in the searched target image and a relationship of the target pattern with respect to the pattern are detected.

8. The image processing device according to claim 7, wherein The relationship is at least any one of a rotation angle, an enlargement / reduction ratio, and a deformation ratio.

9. An image processing device, characterized in that: The image processing device comprises: a template analysis unit that converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values that change the pattern to different degrees, and generates a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively include each of the plurality of conversion patterns; and a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The search processing unit performs the first search or the second search by comparing pixel values of one or more pixels selected from the object extension template with pixel values of pixels of the partial image corresponding to the one or more pixels, wherein the object extension template is an extension template used in the first search or the second search among the plurality of extension templates, and the partial image is cut out of the searched object image with the same size as the object extension template. The number of the one or more pixels is less than the number of all pixels included in the object extension template in the first search and the second search, respectively. The higher the similarity of the multiple extended templates, the smaller the number of the one or more pixels.

10. The image processing device according to claim 9, wherein In the case where the object extension template is one of the multiple conversion templates, a coordinate conversion formula is used to convert the coordinates of the pixels contained in the pattern into the coordinates of the pixels of the conversion pattern contained in the object extension template, and the coordinates of each of the one or more pixels are calculated based on the coordinates of the pixels selected in the template.

11. The image processing device according to claim 9, wherein In the case where the object extension template is the template, selecting the one or more pixels from the template, When the target extension template is one of the plurality of conversion templates, the one or more pixels are selected from the target extension template.

12. An image processing device, characterized in that: The image processing device comprises: a template analysis unit that converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values that change the pattern to different degrees, and generates a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively include each of the plurality of conversion patterns; and a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The template analysis unit analyzes the plurality of extended templates to determine the number of the one or more pixels. The higher the similarity of the multiple extended templates, the smaller the number of the one or more pixels.

13. The image processing device according to claim 12, wherein: In the case where the object extension template is one of the multiple conversion templates, a coordinate conversion formula is used to convert the coordinates of the pixels contained in the pattern into the coordinates of the pixels of the conversion pattern contained in the object extension template, and the coordinates of each of the one or more pixels are calculated based on the coordinates of the pixels selected in the template.

14. The image processing device according to claim 12, wherein: In the case where the object extension template is the template, selecting the one or more pixels from the template, When the target extension template is one of the plurality of conversion templates, the one or more pixels are selected from the target extension template.

15. An image processing device, characterized in that: The image processing device comprises: a template analysis unit that converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values that change the pattern to different degrees, and generates a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively include each of the plurality of conversion patterns; and a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The search processing unit performs the first search or the second search by comparing pixel values of one or more pixels selected from the object extension template with pixel values of pixels of the partial image corresponding to the one or more pixels, wherein the object extension template is an extension template used in the first search or the second search among the plurality of extension templates, and the partial image is cut out of the searched object image with the same size as the object extension template. The number of the one or more pixels is less than the number of all pixels included in the object extension template in the first search and the second search, respectively. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the smaller the number of the one or more pixels.

16. The image processing device according to claim 15, wherein: The one or more pixels are selected in order from pixels having a low co-occurrence probability calculated based on the histogram.

17. The image processing device according to claim 15, wherein: The absolute difference of each pixel in the combination of two extended templates selected from the plurality of extended templates is calculated in all combinations selected from the plurality of extended templates, and the one or more pixels are selected in order from pixels having a low pixel similarity calculated by summing the calculated absolute differences for each pixel.

18. An image processing device, characterized in that: The image processing device comprises: a template analysis unit that converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values that change the pattern to different degrees, and generates a plurality of extended templates consisting of a plurality of conversion templates and the template, wherein the plurality of conversion templates respectively include each of the plurality of conversion patterns; and a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The template analysis unit analyzes the plurality of extended templates to determine the number of the one or more pixels. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the smaller the number of the one or more pixels.

19. The image processing device according to claim 18, wherein The one or more pixels are selected in order from pixels having a low co-occurrence probability calculated based on the histogram.

20. The image processing device according to claim 18, wherein The absolute difference of each pixel in the combination of two extended templates selected from the plurality of extended templates is calculated in all combinations selected from the plurality of extended templates, and the one or more pixels are selected in order from pixels having a low pixel similarity calculated by summing the calculated absolute differences for each pixel.

21. A computer-readable recording medium storing a program for causing a computer to function as: A template analysis unit converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values of different degrees of change, and generates a plurality of extended templates consisting of the plurality of conversion templates and the template, wherein The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; It is characterized by: The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the larger the first division.

22. A computer-readable recording medium storing a program for causing a computer to function as: A template analysis unit converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values of different degrees of change, and generates a plurality of extended templates consisting of the plurality of conversion templates and the template, wherein The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; It is characterized by: The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The search processing unit performs the first search or the second search by comparing pixel values of one or more pixels selected from the object extension template with pixel values of pixels of the partial image corresponding to the one or more pixels, wherein the object extension template is an extension template used in the first search or the second search among the plurality of extension templates, and the partial image is cut out of the searched object image with the same size as the object extension template. The number of the one or more pixels is less than the number of all pixels included in the object extension template in the first search and the second search, respectively. The higher the similarity of the multiple extended templates, the smaller the number of the one or more pixels.

23. A computer-readable recording medium storing a program for causing a computer to function as: A template analysis unit converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values of different degrees of change, and generates a plurality of extended templates consisting of the plurality of conversion templates and the template, wherein The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; It is characterized by: The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The template analysis unit analyzes the plurality of extended templates to determine the number of the one or more pixels. The higher the similarity of the multiple extended templates, the smaller the number of the one or more pixels.

24. A computer-readable recording medium storing a program for causing a computer to function as: A template analysis unit converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values of different degrees of change, and generates a plurality of extended templates consisting of the plurality of conversion templates and the template, wherein The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; It is characterized by: The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The search processing unit performs the first search or the second search by comparing pixel values of one or more pixels selected from the object extension template with pixel values of pixels of the partial image corresponding to the one or more pixels, wherein the object extension template is an extension template used in the first search or the second search among the plurality of extension templates, and the partial image is cut out of the searched object image with the same size as the object extension template. The number of the one or more pixels is less than the number of all pixels included in the object extension template in the first search and the second search, respectively. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the smaller the number of the one or more pixels.

25. A computer-readable recording medium storing a program for causing a computer to function as: A template analysis unit converts the pattern included in the template into a plurality of conversion patterns using a plurality of numerical values of different degrees of change, and generates a plurality of extended templates consisting of the plurality of conversion templates and the template, wherein The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as a search processing unit that performs a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performs a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search; It is characterized by: The search processing unit sets a plurality of conversion templates corresponding to a plurality of numerical values selected by the first division from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first expanded templates, The search processing unit sets as the plurality of second extended templates a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range through a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns were detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. The template analysis unit analyzes the plurality of extended templates to determine the number of the one or more pixels. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the smaller the number of the one or more pixels.

26. An image processing method, The pattern included in the template is converted into a plurality of conversion patterns using a plurality of numerical values having different degrees of change, and a plurality of extended templates consisting of the plurality of conversion templates and the template are generated, wherein: The plurality of conversion templates respectively include each conversion pattern of the plurality of conversion patterns, performing a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performing a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search, It is characterized by: a plurality of conversion templates corresponding to a plurality of numerical values selected by first partitioning from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first extended templates; The plurality of second extended templates are configured as a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range by a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns can be detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the larger the first division.

27. An image processing method, The pattern included in the template is converted into a plurality of conversion patterns using a plurality of numerical values having different degrees of change, and a plurality of extended templates consisting of the plurality of conversion templates and the template are generated, wherein: The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as performing a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performing a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search, It is characterized by: a plurality of conversion templates corresponding to a plurality of numerical values selected by first partitioning from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first extended templates; The plurality of second extended templates are configured as a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range by a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns can be detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. performing the first search or the second search by comparing pixel values of one or more pixels selected from an object extended template with pixel values of pixels of a partial image corresponding to the one or more pixels, wherein the object extended template is an extended template among the plurality of extended templates used in the first search or the second search, and the partial image is cut out of the searched object image with the same size as the object extended template, The number of the one or more pixels is less than the number of all pixels included in the object extension template in the first search and the second search, respectively. The higher the similarity of the multiple extended templates, the smaller the number of the one or more pixels.

28. An image processing method, The pattern included in the template is converted into a plurality of conversion patterns using a plurality of numerical values having different degrees of change, and a plurality of extended templates consisting of the plurality of conversion templates and the template are generated, wherein: The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as performing a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performing a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search, It is characterized by: a plurality of conversion templates corresponding to a plurality of numerical values selected by first partitioning from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first extended templates; The plurality of second extended templates are configured as a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range by a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns can be detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. Analyze the multiple extended templates to determine the number of the one or more pixels, The higher the similarity of the multiple extended templates, the smaller the number of the one or more pixels.

29. An image processing method, The pattern included in the template is converted into a plurality of conversion patterns using a plurality of numerical values having different degrees of change, and a plurality of extended templates consisting of the plurality of conversion templates and the template are generated, wherein: The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as performing a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performing a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search, It is characterized by: a plurality of conversion templates corresponding to a plurality of numerical values selected by first partitioning from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first extended templates; The plurality of second extended templates are configured as a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range by a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns can be detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. performing the first search or the second search by comparing pixel values of one or more pixels selected from an object extended template with pixel values of pixels of a partial image corresponding to the one or more pixels, wherein the object extended template is an extended template among the plurality of extended templates used in the first search or the second search, and the partial image is cut out of the searched object image with the same size as the object extended template, The number of the one or more pixels is less than the number of all pixels included in the object extension template in the first search and the second search, respectively. In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the smaller the number of the one or more pixels.

30. An image processing method, The pattern included in the template is converted into a plurality of conversion patterns using a plurality of numerical values having different degrees of change, and a plurality of extended templates consisting of the plurality of conversion templates and the template are generated, wherein: The plurality of conversion templates respectively include respective conversion patterns of the plurality of conversion patterns; as well as performing a first search for a searched object image using a plurality of first extended templates selected from the plurality of extended templates, and performing a second search for the searched object image using a plurality of second extended templates selected from the plurality of extended templates based on a result of the first search, It is characterized by: a plurality of conversion templates corresponding to a plurality of numerical values selected by first partitioning from a set of the plurality of numerical values arranged in ascending or descending order and the template as the plurality of first extended templates; The plurality of second extended templates are configured as a plurality of conversion templates corresponding to a plurality of values selected from a predetermined range by a second division finer than the first division and including the following one value in the set, wherein the one value corresponds to one or more first extended templates in which the pattern and the plurality of conversion patterns can be detected in the first search; The number of numerical values included in the predetermined range is less than the number obtained by subtracting the number of numerical values corresponding to the first extended template from the number of numerical values included in the set. Analyze the multiple extended templates to determine the number of the one or more pixels, In the histogram of the relationship between the pixel values of a plurality of pixels in a predetermined positional relationship within the template, the smaller the frequency variation, the smaller the number of the one or more pixels.

Images