Image analysis device, image processing device, image processing system, image processing method, and image processing program

Abstract

An image analysis device (110) is a device for generating template information (13) from a template image (11), and is provided with: a feature amount calculation unit (111) for calculating a feature amount for each pixel of the template image (11); and an intra-block selected pixel determination unit (112) that divides the template image (11) into a plurality of blocks on the basis of the block division information (12), and determines, on the basis of the feature amount, selected pixels that are pixels showing features in each block. An image processing device (120) is provided with: a selected pixel extraction unit (121) that extracts selected pixels on the basis of template information (13) and block division information (12) output from an image analysis device (110); a comparison calculation unit (122) that compares a pixel value of a selected pixel of the template image (11) with a pixel value of a position corresponding to the selected pixel of the input image (14), and that calculates a degree of similarity on the basis of the result of the comparison; and a matching calculation unit (123) that performs template matching on the basis of the similarity.

Description

Technical Field

[0001] The present invention relates to an image analysis device, an image processing device, an image processing system, an image processing method, and an image processing program. Background Art

[0002] As a method for detecting a specific pattern from an image, template matching is widely known. Template matching is a method of comparing a template representing a pattern to be detected, which is prepared in advance, with each part of a search target image, and detecting a part of the search target image that is most similar to the template as the specific pattern represented by the template.

[0003] In addition, in template matching, a template is expanded into a plurality of templates by rotating, enlarging / reducing, and deforming the template, and all of the expanded plurality of templates are compared with each part of the search target image, so that not only detection of a pattern identical to the template can be achieved, but also detection of a pattern that has been rotated, enlarged / reduced, or deformed due to the composition of the image captured can be achieved. However, such template matching has a problem of becoming a process with a very large amount of computation. Patent Document 1 proposes an apparatus and method for reducing the amount of computation by limiting pixels used for comparison operations and using a two-stage search of coarse and fine.

[0004] Patent Document 1: Japanese Patent No. 7118295 Summary of the Invention

[0005] However, in the case of the apparatus described in Patent Document 1, the pixels used for template matching differ depending on the template, and a part of the template matching includes a complex process accompanied by branching. Therefore, there is a problem that parallel processing is difficult when constructing a hardware circuit.

[0006] The present invention has been made in view of the above circumstances, and an object thereof is to achieve template matching with high parallelism.

[0007] The image analysis device of the present invention generates template information from a template image, the template information including pairs of positions and pixel values of pixels used for matching for performing template matching. The image analysis device is characterized by including: a feature amount calculation unit that calculates a feature amount of each pixel of the template image; and an intra-block selection pixel determination unit that divides the template image into a plurality of blocks based on block division information and determines, based on the feature amount, a pixel showing a feature in each block, that is, a selection pixel.

[0008] The image processing apparatus of the present invention is characterized by including: a selected pixel extraction unit that extracts the selected pixels based on the template information and the block segmentation information output from the image analysis apparatus; a comparison operation unit that receives the input image, the template information, and the block segmentation information, compares the pixel values of the selected pixels of the template image with the pixel values at the positions corresponding to the selected pixels of the input image, and calculates a similarity based on the result of the comparison; and a matching operation unit that performs template matching based on the similarity and outputs the result of the template matching.

[0009] Effects of the Invention

[0010] According to the present invention, it is possible to achieve template matching with a high degree of parallelism. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram schematically showing the configuration of the image processing system according to Embodiment 1.

[0012] Figure 2 (A) and (B) are diagrams showing examples of the hardware configuration of the image processing system.

[0013] Figure 3 is a diagram showing an outline of template matching.

[0014] Figure 4 is a diagram showing an outline of template matching accompanied by pixel selection.

[0015] Figure 5 is a block diagram schematically showing the configuration of the image analysis apparatus according to Embodiment 1.

[0016] Figure 6 is a diagram showing an example of block segmentation of a template image.

[0017] Figure 7 is a flowchart showing the processing performed by the image analysis apparatus according to Embodiment 1.

[0018] Figure 8 is a block diagram schematically showing the configuration of the image processing apparatus according to Embodiment 1.

[0019] Figure 9 is a flowchart showing the processing performed by the image processing apparatus according to Embodiment 1.

[0020] Figure 10 is a block diagram schematically showing the configuration of the image processing system according to Embodiment 2.

[0021] Figure 11 is a block diagram schematically showing the configuration of the image analysis apparatus according to Embodiment 2.

[0022] Figure 12 This is a flowchart (part 1) showing the processing performed by the image analysis device according to Embodiment 2.

[0023] Figure 13 This is a flowchart (part 2) showing the processing performed by the image analysis device according to Embodiment 2.

[0024] Figure 14 This is a block diagram schematically showing the structure of the image processing device according to Embodiment 2.

[0025] Figure 15 This is a flowchart (part 1) showing the processing performed by the image processing device according to Embodiment 2.

[0026] Figure 16 This is a flowchart (part 2) showing the processing performed by the image processing device according to Embodiment 2. Detailed Embodiment

[0027] Hereinafter, while referring to the drawings, an image analysis device, an image processing device, an image processing system, an image processing method, and an image processing program according to the embodiment will be described. The following embodiments are merely examples, and the embodiments can be appropriately combined and the respective embodiments can be appropriately modified.

[0028] 《1》Embodiment 1

[0029] 《1-1》Image Processing System 100

[0030] Figure 1 This is a block diagram schematically showing the structure of the image processing system 100 according to Embodiment 1. The image processing system 100 can execute the image processing method according to Embodiment 1. As Figure 1 shown, the image processing system 100 includes: an image analysis device 110 that takes the template image 11 as input and outputs template information 13 and block segmentation information 12; and an image processing device 120 that takes the input image 14, template information 13, and block segmentation information 12 as input and outputs the result of template matching, i.e., the matching result 15. The template information 13 is information including pairs of the positions and pixel values of the pixels used for matching in template matching based on the template image 11. The input image 14 is the image to be searched, and the image processing device 120 detects the part of the input image 14 that is most similar (i.e., has the highest similarity) to the template image 11 as the specific pattern represented by the template. In addition, the block segmentation information 12 may be included as part of the template information 13.

[0031] Figure 2 (A) and (B) are diagrams showing examples of the hardware structure of the image processing system 100 according to Embodiment 1. AsFigure 2 As shown in (A), each part constituting the image processing system 100 is implemented by, for example, a processing circuit 101. The processing circuit 101 can be dedicated hardware, or it can also include a CPU (Central Processing Unit) which is a processor that executes a program stored in a memory. When the processing circuit 101 is dedicated hardware, the processing circuit 101 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel-programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or any combination thereof.

[0032] Alternatively, as Figure 2 shown in (B), each part constituting the image processing system 100 is implemented by a memory 103 which is a storage device (including a storage medium) and a processor 102 such as a CPU that reads and executes the image processing program, and the storage device stores a program (for example, the image processing program related to Embodiment 1) which is software. In this case, the image processing system 100 is, for example, a computer. In addition, the memory 103 is, for example, a semiconductor memory such as a RAM (Random Access Memory), a magnetic disk, etc. Furthermore, the image processing system 100 can also be Figure 2 the structure including the processing circuit 101 in (A) and Figure 2 the structure having the processor 102 and the memory 103 in (B) mixed. Additionally, the image analysis device 110 and the image processing device 120 can be a common hardware circuit (or a common computer), but the image analysis device 110 and the image processing device 120 can also be separate hardware circuits (or separate computers).

[0033] Figure 3 is a diagram showing an overview of template matching. As Figure 3As shown, the image processing system 100 performs template matching for the template image 11 and the input image 14 to detect the position of the pattern identical to the template image 11 within the input image 14. The principle of template matching is to cut out image regions at each position of the same size as the template image 11 from the input image 14, compare the cut-out image regions (e.g., cut-out regions 141 to 143) with the template image 11, and detect the cut-out region most similar to the template image 11 (i.e., with the highest similarity) as the matching result 15. The degree of similarity between the template image 11 and the cut-out regions 141 to 143, for example, can be defined by the sum of the squares of the differences between the pixel values of each pixel of the template image 11 and the pixel values of each pixel of the cut-out regions 141 to 143. The smaller this sum (i.e., the difference degree), the higher the similarity, and in the case of complete coincidence, this sum is 0. That is, the image processing system 100 detects the position with the smallest sum (i.e., the position with the smallest difference degree), namely the matching position, as the matching result 15. The coordinates (x, y) representing the matching position are defined by the following formula (1). In addition, the position with the smallest difference degree is the position with the highest similarity.

[0034] [Formula 1]

[0035]

[0036] In formula (1), T(i, j) represents the pixel value of the pixel at the coordinates (i, j) of the template image 11, and I(x + i, y + j) represents the pixel value of the pixel at the coordinates (x + i, y + j) of the cut-out region.

[0037] The value of the sum on the right side of formula (1) is called the sum of squared differences. However, the formula for calculating the coordinates (x, y) representing the matching position is not limited to formula (1). For example, the formula for calculating the coordinates (x, y) representing the matching position can also be defined by other formulas that use the normalized sum of squared differences, cross-correlation, normalized cross-correlation, correlation coefficient, or normalized correlation coefficient instead of the sum of squared differences.

[0038] In addition, in Figure 3 , for simplicity, three image regions are shown as the cut-out regions 141 to 143, but the number of cut-out regions is not limited to 3.

[0039] Figure 4 is a diagram showing the outline of template matching accompanied by pixel selection. The similarity, as Figure 3 shown, can be defined using all the pixels within the template image 11, but it is not necessary to use all the pixels within the template image 11 for definition. The similarity can also be defined in such a way that it is calculated using only a pre-selected part of the pixels within the template image 11. For example, as Figure 4As shown, pixels used for calculating similarity can also be selected in advance from the template image 11 as selected pixels 11a to 11e, and the similarity is calculated using the pixel values of the pixels 11a to 11e within the template image 11 and the pixel values of the pixels at the same positions (also referred to as "selected pixels") 141a to 141e, 142a to 142e, 143a to 143e in the cut-out regions 141 to 143. When calculating the matching position using the squared difference of the selected pixels by the same method as in Equation (1), the calculation formula for the coordinates (x, y) representing the matching position is defined as Equation (2) below.

[0040] [Equation 2]

[0041]

[0042] Here, the function f(i, j) is 1 when the pixel at the coordinates (i, j) is a selected pixel, and 0 otherwise.

[0043] In addition, in Figure 4 , for simplicity, an example of selecting 5 pixels as selected pixels is shown, but the number of selected pixels and the positions of the selected pixels are not limited to the Figure 4 example. As selected pixels, it is generally desirable to select pixels that appropriately represent the characteristics of the template image 11. The selected pixels are, for example, pixels with a large gradient value of the pixel value, pixels at the corners of the cut-out region, pixels selected by feature quantities (e.g., feature quantities such as SIFT, SURF, AKAZE), pixels selected by co-occurrence probability, etc. In addition, the template image 11 is not limited to one image and may include multiple images. For example, the template image 11 may also include one or more basic template images (i.e., basic template images), one or more rotated template images obtained by rotating the basic template image, one or more scaled template images obtained by enlarging or reducing the basic template image, one or more deformed template images obtained by deforming the basic template image, and one or more template images obtained by performing two or more of the processes of rotation, scaling, and deformation on the basic template image.

[0044] 《1-2》 Image analysis device 110

[0045] As Figure 1 shown, the image analysis device 110 analyzes the input template image 11, determines the pixels used for calculating similarity, i.e., the selected pixels, and outputs information associating the positions of the selected pixels with the pixel values as the template information 13.

[0046] Figure 5 is a block diagram schematically showing the structure of the image analysis device 110 according to Embodiment 1. AsFigure 5 As shown in Figure 5 , the image analysis device 110 includes a feature amount calculation unit 111 and an in-block selected pixel determination unit 112. The feature amount calculation unit 111 takes the template image 11 as input, calculates the feature amount of each pixel of the template image 11, and outputs it. The in-block selected pixel determination unit 112 takes the feature amount of each pixel of the template image 11 as input, determines the selected pixel for each block segmented according to a pre-determined block segmentation method, i.e., block segmentation information 12 or block segmentation information 12 input from the outside, associates the position and pixel value of the selected pixel, and outputs it as template information 13.

[0047] Figure 6 FIG. is an example showing the block segmentation of the template image 11. Figure 6 FIG. is an example showing the template image 11 segmented into a plurality of blocks 1101a to 1101n. In Figure 6 this example, each block is a rectangular area with a vertical dimension of 1 pixel and a horizontal dimension corresponding to the width of the template image 11. In Figure 6 this example, the selected pixels (e.g., selected pixels 1102a to 1102n) are extracted one by one from each block. That is, the block segmentation information 12 included in advance or input from the outside includes the shape and configuration of each block, the number of selected pixels extracted from each block (i.e., the number of selected pixels), and the rule for extracting the selected pixels. In addition, the number of selected pixels extracted from each block and the rule for extracting the selected pixels may not be included in the block segmentation information 12, but may be determined by the in-block selected pixel determination unit 112. In Figure 6 this example, the shape of each block is a rectangle with a vertical dimension of 1 pixel and a horizontal dimension equal to the width of the template image 11, the configuration of the plurality of blocks is arranged in a column along the vertical direction, the number of selected pixels in each block is 1, and the selection rule for the selected pixels is a rule for extracting the selected pixels from the corners or boundary parts of each block. However, the shape and configuration of each block, the selection rule for the selected pixels, and the number of selected pixels are not limited to the above examples. In addition, the plurality of blocks obtained by block segmentation in accordance with the block segmentation information 12 may also have adjacent blocks overlapping each other (i.e., may have overlapping regions). In addition, the area of each block may be different, or the area of each block may be changed according to the position within the template image. For example, the area of the block may be smaller closer to the center of the image and larger farther from the center of the image (i.e., closer to the outside of the image).

[0048] As Figure 5As shown, the template information 13 output from the in-block pixel selection determination unit 112 is stored in the information storage unit 130 serving as a storage device. The information storage unit 130 may be a part of the image processing system 100 or the image analysis device 110. The information storage unit 130 may also be a part of another device (e.g., a server on a network) that can communicate with the image processing system 100 or the image analysis device 110.

[0049] Figure 7 is a flowchart showing the processing executed by the image analysis device 110 according to Embodiment 1. First, the feature amount calculation unit 111 reads in the template image 11 (step S101) and selects a pixel to be processed (step S102). There is no limitation on the pixel to be processed first, but generally, the pixel at the upper left corner of the template image 11 is selected as the pixel to be processed first.

[0050] The feature amount calculation unit 111 calculates the feature amount of the selected pixel (step S104). Next, a pixel to be processed is selected (step S105). There is no limitation on the method of selecting the next pixel, but generally, the next pixel is selected in the order of cluster scanning. The calculated feature amount can be arbitrary. For example, it is a gradient value of a pixel value, a corner index, an SIFT feature amount, a SURF feature amount, an AKAZE feature amount, a co-occurrence probability, etc. The processing of steps S104 to S105 is repeatedly executed until all the pixels in the template image 11 are processed (step S103). That is, the processing of steps S103 to S105 is executed by the feature amount calculation unit 111.

[0051] Next, the in-block pixel selection determination unit 112 reads in the block segmentation information 12 (step S106). The block segmentation information 12 may be stored in advance inside the image analysis device 110 or may be input from the outside. The in-block pixel selection determination unit 112 determines the number of selected pixels within each block, that is, the selection pixel number, based on the block segmentation information 12 for each block (step S107) (step S108), sorts the pixels within the block by the feature amount (step S109), and sequentially selects pixels in the amount of the selection pixel number starting from the pixels with a higher feature amount ranking (step S110). The selection pixel number may be a predetermined number or may be a number calculated based on the block segmentation information 12 (e.g., at least one of the shape, the number of pixels, the configuration, and other rules of each block).

[0052] The in-block pixel selection determination unit 112 may, for example, determine the selection pixel number to be a number proportional to the size of the block. Alternatively, the selection pixel number may also be determined by inversely calculating in units of blocks according to the circuit scale allowed by the hardware installed as the image analysis device 110. In addition, although the selection pixel number is calculated in units of blocks, the selection pixel numbers of each block may all be the same number.

[0053] Alternatively, there may be a block that does not select the selected pixels (i.e., a block with 0 selected pixels) among all the blocks. Also, there may be a block that does not select the selected pixels and a block that selects the selected pixels among all the blocks, and the number of selected pixels in the block that selects the selected pixels is all the same. Or, there may be a block that does not select the selected pixels and a block that selects the selected pixels among all the blocks, and the number of selected pixels in the block that selects the selected pixels is set to the minimum number of selected pixels and its integer multiples.

[0054] In addition, the pixel selection (step S110) is not limited to simply selecting pixels in the order of the pixels with higher feature quantity rankings by the amount of the number of selected pixels. In pixel selection, the feature quantity selection pixels determined based on the feature quantity and their surrounding pixels, i.e., the surrounding pixels, may also be set as the selected pixels. For example, in pixel selection, the feature quantity selection pixels with higher feature quantity rankings and their adjacent surrounding pixels may be set as the selected pixels. In addition, in pixel selection, the feature quantity selection pixels with higher feature quantity rankings and the surrounding pixels at a certain distance from the pixels may also be set as the selected pixels. In this way, when there are surrounding pixels, the number of selected pixels is the total number of the number of feature quantity selection pixels and the number of surrounding pixels.

[0055] If more specifically illustrated, when the feature quantity selection pixels determined based on the feature quantity and the 4 surrounding pixels above, below, left, and right adjacent to them are set as the selected pixels, these 5 pixels are the selected pixels.

[0056] In addition, in other illustrations, when the feature quantity selection pixels determined based on the feature quantity and the pixels at a distance of 4 pixels to the right of the pixels, i.e., the surrounding pixels, are set as the selected pixels, these 2 pixels are the selected pixels.

[0057] As illustrated above, by setting restrictions on the number of selected pixels for each block and the positional relationship of the selected pixels, when the image analysis device 110 is composed of a hardware circuit, a circuit for extracting the selected pixels can be efficiently constructed.

[0058] Finally, the in-block selected pixel determination unit 112 outputs the respective pixel values in association with the coordinates as the template information 13 for all the pixels selected in each block (step S111).

[0059] 《1-3》Image processing device 120

[0060] Figure 8 is a block diagram schematically showing the structure of the image processing device 120 according to Embodiment 1. As Figure 8As shown, the image processing device 120 includes a selected pixel extraction unit 121, a comparison operation unit 122, and a matching operation unit 123. The selected pixel extraction unit 121 receives the input of the template information 13 from the information storage unit 130, and also receives the input of the input image 14, and outputs the pixel information of the template image 11 used for the matching operation and the pixel information of the input image 14. The comparison operation unit 122 receives the pixel information output by the selected pixel extraction unit 121 and the input of the template information 13 from the information storage unit 130, and performs a comparison operation on the pixel values between the input image 14 and the template information 13. The matching operation unit 123 receives the operation result of the comparison operation unit 122, calculates the position in the input image 14 that is most similar to the template image 11, and outputs the coordinates of this position as the matching result 15.

[0061] In addition, the selected pixel extraction unit 121 may also be implemented by a pixel selection hardware circuit that is prepared in a predetermined block division manner used in the image analysis device 110 or according to the block division information 12 from the outside.

[0062] Figure 9 It is a flowchart showing the processing executed by the image processing device 120 according to Embodiment 1. First, the selected pixel extraction unit 121 reads in the input image 14, the template information 13, and the block division information 12 (steps S201, S202). Next, the selected pixel extraction unit 121 sets the reference position to the coordinates (0, 0) (step S203), extracts the pixel values and their position information of the pixels selected from the template image 11 included in the template information 13, and extracts the pixels at the position in the input image 14 where the relative distance from the reference position is the same as the position information included in the template information 13 as the selected pixels in association with the pixels selected from the template image 11 corresponding to the position information (step S205).

[0063] For example, if Figure 4 is taken as an example for explanation, when the reference position is set to the pixel 11x in the input image 14, the pixels corresponding to the selected pixels 11a to 11e in the template image 11 are selected from the input image 14 in such a way that the relative positions with respect to the reference position correspond to the positions of the selected pixels 11a to 11e. Therefore, these pixels become Figure 4 the pixels 143a to 143e in. That is, the method of setting the reference position to the pixel 11x can also be said to be a process of comparing the similarity between the region 142 obtained by cutting out from the input image 14 a shape identical to the template image 11 starting from the pixel 11x at the reference position and the template image 11.

[0064] Return Figure 9, the comparison operation unit 122 sets the difference degree of this reference position to 0 in advance as the initial value (step S206), calculates the difference between the pixel value of the pixel selected from the template image 11 and the pixel value of the selected pixel extracted from the input image 14 associated with this pixel, and adds the calculated difference to the difference degree (step S208). The comparison operation unit 122 calculates the difference degree at this reference position by performing this addition process for all the selected pixels (steps S207, S208). Here, the calculated difference degree is the sum of the differences between pixels. Therefore, the smaller the difference degree, the more similar the image patterns are (that is, the smaller the difference degree, the higher the similarity). In addition, the difference mentioned here can be the absolute value of the difference between two pixel values, or the square of the difference between two pixel values. And although the difference degree shown in step S208 is the sum of the differences, it can also be defined by cross-correlation, normalized cross-correlation, correlation coefficient, normalized correlation coefficient, etc. instead of the difference.

[0065] While updating the reference position (step S209), the comparison operation unit 122 calculates the difference degree calculated in this way for all positions within the input image 14 (step S204), and the matching operation unit 123 outputs the reference position with the smallest difference degree as the matching result 15 of the pattern matching (step S210).

[0066] In addition, the method of updating the reference position is usually an update in the cluster scan order, but other update methods can also be adopted. Also, when using cross-correlation, normalized cross-correlation, correlation coefficient, normalized correlation coefficient, etc. instead of the difference operation, the higher this value means the more similar the image patterns are. Therefore, in this case, the difference degree has the same property as the similarity.

[0067] 《1-4》Effect

[0068] According to Embodiment 1, by adjusting the number of selected pixels for each block, the required processing operation amount can be set. Therefore, even when the image processing system 100 is constituted by a hardware circuit, the block division information and the number of selected pixels can be determined in consideration of the operation speed and parallelism that can be achieved by the hardware circuit.

[0069] 《2》Embodiment 2

[0070] 《2-1》Image Processing System 200

[0071] Figure 10 is a block diagram schematically showing the structure of the image processing system 200 according to Embodiment 2. The image processing system 200 can execute the image processing method according to Embodiment 2. Regarding Embodiment 2, the description will be centered on the differences from the above-mentioned Embodiment 1. As Figure 10As shown, the image processing system 200 has: an image analysis device 210 that takes the template image 11 as input and outputs template information 23 and block segmentation information 22; and an image processing device 220 that takes the input image 14, template information 23, and block segmentation information 22 as input and outputs the result of template matching, i.e., the matching result 15. The template information 23 is information containing pairs of the positions and pixel values of the pixels used for matching in template matching from the template image 11. The input image 14 is the image to be searched, and the image processing device 220 detects the part of the input image 14 that is most similar (i.e., has the highest similarity) to the template image 11 as the specific pattern represented by the template. In addition, an example of the hardware structure of the image processing system 200 is the same as that shown in Figure 2 (A) and (B). Additionally, the template image 11 is not limited to one image and may also include multiple images. Similar to the case of Embodiment 1, the template image 11 may also include at least one basic template image, at least one rotated template image, at least one scaled template image, at least one deformed template image, and at least one template image obtained by performing at least two of rotation, scaling, and deformation on the basic template image.

[0072] 《2-2》Image Analysis Device 210

[0073] Figure 11 is a block diagram schematically showing the structure of the image analysis device 210 according to Embodiment 2. In Figure 11 , the same or corresponding structures as those shown in Figure 5 are labeled with the same reference numerals as those shown in Figure 5 . The image analysis device 210 according to Embodiment 2 further has a selection pixel temporary selection unit 211 and a block segmentation determination unit 212, which is different from the image analysis device 110 according to Embodiment 1 in this regard.

[0074] The selection pixel temporary selection unit 211 receives the feature amounts of the respective pixels of the template image 11 from the feature amount calculation unit 111 and selects a predetermined number of temporary selection pixels from the entire template image 11. In addition, the temporary selection pixels selected by the selection pixel temporary selection unit 211 have no relation to the selection pixels selected by the in-block selection pixel determination unit 112 from each block. The block segmentation determination unit 212 determines the block segmentation method based on the distribution of the temporary selection pixels selected by the selection pixel temporary selection unit 211 and outputs the block segmentation information 22, which represents the block segmentation method, to the in-block selection pixel determination unit 112 and the information storage unit 130. In Embodiment 1, the in-block selection pixel determination unit 112 uses a predetermined block segmentation method or receives the input of the block segmentation information 12 from the outside, but in Embodiment 2, the block segmentation information 22 output by the block segmentation determination unit 212 is used instead of the method of Embodiment 1.

[0075] Figure 12 and Figure 13 are flowcharts (Part 1, Part 2) showing the processing performed by the image analysis device 210 according to Embodiment 2. In Figure 12 and Figure 13 , steps that are the same as or corresponding to the steps shown in Figure 7 are labeled with the same labels as those shown in Figure 7 .

[0076] As shown in Figure 12 and Figure 13 , in addition to the processing of the image analysis device 210 according to Embodiment 1, the image analysis device 210 according to Embodiment 2 performs the following processing. After calculating the feature amounts of each pixel (steps S103 to S105) executed by the feature amount calculation unit 111, the pixel temporary selection unit 211 determines the number of selection pixels of the temporary selection pixels to be selected (step S301). After sorting all the pixels in the template image 11 by the feature amounts (step S302), the temporary selection pixels of the amount of the number of selection pixels are sequentially selected starting from the pixels with higher feature amount rankings (step S303). The number of selection pixels of the temporary selection pixels can be a predetermined value, can be determined according to the size of the template image 11, can be calculated from the feature amounts of each pixel calculated by the feature amount calculation unit 111, or can also be determined by a combination of these methods.

[0077] In addition, similar to the in-block selection pixel determination unit 112 in Embodiment 1 when selecting pixels in each block, the pixel temporary selection unit 211 may not simply sequentially select the pixels with higher feature amount rankings as the temporary selection pixels, but may also select the pixels that combine the pixels with higher feature amount rankings and the surrounding pixels of the pixels as the temporary selection pixels. The surrounding pixels are, for example, the pixels adjacent to the pixels with higher feature amount rankings or the pixels located at a certain distance from the pixels with higher feature amount rankings, etc.

[0078] The block division determination unit 212 determines the block division method based on the temporarily selected pixels selected in this way (step S304) and outputs it as the block division information 22 (step S305). The block division method shown in the block division information 22 is, for example, as shown in Figure 6As shown, the shape of each block is a rectangle with a longitudinal dimension of 1 pixel and a transverse dimension equal to the width of the template image 11, and the configuration of multiple blocks is a configuration arranged in a single column longitudinally. The method for determining the block division method in the block division determination unit 212, for example, complies with one or more of the following conditions: the number of temporary selected pixels selected by the pixel temporary selection unit 211 within each block is the same; the number of temporary selected pixels within each block is the minimum value and its constant multiple (e.g., integer multiple); the maximum value of the number of temporary selected pixels within each block does not exceed a predetermined value; the variance of the positions of the pixels selected by the pixel temporary selection unit 211 within each block becomes larger or smaller. In addition, the block division method, for example, may also comply with one or more of the following methods: dividing in such a way that the longitudinal width of each block is the same; dividing in such a way that the transverse width of each block is the same; dividing in such a way that the variance of the areas of each block becomes smaller. In addition, the multiple blocks obtained by dividing according to the block division method determined by the block division determination unit 212 may also be such that adjacent blocks overlap each other (i.e., may have overlapping regions). In addition, the areas of each block may be different, or the area of each block may be changed according to the position within the template image. For example, the area of the block may be smaller the closer it is to the center of the image, and larger the farther it is from the center of the image (i.e., the closer it is to the outer side of the image).

[0079] 《2-3》Image Processing Apparatus 220

[0080] Figure 14 is a block diagram schematically showing the structure of the image processing apparatus 220 according to Embodiment 2. In Figure 14 the same or corresponding structures as those shown in Figure 8 are labeled with the same reference numerals as those shown in Figure 8 The image processing apparatus 220 according to Embodiment 2 further includes a pixel selection circuit construction unit 221, which is different from the image processing apparatus 120 according to Embodiment 1 in this regard.

[0081] The image processing apparatus 220 includes: a selected pixel extraction unit 121 that extracts selected pixels based on template information 23 and block division information 22 output from the image analysis apparatus 210; a comparison operation unit 122 that receives an input image 14, the template information 23, and the block division information 22, compares the pixel values of the selected pixels of the template image with the pixel values at the positions corresponding to the selected pixels of the input image 14, and calculates a similarity based on the comparison result; a matching operation unit 123 that performs template matching based on the similarity and outputs a matching result 15, which is the result of the template matching; and a pixel selection circuit construction unit 221. The selected pixel extraction unit 121 has an arithmetic circuit constructed with a variable structure, and the pixel selection circuit construction unit 221 constructs an arithmetic circuit (for example, switches the states of the switching elements constituting the circuit) based on the template information and the block division information.

[0082] The pixel selection circuit construction unit 221 receives the block division information 22 from the information storage unit 130 and constructs the circuit structure of the hardware circuit of the selected pixel extraction unit 121 that extracts the selected pixels. The selected pixel extraction unit 121 extracts the selected pixels through the hardware circuit constructed by the pixel selection circuit construction unit 221. The hardware circuit constructed by the pixel selection circuit construction unit 221 (that is, the hardware circuit included in the selected pixel extraction unit 121) can be implemented by a device such as an FPGA that can achieve dynamic circuit construction, for example.

[0083] Figure 15 and Figure 16 are flowcharts (Part 1, Part 2) showing the processing performed by the image processing apparatus 220 according to Embodiment 2. In Figure 15 and Figure 16 steps that are the same as or corresponding to the steps shown in Figure 9 are labeled with the same reference numerals as those shown in Figure 9 .

[0084] The image processing apparatus 220 according to Embodiment 2 performs the following processing in addition to the processing of the image analysis apparatus 210 according to Embodiment 1. First, the pixel selection circuit construction unit 221 reads the template information 13 and the block segmentation information 22 (step S401), and extracts the block segmentation information 22 (step S402). The pixel selection circuit construction unit 221 extracts the number of selected pixels for each block according to the extracted block segmentation information 22 (step S403), and constructs a circuit for extracting the selected pixels according to the shape of each block and the number of selected pixels for each block (that is, the arithmetic circuit included in the selected pixel extraction unit 121, namely, the hardware circuit) (step S404). The method of constructing the circuit is not particularly limited. For example, the pixel selection circuit construction unit 221 determines the circuit structure by considering at least one of the maximum value of the number of selected pixels for each block, the circuit scale that can be constructed by devices such as FPGAs for constructing the circuit, and the degree of parallelism of the processing that matches the required operation speed. In particular, the pixel selection circuit construction unit 221 may also perform circuit parallelization by considering the restrictions for pixel selection in the intra-block selected pixel determination unit 112. For example, when the pixel selection circuit construction unit 221 sets five pixels including the feature amount selected pixels selected based on the feature amount and the four surrounding pixels above, below, left, and right adjacent thereto as the selected pixels, a circuit for simultaneously reading the values of these five pixels is constructed.

[0085] In addition, the pixel selection circuit construction unit 221 may also consider the update of the reference position, and by shifting the stored value within the device for devices such as registers and memories that store the pixel values as the read source, the circuit is configured in such a way that the position of the read data can be updated corresponding to the update of the reference position.

[0086] Moreover, the pixel selection circuit construction unit 221 may also pre-construct a selector circuit for selecting blocks when sequentially reading the pixel values of the selected pixels from each block. For example, the block may be determined by switching the selector circuit at a certain period. After the input image 14 is read (step S201), the selected pixel information is extracted from the template information read in step S401 (step S405).

[0087] 《2-4》Effect

[0088] According to Embodiment 2, by adjusting the number of selected pixels for each block, the required processing operation amount can be set. Therefore, even when the image processing system 200 (for example, the selected pixel extraction unit 121) is configured by a hardware circuit, the block segmentation information 22 and the number of selected pixels in each block can be determined by considering the operation speed and degree of parallelism that can be achieved by the hardware circuit.

[0089] In addition, according to Embodiment 2, the pixel selection circuit construction unit 221 constructs a hardware circuit as an arithmetic circuit based on the template information 23 and the block division information 22, and thus can appropriately set the arithmetic speed and parallelism that can be achieved by the hardware circuit.

[0090] Description of reference numerals

[0091] 11 Template image, 12, 22 Block division information, 13, 23 Template information, 14 Input image, 15 Matching result, 100, 200 Image processing system, 110, 210 Image analysis device, 111 Feature quantity calculation unit, 112 Intra-block selected pixel determination unit, 120, 220 Image processing device, 121 Selected pixel extraction unit, 122 Comparison operation unit, 123 Matching operation unit, 130 Information storage unit, 211 Temporary selected pixel unit, 212 Block division determination unit, 221 Pixel selection circuit construction unit, 1101a to 1101n Blocks, 1102a to 1102n Selected pixels, 11x Pixels at the reference position.

Claims

1. An image analysis device that generates template information based on a template image, the template information including pairs of positions and pixel values of pixels used for matching in template matching. The image analysis device is characterized by having: A feature quantity calculation unit that calculates the feature quantity of each pixel of the template image; and An in-block selection pixel determination unit that divides the template image into a plurality of blocks based on block segmentation information and determines selection pixels, which are pixels showing features in each block, based on the feature quantity.

2. The image analysis device according to claim 1, characterized in that: The in-block selection pixel determination unit determines the number of selection pixels, i.e., the selection pixel number, in each block of the template image based on at least one of the shape of each block and the number of pixels in each block.

3. The image analysis device according to claim 2, characterized in that: The in-block selection pixel determination unit sets the maximum value of the selection pixel number to be less than or equal to a pre-determined value.

4. The image analysis device according to claim 2 or 3, characterized in that: The in-block selection pixel determination unit determines the selection pixel number of each block in such a way that the plurality of blocks include a first block with 0 selection pixels and a second block with a pre-determined number of selection pixels.

5. The image analysis device according to any one of claims 1 to 4, characterized in that: The in-block selection pixel determination unit pre-stores the block segmentation information or receives the block segmentation information from the outside.

6. The image analysis device according to any one of claims 1 to 5, characterized in that: The block segmentation information includes the shape, size, and configuration of each block of the template image.

7. The image analysis device according to any one of claims 1 to 6, characterized in that: The block segmentation information includes the number of selection pixels, i.e., the selection pixel number, in each block of the template image.

8. The image analysis device according to any one of claims 1 to 4, characterized in that: It further has: A temporary selection pixel selection unit that selects temporary selection pixels from the whole of the template image based on the feature quantity calculated by the feature quantity calculation unit; and A block segmentation determination unit that determines the block segmentation information based on the distribution state of the temporary selection pixels.

9. An image processing apparatus, characterized in that, Having: A selection pixel extraction unit that extracts the selection pixels based on the template information and the block segmentation information output from the image analysis device according to any one of claims 1 to 8; A comparison operation unit that receives an input image, the template information, and the block segmentation information, compares the pixel values of the selection pixels of the template image with the pixel values at the positions corresponding to the selection pixels of the input image, and calculates a similarity based on the result of the comparison; And A matching operation unit that performs template matching based on the similarity and outputs the result of the template matching.

10. The image processing device according to claim 9, characterized in that: It further has a pixel selection circuit construction unit, The selection pixel extraction unit has an arithmetic circuit that can be constructed variably. The pixel selection circuit construction unit constructs the arithmetic circuit based on the template information and the block segmentation information.

11. An image processing system, characterized in that, It has: The image analysis device according to any one of claims 1 to 8; and An image processing device, The image processing device has: A selected pixel extraction unit that extracts the selected pixels based on the template information and the block segmentation information output from the image analysis device; A comparison arithmetic unit that receives an input image, the template information, and the block segmentation information, compares the pixel values of the selected pixels of the template image with the pixel values at the positions corresponding to the selected pixels of the input image, and calculates the similarity based on the result of the comparison; And A matching arithmetic unit that performs template matching based on the similarity and outputs the result of the template matching.

12. An image processing method executed by an image analysis device that generates template information from a template image, the template information including pairs of positions and pixel values of pixels used for matching in template matching, The image processing method is characterized by having the following steps: Calculating the feature amount of each pixel of the template image; and Based on the block segmentation information, dividing the template image into a plurality of blocks, and determining the pixels showing features in each block, that is, the selected pixels, based on the feature amount.

13. An image processing program executed by a computer that generates template information from a template image, the template information including pairs of positions and pixel values of pixels used for matching in template matching, The image processing program is characterized in that It causes the computer to execute the following steps: Calculating the feature amount of each pixel of the template image; and Based on the block segmentation information, dividing the template image into a plurality of blocks, and determining the pixels showing features in each block, that is, the selected pixels, based on the feature amount.

Images