Image processing device, image processing method, and program

Abstract

To achieve an image processing device capable of acquiring highly accurate feature amount data by detecting feature points spatially distributed within an image with high accuracy even in the case of including a variation, bias, or the like of brightness in an image or a video.SOLUTION: An image processing device 100 performs gradation conversion of each division region image of one image, acquires feature points with high accuracy without being affected by a variation, bias, or the like of brightness within an image and a video because of performing feature point acquisition processing and feature amount data acquisition processing to a division image region after the gradation conversion, can acquire highly accurate feature amount data on the basis of the acquired feature points. Further, the image processing device 100 acquires feature point candidates of each division region image, and further, can acquire feature points distributed (spatially distributed) across the entire image because of acquiring the feature points on the basis of the feature point candidates.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image / video processing technique, and more particularly to a technique for detecting feature points from an image / video and acquiring feature amount data based on the feature points. [Background technology]

[0002] In recent years, technologies have been developed that perform object recognition processing on images and videos captured by image sensors to detect specific objects. In such object detection processing, it is important to properly detect feature points in the images and videos and properly acquire feature data based on the detected feature points. To achieve highly accurate object detection processing, technologies have been developed that acquire feature data that is invariant to changes in image scale and rotation (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Hironobu Fujiyoshi and Mitsuru Abe: Local Gradient Feature Extraction Technology, Journal of the Japan Society for Precision Engineering, Vol. 77, No. 12, 2011. Summary of the Invention [Problem to be solved by the invention]

[0004] When object recognition processing is performed on images and videos acquired by an image sensor, processing such as gradation conversion is performed so that the image and video data acquired by the image sensor falls within a predetermined range (dynamic range). At this time, if the images and videos acquired by the image sensor have variations in brightness or contain noise or abnormal pixels, etc., appropriate processing such as gradation conversion cannot be performed due to the effects of the variations in brightness, noise, abnormal pixels, etc., and as a result, it becomes difficult to properly detect feature points in the images and videos and properly acquire feature amount data based on the detected feature points.

[0005] Furthermore, when continuously detecting (recognizing) a specific object in multiple frame images that are consecutive in time series, the accuracy of continuously detecting (recognizing) the specific object can be improved by acquiring feature points at spatially dispersed positions within the frame images and performing object recognition processing based on the acquired feature points.

[0006] However, if there are variations in brightness among multiple frame images that are consecutive in time series, or if the frame images contain noise, abnormal pixels, etc., the variations in brightness, noise, abnormal pixels, etc. will affect the images, making it impossible to perform appropriate processing such as tone conversion. As a result, it will be difficult to properly detect feature points within the frame images and properly obtain feature data based on the detected feature points.

[0007] In view of the above problems, the present invention aims to provide an image processing device, an image processing method, and a program that can detect spatially dispersed feature points in an image with high accuracy and acquire highly accurate feature amount data, even when the image or video has variations in brightness or contains noise, abnormal pixels, etc. [Means for solving the problem]

[0008] In order to solve the above problem, a representative example (one aspect) of the invention disclosed in this application is an image processing device including a divided area image processing unit, a feature point acquisition processing unit, and a feature amount data acquisition processing unit.

[0009] The divided area image processing unit performs divided area image processing, which is a process of obtaining divided image area data after gradation conversion, by performing gradation conversion processing for each divided image area to be processed based on the divided image area obtained by dividing the image formed by the image data into multiple image areas.

[0010] The feature point acquisition processing unit acquires feature points in the divided image area to be processed formed by the divided image area data after gradation conversion based on the gradation value of the pixel to be processed and the gradation value or distribution of gradation values ​​of the surrounding area of ​​the pixel to be processed.

[0011] The feature data acquisition processing unit sets an area of ​​a predetermined size centered on a feature point as a feature data calculation area in the divided image area to be processed formed by the divided image area data after gradation conversion, and executes a feature data acquisition process to acquire feature data for the feature point based on the pixel values ​​of multiple pixels included in the set feature data calculation area. [Effects of the Invention]

[0012] According to the present invention, it is possible to realize an image processing device, an image processing method, and a program that can detect spatially dispersed feature points in an image with high accuracy and acquire highly accurate feature data, even if the image or video has brightness variations or contains noise, abnormal pixels, etc. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a schematic configuration diagram of an image processing device 100 according to a first embodiment. [Figure 2] 3 is a flowchart of processing executed by the image processing device 100. [Figure 3] 3 is a flowchart of processing executed by the image processing device 100. [Figure 4] 3 is a flowchart of processing executed by the image processing device 100. [Figure 5] 3 is a flowchart of processing executed by the image processing device 100. [Figure 6] FIG. 10 is a diagram for explaining image region division processing. [Figure 7] 5A and 5B are diagrams for explaining a process of acquiring feature amount data of feature points. [Figure 8] 5A and 5B are diagrams for explaining divided region image processing and feature point acquisition processing. [Figure 9] 5A and 5B are diagrams for explaining divided region image processing and feature point acquisition processing. [Figure 10] FIG. 10 is a schematic configuration diagram of an image processing device 200 according to a second embodiment. [Figure 11]10 is a flowchart of a process executed by the image processing device 200. [Figure 12] 10 is a flowchart of a process executed by the image processing device 200. [Figure 13] 10 is a flowchart of a process executed by the image processing device 200. [Figure 14] FIG. 10 is a diagram for explaining image region division processing. [Figure 15] 5A and 5B are diagrams for explaining divided region image processing and feature point acquisition processing. [Figure 16] 5A and 5B are diagrams for explaining divided region image processing and feature point acquisition processing. [Figure 17] FIG. 10 is a diagram for explaining a feature amount data acquisition process executed by an image processing device according to a first modified example of the second embodiment. [Figure 18] FIG. 10 is a diagram for explaining feature point candidate acquisition processing executed by an image processing device according to a second modified example of the second embodiment. [Figure 19] FIG. 10 is a diagram for explaining feature point candidate acquisition processing executed by an image processing device according to a second modified example of the second embodiment. [Figure 20] FIG. 10 is a diagram for explaining feature point candidate acquisition processing executed by an image processing device according to a second modified example of the second embodiment. [Figure 21] FIG. 10 is a diagram for explaining feature point candidate acquisition processing executed by an image processing device according to a second modified example of the second embodiment. [Figure 22] FIG. 10 is a schematic configuration diagram of an image processing device 300 according to a third embodiment. [Figure 23] FIG. 10 is a diagram for explaining a peripheral extended divided image region. [Figure 24] FIG. 10 is a diagram for explaining a peripheral extended divided image region. [Figure 25] FIG. 10 is a schematic configuration diagram of an image processing device 400 according to a fourth embodiment. [Figure 26] 4 is a flowchart of processing executed by the image processing device 400. [Figure 27] 10A and 10B are diagrams showing divided image areas of an image division pattern ptn_1 and an image division pattern ptn_2 having mutually different offset setting values. [Figure 28] 10A and 10B are diagrams showing divided image areas of an image division pattern ptn_1 and an image division pattern ptn_2, which have mutually different image division sizes. [Figure 29] A diagram showing the CPU bus configuration. DETAILED DESCRIPTION OF THE INVENTION

[0014] [First embodiment] The first embodiment will be described below with reference to the drawings.

[0015] <1.1: Configuration of image processing device> FIG. 1 is a schematic diagram of an image processing device 100 according to the first embodiment.

[0016] As shown in FIG. 1, the image processing device 100 includes a divided area image processing unit 1, a feature point acquisition processing unit 2, and a feature amount data acquisition processing unit 3.

[0017] The divided area image processing unit 1 receives input data Din (image data Din) to the image processing device 100, and performs divided area image processing (details will be described later) using the data Din. The divided area image processing unit 1 then outputs the data acquired by this processing as data D1 to the feature point acquisition processing unit 2 and the feature amount data acquisition processing unit 3.

[0018] The feature point acquisition processing unit 2 receives data D1 output from the divided area image processing unit 1. The feature point acquisition processing unit 2 performs feature point acquisition processing using the data D1 to acquire (detect) feature points on the image. The feature point acquisition processing unit 2 then outputs data including information on the acquired (detected) feature points as data D_kp to the feature amount data acquisition processing unit 3.

[0019] The feature data acquisition processing unit 3 receives the data D1 output from the divided area image processing unit 1 and the data D_kp output from the feature point acquisition processing unit 2. The feature data acquisition processing unit 3 performs feature data acquisition processing using the data D1 and the data D_kp to acquire feature data, and outputs the acquired feature data to the outside as data Dout.feature.

[0020] <1.2: Operation of image processing device> The operation of the image processing device 100 configured as above will now be described.

[0021] 2 to 5 are flowcharts of the processes executed by the image processing device 100. FIG.

[0022] FIG. 6 is a diagram for explaining the image region division process.

[0023] FIG. 7 is a diagram for explaining the process of acquiring feature amount data of feature points.

[0024] 8 and 9 are diagrams for explaining the divided region image processing and the feature point acquisition processing.

[0025] For ease of explanation, the operation of the image processing device 100 will be described for the case (as an example) in which image data capable of forming the image Img1 shown on the left in FIG. 6 is input to the image processing device 100 as Din.

[0026] The operation of the image processing device 100 will be described below with reference to the flowcharts of FIGS.

[0027] (Step S1): In step S1, divided area image processing is performed. Specifically, the following processing (processing of steps S11 to S15) is performed.

[0028] (Step S11): In step 11, image region division processing is performed. Specifically, the following processing is performed.

[0029] The divided area image processing unit 1 divides the image Img1 formed from the image data Din into N×M rectangular areas (N, M: natural numbers, N: the number of divisions in the vertical direction of the image, M: the number of divisions in the horizontal direction of the image). The divided areas obtained when the image Img1 is divided into N×M rectangular areas are called divided image areas, and the divided image area in the i-th row and j-th column is represented as div_img(i, j) (i, j: natural numbers, 1≦i≦N, 1≦j≦M). In the case of the right diagram of FIG. 6, N=M=4, and the image is divided equally into four in both the horizontal and vertical directions, resulting in 16 divided image areas div_img(1,1) to div_img(4,4) being set by the divided area image processing unit 1 as shown in the right diagram of FIG. 6.

[0030] (Step S12): In step S12, loop 1 processing (loop processing) is started. Loop 1 processing is executed for each divided image area acquired in step S11. That is, loop 1 processing is executed for each divided image area div_img(1,1) to div_img(N,M) (in this embodiment, N=M=4) (loop 1 processing is executed for divided image area div_img(i,j) (i,j: natural numbers, 1≦i≦N, 1≦j≦M) by incrementing i and j by +1 from the initial value "1", until i=N, j=M).

[0031] (Step S13): In step S13, tone conversion processing is performed. Specifically, the following processing is performed.

[0032] The divided area image processing unit 1 performs processing to obtain the minimum value D_min of the pixel values ​​of the pixels included in the divided image area div_img(i,j).

[0033] Furthermore, the divided area image processing unit 1 performs processing to obtain the maximum value D_max of the pixel values ​​of the pixels included in the divided image area div_img(i,j).

[0034] Then, the divided area image processing unit 1 performs a gradation conversion process on the pixels in the divided image area div_img(i,j). Specifically, the divided area image processing unit 1 converts (gradation conversion) the pixel values ​​of each pixel in the divided image area div_img(i,j) so that the pixel value range [D_min, D_max] of the pixels in the divided image area div_img(i,j) falls within the pixel value range [0, 255] (for example, when the data (dynamic range) after the gradation conversion process is 8-bit data). Note that the gradation conversion may be linear or nonlinear, as long as it converts the pre-conversion gradation value range [D_min, D_max] into the post-conversion gradation value range [0, 255] (for 8-bit data). Furthermore, the range of the data after the gradation conversion is not limited to 8-bit data, and may be data of another range (for example, data of another number of bits).

[0035] (Step S14): In step S14, a process for determining whether or not the loop 1 process has ended is executed. That is, if it is determined that the loop 1 process has been executed for all divided image areas div_img(i,j), the loop 1 process is ended and the process proceeds to step S2. On the other hand, if it is determined that the loop 1 process has not been executed for all divided image areas div_img(i,j), the process returns to step S12 and the loop 1 process (steps S12 to S14) is executed.

[0036] (Step S2): In step S2, loop A processing (loop processing) is started. Loop A processing is executed on the divided image areas after the gradation conversion processing (referred to as "gradation-converted divided image areas") acquired in step S1. That is, loop A processing is executed on each of the divided image areas div_img(1,1) to div_img(N,M) (in this embodiment, N=M=4) after the gradation conversion processing (loop A processing is executed repeatedly for the divided image areas div_img(i,j) (i,j: natural numbers, 1≦i≦N, 1≦j≦M) after the gradation conversion processing, by incrementing i and j by +1 from the initial value "1" until i=N, j=M).

[0037] (Step S3): In step S3, a feature point acquisition process is executed. Specifically, the following processes (processes in steps S31 and S32) are executed.

[0038] (Step S31): In step S31, a feature point candidate acquisition process is executed. Specifically, the following process is executed.

[0039] The feature point acquisition processing unit 2 inputs data D1 (image data of the divided image area div_img(i,j) after tone conversion processing) output from the divided area image processing unit 1. Note that data D1 is the image data of the divided image area div_img(i,j) after tone conversion processing (image data acquired by the divided area image processing unit 1), and is data output from the divided area image processing unit 1 to the feature point acquisition processing unit 2.

[0040] The feature point acquisition processing unit 2 executes processing such as FAST (Features from Accelerated Segment Test) described in Non-Patent Document 1 for each pixel of the divided image area div_img(i,j) after tone conversion to detect key points (feature point candidates) (a decision tree based on the luminance magnitude relationship between a pixel of interest and pixels in its surrounding area (for example, Nc pixels (Nc: natural number) on a circle with a predetermined radius around the pixel of interest and included in the area Area1) and acquired by machine learning, and detects key points (feature point candidates) (for example, points (pixels) that are highly likely to be corners)). Here, for ease of explanation, the set of feature point candidates (key points) detected as above is referred to as S_Pc, and its elements (feature point candidates) are referred to as Pc. i (i: natural number, 1≦i≦N_pc, N_pc: number of detected feature point candidates (keypoints)).

[0041] The feature point acquisition processing unit 2 sets a minimum number n_min of key points (feature point candidates) to be acquired in the divided image area div_img(i,j) after tone conversion, and acquires at least n_min key points (feature point candidates) in the divided image area div_img(i,j) after tone conversion through the above process. This ensures that a minimum number of key points (feature point candidates) are acquired for each divided image area, improving the probability of acquiring key points (feature point candidates) that are spatially dispersed throughout the entire image. Note that if the divided image area div_img(i,j) after tone conversion consists only of flat image areas (if feature point candidates (keypoints) with a predetermined accuracy cannot be acquired), the number of key points (feature point candidates) may be set to zero.

[0042] (Step S32): In step S32, a feature point acquisition process is executed. Specifically, the following process is executed.

[0043] The feature point acquisition processing unit 2 acquires the set of feature point candidates acquired in step S31 as a set of feature points, and designates the feature point candidates included in the set as feature points (feature point P j (j: integer, 0≦j≦N_kp-1, N_kp: number of feature points)

[0044] Then, the N_kp feature points P j The data including the data above is output from the feature point acquisition processing unit 2 to the feature amount data acquisition processing unit 3 as data D_kp.

[0045] (Step S4): In step S4, a feature amount data acquisition process is executed. Specifically, the following processes (processes of steps S41 to S45) are executed.

[0046] (Step S41): In step S41, a process of setting an image area for calculating feature amount data (feature amount data calculation area) is executed. Specifically, the following process is executed.

[0047] The feature data acquisition processing unit 3 sets an area including n×n pixels centered on a pixel of interest (pixel to be processed) in the divided image area div_img(i,j) after the tone conversion processing as the feature data calculation area. For example, the feature data acquisition processing unit 3 sets an area including 31×31 pixels (an area (image area) centered on the pixel of interest (pixel to be processed) and including 31 pixels in the horizontal direction of the image and 31 pixels in the vertical direction of the image) where n=31, as the feature data calculation area.

[0048] (Step S42): In step S42, loop 3 processing (loop processing) is started. The loop 3 processing is performed by calculating the feature points P j (j: integer, 0≦j≦N_kp-1, N_kp: number of detected feature points) j For this, j is incremented by +1 from the initial value "0" and is executed repeatedly until j = N_kp-1.

[0049] (Steps S43 and S44): In steps S43 and S44, a process of selecting L pairs (pairs of pixels) (L: natural number) within the feature amount data calculation region (patch) is performed. j Specifically, the following process is performed to acquire the feature amount data.

[0050] The feature data acquisition processing unit 3 acquires the feature points P j In the feature data calculation region centered on the pixel of , for example, ORB (Oriented FAST and Rotated BRIEF) (FAST: Features from Accelerated Segment Test, BRIEF: Binary Robust Independent Elementary Feature) described in Non-Patent Document 1 is executed to acquire feature data. jIn the feature data calculation region centered on the pixel, for example, as shown in FIG. 7, 256 pairs (256 pairs when the feature data to be acquired is 256 bits) (L=256) of two points (two pixels) specified by a template of a pixel pair selection pattern are selected (step S43).

[0051] Then, the feature data acquisition processing unit 3 determines the value of the i-th bit of the feature data (256-dimensional vector) to be "0" or "1" depending on the magnitude relationship between the pixel values ​​of the two pixels selected as the i-th pixel pair (i: natural number, 1≦i≦256). (For example, if the pixel value of the first pixel of the pixel pair selected by the pixel pair selection pattern template is I1 and the pixel value of the second pixel is I2, then (1) if I1>I2, the value of the i-th bit of the feature data is set to "0," and (2) if I1≦I2, the value of the i-th bit of the feature data is set to "1" (the method of setting "0" and "1" may be reversed from the above).)

[0052] It is preferable that the pixel pair selection pattern be set (for example, by using a Greedy algorithm) so that (1) the variance of the bit values ​​of the feature vector to be acquired (output) is large, and (2) the correlation between the selected pixel pairs is low. A template specifying the pixel pair selection pattern thus set is then stored in the feature data acquisition processing unit 3, and the template is used to determine the pixel pair selection pattern when performing the pixel pair selection process as described above.

[0053] By performing the above process on 256 pixel pairs, the feature data acquisition processing unit 3 obtains 256-bit feature data (256-dimensional vector) (this data (feature point P j feature data) into D_bin(P j ) is acquired (step S44). Then, the feature data acquisition processing unit 3 transfers the acquired feature data to the feature point P j The data (feature points P jData and feature points P j The position of the object on the image Img1 is stored together with the data (coordinate data) that specifies the position of the object on the image Img1.

[0054] In the above, a case has been described in which the value of the ith bit of the feature data (a 256-dimensional vector) is determined based on the magnitude relationship between the pixel values ​​of two pixels selected as the ith (i: natural number, 1≦i≦256) pixel pair. However, this is not limited to this, and the value of the Nout-i+1th (Nout: number of bits of feature data to be acquired) bit of the feature data (a 256-dimensional vector) may also be determined based on the magnitude relationship between the pixel values ​​of two pixels selected as the ith (i: natural number, 1≦i≦256) pixel pair.

[0055] (Step S45): In step S45, the end of the loop 3 process is determined. That is, the feature points P j If it is determined that the loop 3 process has been executed for (j: integer, 0≦j≦N_kp-1, N_kp: number of detected feature points), the loop 3 process is terminated and the process proceeds to step S5. On the other hand, for each feature point P j On the other hand, if it is determined that the loop 3 process has not been executed, the process returns to step S42, and the loop 3 process (steps S42 to S45) is executed.

[0056] (Step S5): In step S5, a process for determining whether or not the loop A process has ended is executed. That is, if it is determined that the loop A process has been executed for all divided image areas div_img(i,j) after the gradation conversion process, the loop A process is ended and the process proceeds to step S6. On the other hand, if it is determined that the loop A process has not been executed for all divided image areas div_img(i,j) after the gradation conversion process, the process returns to step S2 and the loop A process (steps S2 to S5) is executed.

[0057] (Step S6): In step S6, the data output process is executed. Specifically, the following process is executed.

[0058] The feature data acquisition processing unit 3 acquires data including all feature data acquired by the feature data acquisition process as data Dout.feature, and outputs the data Dout.feature to, for example, an external device. The data Dout.feature is as follows: Dout.feature={D_feat(P0),...,D_feat(P N_kp_all-1 )} D_feat(P j )={D_pos(P j ),D_bin(P j )} D_pos(P j ): feature point P j Data for identifying the feature points P j (in this embodiment, data for specifying a position (coordinates) on the image Img1 (image formed by data Din)) D_bin(P j ): feature point P j Feature data (e.g., 256-bit data (256-dimensional vector)) N_kp_all: The number of all feature points contained in the image Img (the total number of feature points obtained by executing the feature point acquisition process for the divided image area div_img(i,j) after all tone conversion processes) Here, the processing results when image data Din (image Img1) is input to the image processing device 100 and processed by the image processing device 100 will be described with reference to FIGS. 8 and 9. FIG.

[0059] As shown in FIG. 8, the image processing device 100 performs a gradation conversion process on the divided image area div_img(i,j). Therefore, even if there is a variation or bias in brightness within the image (for example, even if there is a variation or bias in brightness at a specific position or area within the image due to the positional relationship with the light source), the gradation value will not be biased toward W100% (for example, if the gradation value is 8 bits, then the gradation value is 255) or toward W0% (for example, if the gradation value is 8 bits, then the gradation value is 0) due to the influence of the variation or bias in brightness, and the details and contrast of the image will not be lost, and an image after appropriate gradation conversion will be obtained.

[0060] In the case of FIG. 8, the image processing device 100 sets the minimum number n_min of key points (feature point candidates) acquired in the feature point candidate acquisition process to "1", so that the feature point candidates (key points) Pc0 to Pc1 are spatially distributed in the entire image. 10 In the case of Figure 8, in the divided image areas div_img(1,1), div_img(1,4), div_img(4,1), div_img(4,2), and div_img(4,4), there is no change in brightness within the divided image areas, and they are flat image areas, so no feature point candidates are detected.

[0061] In the case of FIG. 8, as shown in FIG. 9, the feature point candidates Pc0 to Pc 10 The feature points P0 to P 10 It has been acquired as.

[0062] Furthermore, as described above, the image processing device 100 performs gradation conversion for each divided area that is considered to have little luminance variation or bias, thereby widening the range of possible pixel values ​​within the divided area after gradation conversion. Then, the image processing device 100 executes a process of acquiring feature data within the divided image area after gradation conversion, thereby making it possible to acquire highly expressive feature data (highly accurate feature data) (in steps S43 and S44, feature data is acquired based on the difference in gradation values ​​between pixel pairs having pixel values ​​with a wide range of possible values, making it possible to acquire highly expressive feature data (highly accurate feature data)).

[0063] The image processing device 100 then outputs the data of the feature points acquired as described above and the feature quantity data of the feature points to, for example, an object recognition processing device installed downstream of the image processing device 100, thereby enabling the object recognition processing device to perform object detection processing and object recognition processing with high accuracy.

[0064] <Summary> As described above, the image processing device 100 performs tone conversion for each divided-region image of one image (e.g., one frame image) and performs feature point acquisition processing and feature amount data acquisition processing on the divided image regions after tone conversion. This allows feature points to be acquired with high accuracy without being affected by variations or biases in brightness, and high-accuracy feature amount data to be acquired based on the acquired feature points. Furthermore, the image processing device 100 determines the minimum number of feature points to be acquired for each divided-region image, acquires feature point candidates, and further acquires feature points based on the feature point candidates. This allows feature points to be acquired that are dispersed (spatially dispersed) throughout the entire image. Then, by performing processing using the feature points acquired in this manner and the feature amount data based on the feature points, it is possible to perform the processing (processing to continuously detect (recognize) a specific object) with high accuracy, for example, even when the specific object is continuously detected (recognized) in multiple frame images that are consecutive in time series.

[0065] In this way, the image processing device 100 can detect spatially dispersed feature points in an image with high accuracy and acquire highly accurate feature amount data even when the image or video has brightness variations or biases.

[0066] [Second embodiment] Next, a second embodiment will be described below with reference to the drawings. Note that the same parts as those in the above embodiment are given the same reference numerals and detailed description will be omitted.

[0067] <2.1: Configuration of image processing device> FIG. 10 is a schematic diagram of an image processing device 200 according to the second embodiment.

[0068] As shown in FIG. 10, the image processing device 200 has a configuration in which, in the image processing device 100 of the first embodiment, the divided area image processing unit 1 is replaced with a divided area image processing unit 1A, the feature point acquisition processing unit 2 is replaced with a feature point acquisition processing unit 2A, and the feature data acquisition processing unit 3 is replaced with a feature data acquisition processing unit 3A, and further, a noise detection processing unit 4 is added.

[0069] The noise detection processing unit 4 inputs input data (for example, image / video data (image / video signal) acquired by an image sensor) Din (data (image data) capable of forming one image (one frame of image)) to the image processing device 100. The noise detection processing unit 4 performs noise detection processing on the input data, and outputs data including information on the position of noise on the image acquired by the noise detection processing as data D_noise_info to the divided area image processing unit 1A, feature point acquisition processing unit 2A, and feature data acquisition processing unit 3A. The noise detection processing unit 4 also outputs the data D_noise_info to the outside as data Dout.noise_info.

[0070] The noise detection processing unit 4 may be provided with, for example, a memory unit (not shown), and may store and retain data including information on the position of noise on the image acquired by the noise detection processing in the memory unit, read out the data including the information on the position of noise on the image from the memory unit, and output the read out data as data D_noise_info to the divided area image processing unit 1A, the feature point acquisition processing unit 2A, and the feature data acquisition processing unit 3A.

[0071] The divided area image processing unit 1A receives input data Din (image data Din) to the image processing device 100 and data D_noise_info output from the noise detection processing unit 4. The divided area image processing unit 1A performs divided area image processing (details will be described later) using the image data Din and data D_noise_info, and outputs the data acquired by this processing as data D1 to the feature point acquisition processing unit 2A and the feature amount data acquisition processing unit 3A.

[0072] The feature point acquisition processing unit 2A receives the data D1 output from the divided area image processing unit 1A and the data D_noise_info output from the noise detection processing unit 4. The feature point acquisition processing unit 2A performs feature point acquisition processing using the data D1 and the data D_noise_info to acquire (detect) feature points on the image. The feature point acquisition processing unit 2A then outputs data including information on the acquired (detected) feature points as data D_kp to the feature amount data acquisition processing unit 3A.

[0073] The feature data acquisition processing unit 3A receives the data D1 output from the divided area image processing unit 1A, the data D_kp output from the feature point acquisition processing unit 2A, and the data D_noise_info output from the noise detection processing unit 4. The feature data acquisition processing unit 3 performs feature data acquisition processing using the data D1, the data D_kp, and the data D_noise_info to acquire feature data, and outputs the acquired feature data to the outside as data Dout.feature.

[0074] <2.2: Operation of image processing device> The operation of the image processing device 200 configured as above will now be described.

[0075] 11 to 13 are flowcharts of the processing executed by the image processing device 200. FIG.

[0076] FIG. 14 is a diagram for explaining the image region division process.

[0077] 15 and 16 are diagrams for explaining the divided region image processing and the feature point acquisition processing.

[0078] For ease of explanation, the operation of the image processing device 100 will be described for the case (as an example) in which image data capable of forming the image Img1A shown in the left diagram of Fig. 14 is input as Din to the image processing device 100. Assume that the image Img1A is an image that includes isolated point noise Noise1 (isolated area noise) as shown in the left diagram of Fig. 14.

[0079] The operation of the image processing device 200 will be described below with reference to the flowcharts of FIGS.

[0080] (Step SA1): In step SA1, divided area image processing is performed. Specifically, the following processing (processing of steps SA11 to SA16) is performed.

[0081] (Step SA11): In step SA11, noise detection processing is executed. Specifically, the following processing is executed.

[0082] The noise detection processing unit 4 inputs input data Din (image data capable of forming the image Img1A) to the image processing device 200. The noise detection processing unit 4 performs noise detection processing on the input image data to detect noise contained in the image Img1A. For example, the noise detection processing unit 4 detects the difference in gradation value between a pixel of interest (or a pixel region of interest) in the image Img1A and its surrounding pixel regions, and if the detected difference in gradation value is greater than a predetermined value (threshold value), determines the pixel of interest (or pixel region of interest) as noise (a noise point or a noise region) and acquires the position (coordinates) of the noise point or noise region on the image (information that can identify the position or region of the detected noise point or noise region on the image). Note that the noise detection processing is not limited to the above processing, and other methods or processing may be adopted as long as they can detect noise points or noise regions on the image.

[0083] The noise detection processing unit 4 outputs data including information on the position of noise on the image acquired by the noise detection processing as data D_noise_info to the divided area image processing unit 1A, the feature point acquisition processing unit 2A, and the feature amount data acquisition processing unit 3A. The position on the image of image Img1A detected by the noise detection processing unit 4 (data on the x and y coordinates on the image) is represented as Noise1.Pos(x, y).

[0084] (Step SA12): In step A12, image region division processing is executed. Specifically, the following processing is executed.

[0085] The divided area image processing unit 1A divides the image Img1A formed from the image data Din into N×M rectangular areas (N, M: natural numbers, N: the number of divisions in the vertical direction of the image, M: the number of divisions in the horizontal direction of the image). The divided areas obtained when the image Img1A is divided into N×M rectangular areas are called divided image areas, and the divided image area in the i-th row and j-th column is represented as div_img(i, j) (i, j: natural numbers, 1≦i≦N, 1≦j≦M). In the case of the right diagram of FIG. 14, N=M=4, and the image is divided equally into four in both the horizontal and vertical directions, resulting in 16 divided image areas div_img(1,1) to div_img(4,4) set by the divided area image processing unit 1 as shown in the right diagram of FIG. 14.

[0086] (Step SA13): In step SA13, loop 1 processing (loop processing) is started. Loop 1 processing is executed for each divided image area acquired in step SA12. That is, loop 1 processing is executed for each divided image area div_img(1,1) to div_img(N,M) (in this embodiment, N=M=4). (Loop 1 processing is executed for divided image area div_img(i,j) (i,j: natural numbers, 1≦i≦N, 1≦j≦M) by incrementing i and j by +1 from the initial value "1", until i=N, j=M.)

[0087] (Step SA14): In step SA14, a noise point elimination process is performed. Specifically, the following process is performed.

[0088] The divided area image processing unit 1A excludes (1) pixels in the image area of ​​the divided image area div_img(i,j) that are determined to be noise points, and (2) pixels in areas within the image area of ​​the divided image area div_img(i,j) that are determined to be noise areas (excluding these from processing such as tone conversion), based on the data D_noise_info output from the noise detection processing unit 4. Then, the divided area image processing unit 1A acquires, as set S_normal, a set obtained by excluding the pixel values ​​of (1) pixels at noise points and (2) pixels in noise areas that were excluded above from the set of pixel values ​​of the pixels in the divided image area div_img(i,j).

[0089] (Step SA15): In step SA15, the tone conversion process is executed. Specifically, the following process is executed.

[0090] The divided area image processing unit 1A calculates the pixel values ​​D included in the set S_normal. i That is, the divided area image processing unit 1A performs processing corresponding to the following formula to obtain the minimum value of the pixel values ​​D included in the set S_normal. i Get the minimum value D_min of

number

number

[0091] In addition, the divided area image processing unit 1A may perform the above-mentioned gradation conversion by (1) setting the pixel value of a pixel in the divided image area div_img(i,j) to D_min (performing lower limit processing using D_min) if the pixel value of the pixel in the divided image area div_img(i,j) is less than D_min, and (2) setting the pixel value of the pixel in the divided image area div_img(i,j) to D_max (performing upper limit processing using D_max) if the pixel value of the pixel in the divided image area div_img(i,j) is greater than D_max.

[0092] (Step SA16): In step SA16, a process for determining whether or not the loop 1 process has ended is executed. That is, if it is determined that the loop 1 process has been executed for all divided image areas div_img(i,j), the loop 1 process is terminated and the process proceeds to step S2. On the other hand, if it is determined that the loop 1 process has not been executed for all divided image areas div_img(i,j), the process returns to step SA13 and the loop 1 process (steps SA13 to SA16) is executed.

[0093] (Step S2): In step S2, loop A processing (loop processing) is started. Loop A processing is executed on the divided image areas after the gradation conversion processing (referred to as "gradation-converted divided image areas") acquired in step S1. That is, loop A processing is executed on each of the divided image areas div_img(1,1) to div_img(N,M) (in this embodiment, N=M=4) after the gradation conversion processing (loop A processing is executed repeatedly for the divided image areas div_img(i,j) (i,j: natural numbers, 1≦i≦N, 1≦j≦M) after the gradation conversion processing, by incrementing i and j by +1 from the initial value "1" until i=N, j=M).

[0094] (Step SA3): In step SA3, a feature point acquisition process is executed. Specifically, the following processes (processes of steps SA31 to SA37) are executed.

[0095] (Step SA31): In step SA31, feature point candidate acquisition processing is executed. Specifically, the following processing is executed.

[0096] The feature point acquisition processing unit 2A receives as input data D1 (image data of the divided image area div_img(i,j) after tone conversion processing) output from the divided area image processing unit 1A and data D_noise_info output from the noise detection processing unit 4. Note that data D1 is the image data of the divided image area div_img(i,j) after tone conversion processing (image data acquired by the divided area image processing unit 1A), and is data output from the divided area image processing unit 1A to the feature point acquisition processing unit 2A.

[0097] The feature point acquisition processing unit 2A executes processing such as FAST (Features from Accelerated Segment Test) described in Non-Patent Document 1 for each pixel of the divided image area div_img(i,j) after tone conversion, and detects key points (feature point candidates) (for example, detects points (pixels) that are highly likely to be corners using a decision tree acquired by machine learning, which is based on the luminance magnitude relationship between a pixel of interest and pixels in its surrounding area (for example, Nc pixels (Nc: natural number) that are pixels on a circle with a predetermined radius around the pixel of interest and are included in the area Area1)). Here, for ease of explanation, the set of feature point candidates (key points) detected as above is referred to as S_Pc, and its elements (feature point candidates) are referred to as Pc. i (i: natural number, 1≦i≦N_pc, N_pc: number of detected feature point candidates (keypoints)).

[0098] The feature point acquisition processing unit 2A sets a minimum number n_min of key points (feature point candidates) to be acquired in the post-tone conversion divided image area div_img(i,j), and acquires at least n_min key points (feature point candidates) in the post-tone conversion divided image area div_img(i,j) through the above process. This ensures that a minimum number of key points (feature point candidates) are acquired for each divided image area, improving the probability of acquiring key points (feature point candidates) that are spatially dispersed throughout the entire image. Note that if the post-tone conversion divided image area div_img(i,j) consists only of flat image areas (if feature point candidates (keypoints) with a predetermined accuracy cannot be acquired), the number of key points (feature point candidates) may be set to zero.

[0099] (Step SA32): In step SA32, a set of feature points S kp Specifically, the feature point acquisition processing unit 2A performs a process of setting the feature point set S kp The process of setting is performed to an empty set.

[0100] (Step SA33): In step SA33, loop 2 processing (loop processing) is started. The loop 2 processing is performed by i (i: integer, 0≦i≦N_pc-1, N_pc: number of detected feature point candidates (keypoints)) i For i, the process is repeated by incrementing i by 1 from the initial value "0" until i=N_pc-1.

[0101] (Step SA34): In step SA34, the feature point candidates Pc i is a noise point (or a point within a noise region). Specifically, the following process is performed.

[0102] The feature point acquisition processing unit 2A acquires feature point candidates Pc i It is determined whether or not the pixel of the feature point candidate Pc matches a pixel included in the data D_noise_info output from the noise detection processing unit 4 (for example, i (1) The feature point candidate Pc i If it is determined that the pixel in the feature point candidate Pc matches a pixel included in the data D_noise_info (a pixel determined to be noise), the process proceeds to step SA36. i If it is determined that the pixel does not match the pixel included in the data D_noise_info (the pixel determined to be noise), the process proceeds to step SA35.

[0103] (Step SA35): In step SA35, a set of feature point candidates S kp , feature point candidate Pc iThat is, the feature point acquisition processing unit 2A executes a process of adding the feature point candidate Pc that is determined not to match the pixel included in the data D_noise_info (the pixel determined to be noise) in step SA34. i Let S be the set of feature point candidates. kp That is, the feature point acquisition processing unit 2A adds S kp ←{Pc i}∪S kp The equivalent process is performed.

[0104] (Step SA36): In step SA36, the end of the loop 2 process is determined. i If it is determined that the loop 2 process has been executed for all feature point candidates Pc included in the set S_Pc, the loop 2 process is terminated and the process proceeds to step SA37. i If it is determined that the loop 2 process has not been executed, the process returns to step SA33, and the loop 2 process (steps SA33 to SA36) is executed.

[0105] (Step SA37): In step SA37, the feature point acquisition process is executed. Specifically, the following process is executed.

[0106] The feature point acquisition processing unit 2A obtains a set S of feature point candidates at the time when the loop 2 processing is completed. kp is acquired as a set of feature points, and the feature point candidates included in the set are called feature points (feature points P j (j: integer, 0≦j≦N_kp-1, N_kp: number of feature points)

[0107] Then, the N_kp feature points P j The data including the data above is output as data D_kp from the feature point acquisition processing unit 2A to the feature amount data acquisition processing unit 3A.

[0108] (Step S4): In step S4, a feature amount data acquisition process is executed. The process (feature amount data acquisition process) executed in step S4 of this embodiment is the same as the process (feature amount data acquisition process) executed in step S4 of the first embodiment.

[0109] (Step S5): In step S5, a process for determining whether or not the loop A process has ended is executed. That is, if it is determined that the loop A process has been executed for all divided image areas div_img(i,j) after the gradation conversion process, the loop A process is ended and the process proceeds to step SA6. On the other hand, if it is determined that the loop A process has not been executed for all divided image areas div_img(i,j) after the gradation conversion process, the process returns to step S2, and the loop A process (steps S2 to S5) is executed.

[0110] (Step SA6): In step SA6, the data output process is executed. Specifically, the following process is executed.

[0111] The feature data acquisition processing unit 3A acquires data including all feature data acquired by the feature data acquisition process as data Dout.feature, and outputs the data Dout.feature to, for example, an external device. The data Dout.feature is as follows: Dout.feature={D_feat(P0),...,D_feat(P N_kp_all-1 )} D_feat(P j )={D_pos(P j ),D_bin(P j )} D_pos(P j ): feature point P j Data for identifying the feature points P j (in this embodiment, data for specifying a position (coordinates) on the image Img1A (image formed by data Din)) D_bin(P j ): feature point Pj Feature data (e.g., 256-bit data (256-dimensional vector)) N_kp_all: The number of all feature points contained in the image ImgA (the total number of feature points obtained by executing the feature point acquisition process for the divided image area div_img(i,j) after all tone conversion processes) Furthermore, the noise detection processing unit 4 outputs the data D_noise_info acquired by the noise detection processing as data Dout.noise_info, for example, to the outside.

[0112] Here, the processing results when image data Din (image Img1A) is input to image processing device 200 and processed by image processing device 100 will be described with reference to FIGS. 15 and 16. FIG.

[0113] 15, the image processing device 200 performs gradation conversion processing on the divided image area div_img(i,j) after excluding noise points (and noise areas), so that even in the divided image area div_img(i,j), the gradation values ​​are not biased toward W100% (for example, if the gradation value is 8 bits, then gradation value 255) or toward W0% (for example, if the gradation value is 8 bits, then gradation value 0) due to the influence of noise, and image detail and contrast are not lost, and an image after appropriate gradation conversion is obtained. In FIG. 15, the divided image area div_img(4,1) has a noise point Noise1 (white noise), but due to the influence of the noise point, the pixel values ​​of the divided image area div_img(4,1) are not biased overall toward W100%, and have appropriate gradation values.

[0114] In the case of FIG. 15, the image processing device 200 sets the minimum number n_min of key points (feature point candidates) acquired in the feature point candidate acquisition process to "1", so that the feature point candidates (key points) Pc0 to Pc1 are spatially distributed in the entire image. 11In the case of FIG. 15, in the divided image areas div_img(1,1), div_img(4,1), div_img(4,2), and div_img(4,4), there is no change in brightness within the divided image areas, and they are flat image areas, so no feature point candidates are detected.

[0115] 15, the feature point candidate Pc2 coincides with the noise point Noise1 in position on the image, so the image processing device 100 does not regard the feature point candidate Pc2 as a feature point. In other words, as shown in FIG. 16, the feature point candidates Pc0, Pc1, Pc3 to Pc4 excluding the feature point candidate Pc2 are 11 The feature points P0 to P 10 (The feature points are properly obtained.)

[0116] Furthermore, as described above, the image processing device 200 performs tone conversion after excluding noise points and noise regions, thereby widening the range of possible pixel values ​​within the divided region after tone conversion. Then, the image processing device 200 executes a process of acquiring feature data within the divided image region after tone conversion, thereby making it possible to acquire highly expressive feature data (highly accurate feature data) (in steps S43 and S44, feature data is acquired based on the difference in tone values ​​between two pixel pairs having pixel values ​​with a wide range of possible values, making it possible to acquire highly expressive feature data (highly accurate feature data)).

[0117] The image processing device 200 then outputs the data of the feature points acquired as described above, the feature amount data of the feature points, and the data of noise detected by the noise detection process to, for example, an object recognition processing device installed downstream of the image processing device 200, thereby enabling the object recognition processing to be performed with high accuracy by the object recognition processing device (since noise information can also be transmitted to the downstream stage, processing against noise can also be performed with high accuracy).

[0118] <Summary> As described above, the image processing device 200 performs tone conversion for each divided-region image of one image (e.g., one frame image) after removing noise points and noise regions, and then performs feature point acquisition processing and feature data acquisition processing on the divided image regions after tone conversion. This makes it possible to acquire feature points with high accuracy without being affected by noise, and to acquire highly accurate feature data based on the acquired feature points. Furthermore, the image processing device 200 determines the minimum number of feature points to acquire for each divided-region image, acquires feature point candidates, and further acquires feature points based on the feature point candidates. This makes it possible to acquire feature points dispersed (spatially dispersed) throughout the entire image. Then, by processing using the feature points acquired in this manner and the feature data based on the feature points, it is possible to perform the process (process of continuously detecting (recognizing) a specific object) with high accuracy, for example, even when continuously detecting (recognizing) a specific object in multiple frame images that are consecutive in time series.

[0119] In this way, the image processing device 200 can detect spatially dispersed feature points in an image with high accuracy and acquire highly accurate feature amount data even if the image or video contains noise, abnormal pixels, etc.

[0120] <First Modification> Next, a first modified example of the second embodiment will be described. Note that the same parts as those in the above embodiment are given the same reference numerals, and detailed description thereof will be omitted.

[0121] FIG. 17 is a diagram for explaining the feature amount data acquisition process executed by the image processing device according to the first modified example of the second embodiment.

[0122] The image processing device of the first modified example of the second embodiment has the same configuration as the image processing device 200 of the second embodiment, but differs from the second embodiment in the content of the feature data acquisition process by the feature data acquisition processing unit 3A.

[0123] In the image processing device of this modified example, the feature data acquisition processing unit 3A performs the feature data acquisition process using a template (a θ-rotated template) obtained by rotating a template of a selection pattern of pixel pairs by a predetermined angle θ around the center point of the template. This will be described with reference to Fig. 17. The image shown in the left diagram of Fig. 17 is an image in which three noise points Np1, Np2, and Np3 exist in the image shown in the left diagram of Fig. 7.

[0124] The feature data acquisition processing unit 3A counts the number of times a noise point is included (selected) in a pixel pair when the pixel pair is selected using the following four θ-rotated templates (the rotation direction is clockwise) for the image area Area0 (31 × 31 pixel area) shown in the left diagram of Figure 17 (this count number is called the "noise point count number"). (1) Template after θ rotation (θ=0) (2) Template after θ rotation (θ=π / 2) (3) Template after θ rotation (θ=π) (4) Template after θ rotation (θ=3π / 2) And in the case of Figure 17, as shown in Figure 17, (1) Number of noise points counted when applying the template after θ rotation (θ = 0): 2 (2) Number of noise points counted when applying the template after θ rotation (θ = π / 2): 0 (3) Number of noise points counted when applying the template after θ rotation (θ = π): 1 (4) Number of noise points counted when applying the template after θ rotation (θ = 3π / 2): 2 Therefore, the feature data acquisition processor 3A determines that the θ-rotated template with the smallest number of counted noise points is the θ-rotated template where θ=π / 2.

[0125] Then, the feature data acquisition processing unit 3A uses the θ-rotated template (θ=π / 2) that minimizes the number of noise points counted, and selects 256 pairs (256 pairs if the feature data to be acquired is 256 bits) (L=256) of two points (two pixels) identified by the θ-rotated template (θ=π / 2) of the pixel pair selection pattern, as in the second embodiment (step S43).

[0126] Then, based on the selected 256 pixel pairs, the same process as in the second embodiment (the process of step S44) is performed to acquire feature amount data.

[0127] As described above, in the image processing device of this modified example, in the feature data acquisition process, pixel pairs are selected by rotating the template by a predetermined angle, so that the possibility of a noise point (or noise area) being selected as a pixel pair is reduced. Therefore, even if a noise point or noise area is included in the feature data calculation area (a pixel area of ​​a predetermined size (e.g., a 31 × 31 pixel area) centered on a feature point), the influence of noise can be suppressed and highly accurate feature data can be acquired.

[0128] In the above, the template rotation angle θ is set to four angles, each of which is equal to π / 2, but this is not limiting and the template rotation angle θ may be set to any angle. Furthermore, the number of templates after θ rotation may be set to any number depending on the template rotation angle θ to be set.

[0129] Furthermore, in the image processing device, instead of rotating the template by the rotation angle θ, a plurality of different templates may be prepared, noise points may be counted using the plurality of different templates, and pixel pairs may be selected using the template that minimizes the number of counted noise points to acquire feature data.

[0130] <<Second Modification>> Next, a second modified example of the second embodiment will be described. Note that the same parts as those in the above embodiment are given the same reference numerals, and detailed description thereof will be omitted.

[0131] 18 to 21 are diagrams for explaining the feature point candidate acquisition process executed by the image processing device of the second modified example of the second embodiment.

[0132] The image processing device of the second modified example of the second embodiment has the same configuration as the image processing device 200 of the second embodiment, but differs from the second embodiment in the content of the feature point candidate acquisition process by the feature point acquisition processing unit 2A.

[0133] In the image processing device of this modified example, when a noise point exists in the peripheral area (target area of ​​feature point candidate processing) of a pixel of interest (image to be processed) in the feature point candidate acquisition processing by the feature point acquisition processing unit 2A, the effect of the noise point is eliminated and the feature point candidate acquisition processing is executed. This will be described below.

[0134] In the feature point candidate acquisition process (processing of step SA31), the feature point acquisition processing unit 2A of the image processing device of this modified example sets a pixel of interest (pixel to be processed) and a target area for feature point candidate processing. In the case of Fig. 18, area Area1 (an area of ​​7 pixels x 7 pixels centered on the pixel of interest) is the area set as the target area for feature point candidate processing (as an example).

[0135] The feature point acquisition processing unit 2A of this modified example selects pixels included in the target area for feature point candidate processing, on a circumference of a predetermined radius from the center pixel, and performs feature point candidate acquisition processing using the selected pixels. In the case of Fig. 18, the 16 pixels whose pixel values ​​are indicated in the right diagram of Fig. 18 are the pixels selected for performing feature point candidate acquisition processing (as an example).

[0136] Then, the feature point acquisition processing unit 2A of this modified example executes the following process to determine whether or not the pixel of interest should be selected (acquired) as a feature point candidate. This process will be explained below for the case of FIG. (1) The feature point acquisition processing unit 2A of this modified example selects N consecutive pixels (N: natural number, 2≦N≦15; in FIG. 19, N=12) from 16 pixels selected as pixels included in the target area for feature point candidate processing, on a circle of a predetermined radius from the center pixel. When N=12, there are 16 possible selection patterns. (2A) The feature point acquisition processing unit 2A selects the combination with the smallest maximum brightness (of the pixel values ​​of the selected 12 pixels) for N consecutive points (N=12 in the case of FIG. 19) (among the 16 combinations). In the left diagram of FIG. 19, the set (12 pixels) circled in red is the combination with the smallest maximum brightness, and the maximum brightness (maximum pixel value) of this set is "93" as shown in the left diagram of FIG. 19. The set of pixel values ​​of the set with the smallest maximum brightness is set as set S1. That is, in the case of FIG. 19, S1={84, 89, 93, 79, 60, 50, 44, 45, 47, 75, 87, 90}. (2B) Furthermore, the feature point acquisition processing unit 2A selects the combination with the greatest minimum luminance (of the pixel values ​​of the selected 12 pixels) among N consecutive points (N=12 in the case of FIG. 19) (among the 16 combinations). In the case of the right diagram of FIG. 19, the set (12 pixels) circled in red is the combination with the greatest minimum luminance, and the minimum luminance (smallest pixel value) of this set is "60," as shown in the right diagram of FIG. 19. The set of pixel values ​​of the set with the greatest minimum luminance is set S2. That is, in the case of FIG. 19, S2={75, 87, 90, 112, 90, 91, 84, 84, 89, 93, 79, 60}. (3A) When the pixel value P of the target pixel is greater than the maximum luminance of the set with the smallest maximum luminance (in the case of the left diagram of FIG. 19, “93” (=MAX(S1))), the feature point acquisition processing unit 2A: diff_d=P-MAX(S1)=P-93 MAX(): A function that gets the maximum value of an element and obtain the difference diff_d. (3B) When the pixel value P of the target pixel is smaller than the minimum luminance of the set having the largest minimum luminance (in the case of the right diagram of FIG. 19, “60” (=MIN(S2))), the feature point acquisition processing unit 2A: diff_I=MIN(S2)―P MIN(): A function to get the minimum value of an element and obtains the difference diff_I. (4) The feature point acquisition processing unit 2A acquires the maximum value of the differences diff_d and diff_I acquired in (3A) and (3B) above as the value diff. That is, the feature point acquisition processing unit 2A diff=MAX(diff_d,diff_I) In the case of FIG. 19, diff_d has no value (does not satisfy the condition), and diff_I=MIN(S2)−P=60−35=25, so diff=25. (5) The feature point acquisition processing unit 2A compares the value diff (=25) acquired in (4) above with a threshold value th, and if diff>th, acquires the pixel of interest as a feature point candidate. On the other hand, if diff>th is not the case, the pixel of interest is not acquired as a feature point candidate. For example, if the threshold value th=20, then in the case of Figure 19, diff>th, and so the feature point acquisition processing unit 2A acquires the pixel of interest in area Area1 in Figure 19 as a feature point candidate.

[0137] Next, a case where a noise point is included in the processing target area Area1 of the feature point candidate acquisition process will be described with reference to Fig. 20. For ease of explanation, as shown in Fig. 20, a case where noise exists at the position of a pixel whose pixel value is "112" in Fig. 19, and the pixel value of that pixel becomes "0" due to the influence of that noise will be described (a case where a noise point with a pixel value of "0" is included in area Area1).

[0138] First, the processing performed by the feature point acquisition processing unit 2A of the second embodiment will be described.

[0139] The feature point acquisition processing unit 2A executes the following process to determine whether or not the pixel of interest is to be selected (acquired) as a feature point candidate. (1) The feature point acquisition processing unit 2A of this modified example selects N consecutive pixels (N: natural number, 2≦N≦15; in FIG. 19, N=12) from 16 pixels selected as pixels included in the target area for feature point candidate processing, on a circle of a predetermined radius from the center pixel. When N=12, there are 16 possible selection patterns. (2A) The feature point acquisition processing unit 2A selects the combination with the smallest maximum brightness (of the pixel values ​​of the selected 12 pixels) among N consecutive points (N=12 in the case of FIG. 20) (among 16 combinations). In the case of the left diagram of FIG. 20, the set (12 pixels) circled in red is the combination with the smallest maximum brightness, and the maximum brightness (maximum pixel value) of this set is "91" as shown in the left diagram of FIG. 20. The set of pixel values ​​of the set with the smallest maximum brightness is set as set Sn1. That is, in the case of FIG. 20, Sn1={79, 60, 50, 44, 45, 47, 75, 87, 90, 0, 90, 91}. (2B) Furthermore, the feature point acquisition processing unit 2A selects the combination with the greatest minimum luminance (of the pixel values ​​of the selected 12 pixels) among N consecutive points (N=12 in the case of FIG. 19) (among the 16 combinations). In the case of the right diagram of FIG. 20, the set (12 pixels) circled in red is the combination with the greatest minimum luminance, and the minimum luminance (smallest pixel value) of this set is "44" as shown in the right diagram of FIG. 20. The set of pixel values ​​of the set with the greatest minimum luminance is set S2. That is, in the case of FIG. 20, Sn2={90, 91, 84, 84, 89, 93, 79, 60, 50, 44, 45, 47}. (3A) When the pixel value P of the target pixel is greater than the maximum luminance of the set with the smallest maximum luminance (in the case of the left diagram of FIG. 20, “99” (=MAX(Sn1))), the feature point acquisition processing unit 2A: diff_d=P-MAX(Sn1)=P-91 MAX(): A function that gets the maximum value of an element and obtain the difference diff_d. (3B) When the pixel value P of the pixel of interest is smaller than the minimum luminance of the set in which the minimum luminance is the largest (in the case of the right figure in Fig. 20, "44" (= MIN(Sn2))), diff_I = MIN(Sn2) - P MIN(): A function to obtain the minimum value of elements Execute the corresponding process to obtain the difference diff_I. (4) The feature point acquisition processing unit 2A obtains the maximum value of the differences diff_d and diff_I obtained in (3A) and (3B) above as the value diff. That is, the feature point acquisition processing unit 2A diff = MAX(diff_d, diff_I) To obtain the value diff. In the case of Fig. 20, diff_d has no value (the condition is not satisfied), and diff_I = MIN(Sn2) - P = 44 - 35 = 9, so diff = 9. (5) In the case of Fig. 20, since diff < th, the feature point acquisition processing unit 2A does not obtain the pixel of interest in the area Area1 of Fig. 20 as a feature point candidate.

[0140] As described above, when performing the feature point candidate acquisition process by the feature point acquisition processing unit 2A of the second embodiment, as described above, there may be a case where a pixel that should originally be acquired as a feature point candidate cannot be acquired as a feature point candidate due to the influence of noise, and there may also be a problem that a pixel that should not be acquired as a feature point candidate is acquired as a feature point candidate due to the influence of noise.

[0141] Therefore, in the feature point acquisition processing unit 2A of this modification example, noise points are specified based on the data D_noise_info output from the noise detection processing unit 4. When the feature point candidate acquisition processing target area contains N or more noise points (N: natural number), the pixel at the center of the feature point candidate acquisition processing target area (processing target pixel) is excluded from the feature point candidates. Thereby, as described above, when there are noise points in the feature point candidate acquisition processing target area, it is possible to appropriately prevent the feature point candidates from being erroneously acquired (detected) due to the influence of noise, and it is possible to prevent the feature point candidates that should originally be acquired from being erroneously not acquired.

[0142] Furthermore, in the feature point acquisition processing unit 2A of this modified example, when a noise point is included in the processing target area of ​​the feature point candidate acquisition processing, the noise point may be replaced based on pixels in the area surrounding the pixel of the noise point, as follows. Specifically, the feature point acquisition processing unit 2A of this modified example executes the processing as follows.

[0143] For example, when pixel P(i,j) shown in Fig. 21 is a noise point, the feature point acquisition processing unit 2A of this modified example performs processing to set the pixel values ​​derived from four pixels (four pixels adjacent above, below, left and right) P(i,j-1), P(i-1,j), P(i+1,j), and P(i,j+1) in the peripheral area of ​​pixel P(i,j) that is a noise point as the pixel value of pixel P(i,j). In other words, when pixel P(i,j) is a noise point, the feature point acquisition processing unit 2A performs processing to set the pixel values ​​derived from four pixels (four pixels adjacent above, below, left and right) P(i,j-1), P(i-1,j), P(i+1,j), and P(i,j+1) in the peripheral area of ​​pixel P(i,j) that is a noise point as the pixel value of pixel P(i,j). P(i,j)←D_get(D_set_peri1) D_get(): A function that obtains a value derived from an element (for example, a function that obtains the mean, weighted mean, or median of an element) D_set_peri1: A set of pixel values ​​of pixels surrounding pixel P(i,j) D_set_peri1={P(i,j-1),P(i-1,j),P(i+1,j),P(i,j+1)} As a result, the pixel value of pixel P(i,j) that is a noise point is replaced with a value that is not affected by noise, so even if a noise point is present in the processing target area of ​​the feature point candidate acquisition process, it is possible to appropriately prevent a feature point candidate from being erroneously acquired (detected) due to the influence of noise, and to prevent a feature point candidate that should actually be acquired from being erroneously not being acquired.

[0144] Furthermore, the surrounding area for deriving a value to replace a noise point may be set as an area other than the above. For example, the surrounding area for deriving a value to replace a noise point may be set as an area consisting of eight pixels adjacent to the noise point. In this case, if pixel P(i,j) is a noise point, the feature point acquisition processing unit 2A P(i,j)←D_get(D_set_peri1) D_get(): A function that obtains a value derived from an element (for example, a function that obtains the mean, weighted mean, or median of an element) D_set_peri2: A set of pixel values ​​of pixels surrounding pixel P(i,j) D_set_peri2={P(i-1,j-1),P(i,j-1),P(i+1,j-1),P(i-1,j),P(i+1,j),P(i-1,j+1),P(i,j+1),P(i+1,j+1)} As a result, the pixel value of pixel P(i,j) that is a noise point is replaced with a value that is not affected by noise, so even if a noise point is present in the processing target area of ​​the feature point candidate acquisition process, it is possible to appropriately prevent a feature point candidate from being erroneously acquired (detected) due to the influence of noise, and to prevent a feature point candidate that should actually be acquired from being erroneously not being acquired.

[0145] [Third embodiment] Next, a third embodiment will be described. Note that the same parts as those in the above embodiment (including the modified examples) are given the same reference numerals, and detailed description thereof will be omitted.

[0146] FIG. 22 is a schematic diagram of an image processing device 300 according to the third embodiment.

[0147] 23 and 24 are diagrams for explaining the peripheral extended divided image region.

[0148] The image processing device 300 according to the third embodiment has a configuration in which the divided area image processing unit 1A in the image processing device 200 according to the second embodiment is replaced with a divided area image processing unit 1B.

[0149] The divided area image processing unit 1B has the same configuration and functions as the divided area image processing unit 1A, and can also set a peripheral extended divided image area.

[0150] The operation of the image processing device 300 of the third embodiment will be described below.

[0151] The image processing device 200 of the second embodiment executes processing using the divided image area div_img(i,j), but the image processing device 300 executes processing using the extended image divided area ext_div_img(i,j).

[0152] In the image area division process (processing of step SA12), the divided area image processing unit 1B sets an extended image divided area ext_div_img(i,j) instead of setting a divided image area div_img(i,j).

[0153] The divided area image processing unit 1B divides the image Img1A formed from the image data Din into N × M areas (N, M: natural numbers, N: the number of divisions in the vertical direction of the image, M: the number of divisions in the horizontal direction of the image) (corresponding to image divided areas), and adds (1) areas expanded by a predetermined number of pixels above and below the image in the vertical direction, and (2) areas expanded by a predetermined number of pixels to the left and right in the horizontal direction of the image, and sets these areas as extended divided image areas ext_div_img(i, j) (i, j: natural numbers, 1≦i≦N, 1≦j≦M). Figure 23 shows the extended divided image area ext_div_img(2, 3). As shown in Fig. 23, the extended divided image area ext_div_img(2,3) is obtained by adding (1) an area extended by a predetermined number of pixels above and below the divided image area div_img(2,3) in the vertical direction on the image, and (2) an area extended by a predetermined number of pixels to the left and right of the horizontal direction on the image. When the input image is image Img1A, the extended divided image area ext_div_img(i,j) is set as shown in Fig. 24 (Figs. 23 and 24 show the extended divided image area ext_div_img(i,j) after tone conversion).

[0154] Then, the divided area image processing unit 1B outputs the data of the extended divided image area ext_div_img(2,3) set by the above processing as data D1A to the feature point acquisition processing unit 2A and the feature amount data acquisition processing unit 3A.

[0155] The feature point acquisition processing unit 2A and the feature data acquisition processing unit 3A input data D1A and perform the processing that was performed in the second embodiment using the divided image area div_img(i,j), but using the extended image divided area ext_div_img(i,j) instead of the divided image area div_img(i,j).

[0156] In the image processing device 300 of the third embodiment, as described above, the feature point acquisition process is performed using the extended image division area ext_div_img(i,j). Therefore, even if the feature point candidates are located near the edge of the division image area, as shown in Figures 23 and 24, for example, the feature point candidates can be properly acquired without missing any.

[0157] Therefore, the image processing device 300 of the third embodiment can perform the process of acquiring feature point candidates and feature points with even higher accuracy, and as a result, can acquire feature amount data with even higher accuracy.

[0158] In the process of setting the extended image division area ext_div_img(i,j), if there is no image division area div_img(i,j) adjacent to the image division area div_img(i,j) on the top, bottom, left, or right, the extended areas on the top, bottom, left, and right of the image division area div_img(i,j) are set as follows, for example: (1) a region filled with pixels having a predetermined pixel value, or (2) The area is filled with pixels so as to be line-symmetrical about the boundary line at the end of the image division area on the side where there is no adjacent image division area div_img(i,j) (this makes the extended area an image area line-symmetrical about the boundary line). This is how it should be done.

[0159] [Fourth embodiment] Next, a fourth embodiment will be described. Note that the same parts as those in the above-described embodiment (including the modified examples) are given the same reference numerals, and detailed description thereof will be omitted.

[0160] <4.1: Configuration of image processing device> FIG. 25 is a schematic diagram of an image processing device 400 according to the fourth embodiment.

[0161] The image processing device 400 of the fourth embodiment has a configuration in which the divided area image processing unit 1A of the image processing device 200 of the second embodiment is replaced with a divided area image processing unit 1C, and further a feature point determination processing unit 5 is added.

[0162] The divided area image processing unit 1C has the same configuration and functions as the divided area image processing unit 1A, and further receives data Info_offset specifying the offset of the divided image area and data Info_size specifying the size of the divided image area from an external unit or a control unit (not shown) that controls each functional unit of the image processing device 400. The divided area image processing unit 1C sets the divided image area div_img(i,j) based on the data info_offset and / or data Info_size, and outputs the data of the set divided image area div_img(i,j) as data D1B to the feature point acquisition processing unit 2A.

[0163] The feature point acquisition processing unit 2A receives the data D1B output from the divided area image processing unit 1C, executes the feature point acquisition process on the data D1B (data of the divided image area div_img(i,j)) in the same manner as in the second embodiment, and stores the acquired feature point data as data D_kp (ptn_i) and outputs it to the feature point determination processing unit 5. The divided area image processing unit 1C acquires the divided image area using a plurality of patterns (a plurality of setting values) for the offset and / or size of the divided image area, and the i-th pattern among the plurality of patterns is designated as ptn_i. The data of the feature points acquired by the feature point acquisition processing unit 2A for the data of the divided image area div_img(i,j) of the i-th pattern is designated as data D_kp (ptn_i) Let's say.

[0164] The feature point determination processing unit 5 receives the data D_kp output from the feature point acquisition processing unit 2A. (ptn_i) Enter the data D_kp (ptn_i)The feature points are determined based on the above, and data of the determined feature points is output as data D_kp to the feature amount data acquisition processing unit 3A.

[0165] The feature data acquisition processing unit 3A inputs data D_kp output from the feature point determination processing unit 5, performs feature data acquisition processing on the data D_kp in the same manner as in the second embodiment, and outputs the data acquired by this processing, for example, to the outside, as data Dout.feature.

[0166] <4.2: Operation of image processing device> The following describes the operation of the image processing device 400 configured as above. Note that detailed descriptions of the same parts as those in the above embodiment (including the modified examples) will be omitted.

[0167] FIG. 26 is a flowchart of the processing executed by the image processing device 400.

[0168] FIG. 27 is a diagram showing divided image areas of an image division pattern ptn_1 and an image division pattern ptn_2, which have different offset setting values.

[0169] FIG. 28 is a diagram showing divided image areas of an image division pattern ptn_1 and an image division pattern ptn_2, which have different image division sizes.

[0170] (4.2.1: Operation in case of image division pattern with changed offset) The operation of the image processing device 400 (operation in the case of an image division pattern with a changed offset) will be described below with reference to the flowchart in Fig. 26. For ease of explanation, it is assumed that the image division patterns set in the image processing device 400 are two, image division pattern ptn_1 and image division pattern ptn_2, as shown in Fig. 27. The setting values ​​for image division pattern ptn_1 and image division pattern ptn_2 are as follows: (Image division pattern ptn_1): Image division size: The number of vertical divisions = 4, the number of horizontal divisions = 4 Offset: Horizontal offset = 0, Vertical offset = 0 (Image division pattern ptn_2): Image division size: The number of vertical divisions = 4, the number of horizontal divisions = 4 Offset: Horizontal offset = x1, Vertical offset = y1 (Step SB1): In step SB1, loop B processing (loop processing) is started. Loop B processing is executed for each image division pattern ptn_i (i: natural number) (in this embodiment, 1≦i≦2. In other words, loop B processing is repeatedly executed for image division pattern ptn_i by incrementing i by +1 from the initial value "1" until i=2).

[0171] (Step SB2): In step SB2, divided area image processing is performed. Specifically, the following processing is performed. The processing in step SB2 is similar to that in step SA1 in the second embodiment, but the image area division processing in step SA12 differs from that in the second embodiment in that the divided image area div_img(i,j) is set so that it has the image division size and offset set by the image division pattern ptn_i. In the divided area image processing in step SB2, the processing in steps SA11 and SA13 to SA16 is the same as in the second embodiment.

[0172] The divided area image processing unit 1C receives data Info_offset specifying the offset of the divided image area and data Info_size specifying the size of the divided image area from an external unit or a control unit (not shown) that controls each functional unit of the image processing device 400, and sets the divided image area div_img(i,j) based on the data info_offset and data Info_size. Here, the image division size and offset of the image division pattern ptn_1 are set by the data info_offset and data Info_size.

[0173] Then, the divided area image processing unit 1C sets the divided image area div_img(i,j) so that it has the image division size and offset of the set image division pattern ptn_1, and outputs the data of the set divided image area div_img(i,j) to the feature point acquisition processing unit 2A as data D1B.

[0174] (Step SB3): In step SB3, loop C processing (loop processing) is started. Loop C processing is executed on the divided image areas after the gradation conversion processing (gradation-converted divided image areas) acquired in step SB2. That is, loop C processing is executed on each of the divided image areas div_img(1,1) to div_img(N,M) after the gradation conversion processing (N=M=4 in the case of the left diagram in FIG. 27 (in the case of image division pattern ptn_1)). Loop C processing is executed on each of the divided image areas div_img(i,j) after the gradation conversion processing (i,j: natural numbers, 1≦i≦N, 1≦j≦M) by incrementing i and j by +1 from the initial value "1" until i=N, j=M).

[0175] (Step SB4): In step SB4, a feature point acquisition process is executed, which is the same as the feature point acquisition process in step S3.

[0176] (Step SB5): In step SB5, a process for determining whether the loop C process has been completed is executed. That is, if it is determined that the loop C process has been executed for all divided image areas div_img(i,j) after the gradation conversion process in the image division pattern ptn_1, the loop C process is terminated and the process proceeds to step SA6. On the other hand, if it is determined that the loop C process has not been executed for all divided image areas div_img(i,j) after the gradation conversion process in the image division pattern ptn_1, the process returns to step SB3 and the loop C process (steps SB3 to SB5) is executed.

[0177] (Step SB6): In step SB6, a process for determining whether or not the loop B process has ended is executed. That is, if it is determined that the loop B process has been executed for all image division patterns ptn_i, the loop B process is terminated and the process proceeds to step SB7. On the other hand, if it is determined that the loop B process has not been executed for all image division patterns ptn_i, the process returns to step SB1 and the loop B process (steps SB1 to SB6) is executed.

[0178] In the loop B process for the image division pattern ptn_2, the image division size and offset of the image division pattern ptn_2 are set by the data info_offset and data Info_size in step SA2.

[0179] The divided area image processing unit 1C then sets the divided image area div_img(i,j) so that it has the image division size and offset of the set image division pattern ptn_2, and outputs the data of the set divided image area div_img(i,j) as data D1B to the feature point acquisition processing unit 2A. The other processes are then performed in the same manner as in the case of image division pattern ptn_1.

[0180] At the time step SB6 is completed, the data including the data of all the feature points acquired by the image division pattern ptn_1 is data D_kp (ptn_1)The data including the data of all the feature points acquired by the image division pattern ptn_2 is output from the feature point acquisition processing unit 2A to the feature point determination processing unit 5 as data D_kp (ptn_2) are output from the feature point acquisition processing unit 2A to the feature point determination processing unit 5 as

[0181] (Step SB7): In step SBA7, the feature point determination process is executed. Specifically, the following process is executed.

[0182] Data D_kp output from the feature point acquisition processing unit 2A (ptn_1) and data D_kp (ptn_2) The feature point acquisition processing unit 2A performs processing to determine feature points based on, for example, (1) data D_kp (ptn_1) and data D_kp (ptn_2) (2) The feature points included in common in the data D_kp are determined as the feature points for which feature data is to be acquired (feature point determination process by AND processing), or (ptn_1) and data D_kp (ptn_2) All feature points included in are determined as feature points for which feature amount data is to be acquired (feature point determination processing by OR processing). If you want to acquire only feature points with high accuracy, you can use the method (1) above (feature point determination processing by AND processing). On the other hand, if you want to acquire a large number of feature points, you can use the method (2) above (feature point determination processing by OR processing). Here, we will explain the case where the method (1) above (feature point determination processing by AND processing) is used.

[0183] In the case of FIG. 27, the image processing device 400 uses the data D_kp (ptn_1) and data D_kp (ptn_2) In FIG. 27, the circled feature points (feature points P1 to P6) are acquired as feature points commonly included in the above (determined as feature points from which feature amount data is to be acquired).

[0184] Then, the feature point determination processing unit 5 outputs data including the data of the feature points acquired as described above to the feature amount data acquisition processing unit 3A as data D_kp.

[0185] (Step SB8): In step SB8, a feature amount data acquisition process is executed, which is the same as the feature amount data acquisition process in step S4.

[0186] (Step SB9): In step SB9, a data output process is executed, which is the same as the data output process in step SA6 in the second embodiment.

[0187] (4.2.2: Behavior when image division pattern changes division size) The operation of the image processing device 400 (operation in the case of an image division pattern in which the division size is changed) will be described below with reference to the flowchart in Fig. 26. For ease of explanation, it is assumed that the image division patterns set in the image processing device 400 are two, image division pattern ptn_1 and image division pattern ptn_2, as shown in Fig. 28. The setting values ​​for image division pattern ptn_1 and image division pattern ptn_2 are as follows: (Image division pattern ptn_1): Image division size: The number of vertical divisions = 4, the number of horizontal divisions = 4 Offset: Horizontal offset = 0, Vertical offset = 0 (Image division pattern ptn_2): Image division size: The number of vertical divisions = 5, the number of horizontal divisions = 5 Offset: Horizontal offset = 0, Vertical offset = 0 (Step SB1): In step SB1, loop B processing (loop processing) is started. Loop B processing is executed for each image division pattern ptn_i (i: natural number) (in this embodiment, 1≦i≦2). That is, loop B processing is repeatedly executed for image division pattern ptn_i by incrementing i by +1 from the initial value "1" until i=2.

[0188] (Step SB2): In step SB2, divided area image processing is performed. Specifically, the following processing is performed. The processing in step SB2 is similar to that in step SA1 in the second embodiment, but the image area division processing in step SA12 differs from that in the second embodiment in that the divided image area div_img(i,j) is set so that it has the image division size and offset set by the image division pattern ptn_i. In the divided area image processing in step SB2, the processing in steps SA11 and SA13 to SA16 is the same as in the second embodiment.

[0189] The divided area image processing unit 1C receives data Info_offset specifying the offset of the divided image area and data Info_size specifying the size of the divided image area from an external unit or a control unit (not shown) that controls each functional unit of the image processing device 400, and sets the divided image area div_img(i,j) based on the data info_offset and data Info_size. Here, the image division size and offset of the image division pattern ptn_1 are set by the data info_offset and data Info_size.

[0190] Then, the divided area image processing unit 1C sets the divided image area div_img(i,j) so that it has the image division size and offset of the set image division pattern ptn_1, and outputs the data of the set divided image area div_img(i,j) to the feature point acquisition processing unit 2A as data D1B.

[0191] (Step SB3): In step SB3, loop C processing (loop processing) is started. Loop C processing is executed on the divided image areas after the gradation conversion processing (gradation-converted divided image areas) acquired in step SB2. That is, loop C processing is executed on each of the divided image areas div_img(1,1) to div_img(N,M) after the gradation conversion processing (N=M=4 in the case of the left diagram in FIG. 28 (in the case of image division pattern ptn_1)). Loop C processing is executed on each of the divided image areas div_img(i,j) after the gradation conversion processing (i,j: natural numbers, 1≦i≦N, 1≦j≦M) by incrementing i and j by +1 from the initial value "1" until i=N, j=M).

[0192] (Step SB4): In step SB4, a feature point acquisition process is executed, which is the same as the feature point acquisition process in step SA3 of the second embodiment.

[0193] (Step SB5): In step SB5, a process for determining whether or not the loop C process has been completed is executed. That is, if it is determined that the loop C process has been executed for all divided image areas div_img(i,j) after the tone conversion process in the image division pattern ptn_1, the loop C process is terminated and the process proceeds to step SB6. On the other hand, if it is determined that the loop C process has not been executed for all divided image areas div_img(i,j) after the tone conversion process in the image division pattern ptn_1, the process returns to step SB3, and the loop C process (steps SB3 to SB5) is executed.

[0194] (Step SB6): In step SB6, a process for determining whether or not the loop B process has ended is executed. That is, if it is determined that the loop B process has been executed for all image division patterns ptn_i, the loop B process is terminated and the process proceeds to step SB7. On the other hand, if it is determined that the loop B process has not been executed for all image division patterns ptn_i, the process returns to step SB1 and the loop B process (steps SB1 to SB6) is executed.

[0195] In the loop B process for the image division pattern ptn_2, the image division size and offset of the image division pattern ptn_2 are set by the data info_offset and data Info_size in step SB2.

[0196] The divided area image processing unit 1C then sets the divided image area div_img(i,j) so that it has the image division size and offset of the set image division pattern ptn_2, and outputs the data of the set divided image area div_img(i,j) as data D1B to the feature point acquisition processing unit 2A. The other processes are then performed in the same manner as in the case of image division pattern ptn_1.

[0197] At the time step SB6 is completed, the data including the data of all the feature points acquired by the image division pattern ptn_1 is data D_kp (ptn_1) The data including the data of all the feature points acquired by the image division pattern ptn_2 is output from the feature point acquisition processing unit 2A to the feature point determination processing unit 5 as data D_kp (ptn_2) are output from the feature point acquisition processing unit 2A to the feature point determination processing unit 5 as

[0198] (Step SB7): In step SB7, the feature point determination process is executed. Specifically, the following process is executed.

[0199] Data D_kp output from the feature point acquisition processing unit 2A (ptn_1) and data D_kp(ptn_2) The feature point acquisition processing unit 2A performs processing to determine feature points based on, for example, (1) data D_kp (ptn_1) and data D_kp (ptn_2) (2) The feature points included in common in the data D_kp are determined as the feature points for which feature data is to be acquired (feature point determination process by AND processing), or (ptn_1) and data D_kp (ptn_2) All feature points included in are determined as feature points for which feature amount data is to be acquired (feature point determination processing by OR processing). If you want to acquire only feature points with high accuracy, you can use the method (1) above (feature point determination processing by AND processing). On the other hand, if you want to acquire a large number of feature points, you can use the method (2) above (feature point determination processing by OR processing). Here, we will explain the case where the method (1) above (feature point determination processing by AND processing) is used.

[0200] In the case of FIG. 28, the image processing device 400 uses the data D_kp (ptn_1) and data D_kp (ptn_2) In FIG. 28, the circled feature points (feature points P1 to P7) are acquired as feature points commonly included in the above (determined as feature points from which feature amount data is to be acquired).

[0201] Then, the feature point determination processing unit 5 outputs data including the data of the feature points acquired as described above to the feature amount data acquisition processing unit 3A as data D_kp.

[0202] (Step SB8): In step SB8, a feature amount data acquisition process is executed, which is the same as the feature amount data acquisition process in step S4.

[0203] (Step SB9): In step SB9, a data output process is executed, which is the same as the data output process in step SA6.

[0204] As described above, the image processing device 400 performs feature point determination processing by AND processing on feature points acquired by a plurality of image division patterns ptn_i, thereby making it possible to acquire feature amount data limited to feature points with high accuracy, and therefore it is possible to acquire feature points with even higher accuracy and feature amount data with even higher accuracy.Furthermore, the image processing device 400 performs feature point determination processing by OR processing on feature points acquired by a plurality of image division patterns ptn_i, making it possible to acquire a large number of feature points, and for example, it is possible to appropriately deal with cases where it is desired to increase the number of feature points in later processing (e.g., object recognition processing) of the image processing device 400.

[0205] [Other embodiments] In the above embodiments (including the modified examples), the data input to the image processing devices 100, 200, 300, and 400 is described as image data, but this is not limited to this, and the data input to the image processing devices 100, 200, 300, and 400 may be video data (video signals) or data of consecutive frame images in a time series.

[0206] Furthermore, the image processing devices 100, 200, 300, and 400 may be realized by combining some or all of the above-described embodiments (including the modified examples).

[0207] Furthermore, each block (each functional unit) of the image processing devices 100, 200, 300, and 400 described in the above embodiments (including modifications) may be individually implemented as a single chip using a semiconductor device such as an LSI, or may be integrated into a single chip to include some or all of the blocks. Furthermore, each block (each functional unit) of the image processing devices 100, 200, 300, and 400 described in the above embodiments (including modifications) may be realized by multiple semiconductor devices such as LSIs.

[0208] Although we have referred to it as an LSI here, it may also be called an IC, system LSI, super LSI, or ultra LSI depending on the level of integration.

[0209] Furthermore, the method of integration is not limited to LSI, but may be realized by dedicated circuits or general-purpose processors. It is also possible to use FPGAs (Field Programmable Gate Arrays), which can be programmed after the LSI is manufactured, or reconfigurable processors, which allow the connections and settings of circuit cells inside the LSI to be reconfigured.

[0210] Furthermore, part or all of the processing of each functional block in each of the above embodiments (including modified examples) may be realized by a program. And part or all of the processing of each functional block in each of the above embodiments (including modified examples) is performed by a central processing unit (CPU) in a computer. Furthermore, the programs for performing each processing are stored in a storage device such as a hard disk or ROM, and are executed in the ROM or by being read into the RAM.

[0211] Furthermore, each process in the above-described embodiment (including modifications) may be realized by hardware, or by software (including cases where it is realized together with an OS (operating system), middleware, or a predetermined library). Furthermore, it may be realized by a combination of software and hardware.

[0212] For example, when each functional unit of the above embodiment (including the modified examples) is realized by software, each functional unit may be realized by software processing using the hardware configuration shown in Figure 29 (for example, a hardware configuration in which a CPU, GPU, processor, ROM, RAM, memory, input unit, output unit, etc. are connected by a bus).

[0213] Furthermore, when each functional unit of the above embodiment is realized by software, the software may be realized using a single computer having the hardware configuration shown in Figure 29, or may be realized by distributed processing using multiple computers.

[0214] Furthermore, the execution order of the processing method in the above embodiment is not necessarily limited to the description of the above embodiment, and the execution order can be changed within the scope of the gist of the invention. Furthermore, in the processing method in the above embodiment, some steps may be executed in parallel with other steps within the scope of the gist of the invention. Furthermore, in the processing method in the above embodiment, processes that are executed in parallel may be executed serially (sequentially).

[0215] The scope of the present invention includes a computer program for causing a computer to execute the above-described method and a computer-readable recording medium having the program recorded thereon, including, for example, a flexible disk, a hard disk, a CD-ROM, an MO, a DVD, a DVD-ROM, a DVD-RAM, a large-capacity DVD, a next-generation DVD, and a semiconductor memory.

[0216] The computer program is not limited to one recorded on the recording medium, but may be one transmitted via a telecommunications line, a wireless or wired communication line, a network such as the Internet, or the like.

[0217] The term "part" may also include the concept of "circuitry." A circuitry may be realized in whole or in part by hardware, software, or a combination of hardware and software.

[0218] The functions of the elements disclosed herein may be implemented using circuitry or processing circuitry, including general-purpose processors, special-purpose processors, integrated circuits, ASICs ("application-specific integrated circuits"), conventional circuitry, and / or combinations thereof, configured to perform the disclosed elements or programmed to perform the disclosed functions. A processor is considered to be processing circuitry or circuitry when it includes transistors and other circuitry therein. In this disclosure, a circuitry, unit, or means is hardware that performs the recited function or hardware programmed to perform the function. The hardware may be any hardware disclosed herein or other known hardware that is programmed to perform or configured to perform the recited function. When the hardware is a processor, which may be considered as a type of circuitry, the circuitry, means, or unit is a combination of hardware and software, software used to configure the hardware, and / or processor.

[0219] The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications are possible without departing from the gist of the invention.

[0220] [Note] The present invention can also be realized as follows.

[0221] A first invention is an image processing device including a divided region image processing unit, a feature point acquisition processing unit, and a feature amount data acquisition processing unit.

[0222] The divided area image processing unit performs divided area image processing, which is a process of obtaining divided image area data after gradation conversion, by performing gradation conversion processing for each divided image area to be processed based on the divided image area obtained by dividing the image formed by the image data into multiple image areas.

[0223] The feature point acquisition processing unit acquires feature points in the divided image area to be processed formed by the divided image area data after gradation conversion based on the gradation value of the pixel to be processed and the gradation value or distribution of gradation values ​​of the surrounding area of ​​the pixel to be processed.

[0224] The feature data acquisition processing unit sets an area of ​​a predetermined size centered on a feature point as a feature data calculation area in the divided image area to be processed formed by the divided image area data after gradation conversion, and executes a feature data acquisition process to acquire feature data for the feature point based on the pixel values ​​of multiple pixels included in the set feature data calculation area.

[0225] This image processing device performs tone conversion on each processing target divided-area image (e.g., a divided-area image or an area obtained by expanding the divided image area) of one image (e.g., one frame image), and then performs feature point acquisition processing and feature amount data acquisition processing on the divided image area after tone conversion. This makes it possible to acquire feature points with high accuracy without being affected by variations or biases in brightness within the image, and to acquire highly accurate feature amount data based on the acquired feature points. Furthermore, this image processing device acquires feature point candidates for each processing target divided-area image, and further acquires feature points based on the feature point candidates, making it possible to acquire feature points dispersed (spatially dispersed) throughout the entire image. Then, by processing using the feature points acquired in this manner and the feature amount data based on the feature points, it is possible to perform the processing (processing to continuously detect (recognize) a specific object) with high accuracy, for example, even when the specific object is continuously detected (recognized) in multiple frame images that are consecutive in time series.

[0226] In this way, this image processing device can detect spatially dispersed feature points within an image with high accuracy and obtain highly accurate feature data, even if the image or video contains variations or biases in brightness.

[0227] A second invention is an image processing device including a noise detection processing unit, a divided region image processing unit, a feature point acquisition processing unit, and a feature amount data acquisition processing unit.

[0228] The noise detection processing unit performs noise detection processing on image data that can form an image, thereby detecting noise points or noise areas on the image formed by the image data.

[0229] The divided area image processing unit performs divided area image processing, which is a process of obtaining divided image area data after gradation conversion, by excluding from the processing target pixels of noise points detected by the noise detection processing unit and pixels included in the noise area for each divided image area to be processed based on the divided image area obtained by dividing the image formed by the image data into multiple image areas, and performing gradation conversion processing.

[0230] The feature point acquisition processing unit acquires feature point candidates in the divided image area to be processed formed by the divided image area data after gradation conversion, based on the gradation value of the pixel to be processed and the gradation value or distribution of gradation values ​​of the surrounding area of ​​the pixel to be processed, and if the feature point candidate is not a noise point or a pixel included in a noise area detected by the noise detection processing unit, acquires the feature point candidate as a feature point.

[0231] The feature data acquisition processing unit sets an area of ​​a predetermined size centered on a feature point as a feature data calculation area in the divided image area to be processed formed by the divided image area data after gradation conversion, and executes a feature data acquisition process to acquire feature data for the feature point based on the pixel values ​​of multiple pixels included in the set feature data calculation area.

[0232] This image processing device performs tone conversion for each divided-region image of an image (e.g., one frame image) after removing noise points and noise regions, and then performs feature point acquisition processing and feature data acquisition processing on the divided image regions after tone conversion. This allows feature points to be acquired with high accuracy without being affected by noise, and high-accuracy feature data to be acquired based on the acquired feature points. Furthermore, this image processing device acquires feature point candidates for each divided-region image, and further acquires feature points based on the feature point candidates, allowing feature points dispersed (spatially dispersed) throughout the image to be acquired. Then, by processing using the feature points acquired in this manner and the feature data based on the feature points, it is possible to perform the process (process of continuously detecting (recognizing) a specific object) with high accuracy, for example, even when continuously detecting (recognizing) a specific object in multiple frame images that are consecutive in time series.

[0233] In this way, this image processing device can detect spatially dispersed feature points within an image with high accuracy and obtain highly accurate feature data, even if the image or video contains noise, abnormal pixels, etc.

[0234] A third aspect of the present invention is the second aspect of the present invention, wherein the feature point acquisition processing section does not acquire the processing target pixel as a feature point candidate when N or more noise points are included in a peripheral area of ​​the processing target pixel.

[0235] As a result, this image processing device can appropriately acquire feature point candidates without being affected by noise, even when noise points are included in the surrounding area of ​​the pixel to be processed.

[0236] A fourth invention is the second invention, wherein, when a noise point is included in the surrounding area of ​​the pixel to be processed, the feature point acquisition processing unit replaces the pixel value of the noise point pixel with a value derived from the pixel values ​​of pixels in the surrounding area of ​​the noise point pixel, and performs processing to acquire the feature point candidate.

[0237] As a result, this image processing device can appropriately acquire feature point candidates without being affected by noise, even when noise points are included in the surrounding area of ​​the pixel to be processed.

[0238] The fifth invention is the first or second invention, wherein the divided area image processing unit sets an area that is expanded by a predetermined size in the horizontal and / or vertical directions on the image in the divided image area obtained by dividing an image formed by image data into a plurality of image areas as an extended divided image area, and performs gradation conversion processing for each divided image area to be processed using the set extended divided image area as the divided image area to be processed, thereby performing divided area image processing.

[0239] In this image processing device, as described above, the feature point acquisition process is performed using the extended image division area, so that even if the feature point candidates are located near the edge of the divided image area, for example, the feature point candidates can be appropriately acquired without missing any of them.

[0240] Therefore, this image processing device can perform the process of acquiring feature point candidates and feature points with high accuracy, and as a result, can acquire feature amount data with even higher accuracy.

[0241] A sixth invention is the first or second invention, wherein the divided area image processing unit divides the image formed by the image data into a plurality of image areas according to a predetermined offset and / or a predetermined division size, and sets the divided image areas by moving the image areas by the offset amount.

[0242] This allows the image processing device to set a variety of divided image areas.

[0243] A seventh aspect of the present invention is the first or second aspect of the present invention, further comprising a feature point determination processing unit that determines feature points from which feature amount data is to be acquired.

[0244] The divided area image processing unit performs divided area image processing in accordance with M image division patterns specified by M sets (M: a natural number of 2 or more) of offsets and / or division sizes.

[0245] The feature point acquisition processing unit acquires feature point candidates in the divided image area formed by the divided image area data after gradation conversion acquired by the ith (i: natural number, 1≦i≦M) image division pattern of M image division patterns, based on the gradation value of the pixel to be processed and the gradation value or distribution of gradation values ​​of the surrounding area of ​​the pixel to be processed.

[0246] The feature point determination processing unit sets, among the feature points acquired using each of the M image division patterns, feature points that are acquired in common in two or more image division patterns as feature points from which feature amount data is to be acquired.

[0247] The feature amount data acquisition processing unit executes feature amount data acquisition processing on the feature points set by the feature point determination processing unit as targets for acquiring feature amount data.

[0248] As a result, this image processing device can acquire feature data limited to feature points that are commonly acquired using multiple image division patterns, making it possible to acquire feature points with even higher accuracy and feature data with even higher accuracy.

[0249] An eighth aspect of the invention is the first or second aspect of the invention, further comprising a feature point determination processing unit that determines feature points from which feature amount data is to be acquired.

[0250] The divided area image processing unit performs divided area image processing in accordance with M image division patterns specified by M sets (M: a natural number of 2 or more) of offsets and / or division sizes.

[0251] The feature point acquisition processing unit acquires feature point candidates in the divided image area formed by the divided image area data after gradation conversion acquired by the ith (i: natural number, 1≦i≦M) image division pattern of M image division patterns, based on the gradation value of the pixel to be processed and the gradation value or distribution of gradation values ​​of the surrounding area of ​​the pixel to be processed.

[0252] The feature point determination processing unit sets all feature points acquired using each of the M image division patterns as feature points from which feature amount data is to be acquired.

[0253] The feature amount data acquisition processing unit executes feature amount data acquisition processing on the feature points set by the feature point determination processing unit as targets for acquiring feature amount data.

[0254] As a result, this image processing device can subject all feature points acquired using multiple image division patterns to feature data acquisition processing, thereby acquiring a large number of feature points, and can appropriately respond, for example, when it is desired to increase the number of feature points in later processing (e.g., object recognition processing) of this image processing device.

[0255] A ninth invention is the first invention, wherein the feature data acquisition processing unit acquires N pairs of different pixels included in the feature data calculation area based on template data for selecting N (N: natural number) pixel pairs (two sets of pixels) that are pairs of different pixels included in the feature data calculation area, and the template data has data of pixel positions included in the pixel pairs set so that the positions on the image of the N pixel pairs are within a template image area that is an area of ​​the same image size as the feature data calculation area, and acquires feature data for the feature points based on the acquired N pairs of pixels.

[0256] As a result, this image processing device can use template data to efficiently select N pixel pairs, and can acquire feature amount data with high accuracy and high speed.

[0257] A tenth invention is any one of the second to fourth inventions, wherein the feature data acquisition processing unit acquires N pairs of different pixels included in the feature data calculation area based on template data in which data of pixel positions included in the pixel pairs is set so that the positions on the image of the N pixel pairs are within a template image area, which is an area of ​​the same image size as the feature data calculation area, and acquires feature data for the feature points based on the acquired N pairs of pixels.

[0258] As a result, this image processing device can use template data to efficiently select N pixel pairs, and can acquire feature amount data with high accuracy and high speed.

[0259] An eleventh invention is the tenth invention, wherein the feature data acquisition processing unit acquires, from the template data, a plurality of pieces of rotated template data in which pixel positions included in the template data are rotated by a predetermined angle around the center point of the template image area as the center of rotation, and when N pixel pairs are acquired using each of the acquired plurality of rotated template data and template data, measures a noise count, which is the number of pixels included in the acquired pixel pairs that match the positions on the image of pixels of noise points detected by the noise detection processing unit or pixels included in noise areas, and acquires the N pixel pairs using the template data or rotated template data with the smallest noise count, and acquires feature data for the feature points based on the acquired N pixel pairs.

[0260] As a result, this image processing device can efficiently select N pixel pairs using template data that is less affected by noise, and can acquire feature data with high accuracy and high speed.

[0261] A twelfth aspect of the present invention is an image processing method including a divided region image processing step, a feature point acquisition processing step, and a feature amount data acquisition processing step.

[0262] The divided area image processing step performs divided area image processing, which is a process of obtaining divided image area data after gradation conversion, by performing gradation conversion processing for each divided image area to be processed based on the divided image areas obtained by dividing the image formed by the image data into multiple image areas.

[0263] The feature point acquisition processing step acquires feature point candidates in the target divided image area formed by the divided image area data after gradation conversion, based on the gradation value of the target pixel and the gradation value or gradation value distribution of the surrounding area of ​​the target pixel.

[0264] The feature data acquisition processing step sets an area of ​​a predetermined size centered on a feature point as a feature data calculation area in the divided image area to be processed formed by the divided image area data after gradation conversion, and executes a feature data acquisition process to acquire feature data for the feature point based on the pixel values ​​of multiple pixels included in the set feature data calculation area.

[0265] This makes it possible to realize an image processing method that has the same effects as the first aspect of the invention.

[0266] This image processing method can be realized by executing each step using, for example, one or more processors and a memory accessible from the one or more processors.

[0267] A thirteenth aspect of the present invention is an image processing method including a noise detection processing step, a divided region image processing step, a feature point acquisition processing step, and a feature amount data acquisition processing step.

[0268] The noise detection processing step performs noise detection processing on image data that can form an image, thereby detecting noise points or noise areas on the image formed by the image data.

[0269] The divided area image processing step performs divided area image processing, which is a process of obtaining divided image area data after gradation conversion, by excluding from the processing target pixels of noise points detected in the noise detection processing step and pixels included in the noise area for each divided image area to be processed based on the divided image area obtained by dividing the image formed by the image data into multiple image areas, and performing gradation conversion processing.

[0270] The feature point acquisition processing step acquires feature point candidates in the target divided image area formed by the divided image area data after gradation conversion, based on the gradation value of the target pixel and the gradation value or gradation value distribution of the surrounding area of ​​the target pixel, and if the feature point candidate is not a noise point or a pixel included in a noise area detected by the noise detection processing step, the feature point candidate is acquired as a feature point.

[0271] The feature data acquisition processing step sets an area of ​​a predetermined size centered on a feature point as a feature data calculation area in the divided image area to be processed formed by the divided image area data after gradation conversion, and executes a feature data acquisition process to acquire feature data for the feature point based on the pixel values ​​of multiple pixels included in the set feature data calculation area.

[0272] This makes it possible to realize an image processing method that has the same effects as the second aspect of the invention.

[0273] This image processing method can be realized by executing each step using, for example, one or more processors and a memory accessible from the one or more processors.

[0274] This image processing method can be realized by executing each step using, for example, one or more processors and a memory accessible from the one or more processors.

[0275] A fourteenth aspect of the present invention is a program for causing a computer to execute the image processing method of the twelfth or thirteenth aspect of the present invention.

[0276] This makes it possible to realize a program for causing a computer to execute an image processing method that has the same effects as the twelfth or thirteenth aspect of the invention. [Explanation of symbols]

[0277] 100, 200, 300, 400 Image Processing Device 1, 1A, 1B, 1C Image region division processing section 2. 2A Feature point acquisition processing section 3. 3A Feature data acquisition processing section 4 Noise detection processing section 5. Feature point determination processing section

Claims

1. a divided area image processing unit that performs divided area image processing, which is processing to obtain divided image area data after gradation conversion by performing gradation conversion processing on each divided image area to be processed based on divided image areas obtained by dividing an image formed by image data into a plurality of image areas; a feature point acquisition processing unit that acquires feature points in a processing target divided image area formed by the gradation converted divided image area data based on the gradation value of a processing target pixel and the gradation value or gradation value distribution of an area surrounding the processing target pixel; a feature amount data acquisition processing unit that sets an area of ​​a predetermined size centered on the feature point as a feature amount data calculation area in the processing target divided image area formed by the post-tone conversion divided image area data, and executes feature amount data acquisition processing to acquire feature amount data for the feature point based on pixel values ​​of a plurality of pixels included in the set feature amount data calculation area; An image processing device comprising:

2. a noise detection processing unit that performs noise detection processing on image data that can form an image, thereby detecting noise points or noise areas on the image formed by the image data; a divided area image processing unit that executes divided area image processing, which is processing to obtain divided image area data after gradation conversion, by excluding pixels of noise points detected by the noise detection processing unit and pixels included in the noise area from the processing target divided image area based on the divided image areas obtained by dividing an image formed by the image data into a plurality of image areas; a feature point acquisition processing unit that acquires feature point candidates based on the gradation value of a pixel to be processed and the gradation value or gradation value distribution of a peripheral area of ​​the pixel to be processed in the divided image area to be processed formed by the divided image area data after gradation conversion, and acquires the feature point candidates as feature points if the feature point candidates are not noise points or pixels included in noise areas detected by the noise detection processing unit; a feature amount data acquisition processing unit that sets an area of ​​a predetermined size centered on the feature point as a feature amount data calculation area in the processing target divided image area formed by the post-tone conversion divided image area data, and executes feature amount data acquisition processing to acquire feature amount data for the feature point based on pixel values ​​of a plurality of pixels included in the set feature amount data calculation area; An image processing device comprising:

3. The feature point acquisition processing unit If N or more noise points are included in the peripheral area of ​​the pixel to be processed, the pixel to be processed is not acquired as the feature point candidate. The image processing device according to claim 2 .

4. The feature point acquisition processing unit If a noise point is included in the peripheral area of ​​the pixel to be processed, the pixel value of the noise point is replaced with a value derived from the pixel values ​​of pixels in the peripheral area of ​​the pixel of the noise point, and the feature point candidate is acquired. The image processing device according to claim 2 .

5. The divided area image processing unit an image formed by the image data is divided into a plurality of image regions, and an area expanded by a predetermined size in the horizontal and / or vertical directions on the image is set as an expanded divided image region; the expanded divided image region is set as the divided image region to be processed, and a gradation conversion process is performed for each of the divided image regions to be processed, thereby executing the divided area image processing.

3. The image processing device according to claim 1.

6. The divided area image processing unit Dividing an image formed by the image data into a plurality of image regions according to a predetermined offset and / or a predetermined division size, and moving the image regions by the offset amount to set the divided image regions.

3. The image processing device according to claim 1.

7. a feature point determination processing unit that determines feature points from which the feature amount data is to be acquired, The divided area image processing unit performing the divided region image processing in accordance with M image division patterns specified by M sets of offsets and / or division sizes (M: a natural number of 2 or more); The feature point acquisition processing unit In a divided image area formed by the divided image area data after gradation conversion obtained by the ith image division pattern (i: natural number, 1≦i≦M) of the M image division patterns, feature point candidates are obtained based on the gradation value of a pixel to be processed and the gradation value or gradation value distribution of a peripheral area of ​​the pixel to be processed; The feature point determination processing unit Among the feature points acquired using each of the M image division patterns, a feature point acquired in common in two or more of the image division patterns is set as a feature point from which feature amount data is to be acquired; The feature amount data acquisition processing unit the feature point determination processing unit executes the feature amount data acquisition process for the feature points set as targets for acquiring the feature amount data; 3. The image processing device according to claim 1.

8. a feature point determination processing unit that determines feature points from which the feature amount data is to be acquired, The divided area image processing unit performing the divided region image processing in accordance with M image division patterns specified by M sets of offsets and / or division sizes (M: a natural number of 2 or more); The feature point acquisition processing unit In a divided image area formed by the divided image area data after gradation conversion obtained by the ith image division pattern (i: natural number, 1≦i≦M) of the M image division patterns, feature point candidates are obtained based on the gradation value of a pixel to be processed and the gradation value or gradation value distribution of a peripheral area of ​​the pixel to be processed; The feature point determination processing unit setting all of the feature points acquired using each of the M image division patterns as feature points from which the feature amount data is to be acquired; The feature amount data acquisition processing unit the feature point determination processing unit executes the feature amount data acquisition process for the feature points set as targets for acquiring the feature amount data; 3. The image processing device according to claim 1.

9. The feature amount data acquisition processing unit the template data is used to select N pixel pairs (N: a natural number) that are pairs of different pixels included in the feature data calculation region, and pixel position data included in the pixel pairs is set so that the positions of the N pixel pairs on an image are within a template image region that is an region of the same image size as the feature data calculation region; based on the template data, N pairs of different pixels included in the feature data calculation region are acquired; and feature data for the feature points is acquired based on the acquired N pixel pairs. The image processing device according to claim 1 .

10. The feature amount data acquisition processing unit the template data is used to select N pixel pairs (N: a natural number) that are pairs of different pixels included in the feature data calculation region, and pixel position data included in the pixel pairs is set so that the positions of the N pixel pairs on an image are within a template image region that is an region of the same image size as the feature data calculation region; based on the template data, N pairs of different pixels included in the feature data calculation region are acquired; and feature data for the feature points is acquired based on the acquired N pixel pairs.

5. The image processing device according to claim 2.

11. The feature amount data acquisition processing unit a plurality of pieces of rotated template data are acquired from the template data, the pixel positions of which are obtained by rotating pixel positions included in the template data by a predetermined angle around a center point of the template image area as a rotation center; a noise count is measured, which is the number of pixels included in the acquired pixel pairs when N pixel pairs are acquired using each of the acquired plurality of rotated template data and the template data, the noise count being the number of pixels that match the positions on the image of pixels of noise points or pixels included in noise areas detected by the noise detection processing unit; the N pixel pairs are acquired using the template data or rotated template data with the smallest noise count; and feature amount data for the feature points is acquired based on the acquired N pixel pairs. The image processing device according to claim 10.

12. a divided area image processing step of performing divided area image processing, which is processing of acquiring divided image area data after gradation conversion by performing gradation conversion processing on each divided image area to be processed based on divided image areas obtained by dividing an image formed by image data into a plurality of image areas; a feature point acquisition processing step of acquiring feature points in the processing target divided image area formed by the gradation converted divided image area data based on the gradation value of the processing target pixel and the gradation value or gradation value distribution of the surrounding area of ​​the processing target pixel; a feature amount data acquisition processing step of setting an area of ​​a predetermined size centered on the feature point as a feature amount data calculation area in the processing target divided image area formed by the gradation-converted divided image area data, and executing a feature amount data acquisition process to acquire feature amount data for the feature point based on pixel values ​​of a plurality of pixels included in the set feature amount data calculation area; An image processing method comprising:

13. a noise detection processing step of detecting noise points or noise areas on an image formed by the image data by performing noise detection processing on image data capable of forming an image; a divided area image processing step for executing divided area image processing, which is a process for obtaining divided image area data after tone conversion by excluding pixels of noise points detected in the noise detection processing step and pixels included in the noise area from the processing target divided image area based on the divided image areas obtained by dividing the image formed by the image data into a plurality of image areas; a feature point acquisition processing step of acquiring feature point candidates based on the gradation value of a pixel to be processed and the gradation value or gradation value distribution of a peripheral area of ​​the pixel to be processed in the divided image area to be processed formed by the divided image area data after gradation conversion, and acquiring the feature point candidates as feature points if the feature point candidates are not noise points or pixels included in noise areas detected in the noise detection processing step; a feature amount data acquisition processing step of setting an area of ​​a predetermined size centered on the feature point as a feature amount data calculation area in the processing target divided image area formed by the gradation-converted divided image area data, and executing a feature amount data acquisition process to acquire feature amount data for the feature point based on pixel values ​​of a plurality of pixels included in the set feature amount data calculation area; An image processing method comprising:

14. A program for causing a computer to execute the image processing method according to claim 12 or 13.

Images