Template matching device and template matching method

Abstract

The pixel comparison unit (120) extracts a plurality of feature pixels from the template image, extracts a plurality of target pixels from the observed image to be compared with each extracted feature pixel, and calculates pixel distances for each extracted target pixel to the extracted plurality of feature pixels. The position detection unit (130) detects a position corresponding to the template image from the observed image based on the pixel distances for each target pixel.

Description

Technical Field

[0001] This disclosure relates to speeding up template matching.

Background Art

[0002] Template matching is a technique for detecting the position of a template image from an observation image. The observation image is an image obtained by photographing using a camera or the like. The template image is an image containing the object to be searched for.

[0003] Template matching is utilized, for example, as follows. In the fields of crime prevention and security, template matching is utilized for face authentication or object detection. In a factory line, template matching is utilized for product inspection.

[0004] By speeding up the processing of template matching, for example, real-time detection can be achieved. For speeding up the processing, sparse template matching has been proposed. Sparse template matching is a method of sparsifying the pixels to be compared.

[0005] In Patent Document 1, a circuit configuration for speeding up the processing of template matching has been proposed. Specifically, a circuit configuration that can simultaneously compare the feature pixels of the template image and a plurality of target pixels of the observation image has been proposed.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

[0007] The control described in Patent Document 1 is a simple control that sequentially references specific pixels in order to extract target pixels from the observed image. Therefore, in the circuit configuration of Patent Document 1, it is not possible to efficiently reference target pixels that have been sparsified in various ways by sparse template matching. Therefore, the circuit configuration in Patent Document 1 cannot achieve high speed through sparse template matching.

[0008] This disclosure aims to achieve speedup through sparse template matching. [Means for solving the problem]

[0009] The template matching device disclosed herein is A feature pixel extraction unit that extracts multiple feature pixels from a template image, A target pixel extraction unit extracts multiple target pixels from the observed image to be compared with each extracted feature pixel, A distance calculation unit calculates the pixel distance to multiple feature pixels extracted for each extracted target pixel, The system includes a position detection unit that detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel. [Effects of the Invention]

[0010] According to this disclosure, speed improvements can be achieved through sparse template matching. [Brief explanation of the drawing]

[0011] [Figure 1] Configuration diagram of the template matching device 100 in Embodiment 1. [Figure 2] Configuration diagram of the pixel comparison unit 120 in Embodiment 1. [Figure 3]Flowchart of the template matching method in Embodiment 1. [Figure 4] Flowchart of step S110 in Embodiment 1. [Figure 5] Flowchart of step S120 in Embodiment 1. [Figure 6] Configuration diagram of the template matching device 100 in Embodiment 2. [Figure 7] Configuration diagram of the pixel comparison unit 120 in Embodiment 2. [Figure 8] Flowchart of the template matching method in Embodiment 2. [Figure 9] Flowchart of step S220 in Embodiment 2. [Figure 10] Flowchart of the template matching method in Embodiment 3. [Figure 11] Flowchart of step S320 in Embodiment 3. [Figure 12] Hardware configuration diagram of the template matching device 100 in the embodiment.

Modes for Carrying Out the Invention

[0012] In the embodiments and the drawings, the same elements or corresponding elements are denoted by the same reference numerals. The description of the elements denoted by the same reference numerals as those already described will be omitted or simplified as appropriate. The arrows in the figures mainly indicate the flow of data or the flow of processing.

[0013] Embodiment 1. The form of sparse template matching will be described based on FIGS. 1 to 5.

[0014] ***Description of the configuration***

[0015] Based on FIG. 1, the configuration of the template matching device 100 will be described. The template matching device 100 comprises elements such as an observation image acquisition unit 110, a pixel comparison unit 120, a position detection unit 130, and an overall control unit 170. Furthermore, the template matching device 100 includes a memory unit. Specifically, the template matching device 100 includes memory units such as an observation image memory unit 140, a template image memory unit 150, and a coordinate information memory unit 160.

[0016] The components of the template matching device 100 are connected to each other via signal lines.

[0017] The configuration of the pixel comparison unit 120 will be explained based on Figure 2. The pixel comparison unit 120 includes elements such as a feature pixel extraction unit 121, a target pixel extraction unit 122, and a distance calculation unit 123.

[0018] ***Explanation of operation*** The procedure for operating the template matching device 100 corresponds to the template matching method.

[0019] The template matching method will be explained based on Figure 3. When the template matching device 100 is activated, the overall control unit 170 starts the execution of the template matching method. In the template matching method, the observation image acquisition unit 110, the pixel comparison unit 120, the position detection unit 130, and the various storage units are controlled by the overall control unit 170.

[0020] In step S110, the observation image acquisition unit 110 acquires an observation image. An observed image is an image of the target object, obtained by photographing the target object using a camera or other photographic equipment. The target object is the object that will be subjected to template matching.

[0021] The procedure for step S110 will be explained based on Figure 4. In step S111, the observed image is input to the template matching device 100 one or more pixels at a time. For example, one or more pixels of the observed image are input from the camera to the template matching device 100. The observation image acquisition unit 110 receives the input observation image.

[0022] In step S112, the observation image acquisition unit 110 stores one or more received pixels in the observation image storage unit 140.

[0023] Furthermore, the observation image acquisition unit 110 updates the number of times an observation image has been stored by adding 1. The number of times an observed image has been stored is managed using an internal counter. The initial value of the number of stored observed images is zero.

[0024] In step S113, the observation image acquisition unit 110 determines whether enough pixels of the observation image have been stored in the observation image storage unit 140 for the implementation of the template matching method. For example, when the number of stored pixels reaches a threshold, the pixels of the observed image are sufficiently stored. The threshold is predetermined, for example, the number of pixels in the observed image. If the pixels of the observed image are not sufficiently stored, the process proceeds to step S111. Step S110 ends when enough pixels of the observed image have been stored.

[0025] Returning to Figure 3, we will continue the explanation from step S120. In step S120, the pixel comparison unit 120 calculates the pixel distance for each target pixel in the observed image. Pixel distance is the distance between a target pixel and multiple feature pixels in a template image. The template image is an image of the target object and is stored in advance in the template image storage unit 150. Feature pixels are pixels that contain feature points and are extracted from a template image. The target pixels are the pixels that are compared to the feature pixels and are extracted from the observed image.

[0026] The procedure for step S120 will be explained based on Figure 5. In step S121, the feature pixel extraction unit 121 extracts one unextracted feature pixel from the template image. The feature pixel extraction unit 121 then passes the extracted feature pixels to the distance calculation unit 123.

[0027] Feature pixels are extracted as follows: First, the feature pixel extraction unit 121 generates the addresses of feature pixels in the template image based on the coordinate information and an internal counter for the number of feature pixels. The coordinate information indicates the coordinate values ​​of each of the multiple feature pixels in the template image. The coordinate information is specified by the user and is pre-stored in the coordinate information storage unit 160. Then, the feature pixel extraction unit 121 extracts feature pixels from the template image based on the generated addresses.

[0028] The "feature pixel" in step S122 refers to the single feature pixel extracted in step S121.

[0029] In step S122, the target pixel extraction unit 122 extracts one unextracted target pixel from the observed image. The target pixel extraction unit 122 then passes the extracted target pixels to the distance calculation unit 123.

[0030] The target pixels are extracted in the same way as feature pixels, as follows: First, the target pixel extraction unit 122 generates the address of the target pixel in the observed image based on the coordinate information. Then, the target pixel extraction unit 122 extracts the target pixel from the observed image based on the generated address.

[0031] The "feature pixel" in step S123 refers to the single feature pixel extracted in step S121. The "target pixel" in step S123 refers to the single target pixel extracted in step S122.

[0032] In step S123, the distance calculation unit 123 calculates the distance between the feature pixel and the target pixel.

[0033] The distance can be calculated as follows, for example: The distance calculation unit 123 calculates the difference (absolute value) between the pixel value of the feature pixel and the pixel value of the target pixel. The calculated difference (absolute value) is the distance between the feature pixel and the target pixel.

[0034] In step S124, the target pixel extraction unit 122 determines whether there are any unextracted target pixels.

[0035] The judgment is performed as follows: First, the target pixel extraction unit 122 updates the number of target pixels to be processed by adding 1. The number of target pixels to be processed is managed using an internal counter. The initial value of the number of target pixels to be processed is zero. The target pixel extraction unit 122 then compares the number of target pixels processed with a threshold value. The threshold value is a predetermined value specified by the user. If the number of target pixels processed is equal to the threshold, there are no unextracted target pixels. If the number of target pixels processed is less than the threshold, there are unextracted target pixels.

[0036] If there are unextracted target pixels, the process proceeds to step S122. If there are no unextracted target pixels, the process proceeds to step S125.

[0037] In step S125, "target pixel" refers to each of the multiple target pixels extracted in step S122.

[0038] In step S125, the distance calculation unit 123 updates the pixel distance for each target pixel based on the distance between the feature pixel and the target pixel. This calculates the pixel distance for each target pixel.

[0039] The pixel distance is updated as follows, for example: The distance calculation unit 123 updates the pixel distance by adding the distance between the feature pixel and the target pixel. The initial value of the pixel distance is zero.

[0040] In step S126, the feature pixel extraction unit 121 determines whether there are any unextracted feature pixels.

[0041] The judgment is performed as follows: First, the feature pixel extraction unit 121 updates the number of feature pixels to be processed by adding 1. The number of feature pixels processed is managed using an internal counter. The initial value of the feature pixel processing count is zero. The feature pixel extraction unit 121 then compares the number of processed feature pixels with a threshold value. The threshold value is a predetermined value specified by the user. If the number of processed feature pixels is equal to the threshold, there are no unextracted feature pixels. If the number of processed feature pixels is below the threshold, there are unextracted feature pixels.

[0042] If there are any unextracted feature pixels, the process proceeds to step S121. At this time, the feature pixel extraction unit 121 initializes the number of target pixels to be processed. The feature pixel extraction unit 121 also updates the number of feature pixels to be processed by adding 1. The number of feature pixels processed is managed using an internal counter. The initial value of the feature pixel processing count is zero.

[0043] If there are no unextracted feature pixels, step S120 terminates.

[0044] Returning to Figure 3, step S130 will be explained. In step S130, the position detection unit 130 detects the position corresponding to the template image from the observed image based on the pixel distance of each target pixel. The position detection unit 130 then outputs the detection result.

[0045] For example, the position corresponding to the template image is detected as follows: The position detection unit 130 selects the position of the target pixel with the smallest pixel distance. The selected position corresponds to the template pixel.

[0046] After step S130, the template matching method processing is completed.

[0047] ***Effects of Embodiment 1*** Embodiment 1 aims to speed up position detection through efficient sparse template matching processing. Embodiment 1 implements sparse template matching processing. In Embodiment 1, feature pixels and target pixels are extracted based on coordinate information (position information) and an internal counter for the number of feature pixels, and the extracted feature pixels are compared with the extracted target image. As a result, position detection by sparse template matching is accelerated.

[0048] ***Embodiment of Embodiment 1*** The template image storage unit 150 may store all feature pixels in the template image instead of all pixels in the template image. The feature pixel extraction unit 121 extracts feature pixels from the template image storage unit 150 based on an internal counter for the number of feature pixels, without referring to coordinate information. For example, the target pixel extraction unit 122 extracts target pixels that correspond to the feature pixels extracted by the feature pixel extraction unit 121.

[0049] Step S120 may be started before all pixels of a single observation image have been read in step S110. In this case, the number of loops in step S120 will be changed according to the size of the read observation image.

[0050] Multiple template images may be used. In this case, an internal counter for the number of template images is added, and branching processing occurs according to the value of the internal counter. Position detection is then performed by comparing multiple pixel distances for multiple template images.

[0051] Embodiment 2. The main differences between this embodiment and Embodiment 1, which are related to the method for accelerating sparse template matching, will be explained based on Figures 6 to 9.

[0052] ***Explanation of the structure*** The configuration of the template matching device 100 will be explained based on Figure 6. The configuration of the template matching device 100 differs from that of the first embodiment in that it includes multiple observation image storage units 140.

[0053] The configuration of the pixel comparison unit 120 will be explained based on Figure 7. The pixel comparison unit 120 includes elements such as a feature pixel extraction unit 121, multiple target pixel extraction units 122, multiple distance calculation units 123, and a comparison range control unit 124.

[0054] Multiple target pixel extraction units 122 are arranged in parallel and operate simultaneously. Multiple distance calculation units 123 are arranged in parallel and operate simultaneously. The target pixel extraction unit 122 and the distance calculation unit 123 are connected in a one-to-one correspondence.

[0055] The comparison range control unit 124 determines two or more target pixels from among multiple target pixels for which the pixel distance will be calculated.

[0056] ***Explanation of operation*** The template matching method will be explained based on Figure 8. In step S210, the observation image acquisition unit 110 acquires an observation image. The procedure in step S210 is the same as the procedure in step S110 in Embodiment 1. However, the observed images are stored in each of the multiple observed image storage units 140.

[0057] In step S220, the pixel comparison unit 120 calculates the pixel distance for each target pixel in the observed image.

[0058] The procedure for step S220 will be explained based on Figure 9. In step S221, the feature pixel extraction unit 121 extracts one unextracted feature pixel from the template image. Step S221 is the same as step S121 in Embodiment 1.

[0059] In step S222, multiple target pixel extraction units 122 simultaneously extract different target pixels. As a result, multiple target pixels are extracted at once.

[0060] Multiple target pixels are extracted as follows: Each observation image storage unit 140 is assigned a different offset address. Each target pixel extraction unit 122 extracts the target pixel from the observed image stored in the corresponding observed image storage unit 140, based on an address derived from the coordinate information and an internal counter for the number of feature pixels, and the offset address of the corresponding observed image storage unit 140.

[0061] If the number of observation image storage units 140 is greater than the number of target pixel extraction units 122, the comparison range control unit 124 determines two or more observation image storage units 140 to be used in step S222 based on the coordinate information.

[0062] In step S223, multiple distance calculation units 123 simultaneously calculate the distance between feature pixels and target pixels for each other. This allows the distance between each feature pixel and each target pixel to be calculated at once. The calculation method is the same as the method in Embodiment 1.

[0063] Specifically, each distance calculation unit 123 calculates the distance between a feature pixel and a target pixel for the target pixel extracted by the corresponding target pixel extraction unit 122.

[0064] A memory element such as a shift register can be used to control the comparison range. A shift register is an example of a memory element that allows data to be moved through simple shift operations. The shift register stores multiple target pixels. A portion (a certain range) of these target pixels is compared with the feature pixels. In other words, the distance between each of the selected target pixels and the feature pixels is calculated. Some feature pixels are extracted from memory elements by performing a shift operation on the memory elements based on the coordinate information. Some feature pixels are those with a number of digits specified in advance by the user.

[0065] In step S224, the multiple distance calculation units 123 update the pixel distances for different target pixels based on the distance between the feature pixel and the target pixel. As a result, the pixel distance of each target pixel is calculated simultaneously. The update method is the same as the method in Embodiment 1.

[0066] Specifically, each distance calculation unit 123 updates the pixel distance of the target pixel extracted by the corresponding target pixel extraction unit 122.

[0067] In step S225, the feature pixel extraction unit 121 determines whether there are any unextracted feature pixels. Step S225 is the same as step S126 in Embodiment 1.

[0068] If there are any unextracted feature pixels, the process proceeds to step S221. At this point, the feature pixel extraction unit 121 updates the number of processed feature pixels by adding 1. If there are no unextracted feature pixels, step S220 terminates.

[0069] Returning to Figure 8, step S230 will be explained. In step S230, the position detection unit 130 detects the position corresponding to the template image from the observed image based on the pixel distance of each target pixel. Step S230 is the same as step S130 in Embodiment 1.

[0070] After step S230, the template matching method process is completed.

[0071] ***Effects of Embodiment 2*** Embodiment 2 speeds up sparse template matching by simultaneously processing feature pixels of the template image and multiple target pixels of the observed image. In Embodiment 2, multiple target pixel extraction units 122 and distance calculation units 123 are arranged. This allows for the simultaneous calculation of the distances between feature pixels in the template image and multiple target pixels in the observed image. As a result, the sparse template matching process is accelerated. The template matching device 100 includes a comparison range control unit 124, which selects the target pixels to be compared from pixels extracted from multiple observation image storage units 140 by a shift process. This allows for a flexible circuit configuration for other image processing without imposing limitations on the number of divisions when storing the observation image.

[0072] Embodiment 3. The main differences between this embodiment and Embodiment 2 in terms of the method for accelerating sparse template matching will be explained based on Figures 10 and 11.

[0073] ***Explanation of the structure*** The configuration of the template matching device 100 is the same as the configuration in Embodiment 2.

[0074] ***Explanation of operation*** The template matching method will be explained based on Figure 10. In step S310, the observation image acquisition unit 110 acquires an observation image. Step S310 is the same as step S210 in Embodiment 2.

[0075] In step S320, the pixel comparison unit 120 calculates the pixel distance for each target pixel in the observed image.

[0076] The procedure for step S320 will be explained based on Figure 11. In step S321, the feature pixel extraction unit 121 extracts one unextracted set of feature pixels from the template image.

[0077] A feature pixel set consists of one or more feature pixels. Specifically, a feature pixel set is one or more feature pixels that are on the same line in the template image and are within a fixed distance of each other.

[0078] In step S322, the multiple target pixel extraction unit 122 extracts multiple target pixels from the observed image. The extracted multiple target pixels are all different from each other.

[0079] The number of target pixels is equal to the sum of the number of target pixels for a single feature pixel and the distance between the feature pixel pairs.

[0080] In step S323, the multiple distance calculation units 123 calculate the distance between each feature pixel in the feature pixel set and the target pixel corresponding to each feature pixel in the feature pixel set. The calculation method is the same as the method in step S223 in Embodiment 2.

[0081] Some feature pixels are extracted from the memory element by performing a shift operation on the memory element, which involves shifting the coordinate information of the feature pixel by a fixed distance from the feature pixel.

[0082] In step S324, the multiple distance calculation units 123 update the pixel distance for each target pixel based on the distance between each feature pixel in the feature pixel set and the target pixel. The update method is the same as the method in Embodiment 1.

[0083] Specifically, each distance calculation unit 123 updates the pixel distance for the target pixels extracted by the corresponding target pixel extraction unit 122.

[0084] In step S325, the feature pixel extraction unit 121 determines whether there are any unextracted feature pixel sets. Step S325 is the same as step S126 in Embodiment 1.

[0085] If there are any unextracted feature pixel sets, the process proceeds to step S321. At this point, the feature pixel extraction unit 121 updates the number of processed feature pixels by adding 1. If there are no unextracted feature pixel pairs, step S320 terminates.

[0086] Returning to Figure 10, step S330 will be explained. In step S330, the position detection unit 130 detects the position corresponding to the template image from the observed image based on the pixel distance of each target pixel. Step S330 is the same as step S230 in the second embodiment.

[0087] After step S330, the template matching method process is completed.

[0088] ***Effects of Embodiment 3*** Embodiment 3 speeds up sparse template matching when constraints are imposed on the coordinates of feature pixels in the template image. Embodiment 3 improves the efficiency of extracting multiple target pixels. It also reduces the number of iterations in step S320. The more feature pixels included in the feature pixel set, the greater the effect of reducing the number of iterations. Furthermore, the smaller the fixed distance, the more efficient the extraction of multiple target pixels becomes.

[0089] ***Supplementary Information on the Embodiment*** Based on Figure 12, the hardware configuration of the template matching device 100 will be explained. The template matching device 100 includes a processing circuit 101 and an input / output interface 102. The processing circuit 101 is hardware that implements the observation image acquisition unit 110, the pixel comparison unit 120, the position detection unit 130, the observation image storage unit 140, the template image storage unit 150, the coordinate information storage unit 160, and the overall control unit 170. The processing circuit 101 may be dedicated hardware, or it may be a processor that executes a program stored in memory. The program can be recorded (stored) in a computer-readable format on a non-volatile recording medium such as an optical disc or flash memory.

[0090] If the processing circuit 101 is dedicated hardware, the processing circuit 101 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof. ASIC is an abbreviation for Application Specific Integrated Circuit. FPGA is an abbreviation for Field Programmable Gate Array.

[0091] The template matching device 100 may include multiple processing circuits that replace the processing circuit 101.

[0092] In the processing circuit 101, some functions may be implemented by dedicated hardware, while the remaining functions may be implemented by software or firmware.

[0093] Thus, the functions of the template matching device 100 can be realized through hardware, software, firmware, or a combination thereof.

[0094] Each embodiment is an example of a preferred form and is not intended to limit the technical scope of this disclosure. Each embodiment may be implemented in part or in combination with other embodiments. Procedures described using flowcharts, etc., may be modified as appropriate.

[0095] The template matching device 100 may be read as "image processing device" or "image processing circuit". The word "device" in "template matching device 100" may be read as "circuit". The "part" of each element of the template matching device 100 may be read as "processing," "process," "circuit," or "circuit." [Explanation of Symbols]

[0096] 100 Template matching device, 101 Processing circuit, 102 Input / output interface, 110 Observation image acquisition unit, 120 Pixel comparison unit, 121 Feature pixel extraction unit, 122 Target pixel extraction unit, 123 Distance calculation unit, 124 Comparison range control unit, 130 Position detection unit, 140 Observation image storage unit, 150 Template image storage unit, 160 Coordinate information storage unit, 170 Overall control unit.

Claims

1. A feature pixel extraction unit that extracts a plurality of feature pixel sets from the template image, with one or more feature pixels that are located between a fixed distance from two or more feature pixels on the same line in the template image as a feature pixel set; a target pixel extracting unit that extracts, for each extracted feature pixel set, a plurality of target pixels to be compared with the feature pixel set from the observed image; a comparison range control unit that determines, for each of the extracted plurality of target pixels, two or more target pixels for which pixel distances to the plurality of feature pixel sets are calculated from among the plurality of target pixels; a distance calculation unit that calculates a pixel distance to the plurality of feature pixel sets for each of the determined target pixels; a position detection unit that detects a position corresponding to the template image from the observed image based on the pixel distance of each target pixel; A template matching device comprising:

2. A method for extracting a plurality of feature pixel sets from a template image, the feature pixel sets being one or more feature pixels that are located between a fixed distance from two or more feature pixels on the same line in the template image; extracting from the observed image a plurality of target pixels to be compared with each of the extracted feature pixel pairs; determining, for each of the extracted plurality of target pixels, two or more target pixels for which pixel distances to the plurality of feature pixel sets are calculated from among the plurality of target pixels; Calculating a pixel distance for each of the determined target pixels to the plurality of sets of feature pixels; A position corresponding to the template image is detected from the observed image based on the pixel distance of each target pixel. Template matching method.

Images