Verification device and program

Abstract

To provide a technique capable of detecting a collation area even if the collation area exists on a printing base material having unevenness such as a hologram.SOLUTION: A registration image photographing device 20 acquires a registration image 16 and stores it in a registration image DB 50b. A collation image photographing device 22 photographs an ink portion 14 printed on a hologram, extracts the shape of the ink portion 14, and transmits a collation image 18 to a server computer 50. The shape of the ink portion 14 is extracted from an image obtained by simultaneously performing smoothing processing and grayscale difference emphasizing processing, by simultaneously performing the smoothing processing and grayscale difference emphasizing processing on the captured image using a mean shift filter.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a collation device and a program. [Background technology]

[0002] Patent document 1 describes a processing system having a matte-finish forming means for forming a matte pattern on a part or product to which an information display device that displays information about the part, product, or product that includes the part as a component is attached, in order to identify the part, product, or product that includes the part as a component.

[0003] Patent Document 2 describes a technology that generates a layer containing minute particles and having an irregular planar shape on an object, acquires an image of the generated layer, and extracts features that depend on the planar shape of the layer and the distribution of the particles from the image as an individual identifier of the object.

[0004] Patent document 3 describes an individual recognition device that recognizes overlapping objects in a captured image on an individual basis, and includes a means for forming a binary image of the captured image, a means for extracting edges from the captured image, a means for performing differential processing between the binary image and the edges to form an image from which the edges have been removed from the binary image, and a means for combining binary images separated by edges based on the length of the objects in the image. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6455679 [Patent Document 2] Patent No. 6708981 [Patent Document 3] Patent No. 4930789 Summary of the Invention [Problem to be solved by the invention]

[0006] In a system that photographs an image of the surface of an object to obtain a matching area, and then uniquely identifies the object by matching the matching area with a pre-registered random pattern image of the fine pattern on the object's surface.When a user photographs the object's matching area using an imaging means such as a mobile terminal and compares it with the registered image, if the matching area is on a printed substrate with an uneven surface such as a hologram, it becomes difficult to detect the matching area due to the unevenness of the printed substrate.

[0007] An object of the present invention is to provide a technique that can detect a matching area even if the matching area exists on a printing substrate with an uneven surface such as a hologram. [Means for solving the problem]

[0008] A first aspect of the present invention provides a method for implementing a method of controlling a computer system, the method comprising: (a) acquiring a photographed image including a matching area provided on a printing substrate having a texture; (b) simultaneously performing a smoothing process and a shading enhancement process on the captured image; (c) detecting the matching region based on an image obtained by simultaneously executing the smoothing process and the gray-difference enhancement process; a smoothing process and a shading difference enhancement process are simultaneously performed on the captured image at a ratio according to the printing substrate; It is a matching device.

[0010] No. 2 In the aspect of (b), the processor further performs a shade difference enhancement process on the image obtained by simultaneously performing the smoothing process and the shade difference enhancement process between the steps (b) and (c). First Aspect The present invention relates to a matching device.

[0011] No. 3 In the aspect of (b) and (c), the processor removes, as noise, straight lines or curves existing on specific boundaries in the HSV color space from the image obtained by simultaneously executing the smoothing process and the shading difference emphasis process. First Aspect The present invention relates to a matching device.

[0012] No. 4In the aspect of (b), the processor performs a binarization process on the image obtained by simultaneously performing the smoothing process and the gray-difference emphasis process between the steps (b) and (c), and removes, as noise, straight lines or curves that do not fit the predetermined shape of the matching area from the image obtained by the binarization process. First Aspect The present invention relates to a matching device.

[0013] No. 5 In the aspect of (b) and (c), the processor performs a binarization process on an image obtained by simultaneously performing the smoothing process and the gray-difference emphasis process, and calculates, for the image obtained by the binarization process, the coordinates of the centers of gravity of a plurality of intersections that are candidates for vertices constituting the shape of the predetermined matching area, as the coordinates of the vertices. First Aspect The present invention relates to a matching device.

[0014] No. 6 This aspect is a matching device according to the fifth aspect, in which, when the processor is unable to calculate the coordinates of all vertices that make up the shape of the matching area, it uses the shape of the matching area to complement the coordinates of the remaining vertices.

[0015] No. 7 In the embodiment, the printing substrate is paper or a hologram. First Aspect The present invention relates to a matching device.

[0016] No. 8 In the aspect of the present invention, the processor simultaneously performs the smoothing process and the gray-difference enhancement process on the captured image by clustering processing. First Aspect The present invention relates to a matching device.

[0017] No. 9 In the aspect of the present invention, the processor simultaneously performs the smoothing process and the gray-difference enhancement process on the captured image by a mean-shift filtering process. First Aspect The present invention relates to a matching device.

[0018] No. 10A ninth aspect of the present invention is a matching device according to a ninth aspect, wherein the processor variably sets a color space radius sr and a pixel space radius sp in the mean-shift filtering process according to the printing substrate.

[0019] No. 11 A seventh aspect is the matching device according to the tenth aspect, wherein the processor sets the color space radius sr relatively larger when the printing substrate is a hologram than when the printing substrate is paper.

[0020] No. 12 The present invention relates to a method for implementing a method for programming a computer processor, comprising: (a) acquiring a photographed image including a matching area provided on a printing substrate having a texture; (b) simultaneously performing a smoothing process and a shading enhancement process on the captured image; (c) detecting the matching region based on an image obtained by simultaneously executing the smoothing process and the gray-difference enhancement process; death, a smoothing process and a shading difference enhancement process are simultaneously performed on the captured image at a ratio according to the printing substrate; It is a program that executes this. [Effects of the Invention]

[0021] 1st, 1st 8 , th 9 , th 12 According to this aspect, even if the verification area exists on a printing substrate having an uneven surface such as a hologram, the verification area can be detected.

[0022] No. 10 , th 11 According to this aspect, the collation area can be detected without relying on the printing substrate.

[0023] No. 2 ~No. 6 According to this aspect, the matching area can be detected more stably.

[0024] No. 7 According to this aspect, the collation area can be detected even if the print substrate is paper or a hologram. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 2 is a schematic plan view of a hologram-printed substrate and an ink portion according to an embodiment. [Figure 2] 2 is a schematic diagram of a hologram-printed substrate and an ink portion according to an embodiment. FIG. [Figure 3] 1A and 1B are explanatory diagrams showing an original image, an image obtained by smoothing the original image, and an image obtained by emphasizing the difference in brightness of the original image; [Figure 4] FIG. 2 is a configuration diagram of a registration image photographing device and a match image photographing device according to the embodiment. [Figure 5] 1 is a block diagram illustrating a configuration of a verification device according to an embodiment. [Figure 6] 1 is a flowchart of an overall process according to an embodiment. [Figure 7] 10A and 10B are explanatory diagrams illustrating simultaneous execution of smoothing processing and gray-difference emphasis processing according to an embodiment. [Figure 8] FIG. 10 is a diagram illustrating a configuration for simultaneously executing smoothing processing and gray-difference emphasis processing according to an embodiment. [Figure 9] 10 is a detailed flowchart of a mean-shift filtering process according to an embodiment. [Figure 10] 10A and 10B are explanatory diagrams of mean-shift filtering processing for a hologram-printed substrate and a paper-printed substrate according to an embodiment. [Figure 11] 10A to 10C are explanatory diagrams of a shading difference emphasis process according to an embodiment. [Figure 12] FIG. 10 is a diagram illustrating a noise removal process according to an embodiment. [Figure 13] 10A and 10B are explanatory diagrams illustrating a process of removing unnecessary sides according to an embodiment. [Figure 14] 10 is a flowchart of a process for calculating the center of gravity of an intersection according to an embodiment. [Figure 15] 10A to 10C are explanatory diagrams illustrating a process of complementing a missing edge according to an embodiment. [Figure 16] 10A to 10C are explanatory diagrams illustrating processes according to ink portion shapes and printing substrate types in an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0026] Hereinafter, an embodiment of the present invention will be described with reference to the drawings, taking as an example an individual identification system that takes a surface image of an object and uniquely identifies the object by comparing a registered image with a comparison image.

[0027] An individual identification system is a technology that pre-registers a portion of an object's surface—specifically, an image of a size between 0.1 and several millimeters—as unique information about the object. The system then uniquely identifies the object being compared as the registered object, i.e., as genuine. This unique information is, for example, a random pattern based on a fine pattern. A specific example of a random pattern based on a fine pattern is a matte finish. This matte finish is not limited to surface finishes such as frosted glass, but also includes matte finishes applied by processing metals or synthetic resins (e.g., plastics), as well as wrinkled patterns obtained by embossing, randomly woven fiber patterns, random fine dot patterns printed, and random particle distributions printed with ink containing glitter particles. Furthermore, it includes not only matte finishes formed unintentionally and accidentally, but also matte finishes intentionally formed for identification or matching purposes. In short, it is a random pattern that is difficult to control and form. Optically reading such random patterns and using them as information can be considered a type of "artifactometrics."

[0028] Here, we consider a case where a printing substrate with a texture such as a hologram is used as the printing substrate, and polygonal ink portions with dispersed metal particles are printed on such a printing substrate with a texture to create a random pattern.

[0029] FIG. 1 shows an example of an object 10. The object 10 includes a hologram portion 12 as a printing substrate, a QR code (registered trademark) portion 13 printed on the hologram portion 12, and an ink portion 14 printed on the hologram portion 12. The ink portion 14 is polygonal, more specifically, square, and is registered as a matching area. During matching, the matching area 14 is acquired, and the registered image and the matching image 14 are used for comparison and matching. The matching area is the area of ​​the ink portion 14, and the registered image and the matching image use the entire matching area or a portion thereof.

[0030] The ink section 14 is formed by mixing metal particles such as aluminum into a powder. When light is irradiated onto the ink section 14, the light is reflected by the randomly layered metal particles of the powder, creating a pattern of light and shade that varies in accordance with the intensity of the reflected light. The random pattern of light and shade changes when the direction of light irradiation is changed.

[0031] Figure 2 shows a schematic diagram of the relationship between hologram portion 12 and ink portion 14. In order to extract match image 18 from an image obtained by photographing object 10, it is necessary to accurately detect square ink portion 14 as a match area. Specifically, it is necessary to accurately detect square ink portion 14, i.e., the four vertices P1, P2, P3, and P4 that make up the square, from an image in which the concaves and convexes of hologram portion 12 and ink portion 14 are mixed.

[0032] Generally, feature extraction processing and edge extraction processing are used to extract a region of a specific shape from a captured image. However, simple feature extraction processing and edge extraction processing extract both the unevenness of hologram portion 12 and the unevenness of ink portion 14 as feature points, making it difficult to stably extract the shape of ink portion 14 despite the rainbow color changes of hologram portion 12.

[0033] FIG. 3 shows an example of a process for extracting the shape of the ink portion 14 from a captured image. FIG. 3( a ) is an original image obtained by photographing, and includes both the hologram portion 12 and the ink portion 14 . 3(b) shows the original image after a contrast enhancement process has been applied to extract its features. The contrast enhancement process emphasizes the unevenness of the ink portion 14, but at the same time, it also emphasizes the unevenness of the hologram portion 12, so that the unevenness of the ink portion 14 is buried in the unevenness of the hologram portion 12. Figure 3(c) shows an image in which the original image has been smoothed to remove irregularities rather than extracting features. Smoothing alone cannot completely remove the irregularities of the hologram portion 12 and the ink portion 14. Furthermore, excessive smoothing blurs the outline of the ink portion 14, making it impossible to extract the square shape. Figure 3(d) is an image obtained by applying contrast enhancement processing to the image in Figure 3(c). Figures 3(b) and 3(d) are Figure 3(b): Original image → contrast enhancement processing Figure 3(d): Original image → Smoothing process → Gray-scale enhancement process Although there are differences in the processing, it is difficult to extract the square of the ink portion 14 in either case.

[0034] Therefore, in this embodiment, a series of processes are performed on the original image to stably extract the ink portion 14 formed on the hologram portion 12, in other words, the ink portion 14 as the foreground from the hologram portion 12 as the background of the captured image.

[0035] 4 shows the system configuration of this embodiment. The matching system is made up of a registration image photographing device 20, a match image photographing device 22, and a server computer 50. The registration image photographing device 20 and the server computer 50, and the match image photographing device 22 and the server computer 50 are connected via a communication network.

[0036] The object 10 is illuminated by a light source unit 21 such as an LED, and the light reflected from the ink portion 14 of the object 10 is photographed by a registration image photographing device 20 to obtain a registration image 16. The registration image photographing device 20 and the light source unit 21 may be configured as dedicated equipment for registration.

[0037] The irradiation angle φ of the light emitted from the light source unit 21 is set to a certain angle. The acquired registered image 16 is transmitted to the server computer 50 and stored in the registered image DB 50b in the server computer 50.

[0038] On the other hand, a portable terminal such as a smartphone held by a user of the matching system serves as a matching image photographing device 22 to photograph the object 10. A light source unit 22a such as an LED mounted on the smartphone or the like illuminates the object 10, and a camera 22b mounted on the smartphone or the like photographs the light reflected from the ink portion 14 of the object 10. The irradiation angle φ of the light irradiated from the light source unit 22a is set to be substantially the same as the angle φ, which is the condition when the registered image 16 was acquired. The reason for this is that, as described above, the random pattern of the ink portion 14 changes depending on the direction of light irradiation, so the positional relationship between the light source unit 22a, camera 22b, and object 10 needs to be set to be substantially the same as the positional relationship when the registered image 16 was photographed.

[0039] The processor of the match image photographing device 22 performs a series of processes on the photographed image to extract the ink portion 14 from the photographed image, and then cuts out the match image 18 from within the area of ​​the ink portion 14 and transmits it to the server computer 50 via the communications network. The processing of the processor of the match image photographing device 22 will be described in more detail below.

[0040] The server computer 50 includes a matching unit 50a and a registered image DB 50b.

[0041] The registered image DB 50b is configured with a storage device such as a hard disk or SSD (Solid State Drive), and stores an identifier ID for uniquely identifying the object 10 and the registered image 16 in association with each other.

[0042] The matching unit 50a is composed of a processor and stores the registered image 16 received from the registered image photographing device 20 in the registered image DB 50b in association with the ID of the object 10. It also performs image matching between the match image 18 received from the match image photographing device 22 and the registered image 16 stored in the registered image DB 50b, and outputs the matching result to the match image photographing device 22. Specifically, the matching unit 50a reads the registered image 16 from the registered image DB 50b, performs a matching calculation with the match image 18, and calculates the similarity between the two images. The similarity calculation can use feature matching based on feature detection or template matching using image shading comparison. The calculated similarity is compared with a threshold value, and if it exceeds the threshold, it is determined that the two images match; if it does not exceed the threshold, it is determined that the two images do not match. The matching unit 50a transmits the matching result to the match image photographing device 22 via a communication network.

[0043] Image matching involves an error rate due to fluctuations in the input of the image sensor of the registration image capture device 20 or the match image capture device 22, quantization errors, and other factors. The error rate is made up of two components: the false rejection rate, which is the probability that a true value is judged as false, and the false acceptance rate, which is the probability that a false value is judged as true. There is a trade-off between the two, and as one decreases, the other increases. Therefore, the threshold is set to minimize the loss in the target to which the matching judgment is applied.

[0044] It is also possible to obtain a plurality of registered images 16 by changing the direction of light irradiation and register them in the registered image DB 50b of the server computer 50, and perform image matching between these registered images 16 and the match image 18.

[0045] 5 is a block diagram showing the main components of a match image photographing device 22, such as a smartphone. In addition to the light source unit 22a and camera unit 22b described above, the match image photographing device 22 includes a processor 22c, a ROM 22d, a RAM 22e, an input unit 22f, an output unit 22g, and a communication I / F 22h.

[0046] The processor 22c reads out an application program stored in the ROM 22d, executes a series of processes using the RAM 22e as a working memory, extracts the ink portion 14 from the image captured by the camera unit 22b, and cuts out a match image 18. The processor 22c transmits the cut-out match image 18 to the server computer 50 via the communication I / F 22h. The processor 22c also receives a match result from the server computer 50 via the communication I / F 22h.

[0047] The input unit 22f is composed of a keyboard, touch switches, etc., and is operated by the user to start an application program.

[0048] The output unit 22g is configured with a liquid crystal display, an organic EL display, or the like, and displays a preview image when photographing the object 10. Furthermore, when the ink portion 14 is extracted by the processor 22c, the output unit 22g may display a notification indicating detection in response to a control command from the processor 22c. Furthermore, the output unit 22g may display a guide when photographing the object 10 in response to a control command from the processor 22c. The guide is, for example, a guide for ensuring that the irradiation angle of the light emitted from the light source unit 22a forms a certain angle φ. Furthermore, the output unit 22g displays the matching result received from the server computer 50 in response to a control command from the processor 22c. The matching result is either "match" or "mismatch," but other messages related to the matching may also be displayed.

[0049] FIG. 6 is a flowchart showing the processing of a captured image by the processor 22c.

[0050] This processing flowchart aims to obtain the coordinates of the four vertices P1 to P4 of the square (quadrilateral) ink portion 14 from the captured image, and is broadly divided into three processes: a binary image generation process (S1), a rectangular edge extraction process (S2), and a vertex coordinate estimation process (S3).

[0051] <Binarized image generation processing> First, the binary image generation process (S1) will be described.

[0052] In this process, first, a smoothing process and a density difference enhancement process are simultaneously performed on the original image (S101). As mentioned above, simply performing a smoothing process on the original image will blur the shape of the ink portion 14. Furthermore, simply performing a density difference enhancement process on the original image will emphasize the unevenness of the ink portion 14, but will also emphasize the unevenness of the hologram portion 12, making it impossible to extract the ink portion 14.

[0053] Therefore, in this embodiment, smoothing processing and density difference enhancement processing are simultaneously performed on the original image to remove unevenness in the hologram portion 12 and the ink portion 14, thereby distinguishing the ink portion 14 from the hologram portion 12. Specifically, a mean-shift filter can be used to simultaneously perform the smoothing processing and density difference enhancement processing.

[0054] Figure 7(a) shows an original image 11. The original image 11 includes a hologram portion 12 and an ink portion 14. Figure 7(b) shows a processed image 13 obtained by applying a mean-shift filter to the original image 11. By simultaneously performing smoothing processing and contrast enhancement processing, it is possible to remove unevenness in the hologram portion 12 and the ink portion 14 while maintaining the shape of the ink portion 14.

[0055] Figure 8 shows a schematic diagram of the processing in S101. The original image 11 is input to the mean-shift filter 24, which simultaneously performs smoothing and contrast enhancement processing to output the processed image 13. The mean-shift filter 24 is implemented by the processor 22c. The mean-shift filter 24 is a filter that fills similar colors within a specified pixel space with the same color. This causes the silver ink portion 14 to approach the same color, and because the boundary between the rainbow background of the hologram 12 and the silver ink portion 14 is a different color region, the shape of the ink portion 14 is maintained while the contrast between the hologram portion 12 and the ink portion 14 is emphasized.

[0056] There are filtering methods that preserve edges and perform smoothing processing, such as bilateral filters, but the inventors have confirmed that these cannot remove noise from the hologram portion 12 and ink portion 14. By using a mean-shift filter, edges are preserved and smoothing is performed for each color using the color differences between the hologram portion 12 and ink portion 14, making it possible to remove noise without losing edges.

[0057] 9 shows a detailed flowchart of the filtering process of the mean-shift filter 24. The mean-shift filter 24 first searches for the center of gravity of the color distribution of the original image 11 (S201). That is, the mean-shift filter 24 calculates the center of gravity coordinates (xc, yc) and the color (rc, gc, bc) of a color space region of radius sr centered on the color (r, g, b) of a certain pixel (x, y), and searches for the center of gravity under the following conditions: where sp is the radius of the search region. Conditions: |x-xc|≦sp, |y-yc|≦sp, ||(r,g,b)-(rc,gc,bc)||≦sr And when the above conditions are met, (x,y,r,g,b)=(xg,yg,rc,gc,bc) The above-described centroid search process is repeated (NO in S202). Then, the color space distance ε and the number of repetitions n are set in advance, and it is determined whether the following condition is met, and if so, the process ends (YES in S202). Condition: Meet n iterations or |x-xc|+|y-yc|+(r-rc) 2 +(g-gc) 2 +(b-bc) 2 <ε After the centroid search process is completed, the image is smoothed using the centroid value of the color space (S203). That is, after the polar search is completed, each pixel in the space is set to the centroid value of the color space. Then, edges are clarified using a Gaussian pyramid and a threshold value sr.

[0058] The mean-shift filter 24 performs smoothing processing using distance differences in color space, and is therefore an effective smoothing process when there is a difference in color space distance between the foreground and background. For this reason, it is an effective process for an original image 11 in which achromatic ink portions 14 exist in the foreground and chromatic hologram portions 12 exist in the background.

[0059] In the mean-shift filter 24, the performance of the smoothing process and the shading difference enhancement process can be controlled using the color space radius sr and the pixel space radius sp as main parameters. Therefore, by adjusting these parameters, the ratio of the smoothing process to the shading difference enhancement process can be adjusted. Specifically, (1) The pixel space radius sp specifies the search range for pixels to be smoothed (filled in), so Large SP → Wide search range SP is small → Search range is narrow Note that if sp is set too large, it will take a long time to process, so this should be taken into consideration. (2) The range of similar colors to be filled with the same color is determined by the color space radius sr. High SR → Slightly different colors are recognized as the same color Small sr → Similar colors are recognized as the same color and can be adjusted.

[0060] Therefore, it is desirable to adjust the values ​​of the parameters sr and sp as follows when the printing substrate is the hologram section 12 and when it is something else, such as paper. When the printing substrate is the hologram section 12, there is a color difference between the silver ink section 14 and the rainbow hologram section 12, so the color space radius sr is set relatively large, and the pixel space radius sp is set to a desired value taking into account processing time and effectiveness. When the printing substrate is paper, there is a small color difference between the silver ink section 14 and the paper, so the color space radius sr is set relatively small, and the pixel space radius sp is set relatively small.

[0061] FIG. 10 shows the results of filtering processing by the mean-shift filter 24 when the print substrate is the hologram section 12 and when it is paper 15.

[0062] 10(a) and 10(b) show the case where the printing substrate is a hologram portion 12, where FIG. 10(a) is the original image and FIG. 10(b) is the processed image. The parameters sp and sr of the mean-shift filter 24 are (sp,sr)=(10,30) and the shape of the ink portion 14 is extracted.

[0063] On the other hand, Fig. 10(c) and Fig. 10(d) show the case where the printing substrate is paper 15, Fig. 10(c) is the original image and Fig. 10(d) is the processed image. The parameters sp and sr of the mean-shift filter 24 are (sp,sr)=(5,10) and the shape of the ink portion 14 is similarly extracted. When the printing substrate is paper 15, both sp and sr are set relatively small compared to when the hologram portion 12 is used. In other words, when the printing substrate is hologram portion 12, both sp and sr are set relatively large compared to when the printing substrate is paper 15, and the two parameters sp and sr are variably set depending on the printing substrate.

[0064] In this manner, the processor 22c simultaneously performs smoothing processing and contrast enhancement processing on the captured original image as the mean-shift filter 24.

[0065] Returning to FIG. 6, after smoothing and contrast enhancement are simultaneously performed on the original image (S101), additional contrast enhancement is performed on areas where contrast cannot be obtained by the process of S101 (S102).

[0066] When the ink portion 14 is photographed with a matching image photographing device 22 such as a smartphone, the color of the hologram portion 12 around the ink portion 14 changes as the irradiation position of the light source unit 22a changes. In other words, if the color of the hologram portion 12 changes and there is not a sufficient difference in color space distance from the ink portion 14, the foreground and background may blend together, resulting in areas where the shading enhancement is insufficient with only the processing of S101. Therefore, by further performing shading enhancement processing, the shape of the ink portion 14 can be extracted more stably.

[0067] Specifically, the image 13 processed in S101 is decomposed into RGB, and a grayscale enhancement process is performed in each RGB color space. This means flattening the brightness histogram within the image. Then, a Sobel filter is applied to each of the vertical and horizontal RGB color images to extract the edge gradient. Note that the gradient value calculated by the Sobel filter is not 8 bits (256 gradations), so it can be normalized to 8 bits. The normalization method is to take the absolute value of the gradient image and replace all pixel values ​​above 255 with 255. This makes it possible to obtain the edge gradient without relying on external noise.

[0068] FIG. 11 shows a schematic representation of the processing in S102. FIG. 11(a) shows the original image 11, and FIG. 11(b) shows the processed image 13 in S101. FIG. 11(c) shows an image 17 obtained by directly binarizing the processed image in FIG. 11(b), in which the top edges of the square ink portions 14 have disappeared. In contrast, FIG. 11(d) shows an image 19 obtained by further binarizing the processed image 13 in FIG. 11(b) through a density contrast enhancement process. The top edges of the square ink portions 14 have also been extracted.

[0069] Returning to Figure 6, after performing additional shading enhancement processing (S102), noise reduction processing is performed using the HSV color space (S103). Here, the HSV color space is a color space consisting of three components: hue, saturation (Saturation / Chroma), and brightness (Value / Brightness).

[0070] When the rough shape of the ink portion 14 is extracted in S102, noise occurs at the boundary between the white and light blue of the hologram portion 12. In particular, because the gradient between the white and light blue of the R space image is large, edge-like noise occurs when a Sobel filter is applied. Therefore, this noise is removed using the HSV color space. Specifically, (1) HSV decomposition of the processed image 13 in S101 (2) Binarize the S image (3) Apply vertical and horizontal Sobel filters to the binary image. (4) OR composite the vertical and horizontal Sobel image and the black and white inverted binary image of the H image.

[0071] Fig. 12 shows a schematic diagram of the processing in S103. In Fig. 12(a), noise occurring at the boundary between white and light blue in image 19 is indicated by dashed line area 26. Fig. 12(b) shows image 21 processed in S103. This image 21 is an OR composite image of a vertical and horizontal Sobel image and a black and white inverted binary image of an H image, and the noise in dashed line area 26 has been removed.

[0072] Returning to FIG. 6 again, after the process of S103 is executed, a binarized image is created (S104). That is, a total of six vertical and horizontal gradient images of the R, G, and B images are binarized. Different binarization thresholds may be set for R, G, and B. Then, a total of six binarized images of the vertical and horizontal components and RGB color components are OR-combined.

[0073] <Rectangle edge extraction processing> Next, the rectangular edge extraction process will be described.

[0074] After creating a binarized image in S104, the edges of the polygons that make up the square ink portion 14 are obtained from this binarized image (S105). Specifically, this is edge extraction processing using a probabilistic Hough transform. Note that the probabilistic Hough transform is an optimized version of the Hough transform, and instead of using all pixels, calculations are performed by randomly selecting enough points from the image to detect straight lines. A (non-probabilistic) Hough transform can also be used in S104. However, this has the drawback of being difficult to tune the parameters and being too sensitive to rectangular edges in the binarized image.

[0075] After obtaining the polygon edges (S105), a process for removing unnecessary edges is performed (S106). That is, lines that are not rectangular edges (edges) are removed from the lines extracted by the probabilistic Hough transform. Specifically, methods are used such as removing lines with a slope of a certain value or more, lines that are long relative to the size of the matching area 14, lines where the angle between vertical and horizontal lines is within a certain angle, or lines that are tangent to the image frame. Alternatively, unnecessary edges may be removed by using a color space to extract edges with a color that resembles a rectangle.

[0076] FIG. 13 shows a schematic diagram of the processing of S106. FIG. 13(a) shows image 21 before processing, and FIG. 12(b) shows processed image 23. Some of the straight lines that existed on the left side of image 21 have been removed. By removing the unnecessary edges, it is possible to reduce the processing time required for estimating vertex coordinates in the subsequent stage. It is also possible to improve shape acquisition accuracy and disturbance resistance.

[0077] <Vertex coordinate estimation process> Returning to FIG. 6, after the rectangular edge extraction process (S2) is completed, the vertex coordinate estimation process (S3) of the square ink portion 14 is executed.

[0078] In this process, the centroid coordinates of the intersections of the edges are calculated from the image obtained by removing the unnecessary edges in S106 (S107). That is, instead of the intersections of each edge, the centroid coordinates of a group of intersections within a certain neighborhood are calculated. The intersections of vertical and horizontal lines are calculated for the processed image 23 by solving one-dimensional simultaneous equations. However, because the edge width of the binary image after OR compositing is 2 to 3 pixels, multiple lines are extracted for the same edge using the probabilistic Hough transform. As a result, multiple intersections exist near a certain coordinate. Since these intersections are likely to represent the same vertex, the centroid coordinates of the group of intersections are obtained and redefined as the vertices of the shape of the ink portion 14.

[0079] FIG. 14 is a detailed flowchart of the process of calculating the coordinates of the center of gravity of the intersection group.

[0080] First, multiple intersections within a certain neighborhood are combined into one by dilation (S301). The dilation process is a process in which, if there are white pixels in the surrounding pixels of a certain pixel, the pixels are converted to white pixels, thereby successively expanding the white pixels. Next, labeling is performed on each dilated point set (S302). Then, the barycentric coordinates of each labeled point set are calculated (S303). After calculating the barycentric coordinates in this manner, the calculated barycentric coordinates are set as vertex candidates (S304).

[0081] Since the square ink portion 14 has four vertices P1 to P4 (see FIG. 2), four vertex candidates are set in the process of S304. When setting the vertex candidates, known shape characteristics of the ink portion 14, i.e., the lengths of the sides and diagonals, can be used as conditions. If there are multiple pairs of vertices that satisfy the conditions, the most plausible pair of vertices is selected. For example, for the square ink portion 14, the condition that the lengths of the four sides are equal can be used, and the pair of vertices that has the smallest variance in the side lengths can be selected as the pair of vertices.

[0082] Then, it is determined whether all the vertices of the ink portion 14 have been acquired (S108). In the case of a square ink portion 14, it is determined that all the vertices have been acquired when four vertices P1 to P4 have been acquired. If all the vertices cannot be acquired (NO in S108), this means that not all the sides of the ink portion 14 have been extracted, and so next, a process of complementing the missing sides is performed (S109).

[0083] In the edge interpolation process, it is determined whether the three edges that make up the square ink portion 14 have been extracted. Normally, when the ink portion 14 is printed on top of the hologram portion 12 that serves as the printing substrate, extraction of the edge may fail if the red background color of the hologram portion 12 overlaps with the red foreground color. In other words, this occurs when the difference in color space distance between the background and foreground is small. Therefore, first, it is determined whether or not three sides have been extracted. The selection of the three sides can be estimated from the known shape characteristics of the ink portion 14, that is, the lengths and positions of the edges.

[0084] If three sides have been extracted, the length x of the side that does not have an opposite side is calculated from the centroid coordinates calculated in S107. Then, a new parallel side is drawn at a position the length x away from the side. Specifically, the four sides that make up a square are a, b, c, and d, with a and c forming opposite sides and b and d forming opposite sides. When only the three sides a, b, and c are extracted, a side parallel to b is drawn at a position x away from b, and is called d.

[0085] Two parallel edges can be estimated, one on each side of b, that are distance x from b. However, one of them does not exist in the image, so edge d can be drawn uniquely. This completes the missing edge.

[0086] Figure 15 shows a schematic representation of the processing of S109. Figure 15(a) shows an image 25 before processing, in which three sides of the square have been extracted, but the remaining side has disappeared. Figure 15(b) shows an image 27 after processing, in which a new parallel side 28 has been drawn at a distance of the side length x. After the missing side has been interpolated, the barycentric coordinates of the intersections are calculated again to obtain the coordinates of all vertices.

[0087] After the missing edges are complemented, the threshold may be lowered and the probabilistic Hough transform may be performed again on the binarized image obtained in S104 to re-obtain the edges, and the edges thus obtained may be integrated with the edges obtained by complementation in S109, and the vertex coordinate estimation process (S3) may be resumed.

[0088] Once the coordinates of the four vertices P1 to P4 of the ink portion 14 are acquired in this manner, the processor 22c cuts out the match image 18 based on the coordinates of these four vertices and transmits it to the server computer 50. The processor 122c converts the resolution of the match image 18 to a predetermined size and transmits it to the server computer 50. When transmitting it to the server computer 50, the processor 122c attaches the resolution-converted match image 18 to a match request.

[0089] In this embodiment, the ink portion 14 is described as being square, but this embodiment is not limited to a square, and can be applied to any polygonal shape.

[0090] FIG. 16 shows images after each process when a hologram portion 12 and paper 15 are used as the printing substrate, and a triangle and a rectangle (square) are used as the ink portion 14. In FIG. 16, (triangle, hologram) indicates that a hologram portion 12 is used as the printing substrate, and a triangular ink portion 14 is used as the ink portion 14. Similarly, (square, hologram) indicates that a hologram portion 12 is used as the printing substrate, and a rectangle (square) ink portion 14 is used as the ink portion 14. (square, paper) indicates that paper 15 is used as the printing substrate, and a rectangle (square) ink portion 14 is used as the ink portion 14. In addition, the "original image" is the captured image before processing, the "simultaneous processing of smoothing and shading difference enhancement" is the image after processing at S101 in FIG. 6, the "binarized image" is the image after processing at S104 in FIG. 6, the "obtaining polygon edges" is the image after processing at S105 in FIG. 6, and the "obtaining polygon vertex coordinates" is the image after processing at S108 in FIG. 6.

[0091] Furthermore, although the processing of the processor 22c in this embodiment has been described in detail, the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPU Central Processing Unit, etc.) and dedicated processors (e.g., GPU Graphics Processing Unit, ASIC Application Specific Integrated Circuit, FPGA Field Programmable Gate Array, programmable logic device, etc.). Furthermore, the operations of the processor in the embodiment may not only be performed by a single processor, but may also be performed by multiple processors located in physically separate locations working together. Furthermore, the order of each operation of the processor is not limited to the order described in the embodiment, and may be changed as appropriate.

[0092] In addition, in this embodiment, the processing shown in Figure 6 is executed by the processor 22a of the matching image photographing device 22, such as a smartphone, but instead, at least a part of the processing shown in Figure 6 may be executed by the matching unit 50a of the server computer 50.

[0093] Furthermore, in this embodiment, the ink portion 14 is printed on the hologram portion 12 as a printing substrate, i.e., the combination of a chromatic background and an achromatic foreground has been described. However, the processing of this embodiment can also be applied to the opposite combination of an achromatic background and a chromatic foreground. The basic principle of the mean-shift filter 24 is to extract the shape of the foreground using the color space distance difference between the achromatic and chromatic colors, and this principle can be similarly applied to the combination of an achromatic background and a chromatic foreground. An example of a combination of an achromatic background and a chromatic foreground is the combination of paper 15 as a printing substrate and chromatic ink portion 14.

[0094] Furthermore, in this embodiment, if the color space distance difference between the hologram portion 12 and the ink portion 14 is relatively small and the shape of the ink portion 14 cannot be sufficiently extracted using the mean-shift filter 24, the well-known Canny method may be used as a complement.

[0095] That is, the edges obtained by the probabilistic Hough transform are masked for the binarized image obtained in S104.

[0096] Next, assuming that the straight lines on the blue side have been extracted, the blue half of the image is masked. This is because mean-shift filtering is robust to the blue side but vulnerable to the red side, while Canny filtering is vulnerable to the blue side but robust to the red side.

[0097] Next, edges are extracted using the Canny algorithm. Because edges extracted using the Canny algorithm are drawn using a single pixel, they may not be able to be drawn as straight lines using the probabilistic Hough transform. Therefore, the edges extracted using the Canny algorithm are dilated. This extracts edges of about three pixels, which can then be extracted using the probabilistic Hough transform.

[0098] Below, examples of methods that can be used as a complement when shape extraction by the mean-shift filter 24 is insufficient will be listed. Use image contrast enhancement · Lowering the threshold for binarization and probabilistic Hough transform - Extract edges using the Canny algorithm Completing missing edges using the known shape of the ink part 14 Any one of these methods or a combination of two or more of them may be used. [Explanation of symbols]

[0099] 10 object, 12 hologram section, 14 ink section, 16 registered image, 18 match image, 20 registered image capture device, 22 match image capture device, 50 server computer.

Claims

1. The processor executes a program to (a) acquiring a photographed image including a matching area provided on a printing substrate having a texture; (b) simultaneously performing a smoothing process and a shading enhancement process on the captured image; (c) detecting the matching region based on an image obtained by simultaneously executing the smoothing process and the gray-difference enhancement process; a smoothing process and a shading difference enhancement process are simultaneously performed on the captured image at a ratio according to the printing substrate; Collation device.

2. The processor, between (b) and (c), further performing a shade difference enhancement process on the image obtained by simultaneously performing the smoothing process and the shade difference enhancement process; The verification device according to claim 1 .

3. The processor, between (b) and (c), a straight line or a curve existing on a specific boundary in the HSV color space is removed as noise from the image obtained by simultaneously executing the smoothing process and the shading difference enhancement process; The verification device according to claim 1 .

4. The processor, between (b) and (c), performing a binarization process on the image obtained by simultaneously performing the smoothing process and the gray-difference enhancement process; removing, as noise, straight lines or curves that do not fit the predetermined shape of the matching area from the image obtained by the binarization process; The verification device according to claim 1 .

5. The processor, between (b) and (c), performing a binarization process on the image obtained by simultaneously performing the smoothing process and the gray-difference enhancement process; For the image obtained by the binarization process, the coordinates of the centers of gravity of a plurality of intersections that are candidates for vertices constituting the predetermined shape of the matching area are calculated as the coordinates of the vertices. The verification device according to claim 1 .

6. The processor: If it is not possible to calculate the coordinates of all vertices constituting the shape of the matching area, the coordinates of the remaining vertices are interpolated using the shape of the matching area. The verification device according to claim 5 .

7. The printing substrate is paper or a hologram. The verification device according to claim 1 .

8. the processor simultaneously performs the smoothing process and the gray-difference enhancement process on the captured image by clustering processing; The verification device according to claim 1 .

9. the processor simultaneously performs the smoothing process and the gray-difference enhancement process on the captured image by a mean-shift filtering process; The verification device according to claim 1 .

10. the processor variably sets a color space radius sr and a pixel space radius sp in the mean-shift filtering process according to the printing substrate; The verification device according to claim 9.

11. the processor sets the color space radius sr relatively larger when the printing substrate is a hologram than when the printing substrate is paper; The verification device according to claim 10.

12. The computer processor (a) acquiring a photographed image including a matching area provided on a printing substrate having a texture; (b) simultaneously performing a smoothing process and a shading enhancement process on the captured image; (c) detecting the matching region based on an image obtained by simultaneously executing the smoothing process and the gray-difference enhancement process; a smoothing process and a shading difference enhancement process are simultaneously performed on the captured image at a ratio according to the printing substrate; A program that makes it happen.

Images