Verification device and verification program
By capturing multiple images with adjusted light directions, the verification device addresses the issue of inconsistent photographing conditions, ensuring accurate verification image matching.
Patent Information
- Application Number
- JP2020212626
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2020-12-22
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2040-12-22
AI Technical Summary
Existing verification devices struggle to accurately match verification images due to variations in photographing conditions, such as camera position and light direction, leading to false determinations of similarity between images.
The verification device captures multiple images of a verification image by adjusting the direction of light irradiation to ensure consistent positional relationships, allowing for a more accurate comparison with registered images using a processor to determine similarity.
This approach enhances the accuracy of verifying whether an inspection image is a duplicate of a registered image by accounting for variations in light direction and camera position, improving matching precision.
Smart Images

Figure 0007585767000001 
Figure 0007585767000002 
Figure 0007585767000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a matching device and a matching program. [Background technology]
[0002] Patent Document 1 discloses a device authentication system including an authenticated device having a random pattern and a device authentication device that acquires the random pattern of the authenticated device and authenticates its authenticity, wherein the authenticated device has light-reflecting glitter embedded in a highly light-transmitting base material, and the device authentication device includes a light source that irradiates light onto the authenticated device, a random pattern reading means that reads the random pattern of the authenticated device from an image of the light reflected by the authenticated device, a first data acquisition means that acquires first ID data unique to the authenticated device from the random pattern read by the random pattern reading means, a second data recording means that records second ID data for comparison to determine the authenticity of the first ID data, a second data acquisition means that acquires the second ID data recorded in the second data recording means, and an association calculation means that calculates an association between the first ID data and the second ID data and authenticates the authenticity of the authenticated device.
[0003] Patent Document 2 discloses an object identification device that includes an imaging means for imaging a target location on the surface of an object, an irradiation means for irradiating light onto the target location, a control means for sequentially switching the direction of the light irradiated from the irradiation means relative to the target location among a plurality of different directions, and for each of the plurality of directions, causing the imaging means to image the target location when the light is irradiated in that direction, thereby controlling the acquisition of captured images for each of the plurality of directions, and an identification means for identifying the object by comparing feature information indicated by the captured images for each of the plurality of directions with feature information registered for each object. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2004-171109 A [Patent Document 2] JP 2020-154946 A Summary of the Invention [Problem to be solved by the invention]
[0005] A matching device is known that identifies an object by using a pattern of a matching image formed on the surface of the object as identification information. In such a matching device, a captured image of an inspection image to be matched is compared with a registered image of an inspection image captured in advance according to a predetermined matching algorithm, and the inspection image is matched with the registered image.
[0006] However, even if the test image is a genuine match image, if the shooting conditions of the test image and the match image are different, the test image may be determined to be dissimilar to the match image.
[0007] For example, the positions of the camera and the light that illuminates the image differ depending on the model of imaging device, and the angle at which the photographer captures the image also differs, so even if the same area of the same image is captured, the relative positions of the image, camera, and light will be different, and as a result, the appearance of the image will change, and the test image may be determined to be not similar to the comparison image.
[0008] The present invention aims to provide a matching device and a matching program that can accurately match whether an inspection image is a copy of a matching image, compared to when matching an inspection image captured from one direction with a registered image. [Means for solving the problem]
[0009] The matching device of the first aspect includes a processor that acquires a plurality of captured images taken by changing the direction of light applied to the test image to be matched, adjusts the positional relationship between the camera, light source, and images so that it is the same as the positional relationship between the camera capturing the test image, the light source irradiating the test image, and the test image at the time the test image was captured, and then notifies the result of matching between the test image and the match image according to the degree of similarity between the test image contained in each of the captured images and a plurality of registered images that have been captured in advance by changing the direction of light applied to the match image, whose pattern changes depending on the direction of light.
[0010] The matching device of the second aspect is the matching device of the first aspect, in which the processor notifies the matching result using the registered image which includes four images taken by shifting the direction of light irradiating the matching image by 90 degrees around the periphery of the matching image.
[0011] The matching device of the third aspect is the matching device of the second aspect, in which the processor notifies the matching result using the registered images which include, in addition to the four images, four additional images which are taken by shifting the direction of light irradiation by less than 90 degrees around the periphery of the matching image relative to the direction of light irradiation applied to the matching image when capturing the four images.
[0012] The matching device of the fourth aspect is the matching device of the third aspect, in which the processor adds to the registered image as the four additional images images captured by shifting the direction of light irradiation by 45 degrees around the periphery of the match image relative to the direction of light irradiation applied to the match image when capturing the four images.
[0013] A matching device according to a fifth aspect is a matching device according to any one of the first to fourth aspects, in which the processor notifies the result of matching between the inspection image and the matching image using the registered image captured using light irradiated from a single point light source.
[0014] The matching device of the sixth aspect is the matching device of the fifth aspect, in which the processor notifies the matching result between the test image and the matching image using a registered image in which the angle between the optical axis of the point light source and the optical axis of a camera in a first photographing device that captures the registered image has been adjusted to match the angle between the optical axis of a camera in a second photographing device that captures the test image and the optical axis of a built-in point light source built into the second photographing device.
[0015] A matching device according to a seventh aspect is the matching device according to any one of the first to sixth aspects, wherein the matching image is a three-dimensional image formed three-dimensionally on a recording medium.
[0016] A collation device according to an eighth aspect is the collation device according to the seventh aspect, wherein the three-dimensional image is an image formed by fixing powder on the recording medium.
[0017] A matching device according to a ninth aspect is the matching device according to the eighth aspect, wherein the powder image is a captured image of a surface made of black or silver powder.
[0018] A matching device according to a tenth aspect is a matching device according to any one of the first to sixth aspects, in which the matching image is an image of the surface of an object to which a material composed of a mixture of silver powder is attached.
[0019] The matching program of the eleventh aspect is a program that causes a computer to execute a process of acquiring a plurality of captured images taken by changing the direction of light applied to a test image to be matched, adjusting the positional relationship between the camera, light source, and images so that it is the same as the positional relationship between the camera capturing the test image, the light source irradiating the test image, and the test image at the time the test image was captured, and then notifying the result of matching between the test image and the match image according to the degree of similarity between the test image contained in each of the captured images and a plurality of registered images that have been captured in advance by changing the direction of light applied to the match image, whose pattern changes depending on the direction of light application. Effect of the Invention
[0020] The first and eleventh aspects have the advantage that it is possible to accurately compare whether or not an inspection image is a copy of the comparison image, compared to a case in which an inspection image captured from one direction is compared with a registered image.
[0021] According to the second aspect, there is an advantage that the matching result can be notified if there are at least four registered images for one matching image.
[0022] According to the third aspect, there is an advantage that the test image can be compared with a case in which the test image is compared using four registered images.
[0023] According to the fourth aspect, it is possible to improve the accuracy of matching of a test image captured with a photographing device in which the camera and light source are positioned so that the angle between the outline of the photographing device and the line connecting the camera and light source built into the photographing device is 45 degrees, compared to a case in which the test image is matched with a registered image captured with the direction of light irradiating the match image shifted by 90 degrees around the periphery of the true image.
[0024] According to the fifth aspect, it is possible to improve the accuracy of matching an inspection image captured by an imaging device that irradiates light from a single point light source, compared to matching an inspection image using registered images captured using light irradiated from multiple light sources.
[0025] According to the sixth aspect, the test image can be matched more accurately than when the test image is matched using a registration image captured without adjusting the angle between the optical axis of the point light source and the optical axis of the first image capturing device when capturing the registration image to match the angle between the optical axis of the second image capturing device and the optical axis of the built-in point light source when capturing the test image.
[0026] According to the seventh aspect, there is an effect that a plurality of registered images with different patterns can be obtained from one match image.
[0027] According to the eighth aspect, there is an effect that it is possible to perform a comparison using a genuine image generated by an image forming apparatus that forms an image on a recording medium with powder.
[0028] According to the ninth aspect, there is an effect that the accuracy of matching between the inspection image and the match image can be improved compared to the case where a match image made of powder other than black and silver is used.
[0029] According to the tenth aspect, there is an effect that the accuracy of matching between the inspection image and the match image can be improved compared to an image using a material containing a powder other than silver. [Brief description of the drawings]
[0030] [Figure 1] FIG. 13 is an enlarged view of an example of a match image. [Diagram 2] 1A and 1B are diagrams showing an example of a genuine match image and a duplicate image obtained by duplicating the match image; [Diagram 3] FIG. 13 is a diagram illustrating an example of a graph showing the similarity between a match image and a duplicate image. [Figure 4] FIG. 13 is a diagram showing an example of a matching image for left illumination. [Diagram 5] FIG. 13 is a diagram showing an example of a collation image of lower illumination. [Figure 6] 13 is an example of a graph showing the similarity between a match image with left illumination and a match image with bottom illumination. [Figure 7] FIG. 1 is a configuration diagram illustrating an example of a collation system. [Figure 8] FIG. 13 is a diagram showing an example of a photographing form of an inspection image by a smartphone. [Figure 9] FIG. 13 is a diagram showing an example of a form of capturing a collation image using dedicated equipment. [Figure 10] FIG. 13 is a diagram showing an example of the photographing mode of a first smartphone having an orthogonal arrangement. [Figure 11] FIG. 13 is a diagram showing an example of the photographing mode of a second smartphone having an orthogonal arrangement. [Figure 12] FIG. 13 is a diagram showing an example of the shooting mode of a third smartphone having an orthogonal arrangement. [Figure 13] FIG. 13 is a diagram showing an example of the photographing mode of a fourth smartphone having an orthogonal arrangement. [Figure 14] FIG. 13 is a diagram showing an example of the shooting mode of a fifth smartphone equipped with an inclined arrangement. [Figure 15] FIG. 13 is a diagram showing an example of the shooting mode of a sixth smartphone equipped with an inclined arrangement. [Figure 16] FIG. 13 is a diagram showing an example of the shooting mode of a seventh smartphone having an inclined arrangement. [Figure 17] FIG. 13 is a diagram showing an example of the shooting mode of an eighth smartphone having an inclined arrangement. [Figure 18] FIG. 13 is a diagram showing an example of another photographing mode of the second smartphone. [Figure 19] FIG. 13 is a diagram showing an example of another photographing mode of the fourth smartphone. [Figure 20] FIG. 13 is a diagram showing an example of another shooting mode of the sixth smartphone. [Figure 21] FIG. 13 is a diagram showing an example of another shooting mode of the seventh smartphone. [Figure 22] FIG. 13 is a diagram showing an example of another shooting mode of the eighth smartphone. [Figure 23] FIG. 5 is a diagram showing an example of a shooting mode in which the smartphone is placed horizontally or vertically. [Figure 24] FIG. 11 is a diagram showing an example of a shooting mode in which a sixth smartphone is placed horizontally or vertically. [Figure 25] FIG. 13 is a diagram showing an example of a shooting mode in which the seventh smartphone is placed horizontally or vertically. [Figure 26] FIG. 13 is a diagram showing an example of a shooting mode in which the eighth smartphone is placed horizontally or vertically. [Figure 27] FIG. 2 is a diagram illustrating an example of the configuration of a main part of an electrical system in a computer. [Figure 28] 13 is a flowchart showing an example of a matching process executed by a CPU of the matching device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0031] Hereinafter, the present embodiment will be described with reference to the drawings. Note that the same components and processes are given the same reference numerals throughout the drawings, and duplicated explanations will be omitted.
[0032] 1 is an enlarged view of an example of a match image 1 used in this embodiment. The match image 1 is an image formed on a recording medium 4 (see FIG. 8) by an image forming apparatus that forms an image using a powder such as toner. In this embodiment, the match image 1 is described as being formed on the recording medium 4, but the match image 1 may be formed on an object other than the recording medium 4, such as a food container. In other words, the object on which the match image 1 is formed is not limited to a particular type as long as the object is one on which the match image 1 can be formed.
[0033] For example, when the match image 1 formed on the recording medium 4 using black powder is enlarged, it can be seen that the match image 1 is a three-dimensional image formed three-dimensionally by each powder agglomerating and being fixed on the recording medium 4, as shown in Fig. 1. A three-dimensional image is an image in which unevenness of at least the average particle diameter of the powder can be seen as light and shade in the layering direction of the powder on the recording medium 4. Since there are various average particle diameters of powder, powder of any average particle diameter can be used, but the average particle diameter of the powder forming the match image 1 in this embodiment is about 6 µm.
[0034] Also, an image formed by mixing metal particles into powder can be used as the match image 1. An image made of powder mixed with aluminum particles, which is an example of metal particles, appears silver in color, and a print image made of powder mixed with brass particles, which is also an example of metal particles, appears gold in color.
[0035] Since the powder constituting such match image 1 contains metal particles such as aluminum, when light is shone on match image 1, the light is reflected by the metal particles of the randomly layered powder, and match image 1 displays a pattern of light and dark depending on the intensity of the reflected light. Therefore, if match image 1 formed on recording medium 4 is attached to an object or match image 1 is formed directly on the object with an image forming device, the object and match image 1 can be associated with each other by determining the pattern of match image 1.
[0036] However, as the performance of information devices with a camera function, such as a smartphone 6 (see FIG. 7), and image forming devices has improved, if the genuine match image 1 is photographed by the smartphone 6 and the photographed match image 1 is formed on a recording medium by an image forming device, a duplicate image 2 that is a copy of the match image 1 is obtained. Specifically, if a smartphone 6 that can capture a photographed image with a resolution of 1000 dpi or an inkjet printer that forms an image with a resolution of 4800 dpi is used, a duplicate image 2 that shows the pattern characteristics of the match image 1 is formed.
[0037] Fig. 2 is a diagram showing an example of a genuine match image 1 and a duplicate image 2 of the match image 1 created using a smartphone 6 and an inkjet printer as described above. Note that the match image 1 and duplicate image 2 shown in Fig. 2 are each images captured at a resolution of 1012 dpi.
[0038] Although the duplicated image 2 shown in FIG. 2 is less clear than the genuine match image 1, it can be seen that the characteristics of the shading pattern in the match image 1 are reproduced in the duplicated image 2.
[0039] In this embodiment, a rectangular match image 1 is used for explanation, but there is no restriction on the shape of the match image 1, and it may be other shapes such as a circle, an ellipse, or a polygon.
[0040] Fig. 3 is a graph 3A showing an example of a calculation result of the similarity between the match image 1 and the duplicated image 2 shown in Fig. 2, calculated using a known matching algorithm such as ZNCC (Zero-mean Normalized Cross-Correlation). As an example, the resolution of the match image 1 and the duplicated image 2 shown in Fig. 2 is converted to 600 dpi, respectively, before the similarity is calculated. In addition, the matching algorithm used to calculate the similarity can improve the calculation accuracy of the similarity by combining multiple methods, and as an example, the similarity between the match image 1 and the duplicated image 2 is calculated using two types of methods.
[0041] In Fig. 3, the vertical axis indicates the similarity calculated by the first method, and the horizontal axis indicates the similarity calculated by the second method. For both the vertical and horizontal axes, the larger the values, the more similar the images are. Thus, moving to the upper right along the diagonal of Graph 3A indicates that the images are more similar, and moving to the lower left of Graph 3A indicates that the images are less similar.
[0042] The "□" in Figure 3 indicates the results of calculating the similarity between match image 1 and duplicated image 2 for each corresponding point. The "□" points are concentrated in the upper right region of graph 3A, which shows that match image 1 and duplicated image 2 are similar.
[0043] 3 are points that indicate the results of calculating the similarity for each corresponding part between images, for example, two shots of the same match image 1. Naturally, since the same match images 1 are compared, each match image 1 is similar.
[0044] 3 are points showing the results of calculating the similarity between different match images 1 for each corresponding part. Naturally, if the match images 1 are different, the "△" points showing the similarity will be concentrated in the lower left area of graph 3A, indicating that the match images 1 are not similar to each other.
[0045] As shown in Figure 3, the similarity between match image 1 and duplicate image 2 is about the same as the similarity between the same match image 1. This shows that it is possible to create a duplicate of match image 1 with such precision that it is indistinguishable from match image 1.
[0046] However, the shading pattern appearing in match image 1 composed of powder reflects the shadows of the surface unevenness, and changing the direction of light irradiation changes the pattern in match image 1, even for the same match image 1. Powder has a tendency to aggregate in units of, for example, several tens of μm, so the pattern in match image 1 also tends to change with the width of the aggregate unit.
[0047] Match image 1, which is made of powder mixed with metal particles, reflects the random arrangement of tiny metal particle faces that cause specular reflection, and changing the direction of light irradiation changes the pattern of match image 1, even for the same match image 1. When particles of about several μm to 10 μm are used, the brightness tends to change in units of several tens of μm.
[0048] 4, Fig. 4(A) is a diagram showing an example of the match image 1 when light is irradiated from the upper left (diagonally above left) toward the match area along a direction perpendicular to the left side of the match image 1. Hereinafter, this type of light irradiation form will be referred to as "light is irradiated from the left direction."
[0049] Fig. 4(B) is an example of graph 3B showing the results of calculating the similarity between the match image 1 shown in Fig. 4(A) twice for each corresponding part (shown by "○"), and the results of calculating the similarity between the match image 1 shown in Fig. 4(A) and another match image 1 different from Fig. 4(A) that was photographed by irradiating light from the left direction of the match image 1, and showing the results of calculating the similarity between the match image 1 shown in Fig. 4(A) and another match image 1 that was photographed by irradiating light from the left direction of the match image 1. Note that the vertical and horizontal axes in graph 3B have the same meanings as those in graph 3A shown in Fig. 3.
[0050] In other words, as can be seen from graph 3B in Figure 4(B), if the light irradiation direction is the same, the same match images 1 taken respectively are judged to be similar, and even if the light irradiation direction is the same, different match images 1 are judged to be dissimilar.
[0051] On the other hand, in Fig. 5, Fig. 5(A) is a diagram showing an example of the match image 1 which is the same as the match image 1 shown in Fig. 4(A) and in which light is irradiated from above and below toward the match area along a direction perpendicular to the bottom side of the match image 1. Hereinafter, this type of light irradiation form will be referred to as "light irradiated from below."
[0052] Fig. 5(B) is an example of graph 3C showing the results (indicated by "○") of calculating the similarity between the match image 1 shown in Fig. 5(A) twice and the match image 1 taken by irradiating light from below the match image 1 different from that shown in Fig. 5(A) and the results (indicated by "△") of calculating the similarity between the match image 1 shown in Fig. 5(A) and another match image 1 taken by irradiating light from below the match image 1 different from that shown in Fig. 5(A) for each corresponding part. The vertical and horizontal axes in graph 3C have the same meanings as those in graph 3A shown in Fig. 3.
[0053] Even in this case, as long as the light irradiation direction is the same, the same match images 1 taken respectively are judged to be similar, and it can be seen from graph 3C in Figure 5(B) that even if the light irradiation direction is the same, different match images 1 are judged to be dissimilar.
[0054] In contrast, Figure 6 is an example of a graph 3D showing the similarity between the match image 1 shown in Figure 4(A) and the match image 1 shown in Figure 5(A), i.e., the similarity between the same match images 1 illuminated with light from different directions, calculated for each corresponding location (indicated by "○"), and the similarity between different match images 1 illuminated with light from the same direction, calculated for each corresponding location (indicated by "△").
[0055] It can be seen from FIG. 6 that even if the match image 1 is the same, if the light irradiation direction is different, the match image 1 is determined to be different to the same extent as a different match image 1.
[0056] In this embodiment, we will describe a matching system 5 that uses the property of the matching image 1 that the similarity of the matching image 1 changes depending on the direction of light irradiation, even for the same matching image 1, to determine whether an image to be matched (called a "test image 9") captured by a photographing device is similar to the matching image 1.
[0057] The photographing device for photographing the inspection image 9 may be any type of photographing device as long as it is equipped with a camera 6A and one light source 6B for emitting light (see FIG. 8(A)), but in this embodiment, an example in which the inspection image 9 is photographed using a smartphone 6 will be described. The smartphone 6 is an example of the second photographing device.
[0058] 7 is a configuration diagram showing an example of a matching system 5 according to this embodiment. The matching system 5 includes a smartphone 6 that captures an inspection image 9, and a matching device 10 that matches the inspection image 9 included in the captured image with a matching image 1. The smartphone 6 is connected to a communication line 8, for example, via a wireless base station 7, and is connected to the matching device 10 that is also connected to the communication line 8. The communication line 8 may be a wired line or a wireless line, and may be a dedicated line or a public line.
[0059] For convenience of explanation, only one smartphone 6 is illustrated in FIG. 7, but in reality, a plurality of smartphones 6 may be connected to the verification device 10.
[0060] The matching device 10 includes functional units, that is, a communication unit 11 and a matching unit 12, and a registered image DB (Database) 13.
[0061] The communication unit 11 performs data communication with the smartphone 6 through the communication line 8. Specifically, the communication unit 11 acquires from the smartphone 6 a plurality of captured images of the inspection image 9 captured while changing the irradiation direction of the irradiated light, and transfers each of the acquired captured images to the collation unit 12.
[0062] When the matching unit 12 receives a captured image from the communication unit 11, the matching unit 12 uses a known matching algorithm to calculate, for each received captured image, the similarity between the inspection image 9 included in the captured image and the match image 1 included in a registered image that is registered in advance in the registered image DB 13. Then, the matching unit 12 notifies the communication unit 11 of the matching result between the inspection image 9 included in the captured image and the match image 1 included in the registered image.
[0063] Here, the registered image refers to an image obtained by photographing the match image 1 while changing the direction of light irradiation for each match image 1, whose pattern changes depending on the direction of light irradiation, and is the original image of the match image 1. As can be seen from the fact that the match image 1 is photographed while changing the direction of light irradiation for each match image 1, there are multiple registered images for one match image 1. Hereinafter, the multiple registered images obtained from one match image 1 are referred to as a "group of registered images."
[0064] That is, there is a registered image group for each match image 1, and the registered images included in each registered image group are registered in the registered image DB 13 on a registered image group basis. Also, for each registered image group, information associating the match image 1 with the registered image group is managed in the registered image DB 13 so that it is possible to know which match image 1 was captured for that registered image group.
[0065] If each of the inspection images 9 included in the multiple captured images is similar to a plurality of registered images included in a specific group of registered images, the matching unit 12 matches the inspection image 9 included in each of the captured images with the matching image 1 corresponding to the specific group of registered images as an image captured while changing the direction of light irradiation.
[0066] The communication unit 11, which has received the matching result from the matching unit 12, transmits the received matching result to, for example, the smartphone 6 that has transmitted the captured image. As a result, the smartphone 6 is notified of the matching result indicating whether the captured test image 9 matches the matching image 1 included in any of the registered images registered in the registered image DB 13.
[0067] If the test image 9 captured by the user is a duplicated image 2 obtained by duplicating the collation image 1 using an inkjet printer, the duplicated image 2 will not be a three-dimensional image but will be a two-dimensional image. Therefore, even if the duplicated image 2 is captured while changing the direction of light irradiation, the pattern of the duplicated image 2 contained in the captured image will not change, and multiple captured images of the duplicated image 2 showing the same pattern will be obtained.
[0068] In this case, a registered image showing the same pattern as the duplicated image 2 included in any one of the captured images will exist in the registered image DB 13. However, since the pattern of the duplicated image 2 included in the other captured image is the same as the pattern of the duplicated image 2 included in the previous captured image, a registered image showing the same pattern as each of the duplicated images 2 included in the other captured image will not exist in the registered image group other than the previous registered image. Therefore, since each of the duplicated images 2 included in the multiple captured images is not similar to each of the multiple registered images included in a specific registered image group, the inspection image 9 captured by the user is a duplicate of the match image 1, and the matching device 10 can match the authenticity of the match image 1.
[0069] The registered image DB 13 does not necessarily have to be included in the matching device 10, and may be included in an external device (not shown) connected to the communication line 8. In this case, the matching device 10 refers to the registered image DB 13 included in the external device (not shown) via the communication line 8 as necessary.
[0070] Next, a description will be given of the photographing mode of the registration image used for matching with the match image 1. To this end, first, a photographing mode for photographing the test image 9 to be matched using the smartphone 6 will be described.
[0071] FIG. 8 is a diagram showing an example of a form of capturing an inspection image 9 by a smartphone 6. As shown in FIG. 8(A), a camera 6A and a light source 6B are built into the smartphone 6. A point light source that emits light from one point is used as the light source 6B. In other words, the light source 6B is an example of a built-in point light source. Note that the light source 6B may be a light source in which one or more LEDs (Light Emitting Diodes) are concentrated.
[0072] As shown in Figure 8 (B), the user photographs the test image 9 by facing the surface on which the camera 6A of the smartphone 6 is located (referred to as the "capture surface of the smartphone 6") toward the test image 9 so that the camera 6A of the smartphone 6 faces the test image 9 to be compared.
[0073] The shooting environment attributes that affect the shooting condition of the inspection image 9, such as the size of the inspection image 9 included in the captured image, the illuminance distribution on the inspection image 9 illuminated by light from the light source 6B, and the angle of light irradiation by the light source 6B, vary depending on the angle between the optical axis of the camera 6A and the optical axis of the light source 6B (referred to as the "illumination angle φ") and the distance between the shooting surface of the smartphone 6 and the inspection image 9 (referred to as the "shooting distance H").
[0074] In order to perform the matching between the inspection image 9 contained in the captured image taken in this manner and the comparison image 1 as accurately as possible, it is preferable to capture the comparison image 1 using dedicated equipment 15 with an illumination angle φ and shooting distance H close to the illumination angle φ and shooting distance H when the user captures the inspection image 9 using a smartphone 6, as shown in Figure 9.
[0075] For this reason, the dedicated equipment 15 is configured to be able to change the distance (referred to as the "device distance W") between the center of the lens of the camera 15A in the dedicated equipment 15 and the center of the illumination surface of the light source 15B. The photographer of the registered image brings the shooting distance H of the dedicated equipment 15 closer to the shooting distance H when shooting with the smartphone 6, and then adjusts the device distance W to bring the illumination angle φ of the dedicated equipment 15 closer to the illumination angle φ when shooting with the smartphone 6.
[0076] Although it depends on the model of the smartphone 6, when the shooting distance H is 100 mm, the smartphone 6 often captures an image with the highest resolution defined by the smartphone 6. If the shooting distance H is longer than 100 mm, the size of the inspection image 9 included in the captured image becomes small, making it difficult to match it with the registered image, while if the shooting distance H is shorter than 100 mm, the camera 6A does not focus on the inspection image 9, and the resolution of the inspection image 9 tends to decrease. Therefore, it is preferable to set the shooting distance H in the dedicated equipment 15 within an adjustment range of 100 mm ± α mm (α is a real number). α is a predetermined length indicating a fluctuation range within which an image with a resolution that can be considered as the highest resolution can be obtained in the smartphone 6.
[0077] On the other hand, even if the model is different, the device distance W in the smartphone 6 is often within a range of about 10 mm to 15 mm due to the relationship between the size of each of the camera 15A and the light source 15B and the size of the smartphone 6 itself. Therefore, the device distance W of the dedicated equipment 15 is also set to about 10 mm to 15 mm accordingly.
[0078] Note that a single point light source is often used as the light source 6B of the smartphone 6. Therefore, in order to make the illuminance distribution by the light source 15B of the dedicated equipment 15 closer to the illuminance distribution by the light source 6B of the smartphone 6, it is preferable that the light source 15B of the dedicated equipment 15 is also a single point light source.
[0079] Next, the direction of light emitted from light source 15B when a plurality of registered images are captured by dedicated equipment 15 will be considered.
[0080] As already explained, by changing the direction of light irradiation, the pattern of the match image 1 changes, and even the same match image 1 is determined to be a different match image 1.
[0081] Therefore, in order to match the test image 9 contained in the captured image taken by the smartphone 6 with the match image 1 contained in the registered image, it is necessary to capture the match image 1 with the dedicated equipment 15 by irradiating light from the light source 15B of the dedicated equipment 15 in the same direction as the light used when capturing the test image 9 with the smartphone 6.
[0082] In particular, when capturing the test image 9 using a smartphone 6 in which the camera 6A and the light source 6B are arranged so that a straight line connecting the centers of the camera 6A and the light source 6B is perpendicular to one of the contour lines of the smartphone 6, it is sufficient to prepare in advance a number of registered images in which the comparison image 1 is captured by shining light from at least two different directions.
[0083] Note that the arrangement of the camera 6A and the light source 6B such that a straight line connecting the centers of the camera 6A and the light source 6B is perpendicular to any one of the contour lines of the smartphone 6 is referred to as an "orthogonal arrangement."
[0084] When a smartphone 6 in which the camera 6A and light source 6B are arranged orthogonally is used, if a registration image taken of the matching image 1 with left illumination and a registration image taken of the matching image 1 with bottom illumination are prepared, it becomes possible to match the test image 9 contained in the captured image with the matching image 1 contained in the registration image regardless of the relative positions of the camera 6A and the light source 6B.
[0085] In addition, for any image such as the match image 1 or the test image 9, placing the light source 6B on the left side of the camera 6A and illuminating the image from left to right is called "left illumination," placing the light source 6B on the right side of the camera 6A and illuminating the image from right to left is called "right illumination," placing the light source 6B on the upper side of the camera 6A and illuminating the image from top to bottom is called "upper illumination," and placing the light source 6B on the lower side of the camera 6A and illuminating the image from bottom to top is called "lower illumination."
[0086] FIG. 10 is a diagram showing an example of the light irradiation direction in the first smartphone 6 in which the camera 6A and the light source 6B are arranged perpendicular to each other.
[0087] Fig. 10(A) is a diagram of the smartphone 6 viewed from the side opposite to the imaging side of the smartphone 6, that is, the operation side on which the screen is arranged. In the smartphone 6 of Fig. 10(A), the camera 6A and the light source 6B are arranged along the short side of the outline of the smartphone 6, and in the upper half of the smartphone 6, the camera 6A is arranged on the right side as viewed from the operation side of the smartphone 6, and the light source 6B is arranged on the left side. Note that the camera 6A and the light source 6B are not directly visible when viewed from the operation side of the smartphone 6, and are therefore illustrated with dotted lines.
[0088] FIG. 10B is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 10A when capturing an examination image 9 with left illumination.
[0089] FIG. 10C is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 10A when capturing an examination image 9 of bottom illumination.
[0090] FIG. 11 is a diagram showing an example of the light irradiation direction in the second smartphone 6 in which the camera 6A and the light source 6B are disposed perpendicular to each other.
[0091] As shown in FIG. 11(A), the smartphone 6 in FIG. 11 has a camera 6A and a light source 6B arranged along the short side of the contour line of the smartphone 6, and in the upper half of the smartphone 6, the camera 6A is arranged on the left side and the light source 6B is arranged on the right side when viewed from the operation surface of the smartphone 6.
[0092] FIG. 11(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 11(A) when capturing an examination image 9 with left illumination.
[0093] FIG. 11C is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 11A when capturing an examination image 9 of bottom illumination.
[0094] FIG. 12 is a diagram showing an example of the light irradiation direction of a third smartphone 6 in which a camera 6A and a light source 6B are arranged perpendicular to each other.
[0095] As shown in FIG. 12(A), the smartphone 6 in FIG. 12 has a camera 6A and a light source 6B arranged along the long side of the contour line of the smartphone 6, and in the upper half of the smartphone 6, the camera 6A is arranged on the upper side and the light source 6B is arranged on the lower side when viewed from the operation surface of the smartphone 6.
[0096] FIG. 12B is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 12A when capturing an examination image 9 with left illumination.
[0097] FIG. 12C is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 12A when capturing an examination image 9 of bottom illumination with the smartphone 6. In FIG.
[0098] FIG. 13 is a diagram showing an example of the light irradiation direction of a fourth smartphone 6 in which a camera 6A and a light source 6B are arranged perpendicular to each other.
[0099] As shown in FIG. 13(A), the smartphone 6 in FIG. 13 has a camera 6A and a light source 6B arranged along the long side of the contour line of the smartphone 6, and in the upper half of the smartphone 6, the camera 6A is arranged on the lower side and the light source 6B is arranged on the upper side when viewed from the operation surface of the smartphone 6.
[0100] FIG. 13(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 13(A) when capturing an examination image 9 with left illumination.
[0101] FIG. 13(C) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 13(A) when capturing an examination image 9 of bottom illumination with the smartphone 6. In FIG.
[0102] If the smartphone 6 is positioned relative to the test image 9 as shown in Figures 10 to 13, the smartphone 6, in which the camera 6A and light source 6B are arranged orthogonally, can obtain a captured image of the test image 9 with left illumination and a captured image of the test image 9 with bottom illumination.
[0103] In addition, the smartphone 6 may display a guide to assist the user in photographing the test image 9 so that the shooting conditions when the user photographs the test image 9 with the smartphone 6 are closer to the shooting conditions when the comparison image 1 is photographed with the dedicated equipment 15.
[0104] The photographing conditions are conditions that stipulate the positional relationship between the camera 6A of the smartphone 6, the light source 6B, and the inspection image 9. Specifically, the smartphone 6 may display on the screen of the smartphone 6 information informing that the illumination angle φ of the light source 6B and the shooting distance H of the camera 6A have approached the illumination angle φ of the light source 15B and the shooting distance H of the camera 15A when the comparison image 1 was photographed by the dedicated equipment 15.
[0105] On the other hand, the arrangement of the camera 6A and the light source 6B in the smartphone 6 is not limited to an orthogonal arrangement. For example, there are smartphones 6 with an "inclined arrangement" in which the camera 6A and the light source 6B are arranged so that the angle formed by the line connecting the centers of the camera 6A and the light source 6B and any of the contour lines of the smartphone 6 intersects at an angle other than 90 degrees.
[0106] FIG. 14 is a diagram showing an example of the light irradiation direction of a fifth smartphone 6 in which a camera 6A and a light source 6B are arranged at an angle.
[0107] As shown in FIG. 14(A), in the smartphone 6 of FIG. 14, a light source 6B is disposed diagonally downward and to the left from the position of a camera 6A disposed in the upper half of the smartphone 6 when viewed from the operation surface of the smartphone 6.
[0108] FIG. 14(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 14(A) when capturing an examination image 9 with left illumination.
[0109] FIG. 14(C) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 14(A) when capturing an examination image 9 of bottom illumination.
[0110] FIG. 15 is a diagram showing an example of the light irradiation direction of a sixth smartphone 6 in which a camera 6A and a light source 6B are arranged at an angle.
[0111] As shown in FIG. 15(A), in the smartphone 6 of FIG. 15, a light source 6B is disposed diagonally downward to the right from the position of a camera 6A disposed in the upper half of the smartphone 6 when viewed from the operation surface of the smartphone 6.
[0112] FIG. 15(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 15(A) when capturing an examination image 9 with left illumination.
[0113] FIG. 15(C) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 15(A) when capturing an examination image 9 of bottom illumination.
[0114] FIG. 16 is a diagram showing an example of the light irradiation direction of a seventh smartphone 6 in which a camera 6A and a light source 6B are arranged at an angle.
[0115] As shown in FIG. 16(A), in the smartphone 6 of FIG. 16, a light source 6B is disposed diagonally upward to the right from the position of a camera 6A disposed in the upper half of the smartphone 6 when viewed from the operation surface of the smartphone 6.
[0116] FIG. 16(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 16(A) when capturing an examination image 9 with left illumination.
[0117] FIG. 16(C) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 16(A) when capturing an examination image 9 of bottom illumination.
[0118] FIG. 17 is a diagram showing an example of the light irradiation direction of an eighth smartphone 6 in which a camera 6A and a light source 6B are arranged at an angle.
[0119] As shown in FIG. 17(A), in the smartphone 6 of FIG. 17, a light source 6B is disposed diagonally upward and to the left from the position of a camera 6A disposed in the upper half of the smartphone 6 when viewed from the operation surface of the smartphone 6.
[0120] FIG. 17(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 17(A) when capturing an examination image 9 with left illumination.
[0121] FIG. 17(C) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 17(A) when capturing an examination image 9 of bottom illumination.
[0122] If the smartphone 6 is positioned relative to the test image 9 as shown in Figures 14 to 17, a captured image of the test image 9 with left illumination and a captured image of the test image 9 with downward illumination can be obtained even if the smartphone 6 has a tilted arrangement of the camera 6A and light source 6B.
[0123] Here, the arrangement of the smartphone 6 that captures the test image 9 according to the irradiation direction of the light has been described based on an example in which a registered image obtained by photographing the match image 1 with left irradiation and a registered image obtained by photographing the match image 1 with downward irradiation are prepared as a registered image group of the match image 1. However, the registered images included in the registered image group of the match image 1 are not limited to a registered image obtained by photographing the match image 1 with left irradiation and a registered image obtained by photographing the match image 1 with downward irradiation. For example, a registered image obtained by photographing the match image 1 with right irradiation and a registered image obtained by photographing the match image 1 with upward irradiation may be used.
[0124] Then, the user changes the position of the smartphone 6 so that the light irradiation directions match those used when the comparison image 1 was captured by the dedicated equipment 15, and captures the test image 9 in each irradiation direction with the smartphone 6.
[0125] In other words, if there are two registered images with different light irradiation directions as the registered image group for one match image 1, the matching device 10 can match the test image 9 with the match image 1 based on the similarity between each match image 1 included in the two registered images and each test image 9 included in the two captured images.
[0126] However, among the positions of the smartphone 6 shown in Figures 10 to 17, there are some positions that are difficult to realize due to the range of motion of the joints of the body when the user holds the smartphone 6 in his / her hand and tries to take a picture in those positions, and some positions require the user to switch the smartphone 6 from one hand to the other. For example, positions such as those shown in Figures 11(B), 13(C), 15(B), 16(B), 16(C), and 17(C) that require the smartphone 6 to be held upside down are inferior in operability of the smartphone 6 when taking the test image 9 compared to positions that do not.
[0127] In order to deal with such a situation, four registered images with different light irradiation directions may be captured as a registered image group for one match image 1, and registered in the registered image DB 13. Since the more different the light irradiation directions are, the greater the change in the pattern of the match image 1, in this case, it is preferable to capture the registered image by shifting the irradiation direction of the light irradiating the match image 1 by 90 degrees along the periphery of the match image 1, such as a registered image captured of the match image 1 with upper irradiation, a registered image captured of the match image 1 with right irradiation, a registered image captured of the match image 1 with lower irradiation, and a registered image captured of the match image 1 with left irradiation.
[0128] "Taking a photograph while shifting the direction of light irradiation along the periphery of match image 1" means, for example, when match image 1 is viewed from the front, moving light source 15B of dedicated equipment 15 along the contour line of match image 1 while irradiating light in the direction of match image 1, and taking a photograph with dedicated equipment 15. Note that, since halation is likely to occur if the illuminance on match image 1 is too high, match image 1 may be taken in a positional relationship where the optical axis of light source 15B does not intersect with match image 1.
[0129] In the case of the smartphone 6 shown in Figure 11 (A), it is preferable to include in the registered image group a registered image taken of a matching image 1 with upper illumination and a registered image taken of a matching image 1 with right illumination so that the user does not have to capture an examination image 9 with left illumination as shown in Figure 11 (B).
[0130] FIG. 18 is a diagram showing an example of the arrangement of the smartphone 6 shown in FIG. 11(A) when the smartphone 6 captures the inspection image 9. In FIG.
[0131] Of these, FIG. 18(A) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 11(A) when capturing an examination image 9 of upper illumination with the smartphone 6. In FIG.
[0132] FIG. 18(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 11(A) when capturing the test image 9 with right-side illumination.
[0133] In the case of the smartphone 6 shown in Figure 13(A), it is preferable to include in the registered image group a registered image taken of a matching image 1 with an upper illumination and a registered image taken of a matching image 1 with a right illumination so that the user does not have to capture an examination image 9 with a lower illumination as shown in Figure 13(C).
[0134] FIG. 19 is a diagram showing an example of the arrangement of the smartphone 6 shown in FIG. 13(A) when the smartphone 6 captures the inspection image 9. In FIG.
[0135] Of these, FIG. 19(A) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 13(A) when capturing an examination image 9 of upper illumination with the smartphone 6. FIG.
[0136] FIG. 19(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 13(A) when capturing the test image 9 with right-side illumination.
[0137] In the case of the smartphone 6 shown in Figure 15(A), it is preferable to include in the registered image group a registered image taken of a matching image 1 with upper illumination and a registered image taken of a matching image 1 with right illumination so that the user does not have to capture an examination image 9 with left illumination as shown in Figure 15(B).
[0138] FIG. 20 is a diagram showing an example of the arrangement of the smartphone 6 shown in FIG. 15(A) when the smartphone 6 captures the inspection image 9. In FIG.
[0139] Of these, FIG. 20(A) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 15(A) when capturing an examination image 9 of upper illumination with the smartphone 6. FIG.
[0140] FIG. 20(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 15(A) when capturing the test image 9 with right-side illumination.
[0141] In the case of the smartphone 6 shown in Figure 16(A), it is preferable to include in the registered image group a registered image taken of a matching image 1 with upper illumination and a registered image taken of a matching image 1 with right illumination so that the user does not have to capture test images 9 with left illumination and bottom illumination as shown in Figures 16(B) and 16(C).
[0142] FIG. 21 is a diagram showing an example of the arrangement of the smartphone 6 when the inspection image 9 is captured by the smartphone 6 shown in FIG. 16(A).
[0143] Of these, FIG. 21(A) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 16(A) when capturing an examination image 9 of upper illumination with the smartphone 6. FIG.
[0144] FIG. 21(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 16(A) when capturing the test image 9 with right-side illumination.
[0145] In the case of the smartphone 6 shown in Figure 17(A), it is preferable to include in the registered image group a registered image taken of a matching image 1 with an upper illumination and a registered image taken of a matching image 1 with a right illumination so that the user does not have to capture an examination image 9 with a lower illumination as shown in Figure 17(C).
[0146] FIG. 22 is a diagram showing an example of the arrangement of the smartphone 6 when the inspection image 9 is captured by the smartphone 6 shown in FIG. 17(A).
[0147] Of these, FIG. 22(A) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 17(A) when capturing an examination image 9 of upper illumination with the smartphone 6. FIG.
[0148] FIG. 22(B) is a diagram showing an example of the arrangement of the smartphone 6 in FIG. 17(A) when capturing the test image 9 with right-side illumination.
[0149] Also, for example, in the case of the smartphone 6 shown in Fig. 11(A), it is easier for the user to capture the test image 9 when the smartphone 6 is positioned as shown in Fig. 11(C), Fig. 18(A), and Fig. 18(B) than when the smartphone 6 is positioned as shown in Fig. 11(B). Therefore, when the user uses the smartphone 6 shown in Fig. 11(A), the user is more likely to capture the test image 9 in any two of the light irradiation directions of upper illumination, right illumination, and lower illumination.
[0150] In this case, if a registered image group for each match image 1 is registered in the registered image DB 13 as a registered image group for the match image 1 with upward illumination, a registered image for the match image 1 with right illumination, a registered image for the match image 1 with downward illumination, and a registered image for the match image 1 with left illumination, then even if two captured images with different light illumination directions, either up, down, left, or right, are sent to the matching device 10, the matching device 10 can determine whether the test image 9 is a duplicate image 2 of the match image 1 and then match the test image 9 with the match image 1.
[0151] In other words, in order to enable matching of the test image 9 contained in the captured image taken by the smartphone 6 with the match image 1 regardless of how the camera 6A and light source 6B are positioned on the smartphone 6, it is preferable to register a group of registered images in the registered image DB13 of the matching device 10 for each match image 1, the registered image group including at least four registered images with different light illumination directions, such as upward illumination, right illumination, downward illumination, and left illumination.
[0152] On the other hand, when capturing the test images 9 for upper, right, lower, and left illumination using a smartphone 6 in which the camera 6A and light source 6B are arranged orthogonally, the user only needs to position the smartphone 6 horizontally or vertically to capture the images, as shown in Figures 10 to 13, 18, and 19. However, when capturing the test images 9 for upper, right, lower, and left illumination using a smartphone 6 in which the camera 6A and light source 6B are arranged at an angle, the user will have to position the smartphone 6 at an angle to the test images 9 to capture the images, as shown in Figures 14 to 17, and 20 to 22.
[0153] A user tends to find it easier to hold and operate the smartphone 6 when the smartphone 6 is positioned horizontally or vertically than when the smartphone 6 is positioned diagonally.
[0154] Therefore, assuming that a user will capture an inspection image 9 by placing a smartphone 6 in a horizontal or vertical orientation with the camera 6A and light source 6B in an inclined position, it is preferable to also register registered images captured in the same illumination direction as the light illumination direction of the light source 6B when the smartphone 6 is positioned horizontally or vertically as a registered image group in the registered image DB13 for each match image 1.
[0155] Note that arranging the smartphone 6 in the horizontal direction means arranging the smartphone 6 so that the long side of the contour of the smartphone 6 appears along the left-right direction in the user's field of view, and arranging the smartphone 6 in the vertical direction means arranging the smartphone 6 so that the long side of the contour of the smartphone 6 appears along the up-down direction in the user's field of view. Also, arranging the smartphone 6 in the diagonal direction means arranging the smartphone 6 so that the angle between the long side of the contour of the smartphone 6 and a horizontal line that is virtually set along the left-right direction in the user's field of view is an angle other than 90 degrees.
[0156] FIG. 23 is a diagram showing an example of the smartphone 6 in the inclined arrangement shown in FIG. 14 being arranged horizontally or vertically.
[0157] 23(A) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged vertically with the camera 6A on the upper side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from a direction shifted by 45 degrees counterclockwise along the periphery of the inspection image 9 with respect to the light irradiation direction that results in left irradiation (referred to as "left-downward diagonal irradiation").
[0158] 23(B) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the left side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from a direction shifted by 45 degrees counterclockwise along the periphery of the inspection image 9 with respect to the light irradiation direction for downward irradiation (referred to as a "right-downward diagonal irradiation").
[0159] 23(C) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the right side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from a direction shifted by 45 degrees counterclockwise along the periphery of the inspection image 9 with respect to the light irradiation direction of upward irradiation (referred to as "upper left diagonal irradiation").
[0160] In addition, when the smartphone 6 shown in FIG. 23 is arranged vertically with the camera 6A on the lower side, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from a direction (referred to as "upper right diagonal direction") shifted 45 degrees counterclockwise along the periphery of the inspection image 9 with respect to the light irradiation direction of right irradiation (referred to as "upper right diagonal irradiation"). However, in this case, since the smartphone 6 needs to be held upside down, the operability of the smartphone 6 is inferior to the operability of the smartphone 6 in each arrangement shown in FIG. 23. Therefore, in the case of the smartphone 6 arranged in an inclined manner as shown in FIG. 23, in addition to the four registered images of the collation image 1 taken with the upper illumination, right illumination, lower illumination, and left illumination, for example, registered images taken of the collation image 1 taken with the lower left diagonal illumination, the lower right diagonal illumination, and the upper left diagonal illumination may be registered in the registered image DB 13 of the collation device 10 as a registered image group.
[0161] FIG. 24 is a diagram showing an example of the smartphone 6 in the inclined arrangement shown in FIG. 15 being arranged horizontally or vertically.
[0162] 24(A) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged vertically with the camera 6A on the upper side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 from a diagonally lower right direction.
[0163] 24(B) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the left side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from an upper right diagonal direction.
[0164] 24(C) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the right side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 from a diagonally downward left direction.
[0165] In addition, when the smartphone 6 shown in FIG. 24 is arranged vertically with the camera 6A on the lower side, the light source 6B of the smartphone 6 irradiates the inspection image 9 from a diagonally upward left direction. However, in this case, the smartphone 6 needs to be held upside down, and the operability of the smartphone 6 is inferior to that of the smartphone 6 in each arrangement shown in FIG. 24. Therefore, in the case of the smartphone 6 arranged in an inclined manner as shown in FIG. 24, in addition to the four registered images of the collation image 1 taken with the upper illumination, the right illumination, the lower illumination, and the left illumination, for example, registered images of the collation image 1 taken with the lower right illumination, the upper right illumination, and the lower left illumination may be registered as a registered image group in the registered image DB 13 of the collation device 10.
[0166] FIG. 25 is a diagram showing an example of the smartphone 6 in the inclined arrangement shown in FIG. 16 being arranged horizontally or vertically.
[0167] 25(A) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged vertically with the camera 6A on the upper side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from an upper right diagonal direction.
[0168] 25(B) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the right side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 from a diagonally lower right direction.
[0169] 25(C) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the left side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from the diagonally upper left direction.
[0170] In addition, when the smartphone 6 shown in FIG. 25 is arranged vertically with the camera 6A on the lower side, the light source 6B of the smartphone 6 irradiates the inspection image 9 from a diagonally downward left direction. However, in this case, the smartphone 6 needs to be held upside down, and the operability of the smartphone 6 is inferior to that of the smartphone 6 in each arrangement shown in FIG. 25. Therefore, in the case of the smartphone 6 arranged in an inclined manner as shown in FIG. 25, in addition to the four registered images of the collation image 1 taken with the upper illumination, the right illumination, the lower illumination, and the left illumination, for example, registered images of the collation image 1 taken with the upper right illumination, the lower right illumination, and the upper left illumination may be registered as a registered image group in the registered image DB 13 of the collation device 10.
[0171] FIG. 26 is a diagram showing an example of the smartphone 6 in the inclined arrangement shown in FIG. 17 being arranged horizontally or vertically.
[0172] 26(A) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged vertically with the camera 6A on the upper side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from the diagonally upper left direction.
[0173] 26(B) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the left side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 from a diagonally downward left direction.
[0174] 26(C) is a diagram showing an example of an arrangement in which the smartphone 6 is arranged horizontally with the camera 6A on the right side. In this case, the light source 6B of the smartphone 6 irradiates the inspection image 9 with light from an upper right diagonal direction.
[0175] In addition, when the smartphone 6 shown in FIG. 26 is arranged vertically with the camera 6A on the lower side, the light source 6B of the smartphone 6 irradiates the inspection image 9 from a diagonally downward right direction. However, in this case, the smartphone 6 needs to be held upside down, and the operability of the smartphone 6 is inferior to that of the smartphone 6 in each arrangement shown in FIG. 26. Therefore, in the case of the smartphone 6 arranged in an inclined manner as shown in FIG. 26, in addition to the four registered images of the collation image 1 taken with the upper illumination, the right illumination, the lower illumination, and the left illumination, for example, registered images of the collation image 1 taken with the upper left illumination, the lower left illumination, and the upper right illumination may be registered as a registered image group in the registered image DB 13 of the collation device 10.
[0176] Naturally, taking into consideration that there may be users who hold each of the smartphones 6 shown in Figures 23 to 26 upside down to capture test images 9, the registered image DB13 of the matching device 10 may register a total of eight registered images as a registered image group for each matching image 1, including four registered images taken of matching images 1 with upward illumination, right illumination, downward illumination, and left illumination, and four registered images taken of matching images 1 with upward left illumination, downward left illumination, upward right illumination, and downward right illumination.
[0177] Here, as an example, for each light irradiation direction when photographing the match image 1 with upper illumination, right illumination, lower illumination, and left illumination, images photographed by illuminating the match image 1 with light from a direction shifted by 45 degrees counterclockwise along the periphery of the match image 1 are used as registered images, but registered images may also be photographed by illuminating the match image 1 with light from a direction shifted by less than 90 degrees counterclockwise along the periphery of the match image 1, for example, by 30 degrees or 70 degrees. There is also no limit to the number of registered images registered as a registered image group for each match image 1.
[0178] An example of an additional image is a registered image captured by illuminating the match image 1 with light from a direction shifted by less than 90 degrees counterclockwise along the periphery of the match image 1 with respect to the light illumination directions of upward illumination, rightward illumination, downward illumination, and leftward illumination. Therefore, the four registered images captured of the match image 1 with left obliquely upward illumination, left obliquely downward illumination, right obliquely upward illumination, and right obliquely downward illumination are also examples of additional images.
[0179] In addition, it is not necessary for the registered image group to include registered images of matching images 1 taken with upward illumination, right illumination, downward illumination, and left illumination; it may be sufficient to include only registered images of matching images 1 taken with upward left diagonal illumination, downward left diagonal illumination, upward right diagonal illumination, and downward right diagonal illumination.
[0180] Such a collation device 10 can be configured using a computer 20. FIG. 27 is a diagram showing an example of the configuration of a main part of an electrical system in the computer 20.
[0181] The computer 20 includes a CPU (Central Processing Unit) 21, which is an example of a processor that performs each functional unit of the collation device 10 shown in Fig. 7, a ROM (Read Only Memory) 22 that stores a collation program that causes the computer 20 to function as the collation device 10, a RAM (Random Access Memory) 23 used as a temporary work area for the CPU 21, a non-volatile memory 24, and an input / output interface (I / O) 25. The CPU 21, the ROM 22, the RAM 23, the non-volatile memory 24, and the I / O 25 are connected to each other via a bus 26.
[0182] The non-volatile memory 24 is an example of a storage device in which stored information is maintained even if power supplied to the non-volatile memory 24 is cut off. Therefore, the registered image DB 13 is constructed in the non-volatile memory 24, for example.
[0183] Although a semiconductor memory is used as the nonvolatile memory 24, a hard disk may also be used. Furthermore, the nonvolatile memory 24 does not necessarily need to be built into the computer 20, and may be, for example, a portable storage device that is detachable from the computer 20.
[0184] To the I / O 25, for example, a communication unit 27, an input unit 28, and a display unit 29 are connected.
[0185] The communication unit 27 is connected to the communication line 8 and has a communication protocol for performing data communication with the smartphone 6 .
[0186] The input unit 28 is a device that receives instructions from the operator of the collation device 10 and notifies the CPU 21, and may be implemented by, for example, a button, a touch panel, a keyboard, or a mouse.
[0187] The display unit 29 is an example of a device that visually displays information processed by the CPU 21, and may be, for example, a liquid crystal display or an organic EL (Electro Luminescence) display.
[0188] Next, the process of matching the inspection image 9 and the match image 1 in the matching device 10 will be described.
[0189] FIG. 28 is a flowchart showing an example of a matching process executed by the CPU 21 of the matching device 10 when a plurality of captured images of the same inspection image 9 captured by changing the light irradiation direction are received from the user's smartphone 6.
[0190] A collation program that defines the collation process is stored in advance, for example, in the ROM 22 of the collation device 10. The CPU 21 of the collation device 10 reads the collation program stored in the ROM 22 and executes the collation process.
[0191] Here, as an example, a case where two captured images are received from the smartphone 6 will be described.
[0192] First, in step S10, the CPU 21 selects one of the captured images, calculates the similarity between the inspection image 9 contained in the selected captured image and the match image 1 contained in the registered images registered in the registered image DB 13 using a known matching algorithm, and determines whether there is a registered image that includes a match image 1 similar to the inspection image 9.
[0193] In addition, when the test image 9 and the comparison image 1 are similar, it means that, for example, the number of comparison areas with matching characteristics among a plurality of comparison areas preset in each image for comparing the characteristics of the two images is equal to or greater than a reference value.
[0194] For ease of explanation, hereinafter, a registered image including a match image 1 similar to an inspection image 9 will be referred to as a "similar registered image."
[0195] If there is no similar registered image, it means that the test image 9 is not similar to the match image 1 included in any registered image registered in the registered image DB 13. In other words, the test image 9 is not the match image 1. In this case, the process proceeds to step S50.
[0196] In step S50, the CPU 21 sets the matching result to "matching failure", transmits the matching result to the smartphone 6 via the communication unit 27, and ends the matching process shown in FIG.
[0197] On the other hand, if it is determined in the determination process of step S10 that a similar registered image similar to the inspection image 9 exists, the process proceeds to step S20.
[0198] In this case, the inspection image 9 may be the match image 1 included in the similar registered image, but it is also possible that the inspection image 9 is a duplicate image 2 of the match image 1 included in the similar registered image. If the inspection image 9 included in one of the captured images selected in step S10 (referred to as "one inspection image 9") is the genuine match image 1, then a similar registered image similar to the inspection image 9 included in the other captured image (referred to as "the other inspection image 9") should exist in the registered image group that includes similar registered images similar to the one inspection image 9.
[0199] Therefore, in step S20, the CPU 21 acquires a registered image group including a similar registered image similar to one of the inspection images 9 from the registered image DB 13.
[0200] In step S30, the CPU 21 determines whether or not there is a registered image, different from the similar registered image similar to one inspection image 9, in the group of registered images acquired in step S20, that is similar to the other inspection image 9. If there is a similar registered image similar to the other inspection image 9, the process proceeds to step S40.
[0201] In this case, the multiple test images 9 taken with different light directions will be similar to the patterns of the same match image 1 taken with different light directions, so the test image 9 included in the multiple taken images is an image of the genuine match image 1.
[0202] Therefore, in step S40, the CPU 21 sets the matching result to "matching successful", transmits the matching result to the smartphone 6 via the communication unit 27, and ends the matching process shown in FIG.
[0203] On the other hand, if it is determined in the determination process of step S30 that there is no similar registered image similar to the other inspection image 9, the process proceeds to step S50.
[0204] In this case, since the test image 9 does not show the same pattern as the match image 1 according to the irradiation direction of the light, the test image 9 taken by the smartphone 6 is considered to be a duplicate image 2 of the match image 1. Therefore, the CPU 21 sets the match result to "matching failed" and transmits the match result to the smartphone 6 via the communication unit 27, and ends the match processing shown in FIG.
[0205] Here, an example has been described in which the matching device 10 acquires two captured images of the same test image 9 from the smartphone 6 with different light irradiation directions, but the number of captured images used by the matching device 10 to match the match image 1 is not limited to two and may be three or more. In this case, it is possible that a registered image captured in the same irradiation direction as the irradiation direction of light irradiated when the user captured the test image 9 with the smartphone 6 is not necessarily included in the registered image group. Therefore, if there is a specific registered image group including similar registered images similar to the test images 9 included in at least two or more captured images, the matching device 10 may determine that the test image 9 captured by the smartphone 6 is a genuine match image 1 corresponding to the specific registered image group.
[0206] Under the condition that for each match image 1, there exists a group of registered images captured by shining light from all directions around the match image 1, the matching device 10 may determine that the test image 9 captured by the smartphone 6 is a genuine match image 1 corresponding to a specific group of registered images if the specific group of registered images contains similar registered images that are respectively similar to the test images 9 contained in all the captured images.
[0207] Up to this point, the comparison image 1 has been described based on the configuration of the image surface formed from black powder, but as mentioned above, the comparison image 1 may be formed from silver powder, or may be formed by mixing black and silver powders. Black and silver powders tend to have a greater contrast between the powder color and the reflected light compared to powders of other colors. The greater the contrast between the powder color and the reflected light, the more clearly the differences in the patterns appearing in the comparison image 1 depending on the direction of light irradiation can be recognized, so it is preferable that the comparison image 1 is formed using at least one of black and silver powders.
[0208] Note that black refers to the color with RGB values of (0, 0, 0) (called the "reference color" of black), and a color whose deviation from the reference color of black is within the tolerance range for all R, G, and B values. The tolerance range refers to the maximum deviation from the reference color that is considered to be the same color as the reference color even if the RGB values deviate from the reference color. Therefore, silver refers to the color with RGB values of (192, 192, 192), which is the reference color of silver, and a color whose deviation from the reference color of silver is within the tolerance range for all R, G, and B values.
[0209] Although it has been described above that the match image 1 may be silver powder, the match image 1 may also be formed by attaching a material composed of a mixture of silver powder to the surface of an object. Depending on the mixture ratio of the silver powder, the color of the match image 1 may not necessarily be silver defined by RGB values, but differences in the patterns that appear in the match image 1 depending on the direction of light irradiation can be recognized.
[0210] Although an example has been described in which the inspection image 9 captured by the smartphone 6 is compared with the comparison image 1 using the comparison device 10, the comparison program executed by the comparison device 10 may be executed by the smartphone 6, and the inspection image 9 may be compared with the comparison image 1 using the smartphone 6.
[0211] Although one aspect of the matching device 10 has been described above using the embodiment, the disclosed form of the matching device 10 is merely an example, and the form of the matching device 10 is not limited to the scope described in the embodiment. Various modifications or improvements can be made to the embodiment without departing from the gist of the present disclosure, and the form in which the modifications or improvements are made is also included in the technical scope of the disclosure. For example, the order of the matching process shown in FIG. 28 may be changed without departing from the gist of the present disclosure.
[0212] In the above embodiment, the verification process is implemented by software as an example. However, the process equivalent to the flowchart shown in Fig. 28 may be implemented by hardware. In this case, the process can be performed at a higher speed than when the verification process is implemented by software.
[0213] In the above embodiment, the term "processor" refers to a processor in a broad sense, including a general-purpose processor (e.g., CPU 21) and a dedicated processor (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).
[0214] In addition, the operations of the processor in the above embodiment may be performed not only by one processor but also by multiple processors located at physically separate locations working together. The order of the operations of the processor is not limited to the order described in the above embodiment and may be changed as appropriate.
[0215] In the above embodiment, an example in which the verification program is stored in the ROM 22 has been described, but the destination of the verification program is not limited to the ROM 22. The verification program of the present disclosure can also be provided in a form recorded in a storage medium readable by the computer 20. For example, the verification program may be provided in a form recorded in an optical disk such as a CD-ROM (Compact Disk Read Only Memory) or a DVD-ROM (Digital Versatile Disk Read Only Memory). The verification program may also be provided in a form recorded in a portable semiconductor memory such as a USB (Universal Serial Bus) memory or a memory card. The ROM 22, the non-volatile memory 24, the CD-ROM, the DVD-ROM, the USB, and the memory card are examples of non-transitory storage media.
[0216] Furthermore, the collation device 10 may download a collation program from an external device (not shown) connected to the communication line 8 through the communication unit 27, and store the downloaded collation program in the storage device. In this case, the CPU 21 of the collation device 10 reads the collation program downloaded from the external device (not shown) from the storage device and executes the collation process. [Explanation of symbols]
[0217] 1 Matching image 2. Duplicate images Graphs 3A-3D 4 Recording media 5. Matching System 6. Smartphones 6A (Smartphone) Camera 6B (Smartphone) Light Source 7 Radio base station 8. Communication Lines 9. Inspection images 10. Collation device 11 Communications Department 12. Collation Unit 13 Registered Image DB 15 Specialized Equipment 15A (Specialized Equipment) Camera 15B (dedicated equipment) light source 20. Computers 21 CPU 22 ROM 23 RAM 24 Non-volatile memory 25 I / O 26 Bus 27 Communication Unit 28 Input Units 29 Display unit φ Irradiation angle H Shooting Distance W Device Distance
Claims
1. A processor is provided. The processor, A plurality of photographed images are obtained by photographing the test image to be compared with the second photographing device while changing the direction of illumination of light; The positional relationship between the camera, the light source, and the image is adjusted so that it is the same as the positional relationship between the camera capturing the inspection image, the light source irradiating the inspection image, and the inspection image when the inspection image is captured by the second imaging device, and then the direction of light irradiation is changed to irradiate the match image, whose pattern changes depending on the direction of light irradiation, and the match image is notified according to the similarity between a plurality of registered images captured in advance by a first imaging device of a different model from the second imaging device and the inspection image contained in each of the captured images. Collation device.
2. The processor notifies the comparison result using the registered image including four images captured by shifting the irradiation direction of light irradiating the comparison image by 90 degrees each along the periphery of the comparison image. The collation device according to claim 1.
3. The processor notifies the matching result using the registered images including, in addition to the four images, four additional images captured by shifting the light irradiation direction by less than 90 degrees along the periphery of the matching image with respect to the irradiation direction of each light irradiated on the matching image when capturing the four images. The collation device according to claim 2.
4. The processor adds, to the registered image as the four additional images, images captured by shifting the light irradiation direction by 45 degrees along the periphery of the match image with respect to the irradiation direction of each of the lights irradiated on the match image when capturing the four images. The collation device according to claim 3.
5. The processor notifies a result of matching the inspection image and the match image using the registered image captured by light irradiated from one point light source. The collation device according to any one of claims 1 to 4.
6. The processor notifies a result of matching the inspection image and the match image by using the registered image in which an angle between an optical axis of the point light source and an optical axis of a camera in the first photographing device that photographs the registered image is adjusted to match an angle between an optical axis of a camera in the second photographing device that photographs the inspection image and an optical axis of a built-in point light source built into the second photographing device. The collation device according to claim 5.
7. The collation image is a three-dimensional image formed three-dimensionally on a recording medium. The collation device according to any one of claims 1 to 6.
8. The three-dimensional image is an image formed by fixing powder on the recording medium. The collation device according to claim 7.
9. The powder image is an image formed of black or silver powder on the surface of the recording medium. The collation device according to claim 8.
10. The reference image is an image of the surface of an object to which a material composed of a mixture of silver powder is attached. The collation device according to any one of claims 1 to 6.
11. On the computer, A plurality of photographed images are obtained by photographing the test image to be compared with the second photographing device while changing the direction of illumination of light; The positional relationship between the camera, the light source, and the image is adjusted so that it is the same as the positional relationship between the camera capturing the inspection image, the light source irradiating the inspection image, and the inspection image when the inspection image is captured by the second photographing device, and then a process is executed in which a direction of light irradiation is changed to irradiate a match image whose pattern changes depending on the direction of light irradiation, and the match image is notified in accordance with a similarity between a plurality of registered images captured in advance by a first photographing device of a different model from the second photographing device and the inspection image contained in each of the captured images. Matching program.
Citation Information
Patent Citations
Device authentication system
JP2004171109A
Article ascertaining system, imaging device and article ascertaining method
JP2007034747A
Concealed authentication method and device
JP2007534067A
Object recognition device
JP2020154946A
