IMAGE PROCESSING DEVICE, IMAGE PROCESSING SYSTEM, AND IMAGE PROCESSING PROGRAM

Abstract

To provide an imaging processing device, an imaging processing system, and an imaging processing program that can image a position that does not correspond to an overexposed area as an area to be collated.SOLUTION: An overexposure position detection unit 20 acquires a captured image taken by emitting light from a light source, and extracts a region having a predetermined pixel value or more generated in the captured image due to the light emission of a light source as an overexposure portion to detect a position of the overexposed portion, and when the overexposed portion and a frame line overlap each other, a frame line position moving unit 24 moves the frame line to a range that does not overlap with the overexposed portion and displays the frame line on a display.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to an image capture processing device, an image capture processing system, and an image capture processing program. [Background technology]

[0002] Patent Document 1 proposes an imaging device in which a control unit acquires an image captured by an imaging unit as a first captured image, and then acquires an image captured as a second captured image, and if the blown-out areas detected in the first and second captured images are not in approximately the same position, designates the blown-out area in one of the captured images as an area to be corrected, and designates an area in the other captured image at a position equivalent to the area to be corrected as an area to be extracted, and combines the image portion extracted from the area to be extracted with the area to be corrected.

[0003] Patent document 2 proposes a digital pen that includes a first infrared LED and a second infrared LED that illuminate a printed document on which a code image is formed, an emission control unit that selects the LED to be turned on when a user writes on the printed document, an infrared CMOS that receives reflected light from the printed document illuminated with light by the selected LED, and an image processing unit that performs image processing on the light reception result by the infrared CMOS to obtain a code image, decodes the obtained code image to obtain identification information and position information, and detects whether or not whiteout has occurred in the light reception result and determines whether or not to switch the infrared LED to be turned on based on the detection result.

[0004] Patent Document 3 proposes an analytical image capturing and processing system that captures an image of a test object and processes the captured image data to make the image suitable for analysis in order to analyze the image of the test object by analyzing the image of the test object. In detail, when there is a defective part in the image data where color information or shape information is difficult to obtain due to a partial excess or deficiency of the illumination light amount by the light emitting element on the test object, luminance conversion is performed for each color on the image data so that the luminance and color information of the defective part approaches the luminance and color information as the reference, based on the luminance and color information of a predetermined area where the luminance and color information of the image data are unlikely to be excessive or insufficient. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2013-229698 A [Patent Document 2] JP 2009-230287 A [Patent Document 3] JP 2016-018262 A Summary of the Invention [Problem to be solved by the invention]

[0006] When approaching an object and emitting light from a light source to capture an image, the light reflected from the object may cause blown-out white areas in the captured image due to the emission of the light source. When matching a matching target area of ​​the captured image with a predetermined reference image, if the blown-out white areas overlap with the matching target area, accurate matching may not be possible.

[0007] Therefore, an object of the present invention is to provide an image capturing device, an image capturing system, and an image capturing program that are capable of capturing an image of a position that does not fall into a whiteout portion as a region to be compared. [Means for solving the problem]

[0008] In order to achieve the above object, an image capturing and processing device according to a first aspect includes a processor, and the processor acquires an image captured by emitting light from a light source, extracts whiteout parts having a pixel value equal to or greater than a predetermined pixel value that occur in the captured image due to the emission of the light source, and performs processing to set an area in the captured image that is located in a range that does not overlap with the extracted whiteout parts as an area to be compared with a predetermined reference image.

[0009] In addition, in the second aspect of the imaging processing device, in the imaging processing device of the first aspect, the processor acquires the captured image captured at a predetermined distance from an object within a predetermined density range and extracts the blown-out portion.

[0010] In addition, in the photographing processing device of the third aspect, in the photographing processing device of the first or second aspect, the processor extracts the blown out highlight portion while displaying a through image when photographing the photographed image, and displays position information indicating the acquisition position of the image to be compared with the reference image on the through image in an area not overlapping with the blown out highlight portion.

[0011] In addition, in the imaging processing device of the fourth aspect, in the imaging processing device of the first or second aspect, the processor displays position information indicating the acquisition position of the image to be compared with the reference image at a predetermined standard position, accepts an imaging instruction and extracts the blown out highlight portion from the captured image, and when the blown out highlight portion and the position information overlap, moves and displays the position information to an area that does not overlap with the blown out highlight portion.

[0012] In addition, in the image capturing and processing device according to a fifth aspect, in the image capturing and processing device according to the third or fourth aspect, the processor displays a frame line having a rotationally asymmetric shape as the position information.

[0013] In addition, in a sixth aspect of the imaging processing device, in the imaging processing device of any one of the third to fifth aspects, the processor displays the position information in the same quadrant as when the reference image was captured when the origin of the captured image is directly below the center of the light source.

[0014] In addition, in the imaging processing device of the seventh aspect, in the imaging processing device of the sixth aspect, the processor displays the position information at a position moved in a direction away from the origin and not overlapping with the blown-out portion.

[0015] In addition, an imaging processing device according to an eighth aspect is the imaging processing device according to any one of the third to fifth aspects, wherein the processor Center of When the origin is The above Shooting field of view of The position information is displayed in a quadrant in which a triangle formed by three points, namely, the center, the center of the whiteout portion, and a predetermined specific position of the position information, has a shape similar to that of the reference image when it was photographed.

[0016] In addition, in a ninth aspect of the imaging processing device, in the eighth aspect, the processor displays the location information within the quadrant in which the location information is located, at a position that does not overlap with the blown-out portion and that has been moved in a direction away from the blown-out portion.

[0017] In addition, in the photography processing device of a 10th aspect, in the photography processing device of any one of the aspects 3 to 9, the processor further performs processing to cut out an image of the target to be matched from the captured image when the target information representing the area of ​​the target to be matched provided on the object matches the position information, and display the result of matching it with the reference image.

[0018] In addition, a photography processing system according to an eleventh aspect includes a photography processing device according to any one of the first to tenth aspects, and a server that compares a comparison image obtained by cutting out the area to be compared from the photographed image with the reference image and transmits the comparison result to the photography processing device.

[0019] In addition, the photography processing program according to the twelfth aspect causes a computer to execute a process of acquiring a photographed image captured by illuminating a light source, extracting blown-out areas having a pixel value equal to or greater than a predetermined value that occur in the photographed image due to the illumination of the light source, and designating an area in the photographed image that does not overlap with the extracted blown-out areas as an area to be compared with a predetermined reference image. Effect of the Invention

[0020] According to the first aspect, it is possible to provide an imaging and processing device capable of imaging a position that does not correspond to a blown-out highlight portion as a region to be matched.

[0021] According to the second aspect, it is possible to extract blown-out highlights more accurately than in the case where images are obtained by photographing objects with various densities.

[0022] According to the third aspect, it is possible to capture an image with the matching target area aligned in a position that avoids whiteout areas.

[0023] According to the fourth aspect, it is possible to capture an image with the matching target area aligned in a position that avoids whiteout areas.

[0024] According to the fifth aspect, it is possible to capture an image of the area to be compared in a state closer to that in which the reference image is captured, as compared to the case where a frame line having a rotationally symmetric shape is displayed.

[0025] According to the sixth aspect, when the origin of the captured image is directly below the center of the light source, it is possible to capture an image of the area to be compared in a state closer to the positional relationship of the light source when the reference image was captured, compared to when position information is displayed in a different quadrant than when the reference image was captured.

[0026] According to the seventh aspect, when the origin of the captured image is directly below the center of the light source, it is possible to display position information in a position that avoids blown-out areas and capture an image in a state close to the positional relationship of the light source when the reference image was captured, compared to when position information is displayed in a different quadrant than when the reference image was captured.

[0027] According to the eighth aspect, when the field of view is taken as the origin, it is possible to capture an image of the area to be compared in a state closer to the positional relationship of the light source when the reference image was captured, compared to a case in which the positional information is displayed in a quadrant in which the triangle formed by the center of the field of view, the center of the blown-out area, and a predetermined specific position of the positional information is in a different positional relationship from when the reference image was captured.

[0028] According to the ninth aspect, when the field of view is taken as the origin, it is possible to display position information in a position that avoids the blown-out portion and is closer to the positional relationship of the light source when the reference image is captured, compared to when position information is displayed in a quadrant in which the triangle formed by the center of the field of view, the center of the blown-out portion, and a predetermined specific position of the position information has a different positional relationship from when the reference image was captured.

[0029] According to the tenth aspect, it is possible to display the comparison result obtained by comparing the area to be compared, which has been photographed while avoiding whiteout portions, with the reference image.

[0030] According to the eleventh aspect, it is possible to provide an image capturing and processing system capable of capturing an image at a position that does not correspond to a blown-out highlight portion as a region to be matched.

[0031] According to the twelfth aspect, it is possible to provide an image capturing processing program capable of capturing an image at a position that does not correspond to a blown-out highlight portion as a region to be compared. [Brief description of the drawings]

[0032] [Figure 1] 1 is a diagram showing a schematic configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] 1A and 1B are diagrams showing how an object is photographed using a mobile terminal in an information processing system according to the present embodiment. [Diagram 3] FIG. 2 is a block diagram showing the configuration of the main electrical systems of the server and the mobile terminal according to the embodiment; [Figure 4] FIG. 2 is a functional block diagram showing a functional configuration of a mobile terminal according to the present embodiment. [Diagram 5] 11A and 11B are diagrams illustrating an example of a whiteout portion and a frame line in a standard position for each type of mobile terminal. [Figure 6] 13 is a diagram showing an example of a histogram distribution of pixel values ​​in a collation area when an image of the collation mark is captured by mobile terminals A to C. FIG. [Figure 7] 13A and 13B are diagrams illustrating an example of moving the frame line for each mobile terminal to a range that does not overlap with the whiteout portion. [Figure 8] This figure shows an example of a triangle formed by three points directly below the center of the camera lens, directly below the center of the light source which will result in a blown-out highlight area, and a predetermined specific part of the frame line, regardless of the model of mobile terminal. [Figure 9] This figure shows an example in which the triangle formed by three points directly below the center of the camera lens, directly below the center of the light source which will result in a blown-out area, and a predetermined specific part of the frame line is approximately similar in shape, regardless of the model of mobile device, and the frame line is moved in a direction away from the center of the light source within the quadrant in which the specific part of the frame line exists. [Figure 10] FIG. 13 is a diagram for explaining an example of an automatic shutter. [Figure 11] FIG. 4 is a diagram for explaining an example of an operating condition in an example of an automatic shutter. [Figure 12] FIG. 13 is a diagram showing an example of a template corresponding to the shape of a matching mark. [Figure 13] 11 is a flowchart showing an example of a flow of processes performed during preparation for photographing a match image with a mobile terminal of the information processing system according to the present embodiment. [Figure 14] 10 is a flowchart showing an example of a processing flow when a match image is photographed in a mobile terminal of the information processing system according to the present embodiment. [Figure 15] 11 is a diagram for explaining an example in which a captured image of a predetermined range of an area is registered as a reference image. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0033] An example of this embodiment will be described in detail below with reference to the drawings. In this embodiment, an information processing system in which a server and a wireless base station of a mobile terminal are connected to each other via a communication line such as a network will be described as an example of an image processing system. Fig. 1 is a diagram showing a schematic configuration of an information processing system 10 according to this embodiment, and Fig. 2 is a diagram showing how an object is photographed using a mobile terminal in the information processing system according to this embodiment.

[0034] 1, an information processing system 10 according to this embodiment includes a server 12 and a wireless base station 16 for a mobile terminal 14 serving as an image capturing and processing device. The server 12 and the wireless base station 16 are connected via a communication line 18 such as a LAN (Local Area Network), a WAN (Wide Area Network), the Internet, an intranet, or the like.

[0035] Specifically, as shown in FIG. 2, the information processing system 10 according to the present embodiment provides a matching mark 34 on an object 32 to be judged for authenticity, provides a matching area 34A to be matched in a part of the matching mark 34, and registers an image of the matching area 34A captured in advance in the server 12 as a reference image. The matching mark 34 of the object 32 is then captured using the mobile terminal 14 to obtain a matching image, and the server 12 is requested to compare the image with the reference image, whereby the server 12 judges the authenticity of the object 32 by comparing the image with the previously registered reference image. In this embodiment, when the matching mark 34 is captured using a smartphone as the mobile terminal 14, the object 32 is approached and a light source is emitted to capture the image in order to obtain an image with high resolution (for example, 1000 dpi or more). In addition, in order to capture the object 32 provided with the matching mark 34 with sufficient resolution (for example, 1000 dpi or more), the image is captured at the closest possible capture distance of the mobile terminal 14.

[0036] The mobile terminal 14 photographs the surface of an object as a predetermined reference image, or photographs an image of a comparison target to be compared with the reference image. For example, an application for comparison or authenticity determination is installed, and the installed application performs a process of photographing a reference image or a comparison image. Note that a smartphone is used as an example of the mobile terminal 14. The smartphone is provided with a light source such as an LED (Light Emitting Diode) light source. In this embodiment, the comparison mark 34 is photographed by a smartphone in which the position of the light source differs for each model.

[0037] The server 12 receives a photographed image of a predetermined object from the mobile terminal 14 via the wireless base station 16, compares it with a reference image registered in advance, and uniquely identifies the object, and returns the identification result to the mobile terminal 14. In this embodiment, a technology is used that uniquely identifies the object and determines its authenticity by using an image of a random pattern that is accidentally generated on the surface of the object. Note that in this embodiment, the authenticity determination is described as an example, but is not limited to the authenticity determination. For example, a form in which the server 12 receives a photographed image of a two-dimensional barcode or the like from the mobile terminal 14, analyzes the photographed image, and obtains information such as a URL represented by the two-dimensional barcode may be applied.

[0038] Next, the main configuration of the electrical system of the server 12 and the mobile terminal 14 according to this embodiment will be described. Fig. 3 is a block diagram showing the main configuration of the electrical system of the server 12 and the mobile terminal 14 according to this embodiment. Since the server 12 and the mobile terminal 14 are basically configured as general computers, the mobile terminal 14 will be described as a representative, and only the differences will be described with respect to the server 12.

[0039] As shown in FIG. 3, the mobile terminal 14 according to the present embodiment includes a CPU 14A, a ROM 14B, a RAM 14C, a storage 14D, an operation unit 14E, a display 14F, and a communication line interface (I / F) unit 14G. The CPU 14A controls the overall operation of the mobile terminal 14. The ROM 14B stores various control programs and various parameters in advance. The RAM 14C is used as a work area when the CPU 14A executes various programs. The storage 14D stores various data and application programs. The operation unit 14E is used to input various information. The display 14F is used to display various information. The communication line interface unit 14G is connectable to a communication line 18 and transmits and receives various data to and from other devices connected to the communication line 18. The above-mentioned units of the mobile terminal 14 are electrically connected to each other by a system bus 14H. In addition, in the mobile terminal 14 according to the present embodiment, an example is described in which the mobile terminal 14 includes the storage 14D, but the present invention is not limited to this, and the mobile terminal 14 may include other non-volatile storage units such as a flash memory.

[0040] The mobile terminal also includes a camera 14J and a light source 14K, and is capable of acquiring an image captured by the camera 14J by emitting light from the light source 14K and transmitting the image to the server 12. Note that, for example, an LED is used as the light source, but a light source other than an LED may also be used.

[0041] With the above configuration, the mobile terminal 14 according to the present embodiment executes, by the CPU 14A, access to the ROM 14B, the RAM 14C, and the storage 14D, acquisition of various data via the operation unit 14E, and display of various information on the display 14F. In addition, the mobile terminal 14 executes, by the CPU 14A, control of transmission and reception of communication data such as captured images via the communication line interface unit 14G.

[0042] On the other hand, the server 12 is provided with a database (DB) 12I in which predetermined reference images for determining authenticity are stored, and performs a process of determining authenticity by comparing the photographed image received from the mobile terminal 14 with the reference image.

[0043] Next, a description will be given of the functional configuration of the mobile terminal 14. Fig. 4 is a functional block diagram showing the functional configuration of the mobile terminal 14 according to this embodiment.

[0044] 4, the mobile terminal 14 has, as its functional components, a whiteout position detection unit 20, an overlap determination unit 22, a frame line position movement unit 24, a match image acquisition unit 26, a match request unit 28, and a match result display unit 30. Each functional component is realized by the CPU 14A reading out a program for authenticity determination stored in the ROM 14B or the storage 14D, expanding the program in the RAM 14C, and executing the program.

[0045] The blown-out highlight position detection unit 20 obtains a photographed image photographed by the camera 14J by emitting light from the light source 14K, and detects the position of the blown-out highlight portion by extracting an area having a pixel value equal to or greater than a predetermined pixel value that occurs in the photographed image due to the emission of the light source 14K as a blown-out highlight portion. The blown-out highlight position varies in the photographed image depending on the type of the mobile terminal 14. That is, the position of the light source 14K relative to the camera 14J of the mobile terminal 14 varies for each type, and therefore the position of the blown-out highlight portion that occurs in the photographed image varies. For example, the positions of the blown-out highlight portions 38 are different for each of the mobile terminals A to C shown in FIG. 5. In the example of FIG. 5, the circular cross portion indicates the center of emission of the light source 14K, and the portion surrounded by a circle on the outside of the circular cross indicates the blown-out highlight portion 38 having a pixel value equal to or greater than a predetermined pixel value. Therefore, the blown-out highlight position detection unit 20 detects the position of the blown-out highlight portion 38 in the photographed image, which differs for each type of the mobile terminal 14. The detection of blown-out highlight portion 38 is performed, for example, by photographing an object having a uniform density within a predetermined density range at a predetermined distance so that the entire field of view has a uniform density, and extracting from the field of view of the photographed image an area where n or more pixels (for example, 100×100 or more pixels) having a pixel value of 245 or more are adjacent to each other in the case of an 8-bit grayscale image (256 gradations) as blown-out highlight portion 38. Note that when detecting blown-out highlight portion 38, if object 32 has a uniform density within a predetermined density range, object 32 may be photographed at a distance such that the entire field of view has a uniform density.

[0046] The overlap determination unit 22 determines whether the blown-out highlight portion 38 detected by the blown-out highlight position detection unit 20 overlaps with the comparison mark 34 including the comparison region 34A to be compared with a predetermined reference image. For example, it determines whether the blown-out highlight portion 38 overlaps with a predetermined standard position of a frame line for specifying an image acquisition position to be displayed on the display 14F. Specifically, in this embodiment, as shown in FIG. 2, a frame line 36 is displayed as position information representing the acquisition position of an image to be compared with a reference image at a predetermined standard position on the display 14F of the mobile terminal 14. Then, when the frame line 36 is aligned with the comparison mark 34 including the comparison region 34A to be compared with the reference image of the object 32 and the light source 14K is emitted to capture the image, it determines whether the blown-out highlight portion 38 overlaps with the frame line 36. For example, in the mobile terminals A to C shown in FIG. 5, the positional relationship between the frame line 36 shown as a square and the blown-out highlight portion 38 surrounded by a circle is different, and the frame line 36 and the blown-out highlight portion 38 may or may not overlap, so it determines whether they overlap. In the example of Fig. 5, mobile terminal A and mobile terminal C have a frame line 36 and a whiteout portion 38 overlapping, while mobile terminal B has a frame line 36 and a whiteout portion 38 that do not overlap. Note that, in this embodiment, the frame line 36 is described as having a rotationally non-symmetric shape as shown in Fig. 2, but a rectangular frame line 36 as shown in Fig. 5 may also be used. Alternatively, a circular or polygonal frame line may also be used. A non-rotationally symmetric shape is a shape whose direction is uniquely determined, unlike a circle, square, equilateral triangle, etc.

[0047] When the overlap determination unit 22 determines that the whiteout portion 38 and the frame 36 overlap, the frame position moving unit 24 moves the frame 36 to a range where it does not overlap with the whiteout portion 38 and displays it on the display 14F. In this embodiment, the matching area 34A in the matching mark 34 shown in FIG. 2 is matched with a reference image obtained by photographing the matching area 34A in advance. However, when the frame 36 and the whiteout portion 38 overlap as in the mobile terminals A and C shown in FIG. 5, the whiteout portion 38 makes it impossible to accurately match the reference image. Therefore, the frame position moving unit 24 moves the frame 36 to a range where it does not overlap with the whiteout portion 38 and displays it on the display 14F to accept a shooting instruction. For example, FIG. 6 shows an example of a histogram distribution of pixel values ​​of the matching area 34A when the matching mark 34 is photographed by the mobile terminals A to C. As shown in FIG. 6, the mobile terminals A and C have a large distribution of pixel values ​​of 0 and are not uniform, so there is a high possibility that accurate matching will not be possible. For mobile terminal B, the pixel values ​​are distributed uniformly, making it possible to perform accurate matching.

[0048] The movement of the frame line 36 to a range that does not overlap with the blown-out white portion 38 is, for example, performed by displaying the frame line 36 in the same quadrant regardless of the type of mobile terminal 14 when the origin is set directly below the center of the light source 14K visible on the display 14F. Since uneven objects and shining objects are affected by the light direction of the light source 14K, it is necessary to move the position of the frame line 36 so that it is close to the positional relationship of the light source 14K when the reference image is captured. For example, the frame line 36 is always in the first quadrant for the mobile terminals A to C used for matching with the reference image, the mobile terminal 14 for registering the reference image, or the registration device consisting of a camera and lighting, so that the direction of the lighting is always maintained. Then, the frame line 36 is moved to a position that avoids the blown-out white portion in the first quadrant, that is, in a direction away from the origin. For example, as shown in FIG. 7, the mobile terminals A and C move the frame line 36 from the dotted line position to the solid line position. The frame line 36 does not move for the mobile terminal B because the blown-out white portion 38 and the frame line 36 do not overlap.

[0049] Alternatively, when the frame line 36 is displayed on the display 14F, the triangle formed by the points directly below the lens center which is the center of the field of view of the camera 14J, the points directly below the center of the light source 14K which is the blown-out part 38, and a specific part of the frame line 36 is calculated to be similar to the triangle formed by the points directly below the lens center of the camera 14J, the points directly below the center of the light source 14K which is the blown-out part 38, and a specific part of the frame line 36 regardless of the type of the mobile terminal 14. In other words, the triangle formed by the points directly below the lens center of the camera 14J, the points directly below the center of the light source 14K, and the specific part of the frame line 36 is calculated to be similar to the triangle formed by the points directly below the lens center of the camera 14J, the points directly below the center of the light source 14K, and the specific part of the frame line 36 when the reference image is taken. This maintains the positional relationship between the light source 14K and the lens center when the reference image is taken and when the collation image is taken. In this case, the frame line 36 is moved in a direction away from the center of the light source 14K within a quadrant in which the specific part of the frame line 36 exists, with the origin being the point directly below the lens center which is the center of the field of view of the camera. For example, as shown in Fig. 8, for each of mobile terminals A to C, the position of frame line 36 is calculated so that the triangles formed by the center of the field of view of camera 14J, the center of the circular cross directly below the center of light source 14K, and three points of a specific portion of frame line 36 are similar in shape. In Fig. 8, frame line 36 is also shown as a similar shape. Then, as shown in Fig. 8, frame line 36 is moved as necessary so that the frame line does not overlap with blown-out white portion 38. In this case, when the origin is directly below the lens center, which is the center of the field of view of the camera, frame line 36 is moved in a direction away from the center of light source 14K within the quadrant in which the specific portion of frame line 36 exists, as shown by the arrow in Fig. 9.

[0050] The match image acquisition unit 26 extracts an image of an area corresponding to the frame line 36 from the photographed image as a match image, thereby acquiring a match image for matching with the reference image. Here, the match image acquired from the photographed photographed image may be an image in which the overexposed part does not overlap the match area in the match image, and may be an image in which the overexposed part exists outside the match area in the match image. In other words, the overexposed part 38 may appear outside the match area of ​​the match image. In addition, the match image acquisition unit 26 may extract and acquire a match image from a photographed image photographed by accepting a photographing instruction, but may also align the frame line 36 with the match mark 34 shown in FIG. 10, take a photograph by automatically releasing the shutter when the frame line 36 aligns with the match mark 34, and extract and acquire a match image from the photographed image. Note that the hatching in FIG. 10 represents the match mark 34, and the solid line represents the outline of the match mark.

[0051] As an example of automatic shutter, for example, when the average density of y% of adjacent pixels among n×n pixels is d1 and the average density of the other pixels is d2, d1 / d2 is m, and these y and m are determined in advance. For example, in 8-bit grayscale, d1=35, d2=200, etc., and the d1 side is the side of the matching mark printed in solid silver, and the d2 side is the background paper, etc. If there are, for example, 10 rectangles on the frame 36 shown in Figure 10 with appropriate y and m, the shutter is released as a match. In this case, the degree of match with the mark changes depending on parameters such as the rectangle size, y, and m. For example, in the case of the degree of match in the upper row of Figure 11, the automatic shutter is not released, but in the case of the degree of match in the middle row, the automatic shutter is released. In addition, in the case of the degree of match in the lower row, the automatic shutter is released depending on the threshold setting.

[0052] Alternatively, an automatic shutter may be released by template matching to capture an image. For example, a template corresponding to the shape of the verification mark 34 shown in FIG. 12 may be prepared, and the automatic shutter may be released when the degree of match between the captured image and the template is equal to or greater than a predetermined value. For example, a circumscribing frame may be extracted and the degree of match may be calculated using only this frame, or the inside of the frame may be filled uniformly with a certain density and the degree of match may be calculated with the template side being uniformly dense. Then, an image whose degree of match is equal to or greater than a predetermined value is acquired as a captured image. The image is kept in a constant shooting state during the operation period of the verification application like a video, and pattern matching calculations are performed one after another in the mobile terminal 14, and the captured image is acquired when the degree of match is within a predetermined threshold value.

[0053] In this embodiment, the match image acquisition unit 26 performs a process of extracting the area of ​​the match image from the captured image, but the captured image may be sent to the server 12 and the process of extracting the area of ​​the match image may be performed on the server 12 side.

[0054] The matching request unit 28 transmits the match image acquired by the match image acquisition unit 26 to the server 12, thereby requesting matching of the reference image with the match image.

[0055] The matching result display unit 30 receives the matching result between the reference image and the matching image from the server 12, and performs processing to display the matching result on the display 14F.

[0056] Meanwhile, server 12 receives the match image sent from matching request unit 28, matches it with a preregistered reference image, uniquely identifies the object, and performs a process of returning the identification result to mobile terminal 14. In this embodiment, for example, a reference image of matching area 34A in the matching mark provided on object 32 shown in FIG. 2 is preregistered in server 12, and the authenticity of the object is determined by matching the reference image with the match image, and the determination result is returned to mobile terminal 14.

[0057] Next, specific processing performed by the mobile terminal 14 of the information processing system 10 according to this embodiment configured as described above will be described. Fig. 13 is a flow chart showing an example of a flow of processing performed during preparation for photographing a collation image by the mobile terminal 14 of the information processing system 10 according to this embodiment. Note that the processing of Fig. 13 starts, for example, when an application for authenticity determination is installed in the mobile terminal 14 and the application is started for the first time, and the processing of Fig. 13 may be omitted when the application is started for the second or subsequent times.

[0058] In step 100, the CPU 14A issues an instruction to photograph a uniform density, and the process proceeds to step 102. For example, the instruction to photograph a uniform density surface is issued by displaying a message on the display 14F to photograph a uniform density surface close up. Note that, as an example of a uniform density, for example, a surface of uniform density may be provided on a surface of the object 32 different from the surface on which the matching mark 34 is provided, and an instruction to photograph the surface may be issued.

[0059] In step 102, the CPU 14A judges whether or not an image has been captured. This judges whether or not an operation to release the shutter of the camera 14J has been performed on the operation unit 14E. The process waits until the judgement is affirmative, and then proceeds to step 104.

[0060] In step 104, the CPU 14A turns on the light source 14K and causes the camera 14J to take an image, and then the process proceeds to step .

[0061] In step 106, the CPU 14A acquires the image captured by the camera 14J, and the process proceeds to step 108.

[0062] In step 108, the CPU 14A detects the blown-out highlight portion 38 and proceeds to step 110. That is, the blown-out highlight position detection unit 20 detects the position of the blown-out highlight portion 38 by extracting an area having a pixel value equal to or greater than a predetermined pixel value that occurs in the captured image in association with the emission of the light source 14K as the blown-out highlight portion 38.

[0063] In step 110, the CPU 14A determines the position of the frame 36 and ends a series of photographing preparation processes. That is, the overlap determination unit 22 determines whether the whiteout portion 38 detected by the whiteout position detection unit 20 overlaps with a region to be compared with a predetermined reference image. For example, it determines whether the whiteout portion 38 overlaps with a predetermined standard position of the frame 36 for specifying an image acquisition position to be displayed on the display 14F. If the whiteout portion 38 does not overlap with the frame 36, the standard position is determined as the position of the frame 36. On the other hand, if the whiteout portion 38 overlaps with the frame 36, the frame position moving unit 24 moves the frame 36 to a range where it does not overlap with the whiteout portion 38 and displays it on the display 14F. The determined position of the frame 36 may be stored in the storage 14D, and when the application is started for the second or subsequent times, the position of the frame 36 stored in the storage 14D may be used to perform the process of FIG. 14 described later.

[0064] Next, a process for photographing a match image performed after the process for preparing for photographing in Fig. 13 will be described. Fig. 14 is a flowchart showing an example of a process flow for photographing a match image performed in the mobile terminal 14 of the information processing system 10 according to this embodiment. The process in Fig. 14 may be started following the process in Fig. 13. Alternatively, the first start of the application may be started following the process in Fig. 13, and the second start of the application may be started from the process in Fig. 14, and the process in Fig. 14 may be performed using the position of the frame line 36 at the first start of the application.

[0065] In step 200, CPU 14A displays frame line 36 on display 14F and proceeds to step 202. That is, frame line 36 is displayed at the position determined in the above-mentioned step 110. As a result, by aligning comparison mark 34 with frame line 36 and photographing it, a comparison image is photographed while avoiding whiteout portion 38.

[0066] In step 202, CPU 14A judges whether or not photography is to be performed. This judgment may be made by judging whether or not a photography instruction has been received by operation unit 14E. Alternatively, match image acquisition unit 26 may judge whether or not the degree of match between match mark 34 and frame line 36 is equal to or greater than a predetermined value. The process waits until the judgment is affirmative, and then proceeds to step 204.

[0067] In step 204, the CPU 14A acquires an image captured by the camera 14J and then proceeds to step 206.

[0068] In step 206, CPU 14A extracts an image of the region of frame line 36, and proceeds to step 208. That is, match image acquisition unit 26 extracts an image of the region corresponding to frame line 36 from the captured image as a match image, thereby acquiring a match image to be matched with the reference image.

[0069] In step 208, CPU 14A transmits the match image to server 12 to request matching, and proceeds to step 210. That is, matching request unit 28 transmits the match image acquired by match image acquisition unit 26 to server 12, thereby requesting matching of the reference image with the match image.

[0070] In step 210, the CPU 14A waits until the matching result is received from the server 12, and then the process proceeds to step 212.

[0071] In step 212, CPU 14A displays the matching result on display 14F and ends the series of processes. That is, matching result display unit 30 receives the matching result between the reference image and the matching image from server 12 and performs a process of displaying the matching result on display 14F. As a result, the authenticity determination result of object 32 is displayed on display 14F.

[0072] In the above embodiment, in step 102, it is determined whether or not an operation to release the shutter of camera 14J has been performed on operation unit 14E, but the process of step 102 may be omitted. That is, while a through image is being displayed when a photographed image is being photographed, a blown-out highlight portion 38 may be extracted, and a frame line 36 may be displayed in the through image as position information representing the acquisition position of an image to be compared with a reference image in an area not overlapping with blown-out highlight portion 38, and a photographing instruction may be accepted.

[0073] In the above embodiment, the frame line 36 is displayed on the display 14F of the mobile terminal 14, and the frame line 36 is aligned with the verification mark 34 when photographing, and the image of the verification area 34A of the verification mark 34 is extracted and compared with the reference image by the server 12. However, the present invention is not limited to this. For example, the display of the frame line 36 may be omitted, and a photographed image of a predetermined range may be registered as the reference image as shown by hatching in FIG. 15. A frame or the like may be provided on the object 32 so that the registered predetermined range can be recognized, and the photographed image is acquired by photographing the object 32 with the mobile terminal 14 aligned with the frame. Then, the frame shown by hatching in FIG. 15 may be recognized from the photographed image, and an image of a region of the recognized frame that avoids the whiteout portion 38 may be acquired as a verification image, and the verification image may be transmitted to the server 12 together with position information of the verification image relative to the frame. In this way, the server 12 can extract a reference image corresponding to the position of the verification image relative to the frame and compare it with the verification image to determine authenticity. In this case, the size of the hatched area shown in Figure 15 to be registered in the server 12 is made sufficiently larger than the size of the area of ​​the blown out white portion 38, so that the area avoiding the blown out white portion 38 can be used as the match image.

[0074] In the above embodiment, an example has been described in which a match image is photographed using the mobile terminal 14 and a matching request is made to the server 12, but the present invention is not limited to this. For example, a match image may be photographed by the camera 14J, and the photographed match image may be imported into an information processing device such as a personal computer, and a matching request may be made to the server 12. Alternatively, the server 12 may be omitted, and a reference image may be registered in advance in the mobile terminal 14, and matching may be performed by the mobile terminal 14, so that the system is configured only with the mobile terminal 14.

[0075] In addition, in the above embodiment, the CPU has been described as an example of a processor, but the term "processor" refers to a processor in a broad sense, and includes general-purpose processors (e.g., CPUs, etc.) and dedicated processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic devices, etc.).

[0076] In addition, the operations of the processor in the above embodiments may be performed not only by one processor, but also by multiple processors located at physically separate locations working together. The order of the operations of the processor is not limited to the order described in the above embodiments, and may be changed as appropriate.

[0077] Furthermore, the processing performed by the mobile terminal 14 in the information processing system 10 according to the above embodiment may be processing performed by software, processing performed by hardware, or a combination of both. Furthermore, the processing performed by the mobile terminal 14 may be stored as a program in a storage medium and distributed.

[0078] Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit and scope of the present invention. [Explanation of symbols]

[0079] 10. Information Processing Systems 12 Server 14 Mobile devices 14A CPU 20 Whiteout position detection section 22 Duplication determination section 24 Frame position moving section 26 Matching image acquisition unit 28. Verification Request Department 30 Matching result display section 36 Border 38 Whiteout area

Claims

1. a processor, the processor comprising: Obtaining a captured image by emitting light from a light source; extracting a blown-out portion having a pixel value equal to or greater than a predetermined pixel value that occurs in the captured image due to light emission from the light source; The photographing and processing device performs processing to designate an area in the photographed image that does not overlap with the extracted whiteout portion as a comparison target area to be compared with a predetermined reference image.

2. The image capturing and processing device according to claim 1 , wherein the processor obtains the captured image captured at a predetermined distance from an object having a predetermined density range, and extracts the blown-out highlight portion.

3. 3. The photographing processing device according to claim 1, wherein the processor extracts the blown-out portion while displaying a through image when the photographed image is photographed, and displays position information on the through image, in an area not overlapping with the blown-out portion, the position information indicating the acquisition position of the image to be compared with the reference image.

4. the processor displays position information indicating an acquisition position of an image to be compared with the reference image at a predetermined standard position, and extracts the blown-out highlight portion from the captured image captured in response to a shooting instruction; 3. The image capturing and processing device according to claim 1, wherein, when the blown-out highlight portion and the position information overlap, the position information is moved to an area not overlapping the blown-out highlight portion and displayed.

5. The imaging processing device according to claim 3 , wherein the processor displays a frame line having a rotationally asymmetric shape as the position information.

6. The photographing and processing device according to any one of claims 3 to 5, wherein the processor displays the position information in the same quadrant as when the reference image was photographed, when the origin of the photographed image is directly below the center of the light source.

7. The image capturing and processing device according to claim 6 , wherein the processor displays the position information at a position that is moved in a direction away from the origin and does not overlap with the blown-out highlight portion.

8. The photographing processing device described in any one of claims 3 to 5, wherein the processor displays the position information in a quadrant in which a triangle formed by the center of the photographing field of view, the center of the blown-out portion, and a predetermined specific position of the position information, when the center of the photographing field of view is the origin, is similar to the triangle formed by the center of the photographing field of view, the center of the blown-out portion, and a predetermined specific position of the position information, when the reference image is photographed.

9. The image capturing and processing device according to claim 8 , wherein the processor displays the position information at a position within the quadrant in which the position information exists, the position being moved away from the blown-out highlight portion and not overlapping with the blown-out highlight portion.

10. The photographing and processing device according to any one of claims 3 to 9, wherein the processor further performs a process of cutting out an image of the target to be compared from the photographed image when the position information matches target information representing an area of ​​the target to be compared provided on the object, and displaying the result of comparing it with the reference image.

11. The imaging and processing device according to any one of claims 1 to 10, a server that compares a comparison image obtained by cutting out the area to be compared from the photographed image with the reference image and transmits a comparison result to the photographing and processing device; A photographing and processing system including:

12. On the computer, Obtaining a captured image by emitting light from a light source; extracting a blown-out portion having a pixel value equal to or greater than a predetermined pixel value that occurs in the captured image due to light emission from the light source; A photography processing program for executing a process of designating an area in the captured image that does not overlap with the extracted blown-out white portion as a comparison target area to be compared with a predetermined reference image.

Images