Image processing apparatus, image processing system, storage medium, image processing method, and computer program product

Abstract

Provided is a photography processing device, photography processing system, storage medium, photography processing method, and computer program product, capable of capturing a position not corresponding to an overexposed portion as a collation target region. The photography processing device includes a processor that performs the following processing: acquires a photography image captured while a light source is emitting light, extracts an overexposed portion having a pixel value equal to or higher than a predetermined value that is generated in the photography image due to the emission of light from the light source, and sets a region in the photography image that is positioned in a range that does not overlap the extracted overexposed portion as a region that is collated with a predetermined reference image.

Description

Technical Field

[0001] This invention relates to a photographic processing apparatus, a photographic processing system, a storage medium, a photographic processing method, and a computer program product. Background Technology

[0002] Patent Document 1 discloses a camera device in which a control unit acquires an image captured by the camera unit as a first photographic image and then acquires an image captured afterward as a second photographic image. When the overexposed areas detected in the first and second photographic images are not at approximately the same location, the control unit designates the overexposed area in one of the photographic images as a correction target area and designates the area in the other photographic image that corresponds to the correction target area as an extraction target area. The control unit then combines the image portion extracted from the extraction target area into the correction target area.

[0003] Patent Document 2 discloses a digital pen comprising: a light-emitting control unit that selects a first infrared LED, a second infrared LED, and an LED that is illuminated when a user takes notes on the printed document; an infrared CMOS that receives reflected light from the printed document illuminated by the selected LED; and an image processing unit that performs image processing on the light-receiving result based on the infrared CMOS to acquire an coded image, decodes the acquired coded image to obtain identification information and position information, detects overexposure in the light-receiving result, and determines whether to switch the illuminated infrared LED based on the detection result.

[0004] Patent document 3 discloses an image processing system for analysis. In order to analyze an image of an object under inspection, the system captures an image of the object and processes the captured image data to make it suitable for analysis. Specifically, when there are defective portions in the image data where color and shape information is difficult to obtain due to localized over- or under-illumination caused by the light-emitting elements on the object under inspection, the system uses the brightness and color information of a predetermined area in the image data where over- or under-illumination is unlikely to occur as a reference. The system then performs brightness conversion on the image data for each color, so that the brightness and color information of the defective portions approximate the reference brightness and color information.

[0005] Patent Document 1: Japanese Patent Application Publication No. 2013-229698

[0006] Patent Document 2: Japanese Patent Application Publication No. 2009-230287

[0007] Patent Document 3: Japanese Patent Application Publication No. 2016-018262

[0008] If a light source is brought close to an object and an image is taken, the reflected light from the object may sometimes create overexposed areas in the photographic image, accompanied by the light emitted from the light source. When comparing the area of ​​the captured image to a pre-set reference image, inaccurate comparisons may not be possible if the overexposed areas overlap with the reference area. Summary of the Invention

[0009] Therefore, the object of the present invention is to provide a photographic processing apparatus, photographic processing system, storage medium, photographic processing method, and computer program product capable of photographing a location that does not correspond to an overexposed portion as a reference area.

[0010] To achieve the above objectives, the photographic processing apparatus according to the first method includes a processor that performs the following processing: acquiring a photographic image by emitting light from a light source and taking a picture; during the display of a real-time view image while the photographic image is being taken, extracting an overexposed portion in the photographic image that is generated by the emission of the light source at a preset pixel value or higher; designating a region in the photographic image that does not overlap with the extracted overexposed portion as a region for comparison with a preset reference image; and displaying position information indicating the acquisition position of the image compared with the reference image in the real-time view image in a region that does not overlap with the overexposed portion.

[0011] Furthermore, in the photographic processing apparatus of the second method, in the photographic processing apparatus of the first method, the processor performs the following processing: acquiring the photographic image taken by approaching an object within a predetermined concentration range to a predetermined distance, and extracting the overexposed portion.

[0012] Furthermore, in the photographic processing apparatus of the third method, in the photographic processing apparatus of the first method, the process of displaying position information indicating the acquisition position of the image compared with the reference image on the real-time view image includes: extracting the overexposed portion from the photographic image taken by displaying the position information indicating the acquisition position of the image compared with the reference image at a preset standard position and receiving a photographic instruction; when the overexposed portion overlaps with the position information, moving the position information to an area that does not overlap with the overexposed portion for display.

[0013] Furthermore, in the photographic processing apparatus of the fourth method, in the photographic processing apparatus of the first method, the processor performs the following processing: displays a frame line of a non-rotationally symmetric shape as the position information.

[0014] Furthermore, in the photographic processing apparatus of the fifth method, in the photographic processing apparatus of the first method, the processor performs the following processing: when the center of the light source is taken as the origin of the photographic image, the position information is displayed in the same quadrant as when the reference image was captured.

[0015] Furthermore, in the photographic processing apparatus of the sixth method, which is the same as that of the fifth method, the processor performs the following processing: displays the position information at a position that does not overlap with the overexposed portion after moving away from the origin.

[0016] Furthermore, in the photographic processing apparatus of the seventh method, in the photographic processing apparatus of the first method, the processor performs the following processing: when the photographic field of view is taken as the origin, the position information is displayed as a quadrant with a shape similar to that when the reference image is captured, forming a triangle at the center of the photographic field of view, the center of the overexposed portion, and a pre-set specific position of the position information.

[0017] Furthermore, in the photographic processing apparatus of the 8th method, the processor performs the following processing: within the quadrant where the position information exists, displays the position information at a position that does not overlap with the overexposed portion after moving in a direction away from the overexposed portion.

[0018] Furthermore, in the photographic processing apparatus of the ninth method, in any of the photographic processing apparatuses of the first to eighth methods, the processor further performs the following processing: displaying an image of the reference object cropped from the photographic image when the object information representing the area of ​​the reference object set on the object is matched with the position information, and comparing it with the reference image.

[0019] Furthermore, the photographic processing system involved in the 10th method includes: the photographic processing apparatus described in any one of the 1st to 9th methods; and a server that compares a comparison image obtained by cropping the region of the comparison object from the photographic image with the reference image and sends the comparison result to the photographic processing apparatus.

[0020] Furthermore, the storage medium involved in the 11th method stores a photographic processing program that causes a computer to perform the following processing: acquiring a photographic image by emitting light from a light source and taking a picture; during the display of a live view image while the photographic image is being taken, extracting overexposed portions in the photographic image that are generated with the emission of light from the light source and exceeding a predetermined pixel value; in the photographic image, setting a region in the range that does not overlap with the extracted overexposed portion as a region for comparison with a predetermined reference image; and in the region that does not overlap with the overexposed portion, displaying position information indicating the acquisition position of the image compared with the reference image in the live view image.

[0021] Furthermore, the photographic processing method involved in the 12th method includes the following steps: acquiring a photographic image by emitting light from a light source and taking a picture; during the display of a real-time view image while the photographic image is being taken, extracting an overexposed portion in the photographic image that is generated with the emission of light from the light source and exceeds a predetermined pixel value; and in the photographic image, setting a region located in a range that does not overlap with the extracted overexposed portion as a region for comparison with a predetermined reference image, and displaying position information indicating the acquisition position of the image compared with the reference image in the real-time view image in a region that does not overlap with the overexposed portion.

[0022] Furthermore, the computer program product involved in the 13th method includes a computer program for enabling a computer to perform the following functions: acquiring a photographic image by emitting light from a light source and taking a picture; during the display of a live view image while the photographic image is being taken, extracting an overexposed portion in the photographic image that is generated with the emission of light from the light source and exceeds a predetermined pixel value; in the photographic image, setting a region in the range that does not overlap with the extracted overexposed portion as a region for comparison with a predetermined reference image; and in the region that does not overlap with the overexposed portion, displaying position information indicating the acquisition position of the image compared with the reference image in the live view image.

[0023] Invention Effects

[0024] According to the first method, a photographic processing apparatus can be provided that can capture a location that does not correspond to an overexposed portion as a reference object area, and can capture images while aligning the reference object area with a location that avoids the overexposed portion.

[0025] According to the second method, compared with acquiring photographic images of objects with varying concentrations, it is possible to accurately extract overexposed portions.

[0026] According to the third method, it is possible to take a picture while aligning the area of ​​the target object with a position that avoids overexposed parts.

[0027] According to the fourth method, compared with displaying the frame lines of a rotationally symmetric shape, it is possible to capture an image of the area of ​​the comparison object in a state close to that when capturing the reference image.

[0028] According to the fifth method, compared to shooting with position information displayed in a different quadrant than when shooting a reference image, where the center of the light source is directly below the origin of the photographic image, it is possible to shoot an image of the area of ​​the reference object in a state that is closer to the positional relationship of the light source when shooting the reference image.

[0029] According to the sixth method, compared to shooting with the center of the light source directly below the origin of the photographic image and displaying position information in a different quadrant than when shooting the reference image, it is possible to shoot with position information displayed in a position that avoids overexposure, in a state that is closer to the positional relationship of the light source when shooting the reference image.

[0030] According to the seventh method, compared to shooting when the triangle formed by the three points of the center of the photographic field of view, the center of the overexposed portion, and the pre-set specific position information is used as the origin, and the quadrant display position information is used to display position information with a different positional relationship than when shooting the reference image, it is possible to shoot an image of the area of ​​the comparison object in a state that is closer to the positional relationship of the light source when shooting the reference image.

[0031] According to the eighth method, compared to when the triangle formed by the three points of the center of the photographic field of view, the center of the overexposed part, and the pre-set specific position of the position information is used as the origin, and the position information is displayed in a quadrant with a different positional relationship than when shooting the reference image, the position information can be displayed in a position that is closer to the positional relationship of the light source when shooting the reference image, while avoiding the overexposed part.

[0032] According to the ninth method, it is possible to display the comparison results of comparing the area of ​​the control object, which was photographed while avoiding overexposed parts, with the reference image.

[0033] According to the 10th method, a photographic processing system can be provided that can capture a region that does not correspond to an overexposed area as a reference object.

[0034] According to the 11th method, a storage medium can be provided that stores a photographic processing program capable of capturing a region that does not correspond to an overexposed area as a reference object.

[0035] According to the 12th method, a photographic processing method can be provided that can capture a location that does not correspond to the overexposed part as a reference object area.

[0036] According to the 13th method, a computer program product is provided that enables a computer to execute a photographic processing program that captures a region that does not correspond to an overexposed portion as a reference object. Attached Figure Description

[0037] The embodiments of the present invention will be described in detail with reference to the following figures.

[0038] Figure 1 This is a diagram showing the general structure of the information processing system involved in this embodiment;

[0039] Figure 2 This diagram illustrates the situation of taking a picture of an object using a mobile terminal in the information processing system according to this embodiment.

[0040] Figure 3 This is a block diagram showing the main structural components of the electrical system of the server and mobile terminal involved in this embodiment;

[0041] Figure 4 This is a functional block diagram illustrating the functional structure of the mobile terminal involved in this embodiment;

[0042] Figure 5 This is an example diagram showing the frame lines of the overexposed areas and standard positions for each type of mobile terminal;

[0043] Figure 6 This is an example of a histogram distribution showing the pixel values ​​of the control area when the control mark is photographed by mobile terminals A to C;

[0044] Figure 7 This is a diagram showing an example of how the frame lines of each mobile terminal move towards a range that does not overlap with the overexposed portion;

[0045] Figure 8 This diagram illustrates an example where, regardless of the type of mobile device, the triangle formed by the three points directly below the center of the camera lens, the center of the light source that becomes the overexposed part, and a pre-defined specific part of the frame line is set to a roughly similar shape.

[0046] Figure 9 This diagram illustrates an example where, regardless of the type of mobile device, the triangle formed by the three points directly below the center of the camera lens, the center of the light source that becomes the overexposed part, and a pre-defined specific part of the frame line is set to a roughly similar shape, and the frame line is moved away from the center of the light source within the quadrant where the specific part of the frame line exists.

[0047] Figure 10 This is a diagram used to illustrate an example of an automatic shutter;

[0048] Figure 11 This is a diagram illustrating one example of the operating conditions in an example of an automatic shutter;

[0049] Figure 12 This is a diagram showing an example of a template that corresponds to the shape of the reference mark;

[0050] Figure 13 This is a flowchart illustrating an example of the processing flow performed during the preparation for shooting when a mobile terminal of the information processing system according to this embodiment is used to capture a comparison image.

[0051] Figure 14 This is a flowchart illustrating an example of the processing flow when a mobile terminal of the information processing system according to this embodiment is used to capture a comparison image.

[0052] Figure 15 This is an example diagram used to illustrate a pre-defined area for logging, serving as a reference image.

[0053] Symbol Explanation

[0054] 10-Information processing system, 12-Server, 14-Mobile terminal, 14A-CPU, 20-Overexposure position detection unit, 22-Overlap determination unit, 24-Frame position movement unit, 26-Comparison image acquisition unit, 28-Comparison request unit, 30-Comparison result display unit, 36-Frame line, 38-Overexposure part. Detailed Implementation

[0055] Hereinafter, an example of this embodiment will be described in detail with reference to the accompanying drawings. In this embodiment, an information processing system in which the server and the wireless base station of the mobile terminal are connected respectively via communication lines such as a network will be used as an example of a photo processing system. Figure 1 This is a diagram showing the general structure of the information processing system 10 according to this embodiment. Figure 2 This diagram illustrates a situation where an object is photographed using a mobile terminal in the information processing system 10 according to this embodiment.

[0056] like Figure 1 As shown, the information processing system 10 according to this embodiment includes a server 12 and a wireless base station 16 that serves as a mobile terminal 14 for photographic processing. The server 12 and the wireless base station 16 are connected via communication lines 18 such as LAN (Local Area Network), WAN (Wide Area Network), Internet, and intranet.

[0057] Specifically, such as Figure 2As shown, the information processing system 10 of this embodiment sets a comparison mark 34 on the object 32 for authenticity determination. A comparison area 34A of the comparison object is set on a portion of the comparison mark 34. An image obtained by pre-capturing the comparison area 34A is used as a reference image and pre-registered on the server 12. Then, the comparison mark 34 of the object 32 is captured by the mobile terminal 14 to obtain a comparison image. The server 12 is requested to compare the image with the reference image. Thus, the authenticity of the object 32 is determined by comparing the image with the pre-registered reference image by the server 12. In addition, in this embodiment, when the smartphone is used as the mobile terminal 14 to capture the comparison mark 34, in order to acquire an image at a high resolution (e.g., 1000 dpi or higher), the object 32 is brought close to the light source to emit light for the capture. Furthermore, in order to capture the object 32 with the comparison mark 34 at a sufficient resolution (e.g., 1000 dpi or higher), the image is captured at the closest possible shooting distance of the mobile terminal 14.

[0058] The mobile terminal 14 captures images of the surface of an object as a pre-set reference image, or captures an image of a comparison object compared to the reference image. For example, an application for comparison determination or authenticity determination is installed, and the process of capturing the reference image or comparison image is performed through the installed application. Furthermore, a smartphone is used as an example of the mobile terminal 14. Smartphones are equipped with a light source such as an LED (Light Emitting Diode). Moreover, in this embodiment, the comparison mark 34 is captured using a smartphone whose light source position varies depending on the model.

[0059] Server 12 performs the following processing: it receives a pre-set photographic image of an object from mobile terminal 14 via wireless base station 16, compares it with a pre-registered reference image, thereby uniquely identifying the object and sending the identification result back to mobile terminal 14. In this embodiment, the authenticity of the object is determined by using an image of a random pattern that is accidentally generated on the surface of the object. Furthermore, while authenticity determination is described as an example in this embodiment, it is not limited to it. For example, it could also be used to receive photographic images such as two-dimensional barcodes from mobile terminal 14 and analyze the images to obtain information such as the URL represented by the two-dimensional barcode.

[0060] Next, the main structural components of the electrical systems of the server 12 and mobile terminal 14 according to this embodiment will be described. Figure 3This is a block diagram showing the main structural components of the electrical systems of the server 12 and mobile terminal 14 according to this embodiment. Furthermore, since the server 12 and mobile terminal 14 are basically designed with a general-purpose computer structure, the description will focus on the mobile terminal 14, and the description of the server 12 will only address the differences.

[0061] like Figure 3 As shown, the mobile terminal 14 according to this embodiment includes a CPU 14A, a ROM 14B, a RAM 14C, a memory 14D, an operation unit 14E, a display 14F, and a communication line interface (I / F) unit 14G. The CPU 14A controls the overall operation of the mobile terminal 14. The ROM 14B pre-stores various control programs and various parameters. The RAM 14C serves as a working area when the CPU 14A executes various programs. The memory 14D stores various data and application programs. The operation unit 14E is used to input various information. The display 14F is used to display various information. The communication line interface unit 14G is configured to connect to a communication line 18 and to transmit and receive various data with other devices connected to the communication line 18. All of the above components of the mobile terminal 14 are electrically connected to each other via a system bus 14H. In addition, in the mobile terminal 14 according to this embodiment, the example of having a memory 14D is described, but it is not limited to this, and other non-volatile storage units such as flash memory may also be included.

[0062] Furthermore, the mobile terminal is equipped with a camera 14J and a light source 14K, capable of acquiring photographic images obtained by illuminating the light source 14K and capturing them with the camera 14J, and sending them to the server 12. Additionally, an LED can be used as the light source, but other light sources besides LEDs can also be used.

[0063] With the above structure, the mobile terminal 14 according to this embodiment, through the CPU 14A, performs operations such as accessing the ROM 14B, RAM 14C, and memory 14D, acquiring various data via the operation unit 14E, and displaying various information on the display 14F. Furthermore, the mobile terminal 14, through the CPU 14A, controls the transmission and reception of communication data such as photographic images via the communication line interface unit 14G.

[0064] On the other hand, the server 12 has a database (DB) 12I that stores a pre-set reference image for determining authenticity, and performs a process of determining authenticity by comparing the photographed image received from the mobile terminal 14 with the reference image.

[0065] Next, the functional structure of the mobile terminal 14 will be explained. Figure 4 This is a functional block diagram illustrating the functional structure of the mobile terminal involved in this embodiment.

[0066] like Figure 4 As shown, the mobile terminal 14 has the following functional structures: an overexposure position detection unit 20, an overlap determination unit 22, a frame line position movement unit 24, a comparison image acquisition unit 26, a comparison request unit 28, and a comparison result display unit 30. Each functional structure is implemented by the CPU 14A reading the authenticity determination program stored in the ROM 14B or memory 14D, expanding it in the RAM 14C, and executing it.

[0067] The overexposure location detection unit 20 acquires a photographic image obtained by emitting light from the light source 14K and capturing it through the camera 14J, and extracts the area above a predetermined pixel value generated in the photographic image along with the emission of light source 14K as the overexposure portion, thereby detecting the location of the overexposure portion. Regarding the overexposure location, the location in the photographic image varies depending on the type of mobile terminal 14. That is, the position of light source 14K relative to the camera 14J of the mobile terminal 14 differs for each type, therefore the location of the overexposure portion in the photographic image varies. For example, as... Figure 5 The positions of the overexposed portion 38 differ in the mobile terminals A through C shown. Figure 5 In the example, the cross-shaped portion represents the light-emitting center of the 14K light source, and the portion surrounded by a circle outside the cross-shaped portion represents the overexposed portion 38 with a preset pixel value or higher. Therefore, the overexposed position detection unit 20 detects the position of the overexposed portion 38 in the photographic image, which varies depending on the type of mobile terminal 14. In detecting the overexposed portion 38, for example, to make the entire field of view uniform in density, an object with uniform density within a preset density range is brought close to a preset distance and photographed. If it is, for example, an 8-bit grayscale image (256 gray levels), then an area where there are n or more adjacent pixels with a pixel value of 245 or higher (for example, 100×100 or more pixels) within the field of view of the photographic image is extracted as the overexposed portion 38. In addition, when detecting the overexposed portion 38, if the object 32 has a uniform density within a preset density range, the object 32 can be brought close to the object 32 until the entire field of view becomes uniform in density before photographing.

[0068] The overlap determination unit 22 determines whether the overexposed portion 38 detected by the overexposure position detection unit 20 overlaps with a reference mark 34, which includes a reference area 34A of a reference object compared with a preset reference image. For example, it determines whether the overexposed portion 38 overlaps with a preset standard position of the frame line used for determining the image acquisition position displayed on the display 14F. Specifically, in this embodiment, as... Figure 2As shown, a frame line 36, representing the acquisition position of an image compared to a reference image, is displayed at a preset standard position on the display 14F of the mobile terminal 14. Then, when the frame line 36 is aligned with the reference mark 34 containing the reference area 34A for comparison with the object 32, and the light source 14K is illuminated to take a picture, it is determined whether the overexposed portion 38 overlaps with the frame line 36. For example, in... Figure 5 In the mobile terminals A to C shown, the positional relationship between the frame line 36 (represented by quadrilaterals) and the overexposed portion 38 (enclosed by circles) is different in each case. There are cases where the frame line 36 overlaps with the overexposed portion 38, and cases where they do not overlap. Therefore, it is necessary to determine whether they overlap. Figure 5 In the examples shown, frame line 36 overlaps with the overexposed portion 38 in mobile terminal A and mobile terminal C, while frame line 36 does not overlap with the overexposed portion 38 in mobile terminal B. Furthermore, as frame line 36, in this embodiment, as... Figure 2 As shown, an example with a non-rotationally symmetric shape is illustrated, but it can also be illustrated as follows: Figure 5 The rectangular frame 36 is shown as applicable. Alternatively, circular or polygonal frames can also be used. Furthermore, unlike circles, squares, equilateral triangles, etc., non-rotationally symmetric shapes are shapes whose orientation is uniquely determined.

[0069] When the overlap determination unit 22 determines that the overexposed portion 38 overlaps with the frame line 36, the frame line position moving unit 24 moves the frame line 36 to a range that does not overlap with the overexposed portion 38 for display on the display 14F. In this embodiment, for Figure 2 The control region 34A in the control mark 34 shown is compared with the reference image obtained by pre-capturing the control region 34A, but as Figure 5 If the frame line 36 overlaps with the overexposed portion 38 in mobile terminals A and C, it will be impossible to accurately compare with the reference image due to the overexposed portion 38. Therefore, the frame line position moving unit 24 moves the frame line 36 to a range that does not overlap with the overexposed portion 38 and displays it on the display 14F, and receives a photography instruction. For example, an example of the histogram distribution of pixel values ​​in the comparison area 34A when the comparison mark 34 is photographed by mobile terminals A to C is shown below. Figure 6 .like Figure 6 As shown, in mobile terminals A and C, the distribution of pixel values ​​0 is abundant and uneven, making accurate comparison highly unlikely. In contrast, mobile terminal B has a uniform distribution of pixel values, allowing for accurate comparison.

[0070] Regarding the movement of the frame line 36 towards a region that does not overlap with the overexposed portion 38, for example, on the display 14F displaying the frame line 36, when the origin is taken as the area directly below the center of the light source 14K appearing on the display 14F, the frame line 36 is always displayed in the same quadrant regardless of the type of mobile terminal 14. Objects with uneven surfaces or bright objects are affected by the direction of the light from the light source 14K, therefore it is necessary to move the position of the frame line 36 in a manner close to the positional relationship of the light source 14K when capturing the reference image. For example, in mobile terminals A-C used for comparison with the reference image, mobile terminal 14 used for registering the reference image, or a register consisting of a camera and illumination, the direction of illumination is always maintained by keeping the frame line 36 in the first quadrant. Furthermore, within the first quadrant, it is moved towards a position avoiding the overexposed portion, i.e., away from the origin. For example, as... Figure 7 As shown, mobile terminals A and C move frame line 36 from the dashed line position to the solid line position. In mobile terminal B, the overexposed portion 38 does not overlap with frame line 36, therefore it does not move.

[0071] Alternatively, when displaying frame line 36 on display 14F, regardless of the type of mobile terminal 14, the triangle formed by the three points—directly below the center of the lens (which becomes the center of the camera 14J's field of view), directly below the center of the light source 14K (which becomes the overexposed portion 38), and a predetermined specific portion of frame line 36—is made to have a shape approximately similar to the optical system used when capturing the reference image and to various mobile terminals 14. In other words, the triangle formed by the three points—directly below the center of the lens (which becomes the center of the camera 14J's field of view), directly below the center of the light source 14K, and a specific portion of frame line 36—is calculated to have a shape similar to that used when capturing the reference image. Thus, the positional relationship with the light source 14K and the center of the lens is maintained when capturing the reference image and when capturing the comparison image. At this time, within the quadrant where the specific portion of frame line 36, which becomes the center of the camera's field of view, exists as the origin, frame line 36 is moved in a direction away from the center of the light source 14K. For example, as... Figure 8 As shown, the position of the frame line 36 is calculated for mobile terminals A to C in such a way that the triangles formed by the three points—the center of the field of view of camera 14J, the center of the circular cross directly below the center of the light source 14K, and a specific part of the frame line 36—are of similar shape. Figure 8 In the middle, frame line 36 is also shown in a similar shape. Then, as... Figure 8 As shown, move the frame line 36 as needed to avoid overlapping with the overexposed portion 38. At this point, with the origin located directly below the center of the lens, which will become the center of the camera's field of view, as shown... Figure 9 As indicated by the arrow, the frame line 36 is moved away from the center of the light source 14K within the quadrant where a specific portion of the frame line 36 exists.

[0072] The comparison image acquisition unit 26 acquires a comparison image for comparison with a reference image by extracting an image of the area corresponding to the frame line 36 from the photographed image. Here, the comparison image acquired from the photographed image is either an image where the overexposed portion does not overlap with the comparison area in the comparison image, or an image where the overexposed portion exists outside the comparison area in the comparison image. That is, the overexposed portion 38 can be captured outside the comparison area of ​​the comparison image. Furthermore, the comparison image acquisition unit 26 can acquire the comparison image from a photographed image captured by receiving a photographic instruction, but it can also align the frame line 36 with... Figure 10 The reference mark 34 is shown. When the frame line 36 is aligned with the reference mark 34, the automatic shutter is released to take a picture, and the reference image is extracted from the photographed image. Additionally, Figure 10 The shaded line represents reference mark 34, and the solid line represents the outline of the reference mark.

[0073] As an example of automatic shutter, for instance, when the average density of y% of the pixels connected within an n×n pixel area is set to d1, and the average density of the remaining pixels is set to d2, d1 / d2 is set to m, and y and m are predetermined. For example, in 8-bit grayscale, d1 could be set to 35, d2 to 200, etc., with the d1 side being the side of the contrast mark printed in pure silver, and the d2 side being the background paper, etc. If appropriate y and m are used... Figure 10 If, for example, there are 10 rectangles on frame line 36 as shown, they are considered consistent and the shutter is released. At this point, the consistency with the markings changes depending on parameters such as rectangle size, y, and m. For example, Figure 11 The automatic shutter does not release when the upper segment is consistent, but it releases when the middle segment is consistent. Furthermore, the automatic shutter releases when the lower segment is consistent, based on a threshold setting.

[0074] Alternatively, you can use a template to trigger the automatic shutter release for shooting. For example, you can prepare a template with... Figure 12 The template corresponding to the shape of the reference mark 34 shown is automatically shuttered when the consistency between the photographed image and the template reaches a preset value or higher. For example, the outer frame can be extracted and the consistency calculated only by the frame, or the area within the frame can be uniformly coated with a specified concentration, and the template side can also be set to a uniform concentration to calculate the consistency. Then, an image with a consistency of more than the preset value is acquired as the photographed image. As in the animation, the image is kept in photographing mode throughout the reference procedure, and pattern matching calculations are performed sequentially within the mobile terminal 14. When the consistency falls within a preset threshold, the photographed image is acquired.

[0075] In addition, in this embodiment, the comparison image acquisition unit 26 performs the process of extracting the comparison image region from the photographic image, but the photographic image can also be sent to the server 12 and the process of extracting the comparison image region can be performed on the server 12 side.

[0076] The comparison request unit 28 sends the comparison image obtained by the comparison image acquisition unit 26 to the server 12, thereby requesting a comparison between the reference image and the comparison image.

[0077] The comparison result display unit 30 performs the comparison results of the reference image and the comparison image received from the server 12 and displays the comparison results on the display 14F.

[0078] On the other hand, the server 12 performs the following processing: receiving a comparison image sent from the comparison request unit 28, comparing it with a pre-registered reference image, thereby uniquely identifying the object, and replying the identification result to the mobile terminal 14. In this embodiment, for example, the following processing is performed: setting the object to the reference image... Figure 2 The reference image of the reference area 34A in the reference mark of the object 32 shown is pre-registered on the server 12. The reference image is compared with the reference image to determine the authenticity of the object, and the determination result is sent back to the mobile terminal 14.

[0079] Next, the specific processing performed by the mobile terminal 14 of the information processing system 10 according to this embodiment, configured as described above, will be explained. Figure 13 This is a flowchart illustrating an example of the processing flow performed during the preparation for capturing a comparison image when using the mobile terminal 14 of the information processing system 10 according to this embodiment. Furthermore, regarding... Figure 13 The processing, for example, can begin when the authenticity determination application is installed on the mobile terminal 14 and launched for the first time, and be omitted on subsequent launches of the application. Figure 13 The processing.

[0080] In step S100, CPU 14A issues a uniform density shooting instruction and transitions to step S102. For example, a message indicating that a uniform density surface is to be shot at close range is displayed on display 14F, thereby issuing a uniform density shooting instruction. Alternatively, as an example of uniform density, a uniform density surface can be set on a surface of the object 32 that is different from the surface where the reference mark 34 is provided for shooting instruction.

[0081] In step S102, CPU 14A determines whether a shot has been taken. This determination includes checking whether the shutter release operation of camera 14J has been performed on operation unit 14E. The process continues until the determination is affirmative, then proceeds to step S104.

[0082] In step S104, CPU14A illuminates light source 14K to perform shooting based on camera 14J and transitions to step S106.

[0083] In step S106, CPU14A acquires the photographic image captured by camera 14J and proceeds to step S108.

[0084] In step S108, CPU 14A detects the overexposed portion 38 and proceeds to step S110. That is, the overexposed position detection unit 20 extracts the area above a preset pixel value generated in the photographic image due to the emission of the light source 14K as the overexposed portion 38, thereby detecting the position of the overexposed portion 38.

[0085] In step S110, CPU 14A determines the position of frame line 36 and ends a series of photographic preparation processes. Specifically, the overlap determination unit 22 determines whether the overexposed portion 38 detected by the overexposure position detection unit 20 overlaps with the reference area of ​​a reference object compared to a preset reference image. For example, it determines whether the overexposed portion 38 overlaps with a preset standard position of the frame line 36 used for determining the image acquisition position displayed on the monitor 14F. When the overexposed portion 38 does not overlap with the frame line 36, the standard position is determined as the position of the frame line 36. On the other hand, when the overexposed portion 38 overlaps with the frame line 36, the frame line position moving unit 24 moves the frame line 36 to a range that does not overlap with the overexposed portion 38 for display on the monitor 14F. The determined position of the frame line 36 is stored in memory 14D. Upon subsequent startups of the application, the position of the frame line 36 stored in memory 14D can be used for further explanation. Figure 14 The processing.

[0086] Next, regarding Figure 13 The process of preparing for photography and then processing the comparison images during the shooting process will be explained. Figure 14 This is a flowchart illustrating an example of the processing flow when capturing comparison images using the mobile terminal 14 of the information processing system 10 according to this embodiment. Additionally, Figure 14 The processing can be done Figure 13 It begins after processing. Alternatively, it can be set as follows: when the application starts for the first time, in Figure 13 After the processing begins, on the second launch of the application, from Figure 14 The processing begins by utilizing the position of the frame line 36 when the application is first launched. Figure 14 The processing.

[0087] In step S200, CPU 14A displays frame line 36 on display 14F and transitions to step S202. That is, frame line 36 is displayed at the position determined in step S110 above. Thus, the reference mark 34 is aligned with frame line 36 to capture a reference image, thereby avoiding the overexposed portion 38.

[0088] In step S202, the CPU 14A determines whether to take a picture. This determination can be made by checking whether a photography instruction has been received via the operation unit 14E. Alternatively, the comparison image acquisition unit 26 can determine whether the consistency between the comparison mark 34 and the frame line 36 is greater than or equal to a preset value. The process continues until the determination is affirmative, then proceeds to step S204.

[0089] In step S204, CPU14A acquires the photographic image captured by camera 14J and proceeds to step S206.

[0090] In step S206, CPU14A extracts an image of the region corresponding to the frame line 36 and proceeds to step S208. That is, the comparison image acquisition unit 26 extracts an image of the region corresponding to the frame line 36 from the photographic image as a comparison image, and acquires a comparison image for comparison with a reference image.

[0091] In step S208, CPU 14A sends a comparison image to server 12 to make a comparison request and transitions to step S210. That is, the comparison request unit 28 sends the comparison image obtained by the comparison image acquisition unit 26 to server 12, thereby requesting a comparison between the reference image and the comparison image.

[0092] In step S210, CPU14A is in standby mode until it receives the comparison result from server 12 and transitions to step S212.

[0093] In step S212, the CPU 14A displays the comparison results on the display 14F and ends a series of processes. That is, the comparison result display unit 30 performs the process of receiving the reference image and the comparison image from the server 12 and displaying the comparison results on the display 14F. As a result, the authenticity determination result of the object 32 is displayed on the display 14F.

[0094] Furthermore, in the above embodiment, it is set that in step S102, it is determined whether the shutter of the camera 14J has been released by the operation unit 14E, but the processing of step S102 can also be omitted. That is, it can also be set up as follows: during the display of the live view image when taking a photographic image, the overexposed portion 38 is extracted, and in the area that does not overlap with the overexposed portion 38, a frame line 36, which serves as position information indicating the acquisition position of the image compared with the reference image, is displayed in the live view image, and a shooting instruction is received.

[0095] Furthermore, in the above embodiments, an example was described whereby an image of the reference area 34A of the reference mark 34 is extracted by displaying the frame line 36 on the display 14F of the mobile terminal 14 and aligning the frame line 36 with the reference mark 34, and then compared with a reference image in the server 12. However, this is not a limitation. For example, the display of the frame line 36 may be omitted, and the reference image may be used instead. Figure 15 The image, indicated by a shaded line, is a pre-defined area that has been photographed and registered as a reference image. A frame is set on the object 32 to identify the pre-defined area, and the mobile terminal 14 is used to capture the image by aligning it with the frame. Then, the image can also be identified from the photographic image. Figure 15 The image shown is a box indicated by a shaded line. Within the image of the identified box, an image of the area avoiding the overexposed portion 38 is obtained as a reference image. This reference image, along with its position relative to the box, is sent to server 12. Server 12 can then extract a reference image corresponding to the position of the reference image relative to the box and compare it with the reference image to determine authenticity. At this time, by logging into server 12... Figure 15 The area shown in the shaded area is set to be sufficiently large compared to the size of the overexposed portion 38, so that the area that avoids the overexposed portion 38 can be set as the reference image.

[0096] Furthermore, while the above embodiment describes an example of capturing a comparison image using the mobile terminal 14 and requesting a comparison from the server 12, it is not limited to this. For example, it could also be configured to capture a comparison image using the camera 14, read the captured comparison image on an information processing device such as a personal computer, and then request a comparison from the server 12. Alternatively, it could be configured to omit the server 12, pre-register a reference image on the mobile terminal 14, and use the mobile terminal 14 for comparison, thus consisting solely of the mobile terminal 14.

[0097] Furthermore, in the above embodiments, the CPU was described as an example of a processor, but the processor refers to a processor in a broad sense, including general-purpose processors (such as CPU: Central Processing Unit, etc.) or special-purpose processors (such as GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0098] Furthermore, the actions of the processor in the above embodiments can be performed not only by a single processor, but also by a plurality of processors located in physically separate positions working together. Moreover, the order of the processor's actions is not limited to the order described in the above embodiments and can be appropriately modified.

[0099] Furthermore, the processing performed using the mobile terminal 14 in the information processing system 10 according to the above embodiments can be software-based, hardware-based, or a combination of both. Moreover, the processing performed using the mobile terminal 14 can also be stored as a program on a storage medium and circulated.

[0100] Furthermore, the present invention is not limited to the above content. It is natural that various modifications can be made without departing from its spirit.

[0101] The embodiments of the present invention described above are provided for illustrative purposes. Furthermore, these embodiments do not encompass the entirety of the invention, nor do they limit the invention to the disclosed methods. It will be apparent to those skilled in the art that various modifications and variations will be readily understood. These embodiments were chosen and described to most readily explain the principles and applications of the invention. Thus, those skilled in the art can understand the invention through various modifications that are assumed to optimize specific uses of various embodiments. The scope of the invention is defined by the foregoing claims and their equivalents.

Claims

1. A photographic processing apparatus comprising a processor, the processor performing the following processing: To acquire photographic images by illuminating a light source and taking a picture. During the display of the live view image while the photographic image is being captured, the overexposed portion of the photographic image that exceeds a preset pixel value due to the emission of the light source is extracted. In the photographed image, the area located in the range that does not overlap with the extracted overexposed portion is set as the area of ​​the comparison object to be compared with a pre-set reference image, and in the area that does not overlap with the overexposed portion, the position information of the acquisition position of the image compared with the reference image is displayed in the live view image.

2. The photographic processing apparatus according to claim 1, wherein, The processor performs the following processing: acquiring the photographic image taken by approaching an object within a preset concentration range to a preset distance, and extracting the overexposed portion.

3. The photographic processing apparatus according to claim 1, wherein, The process of displaying the location information of the image acquired, which indicates the location to be compared with the reference image, in the live view image includes: extracting the overexposed portion from the photographic image taken after receiving photographic instructions and displaying the location information of the image acquired, which indicates the location to be compared with the reference image, at a preset standard position. When the overexposed portion overlaps with the position information, the position information is moved to an area that does not overlap with the overexposed portion for display.

4. The photographic processing apparatus according to claim 1, wherein, The processor performs the following processing: displays a frame with a non-rotationally symmetric shape as the position information.

5. The photographic processing apparatus according to claim 1, wherein, The processor performs the following processing: when the center of the light source is taken as the origin of the photographic image, the position information is displayed in the same quadrant as when the reference image was captured.

6. The photographic processing apparatus according to claim 5, wherein, The processor performs the following processing: displays the position information at a position that does not overlap with the overexposed portion after moving away from the origin.

7. The photographic processing apparatus according to claim 1, wherein, The processor performs the following processing: when the photographic field of view is taken as the origin, the triangle formed by the three points—the center of the photographic field of view, the center of the overexposed portion, and the pre-set specific position of the position information—becomes a quadrant with a shape similar to that when the reference image was captured, and displays the position information.

8. The photographic processing apparatus according to claim 7, wherein, The processor performs the following processing: within the quadrant where the position information exists, displays the position information at a position that does not overlap with the overexposed portion after moving in a direction away from the overexposed portion.

9. The photographic processing apparatus according to any one of claims 1 to 8, wherein, The processor further performs the following processing: displays an image of the reference object cropped from the photographic image when the object information representing the area of ​​the reference object set on the object is matched with the location information, and compares it with the reference image.

10. A photographic processing system, comprising: The photographic processing apparatus according to any one of claims 1 to 9; and The server compares a comparison image obtained by cropping the region of the comparison object from the photographic image with the reference image and sends the comparison result to the photographic processing device.

11. A storage medium storing a photographic processing program that causes a computer to perform the following processing: To acquire photographic images by illuminating a light source and taking a picture. During the display of the live view image while the photographic image is being captured, the overexposed portion of the photographic image that exceeds a preset pixel value due to the emission of the light source is extracted. In the photographed image, the area located in the range that does not overlap with the extracted overexposed portion is set as the area of ​​the comparison object to be compared with a pre-set reference image, and in the area that does not overlap with the overexposed portion, the position information of the acquisition position of the image compared with the reference image is displayed in the live view image.

12. A photographic processing method, comprising the following steps: To acquire photographic images by illuminating a light source; During the display of the live view image while the photographic image is being captured, the overexposed portion of the photographic image exceeding a preset pixel value caused by the emission of the light source is extracted; and In the photographed image, the area located in the range that does not overlap with the extracted overexposed portion is set as the area of ​​the comparison object to be compared with a pre-set reference image, and in the area that does not overlap with the overexposed portion, the position information of the acquisition position of the image compared with the reference image is displayed in the live view image.

13. A computer program product comprising a computer program for enabling a computer to perform the following functions: To acquire photographic images by illuminating a light source and taking a picture. During the display of the live view image while the photographic image is being captured, the overexposed portion of the photographic image that exceeds a preset pixel value due to the emission of the light source is extracted. In the photographed image, the area located in the range that does not overlap with the extracted overexposed portion is set as the area of ​​the comparison object to be compared with a pre-set reference image, and in the area that does not overlap with the overexposed portion, the position information of the acquisition position of the image compared with the reference image is displayed in the live view image.

Images