Image determination device and program

Abstract

To provide an image determination device and a program having a clear criterion for determining whether a shadow is generated in a face image.SOLUTION: An image determination device 1 includes a face region extraction unit 12 that extracts a face region from a face image of a user which is an evaluation target image, a region division processing unit 14 that divides the face region extracted by the face region extraction unit 12 into left and right regions from a center of a face to obtain the left and right regions, a lightness change value calculation unit 15 that calculates lightness change values of the left and right regions obtained by the region division processing unit 14, a similarity calculation unit 16 that calculates similarity between the lightness change values of the left and right regions calculated by the lightness change value calculation unit 15, and a shadow determination processing unit 17 that compares the similarity calculated by the similarity calculation unit 16 with a threshold value to determine presence or absence of a shadow in the face region.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image assessment device and a program. [Background technology]

[0002] Conventionally, passport photo machines have been used to take facial photographs for use in employee ID cards, membership cards, My Number cards, etc. Passport photo machines are equipped with lighting devices in front, above, or behind the face of the person being photographed, allowing the face to be photographed with the subject's face properly illuminated. As a result, facial images taken with passport photo machines are accurately lit with no shadows on the face, and therefore meet the desired photographic quality.

[0003] As a technique for determining suitability of a photo for identification, for example, a technique for determining whether the eyes are closed, and also a technique for determining the direction of the face, the density, contrast, focus, etc. of the face area has been disclosed (for example, Patent Document 1). Furthermore, as a method for discriminating images for identification photographs, a technique has been disclosed in which discrimination is made based on the orientation of the face, the proportion of the area of ​​the face image to the entire original image, the color tone of the background image, etc. (for example, Patent Document 2). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 4840662 [Patent Document 2] Patent No. 4645952 Summary of the Invention [Problem to be solved by the invention]

[0005] Currently, users use devices such as smartphones to capture images for facial photographs to be used on employee ID cards, My Number cards, etc. When using devices such as smartphones, shadows can appear on the subject's face depending on the shooting environment, and such facial images do not meet the desired photographic quality. On the other hand, in Patent Documents 1 and 2, no determination is made as to whether or not a shadow is cast on the subject's face. When evaluating whether a face image has a shadow, human evaluation results vary depending on the evaluator. While it is possible to use artificial intelligence (AI), the basis for such a judgment is unclear.

[0006] Therefore, an object of the present invention is to provide an image assessment device and program that provide clear criteria for determining whether or not a shadow is present in a facial image. [Means for solving the problem]

[0007] The present invention solves the above problems by the following means. A first invention is an image assessment device comprising: a face area extraction means for extracting a face area from an image to be evaluated; an area division means for dividing the face area extracted by the face area extraction means into left and right areas from the center of the face to obtain a left area and a right area; a brightness change value calculation means for calculating a brightness change value of each of the left area and the right area obtained by the area division means; a similarity calculation means for calculating a similarity between the brightness change value of the left area calculated by the brightness change value calculation means and the brightness change value of the right area; and a shadow assessment means for assessing the presence or absence of a shadow in the face area based on the similarity calculated by the similarity calculation means. A second aspect of the present invention is the image assessment device according to the first aspect of the present invention, wherein the lightness change value calculation means calculates the lightness change value using lightness in a color space of a color system. A third invention is an image assessment device according to the first or second invention, wherein the similarity calculation means calculates the similarity using each brightness change value at a position symmetrical to the center of the face. A fourth invention is an image assessment device in which, in any of the first to third inventions, the brightness change value calculation means calculates the brightness change value for the vertical and horizontal directions of the face in each of the left and right regions obtained by the region division means, and the similarity calculation means calculates the similarity between the brightness change value of the left region and the brightness change value of the right region for each of the vertical and horizontal directions. The fifth invention is an image assessment device that is any of the first to fourth inventions, and is provided with a partial area acquisition means that acquires partial areas that are positioned symmetrically from the center of the face for each of the left area and the right area obtained by the area division means, and the brightness change value calculation means calculates the brightness change value of each of the partial areas for the left area and the right area acquired by the partial area acquisition means. A sixth invention is the image assessment device according to the fifth invention, wherein the partial region is a part that constitutes the face region. A seventh invention is the image assessment device of the sixth invention, wherein the part is a forehead or a cheek of a face. An eighth invention is an image assessment device according to any one of the first to fourth inventions, comprising: a hair recognition means for recognizing a hair portion from the face region extracted by the face region extraction means; and a partial correction means for adding the region of the hair portion recognized by the hair recognition means so that it is symmetrical left and right from the center of the face, and deleting the region of the hair portion and the added region from the image of the face region, wherein the region division means obtains the left region and the right region from the face region after correction by the partial correction means. A ninth invention is an image assessment device according to any one of the first to fourth inventions, further comprising a hair recognition means for recognizing a hair portion from the face region extracted by the face region extraction means, and the similarity calculation means calculates the similarity by excluding the lightness change value of the hair portion recognized by the hair recognition means and the lightness change value of a portion located symmetrically to the hair portion from the center of the face from the lightness change value of the left region and the lightness change value of the right region calculated by the lightness change value calculation means. A tenth invention is an image assessment device in which, in any of the first to ninth inventions, the shadow assessment means compares the similarity calculated by the similarity calculation means with a threshold value to determine whether or not a shadow exists in the face area. The 11th invention is an image assessment device according to the 10th invention, wherein the facial area extraction means extracts the facial area from an identification photo image taken using an identification photo machine, and includes a similarity registration means for storing the similarity calculated by the similarity calculation means in a similarity memory unit, and a threshold determination means for determining the threshold from the similarity stored in the similarity memory unit. A twelfth aspect of the present invention is a program for causing a computer to function as any one of the image assessment devices according to the first to eleventh aspects of the present invention. [Effects of the Invention]

[0008] According to the present invention, it is possible to provide an image assessment device and a program that provide clear criteria for determining whether or not a shadow is present in a face image. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing the overall configuration of an image processing system according to a first embodiment and functional blocks of an image assessment device. [Figure 2] 4 is a flowchart showing a threshold value determination process of the image assessment device according to the first embodiment. [Figure 3] 4 is a flowchart showing a similarity calculation process of the image assessment device according to the first embodiment. [Figure 4]3A to 3C are diagrams for explaining a similarity calculation process in the image assessment device according to the first embodiment. [Figure 5] 3A to 3C are diagrams for explaining a similarity calculation process in the image assessment device according to the first embodiment. [Figure 6] 3A to 3C are diagrams for explaining a similarity calculation process in the image assessment device according to the first embodiment. [Figure 7] 3A to 3C are diagrams for explaining a similarity calculation process in the image assessment device according to the first embodiment. [Figure 8] 3A to 3C are diagrams for explaining a similarity calculation process in the image assessment device according to the first embodiment. [Figure 9] 4 is a flowchart showing a shadow determination process of the image determination device according to the first embodiment. [Figure 10] FIG. 10 is a diagram showing functional blocks of an image assessment device according to a second embodiment. [Figure 11] 10 is a flowchart showing a similarity calculation process of the image assessment device according to the second embodiment. [Figure 12] FIG. 10 is a diagram for explaining a similarity calculation process in the image assessment device according to the second embodiment. [Figure 13] FIG. 10 is a diagram showing functional blocks of an image assessment device according to a third embodiment. [Figure 14] 11 is a flowchart showing a similarity calculation process of the image assessment device according to the third embodiment. [Figure 15] 10A and 10B are diagrams for explaining a similarity calculation process in the image assessment device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, this is merely an example, and the technical scope of the present invention is not limited to this example. (First embodiment) <Image processing system 100> FIG. 1 is a diagram showing the overall configuration of an image processing system 100 and functional blocks of an image assessment device 1 according to the first embodiment.

[0011] 1, the image processing system 100 includes an image assessment device 1, a user terminal 4, a photo ID machine 5, and an image DB (database) server 6. The image assessment device 1, the user terminal 4, the photo ID machine 5, and the image DB server 6 are communicatively connected via a communication network N. The image processing system 100 is operated by a company that modifies and saves facial images (images to be evaluated) taken by users with user terminals 4 such as smartphones into a standardized format, and uses the saved facial images to provide various services for various purposes.

[0012] In the image processing system 100, an image of an ID photo (ID photo image) acquired by the ID photo machine 5 is stored in the image DB server 6. In addition, in the image processing system 100, a face image taken by the user terminal 4 is stored in the image DB server 6 on the condition that the face image satisfies the desired photographic quality. In this way, the image processing system 100 makes it possible to use the face images stored in the image DB server 6 for various purposes. Examples of various uses include issuing ID (identification) certificates with face photographs and using face images for face authentication when passing through security gates.

[0013] <Image assessment device 1> The image assessment device 1 is a device for assessing the presence or absence of a shadow in a facial area of ​​a facial image received from a user terminal 4. The image assessment device 1 can be realized by, for example, a server. Alternatively, the image assessment device 1 may be a general personal computer (PC) or the like. The image assessment device 1 includes a control unit 10, a storage unit 30, and a communication interface unit 39.

[0014] The control unit 10 is a central processing unit (CPU) that controls the entire image assessment device 1. The control unit 10 appropriately reads and executes the operating system (OS) and application programs stored in the storage unit 30, thereby cooperating with the above-mentioned hardware and executing various functions. Before describing the control unit 10 in detail, the storage unit 30 will be described. The storage unit 30 is a storage area such as a hard disk, a semiconductor memory element, etc. for storing programs, data, etc. required for the control unit 10 (described later) to execute various processes. The storage unit 30 includes a program storage unit 31 , a similarity storage unit 32 , and a threshold storage unit 33 .

[0015] The program storage unit 31 is a storage area for storing various programs. The program storage unit 31 stores a similarity calculation program 31a, a judgment processing program 31b, and a threshold determination program 31c. These programs are programs for performing various functions executed by the control unit 10 of the image judgment device 1. More specifically, the similarity calculation program 31a is a program for executing the functions of the image reception unit 11 to the similarity calculation unit 16 of the control unit 10, which will be described later. The judgment processing program 31b is a program for executing the function of the shadow judgment processing unit 17, which will be described later. The threshold determination program 31c is a program for executing the function of the ID photo processing unit 18, which will be described later. In the following explanation, an example including a similarity calculation program 31a, a judgment processing program 31b, and a threshold determination program 31c will be described, but the present invention is not limited to these three programs. There may be one program that fulfills all the functions, or two programs for different purposes.

[0016] The similarity storage unit 32 is a storage area for storing similarities in brightness between the left and right faces of a facial image stored in the image DB server 6. The similarity here refers to the similarity of a graph showing brightness change values ​​between the left and right faces of a facial image. This similarity is calculated, for example, using DTW (Dynamic Time Warping), which will be described later. Furthermore, the similarities stored in the similarity storage unit 32 are for facial images stored in the image DB server 6, and are therefore calculated from facial images that satisfy the quality of a target photograph. The threshold storage unit 33 is a storage area for storing a threshold. The threshold is determined from the similarity stored in the similarity storage unit 32. As described above, the similarity storage unit 32 stores similarities calculated from face images that satisfy the quality of a target photograph. Therefore, the threshold can be used as a clear criterion for judgment.

[0017] Next, the control unit 10 will be described. The control unit 10 includes an image receiving unit 11, a facial area extraction unit 12 (facial area extraction means), a hair recognition processing unit 13 (hair recognition means, partial correction means), an area division processing unit 14 (area division means), a brightness change value calculation unit 15 (brightness change value calculation means), a similarity calculation unit 16 (similarity calculation means), a shadow determination processing unit 17 (shadow determination means), and an ID photo processing unit 18 (similarity registration means, threshold determination means).

[0018] The image receiving unit 11 receives and accepts face images from the user terminal 4. The face images received from the user terminal 4 are face images of the user's upper body, generally from the shoulders up, and are images taken by the user using the user terminal 4. The face images are stored in the image DB server 6 after, for example, confirming that they meet the desired photograph quality. Furthermore, the image receiving unit 11 receives an image of a photo for identification from the image DB server 6. The image of a photo for identification that the image receiving unit 11 receives from the image DB server 6 satisfies the desired photographic quality.

[0019] The face region extraction unit 12 extracts a face region from the face image or ID photo image received by the image receiving unit 11. The face region extraction unit 12 can extract a face region from the face image or ID photo image using various known techniques. As an example, the face region extraction unit 12 can extract a face region by using a function of MediaPipe, a library led by Google, that acquires a shape from a face image.

[0020] The hair recognition processing unit 13 recognizes hair portions from the face region extracted by the face region extraction unit 12. The hair recognition processing unit 13 can be realized using various known technologies to recognize hair portions from the face region. As an example, the hair recognition processing unit 13 can recognize hair portions by using a technology capable of recognizing each part of the head using semantic segmentation or the like. Here, semantic segmentation is an image recognition technology that labels each pixel (picture element) of an image. The hair recognition processing unit 13 then adds the recognized hair portion region symmetrically from the center of the face to the left and right, and deletes the hair portion region and the added region from the face region. By processing by this hair recognition processing unit 13, the hair region is excluded from the face region, and moreover, the exclusion is performed bilaterally symmetrically, so that asymmetric hair portions do not need to be taken into consideration in subsequent processing. The area division processing unit 14 divides the face area corrected by the hair recognition processing unit 13 into left and right areas from the center of the face to obtain a left area and a right area.

[0021] The lightness change value calculation unit 15 calculates the lightness change value of each of the left and right regions obtained by the region division processing unit 14. The lightness change value calculation unit 15 calculates the lightness change value using the lightness in the color space of the color system. As an example, the lightness change value calculation unit 15 calculates the lightness change value using the L * The CIELab color space was standardized by the International Commission on Illumination (CIE) in 1976 and is also adopted in Japan as JIS (JIS Z 8781-4). It is a color system that can express the color of an object in a way that is closer to human perception. *The value represents the brightness, with 0 being black and 100 being white. Here, to evaluate how the light hits the object, L * The degree of change in value is used. As another example, the lightness change value calculation unit 15 may calculate the value L * C * L in h color space * The lightness change value may be calculated using a value representing the lightness in the Hunter Lab color space, RGB color space, HSB color space, HSV color space, HSL color space, OKLCH color space, etc., without being limited to the display system color spaces exemplified above. The brightness change value calculation unit 15 calculates brightness change values ​​in the vertical and horizontal directions of the face in each of the left and right regions obtained by the region division processing unit 14.

[0022] The similarity calculation unit 16 calculates the similarity between the brightness change value of the left region and the brightness change value of the right region calculated by the brightness change value calculation unit 15. In doing so, the similarity calculation unit 16 calculates the similarity using each brightness change value at a position symmetrical about the center of the face. Furthermore, the similarity calculation unit 16 calculates the similarity between the brightness change value of the left region and the brightness change value of the right region for each of the vertical and horizontal directions of the face. The shadow determination processing unit 17 determines whether or not there is a shadow in the face area based on the similarity calculated by the similarity calculation unit 16. The shadow determination processing unit 17 compares the similarity calculated by the similarity calculation unit 16 with a threshold stored in the threshold storage unit 33 to determine whether or not there is a shadow in the face area.

[0023] The ID photo processing unit 18 stores the similarity calculated based on the ID photo image accepted by the image accepting unit 11 in the similarity storage unit 32. The ID photo processing unit 18 also determines a threshold value from the similarity stored in the similarity storage unit 32. The communication interface unit 39 is an interface for communicating between the user terminal 4, the photo ID machine 5, and the image DB server 6 via the communication network N. Here, a computer refers to an information processing device equipped with a control unit, a storage device, etc., and the image assessment device 1 is an information processing device equipped with a control unit 10, a storage unit 30, etc., and is included in the concept of a computer.

[0024] <User terminal 4> The user terminal 4 in FIG. 1 is a terminal used by a user, and is, for example, a mobile information terminal such as a smartphone. The user terminal 4 may also be a tablet terminal, a personal computer (PC), or the like. For example, the user takes a picture of their own face using a camera and sends it to the image assessment device 1 to use the services provided by the image processing system 100. Although not shown, the user terminal 4 includes a control unit, a storage unit, a camera, a touch panel display (input unit and output unit), a communication interface unit, and the like.

[0025] <Passport Photo Machine 5> The ID photo machine 5 is a device that has the function of taking ID photographs. The ID photo machine 5 is installed in places where people gather, such as in front of a station or a shopping center. Although not shown, the ID photo machine 5 includes a control unit, a memory unit, a camera, a touch panel display (input unit and output unit), a lighting unit, a printer unit, a fee payment unit, a communication interface unit, etc.

[0026] <Image DB Server 6> The image DB server 6 is a server having a DB for storing face images that meet the desired photographic quality, including images for ID photos. The image DB server 6 includes a control unit, a memory unit, a DB (memory unit), a communication interface unit, etc., all of which are not shown. There is no limit to the number of pieces of hardware that constitute the image DB server 6. The image DB server 6 may be, for example, a cloud.

[0027] The communication network N is a network between the image assessment device 1, the user terminal 4, the ID photo machine 5, and the image DB server 6, and is, for example, a communication network such as the Internet. The communication network N may include a LAN (Local Area Network) or the like. Furthermore, the communication network N may be wired or wireless.

[0028] <Processing Description> Next, a process using the image assessment device 1 will be described. First, a process for determining a threshold value used to determine whether or not a shadow exists in an image will be described. FIG. 2 is a flowchart showing the threshold determination process of the image assessment device 1 according to the first embodiment. FIG. 3 is a flowchart showing the similarity calculation process of the image assessment device 1 according to the first embodiment. 4 to 8 are diagrams for explaining the similarity calculation process in the image assessment device 1 according to the first embodiment.

[0029] The threshold determination process of the image assessment device 1 is a process that is executed before the shadow determination process described below. The threshold determination process of the image assessment device 1 may be executed, for example, when the number of unprocessed ID photo images registered in the image DB server 6 reaches a predetermined number, or may be executed at regular intervals such as once a month. In step S (hereinafter, "step S" will be simply referred to as "S") 11 of FIG. 2, the control unit 10 (image receiving unit 11) of the image assessment device 1 acquires an unprocessed image of a passport photograph from the image DB server 6. In S12, the control unit 10 performs a similarity calculation process.

[0030] Here, the similarity calculation process will be described with reference to FIG. In S21 of FIG. 3, the control unit 10 (face area extraction unit 12) extracts a face area from the acquired image (image to be evaluated or photo ID image). When the image receiving unit 11 receives, for example, an image 41 shown in FIG. 4(A), the face area extracting unit 12 extracts a face area 42a from an image 42 shown in FIG. 4(B). In S22 of FIG. 3, the control unit 10 (hair recognition processing unit 13) recognizes the hair portion. 5A has a hair image 45a (hair portion) in a left partial region 45L. On the other hand, the image 45 has no hair image in a right partial region 45R, which is positioned symmetrically from the center of the face to the left partial region 45L. The hair recognition processing unit 13 recognizes the hair image 45a.

[0031] In S23 of FIG. 3, the control unit 10 (hair recognition processing unit 13) performs correction processing based on the recognized hair portion. Image 46 shown in Fig. 5(B) is an image after correction processing has been performed on image 45. In image 46, a portion including hair image 45a in left partial region 46L has been excluded from the face region. In addition, in image 46, a portion 46a located symmetrically from the center of the face to the portion including hair image 45a in right partial region 46R has been excluded from the face region.

[0032] In S24 of FIG. 3, the control unit 10 (area division processing unit 14) divides the corrected face area into left and right. The divided areas 43L and 43R shown in Fig. 4(C) are images obtained by dividing the face area 42a shown in Fig. 4(B) into left and right halves from the center of the face. The face area 42a shown in Fig. 4(B) does not need to recognize the hair, and the divided areas 43L and 43R shown in Fig. 4(C) are obtained by performing the process of S24 after the process of S21. Further, divided areas 47L and 47R shown in FIG. 5(B) are images obtained by dividing image 46, in which the hair portion is excluded from the face area, into left and right areas from the center of the face.

[0033] In S25, the control unit 10 (lightness change value calculation unit 15) calculates a lightness change value for each of the left and right regions. A method for calculating the brightness change value will be described with reference to FIG. For the divided region 43L, which is the region on the left side of the face region shown in FIG. 6, the brightness change value calculation unit 15 calculates a brightness region set 51. The brightness region set 51 is, for example, a set of L values ​​(brightness regions) for each pixel value of the divided region 43L. *Only a portion of the brightness region set 51 is shown in this figure. The number of pixel values ​​in the brightness region set 51 depends on, for example, the image size of the face image.

[0034] Next, brightness change value calculation unit 15 calculates average pixel value 52, which is the average value of the pixel values ​​in each row, and calculates average pixel value 53, which is the average value of the pixel values ​​in each column. Then, brightness change value calculation unit 15 creates brightness graph 54, which graphs average pixel values ​​52 in order from top to bottom, and creates brightness graph 55, which graphs average pixel values ​​53 in order from the outside of the face to the inside. Brightness graph 54 and brightness graph 55 are brightness change values. In this way, the brightness change value calculation unit 15 calculates the brightness change value using the average pixel value, so the processing load is reduced. Although not shown, the brightness change value calculation unit 15 also performs similar processing on the divided area 43R (see FIG. 4C), which is the area to the right of the face area, to create a brightness graph.

[0035] In S26 of FIG. 3, the control unit 10 (similarity calculation unit 16) calculates the similarity of the lightness change value between the left region and the right region. The similarity will be explained based on FIG. FIG. 7A shows an example in which the similarity of the brightness change value between the divided region 61L, which is the left region, and the divided region 61R, which is the right region, is calculated in the lateral direction of the face. Brightness graph 62 shows the brightness change value in the horizontal direction of divided region 61L. Brightness graph 63 shows the brightness change value in the horizontal direction of divided region 61R. Here, the arrows in divided regions 61L and 61R indicate the direction in which the graphs showing the brightness change values ​​are created, and in this example, the graphs show the brightness change values ​​in order from the outside to the inside of the face. The similarity calculation unit 16 calculates the similarity between the brightness graph 62 and the brightness graph 63 using DTW (Dynamic Time Warping) as an example of a method for calculating similarity between graphs. Similarity 66 indicates the similarity between the brightness graph 62 and the brightness graph 63. The smaller the similarity value calculated using DTW, the more similar the two graphs are.

[0036] On the other hand, FIG. 7B shows an example in which the similarity of the brightness change value between the divided region 61L, which is the left region, and the divided region 61R, which is the right region, is calculated in the vertical direction of the face. The brightness graph 64 shows the brightness change value in the vertical direction of the divided region 61L. The brightness graph 65 shows the brightness change value in the vertical direction of the divided region 61R. The similarity calculation unit 16 uses DTW to calculate the similarity between the brightness graph 64 and the brightness graph 65. The similarity 67 indicates the similarity between the brightness graph 64 and the brightness graph 65.

[0037] Here, the DTW used to calculate the similarity will be described with reference to FIG. DTW is a method used to measure the distance between time series data. Here, we will explain it by comparing DTW with Euclidean distance, which is one method for calculating the distance between waveforms. FIG. 8(A) shows three graphs g1 to g3. Figure 8(B) shows the Euclidean distance between graph g1 and graph g2, and the Euclidean distance between graph g1 and graph g3. This shows that the relationship between graph g1 and graph g3 is more similar than the relationship between graph g1 and graph g2, which is counterintuitive.

[0038] On the other hand, Figure 8(C) shows the DTW between graphs g1 and g2 and the DTW between graphs g1 and g3. This shows that the relationship between graphs g1 and g2 is more similar than the relationship between graphs g1 and g3, which is in line with our intuition. In this way, DTW calculates the distance (absolute value of error) of each point in two time series in a brute-force manner, and after calculating all of them, finds the path that connects the two time series in the shortest way. Therefore, it is possible to calculate the similarity even if the time series have different lengths or periods. In this example, the similarity is calculated using the DTW, but the present invention is not limited to this, and other methods for calculating the similarity in graphs may also be used.

[0039] In S13 of FIG. 2, the control unit 10 (identification photograph processing unit 18) registers the calculated similarity in the similarity storage unit 32. In S14, the control unit 10 (identification photo processing unit 18) determines the threshold value from the similarities stored in the similarity storage unit 32. Here, the similarities stored in the similarity storage unit 32 are obtained from images that satisfy the quality of the target photograph. Therefore, the identification photo processing unit 18 may, for example, determine the largest similarity value stored in the similarity storage unit 32 as the threshold value. Alternatively, the identification photo processing unit 18 may, for example, determine the largest similarity value within an arbitrary range (for example, a range of 90%) among the similarities stored in the similarity storage unit 32 as the threshold value. In S15, the control unit 10 (identification photo processing unit 18) registers the determined threshold value in the threshold value storage unit 33. After that, the control unit 10 ends this process.

[0040] Next, a process for determining whether or not an image has a shadow will be described. FIG. 9 is a flowchart showing the shadow determination process of the image determination device 1 according to the first embodiment. The shadow determination process of the image determination device 1 is a process that is performed every time a face image (image to be evaluated) is received from the user terminal 4, for example. In S31 of FIG. 9, the control unit 10 (image receiving unit 11) of the image assessment device 1 acquires a face image from the user terminal 4. The process of S32 is the process described above and shown in Fig. 3. The control unit 10 performs the process shown in Fig. 3 on the face image acquired from the user terminal 4.

[0041] In S33, the control unit 10 (shadow determination processing unit 17) performs shadow determination processing based on the calculated similarity. More specifically, the shadow determination processing unit 17 determines whether or not a shadow exists based on whether or not the similarity calculated by the similarity calculation unit 16 is equal to or less than a threshold stored in the threshold storage unit 33. If the calculated similarity is equal to or less than the threshold stored in the threshold storage unit 33, the shadow determination processing unit 17 determines that no shadow exists, and if the calculated similarity exceeds the threshold stored in the threshold storage unit 33, the shadow determination processing unit 17 determines that a shadow exists.

[0042] In S34, the control unit 10 (shadow determination processing unit 17) determines whether or not a shadow is present. If it is determined that a shadow is present, the shadow determination processing unit 17 proceeds to S35. On the other hand, if it is determined that no shadow is present (S34: NO), the shadow determination processing unit 17 proceeds to S36. In S35, the control unit 10 transmits, for example, a request for re-photographing to the user terminal 4, and ends this process. On the other hand, in S36, the control unit 10 registers the calculated similarity in the similarity storage unit 32. In S37, the control unit 10 registers the facial image by transmitting it to the image DB server 6. After that, the control unit 10 ends this process.

[0043] The process of S35 may be to send a message to the user terminal 4 to notify the user of the error. The process of S36 is optional. Furthermore, before processing S37, the control unit 10 may determine whether the facial image satisfies the target photograph quality from another perspective, and then transmit the facial image to the image DB server 6.

[0044] As described above, the image assessment device 1 of the first embodiment has the following advantages. (1) A face region is extracted from a face image, and the extracted face region is divided into left and right regions from the center of the face. The brightness change value of each of the left and right regions is calculated, and the similarity between the calculated brightness change value of the left region and the brightness change value of the right region is calculated. Based on the calculated similarity, the presence or absence of a shadow in the face region is determined. Therefore, by determining whether a shadow is present in a facial image based on the similarity in brightness change values ​​between the left and right areas separated from the center of the face, it is possible to determine whether a shadow is present in a facial image based on clear criteria.Unlike AI-based judgments, which only output results with a black box of judgment grounds, the basis for judgment is not unclear, and the absence of a shadow can be numerically visualized, making it possible to make a clear, objective judgment.

[0045] (2) L in CIELab color space * The value is used to calculate the brightness change value. Therefore, the presence or absence of a shadow can be determined using the brightness of a color system that can be expressed more closely to human perception, and the result of the shadow determination using this brightness also becomes more closely related to human perception.

[0046] (3) The similarity is calculated using the brightness change values ​​at positions symmetrical to the center of the face. Therefore, by comparing the brightness change values ​​at the same positions on the left and right sides of the face, it is possible to determine whether or not there is a shadow.

[0047] (4) In each of the left and right regions of the face region, the brightness change value is calculated for the vertical and horizontal directions of the face, and the similarity between the brightness change value of the left region and the brightness change value of the right region is calculated for each of the vertical and horizontal directions. Therefore, the brightness change value can be calculated comprehensively in each of the left and right regions of the face region to calculate the similarity, so that the presence or absence of a shadow can be determined more accurately.

[0048] (5) The hair portion is recognized from the face area, and the recognized hair portion area is added so that it is symmetrical left and right from the center of the face. A correction is made to delete the hair portion area and the added area from the face area, and a left area and a right area are obtained from the corrected face area. Therefore, by excluding the asymmetrical hair portion, it is possible to obtain a brightness change value without being affected by the hair, and to determine whether or not there is a shadow.

[0049] (6) The calculated similarity is compared with a threshold value to determine whether or not a shadow exists in the face area. Therefore, by using a threshold value, it is possible to determine whether or not a shadow is present in a face image using a clear criterion.

[0050] (7) A facial area is extracted from the image of an ID photograph taken using the ID photo machine 5, and the similarity is calculated. The calculated similarity is stored in the similarity memory unit 32, and a threshold value is determined from the similarity stored in the similarity memory unit 32. Here, the facial image acquired by the ID photo machine 5 is one in which the light hits the face accurately and there is no shadow on the face, so it meets the desired photograph quality. Therefore, by determining a threshold from the similarity obtained from an image of an ID photograph in which there is no shadow on the face and the light is accurately shining, it is possible to determine whether or not the image is shadow-free by using the threshold to determine whether or not there is a shadow.

[0051] (8) The similarity between the brightness change value of the left region and the brightness change value of the right region is calculated using a similarity calculation method for graphs. Therefore, the similarity of the brightness change values ​​can be calculated using a similarity calculation method suitable for determining the similarity of the brightness change values ​​shown in the graph.

[0052] (Second embodiment) In the second embodiment, the processing after recognizing the hair portion is different, and the hair portion is excluded. In the following description, parts that perform the same functions as those in the first embodiment described above are given the same reference numerals or have the same reference numerals at the end, and duplicate descriptions will be omitted as appropriate. <Image processing system 200> Although not shown, the image processing system 200 includes an image assessment device 201, a user terminal 4, a photo ID machine 5, and an image DB server 6.

[0053] <Image assessment device 201> FIG. 10 is a diagram showing functional blocks of an image assessment device 201 according to the second embodiment. The image assessment device 201 includes a control unit 210, a storage unit 230, and a communication interface unit 39. The control unit 210 includes an image receiving unit 11, a face area extraction unit 12, a hair recognition processing unit 213 (hair recognition means), an area division processing unit 214 (area division means), a brightness change value calculation unit 15, a similarity calculation unit 216 (similarity calculation means), a shadow determination processing unit 17, and an ID photo processing unit 18.

[0054] The hair recognition processing unit 213 performs processing to recognize the hair portion from the face region extracted by the face region extraction unit 12 . The region dividing processing unit 214 divides the face region extracted by the face region extracting unit 12 into left and right regions from the center of the face to obtain a left region and a right region. The similarity calculation unit 216 calculates the similarity between the lightness change value of the left region and the lightness change value of the right region calculated by the lightness change value calculation unit 15. In calculating the similarity, the similarity calculation unit 216 excludes the lightness change values ​​of the hair portion recognized by the hair recognition processing unit 213 and the portion located symmetrically to the hair portion from the center of the face from the lightness change value of the left region and the lightness change value of the right region calculated by the lightness change value calculation unit 15.

[0055] The storage unit 230 includes a program storage unit 231 , a similarity storage unit 32 , and a threshold storage unit 33 . The program storage unit 231 stores a similarity calculation program 231a, a determination processing program 31b, and a threshold determination program 31c. The similarity calculation program 231a is a program for executing the functions of the control unit 210 from the image receiving unit 11 to the similarity calculation unit 216.

[0056] <Processing Description> Next, processing using the image assessment device 201 will be described. The threshold value determination process and the shadow determination process are the same as those in the first embodiment (FIGS. 2 and 9). The similarity calculation process performed in the threshold determination process and the shadow determination process will be described. FIG. 11 is a flowchart showing the similarity calculation process of the image assessment device 201 according to the second embodiment. FIG. 12 is a diagram for explaining the similarity calculation process in the image assessment device 201 according to the second embodiment.

[0057] The processes from S221 to S223 in Fig. 11 are the same as the processes of S21, S24, and S25 in the first embodiment (Fig. 3). However, the process of S222 in Fig. 11 differs from S24 in Fig. 3 in that it is not the face region after correction. In S224 of FIG. 11, the control unit 210 (hair recognition processing unit 213) recognizes the hair portion. 12(A), image 45 has hair image 45a in left partial region 45L. On the other hand, image 45 has no hair image in right partial region 45R, which is located symmetrically from the center of the face to left partial region 45L. Hair recognition processing unit 213 recognizes hair image 45a.

[0058] In S225, the control unit 210 (similarity calculation unit 216) excludes the lightness change values ​​of the hair portion recognized by the hair recognition processing unit 213 and the portion located symmetrically to the hair portion from the center of the face from the lightness change values ​​of the left region and the right region calculated by the lightness change value calculation unit 15. 12(B) shows the horizontal brightness change value of divided region 248L, which is the left region of image 45. Furthermore, brightness graph 271R shows the horizontal brightness change value of divided region 248R, which is the right region of image 45. Comparing brightness graph 271L and brightness graph 271R, the values ​​in frame 271a are different due to the influence of hair image 45a. Therefore, the similarity calculation unit 216 excludes, for example, the range from about 0 to 160 on the horizontal axis included in the frame 271a.

[0059] In S226 of FIG. 11, the control unit 210 (similarity calculation unit 216) calculates the similarity of the lightness change value between the left and right regions excluding the excluded range. In this way, the left-right asymmetrical portion of hair image 45a is excluded, and therefore the lightness change value used to calculate the similarity can be one that excludes the influence of hair image 45a.

[0060] As described above, the image assessment device 201 of the second embodiment has the following advantages. The hair portion is recognized from the face region, and the similarity is calculated by excluding the brightness change values ​​of the recognized hair portion and the portion located symmetrically to the hair portion from the center of the face from the calculated brightness change values ​​of the left region and the right region. Therefore, when calculating the similarity, the hair portion can be excluded, and therefore the hair portion can be excluded from the determination of the shadow.

[0061] (Third embodiment) In the third embodiment, a partial area of ​​a face image is processed. <Image processing system 300> Although not shown, the image processing system 300 includes an image assessment device 301, a user terminal 4, a photo ID machine 5, and an image DB server 6.

[0062] <Image assessment device 301> FIG. 13 is a diagram showing functional blocks of an image assessment device 301 according to the third embodiment. The image assessment device 301 includes a control unit 310, a storage unit 330, and a communication interface unit 39. The control unit 310 includes an image receiving unit 11, a facial area extraction unit 12, an area division processing unit 214, a partial area acquisition unit 319 (partial area acquisition means), a brightness change value calculation unit 315 (brightness change value calculation means), a similarity calculation unit 16, a shadow determination processing unit 17, and an ID photo processing unit 18.

[0063] The partial region acquisition unit 319 acquires partial regions that are symmetrical about the center of the face for each of the left and right regions acquired by the region division processing unit 214. Here, it is desirable to acquire partial regions that exclude areas that are little affected by shadows from hair, eyebrows, eyes, etc. The partial region acquisition unit 319 can acquire facial features such as the forehead or cheeks as partial regions that are symmetrical about the center of the face. The brightness change value calculation unit 315 calculates a brightness change value for each of the left and right partial regions obtained by the partial region acquisition unit 319.

[0064] The storage unit 330 includes a program storage unit 331 , a similarity storage unit 32 , and a threshold storage unit 33 . The program storage unit 331 stores a similarity calculation program 331a, a determination processing program 31b, and a threshold determination program 31c. The similarity calculation program 331a is a program for executing the functions of the control unit 310, from the image receiving unit 11 to the similarity calculation unit 16.

[0065] <Processing Description> Next, processing using the image assessment device 301 will be described. The threshold value determination process and the shadow determination process are the same as those in the first embodiment (FIGS. 2 and 9). The similarity calculation process performed in the threshold determination process and the shadow determination process will be described. FIG. 14 is a flowchart showing the similarity calculation process of the image assessment device 301 according to the third embodiment. FIG. 15 is a diagram for explaining the similarity calculation process in the image assessment device 301 according to the third embodiment.

[0066] The processes of S321 and S322 in FIG. 14 are similar to the processes of S221 and S222 in the second embodiment (FIG. 11). In S323 of FIG. 14, the control unit 310 (partial region acquisition unit 319) cuts out a partial region from each of the left region and the right region. For example, divided areas 43L and 43R shown in Fig. 15(A) are images obtained by the process of S322 in Fig. 14. In contrast, Fig. 15(B) shows partial area 344L of divided area 43L and partial area 344R of divided area 43R cut out. The example shown in Fig. 15(B) shows the cheek area of ​​the face cut out.

[0067] In S324 of FIG. 14, the control unit 310 (lightness change value calculation unit 315) calculates a lightness change value for each of the left and right partial regions obtained by the partial region acquisition unit 319. The process of S325 is similar to the process of S26 in the first embodiment (FIG. 3). In this way, the portion that is heavily affected by the shadow is extracted, and the brightness change value used to calculate the similarity can be limited to the portion that is heavily affected by the shadow.

[0068] As described above, the image assessment device 301 of the third embodiment has the following advantages. (1) A partial area located symmetrically from the center of the face is acquired for each of the left and right areas of the face area, and the brightness change value of each of the acquired partial areas is calculated. Therefore, the presence or absence of a shadow can be determined based on the brightness change value between partial regions of the face.

[0069] (2) The partial region is a part that constitutes the facial region, and is set to be the cheek of the face. Therefore, parts of the face that are less affected by shadows, such as hair, eyebrows, and eyes, are excluded, and the presence or absence of a shadow can be determined using the cheeks, which are more susceptible to the influence of shadows.

[0070] In the third embodiment, the cheek areas (features) of the face are acquired as partial areas symmetrically positioned from the center of the face for each of the left and right areas of the face area, but this is not limiting. Other facial features or partial areas that make up the face area, such as the forehead, may also be acquired.

[0071] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Furthermore, the effects described in the embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments. Note that the above-described embodiments and the modified embodiments described below can be used in appropriate combinations, but detailed description thereof will be omitted.

[0072] (Variations) (1) In each embodiment, the brightness change value in the vertical and horizontal directions of the face is calculated, and the similarity is calculated for each of the vertical and horizontal directions. However, the present invention is not limited to this. The similarity may be calculated by calculating the brightness change value in at least one of the vertical and horizontal directions of the face.

[0073] (2) In each embodiment, the threshold value is set to the similarity obtained from the ID photo image acquired from the ID photo machine 5, but this is not limiting. The threshold value may be set in advance by the company that provides the service. Furthermore, although the threshold value is updated at a predetermined timing in the above description, it is not limited to this. Furthermore, the threshold may be different for each service, for example.

[0074] (3) In each embodiment, shadow detection is performed on a face image captured by the user terminal 4, but this is not limiting. The face image on which shadow detection is performed is not limited to one captured by the user terminal, and can be any face image for various purposes. For example, when a new ID photo machine is being manufactured, shadow detection can be performed on images taken with a prototype of the ID photo machine. This process can then be used to evaluate the performance of the prototype machine.

[0075] (4) In each embodiment, a shadow determination is performed on a face image captured by the user terminal 4, and face images determined to have no shadow are registered in the image DB server 6, but this is not limited to this. For example, this processing may be used for online identity verification. In this case, for example, during online operation guidance, an instruction may be given to capture a face image with a light shining from one side, and this processing may be used to confirm whether a shadow is present in the face image acquired after the instruction. For example, a face image without a shadow and a face image with a shadow may be output in the operation guidance, and a shadow determination may be performed on each image, thereby ensuring that the image is actually captured in accordance with the operation guidance. [Explanation of symbols]

[0076] 1, 201, 301 Image judgment device 4. User terminal 5. ID photo machine 6 Image DB server 10, 210, 310 Control unit 11 Image Reception Section 12 Face area extraction unit 13, 213 Hair recognition processing unit 14, 214 Area division processing unit 15, 315 Lightness change value calculation unit 16, 216 Similarity calculation part 17 Shadow detection processing section 18 ID Photo Processing Department 30, 230, 330 storage section 31a, 231a, 331a Similarity calculation program 31b Judgment processing program 31c Threshold Determination Program 32 Similarity storage unit 33 Threshold memory unit 100, 200, 300 Image Processing Systems 319 Partial area acquisition unit

Claims

1. a facial region extraction means for extracting a facial region from the evaluation target image; an area dividing means for dividing the face area extracted by the face area extracting means into left and right areas from the center of the face, to obtain a left area and a right area; a brightness change value calculation means for calculating a brightness change value for each of the left and right regions obtained by the region dividing means; a similarity calculation means for calculating a similarity between the brightness change value of the left region calculated by the brightness change value calculation means and the brightness change value of the right region; a shadow determination means for determining whether or not a shadow exists in the face area based on the similarity calculated by the similarity calculation means; An image assessment device comprising:

2. The image assessment device according to claim 1, The lightness change value calculation means calculates the lightness change value using lightness in a color space of a color system.

3. The image assessment device according to claim 1, The similarity calculation means calculates the similarity using brightness change values ​​at positions symmetrical to the center of the face.

4. The image assessment device according to claim 1, the brightness change value calculation means calculates the brightness change values ​​in the vertical and horizontal directions of the face in each of the left and right regions obtained by the region division means; The similarity calculation means calculates the similarity between the brightness change value of the left region and the brightness change value of the right region for each of the vertical direction and the horizontal direction.

5. The image assessment device according to claim 1, a partial area acquisition means for acquiring partial areas symmetrically positioned with respect to the center of the face for each of the left area and the right area obtained by the area division means, The brightness change value calculation means calculates the brightness change value of each of the left and right partial regions acquired by the partial region acquisition means.

6. The image assessment device according to claim 5, The partial region is a part that constitutes the facial region.

7. The image assessment device according to claim 6, The image assessment device, wherein the part is the forehead or cheek of the face.

8. The image assessment device according to claim 1, a hair recognition means for recognizing a hair portion from the face area extracted by the face area extraction means; a partial correction means for adding the area of ​​the hair part recognized by the hair recognition means so as to be symmetrical on the left and right sides from the center of the face, and deleting the area of ​​the hair part and the added area from the image of the face area; Equipped with The area dividing means obtains the left area and the right area from the face area corrected by the partial correction means.

9. The image assessment device according to claim 1, a hair recognition means for recognizing a hair portion from the face area extracted by the face area extraction means, The similarity calculation means calculates the similarity by excluding the lightness change value of the hair portion recognized by the hair recognition means and the portion located symmetrically to the hair portion from the center of the face from the lightness change value of the left region and the lightness change value of the right region calculated by the lightness change value calculation means.

10. In the image assessment device according to any one of claims 1 to 9, The shadow determination means compares the similarity calculated by the similarity calculation means with a threshold value to determine whether or not a shadow exists in the face area.

11. The image assessment device according to claim 10, the facial region extraction means extracts the facial region from an identification photograph image obtained by taking a photo using an identification photograph machine; a similarity registration means for storing the similarity calculated by the similarity calculation means in a similarity storage unit; a threshold value determining means for determining the threshold value from the similarity stored in the similarity storage unit; An image assessment device comprising:

12. Computer, a facial region extraction means for extracting a facial region from the evaluation target image; an area dividing means for dividing the face area extracted by the face area extracting means into left and right areas from the center of the face, to obtain a left area and a right area; a brightness change value calculation means for calculating a brightness change value for each of the left and right regions obtained by the region dividing means; a similarity calculation means for calculating a similarity between the brightness change value of the left region calculated by the brightness change value calculation means and the brightness change value of the right region; a shadow determination means for determining whether or not a shadow exists in the face area based on the similarity calculated by the similarity calculation means; A program to function as a

Images