Image registration support device, image registration system, and image registration support method

Abstract

To provide an image registration support device capable of acquiring an objective index for registering an authentication image.SOLUTION: The image registration support device for supporting the registration of an image, includes a processor and a memory for storing instructions. The processor is configured to acquire multiple dummy images on which a person is projected, and based on the acquired multiple dummy images, derive a threshold for determining the quality of a registration candidate image as a candidate image to be registered as an authentication image.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an image registration support device, an image registration system, and an image registration support method. [Background technology]

[0002] Conventionally, a personal authentication device that authenticates an individual from an image of a subject is known. As an example of a personal authentication device, a personal authentication device including a detection unit that detects characteristic parts from a subject image obtained by photographing the subject and outputs feature information, a threshold storage unit that stores a performance limit threshold that is a judgment criterion for the subject image, and a judgment unit that obtains the photographing condition of the subject from the feature information and the subject image, and outputs correction information to correct the photographing condition if the photographing condition exceeds the performance limit threshold (see Patent Document 1). This personal authentication device judges the photographing condition of the subject, issues a correction instruction if it is inappropriate, and photographs the subject when the condition becomes appropriate, thereby photographing the subject in the condition desired by the user and authenticating the subject in the appropriate authentication condition. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2005-149370 A Summary of the Invention [Problem to be solved by the invention]

[0004] The personal authentication device of Patent Document 1 arbitrarily sets a performance limit threshold value, which indicates the limit value of the range of face states that are determined to be capable of performing authentication performance, for each user. Therefore, it is easily influenced by the user's personal sense, and for example, a face image that is likely to fail authentication may be mistakenly registered as an authentication image. Therefore, it is difficult to obtain an objective index for registering an authentication image.

[0005] The present disclosure provides an image registration support device, an image registration system, and an image registration support method that can obtain an objective index for registering an image for authentication. [Means for solving the problem]

[0006] One aspect of the present disclosure is an image registration assistance device that includes a processor and a memory for storing instructions and assists in the registration of images, wherein the processor acquires a plurality of dummy images in which a person is reflected, and derives a threshold value for determining the quality of a registration candidate image, which is an image that is a candidate to be registered as an authentication image, based on the plurality of acquired dummy images.

[0007] One aspect of the present disclosure is an image registration system comprising a registration machine that registers an image of a person and an image registration support device that supports the registration of images, wherein the image registration support device acquires a plurality of dummy images in which a person is reflected, and derives a threshold for determining the quality of a registration candidate image, which is a candidate image to be registered as an authentication image, based on the plurality of acquired dummy images, the registration machine acquires an image captured by a camera, generates the registration candidate image based on the captured image, acquires the threshold from the image registration support device, and if the quality of the registration candidate image satisfies the threshold, causes the registration candidate image to be stored in a memory.

[0008] One aspect of the present disclosure is an image registration assistance method that assists in the registration of an image, the image registration assistance method having the steps of acquiring a plurality of dummy images in which a person is reflected, and deriving a threshold value for determining the quality of a registration candidate image, which is an image that is a candidate to be registered as an authentication image, based on the acquired plurality of dummy images. Effect of the Invention

[0009] According to the present disclosure, it is possible to obtain an objective indicator for registering an authentication image. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing an example of the configuration of an image registration system according to a first embodiment; [Diagram 2]FIG. 13 is a diagram for explaining a determination of whether or not a registration candidate image can be registered based on attributes of a dummy data set. [Diagram 3] 1 is a flowchart showing an example of an operation in a first derivation example of a threshold value by an image registration system. [Figure 4] 1 is a flowchart showing an example of a threshold calculation process according to a first derivation example of a threshold. [Diagram 5] Flowchart showing an example of quality determination processing [Figure 6] FIG. 11 is a diagram for explaining a first example of determination as to whether or not a registration candidate image can be registered based on the feature amount of a dummy dataset; [Figure 7] 11 is a flowchart showing an example of a threshold calculation process according to a second derivation example of a threshold. [Figure 8] 11 is a flowchart showing an example of a quality determination process according to a second derivation example of a threshold value. [Figure 9] FIG. 11 is a diagram for explaining a second example of determination of whether or not a registration candidate image can be registered based on the feature amount of a dummy dataset; [Figure 10] 11 is a flowchart showing an example of a threshold calculation process according to a third derivation example of a threshold. [Figure 11] 11 is a flowchart showing an example of a quality determination process according to a third example of derivation of a threshold value. [Figure 12] A diagram showing an example of an ideal image [Figure 13] Flowchart showing an example of an ideal image selection process [Figure 14] Diagram showing the location of the ideal image in feature space [Figure 15] A diagram showing an example of the arrangement of dummy images in the feature space. [Figure 16] A first example of a dummy data set. [Figure 17] A second example of a dummy data set. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters or duplicate explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the attached drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0012] (First embodiment) <Image registration system configuration> 1 is a diagram showing an example of the configuration of an image registration system 5 in the first embodiment. The image registration system 5 includes a face management server 100 and a registration device 200. The face management server 100 and the registration device 200 may be communicatively connected to each other via a communication network, for example, or may be communicatively connected to each other using another communication line.

[0013] The face management server 100 manages information used for authenticating a person. The information used for authenticating a person includes an image of the person. Here, the example mainly illustrates the management of information used for authenticating a person's face (e.g., a facial image), but the face management server 100 may manage other information related to a person for authentication. There may be one or more face management servers 100, and each face management server 100 can be connected to one registration device 200 or multiple registration devices 200.

[0014] The images of persons managed by the face management server 100 include, for example, dummy images for determining whether or not they can be registered by the registration device 200. These dummy images may be images for which the registration device 200 does not actually determine whether or not they can be registered, or may be images based on images that have been previously registered by the registration device 200 or images that have not been registered. The dummy images may be images of the face of a person or images similar to a face.

[0015] The registration device 200 registers information about a person to be authenticated, and registers at least an image of the person (e.g., a facial image). The registered information is used to authenticate the person to be authenticated. The registration device 200 includes, for example, a personal computer (PC), a tablet terminal, or a smartphone. There may be one or more registration devices 200. Each registration device 200 may be connected to the same face management server 100, or may be connected to multiple different face management servers 100.

[0016] <Face management server configuration> The face management server 100 includes a processor 110, a memory 120, and a communication device .

[0017] The processor 110 may be configured using, for example, a Central Processing Unit (CPU) or a Digital Signal Processor (DSP). The processor 110 may be configured using various integrated circuits (for example, a Large Scale Integration (LSI) or a Field Programmable Gate Array (FPGA)). The processor 110 realizes various functions by executing programs stored in the memory 120. The processor 110 comprehensively controls each part of the face management server 100 and performs various processes.

[0018] The processor 110 has a function of executing various processes including a quality score calculation process, a threshold calculation process, and an ideal image selection process. The quality score calculation process is a process of calculating a score indicating the quality of a dummy image (e.g., a face orientation value, an illuminance value, a mask score). Note that the quality score calculation process may include a process of deriving information indicating quality other than the score.

[0019] The threshold calculation process is a process for calculating a threshold th, i.e., a judgment standard, for judging the quality of the registration candidate image CG. The registration candidate image CG is a candidate image to be registered as an authentication image by the registration device 200. The threshold th is, for example, a boundary value that distinguishes between good and bad quality of the registration candidate image CG. Furthermore, the threshold th may be represented by information regarding a predetermined area (for example, a registerable area and an unregisterable area described later) in the feature space described later.

[0020] The ideal image selection process is a process of selecting an ideal image. The ideal image is an image that is ideal as an image for authentication, and is an example of a recommended image that is recommended as an image for authentication.

[0021] The memory 120 includes a primary storage device (e.g., Random Access Memory (RAM) or Read Only Memory (ROM). The memory 120 may include a secondary storage device (e.g., a Hard Disk Drive (HDD) or a Solid State Drive (SSD) or a tertiary storage device (e.g., an optical disk or an SD card), etc. The memory 120 may be an external storage medium and may be removable from the face management server 100. The memory 120 stores various data, information, programs, instructions, etc. The memory 120 may also store, for example, various trained models. The instructions include, for example, various instructions (commands) such as input, output, calculation, or control executed by the processor 110.

[0022] The memory 120 includes a registration database 121 (registration DB), a dummy database 122 (dummy DB), and a threshold database 123 (threshold DB). The registration database 121 registers a registration image used as an authentication image. The registration image is a registration candidate image CG that is permitted to be registered among the registration candidate images CG.

[0023] The dummy database 122 holds a plurality of dummy images. The dummy images are dummy images for deriving a threshold value th for determining whether or not a registration candidate image CG is allowed to be registered. The dummy database 122 holds a dummy data set including a plurality of dummy images. The dummy data set includes at least a dummy image, and may or may not include information related to the dummy image, such as attributes and feature amounts of the dummy image.

[0024] The threshold database 123 holds thresholds th (th1, th2, ...). Furthermore, the threshold database 123 may hold, as the thresholds th, information on a registerable region RO in the feature space CS described later and information on a non-registerable region RN.

[0025] The communication device 130 communicates various data or information in accordance with a wired or wireless communication method. The communication method used by the communication device may include a Local Area Network (LAN), a Wide Area Network (WAN, a mobile phone network, power line communication, short-range wireless communication (e.g., Bluetooth (registered trademark), NFC (registered trademark)), and the like.

[0026] The face management server 100 may also include an input device, a display device, and the like, which are used by an administrator of the face management server 100, for example.

[0027] <Registration machine configuration> The registration device 200 includes a processor 210 , a memory 220 , a communication device 230 , a camera 240 , and a display device 250 .

[0028] The processor 210 may be configured using, for example, a CPU or a DSP. The processor 210 may be configured using various integrated circuits (for example, an LSI or an FPGA). The processor 210 realizes various functions by executing programs stored in the memory 220. The processor 210 comprehensively controls each unit of the registration device 200 and performs various processes.

[0029] The processor 210 has a function of executing various processes including face detection processing, quality judgment processing, feature extraction processing, attribute extraction processing, ideal image presentation processing, and GUI control processing. The face detection processing is processing for detecting a face area from an image captured by the camera 240. As an example, the processor 210 may cut out a face area from the image and set it as a registration candidate image CG.

[0030] The quality determination process is a process for determining the quality of the registration candidate image CG (for example, an image of a face area in a captured image). The quality determination process includes a process for inputting an attribute result output by an attribute extraction process described later and a feature amount related to the attribute (for example, a face attribute feature amount), performing a threshold determination, and outputting whether or not to permit registration of the registration candidate image CG (registration OK / NG). The quality here is an attribute determined by environmental factors (camera installation conditions, illumination conditions, installation facilities, etc.), such as face direction, illuminance, blur, presence or absence of face occlusion, or smile. Presence or absence of face occlusion is, for example, whether or not at least a part of the face is hidden by a hand or an object other than a hand, and possible examples include the presence or absence of a mask, sunglasses, or hat. The processor 210 performs the quality determination process based on the threshold value th derived by the face management server 100. The quality determination process may determine the quality using, for example, a learned model for quality determination stored in the memory 220.

[0031] The feature extraction process is a process of extracting feature amounts (e.g., face feature amounts) of the registration candidate image CG. Each feature amount of the registration candidate image CG is indicated, for example, by a numerical value indicating each feature amount. The attribute extraction process is a process of extracting attributes (e.g., face direction, illuminance) of the registration candidate image CG. The attribute extraction process includes a process of inputting an image and deriving (e.g., calculating) feature amounts related to the attributes (attribute feature amounts, for example, 256-dimensional feature amounts). This process is, for example, a process of outputting an intermediate layer in a neural network. Then, the attribute extraction process includes a process of inputting the derived feature amounts (attribute feature amounts) and deriving numerical values ​​of each attribute. This process is, for example, a process of outputting a fully connected layer (FC layer) in a neural network. Each attribute of the registration candidate image CG is indicated, for example, by a numerical value indicating each attribute. The feature extraction process may extract feature amounts using, for example, a trained model for feature extraction stored in the memory 220. The attribute extraction process may extract features (attribute features) using a trained model for attribute extraction stored in the memory 220, for example.

[0032] The ideal image presentation process is a process of presenting (for example, displaying, outputting sound, or presenting in other ways) an ideal image selected by the face management server 100. The GUI control process is a process of controlling the GUI (for example, the display device 250) in the registration device 200.

[0033] The GUI of the registration machine 200 may also include an input device. The input device may include various buttons, keys, a keyboard, a mouse, a touch panel, a microphone, or other input devices. The input device accepts input of various data or information. The input device can be operated, for example, by a user who uses the registration machine 200. The display device 250 and the input device may be integrally configured, for example, as a touch panel, etc.

[0034] The memory 220 includes a primary storage device (e.g., RAM or ROM). The memory 220 may include a secondary storage device (e.g., HDD or SSD) or a tertiary storage device (e.g., optical disk or SD card), etc. The memory 220 may be an external storage medium, and may be removable from the registration device 200. The memory 220 stores various data, information, programs, etc.

[0035] The communication device 230 communicates various data or information according to a wired or wireless communication method. The communication method used by the communication device may include a LAN, a WAN, a mobile phone network, a power line communication, a short-range wireless communication (e.g., Bluetooth (registered trademark), NFC (registered trademark)), and the like.

[0036] Camera 240 is configured to have at least a lens (not shown) and an image sensor (not shown). The image sensor is a solid-state imaging element such as a Charged-Coupled Device (CCD) or a Complementary Metal-Oxide-Semiconductor (CMOS), and converts an optical image formed on an imaging surface into an electrical signal. Camera 240 captures an image of a subject including a person (e.g., a person's face) to obtain a captured image.

[0037] The camera 240 may capture an image of a subject including a person's face, for example, based on a user operation via an input device. The camera 240 may automatically capture an image according to a predetermined condition. The camera 240 may be provided outside the registration device 200. In this case, the registration device 200 and the camera 240 may be connected by wire via a USB or the like, or may be connected wirelessly via a wireless LAN or the like. The camera 240 may be a camera that can be installed away from the registration device 200, such as a network camera.

[0038] The display device 250 is configured using a display such as a Liquid Crystal Display (LCD) or an organic electroluminescence (organic EL). The display device 250 displays various data or information. The display device 250 displays, for example, a registration candidate image, a registration image, an ideal image, or other information.

[0039] <First derivation example of threshold th> Next, the derivation of the threshold th and an overview of image registration using the threshold th will be described.

[0040] In the first derivation example, the processor 110 of the face management server 100 derives (eg, calculates) the threshold value th based on the attributes of each dummy image DG included in the dummy data set DS.

[0041] FIG. 2 is a diagram for explaining a determination as to whether or not a registration candidate image CG can be registered based on the attributes of the dummy data set DS.

[0042] In FIG. 2, as an example, the dummy database 122 of the face management server 100 holds in advance a dummy data set DS including two dummy images DG (DG11, DG12). The dummy database 122 includes, as information related to the dummy image DG, for example, a dummy image DG including a face, an identification ID, and attribute information (for example, face direction, illuminance of the face area). The information related to the dummy image DG11 includes the dummy image DG11, and includes information indicating that the identification ID is "001", the face direction is "2 degrees", and the illuminance is 40 lux. The information related to the dummy image DG12 includes the dummy image DG12, and includes information indicating that the identification ID is "002", the face direction is "8 degrees", and the illuminance is 60 lux. The processor 110 may extract attributes of the dummy image DG based on the dummy image DG and store the attributes in the dummy database 122 together with information related to the dummy image DG, or information on the attributes extracted from the dummy image DG in advance may be stored in the dummy database 122. When estimating the attributes of the dummy image DG, the processor 110 may estimate the attributes using, for example, a trained model for attribute estimation stored in the memory 120.

[0043] In the face management server 100, the processor 110 derives (e.g., calculates) the threshold value th based on each dummy image DG included in the dummy data set DS held in the dummy database 122. For example, the processor 110 may derive the threshold value th based on the maximum and minimum values ​​of the values ​​indicating each attribute of the dummy image DG. For example, in FIG. 2, the minimum value of the face orientation included in each dummy image DG included in the dummy data set DS is 2 degrees and the maximum value is 8 degrees. In this case, the processor 110 may set the lower threshold value th11 to 2 degrees and the upper threshold value th12 to 8 degrees. For example, in FIG. 2, the minimum value of the illuminance included in each dummy image DG included in the dummy data set DS is 40 lux and the maximum value is 60 lux. In this case, the processor 110 may set the lower threshold value th21 to 40 lux and the upper threshold value th22 to 60 lux. The processor 110 causes the derived thresholds th11, th12, th21, and th22 to be stored in the threshold database 123.

[0044] In the registration machine 200, the processor 210 acquires information on the threshold th stored in the threshold database 123 of the face management server 100 via the communication device 230 at a predetermined timing (for example, before or during the quality judgment process). During the quality judgment process, if the attribute information of the person to be authenticated satisfies the threshold th (i.e., the judgment criterion defined by the threshold th), the processor 210 causes the registration candidate image to be stored (i.e., registered) in the registration database 121. For example, the judgment criterion (threshold th) is whether the face direction is at an angle between 2 degrees and 8 degrees, and whether the illuminance is between 40 lux and 60 lux.

[0045] For example, in FIG. 2, three registration candidate images CG (CG1, CG2, CG3) are present as an example. The registration candidate data regarding the registration candidate image CG has information on the same items as the information regarding the dummy image DG. That is, the registration candidate data includes an image including a face (registration candidate image CG), an identification ID, and attribute information (e.g., face direction, illuminance of the face area). For example, the processor 210 estimates the attribute of the registration candidate image CG based on the registration candidate image CG. The registration candidate data regarding the registration candidate image CG1 includes the registration candidate image CG1, and includes information indicating that the identification ID is "101", the face direction is "5 degrees", and the illuminance is 50 lux. The registration candidate data regarding the registration candidate image CG2 includes the registration candidate image CG2, and includes information indicating that the identification ID is "102", the face direction is "8 degrees", and the illuminance is 40 lux. The registration candidate data regarding the registration candidate image CG3 includes the registration candidate image CG3, and includes information indicating that the identification ID is "103", the face direction is "0 degrees", and the illuminance is 50 lux.

[0046] In this case, the processor 210 determines that the registration candidate image CG1 and the registration candidate image CG2 can be registered because they satisfy the threshold value th during the quality determination process. Therefore, the processor 210 stores (i.e., registers) the registration candidate image CG1 and the registration candidate image CG2 in the registration database 121. In this case, the processor 210 transmits the registration candidate image CG1 and the registration candidate image CG2 to the face management server 100 via the communication device 230. Then, the processor 110 of the face management server 100 acquires the registration candidate image CG1 and the registration candidate image CG2 via the communication device 130 and stores them in the registration database 121. On the other hand, the processor 110 determines that the registration candidate image CG3 cannot be registered because it does not satisfy the threshold value th during the quality determination process. This is because the face direction is 0 degrees, which is outside the range of 2 degrees to 8 degrees, and therefore does not satisfy the threshold value th. Therefore, the processor 110 does not hold the registration candidate image CG3 in the registration database 121 (that is, does not register it).

[0047] FIG. 3 is a flowchart showing an example of the operation of the image registration system 5 in the first derivation example of the threshold value th.

[0048] In the face management server 100, the processor 110 calculates the threshold th based on the dummy data set DS held in the dummy database 122 (S101). In this case, the processor 110 may estimate attributes (e.g., face direction, illuminance) of each dummy image DG based on each dummy image DG included in the dummy data set DS, and calculate the threshold th based on each estimated attribute. The processor 110 stores this threshold th in the threshold database 123. The processor 110 derives whether this threshold th is satisfied (satisfying the threshold th11 or more and the threshold th12 or less).

[0049] In the registration device 200, the processor 210 causes the camera 240 to capture an image of a subject including a person's face, and obtains a captured image (S201). The processor 210 detects and cuts out the person's face area in the registration candidate image CG, and obtains the registration candidate image CG (S202). The processor 210 performs a quality determination process to determine the quality of the registration candidate image CG (S203). In step S203, for example, the processor 210 reads a threshold value th used in the quality determination process from the threshold value database 123, and performs the quality determination process using this threshold value th.

[0050] If the result of the quality assessment process is that the quality is good (quality OK) (Yes in step S203), that is, if the quality of the registration candidate image CG (e.g., the attributes of the registration candidate image CG) meets the assessment criteria, processor 210 extracts features (facial features) of the registration candidate image CG (S204).

[0051] The processor 210 stores the registration information including the extracted feature amount in the registration database 211 (S205). In this case, the processor 210 transmits the registration information to the face management server 100 via the communication device 230. In the face management server 100, the processor 110 receives the registration information via the communication device 130 and stores the registration information in the registration database 211. The registration information includes a registration image. The registration image is a registration candidate image CG of good quality (registration permitted). The registration information may also include the feature amount of the registration image or other information that can be extracted from the registration candidate image.

[0052] The processor 210 causes the display device 250 to display information indicating that the quality of the registration candidate image CG satisfies the judgment criteria (information indicating that registration is OK) (S206).

[0053] On the other hand, if the quality is not good (quality NG) as a result of the quality determination process (No in step S203), that is, if the quality of the registration candidate image CG (e.g., the attributes of the registration candidate image CG) does not satisfy the determination criterion, the processor 210 causes the display device 250 to display information indicating that registration of the registration candidate image CG is not permitted, or re-imaging guidance information (guidance) which is information guiding re-imaging by the camera 240 (S207). Then, the process proceeds to step S201. Therefore, the camera 240 may capture an image again, and the processes of steps S201 to S203 and S207 may be repeated until the quality of the registration candidate image satisfies the determination criterion.

[0054] FIG. 4 is a flowchart showing an example of a threshold calculation process according to a first derivation example of a determination criterion.

[0055] In the face management server 100, the processor 110 refers to the dummy database 122 and estimates the face direction for all dummy images DG included in the dummy data set DS (S111).

[0056] The processor 110 stores the maximum face direction, i.e., the angle at which the absolute value of the angle indicated by the face direction is the maximum, in the threshold database 123 as the face direction threshold th (S112). In this case, face directions equal to or less than the face direction threshold th satisfy the threshold th (determination criterion) in the quality assessment process. The processor 110 may store the minimum face direction, i.e., the angle at which the absolute value of the angle indicated by the face direction is the smallest, in the threshold database 123 as one of the face direction thresholds th. In this case, a face direction between the maximum face direction threshold th and the minimum face direction threshold th satisfies the threshold th (determination criterion) in the quality assessment process.

[0057] The processor 110 refers to the dummy database 122 and estimates the presence or absence of a medical or other mask covering at least a part of a person's face for all dummy images DG included in the dummy data set DS (S113). The processor 110 may derive (e.g., calculate) a mask score indicating the degree to which a mask is estimated to exist by image processing of the registration candidate image CG, and estimate the presence or absence of a mask based on the mask score. The mask score is an example of a quality score. For example, the mask score may be represented by a value between 0 and 1, and it may be determined that a mask exists when the mask score is 0.5 or more, and that a mask does not exist when the mask score is less than 0.5.

[0058] The processor 110 stores the maximum mask score among the derived one or more mask scores as a masking threshold th in the threshold database 123 (S114). In this case, a mask score less than the masking threshold th satisfies the threshold th (criterion) in the quality judgment process.

[0059] FIG. 5 is a flowchart showing an example of the quality determination process.

[0060] In the registration device 200, the processor 210 estimates the face direction of the registration candidate image CG (S211). The processor 210 refers to the threshold value th for face direction stored in the threshold value database 123 of the face management server 100, acquires the threshold value th for face direction from the face management server 100 via the communication device 230, and determines whether the estimated face direction is equal to or less than the threshold value th, that is, satisfies the threshold value th (S212). If the estimated face direction is equal to or less than the threshold value th (Yes in step S212), the processor 210 derives a mask score (S213). For example, if the threshold value database 123 holds a maximum threshold value and a minimum threshold value, the processor 210 may determine that the quality is OK if the score is between the maximum threshold value and the minimum threshold value.

[0061] The processor 210 refers to the mask threshold th stored in the threshold database 123 of the face management server 100, acquires the mask threshold th from the face management server 100 via the communication device 230, and determines whether the derived mask score is less than the mask threshold th, i.e., satisfies the threshold th (S214). If the derived mask score is less than the mask threshold th, the processor 210 determines that the quality of the registration candidate image CG satisfies the threshold th (i.e., quality OK), i.e., determines that registration is possible (S215). In this case, the processor 210 registers the registration information by storing the registration information related to the registration candidate image CG (i.e., the registration image) determined to satisfy the threshold th in the registration database 121 of the face management server 100 via the communication device 230.

[0062] On the other hand, if the estimated face orientation is greater than the threshold th (No in step S212), or if the derived mask score is greater than or equal to the mask threshold th (No in step S214), the processor 210 determines that the quality of the registration candidate image CG does not satisfy the threshold th (i.e., quality NG), that is, determines that registration is not possible (S216). Information related to the registration candidate image CG determined to not satisfy the threshold th is not registered in the registration database 121 of the face management server 100.

[0063] In this way, when there are multiple attributes (e.g., face orientation, presence or absence of mask) as the threshold th in the quality determination process, if the attributes of the registration candidate image CG satisfy a predetermined number or more of index attributes (e.g., all attributes), the processor 210 may determine that the quality of the registration candidate image CG is good and that the registration candidate image CG can be registered. This allows the image registration system 5 to register a registration candidate image CG of higher quality as an authentication image, and to register a registration candidate image CG with high person authentication accuracy.

[0064] In this way, the image registration system 5 allows the face management server 100 to easily derive the threshold th, i.e., the judgment criterion, used in the quality judgment process based on the attributes of each dummy image DG in the dummy data set DS. Then, the face management server 100 can store the threshold th as a criterion for quality judgment. In addition, since the attributes of the dummy image are objective information, not information according to the sense of a specific person, the face management server 100 can obtain an objective judgment criterion.

[0065] In addition, the registration device 200 can perform quality judgment processing on the registration candidate image CG using the objective judgment criteria derived by the face management server 100, so it can objectively judge whether or not the registration candidate image CG can be registered. Therefore, for example, an authentication device that authenticates a person using a registration image can objectively and more accurately perform face authentication of the person.

[0066] <Second derivation example of threshold th> Next, a second derivation example of the threshold value th used in the quality assessment process will be described. In the second derived example, the dummy database 122 holds, as the dummy image DG, one or more dummy images DG1 that can be registered by the quality assessment process.

[0067] In the second derivation example, the face management server 100 derives (e.g., calculates) the threshold value th based on the feature amount of each dummy image DG1 included in the dummy data set DS. In other words, the face management server 100 derives the threshold value th based on the feature amount of the dummy data set DS, and sets it as a criterion for the quality assessment process.

[0068] 6 is a diagram for explaining a first example of a determination as to whether or not a registration candidate image CG can be registered based on the feature amount of the dummy data set DS. In FIG. 6, the description of the same matters as those described in FIG. 2 will be omitted or simplified.

[0069] In FIG. 6, similarly to FIG. 2, as an example, the dummy database 122 of the face management server 100 holds in advance a dummy data set DS including two dummy images DG1 (DG11, DG12).

[0070] In the face management server 100, the processor 110 derives (e.g., calculates) a threshold value th based on each dummy image DG1 included in the dummy data set DS held in the dummy database 122. Specifically, the processor 110 extracts a feature amount indicating the characteristics of the dummy image DG1 by executing, for example, principal component analysis (PSA) or other image processing based on each dummy image DG1. In this case, the processor 110 may extract the feature amount using, for example, a trained model for feature amount extraction stored in the memory 120. The processor 110 derives (e.g., calculates) a position in the feature space CS indicated by the feature amount based on the extracted feature amount. In FIG. 6, the feature space CS is shown as a plane on which two-dimensional feature amounts (e.g., the first principal component a and the second principal component b) can be mapped. Note that the feature amount does not have to be two-dimensional, and may be three or more dimensions (e.g., N dimensions). In this case, the feature space CS is expressed in an N-dimensional space, and the dummy image DG1 is mapped to a specific position in the N-dimensional space.

[0071] The processor 110 derives the threshold value th based on the position of each dummy image DG1 in the feature space CS. In this case, the processor 110 derives (e.g., calculates) a registrable area RO in the feature space CS that permits the registration of the registration candidate image CG. In this case, for example, when the arrangement position (mapping position) based on the feature amount of the registration candidate image CG is included in the registrable area RO, the threshold value th (criterion) for registration is satisfied. The registrable area RO has, for example, a circular outer periphery, but may have other shapes. For example, the processor 110 derives (e.g., calculates) the registrable area RO based on the arrangement position of each dummy image DG1 in the feature space CS.

[0072] 6, the dummy image DG11 and the dummy image DG12 are arranged inside the registrable area RO. In other words, the dummy image DG11 and the dummy image DG12 are images that can be registered in the registration database 121 when the quality assessment process is performed on the dummy image DG.

[0073] In the registration machine 200, the processor 210 acquires information on the threshold th (e.g., information on the registrable area RO) stored in the threshold database 123 of the face management server 100 via the communication device 230 at a predetermined timing (e.g., before or during the quality judgment process). When the arrangement position in the feature space CS based on each feature amount of the registration candidate image CG of the person to be authenticated is within the registrable area RO during the quality judgment process, the processor 210 causes the registration database 121 to store (i.e., register) the registration information on the registration candidate image.

[0074] For example, three registration candidate images CG (CG1, CG2, CG3) are present in Fig. 6. The contents of the registration candidate images CG1, CG2, and CG3 are the same as those in Fig. 2.

[0075] In this case, during the quality determination process, the processor 210 determines that the mapping positions of the registration candidate image CG1 and the registration candidate image CG3 in the feature space CS are inside the registrable area RO. Therefore, the processor 210 determines that the registration candidate image CG1 and the registration candidate image CG2 are registrable. Therefore, the processor 110 causes the registration database 121 to hold (i.e., register) the registration information regarding the registration candidate image CG1 and the registration information regarding the registration candidate image CG2. On the other hand, the processor 110 determines that the mapping position of the registration candidate image CG3 in the feature space CS is outside the registrable area RO. Therefore, the processor 210 determines that the registration candidate image CG3 is not registrable. Therefore, the processor 110 does not cause the registration database 121 to hold (i.e., does not register) the registration information regarding the registration candidate image CG3.

[0076] Furthermore, the processor 210 may display information in which each registration candidate image CG is mapped based on the feature amount in the feature space CS on the display device 250. The mapping information here may be information in which each registration candidate image CG is mapped to a position in the feature space CS based on each feature amount of each registration candidate image CG. By checking the mapping information of each registration candidate image in the feature space CS, the user can check the state of whether each registration candidate image CG can be registered as a result of integrating multiple feature amounts.

[0077] Moreover, the attribute estimation in the first derivation example is performed by threshold judgment for each independent attribute. It is performed based on some of the multiple feature quantities. Therefore, compared to the quality judgment of the registration candidate image CG based on attribute estimation, it can be said that the quality judgment of the registration candidate image CG based on each feature quantity defining the feature space CS can perform a more rigorous and highly accurate quality judgment because it uses a feature quantity in which multiple attributes are integrated (an integrated attribute feature quantity in which multiple attribute feature quantities are integrated).

[0078] FIG. 7 is a flowchart showing an example of a threshold calculation process according to the second derivation example of the threshold th.

[0079] In the face management server 100, the processor 110 refers to the dummy database 122 and extracts feature amounts for all dummy images DG1 included in the dummy data set DS (S121).

[0080] The processor 110 derives the arrangement position of each dummy image DG1 on the feature space CS based on each feature amount of each dummy image DG1. The processor 110 calculates the reference position RP1 of all dummy images DG1 on the feature space CS based on the position of each dummy image DG1 on the feature space CS. The reference position RP1 of all dummy images DG1 on the feature space CS may be, for example, the center position (average position) of all dummy images DG1 (when looking at all dummy images DG1 arranged on the feature space CS as a whole), the center of gravity position of all dummy images DG1, or other statistical position. The processor 110 calculates the distance between the reference position RP1 in the feature space CS and the arrangement position of the dummy image DG1 for each dummy image DG1 based on the reference position RP1 of all dummy images DG1 in the feature space CS. The distance here may be Euclidean distance or cosine similarity, but here, an example of Euclidean distance is shown. The processor 110 calculates the longest distance d1 from among the distances calculated for each dummy image DG1 (S122).

[0081] The processor 110 derives the area inside a circle CR1 having the maximum distance d1 as its radius and centered at the reference position RP1 as the registrable area RO. The processor 110 stores information about the registrable area RO (e.g., the position and size of the registrable area RO, the reference position RP1, and information about the circle CR1) in the threshold database 123 (S123).

[0082] FIG. 8 is a flowchart showing an example of a quality determination process according to the second derivation example of the threshold value th.

[0083] In the registration device 200, the processor 210 extracts each feature amount of the registration candidate image CG (S221). The processor 210 derives (e.g., calculates) the arrangement position of the registration candidate image CG in the feature space CS based on each extracted feature amount. The processor 210 determines whether or not the arrangement position in the feature space CS of the registration candidate image CG generated based on, for example, a dummy image DG is included in the registrable area RO (S222). In this case, the processor 210 may inquire about the information of the registrable area RO to the face management server 100 via the communication device 230, obtain the information of the registrable area RO from the face management server 100, and determine whether or not the information is included in the registrable area RO.

[0084] If the position of the registration candidate image CG in the feature space CS is included in the registrable area RO, the processor 210 determines that the quality of the registration candidate image CG is good (quality OK), that is, that the registration is possible (S223). In this case, the processor 210 registers the registration information by storing the registration information related to the registration candidate image CG determined to be of good quality in the registration database 121 of the face management server 100 via the communication device 230.

[0085] On the other hand, if the position of the registration candidate image CG in the feature space CS is not included in the registrable area RO, the processor 210 determines that the quality of the registration candidate image CG is not good (quality NG), that is, determines that registration is not possible (S224). The registration information related to the registration candidate image CG determined to be of poor quality is not registered in the registration database 121 of the face management server 100.

[0086] In this way, the image registration system 5 allows the face management server 100 to easily derive the registrable area RO, i.e., the judgment criterion, used in the quality judgment process based on the feature amount of each dummy image DG1 in the dummy data set DS. The face management server 100 can store information on the registrable area RO as a criterion for quality judgment. In addition, since the feature amount of the dummy image is objective information rather than information according to the sense of a specific person, the face management server 100 can obtain an objective judgment criterion.

[0087] In addition, the registration device 200 can perform quality judgment processing on the registration candidate image CG using the objective judgment criteria derived by the face management server 100, so it can objectively judge whether or not the registration candidate image CG can be registered. Therefore, for example, an authentication device that authenticates a person using a registration image can objectively and more accurately perform face authentication of the person.

[0088] In the second derivation example of the threshold value th, the process of step S204 shown in FIG. 3 is unnecessary and is omitted.

[0089] <Third derivation example of threshold th>

[0090] The third derivation example is a modified example of the second derivation example. In the face management server 100, the dummy database 122 holds, as dummy images DG, one or more dummy images DG1 that can be registered by the quality assessment process, and one or more dummy images DG2 that cannot be registered by the quality assessment process. The face management server 100 derives the threshold value th based on the feature amounts of each dummy image DG1 and each dummy image DG2 included in the dummy data set DS. At this time, the face management server 100 derives an unregisterable area RN together with a registerable area RO.

[0091] Fig. 9 is a diagram for explaining a second example of determination of whether or not a registration candidate image CG can be registered based on the feature amount of the dummy data set DS. In Fig. 9, the description of the same matters as those described in Fig. 2 or Fig. 6 is omitted or simplified.

[0092] 2 and 6, as an example, the dummy database 122 of the face management server 100 stores in advance a dummy data set DS including two dummy images DG (DG1, DG2). In Fig. 9, the dummy data set DS further includes two dummy images DG (DG21, DG22). Information on each dummy image DG includes information on whether each dummy image DG is data that can be registered or data that cannot be registered, and is stored in the dummy database 122.

[0093] 9, the information on the dummy image DG21 includes the dummy image DG21, and includes information indicating that the identification ID is "011", the face direction is "2 degrees", and the illuminance is 800 lux. The information on the dummy image DG22 includes the dummy image DG22, and includes information indicating that the identification ID is "012", the face direction is "8 degrees", and the illuminance is 20 lux. In other words, the dummy image DG21 is an image that is too dark and of poor quality to be registered, and the dummy image DG22 is an image that is too bright and of poor quality to be registered.

[0094] In the face management server 100, the processor 110 derives a threshold value th based on each dummy image DG1, DG2 included in the dummy data set DS held in the dummy database 122. In this case, the processor 110 derives (e.g., calculates) a registrable area RO in the feature space CS based on each registrable dummy image DG1 included in the dummy data set DS. The processor 110 also derives (e.g., calculates) a non-registrable area RN in the feature space CS that does not permit registration of the registration candidate image CG based on each non-registrable dummy image DG2 included in the dummy data set DS. In this case, for example, when the arrangement position based on the feature amount of the registration candidate image CG is included in the non-registrable area RN, the threshold value th (criterion) for not registering is met.

[0095] Specifically, the processor 110 extracts feature quantities indicating the features of the dummy images DG1 and DG2 based on the dummy images DG1 and DG2. The method of extracting the feature quantities may be the same as the second derivation example of the threshold value th described above. The processor 110 derives (e.g., calculates) the arrangement positions of the dummy images DG1 and DG2 in the feature space CS based on the extracted feature quantities.

[0096] The processor 110 derives a registrable area RO, i.e., a judgment criterion, based on the arrangement position of each dummy image DG1 in the feature space CS. The processor 110 derives a non-registrable area RN, i.e., a judgment criterion, based on the arrangement position of each dummy image DG2 in the feature space CS. Note that the above non-registrable area RN is an explicit non-registrable area. An area in the feature space CS that is not included in either the registrable area RO or the non-registrable area RN is an area that is not explicitly but implicitly considered to be non-registrable.

[0097] Furthermore, when the registrable area RO and the non-registrable area RN overlap in the feature space CS, the processor 110 may give priority to the non-registrable area RN. In other words, when the arrangement position in the feature space CS based on the feature amount of the registration candidate image CG is included in the registrable area RO, the processor 110 may determine that the registration candidate image CG is non-registrable. Furthermore, when the registrable area RO and the non-registrable area RN overlap in the feature space CS, the processor 110 may divide the overlapping area into the registrable area RO and the non-registrable area RN by drawing a boundary line connecting the tangent points of the periphery of the registrable area RO and the periphery of the non-registrable area RN. In other words, the processor 110 may correct the area in the registrable area RO that does not overlap with the non-registrable area RN and the area in the overlapping area that is closer to the registrable area RO to be the registrable area RO. The processor 110 may correct the area in the non-registrable area RN that does not overlap with the registrable area RO and the area in the overlapping area that is closer to the non-registrable area RN to be the non-registrable area RN. The boundary line may be a straight line or a curved line.

[0098] The registerable area RO and the non-registerable area RN have, for example, a circular outer periphery, but may have other shapes. The processor 110 may suppress the occurrence of an overlapping area in which the registerable area RO and the non-registerable area RN overlap by making the shape of the registerable area RO and the non-registerable area RN a shape other than a circle (for example, an ellipse).

[0099] Also, for example, the processor 110 derives (e.g., calculates) the registrable area RO based on the distance between the reference position RP1 derived from each dummy image DG1 and the arrangement position of each dummy image DG1. For example, the processor 110 derives (e.g., calculates) the non-registrable area RN based on the distance between the reference position RP2 derived from each dummy image DG2 and the arrangement position of each dummy image DG2.

[0100] In Fig. 9, the dummy image DG11 and the dummy image DG12 are arranged inside the registrable area RO. That is, the dummy image DG11 and the dummy image DG12 are data that can be registered in the registration database 121 when the quality assessment process is performed. On the other hand, in Fig. 9, the dummy image DG21 and the dummy image DG22 are arranged inside the non-registrable area RN. That is, the dummy image DG21 and the dummy image DG22 are data that cannot be registered in the registration database 121 when the quality assessment process is performed.

[0101] Fig. 10 is a flowchart showing an example of a threshold calculation process according to the third derivation example of the threshold th In Fig. 10, the description of the matters described with reference to Fig. 7 will be omitted or simplified.

[0102] In the face management server 100, the processor 110 refers to the dummy database 122 and extracts feature amounts for all dummy images DG1, DG2 included in the dummy data set DS (S131).

[0103] The processor 110 derives the arrangement position of each dummy image DG1 on the feature space CS based on each feature amount of each dummy image DG1. The processor 110 calculates the reference position RP1 for registration possible on the feature space CS based on the arrangement position of each dummy image DG on the feature space CS. The reference position RP1 for registration possible on the feature space CS may be, for example, the center position (average position) of all dummy images DG1, the center of gravity position of all dummy images DG for registration possible, or other statistical position. The processor 110 calculates the distance between the reference position RP1 for registration possible in the feature space CS and the arrangement position of the dummy image DG1 for registration possible for each dummy image DG1 for registration possible based on the reference position RP1 for registration possible in the feature space CS. The processor 110 calculates the longest distance d1 of the distances calculated for each dummy image DG1 for registration possible (S132).

[0104] The processor 110 derives the area inside a circle CR1 having the longest distance d1 as its radius and centered at the reference position RP1 for registration as the registrable area RO. The processor 110 stores information about the registrable area RO (e.g., the position and size of the registrable area RO, the reference position RP1, and information about the circle CR1) in the threshold database 123 (S133).

[0105] The processor 110 derives the arrangement position of each dummy image DG2 on the feature space CS based on each feature amount of each dummy image DG2. The processor 110 calculates a reference position RP2 for registration not possible on the feature space CS based on the arrangement position of each dummy image DG2 on the feature space CS. The reference position RP2 for registration not possible on the feature space CS may be, for example, the center position (average position) of all dummy images DG2, the center of gravity position of each dummy image DG2, or other statistical position. The processor 110 calculates the distance between the reference position RP2 for registration not possible in the feature space CS and the arrangement position of the dummy image DG2 for each dummy image DG2 based on the reference position RP2 for registration not possible in the feature space CS. The processor 110 calculates the longest distance d11 of the distances calculated for each dummy image DG2 (S134).

[0106] The processor 110 derives the area inside the circle CR2 having the maximum distance d11 as its radius and the reference position RP2 for unregisterable as its center position as the unregisterable area RN. The processor 110 stores information about the unregisterable area RN (e.g., the position and size of the unregisterable area RN, the reference position RP2, and the information about the circle CR2) in the threshold database 123 (S135).

[0107] Fig. 11 is a flowchart showing an example of a quality determination process according to the third derivation example of the threshold value th In Fig. 11, the description of the matters described with reference to Fig. 8 will be omitted or simplified.

[0108] In the registration device 200, the processor 210 extracts each feature amount of the registration candidate image CG (S231). The processor 210 derives (for example, calculates) the position of the registration candidate image CG in the feature space CS based on each extracted feature amount.

[0109] The processor 210 determines whether or not the position in the feature space CS of the registration candidate image CG is included in neither the registrable area RO nor the unregistrable area RN stored in the threshold database 123 (S232). In this case, the processor 210 may inquire of the face management server 100 about the information on the registrable area RO and the unregistrable area RN via the communication device 230, obtain the information on the registrable area RO and the unregistrable area RN from the face management server 100, and determine whether or not the position is included in neither the registrable area RO nor the unregistrable area RN.

[0110] If the position in the feature space CS of the registration candidate image CG is included in either the registration possible area RO or the registration impossible area RN (No in step S232), the processor 210 determines whether the position in the feature space CS of the registration candidate image is included in the registration impossible area RN (S233).

[0111] If the position of the registration candidate image in the feature space CS is not included in the non-registrable region RN (No in step S233), the processor 210 determines that the quality of the registration candidate image is good (quality OK), that is, that the image can be registered (S234). In this case, the processor 210 registers the registration information related to the registration candidate image CG (i.e., the registration image) determined to have good quality by storing the registration information in the registration database 121 of the face management server 100 via the communication device 230.

[0112] On the other hand, if the position in the feature space CS of the registration candidate image CG is not included in either the registerable area RO or the non-registerable area RN (Yes in step S232), or if the position in the feature space CS of the registration candidate image CG is included in the non-registerable area RN (Yes in step S233), the processor 210 determines that the quality of the registration candidate image CG is not good (quality NG), that is, that it is not registerable (S235). Information related to the registration candidate image CG that is determined to be of poor quality is not registered in the registration database 121 of the face management server 100.

[0113] In this way, in the image registration system 5, the face management server 100 can easily derive the registrable area RO and the unregistrable area RN used in the quality judgment process based on the feature amount of each dummy image DG1 for registration and the feature amount of each dummy image DG2 for non-registration in the dummy data set DS, and therefore can easily derive the judgment criteria. The face management server 100 can store information on the registrable area RO and information on the unregistrable area RN as the judgment criteria for quality judgment. Furthermore, the face management server 100 applies the unregistrable area RN preferentially over the registrable area RO, so that if the unregistrable area RN and the registrable area RO overlap, it is judged that registration is not possible, and therefore the registration criteria can be made even stricter.

[0114] In addition, the registration device 200 can perform a quality judgment process on the registration candidate image CG using the objective judgment criteria for whether or not registration is possible, which are derived by the face management server 100, and can therefore objectively judge whether or not the registration candidate image CG can be registered. Therefore, for example, an authentication device that authenticates a person using a registration image can objectively and more accurately perform face authentication of the person.

[0115] <Ideal image display> Next, the selection and presentation of an ideal registration image (also simply called an ideal image) will be described.

[0116] In the registration device 200, when it is determined as a result of the quality determination process that the quality of the registration candidate image is not good (registration is NG), the processor 210 may display an example of the ideal image RG on the display device 250. In this case, the processor 210 may display information (re-imaging guidance information I1) guiding re-imaging, such as "Please capture your face like this", together with the ideal image RG on the display device 250. Fig. 12 is a diagram showing a display example of the ideal image RG.

[0117] In the face management server 100, the processor 110 performs an ideal image selection process to select an ideal image RG. In the registration device 200, the processor 210 performs an ideal image presentation process to display the selected ideal image RG on the display device 250. The ideal image presentation process may be performed in the quality assessment process, and is an example of the process of step S207 in FIG. 3.

[0118] 13 is a flowchart showing an example of the ideal image selection process. The ideal image selection process may be executed in parallel with the threshold calculation process.

[0119] In the face management server 100, the processor 110 refers to the dummy database 122 and extracts feature amounts for all dummy images DG1 that can be registered and are included in the dummy data set DS (S141).

[0120] The processor 110 derives the position of each dummy image DG1 in the feature space CS based on each feature amount of each dummy image DG1. The processor 110 calculates a reference position RP1 for registration in the feature space CS based on the position of each dummy image DG in the feature space CS. The processor 110 determines the dummy image DG1 (for example, the dummy image DG15 in FIG. 14) arranged at the position closest to the reference position RP1 in the feature space CS as the ideal image RG (S142).

[0121] The processor 110 transmits the determined ideal image RG to the registration machine 200 via the communication device 130.

[0122] The processor 110 may select the ideal image RG by a method different from that shown in FIG. 13. For example, the processor 110 may acquire, via the communication device 130, a registration candidate image CG whose quality does not satisfy a threshold th (criterion), that is, whose quality is poor, that is, whose registration is not possible, from the registration machine 200. The processor 110 may determine, as the ideal image RG, a dummy image DG1 (e.g., dummy image DG16 in FIG. 14) that has the longest distance between the arrangement position of the registration candidate image CG (the registration candidate image CG1 in FIG. 14) in the feature space CS acquired from the registration machine 200 and the arrangement position of the dummy image DG1, among the dummy images DG1 included in the registrable area RO.

[0123] Also, for example, the processor 110 may estimate the attribute of the registration candidate image CG and determine the ideal image RG based on the attribute of the registration candidate image CG. In this case, for example, when the face direction as the attribute estimation result of the registration candidate image CG does not satisfy the threshold th, the processor 110 may determine the dummy image DG1 whose face direction satisfies the threshold th as the ideal image RG.

[0124] <Visualization of a dummy dataset> Next, visualization of the dummy dataset DS will be described.

[0125] <First display example of dummy data set> The face management server 100 generates display information DI1 including one or more dummy images DG for each distance from a reference position RP1 derived from the arrangement position of each registrable dummy image DG1 in the feature space CS by the processor 110. The processor 110 causes the display information DI1 to be displayed on the display device of the face management server 100.

[0126] FIG. 15 is a diagram showing an example of the arrangement of the dummy images DG in the feature space CS.

[0127] As shown in Fig. 15, the area RO1 corresponding to distances 1 to 10 from the reference position RP1 of the registration possible area RO includes two dummy images DG1: a dummy image DG11 identified by an identification ID "001" and a dummy image DG11A identified by an identification ID "001-2". The area RO2 corresponding to distances 10 to 20 from the reference position includes a dummy image DG12 identified by an identification ID "002". The area RO3 corresponding to distances 20 to 30 from the reference position includes a dummy image DG12A identified by an identification ID "002-2". The distances here are indicated by a predetermined relative value.

[0128] The distance from the reference position RP1 of the registrable area RO that defines each of the areas RO1 to RO3 may be variable and may be specified by a user via an input device of the face management server 100. Also, the number of areas for each distance from the reference position RP1 of the registrable area RO may be two, four or more, instead of three.

[0129] FIG. 16 is a diagram showing a first display example of the dummy dataset DS.

[0130] In Fig. 16, information indicating areas RO1-RO3 corresponding to the distance from the reference position RP1 of the registrable area RO and one or more dummy images DG1 corresponding to this distance are displayed in association with each other. In Fig. 16, information related to the dummy image DG1 (for example, the dummy image DG1 and attribute information) is included. In Fig. 16, information related to the dummy image DG11 and information related to the dummy image DG11A are displayed in the area RO1. Information related to the dummy image DG12 is displayed in the area RO2. Furthermore, information related to the dummy image DG12A is displayed in the area RO3.

[0131] Since the region RO3 is the outermost of the registrable region RO, the dummy image DG1 included in the region RO3 has a feature amount close to that of the dummy image DG2 included in the non-registrable region RN. In response to this, the processor 110 may specify, via an input device, whether or not to include the dummy image DG1 (e.g., the dummy image DG12A) included in the region RO3 in the dummy data set DS. This allows the user to adjust whether or not to include the dummy image DG belonging to the region RO3 in the dummy data set DS according to the user's intention. In other words, the processor 110 may specify the dummy image DG included in the region RO3 as a registrable image, or as a non-registrable image.

[0132] Furthermore, the processor 110 may specify a distance (range of distance) from the reference position RP1 via an input device, and display a dummy image DG1 according to the specified distance (range of distance) on, for example, a display device included in the face management server 100. This allows a necessary amount of information about the dummy image DG1 to be displayed, allowing the user to easily confirm the information.

[0133] <Second display example of dummy data set> In the face management server 100, the processor 110 generates display information DI2 including attribute information of the dummy image DG and information on the distance of the dummy data from the center of gravity of the registerable area RO in the feature space CS for each dummy image DG included in the dummy data set DS. The processor 110 displays the display information DI2 on the display device of the face management server 100.

[0134] FIG. 17 is a diagram showing a second display example of the dummy data set DS.

[0135] In FIG. 17, a dummy image DG, attribute information of the dummy image DG, and information indicating areas RO1 to RO3 corresponding to the distance of the dummy image DG1 from the reference position RP1 of the registerable area RO on the feature space CS are displayed in association with each other.

[0136] In this way, the face management server 100 can organize and visualize the contents of the dummy data set DS in various display formats, as shown in Fig. 16 and Fig. 17. Furthermore, when a predetermined number or more of dummy images DG1 are classified into an area (e.g., area RO1) corresponding to the distance from the reference position RP1, the processor 110 may delete any of the dummy images DG1 classified into that area. By deleting in this way, the contents of the dummy data set DS may be organized.

[0137] Furthermore, the processor 210 of the registration device 200 may acquire information on the dummy dataset DS via the communication device 230, map the dummy dataset DS and the registration candidate image CG in the feature space CS, and display them on the display device 250. In this way, the registration device 200 can objectively present to the user under what conditions an image can be captured in order to be registered as a registration image, by visualizing the dummy dataset DS and the registration candidate image CG in the feature space CS, rather than presenting information (e.g., attribute information) on the registration candidate image CG only as numerical values.

[0138] 16 and 17, the dummy image DG1 that can be registered as the dummy image DG is visualized, but the present invention is not limited to this. In the visualization of the dummy dataset DS, the dummy image DG2 that cannot be registered as the dummy image DG may be visualized, or both the dummy image DG1 and the dummy image DG2 may be visualized.

[0139] According to such an image registration system 5, the face management server 100 can derive a judgment criterion using objective data, namely, a dummy image, instead of the user's intention. Therefore, the face management server 100 can obtain an objective index for registering an authentication image. In addition, the registration device 200 can judge the quality of the registration candidate image CG using the objective index, and can register the registration candidate image CG that makes it easier to correctly authenticate a person as the authentication image.

[0140] Also, although low-quality images may be input as the registration candidate images CG, the image registration system 5 can distinguish between images with subtle differences (e.g., ID photos and on-site photos) into images that can be registered and images that cannot be registered based on a threshold th (criterion) by performing attribute estimation or feature extraction of the dummy image DG. Therefore, the image registration system 5 does not register registration candidate images CG that cannot be registered, and registers registration candidate images CG that can be registered, so that images with subtle differences are not mixed in, improving authentication accuracy (i.e., the false acceptance rate can be reduced).

[0141] For example, since the imaging environment of on-site photos differs for each site, the face management server 100 can register an image suitable as an authentication image for each site by, for example, preparing a dummy image for each site and deriving a threshold value th. Similarly, since the imaging environment of identification photos and on-site photos differs, the face management server 100 can register an image suitable as an authentication image for the site or lighting by, for example, preparing dummy images separately for lighting and on-site and deriving a threshold value th.

[0142] In this embodiment, the registration candidate image CG is an image of a person's face area, but is not limited to this. For example, when the authentication image is an image of the whole person or an image of a part other than the face, the processor 210 of the registration device 200 may set the image of the whole person or the image of a part other than the face as the registration candidate image CG.

[0143] (Overview of the embodiment) As a result, the present disclosure includes at least the following items. Note that in parentheses, examples of components corresponding to the above-described embodiment are given, but the present disclosure is not limited to these.

[0144] (Item 1) An image registration support device (face management server 100) that supports image registration, comprising a processor (processor 110) and a memory (memory 120) for storing instructions, The processor, Multiple dummy images (dummy images DG) containing people are captured. Deriving a threshold value (threshold value th) for determining the quality of a registration candidate image (registration candidate image CG), which is a candidate image to be registered as an authentication image, based on the acquired multiple dummy images; Image registration support device.

[0145] In this way, the image registration assistance device can derive the determination criteria using objective data, i.e., the dummy image, rather than the user's intention, and can obtain an objective index for registering an authentication image.

[0146] (Item 2) The processor, For each of the dummy images, estimating attributes of the dummy images based on the dummy images; deriving the threshold value based on attributes of each of a plurality of dummy images; Item 1. An image registration support device according to item 1.

[0147] This allows the image registration support device to derive the judgment criteria by taking into account environmental information such as the face direction and illuminance.

[0148] (Item 3) The processor, deriving the threshold value such that all of the values ​​indicating the estimated attributes of the dummy images are within a range of attributes of an image that can be registered as the authentication image; Item 2. An image registration support device.

[0149] This allows the image registration assistance device to use, for example, the range between the maximum and minimum values ​​of the value indicating the attribute of the dummy image as the judgment criterion.

[0150] (Item 4) The threshold value includes information on a boundary value that distinguishes between good and bad quality of the registration candidate image. Item 2. An image registration support device.

[0151] This allows the image registration assistance device to specifically define, by numerical value, the state of an attribute that can identify whether or not the quality is suitable for registration.

[0152] (Item 5) The processor, extracting a feature amount of the dummy image based on the dummy image for each of the dummy images; Deriving the threshold value based on the feature amounts of each of a plurality of dummy images. Item 1. An image registration support device according to item 1.

[0153] This allows the image registration assistance device to derive the judgment criteria by taking into account the feature amounts constituting the dummy image.

[0154] (Item 6) The processor, deriving, for each of the dummy images, a placement position of the dummy image in a feature space (feature space CS) defined by a plurality of feature amounts based on one or more feature amounts of the dummy image; Deriving the threshold value based on the respective arrangement positions of a plurality of dummy images. Item 5. An image registration support device according to item 5.

[0155] This allows the image registration support device to map the dummy image onto a feature space, which is a virtual space configured in multiple dimensions, based on the feature amount of the dummy image. The image registration support device can derive a judgment criterion using the position of the dummy image in the feature space.

[0156] (Item 7) The threshold value includes information regarding a registrable area (registrable area RO) in the feature space that can be registered as the authentication image. Item 6. An image registration support device according to item 6.

[0157] In this way, the image registration support device can determine whether or not an image can be registered by determining whether or not the image can be registered within a specific region in the feature space, and can specify that the interior of the region that can be registered satisfies the determination criterion. Therefore, the image registration support device can obtain an index for whether or not an image can be registered that is easy to understand intuitively by taking into account multiple feature amounts.

[0158] (Item 8) The plurality of dummy images include a plurality of first dummy images (dummy images DG1) that can be registered as the authentication images, The processor, deriving, for each of the first dummy images, a placement position of the first dummy image in the feature space based on one or more feature amounts of the first dummy image; determining the registrable area based on the respective arrangement positions of the plurality of first dummy images; Item 7. An image registration support device according to item 7.

[0159] This allows the image registration assistance device to determine the registrable area taking into account the positional relationship of the registrable dummy images in the feature space.

[0160] (Item 9) The processor, Deriving a reference position (reference position RP1) of the plurality of first dummy images based on the respective arrangement positions of the plurality of first dummy images; Calculating a distance between the reference position and a position of the first dummy image for each of the first dummy images; determining the registrable area based on the reference position and the longest distance (longest distance d1) among the plurality of distances calculated; Item 9. An image registration support device according to item 8.

[0161] This allows the image registration assistance device to specifically derive the registrable area so that the placement positions of all dummy images are included in the registrable area.

[0162] (Item 10) The threshold value includes information regarding a non-registrable region (non-registrable region RN) in the feature space that cannot be registered as the authentication image. Item 6. An image registration support device according to item 6.

[0163] In this way, the image registration support device can determine whether registration is impossible based on a predetermined region in the feature space, and can specify, for example, that the inside of the registration impossible region satisfies the determination criterion. Therefore, the image registration support device can obtain an index for registration impossible that is easy to understand intuitively by taking into account multiple feature amounts.

[0164] (Item 11) The plurality of dummy images include a plurality of second dummy images (dummy images DG2) that cannot be registered as the authentication images, The processor, deriving, for each of the second dummy images, a placement position of the second dummy image in the feature space based on one or more feature amounts of the second dummy image; determining the non-registration area based on the arrangement position of the second dummy image; Item 11. An image registration support device according to item 10.

[0165] This allows the image registration support device to determine the unregistrable area by taking into account the positional relationship of the unregistrable dummy images in the feature space.

[0166] (Item 12) The processor, determining, from among the plurality of first dummy images, a first dummy image as a recommended image (ideal image RG) recommended as the authentication image based on the arrangement positions of the plurality of first dummy images in the feature space; Item 9. An image registration support device according to item 8.

[0167] This allows the image registration support device to recognize, for example, what characteristics the first dummy image has as a registrable image based on the position of the first dummy image in the feature space, and to determine one of the first dummy images as the recommended image.

[0168] (Item 13) The processor, Deriving reference positions of the plurality of first dummy images based on respective arrangement positions of the plurality of first dummy images; Calculating a distance between the reference position and a position of the first dummy image for each of the first dummy images; Among the calculated distances, the first dummy image arranged at a position in the feature space that is the shortest distance (shortest distance d2) from the reference position is determined as the recommended image. Item 13. An image registration support device according to item 12.

[0169] This allows the image registration assistance device to determine an image that is close to the reference position of the registrable area in the feature space as the recommended image, and therefore allows the most typical dummy image to be selected as the recommended image.

[0170] (Item 14) The processor, Acquire feature amounts of registration candidate images having quality that does not satisfy the threshold value; deriving a placement position of the registration candidate image in the feature space based on the feature amount of the registration candidate image; determining, as the recommended image, the first dummy image arranged at a position that is the longest distance (longest distance d3) from the arrangement position of the registration candidate image in the feature space; Item 13. An image registration support device according to item 12.

[0171] In this way, the image registration assistance device can determine a dummy image having characteristics significantly different from the characteristics of the registration candidate image that could not actually be registered as the recommended image. Therefore, when the user checks the recommended image, the user can obtain a more suitable registration candidate image by, for example, changing the user's imaging posture or imaging environment and capturing an image with the camera.

[0172] (Item 15) The processor, Obtaining attributes of a registration candidate image having a quality that does not satisfy the threshold; For each of the dummy images, estimating attributes of the dummy images based on the dummy images; determining the dummy image as the recommended image from among the plurality of dummy images based on the acquired attributes of the registration candidate image and the respective attributes of the plurality of dummy images; Item 13. An image registration support device according to item 12.

[0173] This allows the image registration support device to determine, as the recommended image, a dummy image having attributes (attribute values) significantly different from the attributes (attribute values) of the registration candidate image that could not actually be registered. Therefore, when the user checks the recommended image, the user can obtain a more suitable registration candidate image by, for example, changing the user's imaging posture or imaging environment and capturing an image with the camera.

[0174] (Item 16) The processor, Deriving feature amounts of each of the plurality of dummy images; deriving a position of each of the dummy images in the feature space based on the feature amounts of each of the plurality of dummy images; displaying information about the plurality of dummy images on a display device based on the arrangement positions of the dummy images in the feature space; Item 6. An image registration support device according to item 6.

[0175] This allows the administrator of the image registration support device to confirm the positional relationship of each dummy image in the feature space, and intuitively grasp the features of each dummy image.

[0176] (Item 17) The processor, displaying information about the plurality of dummy images on a display device based on the distances from the reference position to the positions of the respective dummy images; Item 9. An image registration support device according to item 9.

[0177] This allows the administrator of the image registration assistance device to check each dummy image for each distance from the reference position.

[0178] (Item 18) An image registration system (image registration system 5) including a registration device (registration device 200) for registering an image of a person and an image registration support device for supporting the registration of the image, The image registration support device includes: Obtain multiple dummy images with people in them, Derive a threshold value for determining the quality of a registration candidate image, which is a candidate image to be registered as an authentication image, based on the acquired multiple dummy images; The registration machine includes: Acquire the image captured by the camera generating the registration candidate image based on the captured image; obtaining the threshold value from the image registration assistance device; If the quality of the registration candidate image satisfies the threshold, the registration candidate image is stored in a memory. Image registration system.

[0179] This allows the image registration system to derive judgment criteria using objective data, such as dummy images, instead of the user's intentions. Therefore, the image registration support device can obtain objective indices for registering authentication images. In addition, the registration machine can use the objective indices to judge the quality of the registration candidate images, and can register the registration candidate images that are easy to correctly authenticate a person as authentication images.

[0180] (Item 19) When the quality of the registration candidate image does not satisfy the threshold, the registration machine displays re-imaging guidance information (re-imaging guidance information I1) that guides the user to re-imaging the image using the camera. Item 19. The image registration system according to item 18.

[0181] This allows the image registration system to prompt the user to retake an image if the image captured by the registration machine is not suitable as an authentication image.

[0182] (Item 20) The registration machine includes: obtaining a recommended image recommended as the authentication image from the image registration support device; If the quality of the registration candidate image does not satisfy the threshold, display the recommended image. 20. The image registration system according to item 18 or 19.

[0183] This allows the user to check the ideal registration image displayed as the recommended image, and recognizes what image state the image should be in so that it can be registered as an authentication image.

[0184] (Item 21) the threshold value includes information regarding a registrable area in a feature space defined by a plurality of feature amounts that is capable of being registered as the authentication image, The registration machine includes: Extracting features of the registration candidate image; deriving a placement position of the registration candidate image in the feature space based on a feature amount of the registration candidate image; When the arrangement position of the registration candidate image is included in the registrable area in the feature space, the registration candidate image is stored in a memory. Item 19. The image registration system according to item 18.

[0185] This allows the image registration system to use the features of the registration candidate image by the registration machine to register the registration candidate image having features that satisfy the judgment criteria as an authentication image.

[0186] (Item 22) the threshold value includes information regarding a non-registrable area in a feature space defined by a plurality of feature amounts that is not allowed to be registered as the authentication image, The registration machine includes: When the arrangement position of the registration candidate image is not included in the non-registrable area and is included in the registerable area in the feature space, the registration candidate image is stored in the memory. 22. The image registration system according to item 21.

[0187] As a result, the image registration system allows the registration machine to use the features of the registration candidate image, taking into account both non-registrable and registerable areas, and register a registration candidate image having features that meet the determination criteria for registration as an authentication image.

[0188] (Item 23) The registration machine includes: When the arrangement position of the registration candidate image is included in the non-registrable area and also included in the registrable area in the feature space, or when the arrangement position of the registration candidate image is not included in the non-registrable area and also not included in the registrable area, the registration candidate image is not stored in the memory. 23. The image registration system according to item 22.

[0189] In this way, the image registration system can prevent the registration machine from registering a registration candidate image having a feature that does not satisfy the judgment criterion as an authentication image by using the feature of the registration candidate image. In addition, when a registerable area and an unregisterable area overlap, the registration machine can perform a stricter quality judgment by giving priority to the unregisterable area when judging the quality of the registration candidate image.

[0190] (Item 24) An image registration assistance method for assisting in the registration of an image, comprising: Obtaining a plurality of dummy images including a person; A step of deriving a threshold value for determining the quality of a registration candidate image, which is a candidate image to be registered as an authentication image, based on the acquired multiple dummy images; The image registration assistance method includes the steps of:

[0191] As a result, the image registration assistance method can achieve the same effect as item 1.

[0192] Although various embodiments have been described above with reference to the drawings, it goes without saying that the present disclosure is not limited to such examples. It is clear that a person skilled in the art can come up with various modified or amended examples within the scope of the claims, and it is understood that these also naturally belong to the technical scope of the present disclosure. In addition, the components in the above embodiments may be arbitrarily combined within the scope of the invention.

[0193] In addition, the above embodiment may also be applicable to a program that realizes the functions of the image registration assistance method, which is supplied to a computer (e.g., the face management server 100) via a network or various storage media, and which is read and executed by the processor of this computer, as well as to a storage medium on which this program is stored.

[0194] In addition, the above embodiment may also be applicable to a program that realizes the functions of the image registration assistance method, which is supplied to a computer (e.g., the face management server 100) via a network or various storage media, and which is read and executed by the processor of this computer, as well as to a storage medium on which this program is stored. [Industrial Applicability]

[0195] The present disclosure is useful for an image registration support device, an image registration system, an image registration support method, and the like that can obtain an objective index for registering an image for authentication. [Explanation of symbols]

[0196] 5. Image Registration System 100 Face Management Server 110 Processor 120 Memory 130 Communication Devices 200 Registered Machines 210 Processor 220 Memory 230 Communication Devices 240 Camera 250 display devices CS feature space DG, DG1, DG2 dummy images DS Dummy Data Set RO Registerable Area RN Unregisterable Area RP reference position RG Ideal image

Claims

1. An image registration assistance device that assists in the registration of an image, comprising a processor and a memory for storing instructions, The processor, Obtain multiple dummy images with people in them, Deriving a threshold value for determining the quality of a registration candidate image, which is a candidate image to be registered as an authentication image, based on the acquired multiple dummy images; Image registration support device.

2. The processor, For each of the dummy images, estimating attributes of the dummy images based on the dummy images; deriving the threshold value based on attributes of each of a plurality of dummy images; The image registration support device according to claim 1 .

3. The processor, deriving the threshold value such that all of the values ​​indicating the estimated attributes of the dummy images are within a range of attributes of an image that can be registered as the authentication image; The image registration support device according to claim 2.

4. The threshold value includes information on a boundary value that distinguishes between good and bad quality of the registration candidate image. The image registration support device according to claim 2.

5. The processor, extracting a feature amount of the dummy image based on the dummy image for each of the dummy images; Deriving the threshold value based on the feature amounts of each of a plurality of dummy images. The image registration support device according to claim 1 .

6. The processor, deriving, for each of the dummy images, a placement position of the dummy image in a feature space defined by a plurality of feature amounts based on one or more feature amounts of the dummy image; Deriving the threshold value based on the respective arrangement positions of a plurality of dummy images. The image registration support device according to claim 5.

7. The threshold value includes information regarding a registrable area in the feature space that can be registered as the authentication image. The image registration support device according to claim 6.

8. the plurality of dummy images include a plurality of first dummy images that can be registered as the authentication images, The processor, deriving, for each of the first dummy images, a placement position of the first dummy image in the feature space based on one or more feature amounts of the first dummy image; determining the registrable area based on the respective arrangement positions of the plurality of first dummy images; The image registration support device according to claim 7.

9. The processor, Deriving reference positions of the plurality of first dummy images based on respective arrangement positions of the plurality of first dummy images; Calculating a distance between the reference position and a position of the first dummy image for each of the first dummy images; determining the registrable area based on the reference position and the longest distance among the plurality of distances calculated; The image registration support device according to claim 8.

10. The threshold value includes information regarding a non-registrable area in the feature space that cannot be registered as the authentication image. The image registration support device according to claim 6.

11. the plurality of dummy images include a plurality of second dummy images that cannot be registered as the authentication image, The processor, deriving, for each of the second dummy images, a placement position of the second dummy image in the feature space based on one or more feature amounts of the second dummy image; determining the non-registrable area based on the arrangement position of the second dummy image; The image registration support device according to claim 10.

12. The processor, determining, from among the plurality of first dummy images, a first dummy image as a recommended image to be recommended as the authentication image based on the arrangement positions of the plurality of first dummy images in the feature space; The image registration support device according to claim 8.

13. The processor, Deriving reference positions of the plurality of first dummy images based on respective arrangement positions of the plurality of first dummy images; Calculating a distance between the reference position and a position of the first dummy image for each of the first dummy images; determining, as the recommended image, the first dummy image that is arranged at a position in the feature space that has the shortest distance from the reference position among the calculated distances; The image registration support device according to claim 12.

14. The processor, Acquire feature amounts of registration candidate images having quality that does not satisfy the threshold value; deriving a placement position of the registration candidate image in the feature space based on the feature amount of the registration candidate image; determining, as the recommended image, the first dummy image that is arranged at a position that is the longest distance from the arrangement position of the registration candidate image in the feature space; The image registration support device according to claim 12.

15. The processor, Obtaining attributes of a registration candidate image having a quality that does not satisfy the threshold; For each of the dummy images, estimating attributes of the dummy images based on the dummy images; determining the dummy image as the recommended image from among the plurality of dummy images based on the acquired attributes of the registration candidate image and the respective attributes of the plurality of dummy images; The image registration support device according to claim 12.

16. The processor, Deriving feature amounts of each of the plurality of dummy images; deriving a position of each of the dummy images in the feature space based on the feature amounts of each of the plurality of dummy images; displaying information about the plurality of dummy images on a display device based on the arrangement positions of the dummy images in the feature space; The image registration support device according to claim 6.

17. The processor, displaying information about the plurality of dummy images on a display device based on the distances from the reference position to the positions of the respective dummy images; The image registration support device according to claim 9.

18. An image registration system including a registration device that registers an image of a person and an image registration support device that supports the registration of the image, The image registration support device includes: Obtain multiple dummy images with people in them, Derive a threshold value for determining the quality of a registration candidate image, which is a candidate image to be registered as an authentication image, based on the acquired multiple dummy images; The registration machine includes: Acquire the image captured by the camera generating the registration candidate image based on the captured image; obtaining the threshold value from the image registration assistance device; If the quality of the registration candidate image satisfies the threshold, the registration candidate image is stored in a memory. Image registration system.

19. and when the quality of the registration candidate image does not satisfy the threshold, the registration machine displays re-imaging guidance information that guides the user to re-imaging the image using the camera.

20. The image registration system of claim 18.

20. The registration machine includes: obtaining a recommended image recommended as the authentication image from the image registration support device; If the quality of the registration candidate image does not satisfy the threshold, display the recommended image.

20. The image registration system according to claim 18 or 19.

21. the threshold value includes information regarding a registrable area in a feature space defined by a plurality of feature amounts in which the registration candidate image can be registered as the authentication image, The registration machine includes: Extracting features of the registration candidate image; deriving a placement position of the registration candidate image in the feature space based on a feature amount of the registration candidate image; When the arrangement position of the registration candidate image is included in the registrable area in the feature space, the registration candidate image is stored in a memory.

20. The image registration system of claim 18.

22. the threshold value includes information regarding a non-registrable area in a feature space defined by a plurality of feature amounts that is not allowed to be registered as the authentication image, The registration machine includes: When the arrangement position of the registration candidate image is not included in the non-registrable area and is included in the registerable area in the feature space, the registration candidate image is stored in the memory.

22. The image registration system of claim 21.

23. The registration machine includes: When the arrangement position of the registration candidate image is included in the non-registrable area and also included in the registrable area in the feature space, or when the arrangement position of the registration candidate image is not included in the non-registrable area and also not included in the registrable area, the registration candidate image is not stored in the memory.

23. The image registration system of claim 22.

24. An image registration assistance method for assisting in the registration of an image, comprising: Obtaining a plurality of dummy images including a person; A step of deriving a threshold value for determining the quality of a registration candidate image, which is a candidate image to be registered as an authentication image, based on the acquired multiple dummy images; The image registration assistance method includes the steps of:

Images