Face photo creation device and program

The facial photo creation device and program address the issue of inconsistent backgrounds and privacy concerns by extracting and approving human regions from user terminal images, creating uniform and secure facial photos.

JP2024164513APending Publication Date: 2024-11-27DAI NIPPON PRINTING CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2023080048
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-15
Publication Date
2024-11-27

AI Technical Summary

Technical Problem

Existing systems require employees to take facial photos at specific locations, leading to inconsistent backgrounds and potential inclusion of privacy or confidential information, making it difficult to create uniform and secure facial photographs.

Method used

A facial photo creation device and program that extracts human regions from user terminal images, allows user approval of extraction results, and combines the region with a predetermined background, enabling creation of uniform facial photos while removing confidential information.

Benefits of technology

Enables the creation of uniform facial photos with secure backgrounds by user approval, preventing incomplete extractions and ensuring privacy compliance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024164513000001_ABST
    Figure 2024164513000001_ABST
Patent Text Reader

Abstract

To provide a face photo creation device, etc. capable of properly creating a face photo from an image taken with a user terminal.SOLUTION: A face photo creation device 3 comprises: extraction means 310 for extracting a person region from a photographic image of a user taken with a user terminal 2; approval acceptance means 320 for accepting an approval of the user about the extraction results of the person region; and face photo creation means 330 for creating a face photo, which is a synthesis of the person region and a first background after approval of the extraction results by the user. The approval acceptance means 320 creates a synthetic image, which is a synthesis of the person region and a second background different from the first background for confirming the extraction results by the user when accepting a user approval about the extraction results of the person region.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a facial photograph creating device and the like. [Background technology]

[0002] Photo booths on the street are sometimes used to take facial photos of employees for use in personnel ledgers, employee ID cards, etc. By taking facial photos in a photo booth with a good shooting environment, it is possible to take facial photos of many employees in a uniform and good shooting environment. Patent Document 1 also proposes a service that handles everything from taking facial photos in a photo booth to managing them. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2009-276962 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in order to take a facial photograph, the employee must go to a location where a passport photo machine is installed, and from the standpoint of convenience, it is desirable to be able to take the photograph using a user terminal such as a smartphone carried by the employee.

[0005] In this case, the photographs can be taken anywhere, but due to differences in the photographing environment, etc., the background of the person area in the photographed image varies, and there is a problem that it is difficult to create a sense of uniformity in the facial photographs of employees, etc. In addition, there is a risk that in addition to employees, things related to privacy or confidential information may be captured in the photographed image.

[0006] The present invention has been made in consideration of the above problems, and has an object to provide a face photo creation device and the like that can suitably create a face photo from an image captured by a user terminal. [Means for solving the problem]

[0007] A first invention for solving the above-mentioned problem is a face photo creation device comprising: an extraction means for extracting a person area from an image of a user captured by a user terminal; an approval receiving means for receiving approval from the user of the extraction result of the person area; and a face photo creation means for creating a face photo by combining the person area with a first background after the user approves the extraction result.

[0008] In the present invention, a person area is extracted from an image captured by a user terminal, and a facial photograph is created by combining the person area with a specified background. This allows a facial photograph of an employee or the like to be created with a uniform background, and also allows removal of any private or confidential information that may appear in the captured image. In addition, depending on the captured image, extraction of the person area may be incomplete, which may result in an undesirable facial photograph. However, in the present invention, the user's approval of the extraction result of the person area is accepted, so that an incompletely extracted person area is prevented from being used in the facial photograph. Due to these effects, the facial photograph creation device of the present invention allows the facial photograph of an employee or the like to be created in an appropriate manner.

[0009] When receiving user approval for the extraction result of the person area, the approval receiving means preferably creates a composite image by combining the person area with a second background different from the first background, for user confirmation of the extraction result. For example, the first background and the second background are each monochromatic and have different colors. When accepting approval for the extraction result of the person region, the person region is composited with a background different from the background of the face photograph so that the user can confirm the extraction result, and the composite image can make it easier to judge whether the extraction result is acceptable or not. The background of the composite image can be a color that is different from the background of a normal face photograph, such as red or pink, so that the person region stands out easily, which helps with the judgment.

[0010] It is desirable that the extraction means uses a machine learning model to extract the person region. Extraction of a person's region from an image captured by a user terminal can be performed with high accuracy using AI (artificial intelligence) by utilizing a machine learning model. Even if the extraction of a person's region is incomplete, the present invention allows the user to approve the extraction result, thereby preventing the incompletely extracted person's region from being used as a facial photograph.

[0011] It is also preferable that, when receiving user's approval for the extraction result of the person region, the approval receiving means creates an image in which the person region is represented in a single color so that the extraction result can be confirmed by the user. This also makes it possible to easily determine whether the extraction result of the person region is acceptable, as in the above case.

[0012] It is preferable that the face photo creation device of the first invention further comprises a re-photograph receiving means for receiving a selection to take a photo of the user again at the user terminal, instead of the approval. The extraction results of the person region are also affected by the shooting conditions such as the shooting environment, camera settings, etc. Unlike when taking a picture with a passport photo machine, these shooting conditions can be easily changed when taking a picture with a user terminal, so if a user is dissatisfied with the extraction results of the person region, they can improve the extraction results of the person region by taking the picture again under different shooting conditions.

[0013] A second invention is a program for causing a computer to function as a facial photo creation device having an extraction means for extracting a person area from an image of a user captured by a user terminal, an approval receiving means for receiving the user's approval of the extraction result of the person area, and a facial photo creation means for creating a facial photo by combining the person area with a first background after the user approves the extraction result. Effect of the Invention

[0014] According to the present invention, it is possible to provide a face photo creation device etc. that can suitably create a face photo from an image captured by a user terminal. [Brief description of the drawings]

[0015] [Figure 1] FIG. 1 shows a face photo creation system 1. [Diagram 2] FIG. 2 shows a configuration of a user terminal 2. [Diagram 3] FIG. 2 is a diagram showing the configuration of a face photo creation device 3. [Figure 4] FIG. 2 is a diagram showing the functions of the face photo creation system 1. [Diagram 5] 4 is a flowchart showing the processing in the face photo creation system 1. [Figure 6] 3 shows an example of a screen of the display unit 23. [Figure 7] Examples of images 201 to 204 for confirming the extraction results of person area A. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[0017] (1. Face photo creation system 1) Fig. 1 is a diagram showing a face photo creation system 1 including a face photo creation device 3 according to an embodiment of the present invention. As shown in Fig. 1, the face photo creation system 1 has a configuration in which a user terminal 2, a face photo creation device 3, a management device 4, etc. are connected so as to be able to communicate with each other via a network.

[0018] The face photo creation system 1 creates and registers a face photo of a user from an image of the user taken by a user terminal 2. In this embodiment, the user is an employee of a company, and the face photo of the user is used for a personnel ledger, employee ID card, etc. However, the application of the face photo creation system 1 is not limited to this.

[0019] The user terminal 2 is a computer (computer terminal) with a camera that performs various operations related to taking pictures of the user, creating face photos, and registering them. In this embodiment, a mobile terminal such as a smartphone or a tablet terminal is used as the user terminal 2.

[0020] Fig. 2 is a diagram showing the configuration of the user terminal 2. As shown in Fig. 2, the user terminal 2 is configured by connecting a control unit 21, a storage unit 22, a display unit 23, an input unit 24, a communication unit 25, a camera 26, an audio input / output unit 27, etc., via a bus or the like. However, the user terminal 2 is not limited to this and can have various appropriate configurations.

[0021] The control unit 21 is composed of a CPU, a ROM, a RAM, etc. The CPU loads programs related to the processing of the user terminal 2 stored in a storage medium such as the storage unit 22 or ROM into a work area on the RAM, executes the programs, and controls each part of the user terminal 2 to realize the processing described below. The ROM is a non-volatile memory, and permanently stores programs such as a boot program and a BIOS, data, etc. The RAM is a volatile memory, and temporarily stores programs and data loaded from the storage unit 22 or ROM, etc., and has a work area used by the control unit 21 to perform various processing.

[0022] The storage unit 22 is a flash memory or the like, and stores programs executed by the user terminal 2 during processing described below, data necessary for executing the programs, an OS (operating system), etc. These programs and data are read and executed by the control unit 21 as necessary.

[0023] The display unit 23 is realized by a touch panel display or the like that also serves as the input unit 24. The display unit 23 displays various screens related to the operation and selection of the user terminal 2, and accepts user instruction inputs and the like by operating the touch panel on the screen.

[0024] The communication unit 25 is a communication interface that mediates communication via a network, and performs communication with the face photo creation device 3, the management device 4, and the like.

[0025] The camera 26 is an area camera having an imaging element such as a CCD (Charged Coupled Device), and the audio input / output unit 27 includes a microphone for audio input and a speaker for audio output.

[0026] Returning to the explanation of Fig. 1, the face photo creation device 3 performs various processes related to the creation of face photos. Fig. 3 is a diagram showing the configuration of the face photo creation device 3. As shown in Fig. 3, the face photo creation device 3 can be realized by a computer configured by connecting, for example, a control unit 31, a storage unit 32, a communication unit 33, etc. via a bus or the like. However, it is not limited to this, and various configurations can be used as appropriate.

[0027] The control unit 31, the storage unit 32, and the communication unit 33 are substantially the same as the above-mentioned control unit 21, the storage unit 22, and the communication unit 25. The storage unit 32 stores a program for causing the face photo creation device 3 to execute the processing described below.

[0028] The management device 4 collects and manages face photographs, and can be realized by a computer having a configuration similar to that shown in FIG.

[0029] (2. Functions of Face Photo Creation System 1) Fig. 4 is a block diagram showing the functions of the face photo creation system 1. As shown in Fig. 4, in the face photo creation system 1, the face photo creation device 3 has an extraction means 310, an approval receiving means 320, a face photo creation means 330, a re-shooting receiving means 340, a registration means 350, and the like.

[0030] The extraction means 310 is a means for causing the control unit 31 of the face photo creation device 3 to extract a person area from an image of the user captured by the user terminal 2. In this embodiment, a machine learning model is used to extract the person area.

[0031] The approval receiving means 320 is a means for the control unit 31 of the face photo creation device 3 to receive the user's approval for the extraction result of the person area. In this embodiment, when the user's approval is received, an image for confirming the extraction result of the person area (such as a composite image 202 in FIG. 7 described later) is created.

[0032] The face photo creation means 330 is a means for causing the control unit 31 of the face photo creation device 3 to create a face photo by combining the person area with a predetermined background (first background) after the user approves the extraction result.

[0033] The re-photographing acceptance means 340 is a means for the control unit 31 of the face photo creation device 3 to accept a selection to take a picture of the user again at the user terminal 2 instead of the above approval.

[0034] The registration means 350 is a means for registering the face photograph created by the face photograph creation means 330 by the control unit 31 of the face photograph creation device 3 in the management device 4 .

[0035] (3. Processing of face photo creation system 1) Next, the processing executed in the face photo creation system 1 will be described with reference to Fig. 5 etc. Fig. 5 is a flowchart showing the processing in the face photo creation system 1, and the processing of S1, S5 to S8, S11 to S15, and S18 to S19 in Fig. 5 is executed by the control unit 31 of the face photo creation device 3 controlling each unit of the face photo creation device 3. The other processing is executed by the control unit 21 of the user terminal 2 controlling each unit of the user terminal 2.

[0036] In this embodiment, data such as company information, employee information, and settings at the time of shooting are stored in a linked manner in the storage unit of the management device 4, and a user who creates and registers a face photograph is notified in advance of the URL (Uniform Resource Locator) and authentication information of a website provided by the management device 4. The company information and authentication information include a company code for identifying the company to which the user belongs, etc.

[0037] 5, the user accesses a website provided by the management device 4 using the user terminal 2, inputs authentication information on the website, and transmits the authentication information from the user terminal 2 to the management device 4. The management device 4 transmits to the user terminal 2 the URL of a website for creating a face photo provided by the face photo creation device 3. This URL is different for each company.

[0038] When a user accesses the website using the user terminal 2, the face photo creation device 3 transmits a captured image to the user terminal 2 (S1). When the user terminal 2 receives the captured image, it activates the camera 26 and displays the captured image on the display unit 23 (S2).

[0039] FIG. 6(a) is an example of this shooting screen. On the shooting screen, a live view image 231 captured by the camera 26 (in-camera) and a shutter button 232 are displayed. The user checks the live view image 231 and selects the shutter button 232. When the shutter button 232 is selected, the user terminal 2 shoots the user (S3) and acquires the captured image of the user. Although not particularly shown, in addition to the live view image 231 and the shutter button 232, the shooting screen also displays auxiliary lines indicating the position of the face, etc., buttons for performing various settings related to the camera 26, and the like. When shooting the user, a screen display for explaining the shooting method can also be performed on the user terminal 2.

[0040] The user terminal 2 transmits the image of the user taken in S3 to the face photo creation device 3 (S4). When the face photo creation device 3 receives the image of the user (S5), it extracts a person area from the image (S6).

[0041] In this embodiment, extraction of a person region is performed by AI using a machine learning model. The machine learning model for extracting a person region is known, and for example, Robust Video Matting (https: / / github.com / PeterL1n / RobustVideoMatting, searched May 12, 2023) can be used.

[0042] Thereafter, the facial photo creation device 3 creates, as confirmation images for confirming the extraction result of the person area, image 201 in which person area A of the photographed image 200 is shown as a monochrome area, composite image 202 in which person area A is composited with a confirmation background (second background) B for confirming the extraction result of person area A, image 203 in which person area A of composite image 202 is shown as a monochrome area, and composite image 204 in which person area A is composited with background C of the facial photo, as shown in FIG. 7 (S7).

[0043] The background C of the facial photograph is predetermined for each company and is constant for each company. The confirmation background B and the background C of the facial photograph are each a single, plain image, and the color is different between these backgrounds B and C. The confirmation background B is a color that makes the person area A stand out easily, such as red or pink, which is different from the background C of a normal facial photograph, and this helps to determine whether the extraction result of the person area A is acceptable or not. It is also preferable to make the confirmation background B a dark color (for example, a darker color than the background C of the facial photograph).

[0044] The face photo creation device 3 creates a confirmation screen including these images 201 to 204, and transmits it to the user terminal 2 (S8). The user terminal 2 receives the confirmation screen, and displays it on the display unit 23 (S9).

[0045] 6(b) is an example of the confirmation screen displayed in S9. In this embodiment, the photographed image 200 and the above-mentioned images 201 to 204 are displayed as selectable thumbnails on the confirmation screen, and the image selected by the user from among these images 200 to 204 (the composite image 202 in the illustrated example) is displayed large in the upper display area.

[0046] The user checks the extraction results of person region A by looking at images 201-204 on the confirmation screen, and if the user determines that the extraction of person region A has been performed without any problems, selects the "Next" button 233 to approve the extraction results. On the other hand, if the user determines that there is a problem with the extraction of person region A, the user selects the "Retake" button 234 instead of approving the extraction results.

[0047] The user terminal 2 transmits these confirmation results (selection results) to the face photo creation device 3 (S10). The face photo creation device 3 receives and accepts the confirmation results (S11).

[0048] If the confirmation result is selection of the "Next" button 233 (approval of the extraction result) (S12; "Next"), the facial photograph creation device 3 trims the photographed image 200 to the size of the facial photograph (S13), and then combines the person area A extracted from the photographed image 200 with the background C of the facial photograph to create a facial photograph (S14). Note that in S13, a setting screen (not shown) for setting the trimming position is displayed, and the user's setting of the trimming position can be accepted.

[0049] On the other hand, if the confirmation result indicates that the "retake" button 234 is selected (S12; "retake"), the face photo creation device 3 returns to S1 and transmits the photographing screen to the user terminal 2 again.

[0050] After creating the facial photo in S14, the facial photo creation device 3 creates a confirmation screen for confirming the facial photo and transmits it to the user terminal 2 (S15). The user terminal 2 receives the confirmation screen and displays it on the display unit 23 (S16).

[0051] 6(c) is an example of the confirmation screen displayed in S16, which displays the facial photograph 300 created in S14. The user checks the finished facial photograph 300 on the confirmation screen and selects the “Register” button 236.

[0052] When the "Register" button 236 is selected on the confirmation screen, the user terminal 2 transmits an instruction to register the facial photo 300 to the facial photo creation device 3 (S17). Upon receiving the registration instruction (S18), the facial photo creation device 3 transmits the facial photo 300 to the management device 4 and performs registration (S19). Note that the reference numeral 235 on the confirmation screen in Fig. 6(c) is a "Back" button for returning to the previous screen.

[0053] As described above, in this embodiment, the person area A is extracted from the photographed image 200 photographed by the user terminal 2, and the face photo 300 is created by combining the person area A with a predetermined background C. This allows the face photo 300 of an employee or the like to have a uniform background, and also allows the removal of any private or confidential information that may appear in the photographed image 200. In addition, depending on the photographed image 200, the extraction of the person area A may be incomplete, and if left as is, there is a risk that the face photo 300 may be undesirable. However, in this embodiment, the extraction result of the person area A is accepted by the user, so that the incompletely extracted person area A can be prevented from being used in the face photo 300. Due to these effects, the face photo creation device 3 of this embodiment allows the face photo 300 of an employee or the like to be created appropriately.

[0054] Furthermore, when accepting approval for the extraction result of person area A, it is possible to make it easier to judge whether the extraction result is acceptable by using a composite image 202 in which person area A is composited with a different confirmation background B that is different from background C of face photograph 300, or images 201, 203 in which person area A is made clearer by converting it to a single color. The confirmation background B can be made a color that is different from background C of normal face photograph 300, such as red or pink, to make person area A stand out, which helps with the judgment.

[0055] Furthermore, in this embodiment, it is possible to extract the person region A from the captured image 200 captured by the user terminal 2 with high accuracy using AI by utilizing a machine learning model. Even if the extraction of the person region A is incomplete, in this embodiment, the extraction result is approved by the user, so that the incompletely extracted person region A can be prevented from being used as the facial photograph 300.

[0056] Furthermore, in this embodiment, a user who is dissatisfied with the extraction result of the person area A can change the shooting conditions and take the photograph again. In other words, the extraction result of the person area A is also affected by shooting conditions such as the shooting environment and camera settings, and unlike when taking a photograph with a passport photo machine, these shooting conditions can be easily changed when taking a photograph with the user terminal 2, so a user who is dissatisfied with the extraction result of the person area A can improve the extraction result of the person area A by changing the shooting conditions and taking the photograph again. When taking a photograph again, it is also possible to cause the user terminal 2 to display on the screen, etc., shooting methods for improving the extraction result of the person area A.

[0057] However, the present invention is not limited to the above embodiment. For example, in this embodiment, the extraction of the person area A is performed by a machine learning model, but the method of extracting the person area A is not limited to this. For example, other known image recognition techniques that extract a person area from the characteristics of a contour line, etc. may be used.

[0058] In addition, the face photo creation device 3 in this embodiment is a terminal on the administrator side of the face photo creation system 1, but it is also possible to use the user terminal 2 as the face photo creation device 3 by downloading a dedicated application program for performing the processing of the face photo creation device 3 to the user terminal 2. In this case, the user terminal 2 has the functions of the face photo creation device 3 described in FIG.

[0059] Although the preferred embodiment of the present invention has been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0060] 1: Face photo creation system 2: User terminal 3: Face photo creation device 4: Management device 200: Photographed image 201, 203, 231: Images 202, 204: Composite images 300:Face photo 310:Extraction means 320: Approval acceptance means 330:Method of creating face photo 340: Retake acceptance means 350: Registration method

Claims

1. An extraction means for extracting a person area from an image of a user captured by a user terminal; an approval receiving means for receiving approval from a user regarding the extraction result of the person region; a face photo creating means for creating a face photo by combining the person area with a first background after a user approves the extraction result; A face photograph creation device comprising:

2. The facial photo creation device according to claim 1, characterized in that, when receiving user approval for the extraction result of the person area, the approval receiving means creates a composite image by combining the person area with a second background different from the first background so that the user can confirm the extraction result.

3. 3. The face photograph creating device according to claim 2, wherein the first background and the second background are each a single color, but different in color.

4. 2. The face photo creation device according to claim 1, wherein the extraction means uses a machine learning model to extract a person area.

5. 2. The face photo creation device according to claim 1, wherein the approval receiving means, when receiving the user's approval for the extraction result of the person area, creates an image in which the person area is represented in a monochrome color so that the user can confirm the extraction result.

6. 2. The face photo creation device according to claim 1, further comprising a re-photographing accepting unit that accepts, instead of the approval, a selection to take a picture of the user again at the user terminal.

7. Computer, An extraction means for extracting a person area from an image of a user captured by a user terminal; an approval receiving means for receiving approval from a user regarding the extraction result of the person region; a face photo creating means for creating a face photo by combining the person area with a first background after a user approves the extraction result; A program for causing a facial photo creation device to function as such.

Citation Information

Patent Citations

  • Identification photograph management system

    JP2009276962A