Terminal device and program

Abstract

To provide a terminal device, or the like, allowing a representative user to easily apply for a service for one of a plurality of users.SOLUTION: A storage unit 330 stores user information including profiles of a plurality of users and face images of the user, the user information relating to the users. An acquisition unit 341 tries to acquire a first captured image and a second captured image captured by a camera 310. A recognition unit 342 recognizes a user concerned from among a plurality of users, on the basis of the captured images acquired by the acquisition unit 341 and the face images included in the user information 331. A display control unit 343 displays an application form according to a result recognized by the recognition unit 342 based on the first captured image, and a result recognized by the recognition unit 342 based on the second captured image.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a terminal device and a program. [Background technology]

[0002] In recent years, reservation systems (reservation application systems) that allow users (customers) to make reservation applications for various facilities from their terminal devices via the Internet, etc. Among users of such reservation systems, for example, there are cases where a representative user (parent user) applies for a service for one of multiple users (child users, etc.) in a family.

[0003] As an example, Patent Document 1 discloses an invention for a reservation system (childcare facility reservation system) that allows users to make reservations to leave their children at childcare facilities. In this reservation system, parents make reservations by inputting information about their children from a terminal device (user terminal) used by the parents. In addition, in reservation systems for beauty salons, music classes, sports classes, language classes, etc., when a parent wishes to have their child receive a service, the parent enters information about the child from a terminal device to make a reservation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 6338310 Summary of the Invention [Problem to be solved by the invention]

[0005] In the conventional reservation system described in the above-mentioned Patent Document 1, the representative user (parent user) had to input information about other users (child users, etc.), which was cumbersome. In particular, in a household with multiple children, different information had to be input for each child who was to receive the service, which made it easy for input errors to occur. In addition, beauty salons and other beauty facilities are often used by families (for example, a mother and her underage daughter), and in such cases, a reservation application must be made for each user in the beauty facility's reservation system, which is also cumbersome. Furthermore, the same problems as those described above arise when using a system for making registration applications to receive various services from public institutions and the like, in addition to making such reservation applications.

[0006] In view of this situation, there has been a demand for a technology that allows a representative user to easily apply for a service for any one of a plurality of users.

[0007] The present invention has been made in consideration of the above-mentioned situation, and aims to provide a terminal device and a program that enable a representative user to easily apply for a service for any one of multiple users. [Means for solving the problem]

[0008] A terminal device according to a first aspect of the present invention comprises: A terminal device having a camera, A storage unit that stores user information for each of a plurality of users, the user information including a user profile and a face image of the user; an acquisition unit that acquires a captured image including a face of a person captured by the camera; a recognition unit that recognizes a user from among the plurality of users based on the captured image acquired by the acquisition unit and each face image included in the user information stored in the storage unit; a display control unit for displaying an application form for applying for a service that the user recognized by the recognition unit wishes to receive, The acquisition unit attempts to acquire a first captured image and a second captured image captured by the camera, the display control unit displays the application form according to a recognition result of the recognition unit based on the first captured image and a recognition result of the recognition unit based on the second captured image. It is characterized by:

[0009] Further, the terminal device according to the above aspect includes a plurality of cameras, The first captured image and the second captured image may be captured by different cameras.

[0010] In addition, the terminal device according to the above aspect is A display unit for displaying the application form is further provided, the camera is composed of a front camera provided on the same surface as the display unit and a rear camera provided on the back surface of the display unit, The first captured image may be an image captured by the front camera, and the second captured image may be an image captured by the rear camera.

[0011] In addition, in the terminal device according to the above aspect, The display control unit may be configured to display the application form with some content already entered based on the profile of the first corresponding user when the recognition unit recognizes a first corresponding user based on the first captured image but no person is included in the second captured image.

[0012] In addition, in the terminal device according to the above aspect, The display control unit may be configured to display the application form with some content already entered based on the profile of the second relevant user when the recognition unit recognizes a first relevant user based on the first captured image and recognizes a second relevant user based on the second captured image.

[0013] In addition, in the terminal device according to the above aspect, The display control unit may be configured to display the application form for multiple users, which consolidates applications for multiple users, when the recognition unit recognizes a first relevant user based on the first captured image and recognizes a second relevant user based on the second captured image.

[0014] In addition, in the terminal device according to the above aspect, The display control unit may be configured to display the application form for multiple users with some content already filled in based on the profile of the first relevant user and the profile of the second relevant user.

[0015] In addition, the terminal device according to the above aspect is and a registration unit that registers new user information including the second captured image in the storage unit when the recognition unit outputs a recognition result indicating that information of a person included in the second captured image is not registered.

[0016] In addition, in the terminal device according to the above aspect, The registration unit may copy at least a portion of information of the profile of the first relevant user recognized by the recognition unit based on the first captured image to the new user information.

[0017] A program according to a second aspect of the present invention comprises: A computer having a camera and a storage unit for storing user information for each of a plurality of users, the user information including a user profile and a face image of the user, an acquisition step of acquiring a photographed image including a face portion of a person photographed by the camera; a recognition step of recognizing a user from among the plurality of users based on the photographed image acquired in the acquisition step and each face image included in the user information stored in the storage unit; executing a display control procedure for displaying an application form for applying for a service that the user recognized in the recognition procedure wishes to receive; In the acquisition step, an attempt is made to acquire a first captured image and a second captured image captured by the camera; In the display control step, the application form is displayed according to a recognition result in the recognition step based on the first captured image and a recognition result in the recognition step based on the second captured image. It is characterized by:

[0018] The above program may be distributed or sold via a computer communication network independently of the computer on which the program is executed. Furthermore, the information recording medium on which the above-mentioned program is recorded is a non-transitory recording medium, and can be distributed and sold independently of a computer. Here, the non-transitory recording medium refers to a tangible recording medium. Examples of non-transitory recording media include compact discs, flexible disks, hard disks, magneto-optical disks, digital video disks, magnetic tapes, and semiconductor memories. Also, the transitory recording medium refers to the transmission medium (propagation signal) itself. Examples of the transitory recording medium include electrical signals, optical signals, and electromagnetic waves. In addition, the temporary storage area refers to an area for temporarily storing data and programs, and is, for example, a volatile memory such as a RAM (Random Access Memory). Effect of the Invention

[0019] According to the present invention, a representative user can easily apply for a service for any one of a plurality of users. [Brief description of the drawings]

[0020] [Figure 1] 1 is a block diagram showing an example of the overall configuration of an application system according to an embodiment of the present invention. [Diagram 2] 1A and 1B are schematic diagrams showing an example of the external appearance of a terminal device according to the present embodiment, in which (a) is a schematic diagram of the front side, and (b) is a schematic diagram of the rear side. [Diagram 3]1 is a block diagram showing an example of a schematic configuration of a typical information processing device in which a terminal device or the like according to the present embodiment is realized. [Figure 4] FIG. 2 is a block diagram showing an example of a functional configuration of a terminal device according to the present embodiment. [Diagram 5] FIG. 4 is a schematic diagram showing an example of user information. [Figure 6] 13(a) and 13(b) are schematic diagrams for explaining a specific shooting situation. [Figure 7] FIG. 13 is a schematic diagram showing an example of an application form at the start of display. [Figure 8] 13 is a schematic diagram showing an example of an application form reflecting input operations by a representative user. [Figure 9] 10 is a flowchart for explaining an application process according to the embodiment. [Figure 10] FIG. 13 is a schematic diagram for explaining an application form according to another embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] An embodiment of the present invention will be described below. In the embodiment of the present invention, an application system for making a reservation request to a beauty facility (beauty salon, etc.) that provides beauty services to users will be described as an example, but the present invention is not limited to such beauty facilities, and can also be applied to application systems for making reservation requests to other facilities that provide various services to users (medical facilities, music classes, sports classes, language classes, etc.). In addition to reservation requests, the present invention can also be applied to application systems for making registration requests to receive various services from public institutions, etc. In addition, in an embodiment of the present invention, an example is described in which a representative user (parent user) applies for a service for one of multiple users (parent user, child user, etc.) in a family. However, the present invention is not limited to such a family, and can also be applied to a case in which a representative user applies for a service for one of multiple users in a close friend group. It should be noted that the following embodiments are for illustrative purposes only and do not limit the scope of the present invention. Therefore, a person skilled in the art may adopt embodiments in which any or all of the elements are replaced with equivalents, and these embodiments are also included in the scope of the present invention.

[0022] (Overall composition) The application system 100 according to an embodiment of the present invention is a system in which a user makes a reservation application for the service he or she wishes to receive at a beauty facility that provides beauty services to users, and is configured to include a management server 200 and a terminal device 300, as shown in FIG. 1. The management server 200 and the terminal device 300 are communicatively connected via the Internet 900. Although only three terminal devices 300 are shown in Fig. 1, this is for ease of understanding of the invention, and in reality, a huge number of terminal devices 300 is included depending on the number of users (the number of representative users).

[0023] The management server 200 is, for example, a server computer that manages information about beauty facilities and transmits and receives necessary information between the terminal device 300 used by the representative user, thereby executing processing in response to a reservation request for a service that any of multiple users within a family wishes to receive.

[0024] The terminal device 300 is, for example, a smartphone with a camera or a portable game device, and is used by a representative user who intends to use the beauty facility with his or her family. As an example, the terminal device 300 has an appearance as shown in Figures 2(a) and (b). Figure 2(a) shows the front side (operation surface side) of the terminal device 300, on which a front camera 311 and a display unit 350 are arranged. Figure 2(b) shows the back side (rear side) of the terminal device 300, on which a rear camera 312 is arranged. Details will be described later, but based on an image including a person's face captured by the front camera 311 and / or the rear camera 312, the terminal device 300 can recognize the relevant user from multiple users in the same family.

[0025] (General configuration of information processing device) A typical information processing device 400 in which the terminal device 300 according to the present embodiment is realized will be described.

[0026] As shown in FIG. 3, the information processing device 400 includes a CPU (Central Processing Unit) 401, a ROM (Read Only Memory) 402, a RAM (Random Access Memory) 403, a NIC (Network Interface Card) 404, an image processing unit 405, an audio processing unit 406, an auxiliary memory unit 407, an interface 408, a photographing unit 409, an operation unit 410, and a display unit 411.

[0027] A CPU 401 controls the overall operation of the information processing device 400, and is connected to each component to exchange control signals and data.

[0028] The ROM 402 stores an IPL (Initial Program Loader) that is executed immediately after power is turned on. When this IPL is executed, a program stored in the auxiliary memory unit 407 is read into the RAM 403 and execution of the program by the CPU 401 begins.

[0029] The RAM 403 is for temporarily storing data and programs, and holds programs and data read from the auxiliary storage unit 407, as well as other data necessary for communication.

[0030] The NIC 404 is used to connect the information processing device 400 to a computer communication network such as the Internet, and may be configured as a 10BASE-T / 100BASE-T standard used in constructing a LAN (Local Area Network), an analog modem for connecting to the Internet using a telephone line, an ISDN (Integrated Services Digital Network) modem, an ADSL (Asymmetric Digital Subscriber Line) modem, a cable modem for connecting to the Internet using a cable television line, or the like.

[0031] The image processing unit 405 processes image data read from the auxiliary storage unit 407 or the like using an image calculation processor (not shown) provided in the CPU 401 or the image processing unit 405, and then records the processed image data in a frame memory (not shown) provided in the image processing unit 405. The image information recorded in the frame memory is converted into a video signal at a predetermined synchronous timing, and is output to the display unit 411 via the interface 408 or the like. In other words, under the control of the CPU 401, the image processing unit 405 generates images required in the progress of processing performed by the information processing device 400, and causes the display unit 411 to display the images.

[0032] The audio processing unit 406 converts audio data read from the auxiliary storage unit 407 or the like into an audio signal, and outputs it to a speaker via the interface 408. In other words, under the control of the CPU 401, the audio processing unit 406 generates audio to be generated during the progress of processing performed by the information processing device 400, and outputs the audio from the speaker.

[0033] The auxiliary storage unit 407 is a hard disk or the like, and stores various programs and various data necessary for controlling the operation of the entire information processing device 400. For example, the auxiliary storage unit 407 stores a program for realizing the terminal device 300 according to the present embodiment. Then, under the control of the CPU 401, the auxiliary storage unit 407 appropriately reads out the stored programs and data, and temporarily stores them in the RAM 403 or the like.

[0034] A photographing unit 409, an operation unit 410, and a display unit 411 are connected to the interface 408. Note that the interface 408 may also transmit and receive necessary information to and from other connected external devices.

[0035] The photographing unit 409 is, for example, a single-focus lens and an image sensor (CMOS: Complementary Metal Oxide Semiconductor or CCD: Charge Coupled Device) with a predetermined number of pixels, and captures still images or moving images.

[0036] The operation unit 410 accepts operation inputs from an operator or the like who uses the information processing device 400 .

[0037] The display unit 411 renders an image according to the image data output by the image processing unit 405 , and displays it to an operator or the like who uses the information processing device 400 .

[0038] Additionally, the information processing device 400 may include a drive unit such as a DVD (Digital Versatile Disc)-ROM instead of the auxiliary storage unit 407. In this case, the information processing device 400 reads out programs and data from a DVD-ROM or the like that is loaded into the drive unit, and operates in the same manner as described above.

[0039] (Embodiment) Hereinafter, the functional configuration of the terminal device 300 according to this embodiment will be described with reference to Fig. 4 to Fig. 8. This terminal device 300 is realized by the above-mentioned information processing device 400. For example, when the information processing device 400 is powered on, a program that causes it to function as the terminal device 300 is executed, and the terminal device 300 according to this embodiment is realized. The management server 200 is also realized by an information processing device 400, but a description thereof will be omitted and the following will describe the terminal device 300, which is the most characteristic feature of this embodiment.

[0040] (Functional configuration of terminal device) 4 is a block diagram showing an example of a functional configuration of the terminal device 300 according to this embodiment. As shown in the figure, the terminal device 300 includes a camera 310, a transmission / reception unit 320, a storage unit 330, a control unit 340, and a display unit 350.

[0041] The camera 310 includes a front camera 311 provided on the same surface as the display unit 350, and a rear camera 312 provided on the back surface of the display unit 350. In other words, the camera 310 is composed of the front camera 311 arranged on the front side as shown in Fig. 2(a) above, and the rear camera 312 arranged on the back side as shown in Fig. 2(b). In the following explanation, it is assumed that the terminal device 300 is incapable of operating (taking pictures) the front camera 311 and the rear camera 312 simultaneously, and a case will be explained in which the front camera 311 and the rear camera 312 are operated alternately (taking pictures alternately) every short period of time (for example, every few hundred milliseconds). However, if the terminal device 300 is capable of operating the front camera 311 and the rear camera 312 simultaneously, it may be possible to simultaneously obtain images taken in two directions (a first shadow image and a second taken image, which will be described later) in parallel.

[0042] It should be noted that the above-mentioned photographing unit 409 can function as such a camera 310.

[0043] The transmitting / receiving unit 320 transmits and receives various information to and from the management server 200 via the Internet 900 . For example, the transmitting / receiving unit 320 accesses the management server 200 and receives from the management server 200 an application form (details of which will be described later) for applying for a service that the user wishes to receive. Furthermore, the transmitting / receiving unit 320 transmits the application form, in which all necessary information has been filled in (set), to the management server 200, as will be described later.

[0044] The above-mentioned NIC 404 or the like can function as such a transmission / reception unit 320.

[0045] The storage unit 330 stores various information necessary for processing in the terminal device 300. For example, the storage unit 330 stores user information 331 about a plurality of users in a family.

[0046] The user information 331 includes, as an example, information as shown in Fig. 5. That is, the user information 331 includes information such as a user ID 331a, a user name 331b, an address 331c, a telephone number 331d, an age 331e, a gender 331f, and a facial image 331g. The user name 331b to the gender 331f are an example of a user profile. The facial image 331g is a file name of a facial image (a photographed image including the user's face) stored in a predetermined folder in the storage unit 330. The facial image 331g is not limited to one for each user, and may be multiple.

[0047] In addition to this user information 331, the storage unit 330 stores the representative user's account information, payment information, etc., which are necessary to use the application system 100.

[0048] The above-mentioned auxiliary storage unit 407 or the like can function as such a storage unit 330.

[0049] 4, the control unit 340 controls the entire terminal device 300. The control unit 340 includes an acquisition unit 341, a recognition unit 342, a display control unit 343, a reception unit 344, an update unit 345, and a registration unit 346.

[0050] The acquisition unit 341 acquires a captured image including a face of a person captured by the camera 310. More specifically, the acquisition unit 341 attempts to acquire a first captured image captured by the front camera 311 and a second captured image captured by the rear camera 312. Specifically, the acquisition unit 341 attempts to acquire a first captured image and a second captured image by alternately operating the front camera 311 and the rear camera 312 (switching the capture operation) every short period of time (for example, every few hundred milliseconds). In addition, in the case where the front camera 311 and the rear camera 312 can be operated simultaneously in the terminal device 300, the acquisition unit 341 may attempt to acquire the first captured image and the second captured image in parallel.

[0051] When the representative user actually uses the terminal device 300 to make a reservation request, the following several situations are assumed. First, in a situation where the representative user wishes to use a beauty salon by himself, for example, as shown in Fig. 6(a), the terminal device 300 is operated without other users being present on the rear side of the terminal device 300 (the side of the rear camera 312, not shown). In this case, the acquisition unit 341 acquires a first captured image including the face of person H1 (representative user) captured by the front camera 311. Note that the acquisition unit 341 also acquires an image captured by the rear camera 312, but since this captured image does not include a person (face), it is not used as the second captured image. Next, in a situation where the representative user wants another user (as an example, a child user) to use a beauty facility, for example, as shown in Fig. 6(b), the representative user operates the terminal device 300 in a state where the target user (child user) is present on the rear side (the side of the rear camera 312, not shown) of the terminal device 300. In this case, the acquisition unit 341 acquires a first captured image including the face of a person H1 (representative user) captured by the front camera 311, and a second captured image including the face of a person H2 (child user) captured by the rear camera 312. In this case, as will be described later, recognition using the second captured image is given priority. In reality, it is possible that another user operates the terminal device 300 on behalf of the representative user. However, to make the invention easier to understand, this embodiment will be described on the assumption that the terminal device 300 is operated by the representative user.

[0052] 4, the recognition unit 342 recognizes a relevant user from among multiple users in a family by calculating the similarity between the photographed image acquired by the acquisition unit 341 and each face image 331g included in the user information 331 stored in the storage unit 330. For example, when the similarity between any one of the face images 331g and the photographed image is the highest and the similarity is higher than a reference value that satisfies the recognition requirement, the recognition unit 342 recognizes that the person included in the photographed image (the person photographed by the camera 310) is a user corresponding to (matching) the face image 331g.

[0053] More specifically, the recognition unit 342 recognizes a first corresponding user based on a first captured image captured by the front camera 311, and recognizes a second corresponding user based on a second captured image captured by the rear camera 312. 6(a) described above, the recognition unit 342 recognizes a first relevant user (representative user) from among multiple users in a family by calculating the similarity between the first captured image acquired by the acquisition unit 341 and each face image 331g included in the user information 331. Note that in this scene, since the captured image captured by the rear camera 312 does not include a person, the recognition unit 342 does not use such a captured image as the second captured image. 6(b), the recognition unit 342 recognizes a first relevant user (representative user) and a second relevant user (child user, etc.) from among a plurality of users in a family by calculating the similarity between the first captured image and the second captured image acquired by the acquisition unit 341 and each face image 331g included in the user information 331. Note that when both the first relevant user and the second relevant user are recognized, the second relevant user is given priority as described below.

[0054] However, in a scene as shown in FIG. 6(b), a situation may occur in which information (such as a facial image) of person H2 is not registered in the user information 331. For example, when a youngest child user is to use a beauty facility for the first time, the information of the child user may not be registered in the user information 331. That is, even if the face of a person (person H2) is included in the second captured image, the recognition unit 342 cannot recognize the user from among multiple users in the family. In this case, the recognition unit 342 cancels the recognition of the first corresponding user and outputs a recognition result indicating that the information of the person included in the second captured image is not registered.

[0055] When the relevant user is recognized by the recognition unit 342, the display control unit 343 displays an application form for applying for a service that the relevant user wishes to receive on the display unit 350. At that time, the display control unit 343 displays on the display unit 350 the application form in which values ​​and the like are set for some of the contents based on the profile of the relevant user in the user information 331 (for example, user name 331b to gender 331f in FIG. 4, and the like). For example, the display control unit 343 displays an application form 510 as shown in Fig. 7 on the display unit 350. This application form 510 includes setting contents consisting of a name 511, a phone number 512, an address 513, adult / child 514, male / female 515, a desired service 516, a desired reservation date 517, and a desired reservation time 518, and a send button SB. When displaying such an application form 510 for the first time, the display control unit 343 sets values, etc. for the name 511 to male / female 515 based on the profile of the corresponding user recognized by the recognition unit 342 (for example, the user name 331b to the gender 331f in Fig. 4, etc.), and displays it on the display unit 350.

[0056] When both the first and second applicable users are recognized by the recognition unit 342, the second applicable user is given priority. This is assumed to be the case as shown in Fig. 6(b) above (a case where the representative user wants other users to use the beauty facility), so the display control unit 343 displays on the display unit 350 an application form with values ​​set for some of the contents (application form 510 as shown in Fig. 7) based on the profile of the second applicable user in the user information 331.

[0057] Furthermore, when the recognition unit 342 recognizes the user, the display control unit 343 may not immediately display the application form on the display unit 350, but may pop up a confirmation screen including a part of the profile of the recognized user (e.g., user name 331b in FIG. 4) on the display unit 350, and may display the application form on the display unit 350 after confirmation from the representative user.

[0058] Returning to FIG. 4, the reception unit 344 receives an input operation from the representative user for adding or changing content in the application form displayed on the display unit 350. That is, the reception unit 344 receives an input operation from the representative user to the application form 510 in which values ​​and the like have been set for some of the contents as shown in Fig. 7 described above. For example, in the application form 510 in Fig. 7, the settings for the desired service 516, the desired reservation date 517, and the desired reservation time 518 are not entered (not set), so the representative user performs an input operation to set values ​​and the like for these contents. The reception unit 344 then receives this kind of input operation. Note that the reception unit 344 may also receive an input operation to change the contents that have already been set (filled in). Furthermore, the above-mentioned display control unit 343 displays an application form reflecting the contents on the display unit 350 in response to the input operation received by the reception unit 344. For example, the display control unit 343 displays an application form 510 in which values ​​and the like have been set for the contents that have not been input (desired service 516, desired reservation date 517, and desired reservation time 518) as shown in Fig. 8. In this state, when the representative user presses the send button SB, the above-mentioned transmitting / receiving unit 320 transmits the application form 510 in which values ​​and the like have been set for all the contents (more specifically, data in which information is listed according to the contents of the application form 510) to the management server 200.

[0059] Returning to Fig. 4, the update unit 345 adds or replaces the captured image as a new facial image in the user information of the relevant user recognized by the recognition unit 342 and stores it. For example, when the send button SB of the application form 510 as shown in Fig. 8 described above is pressed by the representative user, the update unit 345 can determine that the captured image captured by the camera 310 has been correctly recognized, and therefore adds or replaces the captured image as a new facial image in the facial image 331g of the recognized relevant user in the user information 331 and stores it. More specifically, the update unit 345 gives a name to the captured image and stores it in a specified folder, and adds or replaces the name (file name) of the captured image to or stores the name of the captured image in the facial image 331g. In addition, when both the first relevant user and the second relevant user are recognized by the recognition unit 342, as described above, the second captured image is given priority, and the second captured image is added to or replaced as a new facial image and stored in the user information of the second relevant user. In this way, by updating (adding or replacing) the facial image 331g of the relevant user, the update unit 345 can handle, for example, a situation in which the facial expression of a child user gradually changes as the child grows. In other words, since the update unit 345 updates the facial image 331g of the child user every time a reservation request is made for the child user (at the interval at which the child user uses a beauty facility), it is expected that the child user will be correctly recognized even when a child user in the growing period is photographed.

[0060] When the recognition unit 342 does not recognize a person included in a captured image from among the face images 331g of the user information 331, the registration unit 346 registers new user information using the captured image in the storage unit 330. For example, when a user (face) not registered in the user information 331 is photographed by the camera 310, the above-mentioned recognition unit 342 outputs as a recognition result that the information of the person included in the photographed image is not registered, so that the registration unit 346 adds a new record to the user information 331 in Fig. 4, names the photographed image, and stores it in a predetermined folder in the storage unit 330. Then, the registration unit 346 copies the same information from the address 331c, telephone number 331d, etc. of the representative user record (for example, the first record) to the address 331c, telephone number 331d, etc. of the added record, and sets the file name of the photographed image to the face image 331g of the added record.

[0061] The above-mentioned CPU 401 and the like can function as the control unit 340 having such a configuration.

[0062] The display unit 350 is controlled by the display control unit 343 and displays the application form 510 as shown in FIG. 7 and FIG. Furthermore, the display unit 350 may display an image captured by the camera 310 while the above-mentioned recognition unit 342 is attempting to recognize the user.

[0063] The above-mentioned display unit 411 can function as such a display section 350.

[0064] (Terminal device operation) The operation of the terminal device 300 configured as above will be described below with reference to Fig. 9. Fig. 9 is a flow chart showing the flow of application processing executed by the terminal device 300. This application processing is started, for example, after the representative user's terminal device 300 accesses the management server 200 and makes a request for reservation application on the reservation page of the desired beauty facility.

[0065] First, the terminal device 300 receives a blank application form from the management server 200 (step S11). That is, the control unit 340 controls the transmitting / receiving unit 320 to receive the initial application form (an application form with no content filled in) sent from the management server 200.

[0066] The terminal device 300 acquires an image captured by the camera 310 (step S12). That is, the control unit 340 (acquisition unit 341) acquires a captured image that may include a face of a person captured by the camera 310. More specifically, the control unit 340 acquires a first captured image captured by the front camera 311 and a second captured image captured by the rear camera 312.

[0067] The terminal device 300 determines whether or not a person (face) is included in the captured image (step S13). That is, the control unit 340 determines whether or not the captured image captured in step S12 includes a person's face. More specifically, the control unit 340 determines whether or not at least one of the first captured image captured by the front camera 311 and the second captured image captured by the rear camera 312 includes a person's face.

[0068] When the terminal device 300 determines that the captured image does not include a person (step S13; No), the terminal device 300 returns the process to the above-mentioned step S12.

[0069] On the other hand, when it is determined that the photographed image includes a person (step S13; Yes), the terminal device 300 calculates the similarity between the photographed image and each face image 331g of the user information 331 (step S14). For example, in a scene as shown in Fig. 6(a) (a scene in which the representative user is going to use a beauty salon by himself / herself), the control unit 340 (recognition unit 342) calculates the similarity between the first captured image acquired in step S12 above and each face image 331g included in the user information 331. Note that in this scene, since the captured image captured by the rear camera 312 does not include a person, the control unit 340 does not use such a captured image as the second captured image. In addition, in a situation such as that shown in FIG. 6(b) (where the representative user is trying to get other users to use the beauty facility), the control unit 340 calculates the similarity between the first and second captured images acquired in step S12 above and each facial image 331g included in the user information 331.

[0070] The terminal device 300 determines whether or not the user has been recognized (step S15). For example, if any of the facial images 331g has the highest similarity between the captured image and the facial image 331g and the similarity is higher than a reference value that satisfies the recognition requirements, the control unit 340 (recognition unit 342) recognizes that the person included in the captured image (the person photographed by the camera 310) is a user who corresponds (matches) to that facial image 331g.

[0071] When the terminal device 300 determines that the user has been recognized (step S15; Yes), the process proceeds to step S17, which will be described later.

[0072] On the other hand, if it is determined that the user is not recognized (step S15; Yes), new user information is registered using the captured image (step S16). 6(b), a situation may occur in which information (such as a facial image) of person H2 is not registered in the user information 331. That is, even if the face of a person (person H2) is included in the second captured image, the control unit 340 (recognition unit 342) cannot recognize the relevant user from among multiple users in a family. In this case, since information on the person included in the captured image captured in step S12 described above has not been registered, the control unit 340 (registration unit 346) registers new user information using the captured image in the storage unit 330. For example, the control unit 340 adds a new record to the user information 331 in Fig. 4, names the captured image, and stores it in a predetermined folder in the storage unit 330. Then, the control unit 340 copies the same information from the address 331c, telephone number 331d, etc. of the representative user record (for example, the first record) to the address 331c, telephone number 331d, etc. of the added record, and sets the file name of the captured image to the face image 331g of the added record.

[0073] The terminal device 300 displays the application form, some of whose contents have been set, on the display unit 350 (step S17). That is, the control unit 340 (display control unit 343) displays an application form on the display unit 350, in which values ​​and the like are set for some of the contents, based on the profile of the corresponding user in the user information 331 (for example, user name 331b to gender 331f in FIG. 4, etc.). For example, the control unit 340 displays an application form 510 as shown in Fig. 7 on the display unit 350. That is, the control unit 340 displays the application form 510 on the display unit 350 in which values, etc. are set for the name 511 to the male / female 515 based on the profile of the recognized user (for example, the user name 331b to the gender 331f in Fig. 4).

[0074] In response to the input operation, the terminal device 300 displays the application form reflecting the contents on the display unit 350 (step S18). For example, in the application form 510 of FIG. 7, the settings for the desired service 516, the desired reservation date 517, and the desired reservation time 518 are left blank (unset), so the representative user will perform input operations to set values ​​for these settings. Therefore, the control unit 340 (display control unit 343) displays an application form reflecting the contents in response to the input operation by the representative user on the display unit 350. For example, the control unit 340 displays an application form 510 in which values ​​and the like are set for the contents that have not been input (desired service 516, desired reservation date 517, and desired reservation time 518) as shown in Fig. 8 .

[0075] The terminal device 300 determines whether or not the send button has been pressed (step S19). That is, the control unit 340 determines whether or not the send button SB has been pressed by the representative user in the application form 510 as shown in FIG.

[0076] If the terminal device 300 determines that the send button has not been pressed (step S19; No), the terminal device 300 returns the process to the above-mentioned step S18.

[0077] On the other hand, if it is determined that the send button has been pressed (step S19; Yes), the terminal device 300 transmits the application form to the management server 200 (step S20). That is, the control unit 340 controls the transmission / reception unit 320 to transmit an application form 510 with values ​​etc. set for all contents (more specifically, data in which information is listed according to the contents of the application form 510) to the management server 200.

[0078] With this type of application process, the representative user only needs to take a picture of the target user with the camera 310 of the terminal device 300, and an application form with some contents filled in based on the user's profile will be displayed on the display unit 350. Furthermore, at this time, the representative user does not need to switch between the front camera 311 and the rear camera 312, and only needs to take a picture of the target user with the front camera 311 or the rear camera 312, and an application form with some contents filled in based on the user's profile will be displayed on the display unit 350. From this state, the representative user can complete the application form by simply filling in any blanks, and then send the completed application form to the management server 200 by simply pressing the send button.

[0079] As a result, the representative user can easily apply for a service for any one of the multiple users.

[0080] (Other embodiments) In the above embodiment, a case has been described in which priority is given to a second relevant user recognized using a second captured image taken by the rear camera 312 in a scene such as that shown in FIG. 6(b). However, when a first relevant user and a second relevant user are recognized, reservation requests for both users may be made at the same time.

[0081] For example, when the recognition unit 342 recognizes both the first applicable user and the second applicable user, the display control unit 343 displays on the display unit 350 an application form (application form for multiple people) with values ​​etc. set for some contents based on the profile of the first applicable user in the user information 331 (for example, user name 331b to gender 331f in FIG. 4, etc.) and the profile of the second applicable user.

[0082] Specifically, the display control unit 343 displays an application form 520 as shown in Fig. 10 on the display unit 350. This application form 520 allows the details of each relevant user (name 511 to desired service 516) to be switched and displayed using tabs TB, and among these, values ​​etc. are set for name 511 to male / female 515 based on the profile of the recognized relevant user (for example, user name 331b to gender 331f in Fig. 4, etc.). In this application form 520, the desired reservation date 517 and the desired reservation time 518 are shared, but the desired reservation date 517 and the desired reservation time 518 may be set for each user.

[0083] In this case, too, the representative user can complete the application form 520 by simply setting values ​​in the blank settings (each user's desired service 516, desired reservation date 517, and desired reservation time 518), and then can send the completed application form 520 to the management server 200 by simply pressing the send button SB.

[0084] In this case as well, the representative user can easily apply for a service for any one of the multiple users.

[0085] In the above embodiment, the application system 100 for making reservation requests to beauty facilities has been described as an example, but the application system is not limited to such beauty facilities and can also be applied to application systems for making reservation requests to other facilities that provide various services to users (medical facilities, music classes, sports classes, language classes, etc.). In addition to reservation applications, the present invention can also be applied to application systems for making registration applications to receive various services from public institutions and the like.

[0086] In addition, in the above embodiment, a case has been described in which a representative user applies for a service for one of multiple users within a family. However, the present invention is not limited to such a family, and can also be applied to a case in which a representative user applies for a service for one of multiple users within a close friend group.

[0087] As described above, according to the present invention, it is possible to provide a terminal device and a program that allow a representative user to easily apply for a service for any one of a plurality of users. [Explanation of symbols]

[0088] 100 Application System 200 Management Server 300 Terminal Equipment 310 Camera 311 Front Camera 312 Rear Camera 320 Transmitter / Receiver 330 Storage section 331 User Information 340 Control Unit 341 Acquisition Department 342 Recognition part 343 Display control section 344 Reception Department 345 Update Department 346 Registration Department 400 Information processing device 401 CPU 402 ROM 403 RAM 404 NIC 405 Image Processing Unit 406 Audio Processing Unit 407 Auxiliary storage 408 Interface 409 Photography Unit 410 Operation Unit 411 Display unit 900 Internet

Claims

1. A storage unit that stores user information for each of a plurality of users, the user information including a user profile and a face image of the user; An acquisition unit that acquires a photographed image including a face of a person; a recognition unit that recognizes a user from among the plurality of users based on the captured image acquired by the acquisition unit and each face image included in the user information stored in the storage unit; a display control unit for displaying an application form for applying for a service that the user recognized by the recognition unit wishes to receive, the acquiring unit attempts to acquire a first captured image and a second captured image that are captured simultaneously or within a predetermined time period; the display control unit displays different application forms depending on whether the recognition unit recognizes a first corresponding user from the first captured image and does not recognize a second corresponding user from the second captured image or whether the recognition unit recognizes the first corresponding user from the first captured image and recognizes the second corresponding user from the second captured image; A terminal device comprising:

2. The terminal device includes a plurality of cameras, The first captured image and the second captured image are captured by a plurality of cameras having different capture directions.

2. The terminal device according to claim 1 .

3. A display unit for displaying the application form is further provided, the plurality of cameras are composed of a front camera provided on the same surface as the display unit and a rear camera provided on the back surface of the display unit, The first captured image is an image captured by the front camera, and the second captured image is an image captured by the rear camera.

3. The terminal device according to claim 2.

4. The display control unit, when the recognition unit recognizes the first corresponding user based on the first captured image but the second captured image does not include a person, displays the application form with some contents already entered based on the profile of the first corresponding user.

4. The terminal device according to claim 1, wherein the terminal device is a communication device.

5. the display control unit, when the recognition unit recognizes the first corresponding user based on the first captured image and recognizes the second corresponding user based on the second captured image, displays the application form with some contents already entered based on the profile of the second corresponding user.

4. The terminal device according to claim 1, wherein the terminal device is a communication device.

6. The display control unit displays the application form for multiple users that consolidates applications for multiple users when the recognition unit recognizes the first corresponding user based on the first captured image and recognizes the second corresponding user based on the second captured image.

4. The terminal device according to claim 1, wherein the terminal device is a communication device.

7. the display control unit displays the application form for multiple users, some of which has been filled in, based on the profile of the first corresponding user and the profile of the second corresponding user.

7. The terminal device according to claim 6,

8. a registration unit that registers new user information including the second captured image in the storage unit when the recognition unit outputs a recognition result indicating that information of a person included in the second captured image is not registered, 4. The terminal device according to claim 1, wherein the terminal device is a communication device.

9. The registration unit copies at least a part of information of the profile of the first corresponding user recognized by the recognition unit based on the first captured image to the new user information.

9. The terminal device according to claim 8,

10. A computer having a storage unit for storing user information for each of a plurality of users, the user information including a user profile and a face image of the user, An acquisition step of acquiring a photographed image including a face portion of a person; a recognition step of recognizing a user from among the plurality of users based on the photographed image acquired in the acquisition step and each face image included in the user information stored in the storage unit; executing a display control procedure for displaying an application form for applying for a service that the user recognized in the recognition procedure wishes to receive; The acquisition step includes attempting to acquire a first captured image and a second captured image that are captured simultaneously or within a predetermined time period; In the display control step, different application forms are displayed depending on whether the first relevant user is recognized from the first captured image and the second relevant user is not recognized from the second captured image in the recognition step or whether the first relevant user is recognized from the first captured image and the second relevant user is recognized from the second captured image. A program characterized by:

Images