Server device, control method for server device, and program

Abstract

To provide a server device that further improves the convenience of the wearer of a wearable terminal.SOLUTION: The server device comprises storage means, identification means, generation means, and transmission means. The storage means stores a plurality of user information pieces including the biological information of each of the plurality of users. The identification means identifies the person being imaged by cross checking using the biological information of the person being imaged that is included in the image data captured by a wearable terminal and the stored biological information. The generation means generates important information which is assumed to be important for the proprietor of a mobile terminal on the basis of the attribute information of the proprietor of the mobile terminal and the user information of an identified user. The generation means generates detail information pertaining to the identified person being imaged, on the basis of the user information of the identified user. The transmission means transmits the generated important information to the wearable terminal and transmits the generated detail information to the mobile terminal.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a server device, a control method for a server device, and a program. [Background technology]

[0002] There is technology that utilizes wearable devices.

[0003] For example, U.S. Patent Publication 2009 / 0133664 describes a wearable device capable of presenting virtual content to a wearer for many applications in a healthcare setting. The wearer of the wearable device in U.S. Patent Publication 2009 / 0133664 may be a patient or a healthcare provider. In U.S. Patent Publication 2009 / 0133664, patient medical records are shared among authorized healthcare providers. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2020-505675 Summary of the Invention [Problem to be solved by the invention]

[0005] Wearable devices are being used to provide information, as disclosed in Patent Document 1. A person wearing a wearable device can instantly access information that he or she needs.

[0006] On the other hand, whether the wearable device is a glasses type or a head mounted display (HMD) type, the size of the display area is limited. Therefore, the amount of information provided to the user via the wearable device is limited. As a result, there is a possibility that the information required by the wearer of the wearable device is not provided to the wearer of the wearable device.

[0007] A primary object of the present invention is to provide a server device, a control method for a server device, and a program that contribute to further improving the convenience of a wearable device wearer. [Means for solving the problem]

[0008] According to a first aspect of the present invention, a server device is provided comprising: a storage means for storing a plurality of user information including biometric information of each of a plurality of users; an identification means for identifying a subject from among a plurality of users by performing a matching process using the biometric information of the subject contained in image data captured by a wearable terminal worn by a holder of a mobile terminal and the stored biometric information; a generation means for generating information about the identified subject based on attribute information of the holder of the mobile terminal and the user information of the identified user, the information being important information that is assumed to be important to the holder of the mobile terminal, and detailed information about the identified subject based on the user information of the identified user; and a transmission means for transmitting the generated important information to the wearable terminal and transmitting the generated detailed information to the mobile terminal.

[0009] According to a second aspect of the present invention, there is provided a method for controlling a server device, comprising: a storage step of storing a plurality of user information including biometric information of each of a plurality of users; an identification step of identifying the subject from among the plurality of users by performing a matching process using the biometric information of the subject contained in image data captured by a wearable terminal worn by a holder of a mobile terminal and the stored biometric information; a generation step of generating information about the identified subject based on attribute information of the holder of the mobile terminal and the user information of the identified user, the information being important information that is assumed to be important to the holder of the mobile terminal, and detailed information about the identified subject based on the user information of the identified user; and a transmission step of transmitting the generated important information to the wearable terminal and transmitting the generated detailed information to the mobile terminal.

[0010] According to a third aspect of the present invention, there is provided a program for causing a computer mounted on a server device to execute a storage process for storing multiple user information including biometric information of each of multiple users; an identification process for identifying a subject from among the multiple users by a matching process using the biometric information of the subject contained in image data captured by a wearable terminal worn by a holder of a mobile terminal and the stored biometric information; a generation process for generating information about the identified subject based on attribute information of the holder of the mobile terminal and the user information of the identified user, the information being important information that is assumed to be important to the holder of the mobile terminal, and detailed information about the identified subject based on the user information of the identified user; and a transmission process for transmitting the generated important information to the wearable terminal and transmitting the generated detailed information to the mobile terminal. Effect of the Invention

[0011] According to each aspect of the present invention, a server device, a control method for a server device, and a program are provided that contribute to further improving the convenience of a wearable device wearer. Note that the effects of the present invention are not limited to the above. The present invention may achieve other effects instead of or in addition to the effects. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram for explaining an overview of an embodiment. [Diagram 2] FIG. 2 is a flowchart for explaining an outline of the operation of one embodiment. [Diagram 3] FIG. 3 is a diagram illustrating an example of a schematic configuration of an information processing system according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram for explaining the operation of the information processing system according to the embodiment of the present disclosure. [Diagram 5] FIG. 5 is a diagram illustrating an example of a display of a mobile terminal according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a diagram illustrating an example of a display of the wearable terminal according to an embodiment of the present disclosure. [Figure 7] FIG. 7 is a diagram illustrating an example of a processing configuration of a server device according to an embodiment of the present disclosure. [Figure 8] FIG. 8 is a diagram illustrating an example of a related party management database according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a diagram illustrating an example of a user management database according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a flowchart illustrating an example of the operation of the provided information control unit according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a diagram illustrating an example of a processing configuration of a mobile terminal according to an embodiment of the present disclosure. [Figure 12] FIG. 12 is a sequence diagram illustrating an example of the operation of the information processing system according to the embodiment of the present disclosure. [Figure 13] FIG. 13 is a diagram illustrating an example of a hardware configuration of a server device according to the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] First, an outline of one embodiment will be described. The reference numerals in the drawings are added to each element for convenience as an example to aid in understanding, and the description of the outline is not intended to be limiting. Furthermore, unless otherwise specified, the blocks shown in each drawing represent functional units rather than hardware units. The connection lines between the blocks in each drawing include both bidirectional and unidirectional. The unidirectional arrows are used to diagrammatically indicate the flow of the main signal (data) and do not exclude bidirectionality. In this specification and the drawings, elements that can be similarly described may be given the same reference numerals to avoid redundant description.

[0014] The server device 100 according to an embodiment includes a storage unit 101, a specification unit 102, a generation unit 103, and a transmission unit 104 (see FIG. 1). The storage unit 101 stores a plurality of pieces of user information including biometric information of each of a plurality of users (step S1 in FIG. 2). The specification unit 102 specifies the subject from among the plurality of users by a matching process using the biometric information of the subject included in image data captured by a wearable terminal worn by a holder of a mobile terminal and the stored biometric information (step S2). The generation unit 103 generates important information about the specified subject, which is information about the specified subject and is assumed to be important to the holder of the mobile terminal, based on attribute information of the holder of the mobile terminal and the user information of the specified user (step S3). The generation unit 103 generates detailed information about the specified subject based on the user information of the specified user (step S4). The transmission unit 104 transmits the generated important information to the wearable terminal, and transmits the generated detailed information to the mobile terminal (step S5).

[0015] The server device 100 transmits information that is assumed to be important to the wearer of the wearable device (the owner of the mobile device) to the wearable device. As a result, the wearer of the wearable device can instantly access information necessary for the wearer's work. The server device 100 also transmits detailed information about the person photographed by the wearable device to the mobile device. Unlike wearable devices, mobile devices have virtually no limitations on the display area, etc. As a result, the wearer of the wearable device (the owner of the mobile device) can quickly refer to information necessary for work, even though it is not provided by the wearable device. Such an operation of the server device 10 further improves the convenience of the wearer of the wearable device.

[0016] Specific embodiments will be described in more detail below with reference to the drawings.

[0017] [First embodiment] The first embodiment will be described in more detail with reference to the drawings.

[0018] The information processing system according to the first embodiment is a system that provides information to organizations (those involved in the organization) to which people with different positions belong. In the first embodiment, an example will be described in which the information processing system provides information to those involved in a hospital (e.g., doctors, nurses, hospital staff, security guards, etc.). More specifically, the information providing system provides information on hospital users (e.g., patients) according to the respective positions of those involved in the hospital (e.g., doctors, nurses, hospital staff, security guards, etc.).

[0019] [System configuration] 3 is a diagram showing an example of a schematic configuration of an information processing system according to an embodiment of the present disclosure. As shown in FIG. 3, the information processing system includes a server device 10.

[0020] The server device 10 realizes the main function of the information providing system. For example, the server device 10 is managed and operated by an organization such as a hospital. The server device 10 may be installed in a building of the hospital or the like, or may be installed on a network (on the cloud).

[0021] The server device 10 stores information on hospital personnel (e.g., doctors, nurses, hospital staff, security guards, etc.) and information on hospital users (e.g., patients). In the following description, information on hospital personnel is referred to as "personnel information" and information on hospital users is referred to as "user information."

[0022] For example, the server device 10 stores, as related party information, the name, sex, date of birth, occupation (doctor, nurse, hospital staff, security guard, etc.), affiliation (medical department), and biological information of a doctor or the like.

[0023] The server device 10 stores user information (patient information, medical record information) such as the patient's name, gender, date of birth, address, patient card number, disease name, test results, medication information, appointment information, doctor information, hospital ward information, and biological information.

[0024] Examples of biometric information include data (feature amounts) calculated from physical characteristics unique to an individual, such as a face, fingerprint, voiceprint, veins, retina, or iris pattern. Alternatively, the biometric information may be image data such as a face image or fingerprint image. The biometric information may be any information that includes a user's physical characteristics. In the embodiment disclosed in the present application, a case will be described in which biometric information related to a person's "face" (a face image or feature amounts generated from a face image) is used.

[0025] 3, a hospital staff member wears a wearable terminal 21. For example, the hospital staff member wears a glasses-type device or a goggle-type device (HMD (Head Mounted Display)). The wearable terminal 21 is a device that has functions such as information display function, communication function, and image capture function.

[0026] Furthermore, the hospital staff carries a mobile terminal 22. The hospital staff carries a device such as a smartphone or a tablet.

[0027] The wearable terminal 21 and the mobile terminal 22 are configured to be able to communicate with each other by a short-range wireless communication means such as Bluetooth (registered trademark). The mobile terminal 22 is also configured to be able to communicate with the server device 10 via a wireless LAN (Local Area Network) or a mobile line.

[0028] The configuration of the information processing system shown in Fig. 3 is an example, and is not intended to be limiting. For example, the system may include multiple server devices 10. Load balancing and redundancy may be achieved by the multiple server devices 10.

[0029] [General operation] Next, an outline of the operation of the information processing system according to the first embodiment will be described.

[0030] <Registering related party information> The server device 10 acquires related person information. For example, a hospital staff member such as a doctor operates the mobile terminal 22 to register his / her related person information in the server device 10. Specifically, the hospital staff member registers his / her name, sex, date of birth, occupation, affiliation (medical department), biometric information (face image), etc. in the server device 10.

[0031] When the server device 10 acquires the related person information, it generates a staff ID for identifying the hospital related person. The server device 10 stores the generated staff ID, name, biological information, etc. in a related person management database. The related person management database will be described in detail later.

[0032] The server device 10 then issues the generated staff ID to the mobile terminal 22. The mobile terminal 22 then stores the acquired staff ID therein.

[0033] <User information registration> When a hospital user (patient) visits the hospital for the first time, hospital staff or the like registers the patient's name, sex, date of birth, address, patient card number, biological information, etc. in the server device 10. In addition, the patient's illness name, test results, medication information, appointment information, doctor in charge, hospital ward information, etc. are updated by doctors, nurses, etc.

[0034] The user information is stored in a user management database, the details of which will be described later.

[0035] <Information provision> Hospital personnel, as necessary, perform their duties while wearing the wearable terminal 21 and carrying the mobile terminal 22. For example, a doctor wears the wearable terminal 21 and carries the mobile terminal 22 when making rounds.

[0036] The wearable device 21 captures an image of the wearer's line of sight. The wearable device 21 transmits image data obtained by capturing the image to the mobile device 22 (see FIG. 4). The mobile device 22 transmits the acquired image data and the staff ID stored therein to the server device 10.

[0037] The server device 10 extracts biometric information (face image) from the acquired image data. When the biometric information is extracted from the image data, the server device 10 executes a matching process (authentication process) using the extracted biometric information and the biometric information stored in the user management database. Through the matching process, the server device 10 identifies a patient (a person photographed by the wearable device 21; a hospital user) who is located in front of a person wearing the wearable device 21 (e.g., a doctor, a nurse, a hospital staff member, a security guard).

[0038] When the patient is identified, the server device 10 generates information (hereinafter referred to as provided information) to be provided to the wearer of the wearable device 21 (the hospital personnel corresponding to the staff ID). At that time, the server device 10 generates two pieces of provided information: information to be provided to the hospital personnel via the wearable device 21 and information to be provided to the hospital personnel via the mobile device 22.

[0039] Specifically, server device 10 generates information about the subject identified by the matching process and that is assumed to be important to the wearer of wearable device 21, as information to be provided to hospital personnel via wearable device 21. In the following description, information that is assumed to be important (useful) to the wearer of wearable device 21 and that is provided to the wearer (hospital personnel) via wearable device 21 will be referred to as "important information."

[0040] For example, if a doctor is wearing the wearable device 21, the patient's name and disease name are generated as important information. Alternatively, if a nurse is wearing the wearable device 21, the patient's name and medication information are generated as important information.

[0041] Furthermore, the server device 10 generates detailed information about the identified patient based on the user information of the identified patient, as information to be provided to hospital personnel via the mobile terminal 22. In the following description, the detailed information about the patient identified by the matching process using biometric authentication is referred to as "detailed information."

[0042] For example, all or part of the user information about a patient stored in the user management database corresponds to the "detailed information." In principle, the content of the detailed information does not depend on the attribute information (occupation, etc.) of the wearer of the wearable terminal 21. If the identified patient is the same, the server device 10 generates detailed information with the same content.

[0043] The server device 10 transmits to the mobile terminal 22 important information and detailed information regarding the hospital user (patient) identified by the biometric information.

[0044] The mobile terminal 22 displays the received detailed information on a display etc. For example, the mobile terminal 22 displays as shown in FIG.

[0045] In addition, the mobile terminal 22 transmits the received important information to the wearable terminal 21.

[0046] The wearable terminal 21 displays important information on a display or the like. For example, the wearable terminal 21 displays as shown in Fig. 6. Fig. 6 shows an example of a display on the wearable terminal 21 when the person wearing the wearable terminal 21 visually recognizes the person to be photographed (patient). Also, in Fig. 6, the patient who is the person to be photographed is not shown.

[0047] In this way, server device 10 provides important information according to attribute information (such as occupation) of the wearer of wearable device 21 to the wearer of wearable device 21. Server device 10 also provides detailed information on the person photographed by wearable device 21 to the owner of mobile terminal 22.

[0048] Next, each device included in the information processing system according to the first embodiment will be described in detail.

[0049] [Server device] 7 is a diagram showing an example of a processing configuration (processing module) of the server device 10 according to the embodiment of the present disclosure. Referring to FIG. 7, the server device 10 includes a communication control unit 201, a related party information control unit 202, a user information control unit 203, a provision information control unit 204, and a storage unit 205.

[0050] The communication control unit 201 is a means for controlling communication with other devices. For example, the communication control unit 201 receives data (packets) from the mobile terminal 22. The communication control unit 201 also transmits data to the mobile terminal 22. The communication control unit 201 passes data received from other devices to other processing modules. The communication control unit 201 transmits data acquired from other processing modules to other devices. In this way, the other processing modules transmit and receive data to and from other devices via the communication control unit 201. The communication control unit 201 has a function as a receiving unit that receives data from other devices and a function as a transmitting unit that transmits data to other devices. The transmitting unit transmits the generated important information to the wearable terminal 21 and transmits the generated detailed information to the mobile terminal 22.

[0051] The related person information control unit 202 is a means for mainly controlling the registration of related person information.

[0052] For example, when a hospital staff member operates the mobile terminal 22 to access a specific web page (information registration page), the staff member information control unit 202 acquires staff member information using a GUI (Graphical User Interface) etc. Specifically, the staff member information control unit 202 acquires the name, sex, date of birth, occupation, affiliation (medical department), biological information (face image) etc. of the hospital staff member.

[0053] When a face image is acquired as biometric information, the related person information control unit 202 generates feature amounts from the face image.

[0054] Since existing technology can be used for the process of generating the feature amounts, detailed description thereof will be omitted. For example, the related party information control unit 202 extracts the eyes, nose, mouth, etc. as feature points from the face image. After that, the related party information control unit 202 calculates the position of each feature point and the distance between each feature point as feature amounts (generates a feature vector consisting of multiple feature amounts).

[0055] Furthermore, the related party information control unit 202 generates a staff ID for uniquely identifying the hospital related party. The staff ID may be any information that can uniquely identify the hospital related party. For example, the related party information control unit 202 may assign a unique value each time related party information is acquired, and use the unique value as the staff ID.

[0056] When the biometric information (feature amount) and staff ID are generated, the related person information control unit 202 stores the related person information including the staff ID and the like in the related person management database (see FIG. 8).

[0057] As shown in Fig. 8, the related party management database stores the staff ID, biometric information (facial image, feature amount), name, occupation, etc. of the hospital related party in association with each other. Note that the related party management database shown in Fig. 8 is an example, and is not intended to limit the items to be stored.

[0058] When the related person information is stored in the related person management database, the related person information control unit 202 issues the generated staff ID to the hospital related person. The related person information control unit 202 transmits the staff ID to the mobile terminal 22.

[0059] The user information control unit 203 is a means for mainly controlling the registration of user information (patient information, medical record information).

[0060] The user information control unit 203 acquires the patient's name, sex, date of birth, address, patient card number, biometric information (facial image), etc. from a terminal operated by a hospital staff member or the like. The user information control unit 203 generates features from the facial image. The user information control unit 203 stores the user information including the features in a user management database (see FIG. 9).

[0061] In addition, the user information control unit 203 acquires the patient's disease name, test results, medication information, consultation appointment information, information on the doctor in charge, hospital ward information, etc. from doctors, nurses, etc. The user information control unit 203 updates the user management database using the acquired disease name, etc.

[0062] Note that the registration of user information and the updating of the user management database will be apparent to those skilled in the art and are not within the scope of the present disclosure, so a detailed explanation will be omitted.

[0063] The provided information control unit 204 is a means for controlling the provided information provided to hospital personnel.

[0064] The provided information control unit 204 has functions as a specifying unit (specifying means) and a generating unit (generating means).

[0065] The identification unit identifies the subject from among multiple hospital users (patients) by performing a matching process using the biometric information of the subject contained in image data captured by the wearable terminal 21 worn by the holder of the mobile terminal 22 and the biometric information stored in the user management database.

[0066] The generation unit generates important information about the specified person to be photographed, which is information about the specified person to be photographed, based on the attribute information of the holder of the mobile terminal 22 and the user information of the hospital user specified by the matching process, and which is assumed to be important to the holder of the mobile terminal 22. Furthermore, the generation unit generates detailed information about the specified person to be photographed, based on the user information of the specified user.

[0067] The provided information control unit 204 transmits the generated important information to the wearable terminal 21 via the communication control unit 201. The provided information control unit 204 transmits the generated detailed information to the mobile terminal 22 via the communication control unit 201.

[0068] 10 is a flowchart showing an example of the operation of the provided information control unit 204 according to the embodiment of the present disclosure. The operation of the provided information control unit 204 will be described with reference to FIG.

[0069] The provided information control unit 204 acquires image data captured by the wearable terminal 21, etc., via the mobile terminal 22. When the provided information control unit 204 receives the image data and the staff ID from the mobile terminal 22, it attempts to extract biometric information (face image) from the received image data (biometric information extraction process; step S101).

[0070] Since the facial image extraction process by the provided information control unit 204 can use existing technology, detailed description will be omitted. For example, the provided information control unit 204 may extract a facial image (facial region) from image data using a learning model learned by a CNN (Convolutional Neural Network). Alternatively, the provided information control unit 204 may extract a facial image using a method such as template matching.

[0071] If a face image is not extracted (step S102, No branch), the provided information control unit 204 does not execute any particular process. The provided information control unit 204 discards the received image data and staff ID.

[0072] When a face image is extracted (step S102, Yes branch), the provided information control unit 204 executes a matching process using the extracted face image (biometric information) and the biometric information stored in the user management database (step S103).

[0073] More specifically, the provided information control unit 204 generates a feature from the extracted face image. The provided information control unit 204 sets the generated feature as a collation target, and performs a collation process (1:N collation; N is a positive integer, the same applies below) between the feature and the feature stored in the user management database.

[0074] The provided information control unit 204 calculates the similarity between the feature to be matched and each of the multiple feature on the registration side. The similarity can be calculated using chi-square distance, Euclidean distance, etc. Note that the greater the distance, the lower the similarity, and the closer the distance, the higher the similarity.

[0075] The provided information control unit 204 determines that the matching process has failed if there is no feature among the multiple feature amounts stored in the user management database that has a similarity with the feature amount to be matched that is equal to or greater than a predetermined value.

[0076] The provided information control unit 204 determines that the matching process is successful if there is a feature that has a similarity between the feature to be matched and a predetermined value or more among the multiple feature amounts stored in the user management database. In this case, the hospital user (patient) with the entry with the highest similarity is identified as the person photographed by the wearable device 21.

[0077] If the matching process fails (step S104, No branch), the provided information control unit 204 does not execute any particular process. For example, if the wearable device 21 captures an image of a person other than a patient (for example, a hospital staff member such as a doctor or nurse), the matching process fails because the person is not stored in the user management database.

[0078] If the collation process is successful (step S104, branch Yes), the provided information control unit 204 generates provided information to be provided to the wearer of the wearable terminal 21 (step S105).

[0079] First, the provided information control unit 204 searches the related person management database using the staff ID received from the mobile terminal 22 as a key, and identifies the wearer of the wearable terminal 21.

[0080] The provided information control unit 204 acquires, from the identified entry, attribute information of the wearer who wears the wearable device 21. For example, the provided information control unit 204 acquires the occupation of the wearer (e.g., doctor, nurse, hospital staff, security guard, etc.) from the occupation field of the identified entry.

[0081] The provided information control unit 204 generates important information using the acquired attribute information and the user information of the hospital user identified by the collation process. For example, if the wearer is a doctor, the provided information control unit 204 generates the patient's name and disease name as important information. The provided information control unit 204 generates the important information by referring to table information that stores the occupation type and the items (user information items) adopted as important information in association with each other.

[0082] Furthermore, the provision information control unit 204 generates detailed information by using the user information of the hospital user identified by the matching process. For example, the provision information control unit 204 generates all or part of the information described in the entry identified by the matching process as detailed information.

[0083] The provided information control unit 204 transmits the generated provided information (important information, detailed information) to the mobile terminal 22 (step S106). At that time, the provided information control unit 204 transmits the important information and the detailed information to the mobile terminal 22 while distinguishing between the important information and the detailed information.

[0084] For example, the provided information control unit 204 transmits provided information (important information, detailed information) to the mobile terminal 22, specifying that the important information is information intended for the wearable terminal 21. In this manner, the important information is transmitted to the wearable terminal 21 via the mobile terminal 22.

[0085] The storage unit 205 is a means for storing information necessary for the operation of the server device 10. The storage unit 205 stores a plurality of pieces of user information including the biological information of each of a plurality of users (hospital users).

[0086] [Mobile device] Examples of the mobile terminal 22 include a mobile terminal device such as a smartphone, a mobile phone, a game machine, a tablet, etc. The mobile terminal 22 may be any equipment or device that can accept a user's operation and communicate with the wearable terminal 21 and the server device 10.

[0087] 11 is a diagram showing an example of a processing configuration (processing module) of the mobile terminal 22 according to an embodiment of the present disclosure. Referring to FIG. 11, the mobile terminal 22 includes a communication control unit 301, a registration control unit 302, an image data control unit 303, a provision information control unit 304, and a storage unit 305.

[0088] The communication control unit 301 is a means for controlling communication with other devices. For example, the communication control unit 301 receives data (packets) from the server device 10. The communication control unit 301 also transmits data to the server device 10. The communication control unit 301 passes the data received from the other devices to the other processing modules. The communication control unit 301 transmits data acquired from the other processing modules to the other devices. In this way, the other processing modules transmit and receive data to and from the other devices via the communication control unit 301. The communication control unit 301 has a function as a receiving unit that receives data from the other devices and a function as a transmitting unit that transmits data to the other devices.

[0089] The communication control unit 301 supports short-range wireless communication means such as Bluetooth (registered trademark), and communicates with the wearable terminal 21 via the short-range wireless communication means. Note that a pairing process required when the mobile terminal 22 and the wearable terminal 21 communicate via Bluetooth (registered trademark) is performed, for example, when a hospital staff member starts work.

[0090] The registration control unit 302 is a means for registering the person-in-charge information of the hospital-related person in the server device 10. In response to an operation by the hospital-related person, the registration control unit 302 accesses an information registration page provided by the server device 10. The registration control unit 302 transmits to the server device 10 information (e.g., name, gender, etc.) input by the hospital-related person in response to the GUI of the information registration page.

[0091] The registration control unit 302 stores the staff ID received from the server device 10 therein.

[0092] The image data control unit 303 is a unit that performs control related to image data acquired from the wearable device 21.

[0093] When the image data control unit 303 acquires image data from the wearable device 21, it transmits the acquired image data and the staff ID stored therein to the server device 10.

[0094] The provided information control unit 304 is a unit that controls the provided information received from the server device 10.

[0095] When the provided information control unit 304 receives the provided information consisting of important information and detailed information, it displays the received detailed information on a display etc. For example, the provided information control unit 304 performs a display as shown in FIG.

[0096] In addition, the provided information control unit 304 transmits important information received from the server device 10 to the wearable device 21.

[0097] The storage unit 305 is a means for storing information necessary for the operation of the mobile terminal 22 .

[0098] [Wearable devices] Detailed description of the configuration and operation of the wearable terminal 21 will be omitted because the configuration and operation of the wearable terminal 21 are obvious to those skilled in the art. The wearable terminal 21 captures an image of a predetermined area periodically or at a predetermined timing, and transmits the obtained image data to the mobile terminal 22. The wearable terminal 21 displays important information received from the mobile terminal 22 in its own display area. For example, the wearable terminal 21 may perform a display as shown in FIG. 6.

[0099] [System Operation] Next, the operation of the information processing system according to the first embodiment will be described.

[0100] 12 is a sequence diagram showing an example of the operation of the information processing system according to the embodiment of the present disclosure, with reference to FIG.

[0101] The wearable terminal 21 transmits image data to the mobile terminal 22 (step S01).

[0102] The mobile terminal 22 transmits the image data and the staff ID to the server device 10 (step S02).

[0103] If a hospital user (patient) is captured in the image data, the server device 10 identifies the patient by biometric authentication and generates information to be provided about the identified patient (step S03). At that time, the server device 10 generates important information that is important to the wearer of the wearable device 21 and detailed information including details about the patient.

[0104] The server device 10 transmits the generated information to be provided to the mobile terminal 22 (step S04).

[0105] The mobile terminal 22 displays the detailed information (step S05).

[0106] The mobile terminal 22 transmits the important information to the wearable terminal 21 (step S06).

[0107] The wearable device 21 displays the important information (step S07).

[0108] Below, specific examples of information to be provided to each of the hospital personnel (doctors, nurses, hospital staff, and security guards) will be described.

[0109] <If the hospital staff member is a doctor> When a doctor is wearing the wearable terminal 21, the server device 10 transmits information important to the doctor (for example, the patient's name and the name of the disease) to the wearable terminal 21 via the mobile terminal 22. Furthermore, the server device 10 transmits detailed information about the patient, such as test results and medication information, in addition to the patient's name and the name of the disease, to the mobile terminal 22. The doctor uses the information displayed on the wearable terminal 21 to interview the patient, and refers to the detailed information displayed on the mobile terminal 22 when more detailed information is required.

[0110] <If the hospital staff member is a nurse> When a nurse wears the wearable device 21, the server device 10 transmits information important to the nurse (for example, the patient's name and medication information) to the wearable device 21 via the mobile device 22. The server device 10 also transmits detailed information about the patient, such as the patient's name, disease name, and information about the doctor in charge (such as the doctor's name), to the mobile device 22. The nurse uses the information displayed on the wearable device 21 to provide care for the patient, and refers to the detailed information displayed on the mobile device 22 when more detailed information is required.

[0111] <If the hospital-related person is an in-house employee> When a hospital staff member (e.g., a receptionist) wears the wearable terminal 21, the server device 10 transmits information important to the hospital staff member (e.g., the patient's name and appointment information) to the wearable terminal 21 via the mobile terminal 22. Furthermore, the server device 10 transmits detailed information about the patient, such as the patient's name, appointment information, and the like, to the mobile terminal 22. The hospital staff member uses the information displayed on the wearable terminal 21 to guide the patient to the appointment, and refers to the detailed information displayed on the mobile terminal 22 when more detailed information is required.

[0112] <When the hospital staff is a security guard> When the security guard wears the wearable terminal 21, the server device 10 transmits information important to the security guard (for example, the patient's name) to the wearable terminal 21 via the mobile terminal 22. Furthermore, the server device 10 transmits detailed information about the patient, such as information about the ward in which the patient is admitted, to the mobile terminal 22 in addition to the patient's name. The security guard uses the information displayed on the wearable terminal 21 to detect suspicious individuals, etc. For example, if a person photographed by the wearable terminal 21 is not an inpatient in the hospital, the security guard calls out to the person. Alternatively, if the security guard needs more detailed information about the patient in front of him, he refers to the detailed information displayed on the mobile terminal 22. For example, if a patient is lost in the hospital, the security guard guides the patient to the inpatient ward. Alternatively, the security guard contacts a nurse in the inpatient ward.

[0113] Next, a modification of the first embodiment will be described.

[0114] <Variation 1> The provided information control unit 204 may use the user information and other information when generating important information and detailed information. For example, when providing information to hospital staff, the provided information control unit 204 may generate detailed information including a schedule (examination date, examination time, etc.) related to the doctor in charge of the patient. In other words, the provided information control unit 204 may generate detailed information with different contents according to attribute information of the wearer of the wearable terminal 21.

[0115] <Variation 2> In the above embodiment, the provided information control unit 204 uses the user management database as a database for biometric authentication. However, the provided information control unit 204 may use the related party management database in addition to the user management database for biometric authentication.

[0116] Specifically, if the authentication process (matching process) using the user management database fails, the provided information control unit 204 may execute authentication process (matching process) using the related person management database. The provided information control unit 204 may identify the hospital related person photographed by the wearable device 21 by executing matching process using biometric information stored in the related person management database.

[0117] In this case, the provision information control unit 204 may generate important information or detailed information using the information of the hospital personnel identified by the collation process.

[0118] For example, if the wearer of the wearable device 21 is a security guard, the provided information control unit 204 may generate important information including the name of the hospital personnel identified by the matching process. As a result, the wearable device 21 worn by the security guard can display the name of the person to be authenticated, regardless of whether the person is a hospital personnel or a patient. In other words, a person whose name is not displayed on the wearable device 21 is neither a hospital personnel nor a patient, and is therefore likely to be a suspicious person. The security guard can more reliably and easily detect suspicious people, ensuring safety within the hospital.

[0119] As described above, in the first embodiment, the server device 10 transmits information assumed to be important to the wearer of the wearable terminal 21 (the holder of the mobile terminal 22) to the wearable terminal 21. At that time, the server device 10 transmits important information according to the attribute information of the wearer of the wearable terminal 21 to the wearer of the wearable terminal 21. As a result, the wearer of the wearable terminal 21 can immediately access information that is important (high priority) in his / her work. Furthermore, the server device 10 transmits detailed information about the photographed person to the mobile terminal 22. The size of the display area of ​​the wearable terminal 21 is limited, but there is no substantial limit on the display area of ​​the mobile terminal 22. Therefore, when more information than the information provided by the wearable terminal 21 becomes necessary, the wearer of the wearable terminal 21 can quickly perform his / her work by referring to the detailed information provided by the mobile terminal 22. That is, the cooperation between the wearable terminal 21 and the mobile terminal 22 improves the convenience of the user (the holder of the mobile terminal 22).

[0120] Next, the hardware of each device constituting the information processing system will be described.

[0121] The server device 10 can be configured by an information processing device (so-called a computer), and has the configuration exemplified in Fig. 13. For example, the server device 10 has a processor 311, a memory 312, an input / output interface 313, and a communication interface 314. The components such as the processor 311 are connected by an internal bus or the like, and are configured to be able to communicate with each other.

[0122] However, the configuration shown in Fig. 13 is not intended to limit the hardware configuration of the server device 10. The server device 10 may include hardware not shown, and may not include the input / output interface 313 as necessary. In addition, the number of processors 311 and the like included in the server device 10 is not intended to be limited to the example shown in Fig. 13, and for example, the server device 10 may include multiple processors 311.

[0123] The processor 311 is, for example, a programmable device such as a central processing unit (CPU), a micro processing unit (MPU), or a digital signal processor (DSP). Alternatively, the processor 311 may be a device such as a field programmable gate array (FPGA), or an application specific integrated circuit (ASIC). The processor 311 executes various programs including an operating system (OS).

[0124] The memory 312 is a random access memory (RAM), a read only memory (ROM), a hard disk drive (HDD), a solid state drive (SSD), etc. The memory 312 stores an OS program, application programs, and various data.

[0125] The input / output interface 313 is an interface for a display device and an input device (not shown). The display device is, for example, a liquid crystal display, etc. The input device is, for example, a keyboard, a mouse, a touch panel, etc. that accepts user operations.

[0126] The communication interface 314 is a circuit, a module, etc. that communicates with other devices. For example, the communication interface 314 includes a network interface card (NIC).

[0127] The functions of the server device 10 are realized by various processing modules. The processing modules are realized, for example, by the processor 311 executing a program stored in the memory 312. The program can be recorded in a computer-readable storage medium. The storage medium can be a non-transitory medium such as a semiconductor memory, a hard disk, a magnetic recording medium, or an optical recording medium. That is, the present invention can also be embodied as a computer program product. The program can be downloaded via a network or updated using a storage medium storing the program. The processing modules may also be realized by a semiconductor chip.

[0128] In addition, the wearable terminal 21 and the mobile terminal 22 can also be configured by an information processing device like the server device 10, and since their basic hardware configurations are no different from that of the server device 10, a description thereof will be omitted.

[0129] The server device 10 is equipped with a computer, and functions of the server device 10 can be realized by having the computer execute a program. The server device 10 also executes a control method for the server device 10 by the program. Similarly, the mobile terminal 22, etc. is equipped with a computer, and functions of the mobile terminal 22, etc. can be realized by having the computer execute a program. The mobile terminal 22, etc. also executes a control method for the mobile terminal 22, etc. by the program.

[0130] [Variations] The configuration, operation, etc. of the information processing system described in the above embodiment are merely examples, and are not intended to limit the system configuration, etc.

[0131] In the above embodiment, the configuration and operation of the information processing system have been described using a hospital as an example. However, the information processing system disclosed in the present application is naturally applicable to other organizations, groups, and the like. For example, the information processing system disclosed in the present application can be applied to theme parks and the like. For example, the wearable terminal 21 worn by a security guard displays information on employees working at the theme park and customers who have entered the park after purchasing a legitimate ticket. The security guard guards the park while referring to the displayed information.

[0132] Furthermore, when multiple hospital users (patients) are captured in one image data, the server device 10 may identify each of the subjects. The server device 10 may generate important information and detailed information for each of the identified subjects and provide the information to the wearable terminal 21 and the mobile terminal 22. The wearable terminal 21 may simultaneously display important information about multiple hospital users. At the same time, the mobile terminal 22 may simultaneously display detailed information about multiple hospital users.

[0133] In the above embodiment, the case has been described where server device 10 identifies the wearer of wearable device 21 by using the staff ID. The wearer of wearable device 21 may be identified by biometric authentication. For example, when starting work, a hospital staff member operates mobile device 22 to capture an image of his or her face (taking a so-called selfie). Mobile device 22 transmits the captured face image to server device 10. Server device 10 may identify the hospital staff member by performing a matching process using the captured face image and biometric information stored in the staff member management database.

[0134] In the above embodiment, the case has been described where image data captured by the wearable device 21 is transmitted to the server device 10 via the mobile device 22. However, the wearable device 21 may transmit the acquired image data directly to the server device 10. In this case, the wearable device 21 may acquire a staff ID from the mobile device 22 at the time of startup, etc., and transmit the image data together with the acquired staff ID to the server device 10. Furthermore, when image data is transmitted from the wearable device 21 to the server device 10, the server device 10 may transmit important information to the wearable device 21 without going through the mobile device 22. In this case, the server device 10 may transmit detailed information to the mobile device 22 while transmitting important information to the wearable device 21.

[0135] In the above embodiment, the case where the related party management database and the user management database are configured inside the server device 10 has been described, but these databases may be constructed in an external database server or the like. That is, some of the functions of the server device 10 may be implemented in another device. More specifically, the above-described "provided information control unit (provided information control means)" and the like may be implemented in any of the devices included in the system.

[0136] Although the form of data transmission and reception between the devices (mobile terminal 22, server device 10) is not particularly limited, the data transmitted and received between these devices may be encrypted. Biometric information and the like is transmitted and received between these devices, and in order to appropriately protect this information, it is desirable to transmit and receive encrypted data.

[0137] In the flow charts (flow charts, sequence diagrams) used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps executed in the embodiments is not limited to the order described. In the embodiments, the order of the steps shown in the figures can be changed to a degree that does not interfere with the content, such as executing each process in parallel.

[0138] The above-mentioned embodiments have been described in detail to facilitate understanding of the present disclosure, and it is not intended that all of the configurations described above are necessary. In addition, when a plurality of embodiments are described, each embodiment may be used alone or in combination. For example, it is possible to replace a part of the configuration of an embodiment with the configuration of another embodiment, or to add the configuration of another embodiment to the configuration of an embodiment. Furthermore, it is possible to add, delete, or replace a part of the configuration of an embodiment with another configuration.

[0139] From the above explanation, it is clear that the present invention has industrial applicability, and the present invention can be suitably applied to an information processing system that provides information to a user.

[0140] A part or all of the above-described embodiments can be described as, but is not limited to, the following supplementary notes.

[0141] [Appendix 1] A storage means for storing a plurality of pieces of user information including biometric information of each of the plurality of users; an identification means for identifying the person to be photographed from among the plurality of users by a matching process using biometric information of the person to be photographed included in image data captured by a wearable device worn by a person carrying the mobile device and the stored biometric information; a generating means for generating information about the specified photographed person, the information being important information that is assumed to be important to the owner of the mobile terminal, based on attribute information of the owner of the mobile terminal and user information of the specified user, and detailed information about the specified photographed person, based on the user information of the specified user; A transmission means for transmitting the generated important information to the wearable device and transmitting the generated detailed information to the mobile device; A server device comprising: [Appendix 2] The server device according to claim 1, wherein the identification means acquires image data captured by the wearable device via the mobile device. [Appendix 3] 3. The server device according to claim 2, wherein the transmission means transmits the important information to the wearable device via the mobile device. [Appendix 4] The server device according to any one of appendix 1 to 3, wherein each of the plurality of users is a patient of a hospital, and the owner of the mobile terminal is an affiliate of the hospital. [Appendix 5] The server device according to claim 4, wherein the generating means generates the important information based on the occupations of the hospital personnel and user information of the identified user. [Appendix 6] The server device according to claim 4, wherein the generation means generates all or part of the user information of the identified user as the detailed information. [Appendix 7] a storage step of storing a plurality of pieces of user information including biometric information of each of the plurality of users; an identification process for identifying the subject from among the plurality of users by a matching process using biometric information of the subject included in image data captured by a wearable device worn by a user of the mobile device and the stored biometric information; a generating step of generating information about the specified person to be photographed, the information being important information that is assumed to be important to the owner of the mobile terminal, based on attribute information of the owner of the mobile terminal and user information of the specified user, and generating detailed information about the specified person to be photographed, based on the user information of the specified user; a transmission step of transmitting the generated important information to the wearable device and transmitting the generated detailed information to the mobile device; A method for controlling a server device comprising: [Appendix 8] The method for controlling a server device according to claim 7, wherein the identifying step acquires image data captured by the wearable terminal via the mobile terminal. [Appendix 9] 9. The method for controlling a server device according to claim 8, wherein the transmitting step transmits the important information to the wearable device via the mobile device. [Appendix 10] A method for controlling a server device according to any one of appendices 7 to 9, wherein each of the plurality of users is a patient of a hospital, and the owner of the mobile terminal is an affiliate of the hospital. [Appendix 11] The server device control method according to claim 10, wherein the generating step generates the important information based on the occupations of the hospital's personnel and user information of the identified user. [Appendix 12] The server device control method according to claim 10, wherein the generating step generates all or part of the user information of the identified user as the detailed information. [Appendix 13] A computer installed in the server device A storage process for storing a plurality of pieces of user information including biometric information of each of the plurality of users; an identification process for identifying the subject from among the plurality of users by a matching process using biometric information of the subject included in image data captured by a wearable device worn by a user of the mobile device and the stored biometric information; a generation process for generating information about the specified photographed person, the information being important information that is assumed to be important to the owner of the mobile terminal, based on attribute information of the owner of the mobile terminal and user information of the specified user, and generating detailed information about the specified photographed person, based on the user information of the specified user; a transmission process of transmitting the generated important information to the wearable device and transmitting the generated detailed information to the mobile device; A program for executing the above. [Appendix 14] The program described in Appendix 13, wherein the specific processing acquires image data captured by the wearable terminal via the mobile terminal. [Appendix 15] The program described in Appendix 14, wherein the transmission process transmits the important information to the wearable terminal via the mobile terminal. [Appendix 16] The program described in any one of appendix 13 to 15, wherein each of the multiple users is a patient of a hospital, and the owner of the mobile terminal is an affiliate of the hospital. [Appendix 17] The program described in Appendix 16, wherein the generation process generates the important information based on the occupations of the hospital's staff and the user information of the identified user. [Appendix 18] The program described in Appendix 16, wherein the generation process generates all or part of the user information of the identified user as the detailed information.

[0142] In addition, some or all of the configurations described in Supplementary Notes 2 to 6 that are dependent on Supplementary Note 1 may also be dependent on Supplementary Note 7 and Supplementary Note 13 in the same dependent relationship as Supplementary Note 2 to Supplementary Note 6. Furthermore, not limited to Supplementary Note 1, Supplementary Note 7, and Supplementary Note 13, various hardware, software, various recording means for recording software, or systems may similarly be made to be dependent on some or all of the configurations described as the supplementary notes, within the scope of each of the above-mentioned embodiments.

[0143] The disclosures of the above cited prior art documents are incorporated herein by reference. Although the embodiments of the present invention have been described above, the present invention is not limited to these embodiments. It will be understood by those skilled in the art that these embodiments are merely illustrative and that various modifications are possible without departing from the scope and spirit of the present invention. In other words, the present invention naturally includes various modifications and amendments that a person skilled in the art can make in accordance with the entire disclosure, including the scope of the claims, and the technical ideas. [Explanation of symbols]

[0144] 10. Server equipment 21 Wearable devices 22 Mobile devices 100 Server device 101 Memory means 102 Specific means 103 Generation means 104 Transmission means 201 Communication control section 202 Personnel Information Control Unit 203 User Information Control Unit 204 Provision information control section 205 Storage section 301 Communication Control Unit 302 Registration control section 303 Image data control section 304 Provision information control section 305 Storage section 311 Processor 312 Memory 313 Input / Output Interface 314 Communication Interface

Claims

1. A storage means for storing a plurality of pieces of user information including biometric information of each of the plurality of users; an identification means for identifying the person to be photographed from among the plurality of users by a matching process using biometric information of the person to be photographed included in image data captured by a wearable device worn by a person carrying the mobile device and the stored biometric information; a generating means for generating information about the specified photographed person, the information being important information that is assumed to be important to the owner of the mobile terminal, based on attribute information of the owner of the mobile terminal and user information of the specified user, and detailed information about the specified photographed person, based on the user information of the specified user; A transmission means for transmitting the generated important information to the wearable device and transmitting the generated detailed information to the mobile device; A server device comprising:

2. The server device according to claim 1 , wherein the identification unit acquires image data captured by the wearable device via the mobile device.

3. The server device according to claim 2 , wherein the transmission means transmits the important information to the wearable device via the mobile device.

4. The server device according to claim 1 , wherein each of the plurality of users is a patient of a hospital, and the owner of the mobile terminal is a person associated with the hospital.

5. The server device according to claim 4 , wherein the generating means generates the important information based on the occupations of the hospital personnel and user information of the identified users.

6. The server device according to claim 4 , wherein the generating means generates all or a part of the user information of the identified user as the detailed information.

7. a storage step of storing a plurality of pieces of user information including biometric information of each of the plurality of users; an identification process for identifying the subject from among the plurality of users by a matching process using biometric information of the subject included in image data captured by a wearable device worn by a user of the mobile device and the stored biometric information; a generating step of generating information about the specified person to be photographed, the information being important information that is assumed to be important to the owner of the mobile terminal, based on attribute information of the owner of the mobile terminal and user information of the specified user, and generating detailed information about the specified person to be photographed, based on the user information of the specified user; a transmission step of transmitting the generated important information to the wearable device and transmitting the generated detailed information to the mobile device; A method for controlling a server device comprising:

8. A computer installed in the server device A storage process for storing a plurality of pieces of user information including biometric information of each of the plurality of users; an identification process for identifying the subject from among the plurality of users by a matching process using biometric information of the subject included in image data captured by a wearable device worn by a user of the mobile device and the stored biometric information; a generation process for generating information about the specified photographed person, the information being important information that is assumed to be important to the owner of the mobile terminal, based on attribute information of the owner of the mobile terminal and user information of the specified user, and generating detailed information about the specified photographed person, based on the user information of the specified user; a transmission process of transmitting the generated important information to the wearable device and transmitting the generated detailed information to the mobile device; A program for executing the above.

Images