Information processing device, information processing method, program, and information provision system

Abstract

To provide an information processing device, information processing method, program, and information provision system that can appropriately provide users with information for communicating with the person in front of them in organizations such as schools and companies. [Solution] The information processing device is wearable by the user and comprises a display unit that displays information, an imaging unit that captures a facial image of the person the user is looking at, an acquisition unit that acquires personal information including the name of the person identified using the captured facial image and information indicating the organization to which that person belongs, and a display processing unit that displays the acquired personal information on the display unit.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, a program, and an information providing system.

Background Art

[0002] Patent Document 1 describes an example of smart glasses that identify a user in the line of sight (care recipient) in a facility for the elderly or a medical institution, and display personal information of the user in the line of sight (basic information such as name, age, degree of care required, date of birth, room number, etc., behavior information, messages, etc.) on a display. The technology described in Patent Document 1 improves the quality of care services and reduces the burden on caregivers by efficiently transmitting and sharing information such as the characteristics of care recipients, their condition on a given day, and messages among caregivers in a facility for the elderly or the like.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In organizations such as schools and companies, the increase in online classes and remote work has reduced the opportunities for people to communicate face-to-face directly. As a result, when people meet face-to-face on campus or in the office, it is becoming increasingly difficult to recall information about the person in front of them. Since the technology described in Patent Document 1 above efficiently shares information indicating the daily conditions of care recipients in a facility for the elderly or the like among caregivers, it cannot address such problems.

Means for Solving the Problems

[0005] An information processing apparatus according to one aspect of the present disclosure is A means for displaying information, An imaging means for capturing an image of the face of the person the user is looking at, A means for acquiring personal information, including the name of a person identified using a facial image of the person captured, and information indicating the organization to which that person belongs. A display processing means for displaying the acquired personal information on the display means, Equipped with, It can be attached to the aforementioned user.

[0006] The information provision system in one aspect of this disclosure is The user wears facial recognition glasses, The system comprises a server device connected to the aforementioned facial recognition glasses and having a database within the organization, The aforementioned facial recognition glasses are A means for displaying information, An imaging means for capturing an image of the face of the person the user is looking at, An acquisition means that transmits the captured facial image of the person to the server device and compares it with images registered in the database to obtain personal information from the server device, including the name of the identified person and information indicating the organization to which that person belongs. The system comprises a display processing means for displaying the acquired personal information on the display means, The server device is A means for identifying a person by comparing the facial image obtained from the facial recognition glasses with the facial image registered in the database, The system includes a transmission means for transmitting personal information, including the name of the identified person and information indicating the organization to which that person belongs, to the facial recognition glasses.

[0007] The information processing method in one aspect of this disclosure is: A means for displaying information, One or more computers, comprising an imaging means, The aforementioned imaging means is instructed to capture an image of the face of the person the user is looking at. Using the captured facial image of the person, personal information including the name of the person identified and information indicating the organization to which that person belongs is obtained. This includes displaying the acquired personal information on the display means.

[0008] One aspect of this disclosure is that the program is A means for displaying information, A computer equipped with an imaging means, The imaging means performs an imaging process to capture an image of the face of the person the user is looking at. An acquisition process that obtains personal information including the name of the person identified using the captured facial image of the person and information indicating the organization to which that person belongs. This is a program for performing a display process that causes the acquired personal information to be displayed on the display means. [Effects of the Invention]

[0009] According to one example of this disclosure, it is possible to obtain an information processing device, information processing method, program, and information provision system that can appropriately provide users with information for communicating with a person in front of them in organizations such as schools and companies. [Brief explanation of the drawing]

[0010] [Figure 1] This is a functional block diagram showing an example configuration of the information processing device related to this disclosure. [Figure 2] This flowchart shows an example of the processing operation of the information processing device related to this disclosure. [Figure 3] This diagram conceptually illustrates an example of the system configuration of the information provision system related to this disclosure. [Figure 4] This figure shows an example of the configuration of a computer that implements the information processing device described herein. [Figure 5] This is a functional block diagram showing an example configuration of the server device related to this disclosure. [Figure 6] This is a diagram illustrating examples of various types of information contained in a storage device. [Figure 7] It is a diagram showing an example of the data structure of employee information. [Figure 8] It is a flowchart showing an example of the processing operation of the server device according to the present disclosure. [Figure 9] It is a diagram showing an example of the data structure of communication-related information. [Figure 10] It is a functional block diagram showing a configuration example of the second information processing device according to the present disclosure. [Figure 11] It is a flowchart showing an example of a partial processing operation of the information processing device according to the present disclosure. [Figure 12] It is a diagram conceptually showing another system configuration example of the information providing system according to the present disclosure.

Mode for Carrying Out the Invention

[0011] Hereinafter, in the present disclosure, the drawings are associated with one or more embodiments. Also, in all the drawings, the same reference numerals are given to the same components, and the description thereof will be omitted as appropriate. Further, in each of the following drawings, the configuration of the part not related to the essence of the present disclosure is omitted and not shown.

[0012] In the present disclosure, "acquisition" includes at least one of the actions of the self-device going to obtain data or information stored in other devices or storage media (active acquisition), and the self-device inputting data or information output from other devices (passive acquisition). Examples of active acquisition include requesting or inquiring other devices and receiving their replies, and accessing other devices or storage media to read data. Examples of passive acquisition include receiving information distributed (or transmitted, push-notified, etc.). Further, "acquisition" may also mean selecting and acquiring from the received data or information, or selecting and receiving the distributed data or information.

[0013] <Functional Configuration Example> As shown in Figure 1, the information processing device 100 includes a display unit 102, an imaging unit 104, an acquisition unit 106, and a display processing unit 108. The display unit 102 displays information. The imaging unit 104 captures a facial image of the person the user is looking at. The acquisition unit 106 acquires personal information, including the name of the person identified using the captured facial image of the person, and information indicating the organization to which that person belongs. The display processing unit 108 displays the acquired personal information on the display unit 102.

[0014] <Example of operation> The information processing device 100 operates as shown in the flowchart in Figure 2. First, the imaging unit 104 captures a facial image of the person the user is looking at (step S101). Then, the acquisition unit 106 acquires personal information, including the name of the person identified using the captured facial image and information indicating the organization to which that person belongs (step S103). The display processing unit 108 displays the acquired personal information on the display unit 102 (step S105).

[0015] This information processing device 100 can capture a facial image of the person the user is looking at, and use that facial image to display personal information, including the person's name and information indicating the organization to which that person belongs, on the display unit 102. As a result, the information processing device 100 can appropriately provide users with information for communicating with the person in front of them in organizations such as schools and companies.

[0016] The following describes a detailed example of the information processing device 100.

[0017] (First Embodiment) <System Overview> The information provision system 1 includes an information processing device 100 and a server device 200, as shown in Figure 3. The information processing device 100 is a glasses-type device, shown as smart glasses 50 in Figure 3. The information processing device 100 and the server device 200 are connected via a communication network 3.

[0018] The server device 200 includes a storage device 220. The storage device 220 may be located inside the server device 200 or outside of it. In other words, the storage device 220 may be hardware integrated with the server device 200 or hardware separate from the server device 200.

[0019] As described above, the information processing device 100 is a smart glasses 50, but is not limited to this; it may be any other wearable device as long as it is in a form that can be worn by the user 10. For example, the smart glasses 50 may be AR (Augmented Reality) glasses. Server device 200 is a computer such as a server computer or a personal computer.

[0020] The smart glasses 50 are glasses-type devices that can be worn by user 10. The smart glasses 50 have a display (not shown) that projects information onto a screen in user 10's field of view when user 10 is wearing the smart glasses 50. The display is included in at least one of the lenses of the smart glasses 50 and is a liquid crystal display or an organic EL (Electro-Luminescence) display, etc. In other examples, the display may be a retinal projection type display.

[0021] The smart glasses 50 also have a camera (not shown) positioned to capture an image of the person the user 10 wearing the smart glasses 50 is looking at.

[0022] The camera is equipped with a lens and an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor. The smart glasses 50 have wireless communication capabilities (for example, a wireless LAN (Local Area Network) communication function) and communicate with the server device 200. Images captured by the camera are transmitted to the server device 200 using the wireless communication function of the smart glasses 50.

[0023] The images generated by the camera may be captured in real time and transmitted to the server device 200, or they may be images with a predetermined time delay. The images captured by the camera may be temporarily stored in another storage device, and the server device 200 may read them sequentially or at predetermined intervals from the storage device. Furthermore, the images transmitted to the server device 200 may be moving images, frame images at predetermined intervals, or still images.

[0024] The smart glasses 50 further have a speaker for outputting sound. The smart glasses 50 may also have a touch sensor for receiving user input 10. The smart glasses 50 may further have a microphone for inputting sound.

[0025] <Example Hardware Configuration> The information processing device 100 described herein is implemented by the computer 1000 shown in Figure 4. The server device 200 shown in Figure 3 is also implemented by the computer 1000. Furthermore, the functions of the information processing device 100 may be shared between the computer 1000 of the server device 200 and the computer 1000 of the information processing device 100. In addition, the communication device 300, which will be described later, is also implemented by the information processing device 100.

[0026] Computer 1000 has a bus 1010, a processor 1020, memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0027] Bus 1010 is a data transmission path for the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060 to send and receive data to and from each other. However, the method of connecting the processor 1020 and the other components to each other is not limited to bus connection.

[0028] The 1020 processor is a processor implemented in components such as the CPU (Central Processing Unit) and GPU (Graphics Processing Unit).

[0029] Memory 1030 is a main memory device implemented using RAM (Random Access Memory), etc.

[0030] The storage device 1040 is an auxiliary storage device implemented as an HDD (Hard Disk Drive), SSD (Solid State Drive), memory card, or ROM (Read Only Memory). The storage device 1040 stores program modules that implement each function of the information processing device 100 (for example, the imaging unit 104, the acquisition unit 106, the display processing unit 108, the reading unit 110 described later, etc.). Alternatively, the storage device 1040 enables each function of the server device 200 (for example, the specific unit 202, the transmission unit 204 described later, etc.) by having the processor 1020 read these program modules into the memory 1030 and execute them, thereby realizing each function corresponding to the program module. Furthermore, the storage device 1040 may also function as the storage device 220 of the server device 200.

[0031] The program module may be recorded on a recording medium. The recording medium on which the program module is recorded includes a non-temporary, tangible medium usable by a computer 1000, and program code that can be read by the computer 1000 (processor 1020) may be embedded in that medium.

[0032] The input / output interface 1050 is an interface for connecting the computer 1000 to various input / output devices. The input / output interface 1050 also functions as a communication interface for short-range wireless communication such as Bluetooth (registered trademark) and NFC (Near Field Communication). Furthermore, the input / output interface 1050 also functions as a wired communication interface such as USB (Universal Serial Bus) and HDMI (High Definition Multimedia Interface) (registered trademark).

[0033] The network interface 1060 is an interface for connecting the computer 1000 to the communication network 3. This communication network 3 is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1060 connects to the communication network 3 may be a wireless connection or a wired connection. Furthermore, the network interface 1060 also functions as a communication interface for communication using mobile communication systems such as CDMA (Code Division Multiple Access), LTE (Long Term Evolution), 4th generation communication (4G), 5th generation communication (5G), and 6th generation communication (6G) and later, via a mobile communication network.

[0034] The computer 1000 then connects to the necessary devices (for example, the display, sensors, operation buttons, camera, speaker, microphone, etc. of the information processing device 100 (smart glasses 50), the display, touch panel, touchpad, operation buttons, keyboard, mouse, speaker, microphone, camera, printer, etc. of the server device 200) via the input / output interface 1050 or the network interface 1060.

[0035] Each component of the information processing device 100 and server device 200 in each figure of this disclosure can be realized by any combination of the hardware and software of the computer 1000 in Figure 4. It will be understood by those skilled in the art that there are various modifications to the implementation method and apparatus. The functional block diagrams showing the information processing device 100 and server device 200 in each figure of this disclosure show blocks of logical functional units, not hardware-level configurations.

[0036] <Example of Functional Configuration> The following explanation will use Figure 1. In the following explanation, the organization will be assumed to be a company, but the organization may also be a school or other similar institution. Furthermore, the organization may be temporarily formed, for example, a group of disaster relief volunteers. In the case of a school, the target person 20 is a student or teacher, and in the case of a company, it is an employee. However, the target person 20 may also include employees of other companies or contractors who frequent the company. In that case, however, it is assumed that information regarding employees of other companies or contractors has been registered in the storage device 220 in advance.

[0037] As described above, the information processing device 100 includes a display unit 102, an imaging unit 104, an acquisition unit 106, and a display processing unit 108. The display unit 102 displays information. The display unit 102 is the display of the smart glasses 50 described above.

[0038] The imaging unit 104 is the camera of the smart glasses 50 described above, and captures a facial image of the person the user is looking at. There are various possible timings for generating the facial image, as exemplified below, but are not limited to these. In addition, multiple timings described below may be combined. For example, the timing for generating the facial image may be after the face in (a2) is detected AND when the operation of user 10 in (a1) is detected. (a1) When it is detected that user 10 has operated the touch center of the smart glasses 50 (a2) When the camera image of the Smart Glasses 50 is analyzed and a face is detected (a3) When it is detected that user 10's gaze is directed towards the face detected in (a2)

[0039] In the case of (a3), the smart glasses 50 may be further equipped with an eye-tracking camera that detects the gaze of the user 10, and the direction of the user 10's gaze may be determined using the eye-tracking camera. For example, when a face is detected in the camera's image in (a2), it may be determined whether the user 10's gaze is directed towards the face detected in (a2) by checking whether the position of the face matches the position of the user 10's gaze.

[0040] The acquisition unit 106 acquires personal information, including the name of the person identified using the facial image of the person captured (hereinafter also referred to as the target person 20) and information indicating the organization to which the target person 20 belongs. The acquisition unit 106 transmits the facial image to the server device 200, where the server device 200 identifies the identity of the target person 20 using the facial image. The process of identifying the target person 20 by the server device 200 will be described later. The acquisition unit 106 acquires personal information from the server device 200, including the name of the identified target person 20 and information indicating the organization to which the target person 20 belongs.

[0041] Personal information includes, for example, the name of the subject 20 and the organization to which they belong (e.g., department, grade, class, etc.). Personal information is not limited to this and may also include, for example, the subject 20's job title and title. Furthermore, if the subject 20 is an employee of an external company or school, the personal information may also include the name of the organization (e.g., company name, office name, school name, etc.).

[0042] The display processing unit 108 displays the acquired personal information on the display unit 102 (the display of the smart glasses 50). This allows the user 10 to instantly learn the personal information of the person 20 in front of them, including their name and organization.

[0043] As shown in Figure 5, the server device 200 includes a specific unit 202 and a transmission unit 204. Furthermore, as shown in Figure 6, the storage device 220 includes at least an employee face database (referred to as employee face DB in the figure) 230 and an employee information database (referred to as employee information DB in the figure) 240. The identification unit 202 identifies the target person 20 by comparing the facial image acquired from the information processing device 100 (smart glasses 50) with the facial image registered in the employee facial database 230.

[0044] For example, the employee face database 230 includes, for each employee, identification information that can identify that employee, associated with the employee's face photograph or facial feature information. The identification unit 202 receives a face image (or facial feature information extracted from the face image) from the information processing device 100. The identification unit 202 compares the feature information extracted from the face image with the facial feature information registered in the employee face database 230 and identifies the employee identification information corresponding to the feature information whose similarity is above a threshold. Then, the identification unit 202 refers to the employee information database 240 and obtains information corresponding to the employee identification information from the employee information database 240.

[0045] In this disclosure, image processing includes various image data analysis processes such as performing pattern recognition based on predetermined conditions, determining goodness or badness by comparing with reference values, recognizing an object (the face of the target person 20) by matching it with the feature quantities of a predetermined object (registered face), and measuring the size of the object (the face of the target person 20). Feature data extraction may be achieved by methods such as SIFT (Scale-Invariant Feature Transform) or HOG (Histograms of Oriented Gradients).

[0046] The employee information database 240 includes, for example, employee information 242 as shown in Figure 7. For each employee, employee information 242 includes, at a minimum, the employee's name, the phonetic spelling of the name, and information indicating their department, associated with an identifying information that can identify the employee (shown as employee ID in the figure). Furthermore, employee information 242 may also include information such as the employee's face and job title. In addition, employee information 242 may include information indicating the employee's attributes (e.g., age, gender, etc.), nicknames (nicknames or pet names), etc.

[0047] The identification unit 202 obtains information indicating the name and department of the target person 20 identified from the employee information 242.

[0048] The transmission unit 204 transmits information indicating the name and department of the target person 20 identified by the identification unit 202 to the information processing device 100 (smart glasses 50). The name of the target person 20 may include, for example, a surname and a given name, but the personal information transmitted to the information processing device 100 may include at least one of the surname and given name.

[0049] <Example of operation> The following explanation will be given using Figures 2 and 8. First, the information processing device 100 and the server device 200 are assumed to be pre-connected to enable wireless communication. User 10 is wearing the information processing device 100 (smart glasses 50). User 10 is then assumed to be in a designated organization, such as an office, and is face-to-face with the target person 20.

[0050] The imaging unit 104 of the information processing device 100 captures a facial image of the target person 20 that the user 10 is looking at (step S101 in Figure 2). As described above, the imaging unit 104 may also capture a facial image of the target person 20 when it detects that the user 10 has operated the touch center of the smart glasses 50.

[0051] The acquisition unit 106 then transmits the captured facial image of the target person 20 to the server device 200. When the identification unit 202 of the server device 200 receives the facial image from the information processing device 100, it identifies the target person 20 by comparing the received facial image with the facial images registered in the employee facial database 230 (step S201 in Figure 8).

[0052] The identification unit 202 further refers to the employee information database 240 and retrieves the personal information of the target person 20. Then, the transmission unit 204 of the server device 200 transmits the personal information of the target person 20 to the information processing device 100 (step S203 in Figure 8). Specifically, the personal information includes the name and affiliation of the target person 20 (for example, the department name).

[0053] The acquisition unit 106 of the information processing device 100 acquires personal information from the server device 200, including the name of the target person 20 and information indicating the organization to which that person belongs (step S103 in Figure 2). The display processing unit 108 of the information processing device 100 displays the acquired personal information on the display unit 102 (the display of the smart glasses 50 worn by the user 10) (step S105 in Figure 2).

[0054] In this way, user 10 can see the personal information of the person 20 in front of them displayed on the smart glasses 50's screen, and can immediately identify who the person 20 is.

[0055] As described above, according to this embodiment, the information processing device 100 includes a display unit 102, an imaging unit 104, an acquisition unit 106, and a display processing unit 108. The imaging unit 104 captures a facial image of the person the user is looking at. The acquisition unit 106 uses the captured facial image to acquire personal information, including the name of the person identified and information indicating the organization to which that person belongs. The display processing unit 108 displays the acquired personal information on the display unit 102. Thus, with the information processing device 100, in organizations such as schools and companies, it is possible to view the personal information of the person 20 in front of the user and confirm who the person 20 is, enabling the user 10 to communicate appropriately with the person 20.

[0056] In recent years, the spread of online classes and remote work has reduced opportunities for face-to-face communication. This is particularly difficult for teachers, supervisors, or HR personnel who need to remember the faces, names, and affiliations of numerous students and employees, as face-to-face interactions are infrequent.

[0057] For example, according to the information processing device 100, even if the user only vaguely remembers the name of the person in front of them, the person's name is displayed on the display unit 102, so the user 10 can call out the name of the person 20 and start a conversation. In addition, information such as the person 20's affiliation is displayed on the display unit 102, so the user 10 can decide what to talk about and start a conversation after knowing the person 20's affiliation. This allows the user 10 to communicate appropriately with the person 20.

[0058] (Second Embodiment) The second information processing device 100 according to this disclosure is the same as the information processing device 100 in Figure 1, except that the personal information displayed on the display unit 102 is different. Note that each element of the second information processing device 100 may be combined to the extent that it does not conflict with the elements of one or more embodiments. The following explanation will be given using Figures 1 and 5.

[0059] <Status display> Personal information may include information that can identify the current status of the person(s) concerned. The display processing unit 108 displays the current status of the target person 20 on the display unit 102 so that the user 10 can identify it.

[0060] As shown in Figure 6, the storage device 220 further includes a schedule storage unit 250 (referred to as "schedule" in the figure). The schedule storage unit 250 includes, for each employee, information indicating the employee's schedule associated with identification information that can identify the employee. The employee's schedule is, for example, an appointment from the current time onward. The employee's schedule may also include, for example, appointments for meetings (including web conferences), consultations, education, training, outings, and breaks. Alternatively, the employee's schedule may include, for example, appointments set by the employee themselves for times when they want to concentrate on work while avoiding contact with others, or times when they can interact with others.

[0061] The display processing unit 108 refers to the schedule storage unit 250 to obtain the schedule of the employee corresponding to the target person 20 from the current time onward, and uses the obtained schedule to identify the current status of the target person 20. The status indicates whether or not the subject 20 currently has time to talk. The status may also include information indicating how much time is available to talk. Furthermore, the status may also include information indicating that the subject 20 has free time in their future schedule (i.e., information indicating that they have time to talk), and information indicating that the subject 20 is currently busy (i.e., information indicating that they do not have time to talk).

[0062] For example, if person 20 has a meeting scheduled within 10 minutes from the current time, they may be busy preparing for the meeting, so their status should indicate that they are not available to talk right now (e.g., "Busy"). Conversely, if person 20 has no particular appointments and is free, their status should indicate that they are available to talk (e.g., "Available"). Also, if they have a meeting scheduled in 30 minutes, their status may indicate that they have very little time to talk (e.g., "Available for a short chat").

[0063] The display processing unit 108 identifies the status of the target person 20 and displays that status on the display unit 102 in an identifiable manner. For example, the display processing unit 108 may simply display information indicating the status of the target person 20 (e.g., "busy" or "schedule available"). Alternatively, the display processing unit 108 may display the status of the target person 20 using a symbol corresponding to the target person 20's status (e.g., an icon or badge).

[0064] For example, if the status of person 20 is "available for interaction," a green badge or an icon indicating "○" or "OK" may be displayed. If the status of person 20 is "unavailable for interaction," a red badge or an icon indicating "×" or "NG" may be displayed. If the status of person 20 is "available for short-term interaction," a yellow badge or a triangle icon may be displayed.

[0065] There are various possible locations for displaying the status, but the display processing unit 108 displays the status of the target person 20 next to (near) the name of the target person 20. Furthermore, if multiple people are in the field of view, for example, the display processing unit 108 may display the name and status (e.g., available / busy) of the identified target person 20, associated with an icon or badge, near the face of the identified target person 20. Alternatively, the display processing unit 108 may superimpose an image element (e.g., a rectangular shape) surrounding the face of the identified target person 20, and then display the name and status of the target person 20, associated with that image element, using an icon or badge.

[0066] This configuration allows user 10 to see the current status of the person 20 whose gaze is directed at them at a glance, enabling them to decide whether or not it is appropriate to speak to person 20. Since they can speak when person 20 is available to talk, the likelihood of smooth communication increases. Furthermore, if person 20 is not available to talk, they can refrain from speaking, thus avoiding inconveniencing person 20.

[0067] Status-identifying information may further include information regarding the schedules of the 20 individuals concerned. The display processing unit 108 may display information indicating the subject person 20's future schedule, such as "10:00-11:00: Meeting," on the display unit 102. Alternatively, the display processing unit 108 may display the subject person 20's entire schedule for the day.

[0068] <Communication Log Display> As shown in Figure 6, the storage device 220 may further include an SNS (Social Networking Service) information storage unit 260 (referred to as SNS in the figure) and an email information storage unit 270 (referred to as email in the figure). The SNS information storage unit 260 and the email information storage unit 270 store communication records such as emails, SMS (Short Message Service), and chats. The employee information database 240 also further includes communication-related information 244 as shown in Figure 9. For each employee, the communication-related information 244 includes identification information that can identify the employee (referred to as employee ID in the figure), associated with the employee's email address, mobile phone number (referred to as mobile phone in the figure), and SNS user ID (referred to as SNS ID in the figure).

[0069] The acquisition unit 106 acquires the communication record between the target person 20 and the user 10. Specifically, the acquisition unit 106 requests the server device 200 to record the communication between the target person 20 and the user 10. At this time, the information processing device 100 transmits information that can identify the user 10 (for example, an employee ID) to the server device 200. The identification unit 202 of the server device 200 obtains the employee's email address, mobile phone number, and SNS ID from the user 10's employee ID. Furthermore, the identification unit 202 obtains the employee's email address, mobile phone number, and SNS ID from the identified target person 20's employee ID.

[0070] The identification unit 202 then refers to the SNS information storage unit 260 and the email information storage unit 270 and extracts records of communication between the user 10 and the target person 20. The target communication records include communication records such as email, SMS (Short Message Service), and chat. The extracted communication records may include all of these, or at least one. The communication records to be extracted may be specified in advance by the user 10, or they may be selected by the user 10 at any time using the operation menu of the smart glasses 50. In the latter case, information indicating the communication records to be extracted is also transmitted from the smart glasses 50 to the server device 200.

[0071] The communication records may cover, for example, the most recent few days (1 to 3 days), or a predetermined period (within one week, etc.). The period of the communication records may also be specified by user 10. The period of the communication records may be specified in advance by user 10, or it may be specified at any time by displaying an operation menu on the smart glasses 50.

[0072] The transmission unit 204 of the server device 200 transmits information indicating the extracted communication record to the information processing device 100.

[0073] The display processing unit 108 of the information processing device 100 displays the communication record received from the server device 200 on the display unit 102.

[0074] For example, the display processing unit 108 may display information showing recent email or chat exchanges in chronological order. In the case of emails, the display processing unit 108 may first display only the subject lines in a list, and then display the details of the selected email.

[0075] The display processing unit 108 may display the summarized communication record. For example, the identification unit 202 of the server device 200 generates a summary of the extracted communication records. Then, the transmission unit 204 transmits the generated summary of communication records to the information processing device 100. The communication record summary may be created, for example, using generative AI (Artificial Intelligence) or large-scale language models.

[0076] Furthermore, the information regarding the schedule of the aforementioned person 20 may also be summarized. Alternatively, both the communication records (email, SMS, chat) and the schedule may be summarized. The information to be summarized may be specified in advance by the user 10, or it may be selected by the user 10 at any time using the operation menu of the smart glasses 50. In the latter case, information regarding the information to be summarized is also transmitted from the smart glasses 50 to the server device 200.

[0077] This configuration makes the displayed content summarized, making it easier for user 10 to grasp information about the target person 20.

[0078] As described above, in the information processing device 100 of this embodiment, the display processing unit 108 can display the status of the target person 20 in an identifiable manner. Thus, the information processing device 100 of this embodiment provides the same effects as the above embodiment, and furthermore, it can check the state of the target person 20 and determine the timing to speak to them, thus enabling communication that is tailored to the other person's convenience.

[0079] Furthermore, the information processing device 100 can display the communication record between the user 10 and the target person 20 using the display processing unit 108, so that the user can communicate with the target person 20 in a way that is appropriate for what to say to the target person 20, and without omitting anything that should be said.

[0080] (Third embodiment) The third information processing device 100 in Figure 10 of this disclosure is the same as the information processing device 100 in Figure 1, except that it further has a reading section 110. Note that each element in Figure 10 may be combined to the extent that it does not conflict with each element of one or more embodiments.

[0081] The reading unit 110 outputs the name of the target person 20 correctly from a speaker located on the ear side of the glasses-type device (smart glasses 50).

[0082] Specifically, the acquisition unit 106 further acquires information from the server device 200 indicating the phonetic reading of the name of the target person 20. The reading unit 110 uses information indicating the phonetic reading of the name of the target person 20, obtained from the server device 200, to correctly read aloud and output the name of the target person 20.

[0083] For example, even if the spelling of a person's name is the same, the pronunciation may differ. For instance, the name "Ogiwara" can be read as "Ogiwara" or "Ogihara." Also, if the target person 20 is a foreigner, it may be difficult to pronounce their name correctly, causing hesitation in approaching them. When the reading unit 110 outputs the correct pronunciation of the name, user 10 can call out the name correctly and approach the target person 20.

[0084] As described above, the information processing device 100 of this embodiment has a reading unit 110. The reading unit 110 correctly reads aloud and outputs the name of the target person 20 from the speaker of the smart glasses 50. Thus, the information processing device 100 of this embodiment provides the same effects as the above embodiment, and furthermore, it allows the user to know the correct pronunciation of the name of the person 20, thus preventing mispronunciation of the name. In addition, the user 10 can address the person 20 by name without hesitation.

[0085] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0086] (Other Embodiment 1) In another embodiment, the display processing unit 108 may switch the information to be displayed on the display unit 102 when it detects a predetermined trigger. The predetermined triggers could be, for example, gaze direction, blinking, or staring for a predetermined period of time or longer. These can be detected using the eye-tracking camera described above. Furthermore, if the smart glasses 50 have a microphone, the predetermined trigger may be the user's words (instructions).

[0087] The triggers, such as gaze direction, blink count, gaze duration, and words, are predetermined and may be changed by user 10. Furthermore, user 10 may pre-select which trigger to use via a settings menu. These settings can be changed by displaying an operation menu on the smart glasses 50, thereby accepting user 10's input.

[0088] As shown in Figure 11, when the information processing device 100 detects a predetermined trigger (YES in step S111) while displaying information in step S105 of Figure 2, the display processing unit 108 switches the display information (step S113).

[0089] For example, the display processing unit 108 can switch the display of the schedule and communication records of the target person 20, or switch the display of email, chat, and SMS records among the communication records, each time it detects a predetermined trigger.

[0090] (Another Embodiment 2) In the above embodiment, various settings were made using an operation menu displayed on the smart glasses 50. In other embodiments, various settings may be received using a communication device 300 available to the user 10. As shown in Figure 12, the information provision system 1 further includes a communication device 300 for use by the user 10. The communication device 300 is capable of communicating with at least one of the smart glasses 50 and the server device 200. The communication device 300 is, for example, a computer 1000 such as a smartphone, tablet terminal, or personal computer.

[0091] The communication device 300 may have an application pre-installed, and by launching this application, a screen for performing various settings may be displayed on the display of the communication device 300. Alternatively, the communication device 300 may launch a predetermined browser, access a predetermined URL (Uniform Resource Locator), and access the information provision system 1. After user authentication, the communication device 300 may log in to the server device 200 or a web server (not shown) that provides services for the information provision system 1, thereby displaying a web page for performing various settings on the communication device 300. In any case, user 10 must first complete the user registration procedure to use the services of the information provision system 1 and obtain a user account (e.g., user ID and password) for user authentication processing.

[0092] The communication device 300 allows for various settings. These are exemplified below, but are not limited to these. Furthermore, multiple settings can be combined. (b1) Specify the display order for multiple display items (schedule, email, chat, SMS, etc.) (b2) Specify the target period for each display item (today only, the most recent days, a specific date and time, a specific period, etc.) (b3) Specify the items to be included in the summary (schedule, email, chat, SMS, etc.) (b4) Specify the time period for the items to be included in the summary (e.g., today only, most recent days, specific date and time, specific period, etc.) (b5) Specify that only the free time slots of the 20 target individuals' schedules should be displayed.

[0093] The information set by the communication device 300 is stored in the storage device 220 of the server device 200, associated with the user 10. Alternatively, the information set by the communication device 300 may be stored in the storage device (not shown) of the communication device 300 and transmitted to the smart glasses 50 or the server device 200 as needed.

[0094] Furthermore, the communication device 300 may accept registration of individuals that user 10 wishes to communicate with today. There are various methods for accepting the designation of registered individuals, and are not particularly limited, but examples are given below. (c1) The system accepts the selection of registered individuals using a UI that allows users to choose from a list of members (employees) of a designated group (e.g., department) (including names, photos, etc.). (c2) A UI is used that accepts input of at least some of the information, such as the department name and name. Based on the input information, the server device 200 searches the employee information database 240 to identify the person and accepts them as a registered person.

[0095] The communication device 300 queries the server device 200 for the schedules of the registered person and user 10, and identifies the available time for communication between them. This identification process can be performed by either the communication device 300 or the server device 200. The communication device 300 then displays the identified available time for communication on its screen. Alternatively, the communication device 300 may display the time on the smart glasses 50.

[0096] Furthermore, the communication device 300 may output a notification to the communication device 300 or smart glasses 50 a predetermined time before the time (for example, 10 minutes before, 5 minutes before, etc.) informing the user 10 that the time for communication is approaching. The timing of the notification may also be predetermined by the user 10 and accepted in advance on the communication device 300.

[0097] Furthermore, regarding the person that user 10 has registered to communicate with today, the smart glasses 50's camera may be used to determine whether or not the registered person is in front of user 10. The smart glasses 50 continuously (or periodically) transmits the captured image to the server device 200. However, the captured image may also be transmitted to the server device 200 only when there is a change in the person in front of user 10. If only the same person is included in the image, the image does not need to be transmitted. In this case, the imaging unit 104 processes the image to determine whether or not there has been a change in the person in the image. The identification unit 202 of the server device 200 extracts the face image of the person in the image through image processing. The identification unit 202 then compares the face image (or facial feature information) with the face image (or facial feature information) of the person registered by user 10 to determine whether or not the registered person is in front of user 10.

[0098] When the system detects that a registered person is in front of user 10, the transmission unit 204 transmits information to the smart glasses 50 indicating that a registered person has been detected. This information includes information that allows the registered person in the image to be identified. Upon receiving this information, especially if the image contains multiple people, the display processing unit 108 may overlay information indicating which person is the registered person, such as an image element to be highlighted, onto the registered person displayed on the smart glasses 50. The image element to be highlighted may be, for example, a frame around the person's face or an arrow icon, but is not limited to these. The highlighting method is not particularly limited as long as the person can be identified at a glance. Alternatively, the smart glasses 50 may output a voice or alert sound from its speaker to notify the user that a registered person is in front of it.

[0099] Furthermore, the display processing unit 108 may further display various personal information and status information of the registered person, as described above.

[0100] Furthermore, in this configuration, the process of detecting registered individuals from captured images through image processing may be performed by the communication device 300 or the smart glasses 50 instead of the server device 200. In that case, it becomes unnecessary to send the captured images to the server device 200. When image processing is performed by the communication device 300, the captured images are sent from the smart glasses 50 to the communication device 300. In this case, the communication device 300 also obtains a face image or facial feature information from the server device 200 in advance to detect the registered individual that the communication device 300 has received. When image processing is performed by the smart glasses 50, the communication device 300 further sends the face image or facial feature information of the registered individual to the smart glasses 50. The communication device 300 or the smart glasses 50 then detects the registered individual by processing the captured images using the acquired face image or facial feature information of the registered individual.

[0101] This configuration allows user 10 to initiate a conversation with someone they don't know, even if they've never met that person before. Alternatively, it makes it easier for user 10 to find someone they want to talk to in a large group of people.

[0102] Other examples of the disclosure may include a program that causes at least one computer to execute the method disclosed above, or a recording medium that is readable by a computer on which such a program is recorded. This recording medium may include non-temporary tangible media. This computer program includes computer program code that, when executed by a computer, causes the computer to perform an information processing method on an information processing device or server device.

[0103] Furthermore, any combination of the above components, as well as any conversion of the expressions of this disclosure between methods, apparatus, systems, recording media, computer programs, etc., are also valid forms of this disclosure.

[0104] Furthermore, the various components of this disclosure do not necessarily have to be independent entities; multiple components may be formed as a single member, one component may be formed from multiple members, one component may be part of another component, or a part of one component may overlap with a part of another component, and so on.

[0105] Furthermore, although the methods and computer programs disclosed herein describe multiple steps in sequence, the order in which these steps are described does not limit the order in which they must be performed. Therefore, when implementing the methods and computer programs disclosed herein, the order of the steps can be changed to the extent that it does not impair the content.

[0106] Furthermore, the methods and computer programs described herein are not limited to being executed at individually different times. For example, other procedures may be executed while one procedure is being executed, or the timing of one procedure's execution may overlap with the timing of another procedure's execution in whole or in part.

[0107] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate. Furthermore, any information obtained or used in this disclosure regarding users (User 10 and Subject Person 20) shall be conducted lawfully.

[0108] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment is not limited to the order in which they are described. In each embodiment, the order of the illustrated steps can be changed to the extent that it does not impair the content.

[0109] Some or all of the above embodiments may also be described as follows, but are not limited to the following: 1. A means for displaying information, An imaging means for capturing an image of the face of the person the user is looking at, A means for acquiring personal information, including the name of a person identified using a facial image of the person captured, and information indicating the organization to which that person belongs. A display processing means for displaying the acquired personal information on the display means, Equipped with, An information processing device that can be attached to the aforementioned user. 2. In the information processing device described in 1., An information processing device that resembles eyeglasses. 3. In the information processing device described in 1. or 2., The aforementioned personal information further includes information that can identify the current status of the person, The display processing means is an information processing device that displays the current status of the person in a way that the user can identify. 4. In the information processing device described in any one of items 1 to 3, The acquisition means acquires the communication record between the person and the user, The display processing means is an information processing device that further displays the acquired communication record. 5. In the information processing device described in any one of items 1 to 4, The display processing means is an information processing device that switches the information to be displayed when it detects a predetermined trigger. In the information processing device described in 6.2, An information processing device further comprising a reading means that correctly reads out and outputs the person's name from a speaker provided on the ear side of the aforementioned glasses-type device. In the information processing device described in 7.3, The information processing device further includes information relating to the schedule of the person, which allows for the identification of the aforementioned status. In the information processing device described in 8.4, The display processing means is an information processing device that displays the summarized communication record.

[0110] 9. Face recognition glasses worn by the user, The system comprises a server device connected to the aforementioned facial recognition glasses and having a database within the organization, The aforementioned facial recognition glasses are A means for displaying information, An imaging means for capturing an image of the face of the person the user is looking at, An acquisition means that transmits the captured facial image of the person to the server device and compares it with images registered in the database to obtain personal information from the server device, including the name of the identified person and information indicating the organization to which that person belongs. The system comprises a display processing means for displaying the acquired personal information on the display means, The server device is A means for identifying a person by comparing the facial image obtained from the facial recognition glasses with the facial image registered in the database, The system includes a transmission means for transmitting personal information, including the name of the identified person and information indicating the organization to which that person belongs, to the facial recognition glasses. Information provision system. 10. A means for displaying information, One or more computers, comprising an imaging means, The aforementioned imaging means is instructed to capture an image of the face of the person the user is looking at. Using the captured facial image of the person, personal information including the name of the person identified and information indicating the organization to which that person belongs is obtained. The acquired personal information is displayed on the display means. Information processing methods. 11. A means for displaying information, A computer equipped with an imaging means, The imaging means performs an imaging process to capture an image of the face of the person the user is looking at. An acquisition process that obtains personal information including the name of the person identified using the captured facial image of the person and information indicating the organization to which that person belongs. A program for performing a display process that causes the acquired personal information to be displayed on the display means. 12. A means for displaying information, A computer equipped with an imaging means, The imaging means performs an imaging process to capture an image of the face of the person the user is looking at. An acquisition process that obtains personal information including the name of the person identified using the captured facial image of the person and information indicating the organization to which that person belongs. A computer-readable recording medium that stores a program for executing a display process that causes the acquired personal information to be displayed on the display means.

[0111] Furthermore, some or all of the configurations described in Appendices 2 to 8, which are subordinate to Appendice 1 (Information Processing Device) as described above, may also be subordinate to Appendice 9 (Information Provisioning System), Appendice 10 (Information Processing Method), Appendice 11 (Program), and Appendice 12 (Recording Medium) in the same subordinate relationship as in Appendices 2 to 8. Moreover, not limited to Appendices 9, 10, 11, and 12, some or all of the configurations described as appendices may also be subordinate to various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above. [Explanation of Symbols]

[0112] 1. Information Provision System 3. Communication Network 10 users 20 Target Persons 50 Smart Glasses 100 Information Processing Devices 102 Display section 104 Imaging Unit 106 Acquisition Department 108 Display Processing Unit 110 Reading section 200 Server Devices 202 Specific section 204 Transmitter 220 Storage device 230 Employee Facial Database 240 Employee Information Database 242 Employee Information 244 Communications-related information 250 Schedule memory unit 260 Information storage section 260 SNS information storage section 270 Email Information Storage Unit 300 Communication devices 1000 computers 1010 Bus 1020 Processor 1030 memory 1040 Storage Devices 1050 Input / Output Interface 1060 Network Interfaces

Claims

1. A means for displaying information, An imaging means for capturing an image of the face of the person the user is looking at, A means for acquiring personal information, including the name of a person identified using a facial image of the person captured, and information indicating the organization to which that person belongs. A display processing means for displaying the acquired personal information on the display means, Equipped with, An information processing device that can be attached to the aforementioned user.

2. In the information processing apparatus according to claim 1, An information processing device that resembles eyeglasses.

3. In the information processing apparatus according to claim 1 or 2, The aforementioned personal information further includes information that can identify the current status of the person, The display processing means is an information processing device that displays the current status of the person in a way that the user can identify.

4. In the information processing apparatus according to claim 1 or 2, The acquisition means acquires the communication record between the person and the user, The display processing means is an information processing device that further displays the acquired communication record.

5. In the information processing apparatus according to claim 2, An information processing device further comprising a reading means that correctly reads out and outputs the person's name from a speaker provided on the ear side of the aforementioned glasses-type device.

6. In the information processing apparatus described in claim 3, The information processing device further includes information relating to the schedule of the person, which allows for the identification of the aforementioned status.

7. In the information processing apparatus according to claim 4, The display processing means is an information processing device that displays the summarized communication record.

8. The user wears facial recognition glasses, The system comprises a server device connected to the aforementioned facial recognition glasses and having a database within the organization, The aforementioned facial recognition glasses are A means for displaying information, An imaging means for capturing an image of the face of the person the user is looking at, An acquisition means that transmits the captured facial image of the person to the server device and compares it with images registered in the database to obtain personal information from the server device, including the name of the identified person and information indicating the organization to which that person belongs. The system comprises a display processing means for displaying the acquired personal information on the display means, The server device is A means for identifying a person by comparing the facial image obtained from the facial recognition glasses with the facial image registered in the database, The system includes a transmission means for transmitting personal information, including the name of the identified person and information indicating the organization to which that person belongs, to the facial recognition glasses. Information provision system.

9. A means for displaying information, One or more computers, equipped with imaging means, The aforementioned imaging means is instructed to capture an image of the face of the person the user is looking at. Using the captured facial image of the person, personal information including the name of the person identified and information indicating the organization to which that person belongs is obtained. The acquired personal information is displayed on the display means. Information processing methods.

10. A means for displaying information, A computer equipped with an imaging means, The imaging means performs an imaging process to capture an image of the face of the person the user is looking at. An acquisition process that obtains personal information including the name of the person identified using the captured facial image of the person and information indicating the organization to which that person belongs. A program for performing a display process that causes the acquired personal information to be displayed on the display means.

Images