Information processing device, information processing system, equipment management method, and program

Abstract

To configure settings for an electronic device using information used in another electronic device.SOLUTION: A management apparatus configured to communicate with a plurality of electronic devices and an imaging apparatus over a communication link includes: a state storage unit; a state management unit which includes an object state management unit, a device state management unit, and an output information determination unit; an image receiving unit which manages the electronic devices and a state of an object, decides content of an operation instruction to be transmitted to the electronic devices, updates, based on a received image, a registered object list stored in the state storage unit, updates, based on received device operation information, a registered device list and the registered object list, and transmits the received image to the state management unit; an operation information receiving unit which receives the device operation information from the electronic devices and transmits the received device operation information to the state management unit; an output information determination unit which determines content of an operation instruction to be transmitted to the electronic devices on the basis of the stored state management information, and transmits the determined operation instruction to an operation instruction unit; and the operation instruction unit which transmits the operation instruction received from the state management unit to the electronic devices.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] This invention relates to an information processing apparatus, an information processing system, a device management method, and a program.

Background Art

[0002] In cloud computing, there is a technology for realizing single sign-on by coordinating software. For example, Patent Document 1 discloses an invention in which, by using a common authentication base in a plurality of services, a user authenticated in one service can omit authentication in other services.

[0003] In authentication by hardware such as electronic devices, there is a technology for associating the authentication processes of each hardware. For example, Patent Document 2 discloses an invention in which, by coordinating a system for managing entry and exit to / from an area handling confidential information and an electronic device within the area, authentication by the electronic device is not permitted unless the user is authenticated in the entry / exit management system.

[0004] In cooperation between hardware, it is difficult to determine the identity of a user. The invention disclosed in Patent Document 2 uses the authentication status of the entry / exit management system in authentication by an electronic device, but the user needs to perform an authentication operation for each. For example, if the identity between a user who has authenticated with a first electronic device and a user in the vicinity of a second electronic device can be determined, the second electronic device can perform authentication using the authentication information input by the user with the first electronic device.

Summary of the Invention

Problems to be Solved by the Invention

[0005] An embodiment of this invention aims to set a second electronic device using information used in a first electronic device.

Means for Solving the Problems

[0006] To solve the above problems, an information processing device, which is one embodiment of the present invention, is an information processing device that can communicate with a first electronic device, a second electronic device and a shooting device via a network, and comprises: a first image receiving unit that receives a first image of a first user using the first electronic device from the shooting device; a first user identification unit that identifies the first user using the first electronic device based on usage information of the first electronic device and the first image; an input information receiving unit that receives authentication information entered by the first user identified by the first user identification unit into the first electronic device from the first electronic device; a second image receiving unit that receives a second image of a second user located in an operable position of the second electronic device from the shooting device; a second user identification unit that identifies the second user located in an operable position of the second electronic device from the second image; and an operation instruction unit that transmits authentication information or screen setting information in the authentication information to the second electronic device based on information of the first user identified by the first user identification unit and the second user identified by the second user identification unit and authentication information. [Effects of the Invention]

[0007] According to one embodiment of this invention, a second electronic device can be configured using information used in a first electronic device. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the overall configuration of a device management system in one embodiment. [Figure 2] This figure shows an example of a computer hardware configuration in one embodiment. [Figure 3] This figure shows an example of the hardware configuration of a 360-degree imaging device in one embodiment. [Figure 4] This figure shows an example of the hardware configuration of an MFP in one embodiment. [Figure 5] This figure shows an example of the hardware configuration of an electronic whiteboard in one embodiment. [Figure 6]This figure shows an example of the functional configuration of an equipment management system in one embodiment. [Figure 7] This figure shows an example of state management information in one embodiment. [Figure 8] This figure shows an example of a first basic flowchart in one embodiment. [Figure 9] This figure shows an example of a second basic flowchart in one embodiment. [Figure 10] This figure shows an example of a device management method in one embodiment. [Figure 11] This figure shows an example of object state update processing in one embodiment. [Figure 12] This figure shows an example of the process for creating a list of detected objects in one embodiment. [Figure 13] This figure shows an example of the process for updating the registered object list in one embodiment. [Figure 14] This figure shows an example of the process for updating the registered object list in one embodiment. [Figure 15] This figure shows a first example of the group ID assignment process in one embodiment. [Figure 16] This figure shows a second example of the group ID assignment process in one embodiment. [Figure 17] This figure shows an example of equipment status update processing in one embodiment. [Figure 18] This figure shows an example of the output information determination process in one embodiment. [Figure 19] This figure shows an example of the output condition determination process in one embodiment. [Figure 20] This figure shows an example of device output information in one embodiment. [Modes for carrying out the invention]

[0009] The embodiments of this invention will be described in detail below with reference to the drawings. Note that components having the same function are numbered the same in the drawings, and redundant explanations are omitted.

[0010] [Embodiment] One embodiment of the present invention is a device management system for managing a plurality of electronic devices. The device management system in this embodiment realizes cooperation between electronic devices by instructing an operation using information input by a certain electronic device to other electronic devices.

[0011] Conventionally, in cooperation between hardware, the identity of a user was determined by the user possessing the same device (for example, an IC (Integrated Circuit) card or a portable information terminal, etc.). As a result, even when cooperating between hardware, an authentication operation on each hardware was necessary. In single sign-on by software cooperation, when a user authenticated by a certain service accesses another service, since the operation is performed from the same device, the identity of the user can be easily determined.

[0012] In the device management system in this embodiment, the management device tracks a person moving between electronic devices based on an image of a space where a plurality of electronic devices are installed. Thereby, the management device can determine the identity between a user who operated a certain electronic device and a user who is near another electronic device.

[0013] If the identity of the user can be determined, an operation using information input by a certain electronic device can be executed on other electronic devices. For example, if authentication using authentication information input by the user on another electronic device is performed on an electronic device near the user, single sign-on by hardware cooperation can be realized.

[0014] <Overall Configuration of the Device Management System> First, the overall configuration of the device management system in this embodiment will be described while referring to FIG. 1. FIG. 1 is a diagram showing an example of the overall configuration of the device management system in this embodiment.

[0015] As shown in Figure 1, the equipment management system 1 in this embodiment includes a management device 10, one or more monitoring devices 20, and a plurality of electronic devices 30. The management device 10, the monitoring devices 20, and the electronic devices 30 are each connected to a communication network N1.

[0016] The communication network N1 is configured so that each connected device can communicate with one another. The communication network N1 is constructed using wired communication networks such as the Internet, a LAN (Local Area Network), or a WAN (Wide Area Network).

[0017] The communication network N1 may include not only wired communication but also wireless communication such as Wi-Fi or short-range wireless communication, or mobile communication networks such as WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution), or 5G (5th Generation).

[0018] The monitoring device 20 and the electronic equipment 30 are installed in the managed space R1. The managed space R1 may be a single space such as a room in a building, or it may be multiple spaces connected by an accessible space such as a door or corridor.

[0019] Examples of managed spaces R1 include office conference rooms or offices. Other examples of managed spaces R1 include sales areas or back rooms in small stores such as convenience stores. Other examples of managed spaces R1 include hotel lobbies or guest rooms. Other examples of managed spaces R1 include airports, bookstores, or factories. Managed spaces R1 are not limited to these, and can be any space where electronic devices that can be operated by multiple users are installed.

[0020] The management device 10 is an information processing device such as a PC (Personal Computer), workstation, or server that manages the electronic equipment 30. Based on the images acquired by the monitoring device 20, the management device 10 transmits operation instructions to the electronic equipment 30. An example of the management device 10 is a computer.

[0021] The monitoring device 20 is an electronic device that acquires images including the vicinity of the electronic devices 30 installed in the managed space R1. The monitoring device 20 may acquire video (i.e., a time series of images). The monitoring device 20 is installed in a position where it can photograph all of the multiple electronic devices 30.

[0022] One example of the monitoring device 20 is a 360-degree camera. Another example of the monitoring device 20 is a set of multiple network cameras. When the monitoring device 20 consists of multiple network cameras, each network camera is positioned with its field of view adjusted so that there are no blind spots within the managed space R1.

[0023] Hereafter, when there are multiple monitoring devices 20, they will be distinguished using sub-numbers such as "monitoring device 20-1," "monitoring device 20-2," etc.

[0024] The electronic device 30 is a variety of electronic devices used by the user. The electronic device 30 is configured to be usable only by authenticated users. The electronic device 30 is configured to enable authentication using hardware such as an IC (Integrated Circuit) card.

[0025] An example of electronic equipment 30 is an image forming apparatus (printer, fax machine, MFP (Multifunction Peripheral / Product / Printer: digital multifunction device), scanner, etc.). Another example of electronic equipment 30 is an electronic whiteboard (IWB (Interactive White Board: a whiteboard with electronic blackboard functionality that enables mutual communication)).

[0026] Hereafter, when distinguishing between multiple electronic devices 30, they will be described using sub-numbers such as "electronic device 30-1," "electronic device 30-2," etc.

[0027] The electronic device 30 is not limited to an image forming apparatus or an electronic whiteboard, as long as it is a device equipped with communication functions. In other words, the electronic device 30 may be, for example, an output device such as a PJ (Projector) or digital signage, a HUD (Head Up Display) device, industrial machinery, an imaging device, a sound collection device, medical equipment, networked home appliances, an automobile (Connected Car), a notebook PC (Personal Computer), a mobile phone, a smartphone, a tablet device, a game console, a PDA (Personal Digital Assistant), a digital camera, a wearable PC, or a desktop PC.

[0028] <Hardware configuration of the equipment management system> Next, the hardware configuration of each device included in the equipment management system in this embodiment will be described with reference to Figures 2 to 5.

[0029] Computer Hardware Configuration Figure 2 shows an example of a hardware configuration when the management device 10 is implemented as a computer.

[0030] As shown in Figure 2, a computer in one embodiment includes a CPU (Central Processing Unit) 501, ROM (Read Only Memory) 502, RAM (Random Access Memory) 503, HD (Hard Disk) 504, HDD (Hard Disk Drive) controller 505, display 506, external device connection I / F (Interface) 508, network I / F 509, bus line 510, keyboard 511, pointing device 512, DVD-RW (Digital Versatile Disk Rewritable) drive 514, and media I / F 516.

[0031] Of these components, the CPU 501 controls the operation of the entire computer. The ROM 502 stores programs used to drive the CPU 501, such as the IPL (Initial Program Loader). The RAM 503 is used as the work area for the CPU 501. The HD 504 stores various data, such as programs. The HDD controller 505 controls the reading or writing of various data to the HD 504 according to the control of the CPU 501.

[0032] The display 506 displays various information such as cursors, menus, windows, characters, or images. The external device connection interface 508 is an interface for connecting various external devices. In this case, external devices include, for example, USB (Universal Serial Bus) memory and printers. The network interface 509 is an interface for data communication using the communication network N1. The bus line 510 is an address bus, data bus, etc., for electrically connecting each component such as the CPU 501 shown in Figure 2.

[0033] The keyboard 511 is a type of input means equipped with multiple keys for inputting characters, numbers, and various instructions. The pointing device 512 is a type of input means for selecting and executing various instructions, selecting processing targets, and moving the cursor. The DVD-RW drive 514 controls the reading or writing of various data to the DVD-RW 513, which is an example of a removable recording medium. Note that it is not limited to DVD-RW, but may also be DVD-R, etc. The media interface 516 controls the reading or writing (storage) of data to the recording medium 515, such as flash memory.

[0034] ≪Hardware configuration of the 360-degree imaging system≫ Figure 3 shows an example of the hardware configuration when the monitoring device 20 is implemented as a 360-degree imaging device. In the following, the 360-degree imaging device is assumed to be a 360-degree (omnidirectional) imaging device using two image sensors, but any number of image sensors other than two is acceptable. Furthermore, it is not necessarily required to be a device dedicated to omnidirectional imaging; a standard digital camera or smartphone can be retrofitted with an omnidirectional imaging unit to achieve essentially the same functionality as a 360-degree imaging device.

[0035] As shown in Figure 3, the 360-degree imaging device in one embodiment consists of an imaging unit 601, an image processing unit 604, an imaging control unit 605, a microphone 608, a sound processing unit 609, a CPU (Central Processing Unit) 611, a ROM (Read Only Memory) 612, an SRAM (Static Random Access Memory) 613, a DRAM (Dynamic Random Access Memory) 614, an operation unit 615, an external device connection interface 616, a communication unit 617, an antenna 617a, an acceleration / direction sensor 618, a gyroscope sensor 619, an acceleration sensor 620, and a recessed terminal 621 for Micro USB.

[0036] Of these, the imaging unit 601 includes wide-angle lenses (so-called fisheye lenses) 602a and 602b, each having a field of view of 180° or more for forming hemispherical images, and two image sensors 603a and 603b, each corresponding to the wide-angle lens. The image sensors 603a and 603b include an image sensor such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor that converts the optical image from the fisheye lenses 602a and 602b into electrical signal image data and outputs it, a timing generation circuit that generates horizontal or vertical synchronization signals and pixel clocks for the image sensors, and a group of registers in which various commands and parameters necessary for the operation of the image sensors are set.

[0037] The image sensors 603a and 603b of the imaging unit 601 are each connected to the image processing unit 604 via a parallel I / F bus. On the other hand, the image sensors 603a and 603b of the imaging unit 601 are connected to the imaging control unit 605 via a serial I / F bus (such as an I2C bus). The image processing unit 604, the imaging control unit 605, and the sound processing unit 609 are connected to the CPU 611 via bus 610. Furthermore, ROM 612, SRAM 613, DRAM 614, operation unit 615, external device connection I / F (Interface) 616, communication unit 617, and acceleration / direction sensor 618 are also connected to bus 610.

[0038] The image processing unit 604 receives image data output from image sensors 603a and 603b via a parallel I / F bus, performs predetermined processing on each image data, and then combines these image data to create equirectangular projection image data.

[0039] The imaging control unit 605 generally uses the I2C bus to set commands and other information in the registers of the image sensors 603a and 603b, with the imaging control unit 605 acting as the master device and the image sensors 603a and 603b as slave devices. It receives necessary commands and other information from the CPU 611. The imaging control unit 605 also uses the I2C bus to acquire status data and other information from the registers of the image sensors 603a and 603b and send it to the CPU 611.

[0040] Furthermore, the imaging control unit 605 instructs the image sensors 603a and 603b to output image data when the shutter button on the operation unit 615 is pressed. Some 360-degree imaging devices may also have functions to display a preview image on a display (for example, a smartphone display) or to display video. In this case, the image data output from the image sensors 603a and 603b is performed continuously at a predetermined frame rate (frames / minute).

[0041] Furthermore, as will be described later, the imaging control unit 605 also functions as a synchronization control means that works in cooperation with the CPU 611 to synchronize the output timing of image data from the image sensors 603a and 603b. In this embodiment, the 360-degree imaging device is not provided with a display, but a display unit may be provided.

[0042] Microphone 608 converts sound into sound (signal) data. Sound processing unit 609 receives the sound data output from microphone 608 through the I / F bus and performs predetermined processing on the sound data.

[0043] The CPU 611 controls the overall operation of the 360-degree imaging system and performs necessary processing. The ROM 612 stores various programs for the CPU 611. The SRAM 613 and DRAM 614 are work memories that store programs executed by the CPU 611 and data in progress. In particular, the DRAM 614 stores image data in progress from the image processing unit 604 and data of completed equirectangular projection images.

[0044] The control unit 615 is a collective term for the operation buttons, such as the shutter button 615a. The user inputs various shooting modes and shooting conditions by operating the control unit 615.

[0045] The external device connection interface 616 is an interface for connecting various external devices. These external devices include, for example, USB (Universal Serial Bus) memory and PCs (Personal Computers). The equirectangular projection image data stored in the DRAM 614 can be recorded to external media via the external device connection interface 616, or transmitted to external terminals (devices) such as smartphones via the external device connection interface 616 as needed.

[0046] The communication unit 617 communicates with external terminals (devices) such as smartphones via an antenna 617a provided on the 360-degree imaging device, using short-range wireless communication technologies such as Wi-Fi, NFC (Near Field Communication), and Bluetooth (registered trademark). This communication unit 617 can also transmit equirectangular projection image data to external terminals (devices) such as smartphones.

[0047] The acceleration / direction sensor 618 calculates the direction of the 360-degree camera from the Earth's magnetic field and outputs direction information. This direction information is an example of related information (metadata) in accordance with Exif and is used for image processing such as image correction of captured images. The related information also includes the date and time the image was captured and the data size of the image data. The acceleration / direction sensor 618 is also a sensor that detects changes in angle (Roll angle, Pitch angle, Yaw angle) associated with the movement of the 360-degree camera 6. Changes in angle are an example of related information (metadata) in accordance with Exif and are used for image processing such as image correction of captured images.

[0048] Furthermore, the acceleration / direction sensor 618 is a sensor that detects acceleration in three axes. The 360-degree imaging device calculates its orientation (angle relative to the direction of gravity) based on the acceleration detected by the acceleration / direction sensor 618. By providing the acceleration / direction sensor 618 to the 360-degree imaging device, the accuracy of image correction is improved.

[0049] ≪MFP Hardware Configuration≫ Figure 4 shows an example of a hardware configuration when the electronic device 30 is implemented using an MFP.

[0050] As shown in Figure 4, one embodiment of the MFP includes a controller 910, a short-range communication circuit 920, an engine control unit 930, an operation panel 940, and a network interface 950.

[0051] Of these, the controller 910 includes the main components of the computer: the CPU (Central Processing Unit) 901, system memory (MEM-P) 902, northbridge (NB) 903, southbridge (SB) 904, ASIC (Application Specific Integrated Circuit) 906, local memory (MEM-C) 907, HDD (Hard Disk Drive) controller 908, and HD (Hard Disk) 909, with the NB 903 and ASIC 906 connected by an AGP (Accelerated Graphics Port) bus 921.

[0052] Of these, the CPU901 is a control unit that performs overall control of the MFP. The NB903 is a bridge that connects the CPU901 to the MEM-P902, SB904, and AGP bus 921, and has a memory controller that controls reading and writing to the MEM-P902, as well as a PCI (Peripheral Component Interconnect) master and an AGP target.

[0053] MEM-P902 consists of a ROM (Read Only Memory) 902a, which is a memory for storing programs and data that realize the various functions of the controller 910, and a RAM (Random Access Memory) 902b, which is used for program and data deployment and drawing during memory printing. The programs stored in RAM 902b may be configured to be provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

[0054] SB904 is a bridge for connecting NB903 to PCI devices and peripherals. ASIC906 is an integrated circuit (IC) for image processing applications that has hardware elements for image processing, and acts as a bridge connecting the AGP bus 921, PCI bus 922, HDD908, and MEM-C907, respectively. This ASIC906 consists of a PCI target and AGP master, an arbiter (ARB) that forms the core of the ASIC906, a memory controller that controls the MEM-C907, multiple DMACs (Direct Memory Access Controllers) that perform image data rotation using hardware logic, and a PCI unit that performs data transfer via PCI bus 922 between the scanner unit 931 and the printer unit 932. Note that the ASIC906 may also be connected to a USB (Universal Serial Bus) interface or an IEEE1394 (Institute of Electrical and Electronics Engineers 1394) interface.

[0055] MEM-C907 is local memory used as a copy image buffer and code buffer. HD909 is storage for storing image data, font data used during printing, and forms. HD909 controls data reading or writing to it according to the control of CPU901. The AGP bus 921 is a bus interface for graphics accelerator cards proposed to speed up graphics processing, and by directly accessing MEM-P902 with high throughput, the graphics accelerator card can be made faster.

[0056] Furthermore, the short-range communication circuit 920 is equipped with a short-range communication circuit 920a. The short-range communication circuit 920 is a communication circuit such as NFC or Bluetooth.

[0057] Furthermore, the engine control unit 930 is composed of a scanner unit 931 and a printer unit 932. The operation panel 940 includes a panel display unit 940a, such as a touch panel, which displays current settings and selection screens and accepts input from the operator, and an operation panel 940b, which consists of a numeric keypad that accepts setting values ​​for image formation conditions such as density settings and a start key that accepts a copy start command. The controller 910 controls the entire MFP, for example, controlling drawing, communication, and input from the operation panel 940. The scanner unit 931 or the printer unit 932 includes an image processing section, such as error diffusion and gamma conversion.

[0058] Furthermore, the MFP allows you to sequentially switch between document box function, copy function, printer function, and facsimile function using the application switching key on the control panel 940. When the document box function is selected, it enters document box mode; when the copy function is selected, it enters copy mode; when the printer function is selected, it enters printer mode; and when the facsimile mode is selected, it enters facsimile mode.

[0059] Furthermore, the network interface 950 is an interface for data communication using the communication network N1. The short-range communication circuit 920 and the network interface 950 are electrically connected to the ASIC 906 via the PCI bus 922.

[0060] <<Hardware configuration of the interactive whiteboard>> Figure 5 shows an example of a hardware configuration when the electronic device 30 is implemented as an electronic whiteboard.

[0061] As shown in Figure 5, the electronic whiteboard in one embodiment includes a CPU (Central Processing Unit) 201, ROM (Read Only Memory) 202, RAM (Random Access Memory) 203, SSD (Solid State Drive) 204, network I / F (Interface) 205, and external device connection I / F 206.

[0062] Of these components, the CPU 201 controls the overall operation of the interactive whiteboard. The ROM 202 stores programs used to drive the CPU 201, such as the CPU 201 and the IPL (Initial Program Loader). The RAM 203 is used as the work area for the CPU 201. The SSD 204 stores various data, such as programs for the interactive whiteboard. The network controller 205 controls communication with the communication network N1. The external device connection I / F 206 is an interface for connecting various external devices. In this case, external devices include, for example, a USB (Universal Serial Bus) memory 230 and external devices (microphone 240, speaker 250, camera 260).

[0063] The electronic whiteboard also includes a capture device 211, a GPU 212, a display controller 213, a contact sensor 214, a sensor controller 215, an electronic pen controller 216, a short-range communication circuit 219, and an antenna 219a for the short-range communication circuit 219, a power switch 222, and selector switches 223.

[0064] Of these, the capture device 211 displays video information as a still image or video on the display of an external PC (Personal Computer) 270. The GPU (Graphics Processing Unit) 212 is a semiconductor chip specializing in graphics. The display controller 213 controls and manages the screen display in order to output the image output from the GPU 212 to the display 280, etc.

[0065] The contact sensor 214 detects when an electronic pen 290 or a user's hand H comes into contact with the display 280. The sensor controller 215 controls the processing of the contact sensor 214. The contact sensor 214 performs coordinate input and coordinate detection using an infrared blocking method. This method of coordinate input and detection involves two light-emitting and receiving devices installed at both ends of the upper part of the display 280 emitting multiple infrared rays parallel to the display 280, which are reflected by reflective members provided around the display 280, and the light received by the light-receiving element is received along the same optical path as the light emitted. The contact sensor 214 outputs the ID of the infrared rays emitted by the two light-emitting and receiving devices blocked by the object to the sensor controller 215, and the sensor controller 215 identifies the coordinate position, which is the contact position of the object.

[0066] The electronic pen controller 216 communicates with the electronic pen 290 to determine whether the pen tip or the pen tip has touched the display 280. The near-field communication circuit 219 is a communication circuit such as NFC (Near Field Communication) or Bluetooth (registered trademark). The power switch 222 is a switch for turning the electronic whiteboard's power ON / OFF. The selection switches 223 are a group of switches for adjusting, for example, the brightness and color of the display 280.

[0067] Furthermore, the electronic whiteboard is equipped with a bus line 210. The bus line 210 is an address bus, data bus, etc., for electrically connecting each component, such as the CPU 201 shown in Figure 5.

[0068] Furthermore, the contact sensor 214 is not limited to the infrared blocking method. Various detection means may be used, such as a capacitive touch panel that identifies the contact position by detecting changes in capacitance, a resistive touch panel that identifies the contact position by voltage changes between two opposing resistive films, or an electromagnetic induction touch panel that identifies the contact position by detecting electromagnetic induction caused by contact between an object and the display. In addition, the electronic pen controller 216 may be configured to determine whether or not there is touch not only at the tip and end of the electronic pen 290, but also at the part of the electronic pen 290 held by the user or other parts of the electronic pen.

[0069] <Functional Configuration of the Equipment Management System> Next, an example of the functional configuration of the equipment management system in this embodiment will be described with reference to Figure 6. Figure 6 is a block diagram illustrating the functional configuration of each device included in the equipment management system in this embodiment.

[0070] ≪Functional Configuration of the Management Device≫ As shown in Figure 6, the management device 10 in this embodiment includes an image receiving unit 11, an operation information receiving unit 12, a state management unit 13, an operation instruction unit 14, and a state storage unit 100. The state management unit 13 in this embodiment includes an object state management unit 131, an equipment state management unit 132, and an output information determination unit 133.

[0071] The image receiving unit 11, operation information receiving unit 12, state management unit 13, and operation instruction unit 14 are realized, for example, by a process in which a program deployed from the HD 504 shown in Figure 2 onto the RAM 503 causes the CPU 501 and HDD controller 505 to execute.

[0072] The state storage unit 100 is implemented, for example, using the HD504 shown in Figure 2. Reading or writing data stored in the HD504 is performed, for example, via the HDD controller 505.

[0073] The state storage unit 100 stores state management information for managing the state of equipment and objects present in the managed space R1. In this embodiment, the state management information includes a registered object list, a registered equipment list, and a detected object list.

[0074] The registered object list is a list that manages information about objects present in the managed space R1. In this embodiment, the object is a person.

[0075] The registered device list is a list that manages information about devices located in the managed space R1. In this embodiment, the device is an electronic device 30.

[0076] The detected object list is a list that manages objects detected in the managed space R1. The detected object list is a temporary list used to update the registered object list.

[0077] Here, the state management information in this embodiment will be explained with reference to Figure 7. Figure 7(A) is a conceptual diagram showing an example of a registered object list.

[0078] As shown in Figure 7(A), the registered object list in this embodiment includes, as data items, a registered object ID, a group ID, an existence confirmation flag, attribute information, and equipment operation information.

[0079] The registered object ID is identification information that identifies an object included in the registered object list.

[0080] A group ID is an identifier that identifies a group to which multiple registered objects belong. A group ID is assigned when it is determined that some of the registered objects in the registered object list belong to the same group. For registered objects that are not determined to belong to a group, the group ID is set to an initial value (e.g., zero).

[0081] A group can be, for example, colleagues, parent and child, or friends. Members of a group work together to carry out tasks such as meetings or shopping. Therefore, recognizing groups can be expected to allow for a more accurate understanding of individuals' actions. For example, when multiple employees are having a meeting in a conference room equipped with an MFP and an interactive whiteboard, it would be highly convenient if one employee authenticated using the MFP, allowing other employees to use the interactive whiteboard without needing to authenticate.

[0082] The existence confirmation flag indicates whether or not the registered object exists in the managed space R1. For example, the existence confirmation flag is set to 1 if it exists, and 0 if it does not.

[0083] Attribute information is information associated with a registered object and is necessary for its management. In this embodiment, it is the position of the registered object and the time when that position was confirmed (hereinafter also referred to as the "image acquisition time"). The position of the registered object is represented by three-dimensional coordinates (i.e., numerical values ​​on each axis in a Cartesian coordinate system such as the X, Y, and Z axes).

[0084] The method for acquiring location information varies depending on the type of monitoring device 20. For example, if the monitoring device 20 is a network camera, known techniques can be used, such as acquiring the distance to an object with a stereo camera and mapping it to three-dimensional coordinates along with its direction.

[0085] Furthermore, attribute information may include various pieces of information that describe the characteristics of the registered object, in addition to its location. For example, the color or shape of the registered object can be used. If the object is a person, the shape changes depending on their posture, so it is preferable to use color. Since the number of different colors or the area of ​​each color also changes depending on the posture, it is preferable to use the condition that at least one color is continuous when determining whether objects are the same.

[0086] Device operation information refers to information about device operations performed by a registered object. This information may include input information entered by the user during the device operation. Examples of device operation information include the functions performed by the user on the electronic device, the time of operation, the fee information for the operation, and the language used. Input information includes authentication information in authentication operations or configuration information in device configuration operations.

[0087] The number of objects in the managed space R1 changes constantly. Therefore, the number of registered objects included in the registered object list is variable.

[0088] Figure 7(B) is a conceptual diagram showing an example of a registered device list. As shown in Figure 7(B), the registered device list in this embodiment includes registered device ID, attribute information, device input information, and device output information as data items.

[0089] The registered device ID is identification information that identifies the device included in the registered device list.

[0090] Attribute information is information associated with the registered device and is necessary for its management. In this embodiment, this includes the location of the registered device, power on / off status, whether it is in sleep mode or not, whether authentication is required, whether options are installed, and status information such as maintenance in progress / scheduled maintenance time / energy saving mode. The location of the registered device is represented by three-dimensional coordinates, similar to the registered object.

[0091] Device input information (hereinafter also referred to as "usage information") is predetermined information managed when a registered device is operated. Device input information may include input information entered by the user during the operation of the device. Device input information may include, for example, the functions performed by the user on the registered device, the time of operation, and the fee information to be charged as compensation for the operation. Device input information may also include screen setting information set by the user during the operation of the device. Screen setting information may include, for example, the language used, the font used, and the size of the characters used.

[0092] Device output information is information pre-registered for registered devices. It is a list that associates the output conditions that trigger the transmission of an operation command with the operation command to be transmitted to the registered device when those output conditions are met.

[0093] The number of devices installed in the managed space R1 may change. Therefore, the number of registered devices included in the registered device list is variable.

[0094] Figure 7(C) is a conceptual diagram showing an example of a detection object list. As shown in Figure 7(C), the detection object list in this embodiment has a detection object ID, a group ID, and attribute information as data items.

[0095] The detected object ID is identification information that identifies an object included in the detected object list.

[0096] A group ID is an identifier that identifies a group containing multiple detected objects. A group ID is assigned when it is determined that some of the detected objects in the detected object list belong to the same group. For detected objects that are not determined to belong to a group, the group ID is set to an initial value (e.g., zero).

[0097] Attribute information is information associated with the detected object and is necessary for management. In this embodiment, it is the position of the detected object and the time of image acquisition. The position of the detected object is represented by three-dimensional coordinates, similar to registered objects.

[0098] The number of objects detected in the managed space R1 changes moment by moment. Therefore, the number of detected objects included in the detected object list is variable.

[0099] Let's return to Figure 6 for explanation. The image receiving unit 11 receives an image from the monitoring device 20. The image receiving unit 11 sends the received image to the status management unit 13 in response to a request from the status management unit 13.

[0100] The operation information receiving unit 12 receives device operation information from the electronic device 30. The operation information receiving unit 12 sends the received device operation information to the status management unit 13 in response to a request from the status management unit 13.

[0101] The state management unit 13 manages the state of the electronic equipment 30 and objects using the object state management unit 131 and the equipment state management unit 132. The state management unit 13 also determines the content of the operation instructions to be transmitted to the electronic equipment 30 using the output information determination unit 133.

[0102] The object state management unit 131 updates the registered object list stored in the state storage unit 100 based on the image received from the image receiving unit 11.

[0103] The device status management unit 132 updates the registered device list and registered object list stored in the status storage unit 100 based on the device operation information received from the operation information receiving unit 12.

[0104] The output information determination unit 133 determines the content of the operation instruction to be transmitted to the electronic device 30 based on the state management information stored in the state storage unit 100. The output information determination unit 133 sends the determined operation instruction to the operation instruction unit 14.

[0105] The operation instruction unit 14 transmits the operation instruction received from the status management unit 13 to the electronic device 30.

[0106] ≪Functional Configuration of Monitoring Devices≫ As shown in Figure 6, the monitoring device 20 in this embodiment includes an image acquisition unit 21 and an image transmission unit 22.

[0107] The image acquisition unit 21 acquires an image that includes the vicinity of the electronic device 30 installed in the managed space R1. The image acquisition unit 21 is implemented, for example, by a process in which a program loaded from the ROM 612 shown in Figure 3 onto the SRAM 613 causes the CPU 611 and the imaging control unit 605 to execute.

[0108] The image transmission unit 22 transmits the image acquired by the image acquisition unit 21 to the management device 10. The image transmission unit 22 is implemented, for example, by a process in which a program loaded from the ROM 612 shown in Figure 3 onto the SRAM 613 is executed by the CPU 611 and the external device connection I / F 616.

[0109] ≪Functional Configuration of Electronic Devices≫ As shown in Figure 6, the electronic device 30 in this embodiment includes an operation information transmission unit 31 and an equipment control unit 32.

[0110] The operation information transmission unit 31 and the device control unit 32 are implemented, for example, by a process in which a program deployed from the HD909 to the RAM902b, as shown in Figure 4, causes the CPU901 and HDD controller 908 to execute.

[0111] The operation information transmission unit 31 transmits equipment operation information related to the operations performed by the user to the management device 10.

[0112] The device control unit 32 receives operation instructions from the management device 10. Based on the received operation instructions, the device control unit 32 controls the operation of the electronic device 30. For example, the device control unit 32 authenticates the user using the authentication information included in the operation instructions. Also, for example, the device control unit 32 changes the settings of the electronic device 30 using the setting information included in the operation instructions.

[0113] <Procedures for managing equipment> Next, the device management method performed by the device management system in this embodiment will be described with reference to Figures 8 to 20.

[0114] ≪Basic Flowchart≫ The device management method in this embodiment involves many nested loop processes that process all combinations of each data item in one list and each data item in another list. Therefore, in this embodiment, a basic flowchart illustrating the framework of nested loop processes is introduced, and the following explanation will focus on the processes in the basic flowchart. Note that each process in the basic flowchart is performed on one combination of two data items.

[0115] The basic flowchart in this embodiment includes two basic flowcharts. The first basic flowchart is used when processing two data items in different lists using a nested loop. The second basic flowchart is used when processing two data items in the same list using a nested loop.

[0116] Figure 8 is a flowchart showing an example of the first basic flowchart in this embodiment.

[0117] In step S101, list A is read. The number of data points in list A is N. In step S102, list B is read. The number of data points in list B is M.

[0118] In step S103, the variable n is initialized to 1. In step S104, the variable m is initialized to 1. In step S105, process A1 is executed.

[0119] In step S106, it is determined whether the variable m is equal to the number of data points M. This means whether all combinations of the nth data point in list A with all the data points in list B have been processed. If the variable m is different from the number of data points M (NO), proceed to step S107. If the variable m is equal to the number of data points M (YES), proceed to step S109.

[0120] In step S107, process A3 is executed. In step S108, the variable m is incremented. Then, the process returns to step S106.

[0121] In step S109, process A2 is executed. In step S110, it is determined whether the variable n is equal to the number of data points N. This means whether all combinations of all data points in list A with all data points in list B have been processed. If the variable n is different from the number of data points N (NO), proceed to step S111. If the variable n is equal to the number of data points N (YES), proceed to step S113.

[0122] In step S111, process A5 is executed. In step S112, the variable n is incremented. Then, the process returns to step S104.

[0123] In step S113, process A4 is executed. This completes the processing for all combinations of data included in list A and list B.

[0124] Figure 9 is a flowchart showing an example of a second basic flowchart in this embodiment.

[0125] In step S201, list A is read. The number of data points in list A is N.

[0126] In step S202, the variable n is initialized to 1. In step S203, n+1 is assigned to the variable m. In step S204, process B1 is executed.

[0127] In step S205, it is determined whether the variable m is equal to N. This means whether the combinations of the nth data item in list A with all the data items in list A have been processed. If the variable m is different from N (NO), proceed to step S206. If the variable m is equal to N (YES), proceed to step S208.

[0128] In step S206, process B3 is executed. In step S207, the variable m is incremented. Then, the process returns to step S204.

[0129] In step S208, process B2 is executed. In step S209, it is determined whether the variable n is equal to N-1. This means that all the data in list A have been processed in combination with all the other data in list A. If the variable n is different from N-1 (NO), proceed to step S210. If the variable n is equal to N-1 (YES), proceed to step S212.

[0130] In step S210, process B5 is executed. In step S211, the variable n is incremented. Then, the process returns to step S203.

[0131] In step S212, process B4 is executed. This completes the processing for all combinations of data included in list A.

[0132] ≪Device management method≫ Figure 10 is a flowchart showing an example of a device management method performed by the device management system in this embodiment.

[0133] The equipment management system repeatedly executes the flowchart shown in Figure 10 at predetermined time intervals. The time interval can be set arbitrarily, but for example, it is 3 seconds.

[0134] In step S1, the image acquisition unit 21 of the monitoring device 20 acquires a first image of the vicinity of electronic device 30-1. The image acquisition unit 21 also acquires a second image of the vicinity of electronic device 30-2. The monitoring device 20 is installed so that the vicinity of electronic devices 30-1 and 30-2, which are installed in the managed space R1, is within its field of view. Therefore, the first image acquired by the image acquisition unit 21 captures the vicinity of electronic device 30-1. The second image acquired by the image acquisition unit 21 captures the vicinity of electronic device 30-2.

[0135] The vicinity of electronic device 30-1 or electronic device 30-2 refers to the range of positions from which a user can operate electronic device 30-1 or electronic device 30-2 (hereinafter also referred to as the "operable position"). The operable position is the position from which a user can reach electronic device 30-1 or electronic device 30-2. The operable position is, for example, within 1 meter of electronic device 30-1 or electronic device 30-2. Therefore, the first image is an image of electronic device 30-1 and the user using electronic device 30-1. The second image is an image of electronic device 30-2 and the user using electronic device 30-2.

[0136] If the operable positions of electronic device 30-1 and electronic device 30-2 can be captured in a single image, then the operable positions of electronic device 30-1 and electronic device 30-2 can be captured in a single image. In this case, the first image and the second image will be the same image.

[0137] The first and second images acquired by the image acquisition unit 21 are not limited to still images, but may also be videos.

[0138] Next, the image acquisition unit 21 sends the acquired first image and second image to the image transmission unit 22. The image transmission unit 22 receives the first image and second image from the image acquisition unit 21. Next, the image transmission unit 22 transmits the received first image and second image to the management device 10. In the management device 10, the image receiving unit 11 receives the first image and second image from the monitoring device 20.

[0139] In step S2, the state management unit 13 of the management device 10 requests the first image and the second image from the image receiving unit 11. Next, the state management unit 13 inputs the first image and the second image received from the image receiving unit 11 to the object state management unit 131.

[0140] Next, the object state management unit 131 executes the object state update process described later and updates the registered object list of state management information stored in the state storage unit 100. If there are multiple monitoring devices 20, the object state management unit 131 executes the object state update process for each of the multiple images received from each of the multiple monitoring devices 20.

[0141] ≪Object State Update Process≫ Here, the object state update process in this embodiment will be explained with reference to Figures 11 to 16. Figure 11 is a flowchart showing an example of the object state update process (step S2 in Figure 10) executed by the object state management unit 131 in this embodiment.

[0142] In step S21, the object state management unit 131 executes the detected object list creation process described later and creates a detected object list based on the image input from the state management unit 13.

[0143] Figure 12 is a flowchart showing an example of the process for creating a detected object list (step S21 in Figure 11) performed by the object state management unit 131 in this embodiment.

[0144] In step S21-1, the object state management unit 131 clears the detected object list stored in the state storage unit 100. That is, it deletes all data included in the detected object list.

[0145] In step S21-2, the object state management unit 131 acquires an image input from the state management unit 13. The image may be a single image received from one monitoring device 20, or multiple images received from multiple monitoring devices 20.

[0146] Next, the object state management unit 131 divides the image into predetermined blocks. Subsequently, the object state management unit 131 measures the distance between the object captured in the image and the camera for each block. The object state management unit 131 also calculates three-dimensional coordinates from the measured distances. A known method can be used to convert the distance to three-dimensional coordinates.

[0147] In step S21-3, the object state management unit 131 obtains the image acquisition time T from the acquired image. If the image acquisition time cannot be obtained from the image, the current time may be used as the image acquisition time T.

[0148] In step S21-4, the object state management unit 131 analyzes the acquired image and detects the objects captured in the image. In this embodiment, it detects the people captured in the image.

[0149] Human detection can be performed using known methods. For example, pattern matching using machine learning models can be used. Specifically, known region-based convolutional neural networks (R-CNNs) can be used to extract humans from images.

[0150] Furthermore, the accuracy of the detection can be improved by pattern matching the extracted image of the person with a pre-stored comparison image. In addition, various other known person detection methods can also be used.

[0151] In step S21-5, the object state management unit 131 assigns a detected object ID to identify the detected object.

[0152] In step S21-6, the object state management unit 131 registers the detected object ID and attribute information (location of the detected object and image acquisition time) in the detected object list. At this stage, the group ID is set to an initial value indicating that it does not belong to a group.

[0153] Let's return to Figure 11 for explanation. In step S22, the object state management unit 131 executes the registered object list update process described later and updates the registered object list based on the detected object list.

[0154] Figures 13 and 14 are flowcharts showing an example of the registered object list update process (step S22 in Figure 11) performed by the object state management unit 131 in this embodiment.

[0155] The registered object list update process is performed within the framework of the first basic flowchart (see Figure 8). Specifically, the detected object list is denoted as List A, and the registered object list as List B. Processes A1 (Figure 13) and A4 (Figure 14) are executed for all combinations of detected and registered objects. There are no processes corresponding to A2, A3, and A5.

[0156] Figure 13 is a flowchart showing an example of process A1 in the registered object list update process. Process A1 is a process for determining the identity between a registered object and a detected object and updating the position of the registered object.

[0157] Here, we will perform processing on the combination of the nth detected object in the detected object list and the mth registered object in the registered object list.

[0158] In step S22-1, the object state management unit 131 clears the existence confirmation flag for the m-th registered object. That is, it sets the existence confirmation flag for the m-th registered object to 0. This means that it is unknown whether or not the registered object exists in the managed space R1.

[0159] In step S22-2, the object state management unit 131 calculates the distance X between the three-dimensional coordinates of the m-th registered object and the three-dimensional coordinates of the n-th detected object.

[0160] In step S22-3, the object state management unit 131 determines whether the distance X is less than or equal to a predetermined threshold (for example, 1 meter). If the distance X exceeds the threshold (NO), the object state management unit 131 terminates process A1. This means that it has been determined that the two objects are not the same person because they are far apart. If the distance X is less than or equal to the threshold (YES), the object state management unit 131 proceeds to step S22-4.

[0161] In step S22-4, the object state management unit 131 adds the three-dimensional position and video acquisition time of the nth detected object to the attribute information of the mth registered object. This means that because the distance between the two is small, it is determined that they are the same person, and the position of the nth detected object is the position of the mth registered object at the image acquisition time T.

[0162] The attribute information in the registered object list increases each time the object state update process is performed, as long as the registered object is detected in the image. Conversely, the attribute information in the registered object list represents the time series of positions in which the registered object was detected within the managed space. Therefore, the movement of the registered object can be tracked using the attribute information in the registered object list.

[0163] In step S22-5, the object state management unit 131 sets the existence confirmation flag for the m-th registered object. That is, it sets the existence confirmation flag for the m-th registered object to 1. This means that since an object matching the registered object was detected in step S22-3, it was determined that the registered object was also present in the managed space R1 at the image acquisition time T.

[0164] Figure 14 is a flowchart showing an example of process A4 in the registered object list update process. Process A4 is the process of deleting registered objects from the registered object list whose existence could not be confirmed in process A1.

[0165] In step S22-6, the object state management unit 131 initializes the variable m to 1.

[0166] In step S22-7, the object state management unit 131 determines whether the existence of the m-th registered object has been confirmed. Specifically, it determines whether the existence confirmation flag is 1 or 0. If the existence confirmation flag is 1 (YES), the object state management unit 131 proceeds to step S22-9. If the existence confirmation flag is 0 (NO), the object state management unit 131 proceeds to step S22-8.

[0167] In step S22-8, the object state management unit 131 deletes the m-th registered object from the registered object list.

[0168] In step S22-9, the object state management unit 131 determines whether the variable m is equal to the number of data points M. If the variable m is different from the number of data points M (NO), the object state management unit 131 proceeds to step S22-10. If the variable m is equal to the number of data points M (YES), the object state management unit 131 terminates the process.

[0169] In step S22-10, the object state management unit 131 increments the variable m. After that, the object state management unit 131 returns to step S22-7.

[0170] Let's return to Figure 11 for explanation. In step S23, the object state management unit 131 executes the group ID assignment process described later and assigns a group ID to the registered objects included in the registered object list.

[0171] The group ID assignment process is performed within the framework of the second basic flowchart (see Figure 9). Specifically, the registered object list is designated as List A, and process B1 (Figure 15 or Figure 16) is executed for all combinations of two registered objects. Note that there are no processes corresponding to processes B2-B5.

[0172] Various methods can be considered for determining whether one object belongs to the same group as another object. In this embodiment, group determination by proximity and group determination by group behavior will be described. However, the group determination method is not limited to these, and any technology that enables group determination of objects from an image may be used.

[0173] Figure 15 is a flowchart showing a first example of the group ID assignment process (step S23 in Figure 11) performed by the object state management unit 131 in this embodiment. The first example of the group ID assignment process is group determination based on proximity.

[0174] Group determination based on proximity is a process that registers two objects detected in close proximity for a predetermined number of consecutive times as a group. The predetermined number of times is, for example, 5 times. If the time interval for executing the device management method is 3 seconds, then two objects detected in close proximity for 15 seconds consecutively will be determined to be the same group.

[0175] Here, we will perform processing on the combination of the mth registered object and the nth registered object in the registered object list.

[0176] In step S23A-1, the object state management unit 131 initializes variables k and j to 1. Variable k is a counter that represents the number of times it has been determined whether or not two objects are in close proximity. Variable j is a counter that represents the number of times it has been determined that two objects are in close proximity.

[0177] In step S23A-2, the object state management unit 131 calculates the distance X between the three-dimensional coordinates of the m-th registered object k steps prior and the three-dimensional coordinates of the n-th registered object k steps prior.

[0178] In step S23A-3, the object state management unit 131 determines whether the distance X is less than a predetermined threshold (for example, 1 meter). If the distance X is less than the threshold (YES), the object state management unit 131 proceeds to step S23A-4. If the distance X is greater than or equal to the threshold (NO), the object state management unit 131 proceeds to step S23A-5.

[0179] In step S23A-4, the object state management unit 131 increments the variable j.

[0180] In step S23A-5, the object state management unit 131 increments the variable k.

[0181] In step S23A-6, the object state management unit 131 determines whether the variable k is equal to a predetermined number of times K. If the variable k is different from the predetermined number of times K (NO), the object state management unit 131 returns to step S23A-2. If the variable k is equal to the predetermined number of times K (YES), the object state management unit 131 proceeds to step S23A-7.

[0182] In step S23A-7, the object state management unit 131 determines whether the variable j is equal to a predetermined number of times K. If the variable j is different from the predetermined number of times K (NO), the object state management unit 131 terminates the process. If the variable j is equal to the predetermined number of times K (YES), the object state management unit 131 proceeds to step S23A-8.

[0183] In step S23A-8, the object state management unit 131 determines whether a group ID has been assigned to either the m-th registered object or the n-th registered object. If a group ID has been assigned (YES), the object state management unit 131 proceeds to step S23A-9. If a group ID has not been assigned (NO), the object state management unit 131 proceeds to step S23A-10.

[0184] In step S23A-9, the object state management unit 131 determines which of the group IDs assigned to the m-th registered object and the n-th registered object will be assigned as the group ID to be assigned. Next, the object state management unit 131 sets the determined group ID for the group IDs of the m-th registered object and the n-th registered object in the registered object list.

[0185] The object state management unit 131 determines the group ID to be assigned as follows: If a group ID has been assigned to only one registered object, that group ID will be used as the group ID to be assigned. In other words, registered objects that have not been assigned a group ID will be added as members of an existing group.

[0186] If both registered objects have a group ID, first, sort the m+1th and subsequent registered objects by group ID to identify the registered objects that share the same group ID as the mth registered object. Next, update the group IDs of the mth registered object and all identified registered objects with the group ID of the nth registered object. This ensures that all members of the group to which the mth registered object belongs also belong to the same group as the nth registered object. This allows us to identify groups that contain three or more registered objects.

[0187] In step S23A-10, the object status management unit 131 issues a new group ID that does not overlap with other group IDs. Next, the object status management unit 131 sets the newly issued group ID to the group IDs of the m-th and n-th registered objects in the registered object list.

[0188] Figure 16 is a flowchart showing a second example of the group ID assignment process (step S23 in Figure 11) executed by the object state management unit 131 in this embodiment. The second example of the group ID assignment process is group determination based on group actions.

[0189] Group determination by group action is a process that performs group determination by image analysis after creating a list of detected objects. Therefore, when performing group determination by group action, step S23 is executed between step S21 and step S22.

[0190] Here, we will perform processing on the combination of the mth detected object and the nth detected object in the list of detected objects.

[0191] In step S23B-1, the object state management unit 131 calculates the distance X between the three-dimensional coordinates of the nth detected object and the three-dimensional coordinates of the mth detected object.

[0192] In step S23B-2, the object state management unit 131 determines whether the distance X is less than a predetermined threshold (for example, 1 meter). If the distance X is greater than or equal to the threshold (NO), the object state management unit 131 terminates the process. If the distance X is less than the threshold (YES), the object state management unit 131 proceeds to step S23B-3.

[0193] In step S23B-3, the object state management unit 131 calculates the midpoint between the three-dimensional coordinates of the nth detected object and the three-dimensional coordinates of the mth detected object. Specifically, the object state management unit 131 divides the sum of the X, Y, and Z coordinates of the two detected objects by 2.

[0194] In step S23B-4, the object state management unit 131 extracts an image P with radius Y pixels centered on the midpoint. The radius Y is set according to the resolution of the image so that the area included in the image P is approximately 1m in real space.

[0195] In step S23B-5, the object state management unit 131 reads a pre-stored group action image. A group action image is a set of images representing actions that are determined to belong to the same group. For example, an image of two people greeting or shaking hands.

[0196] In step S23B-6, the object state management unit 131 calculates the similarity between the video P and the group action image using pattern matching or the like.

[0197] In step S23B-7, the object state management unit 131 determines whether any of the calculated similarities exceed a predetermined threshold. If there are similarities that exceed the threshold (YES), the object state management unit 131 proceeds to step S23B-8. If there are no similarities that exceed the threshold (NO), the object state management unit 131 terminates the process.

[0198] In step S23B-8, the object state management unit 131 issues a new group ID that does not overlap with any other group IDs. Next, the object state management unit 131 sets the newly issued group ID to the group IDs of the m-th detected object and the n-th detected object in the detected object list.

[0199] In step S22-4 (see Figure 13) of the subsequent registered object list update process, the object state management unit 131 sets the group ID of the detected object determined to be the same object as the group ID of the registered object. If an existing group ID has been assigned to the registered object, steps S23A-8 to S23A-10 (see Figure 15) of the first example of the group ID assignment process are executed.

[0200] Let's return to Figure 10 for explanation. In step S3, the operation information transmission unit 31 of the electronic device 30-1 determines whether the operation performed by the user on the electronic device 30-1 is information that should be notified to the management device 10. This determination is made based on whether or not it matches one of the predetermined operation contents.

[0201] If the operation information transmission unit 31 determines that the operation content should be notified, it transmits equipment operation information related to the equipment operation to the management device 10. In the management device 10, the operation information receiving unit 12 receives the equipment operation information from the electronic device 30-1.

[0202] The operation content and operation information in this embodiment are illustrated below.

[0203] <Example 1> Operation details: Device login Operation information: Registered device ID, device authentication ID (if authentication is required), login time

[0204] <Example 2> Operation details: Processes that incur charges (copying, outputting, use of pay-per-use software, etc.) Operation information: Registered device ID, device authentication ID (if authentication is required), fee, input data, language used

[0205] <Example 3> Operation details: None Operation information: Status information such as maintenance or power saving mode.

[0206] The registered device ID is identification information included in the registered device list stored in the state storage unit 100 of the management device 10. A registered device ID is assigned to each electronic device 30 installed in the managed space R1. The device authentication ID is authentication information used for authentication by the user of the electronic device 30 to use that electronic device 30.

[0207] In step S4, the status management unit 13 of the management device 10 requests operation information from the operation information receiving unit 12. Next, the status management unit 13 inputs the operation information received from the operation information receiving unit 12 to the equipment status management unit 132.

[0208] Next, the device status management unit 132 executes the device status update process described later and updates the registered device list of status management information stored in the status storage unit 100.

[0209] <<Equipment Status Update Process>> Here, the equipment status update process in this embodiment will be explained with reference to Figure 17. Figure 17 is a flowchart showing an example of the equipment status update process (step S4 in Figure 10) executed by the equipment status management unit 132 in this embodiment.

[0210] In step S41, the device status management unit 132 identifies a registered device included in the registered device list based on the registered device ID contained in the device operation information. Next, the device status management unit 132 sets the received device operation information in the device input information of the identified registered device.

[0211] In step S42, the device status management unit 132 initializes the variable n to 1.

[0212] In step S43, the equipment status management unit 132 calculates the distance X between the three-dimensional coordinates of the identified registered equipment and the latest three-dimensional coordinates of the nth registered object. The three-dimensional coordinates of the registered equipment are set in advance using the same method as the three-dimensional coordinates of the registered objects. Since the installation location of the registered equipment may be moved, it is necessary to update the three-dimensional coordinates of the registered equipment periodically, but the update frequency may be low.

[0213] In step S44, the equipment status management unit 132 determines whether the distance X is less than a predetermined threshold (for example, 1 meter). If the distance X is greater than or equal to the threshold (NO), the equipment status management unit 132 proceeds to step S47. If the distance X is less than the threshold (YES), the equipment status management unit 132 proceeds to step S45.

[0214] In step S45, the equipment status management unit 132 adds the equipment input information of the identified registered equipment to the equipment operation information of the nth registered object. The equipment operation information in the registered object list is configured to store a predetermined number of equipment input information.

[0215] In step S46, the device status management unit 132 determines whether the variable n is equal to the number of data points N. If the variable n is different from the number of data points N (NO), the device status management unit 132 proceeds to step S47. If the variable n is equal to the number of data points N (YES), the device status management unit 132 terminates the process.

[0216] In step S47, the equipment status management unit 132 increments the variable n. Then, the equipment status management unit 132 returns to step S43.

[0217] Let's return to Figure 10 for explanation. In step S5, the output information determination unit 133 of the management device 10 executes the output information determination process described later and determines the operation instruction to be transmitted to the electronic device 30 based on the state management information stored in the state storage unit 100.

[0218] <<Output Information Determination Process>> Here, the output information determination process in this embodiment will be explained with reference to Figures 18 and 19. Figure 18 is a flowchart showing an example of the output information determination process (step S5 in Figure 10) executed by the output information determination unit 133 in this embodiment.

[0219] The output information determination process is carried out within the framework of the first basic flowchart (see Figure 8). Specifically, the registered equipment list is designated as List A, and the registered object list as List B. Process A1 (Figure 18) is executed for all combinations of registered equipment and registered objects. Note that there are no processes corresponding to A2-A5.

[0220] Here, processing will be performed on the combination of the nth registered device in the registered device list and the mth registered object in the registered object list.

[0221] In step S51, the output information determination unit 133 calculates the distance X between the latest three-dimensional coordinates of the nth registered device and the latest three-dimensional coordinates of the mth registered object.

[0222] In step S52, the output information determination unit 133 determines whether the distance X is less than a predetermined threshold (for example, 1 meter). If the distance X is greater than or equal to the threshold (NO), the equipment status management unit 132 terminates the process. If the distance X is less than the threshold (YES), the equipment status management unit 132 proceeds to step S53.

[0223] In step S53, the output information determination unit 133 obtains the output conditions included in the device output information of the nth registered device from the registered device list stored in the state storage unit 100.

[0224] In step S54, the output information determination unit 133 obtains the equipment operation information of the m-th registered object from the registered object list stored in the state storage unit 100.

[0225] In step S55, the output information determination unit 133 determines whether any of the equipment operation information acquired in step S54 satisfies the output conditions acquired in step S53. If none of the equipment operation information satisfies the output conditions (NO), the equipment status management unit 132 terminates the process. If any of the equipment operation information satisfies the output conditions (YES), the equipment status management unit 132 proceeds to step S56.

[0226] In step S56, the output information determination unit 133 obtains an operation instruction corresponding to the output condition acquired in step S53 from the device output information of the nth registered device. Note that the operation instruction for the registered device may include instructions that are processed adaptively according to the attribute information of the registered device. For example, it may include a condition that does not send an operation instruction when the registered device is in an inoperable state. An inoperable state is, for example, a power-off state or a hibernation state.

[0227] Figure 19 is a flowchart showing a modified example of the output condition determination process (step S55 in Figure 18) executed by the output information determination unit 133 in this embodiment.

[0228] In the output information determination process shown in Figure 18, step S55 determines whether the output conditions are met based solely on the equipment operation information of the m-th registered object. In the output information determination process shown in Figure 19, the output conditions are met by referring to the equipment operation information of devices other than the m-th registered device.

[0229] In step S55-1, the output information determination unit 133 determines whether any of the device operation information of the m-th registered object satisfies the output conditions of the n-th registered device. If any of the device operation information satisfies the output conditions (YES), the output information determination unit 133 proceeds to step S55-10. If none of the device operation information satisfies the output conditions (NO), the output information determination unit 133 proceeds to step S55-2.

[0230] In step S55-2, the output information determination unit 133 determines whether or not device operation information for other registered objects is required based on the output conditions of the nth registered device. If device operation information for other registered objects is required (YES), the output information determination unit 133 proceeds to step S55-3. If device operation information for other registered objects is not required (NO), the output information determination unit 133 proceeds to step S55-9.

[0231] In step S55-3, the output information determination unit 133 initializes the variable k to 1.

[0232] In step S55-4, the output information determination unit 133 determines whether variable k is equal to variable m. If variable k is equal to variable m (YES), the output information determination unit 133 proceeds to step S55-5. If variable k is not equal to variable m (NO), the output information determination unit 133 proceeds to step S55-7.

[0233] In step S55-5, the output information determination unit 133 obtains the equipment operation information of the k-th registered object from the registered object list stored in the state storage unit 100.

[0234] In step S55-6, the output information determination unit 133 determines whether any of the equipment operation information of the k-th registered object satisfies the output conditions of the n-th registered equipment. If any of the equipment operation information satisfies the output conditions (YES), the output information determination unit 133 proceeds to step S55-10. If none of the equipment operation information satisfies the output conditions (NO), the output information determination unit 133 proceeds to step S55-7.

[0235] In step S55-7, the output information determination unit 133 determines whether variable k is equal to the number of data points M. If variable k is different from the number of data points M (NO), the output information determination unit 133 proceeds to step 55-8. If variable m is equal to the number of data points M (YES), the output information determination unit 133 proceeds to step S55-9.

[0236] In step S55-8, the output information determination unit 133 increments the variable k. After that, the output information determination unit 133 returns to step S55-4.

[0237] In step S55-9, the output information determination unit 133 determines that the equipment operation information acquired in step S54 does not satisfy the output conditions acquired in step S53, and terminates the process.

[0238] In step S55-10, the output information determination unit 133 determines that the equipment operation information acquired in step S54 satisfies the output conditions acquired in step S53, and terminates the process.

[0239] Figure 20 shows an example of device output information in this embodiment. The first example of the output conditions for device output information in this embodiment is an output condition in which device operation information of other registered objects is not required. The second example of the output conditions for device output information in this embodiment is an output condition in which device operation information of other registered objects is required.

[0240] As shown in Figure 20, a first example of the output conditions in this embodiment is that the authentication information entered by the m-th registered object (hereinafter also referred to as "person α") on another electronic device (here, an MFP) is authentication information that can be authenticated on the n-th electronic device (here, an electronic whiteboard).

[0241] As shown in Figure 20, if it is determined that the first example of the output conditions is met, one or more of the following operation instructions will be determined to be sent to the registered device. The first operation instruction is to automatically authenticate person α on the electronic whiteboard. In this case, the operation instruction to be sent will include the authentication information entered by person α on the MFP. The second operation instruction is to display "α, automatic authentication completed" on the electronic whiteboard for a few seconds.

[0242] The third instruction is to display the message "Do you want to display the information just entered on device XX (name of MFP)?" and a "Yes" or "No" selection button if person α is inputting information on the MFP (for example, performing a scan). If "Yes" is selected, the information entered by the scanner will be displayed on the electronic whiteboard.

[0243] The fourth instruction is to identify the language used by person α, as set in the MFP, from the device operation information, and reflect this in the display on the electronic whiteboard. The fifth instruction is not to send an instruction if the electronic whiteboard's attribute information indicates a dormant state such as "power saving mode".

[0244] As shown in Figure 20, a second example of the output conditions in this embodiment is that the authentication information entered by the k-th registered object (hereinafter also referred to as "person β") on another electronic device (here, an MFP) is authentication information that can be authenticated on the n-th electronic device (here, an electronic whiteboard). Person β is assumed to have the same group ID as person α.

[0245] As shown in Figure 20, if it is determined that the second example of the output conditions is met, one or more of the following operation instructions will be determined to be sent to the registered device. The first operation instruction is to automatically authenticate person α on the electronic whiteboard. In this case, the operation instruction to be sent will include the authentication information entered by person β on the MFP. The second operation instruction is to display "Automatic authentication performed with β's authentication information" on the electronic whiteboard for a few seconds.

[0246] The third instruction is to display the message "Do you want to display the information entered by person β on the MFP (e.g., a scan operation)?" along with a "Yes" or "No" selection button, if person β is inputting information on the MFP. If "Yes" is selected, the information entered by the scanner will be displayed on the electronic whiteboard.

[0247] Let's return to Figure 10 for explanation. In step S6, the output information determination unit 133 of the management device 10 sends operation instruction information representing an operation instruction to the operation instruction unit 14. The operation instruction information includes the determined operation instruction and information indicating the destination electronic device 30-2.

[0248] The operation instruction unit 14 receives operation instruction information from the output information determination unit 133. Next, the operation instruction unit 14 transmits the operation instruction contained in the operation instruction information to the destination electronic device 30-2 included in the operation instruction information.

[0249] In the electronic device 30-2, the device control unit 32 receives an operation instruction from the management device 10. Next, the device control unit 32 executes an operation using the input information contained in the received operation instruction. For example, the device control unit 32 authenticates the user using the authentication information contained in the operation instruction. Alternatively, for example, the device control unit 32 changes the settings of the electronic device 30-2 using the setting information contained in the operation instruction.

[0250] <Effects of the Embodiment> In this embodiment, the device management system 1 tracks the movement of a person based on images taken near the electronic device 30, and transmits an operation instruction using information entered by the user in the first electronic device 30-1 to the second electronic device 30-2, which has detected the presence of the user nearby. Therefore, according to the device management system 1 in this embodiment, the second electronic device can be made to execute an operation using information entered in the first electronic device.

[0251] In particular, by sending an authentication instruction to the second electronic device 30-2 using the authentication information authenticated by the first electronic device 30-1, authentication can be automatically performed when a user authenticated by the first electronic device 30-1 moves near the second electronic device 30-2. In other words, the device management system in this embodiment enables single sign-on through hardware cooperation.

[0252] [Application Examples] In the above embodiment, the managed space was assumed to be an office conference room or workspace, and the example of managing the electronic device 30 as an OA (Office Automation) device was mainly described. However, the applicable usage scenarios for the device management system 1 are not limited to this, and it can be applied to various usage scenarios.

[0253] For example, the equipment management system 1 can be configured to manage electronic devices installed in a hotel. Hotels have various electronic devices installed. For example, there may be an automated check-in machine in the lobby. Also, guest rooms may have set-top boxes that can play television broadcasts and on-demand videos. By managing these devices with the equipment management system 1, it becomes possible to automatically set the language used by the hotel guest on the automated check-in machine to the language used on the set-top box.

[0254] Furthermore, for example, the equipment management system 1 can be configured to manage electronic devices installed at an airport. At an airport, passengers must go through procedures such as ticketing, check-in, and baggage inspection before boarding an aircraft, and boarding passes are processed using dedicated electronic devices at each step. By managing these devices with the equipment management system 1, it becomes possible to automatically set the language used by the electronic device performing the previous procedure to the language used by the electronic device performing the next procedure.

[0255] Furthermore, for example, the device management system 1 can be configured to manage electronic devices such as search terminals installed in bookstores. By linking the searches performed by customers on the search terminals with the information of the bookshelves they subsequently moved to, it can be used to enhance marketing efforts.

[0256] Furthermore, for example, the equipment management system 1 can be configured to manage electronic devices such as PCs installed in the factory. By correlating and verifying the actions of workers with the operations performed on equipment installed at the work site, the appropriateness of the workers' work can be monitored.

[0257] Furthermore, for example, the equipment management system 1 can be configured to manage electronic devices installed in an unmanned convenience store. Convenience stores are equipped with electronic devices such as MFPs and payment machines. By managing these devices with the equipment management system 1, it becomes possible, for example, to set the language used by the customer on the MFP as the language used by the payment machine.

[0258] [supplement] In each of the above embodiments, the management device 10 is an example of an information processing device. The equipment management system 1 is an example of an information processing system. The monitoring device 20 is an example of a camera. The status management unit 13 is an example of a user identification unit. The operation information receiving unit 12 is an example of an input information receiving unit.

[0259] Each of the embodiments described above can be implemented by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each of the functions described above.

[0260] The apparatus described in the examples represents only one of several computing environments for carrying out the embodiments disclosed herein. In one embodiment, the management device 10 includes multiple computing devices, such as a server cluster. The multiple computing devices are configured to communicate with each other via any type of communication link, including a network or shared memory, and perform the processing disclosed herein.

[0261] Although embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications or changes are possible within the scope of the gist of the present invention as described in the claims. [Explanation of Symbols]

[0262] 1. Equipment Management System 10 Management device 11 Image receiving unit 12 Operation Information Receiving Unit 13. Status Management Department 14 Operation instruction section 100 State memory unit 131 Object State Management Unit 132 Equipment Status Management Department 133 Output Information Determination Unit 20 Monitoring equipment 21 Image acquisition unit 22 Image transmission unit 30 Electronic equipment 31 Operation Information Transmission Unit 32 Equipment Control Unit [Prior art documents] [Patent Documents]

[0263] [Patent Document 1] Patent No. 6064636 [Patent Document 2] Patent No. 5238409

Claims

1. An information processing device capable of communicating with a first electronic device and a second electronic device via a network, A first image receiving unit that receives a first image of a first user who is located in an operable position of the first electronic device, A first user identification unit identifies the first user based on the first image, An input information receiving unit that receives input information entered by the first user on the first electronic device from the first electronic device, A second image receiving unit that receives a second image of a second user who is located in an operable position of the second electronic device, A second user identification unit identifies the second user based on the second image, When the first user identified by the first user identification unit matches the second user identified by the second user identification unit, and the input information satisfies the output conditions, a transmission unit transmits information to the second electronic device based on the input information. An acquisition unit that acquires an operation instruction corresponding to the output conditions, for controlling the operation of the second electronic device, Equipped with, The transmitting unit transmits the acquired operation instruction to the second electronic device. Information processing device.

2. An information processing device capable of communicating with a first electronic device and a second electronic device via a network, A first image receiving unit that receives a first image of a first user who is located in an operable position of the first electronic device, A first user identification unit identifies the first user based on the first image, An input information receiving unit that receives authentication information entered by the first user on the first electronic device from the first electronic device, A second image receiving unit that receives a second image of a second user who is located in an operable position of the second electronic device, A second user identification unit identifies the second user based on the second image, A transmitting unit transmits the authentication information and a request for authentication processing based on the authentication information to the second electronic device when the first user identified by the first user identification unit matches the second user identified by the second user identification unit. An information processing device equipped with the following features.

3. An information processing device capable of communicating with a first electronic device and a second electronic device via a network, A first image receiving unit that receives a first image of a first user who is located in an operable position of the first electronic device, A first user identification unit identifies the first user based on the first image, An input information receiving unit that receives input information entered by the first user on the first electronic device from the first electronic device, A second image receiving unit that receives a second image of a second user who is located in an operable position of the second electronic device, A second user identification unit identifies the second user based on the second image, A transmission unit transmits information to the second electronic device based on the input information when the first user identified by the first user identification unit matches the second user identified by the second user identification unit. Equipped with, The input information receiving unit further receives state information representing the state of the second electronic device, The transmitting unit does not transmit the information to the second electronic device in which the status information indicates a paused state. Information processing device.

4. The input information includes usage information of the first user using the first electronic device. The information transmitted by the transmission unit includes the input information. The information processing apparatus according to claim 1 or 3.

5. The aforementioned input information includes authentication information, The information processing apparatus according to claim 1 or 3.

6. The information transmitted by the transmission unit includes the input information. The information processing apparatus according to claim 1 or 3.

7. The information transmitted by the transmitting unit includes instructions for operation in the second electronic device. The information processing apparatus according to claim 1 or 3.

8. The system further includes a state management unit that records the location information of the first user and the second user in chronological order using the first image and the second image. An information processing apparatus according to any one of claims 1 to 3.

9. The state management unit identifies the first user who operated the first electronic device based on the time series of location information. The information processing apparatus according to claim 8.

10. The state management unit records the position information using a plurality of first images and second images. The information processing apparatus according to claim 9.

11. An information processing system in which a first electronic device, a second electronic device, and an information processing device can communicate via a network, The aforementioned information processing device is A first image receiving unit that receives a first image of a first user who is located in an operable position of the first electronic device, A first user identification unit identifies the first user based on the first image, An input information receiving unit that receives input information entered by the first user on the first electronic device from the first electronic device, A second image receiving unit that receives a second image of a second user who is located in an operable position of the second electronic device, A second user identification unit identifies the second user based on the second image, When the first user identified by the first user identification unit matches the second user identified by the second user identification unit, and the input information satisfies the output conditions, a transmission unit transmits information to the second electronic device based on the input information. An acquisition unit that acquires an operation instruction corresponding to the output conditions, for controlling the operation of the second electronic device, Equipped with, The transmitting unit transmits the acquired operation instruction to the second electronic device. Information processing system.

12. An information processing system in which a first electronic device, a second electronic device, and an information processing device can communicate via a network, The aforementioned information processing device is A first image receiving unit that receives a first image of a first user who is located in an operable position of the first electronic device, A first user identification unit identifies the first user based on the first image, An input information receiving unit that receives authentication information entered by the first user on the first electronic device from the first electronic device, A second image receiving unit that receives a second image of a second user who is located in an operable position of the second electronic device, A second user identification unit identifies the second user based on the second image, A transmitting unit transmits the authentication information and a request for authentication processing based on the authentication information to the second electronic device when the first user identified by the first user identification unit matches the second user identified by the second user identification unit. An information processing system equipped with the following features.

13. An information processing system in which a first electronic device, a second electronic device, and an information processing device can communicate via a network, The aforementioned information processing device is A first image receiving unit that receives a first image of a first user who is located in an operable position of the first electronic device, A first user identification unit identifies the first user based on the first image, An input information receiving unit that receives input information entered by the first user on the first electronic device from the first electronic device, A second image receiving unit that receives a second image of a second user who is located in an operable position of the second electronic device, A second user identification unit identifies the second user based on the second image, A transmission unit transmits information to the second electronic device based on the input information when the first user identified by the first user identification unit matches the second user identified by the second user identification unit. Equipped with, The input information receiving unit further receives state information representing the state of the second electronic device, The transmitting unit does not transmit the information to the second electronic device in which the status information indicates a paused state. Information processing system.

14. A computer capable of communicating with the first electronic device and the second electronic device via a network, A first image receiving procedure for receiving a first image of a first user who is located in an operable position of the first electronic device, A first user identification procedure for identifying the first user based on the first image, An input information receiving procedure for receiving input information entered by the first user on the first electronic device from the first electronic device, A second image receiving procedure for receiving a second image of a second user who is located in an operable position of the second electronic device, A second user identification procedure for identifying the second user based on the second image, A transmission procedure to transmit information to the second electronic device based on the input information when the first user identified by the first user identification procedure matches the second user identified by the second user identification procedure, and the input information satisfies the output conditions, An acquisition procedure for acquiring an operation instruction corresponding to the output conditions, which is for controlling the operation of the second electronic device, Execute, The transmission procedure involves transmitting the acquired operation instruction to the second electronic device. Equipment management method.

15. A computer capable of communicating with the first electronic device and the second electronic device via a network, A first image receiving procedure for receiving a first image of a first user who is located in an operable position of the first electronic device, A first user identification procedure for identifying the first user based on the first image, An input information reception procedure for receiving authentication information entered by the first user on the first electronic device from the first electronic device, A second image receiving procedure for receiving a second image of a second user who is located in an operable position of the second electronic device, A second user identification procedure for identifying the second user based on the second image, A transmission procedure that, if the first user identified by the first user identification procedure matches the second user identified by the second user identification procedure, transmits the authentication information and a request for authentication processing based on the authentication information to the second electronic device. A device management method for performing this task.

16. A computer capable of communicating with the first electronic device and the second electronic device via a network, A first image receiving procedure for receiving a first image of a first user who is located in an operable position of the first electronic device, A first user identification procedure for identifying the first user based on the first image, An input information receiving procedure for receiving input information entered by the first user on the first electronic device from the first electronic device, A second image receiving procedure for receiving a second image of a second user who is located in an operable position of the second electronic device, A second user identification procedure for identifying the second user based on the second image, A transmission procedure to transmit information to the second electronic device based on the input information when the first user identified by the first user identification procedure matches the second user identified by the second user identification procedure, Execute, The input information receiving procedure further receives state information representing the state of the second electronic device, The transmission procedure does not transmit the information to the second electronic device in which the status information indicates a paused state. Equipment management method.

17. A computer capable of communicating with the first electronic device and the second electronic device via a network, A first image receiving procedure for receiving a first image of a first user who is located in an operable position of the first electronic device, A first user identification procedure for identifying the first user based on the first image, An input information receiving procedure for receiving input information entered by the first user on the first electronic device from the first electronic device, A second image receiving procedure for receiving a second image of a second user who is located in an operable position of the second electronic device, A second user identification procedure for identifying the second user based on the second image, A transmission procedure to transmit information to the second electronic device based on the input information when the first user identified by the first user identification procedure matches the second user identified by the second user identification procedure, and the input information satisfies the output conditions, An acquisition procedure for acquiring an operation instruction corresponding to the output conditions, which is for controlling the operation of the second electronic device, Make it run, The transmission procedure involves transmitting the acquired operation instruction to the second electronic device. program.

18. A computer capable of communicating with the first electronic device and the second electronic device via a network, A first image receiving procedure for receiving a first image of a first user who is located in an operable position of the first electronic device, A first user identification procedure for identifying the first user based on the first image, An input information reception procedure for receiving authentication information entered by the first user on the first electronic device from the first electronic device, A second image receiving procedure for receiving a second image of a second user who is located in an operable position of the second electronic device, A second user identification procedure for identifying the second user based on the second image, A transmission procedure that, if the first user identified by the first user identification procedure matches the second user identified by the second user identification procedure, transmits the authentication information and a request for authentication processing based on the authentication information to the second electronic device. A program to execute.

19. A computer capable of communicating with the first electronic device and the second electronic device via a network, A first image receiving procedure for receiving a first image of a first user who is located in an operable position of the first electronic device, A first user identification procedure for identifying the first user based on the first image, An input information receiving procedure for receiving input information entered by the first user on the first electronic device from the first electronic device, A second image receiving procedure for receiving a second image of a second user who is located in an operable position of the second electronic device, A second user identification procedure for identifying the second user based on the second image, A transmission procedure to transmit information to the second electronic device based on the input information when the first user identified by the first user identification procedure matches the second user identified by the second user identification procedure, Make it run, The input information receiving procedure further receives state information representing the state of the second electronic device, The transmission procedure does not transmit the information to the second electronic device in which the status information indicates a paused state. program.

Images