Information processing device and program

Abstract

This configuration reduces the user's operational burden compared to a system where operations performed on one device are not automatically transferred to the other when an operation is performed on the other device before an operation on the other device is completed. [Solution] An information processing device 10 comprising an operation panel 15 on which a user operates in response to a display, and a processor that receives operations on the operation panel 15 and controls the display output to the operation panel 15 in accordance with the received operations, wherein the processor outputs a display to the user terminal 20 when a user terminal 20 is connected to the information processing device 10 in a communicable manner, receives operations on the user terminal 20, and when predetermined conditions are met, switches the display output to either the operation panel 15 or the user terminal 20 and the acceptance of operations on one to the other and continues control.

Description

Technical Field

[0003] , , , , , , , ,

[0004] , , , ,

[0001] The present invention relates to an information processing apparatus and a program.

Background Art

[0002] For example, Patent Document 1 discloses a function expansion type system having a second operation panel for performing operations on the entire expanded device in a device having a first operation panel whose functions can be expanded by connecting an external device. Such a system enables status display and operation to be performed from the first operation panel separately from the status display and operation of the entire device by the second operation panel. Further, Patent Document 2 discloses a configuration for remotely controlling an image processing apparatus with an information terminal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0004] Here, consider a case where a user who is operating on either the operation unit of the information processing apparatus or the user terminal connected to the information processing apparatus performs an operation on the other before finishing the operation on one side. In such a case, if the operation performed on one side has to be repeated on the other side, the operation burden on the user increases. If a configuration is adopted in which the operation performed on one side is taken over by the other side, it becomes possible to reduce the operation burden on the user. An object of the present invention is to reduce the operation burden on the user as compared with a case where a configuration in which an operation performed on one side is taken over by the other side is not adopted when performing an operation on the other side before finishing the operation on one side between the operation unit of the information processing apparatus and the user terminal connected to the information processing apparatus. [Means for solving the problem]

[0005] The invention described in claim 1 is an information processing device comprising: an operation unit in which a user operates in response to a display; and a processor that receives the operation to the operation unit and controls the display output to the operation unit in response to the received operation, wherein the processor, when a user terminal is connected to the information processing device in a communicable manner, outputs the display to the user terminal, receives the operation to the user terminal, and, when predetermined conditions are met, switches the output of the display to either the operation unit or the user terminal and the acceptance of the operation to either one to the other and continues control. The invention described in claim 2 is an information processing apparatus according to claim 1, characterized in that the predetermined condition is a condition indicating that the operating unit has become usable by another user when it is currently unavailable to another user due to being in use, and one of the terms is the user terminal, and the other is the operating unit of the information processing apparatus. The invention described in claim 3 is an information processing device according to claim 2, characterized in that the condition indicating that the above-mentioned possible situation has occurred is the detection of the occurrence of a state in which it is highly likely that the operating unit is not in use. The invention described in claim 4 is an information processing apparatus according to claim 2, characterized in that the condition indicating that the above-mentioned possible situation has been reached is that an operation indicating the end of use of the operating unit has been performed. The invention described in claim 5 is an information processing apparatus according to claim 1, characterized in that, in the case where one of the members is the user terminal and the other is the operation unit of the information processing apparatus, the user terminal is in a predetermined positional relationship with respect to the information processing apparatus. The invention described in claim 6 is an information processing device according to claim 5, characterized in that when a plurality of user terminals are in the predetermined positional relationship, the user terminal that is one of the plurality of user terminals is the user terminal closest to the information processing device. The invention described in claim 7 is an information processing device according to claim 5, characterized in that when a plurality of user terminals are in the predetermined positional relationship, the user terminal that is one of them is the user terminal that has been connected to the information processing device for the longest time. The invention described in claim 8 is an information processing device according to claim 5, characterized in that the user terminal, which is one of the aforementioned users, displays a notification that enables the user terminal's display to be transferred to the operation unit, and a query asking whether to perform the transfer. The invention described in claim 9 is an information processing device according to claim 5, characterized in that it displays information indicating that a user terminal located farther from the information processing device than the predetermined positional relationship needs to approach the information processing device in order to perform a handover. The invention described in claim 10 is an information processing device according to claim 1, characterized in that the predetermined condition is that a predetermined operation is performed, one of the persons is the operation unit of the information processing device, and the other of the persons is the user terminal. The invention described in claim 11 is an information processing device according to claim 10, characterized in that the predetermined operation is an operation that instructs the operation unit to transfer the display of the operation unit to the user terminal which is communicably connected to the information processing device. The invention described in claim 12 is an information processing device according to claim 1, characterized in that the display output to the operation unit and the display output to the user terminal which is communicatively connected to the information processing device have the same display mode. The invention described in claim 13 is an information processing device according to claim 12, characterized in that, among the items that the information processing device can execute, an item that is being executed by operation of either the user terminal or the operation unit is indicated on the display of the other as being unoperable. The invention described in claim 14 is a program that enables the following functions for an information processing device, which includes an operation unit on which a user operates in response to a display, and a processor that receives the operation on the operation unit and controls the display output to the operation unit in response to the received operation: when a user terminal is connected to the information processing device in a communicative manner, the program outputs the display to the user terminal and receives the operation on the user terminal, and when predetermined conditions are met, switches the output of the display to either the operation unit or the user terminal and the acceptance of the operation on either one to the other and continues control. [Effects of the Invention]

[0006] According to the invention of claim 1, compared to a configuration in which an operation performed on one device is not carried over to the other when an operation is performed on the other device before an operation on the other device is completed, the user's operational burden can be reduced. According to the invention of claim 2, the predetermined conditions are not conditions indicating that the operating unit is in use and therefore unavailable to other users, but have become available to other users. In this case, the handover from the user terminal to the operating unit can be carried out more smoothly than when one is a user terminal and the other is an operating unit of an information processing device. According to the invention of claim 3, the condition indicating that the operating unit is ready for use is the detection of a state in which there is a high probability that the operating unit is not in use. Compared to a configuration in which such a configuration is not adopted, the burden on the user using the operating unit can be reduced. According to the invention of claim 4, the condition indicating that the operating unit is ready for use is that an operation indicating the end of use of the operating unit has been performed. Compared to a configuration that does not employ this, the timing at which the user who is using the operating unit has finished using it can be reliably determined. According to the invention of claim 5, when one is a user terminal and the other is an operating unit of an information processing device, the user experience can be improved compared to a configuration in which the user terminal is not in a predetermined positional relationship with the information processing device. According to the invention of claim 6, when multiple user terminals are in a predetermined positional relationship, one of the user terminals can resume operations after the handover more quickly than in a configuration where the user terminal closest to the information processing device is not the one among the multiple user terminals. According to the invention of claim 7, when multiple user terminals are in a predetermined positional relationship, one of the user terminals is the user terminal that has been connected to the information processing device for the longest time. This configuration prevents prolonged waiting times for users. According to the invention of claim 8, compared to a configuration in which a user terminal does not employ a notification that allows the user terminal's display to be transferred to the operation unit and a query to ask whether the transfer should be performed, the user's intention regarding the transfer can be confirmed. According to the invention of claim 9, the handover can be performed more smoothly compared to a configuration in which information indicating that a user terminal located farther from the information processing device than a predetermined positional relationship needs to approach the information processing device in order to perform the handover. According to the invention of claim 10, in cases where the predetermined conditions do not involve the performance of predetermined operations, it is possible to enable other users to use the operation unit compared to a case where one is an operation unit of an information processing device and the other is a user terminal. According to the invention of claim 11, compared to a configuration in which a predetermined operation is an operation that instructs the operation unit to transfer the display of the operation unit to a user terminal that is communicably connected to the information processing device, it is possible to confirm the user's intention regarding the transfer. According to the invention of claim 12, compared to a configuration in which the display output to the operation unit and the display output to the user terminal connected to the information processing device have the same display mode, it is possible to prevent the user from feeling unfamiliar with the display on the receiving end. According to the invention of claim 13, compared to a configuration in which the information processing device does not employ a configuration in which an item being executed by either the user terminal or the operation unit is displayed as unoperable on the other, it is possible to prevent conflicts in operations between the user terminal and the operation unit. According to the invention of claim 14, compared to a configuration in which an operation performed on one device is not carried over to the other when an operation is performed on the other device before an operation on the other device is completed, the user's operational burden can be reduced. [Brief explanation of the drawing]

[0007] [Figure 1] This is a diagram illustrating an information processing system. [Figure 2] This figure shows an example of the hardware configuration of an information processing device. [Figure 3] This figure shows an example of the hardware configuration of a user terminal. [Figure 4] This is a functional block diagram of this embodiment, where (a) represents an information processing device and (b) represents a user terminal. [Figure 5] This diagram illustrates examples of screens on the control panel of an information processing device and on AR glasses. [Figure 6] This diagram illustrates an example of a screen displayed on the control panel of an information processing device. [Figure 7] This diagram illustrates various screen examples for AR glasses, with (a), (b), and (c) showing sequential screen transitions. [Figure 8] This is a sequence diagram showing the basic sequence of the information processing system according to this embodiment. [Figure 9] This is a sequence diagram showing the first sequence. (a) shows the case where user A wishes to migrate to the information processing device, and (b) shows the case where user A does not wish to migrate to the information processing device. [Figure 10] This is a diagram for explaining an example of a confirmation screen. [Figure 11] This is a sequence diagram showing the second sequence. [Figure 12] This is a diagram for further explaining the second sequence. (a) is a diagram for explaining an example of a predefined positional relationship with respect to the information processing device, and (b) is a diagram for explaining an example of the screen of the AR glasses. [Figure 13] This is a diagram for explaining an example of the screen on the operation panel of the information processing device and an example of the screen on the AR glasses. [Figure 14] This is a sequence diagram showing the third sequence. [Figure 15] This is a diagram for explaining another example of the predefined positional relationship with respect to the information processing device.

Embodiments for Carrying Out the Invention

[0008] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. <System Configuration> FIG. 1 is a diagram for explaining an information processing system 1. The information processing system 1 includes an information processing device 10 that performs information processing and a user terminal 20 that is operated by a user. The user terminal 20 can be connected to the information processing device 10 via a network 30.

[0009] A user of the information processing system 1 can cause the information processing device 10 to execute information processing from the user terminal 20. The information processing here refers to causing the information processing device 10 to exert its functions, and involves display control to the display unit provided in the information processing device 10 and the display unit provided in the user terminal 20.

[0010] If the information processing device 10 is a printing device, it has at least the function of printing documents onto paper. In this sense, the information processing device 10 is also called an image forming device. The information processing device 10 may be a device dedicated to printing, but it may also have other functions. Other functions include, for example, duplication printing (so-called copy printing), original document reading (scanning), and faxing (facsimile). The information processing device 10 can be called an image forming device. The location where the information processing device 10 will be installed is a predetermined location.

[0011] The user terminal 20 shown in Figure 1 is, for example, a desktop computer, a notebook computer, a tablet computer, a smartphone, or a mobile device. The user terminal 20 may also be a so-called wearable device. Wearable devices include watch-type, ring-type, and glasses-type devices. Examples of glasses-type wearable devices include smart glasses and AR (=Augmented Reality) glasses. In Figure 1, the user terminal 20 operated by user A is referred to as "user terminal 20A," the user terminal 20 operated by user B is referred to as "user terminal 20B," and the user terminal 20 operated by user C is referred to as "user terminal 20C." Hereafter, "user terminal 20" will be used as a general term for user terminals 20A, 20B, 20C, etc.

[0012] <Hardware configuration of the information processing device> Figure 2 shows an example of the hardware configuration of the information processing device 10. The information processing device 10 includes a processor 11, a ROM 12 storing the BIOS, firmware, etc., a RAM 13 used as the work area of ​​the processor 11, and an auxiliary storage device 14. The information processing device 10 also includes an operation panel 15 (see Figure 1), a scanner unit 16, an image processing unit 17, a printing unit 18, and a communication interface 19. The information processing device 10 may also include a facsimile device (not shown).

[0013] The processor 11 is a device that implements various functions through the execution of firmware. The processor 11, ROM 12, and RAM 13 function as a computer. The auxiliary storage device 14 is composed of, for example, a hard disk drive or semiconductor storage. The auxiliary storage device 14 stores, for example, print jobs and images of original documents. The control panel 15 is a UI (User Interface) that the user operates in response to its display, and is composed of, for example, a touch panel (see Figure 1). The control panel 15 is sometimes referred to simply as the UI. The scanner unit 16 includes, for example, a light source that illuminates the original document with illumination light, an image sensor that captures an image of the original document, and a mechanism for transporting the original document.

[0014] The image processing unit 17 is a processing unit that performs color correction, gradation correction, and other image processing on, for example, the image of a print job or original document. The printing unit 18 is a mechanism that forms an image on paper, for example, by an electrophotographic method or an inkjet method.

[0015] The communication interface 19 is an interface that enables communication with the user terminal 20 (see Figure 1). The communication interface 19 transfers the data of the document scanned by the scanner unit 16 to the user terminal 20. The communication interface 19 also receives print data from the user terminal 20. Communication may be wired or wireless. The communication interface 19 supports various communication standards. These communication standards include, for example, Ethernet (registered trademark), Wi-Fi (registered trademark), and mobile communication systems.

[0016] <User terminal hardware configuration> Figure 3 shows an example of the hardware configuration of user terminal 20. The user terminal 20 includes a control unit 21 that controls the operation of the entire device, a secondary storage unit 22 that stores various information, and a communication unit 23 that sends and receives information via the network 30 (see Figure 1). The control unit 21, the secondary storage unit 22, and the communication unit 23 are connected via buses and signal lines.

[0017] The control unit 21 includes a CPU 21a that controls the entire device as an example of a processor, a RAM 21b used as working memory for the CPU 21a, and a ROM 21c that stores programs executed by the CPU 21a. The control unit 21 also includes a rewritable non-volatile memory 21d that can retain data even if the power supply is interrupted, and an interface unit 21e that controls various parts such as the communication unit 23 connected to the control unit 21. The non-volatile memory 21d is composed of, for example, SRAM or flash memory backed up by a battery. Furthermore, each process in this embodiment is executed by the control unit 21 reading the program stored in the secondary storage unit 22. The secondary storage unit 22 is composed of, for example, a hard disk drive (HDD) or semiconductor memory. The secondary storage unit 22 varies depending on the product form of each device. The secondary storage unit 22 stores programs executed by the control unit 21 and various other information.

[0018] The user terminal 20 includes an input unit 24 such as a keyboard or mouse, and a display unit 25 consisting of an LCD display or the like. If the input unit 24 is a touch panel, it may be provided as an integral part of the display unit 25.

[0019] Here, the information processing device 10 is equipped with multiple functions, such as a printing function, and can execute multiple jobs simultaneously using each function. However, while one user is using the operation panel 15 of the information processing device 10, the operation panel 15 is occupied, and other users cannot use the operation panel 15. In other words, the number of users who can use the operation panel 15 of the information processing device 10 simultaneously is limited to the number of operation panels 15. For example, if the information processing device 10 has one operation panel 15, only one user can issue job execution instructions, etc., by operating the operation panel 15. Therefore, when the information processing device 10 is shared by multiple users, waiting times will occur for users other than the one operating the control panel 15. This may reduce the work efficiency of the other users.

[0020] On the other hand, the information processing device 10 can receive job execution instructions and the like from user terminals 20 connected to the information processing device 10. This allows more users than the number of operation panels 15 installed on the information processing device 10 to issue job execution instructions and the like to the information processing device 10.

[0021] Users can issue job execution instructions to the information processing device 10 from either the operation panel 15 or the user terminal 20. Therefore, various situations can be envisioned. The first situation is when another user is using the operation panel 15 of the information processing device 10, and the user is issuing execution commands to the information processing device 10 from the user terminal 20. In this case, the user may wish to switch to the operation panel 15 and continue operations after the other user has finished using it. The second situation is when a user operating on the operation panel 15 wishes to switch to the user terminal 20 and continue operations in the middle of an operation. Such a migration reduces the user's operational burden by ensuring that the operations performed after the migration do not overlap with those performed before the migration.

[0022] Incidentally, in recent years, devices such as AR (Augmented Reality) and MR (Mixed Reality) have become available, and technologies that display UI screens in an AR space different from the real world are known. However, when a user terminal 20 gives job execution instructions to the information processing device 10, if the user terminal 20 is a smartphone or AR glasses as described above, inconvenient situations can be anticipated. For example, if the user is holding a smartphone or AR glasses controller in one or both hands, their hands will be occupied. This could lead to situations such as being unable to set a document in the information processing device 10 or being unable to operate the paper tray of the information processing device 10. Furthermore, wearing an AR device can narrow the field of view, potentially negatively impacting the aforementioned operability. Therefore, the first situation described above is expected.

[0023] In view of the above, this embodiment has the following configuration. Figure 4 is a functional block diagram in this embodiment, where (a) shows the information processing device 10 and (b) shows the user terminal 20. As shown in Figure (a), the information processing device 10 comprises a UI control unit 10a, a communication control unit 10b, and a job control unit 10c. Furthermore, the information processing device 10 is equipped with a human presence sensor 10d (see Figure 1). The human presence sensor 10d is a detection unit that detects whether or not there is a user operating the operation panel 15 or the like in front of the information processing device 10.

[0024] The UI control unit 10a of the information processing device 10 creates operation screens for displaying job status, various job parameters, and job instructions, and displays them on the operation panel 15. The UI control unit 10a also sets job parameters and starts jobs based on operations input from the operation panel 15. Furthermore, the UI control unit 10a manages the user's operation status using a human presence sensor 10d and an operation timer (not shown), etc.

[0025] Furthermore, the UI control unit 10a creates an operation screen to be displayed to the user terminal 20. In addition, the UI control unit 10a receives operations on the operation panel 15 and also receives operations on the user terminal 20.

[0026] The communication control unit 10b controls communication with the user terminal 20. The job control unit 10c issues control instructions to various devices based on the job instructions received from the UI control unit 10a. The job control unit 10c also notifies the UI control unit 10a of the current job status.

[0027] As shown in Figure 4(b), the user terminal 20 comprises a UI control unit 26a and a communication control unit 26b. The UI control unit 26a of the user terminal 20 displays display instructions from the information processing device 10 on the user terminal 20. It receives input instructions from the operation controller attached to the user terminal 20 and executes operation instructions on the displayed screen to the information processing device 10. The communication control unit 26b controls communication with the information processing device 10.

[0028] In this embodiment, each process is executed on any computer. Furthermore, any computer may execute these processes using a processor as hardware, a program as software, or a combination thereof. In that case, the processor is configured to work in cooperation with the program to execute the various processes in this embodiment, and can function as a unit or means in this embodiment. Also, the execution order of the processes by the processor is not limited to the order described and may be changed as appropriate. Any computer may be a general-purpose computer, a computer designed for a specific purpose, a workstation, or any other system capable of executing each process.

[0029] A processor may consist of one or more hardware components, and the type of hardware is not limited. For example, a processor may consist of a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a programmable logic device such as an FPGA (Field Programmable Gate Array), a dedicated circuit for executing a specific process such as an ASIC (Application Specific Integrated Circuit), a GPU (Graphic Processing Unit), or an NPU (Neural Processing Unit). Furthermore, the type of hardware may be a combination of different types of hardware. When multiple hardware components are configured to execute one or more processes of a given processor, these components may reside in physically separate devices or in the same device. Also, in any embodiment, the order of each process performed by the processor is not limited to the order described above and may be changed as appropriate. Hardware is composed of electrical circuits (circuitry) that combine circuit elements such as semiconductor elements.

[0030] Furthermore, the program may be firmware or software such as microcode. Alternatively, the program may be, for example, a set of program modules, each function of which may be implemented by a processor configured to perform its respective function. The program may be program code or multiple code segments stored on one or more non-temporary computer-readable media (e.g., storage media or other storage). The program may be divided and stored on multiple non-temporary computer-readable media located on physically separate devices. Program code or code segments may represent any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Program code or code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0031] <First Embodiment> Next, the first embodiment will be described. First, the relationship between the operation panel 15 of the information processing device 10 and the AR glasses 51 in the first embodiment will be explained. Note that the AR glasses 51 are not part of the information processing device 10. Figure 5 illustrates example screens 41 on the operation panel 15 of the information processing device 10 and example screens 61 on the AR glasses 51. The AR glasses 51 and controller 52 shown in the figure constitute the user terminal 20. The AR glasses 51 correspond to the display unit, and the controller 52 corresponds to the input unit.

[0032] As shown in Figure 5, the operation panel 15 is operated by user Z, and the AR glasses 51 are used by user A. The operation panel 15 is an example of an operation unit, and the AR glasses 51 and controller 52 are examples of user terminals.

[0033] As shown in Figure 5, the operation panel 15 of the information processing device 10 displays screen example 41. Additionally, the AR glasses 51 display screen example 61. Screen examples 41 and 61 are examples of screens showing the same content. To further explain, the example screen 41 of the control panel 15 and the example screen 61 of the AR glasses 51 have the same design and screen specifications. The display manner is standardized between the example screen 41 of the control panel 15 and the example screen 61 of the AR glasses 51. For example, the overall layout and font size are the same. No layout issues or font differences occur between the two. This reduces the feeling of unfamiliarity when switching from either the control panel 15 or the AR glasses 51 to the other. In the example shown in Figure 5, the size of the screen example 61 of the AR glasses 51 is different from that of the control panel 15, but when user A wears the AR glasses 51, it appears to be about the same size as the control panel 15.

[0034] The example screen 41 of the operation panel 15 shown in Figure 5 is created by the UI control unit 10a of the information processing device 10 (see Figure 4(a)). The example screen 61 of the AR glasses 51 is created by the UI control unit 10a (see Figure 4(a)) of the information processing device 10 and its display is controlled by the UI control unit 26a (see Figure 4(b)). Thus, the example screen 41 of the operation panel 15 and the example screen 61 of the AR glasses 51 are created by the UI control unit 10a of the information processing device 10. The AR glasses 51, which serve as a user terminal 20 as shown in Figure 5, are connected to the information processing device 10 in a communication-enabled manner. The information processing device 10 can be connected to as many user terminals 20 as there are users who wish to connect.

[0035] To explain further, user A wears AR glasses 51, which serve as the user terminal 20, and connects to the information processing device 10. This connection is made, for example, using an AR marker 10e (see Figure 1) provided on the outer surface of the information processing device 10.

[0036] The AR marker 10e is recognized by the user terminal 20. The AR marker 10e is positioned vertically, but is not limited to this arrangement; it may also be positioned horizontally. In this embodiment, two AR markers 10e are provided, but any other number may be used. For example, if the user terminal 20 reads either of the two AR markers 10e positioned on the left and right, the information processing device 10 can recognize the position of user A relative to the information processing device 10. It is conceivable that the result of such recognition could be reflected in the display of the AR glasses 51.

[0037] When the user terminal 20 reads the AR marker 10e, communication with the information processing device 10 begins. For this communication, one possibility is to use BLE (Bluetooth Low Energy) communication. BLE communication, as used here, is a type of Bluetooth® and is a low-power wireless communication method. BLE communication enables communication and location detection within a range of up to approximately 50 meters. The information processing device 10 can estimate the distance to the connected user terminal 20 based on the strength of the radio waves transmitted via BLE communication.

[0038] The example screen 61 of the AR glasses 51 displays a virtual AR space superimposed on the real world. More specifically, the example screen 61 displays the same content as when operating the operation panel 15 of the information processing device 10. In this embodiment, by displaying the example screen 61 of the AR glasses 51 in the AR space, as described above, the example screen 41 of the operation panel 15 and the example screen 61 have the same design, thus ensuring identity. In addition to displaying Screen Example 61 in AR space, other technologies can also be considered. For example, although it would be difficult to ensure screen consistency, it is also possible to display Screen Example 61 using general browser technology.

[0039] <Examples of screens on the control panel 15 and AR glasses 51> Figure 6 illustrates an example screen 42 displayed on the operation panel 15 of the information processing device 10, and Figure 7 illustrates various example screens of the AR glasses 51. Figures 7(a), (b), and (c) show sequentially transitioning example screens 62, 63, and 64. It is assumed that the operation panel 15 is operated by user Z (see Figure 5), and the AR glasses 51 are being used by user A (see the same figure).

[0040] Screen example 42 shown in Figure 6 is displayed when a copy job is running. Screen example 62 shown in Figure 7(a) is an example of the initial screen that is displayed first. Screen example 62 is displayed in AR space.

[0041] To explain further, as shown in Figure 6, screen example 42 shows the state in which a copy job is being performed in accordance with the instructions on the operation panel 15. Therefore, in screen example 62 of the AR glasses 51 shown in Figure 7(a), exclusive control is in place so that the copy button 62a cannot be pressed. In Figure 7(a), the copy button 62a is shaded to visualize the exclusive control.

[0042] In the AR glasses 51 shown in Figure 7(a), the nine job selection buttons, including the copy button 62a in the example screen 62, are items that the information processing device 10 can normally execute. However, the shaded copy button 62a in Figure 7(a) is an item that is currently being executed by the information processing device 10. The copy button 62a in Figure 7(a) is an example of a display indicating that it cannot be operated on the AR glasses 51 side.

[0043] To further explain, as shown in Figure 7(a), in addition to indicating that each job selection button is unavailable, it is also possible to disable related items as well. In the former case, for example, when a copy job is running, the copy job is unavailable, but the scan transmission job is not. In the latter case, for example, when a copy job is running, both the copy job and the scan transmission job are unavailable.

[0044] In the example screen 62 of the AR glasses 51 shown in Figure 7(a), assume that a button other than the copy button 62a, such as the security print button 62b, is selected. This will transition to example screen 63.

[0045] In the example screen 63 shown in Figure 7(b), the login name of user A, "002," is displayed, and below it, multiple print documents are listed. When area 63a is tapped from among the multiple print documents and the document "bbb" is selected, the screen transitions to the example screen 64 shown in Figure 7(c).

[0046] In the example screen 64 shown in Figure 7(c), a preview image 64a of the document "bbb" is displayed. In this way, the AR glasses 51 can execute jobs or processes other than the copy job being executed on the operation panel 15.

[0047] <Example of a basic sequence> Figure 8 is a sequence diagram showing the basic sequence of the information processing system according to this embodiment. The AR glasses 51 and controller 52 (see Figure 5), which serve as the user terminal 20, will be referred to as the "AR device 53" below. In the example shown in Figure 8, it is assumed that user Z (see Figure 5) operates the operation panel 15 (see Figure 5) on the information processing device 10 to execute a copy job. Therefore, the information processing device 10 displays a copy screen, which is a screen for copying, on the operation panel 15 (see Figure 5) (step 101). The copy job referred to here is executed by the job control unit 10c (see Figure 4(a)).

[0048] When user A (see Figure 5) operates the AR device 53 to initiate a connection to the information processing device 10 (step 102), the information processing device 10 issues a connection confirmation notification (step 103). This notification can be the transmission of a predetermined signal.

[0049] Next, when the AR device 53 requests the information processing device 10 to display a screen (step 104), the information processing device 10 transmits screen display data (step 105). This screen display data is the data for the initial screen and is created by the UI control unit 10a of the information processing device 10 (see Figure 4). As a result, the initial screen is displayed on the AR glasses 51 (see Figure 5) of the AR device 53.

[0050] Subsequently, when the controller 52 (see Figure 5) of the AR device 53 is operated by user A (see the same figure), the UI control unit 26a (see Figure 4) of the AR device 53 identifies the content of the operation. The identified content of the operation is an instruction to execute a job other than copy job execution, such as sending an email. In the AR device 53, the communication control unit 26b (see Figure 4) transmits an operation instruction corresponding to the identified operation to the information processing device 10 (step 106).

[0051] The information processing device 10 transmits screen display data corresponding to the received operation instruction (step 107). The screen display data transmitted in step 107 is created by the UI control unit 10a of the information processing device 10 (see Figure 4).

[0052] <An example of the first sequence> Figure 9 is a sequence diagram showing a first sequence of operations of the information processing system when user Z (see Figure 5) stops operating the information processing device 10. Figure 9(a) shows the case where user A (see Figure 5) wishes to switch to the information processing device 10, and (b) shows the case where user A does not wish to switch to the information processing device 10.

[0053] The example shown in Figure 9(a) illustrates the case where User A (see Figure 5) wishes to migrate to the information processing device 10. As shown in Figure (a), the information processing device 10 detects that no operation has been performed by user Z (see Figure 5), indicating an inoperable state (step 201).

[0054] Detection of an inactive state can be achieved through absence detection using the human presence sensor 10a (see Figure 4(a)). Alternatively, detection of an inactive state can be achieved by detecting that the operation panel 15 (see Figure 5) has not been operated for a predetermined period of time. Furthermore, detection of an inactive state can occur when user Z (see the same figure) logs out, or when a user instructs the system to operate an unshown exit button after logging out. In addition, detection can occur when a user instructs the system to operate the operation handover button (button 57a ​​in Figure 13), which will be described later. Finally, detection of an inactive state can also be achieved by combining several of the above examples.

[0055] The detection of inactivity as referred to herein includes detections related to the operation panel 15, such as the absence detection and the detection of the elapsed period of inactivity as described above, as well as detections related to user actions, such as when the user logs out or when instructed to do so.

[0056] The information regarding the control panel 15 and the user's operations is an example of predetermined conditions, and also an example of conditions indicating that the system is ready for use by other users. Furthermore, the aforementioned absence detection and the detection of the elapsed period of inactivity are examples of detecting a state in which there is a high probability that the operation panel 15 of the information processing device 10 is not in use. In addition, logging out or user instruction are examples of operations that indicate the end of use of the operation panel 15 of the information processing device 10. The operation transfer button is one example of a predetermined operation.

[0057] Other users, such as user A (see Figure 5), are waiting for user Z (see Figure 5) to finish operating the control panel 15. Therefore, control is performed to notify user A that the control panel 15 is now free. In other words, the information processing device 10 responds to the detection of an inactive state (step 201) and notifies the AR device 53 being used by user A (see Figure 5) that it is in an inactive state (step 202). Such notification can be carried out by transmitting a signal that indicates the usage status.

[0058] Upon receiving notification of usage status, the AR device 53 displays a confirmation screen on the control panel 15 to ask whether to continue the operation (step 203). That is, the confirmation screen is displayed on the AR glasses 51 of the AR device 53 (see Figure 5). This allows user A (see Figure 5) to choose whether to continue the operation using the AR device 53 or the control panel 15. The confirmation screen will be described later (see Figure 10).

[0059] For example, the confirmation screen of the AR glasses 51 (see Figure 5) displays options such as "Yes" and "No". If User A (see diagram) wishes to continue operating on the control panel 15, select "Yes" on the controller 52 (see diagram). If User A (see diagram) wishes to continue operating on the AR device 53, select "No" on the controller 52 (see diagram).

[0060] In response to the confirmation screen on the control panel 15 asking whether to continue the operation, user A (see Figure 5) inputs to the controller 52 (see the same figure) of the AR device 53 that they want to continue the operation by transferring it to the control panel 15. In other words, this is the case where "Yes" is selected in the example described above. Therefore, the AR device 53 makes a transition request to the information processing device 10 (step 204).

[0061] When the information processing device 10 receives a transition request, it performs a switching process. This switching process is a process to continue control by switching the display output to user A (see Figure 5) and the acceptance of operations from user A from the AR device 53 to the information processing device 10. In other words, the information processing device 10 acquires the currently displayed screen information, which is the information of the screen being displayed by the AR glasses 51 (see Figure 5) of the AR device 53 (step 205). The information processing device 10 displays the display screen based on the acquired display screen information on the operation panel 15 (step 206). The display screen data displayed by the AR device 53 is created by the UI control unit 10a of the information processing device 10 (see Figure 4(a)) as described above and is therefore available for acquisition.

[0062] Then, the information processing device 10 notifies the AR device 53 that the switching process is complete (step 207). The AR device 53 clears the confirmation screen on the AR glasses 51 (see Figure 5) (step 208). In this way, user A (see the same figure) can move to the location where the information processing device 10 is installed and continue to perform subsequent operations using the operation panel 15 (see the same figure) of the information processing device 10.

[0063] The example shown in Figure 9(b) illustrates a case where User A (see Figure 5) wishes not to transfer the operation from the AR device 53 to the information processing device 10. That is, User A (see Figure 5) has entered into the controller 52 of the AR device 53 (see the same figure) that they wish to continue the operation on the AR device 53 without transferring to the information processing device 10 on the confirmation screen of the information processing device 10 (see step 203 above). In other words, this is the case where "No" is selected in the example above. Therefore, the AR device 53 makes a maintenance request to the information processing device 10 (step 211).

[0064] In response to this, the information processing device 10 notifies user A (see Figure 5) that the AR device 53 will continue to output the display and accept operations from user A (step 212). Then, the AR device 53 clears the confirmation screen on the AR glasses 51 (see Figure 5) (step 213). This allows user A (see Figure 5) to continue operating the AR device 53.

[0065] Figure 10 is a diagram illustrating Screen Example 65, which shows an example of a confirmation screen (see step 203 in Figure 9). Screen Example 65 is displayed on the AR glasses 51. As shown in Figure 10, Screen Example 65 displays the notification "The main unit panel is available," based on the detection of the aforementioned inactive state. Such a notification is an example of a notification that allows the user terminal display to be transferred to the control unit. Here, "main unit panel" refers to the control panel 15 (see Figure 5). Furthermore, Screen Example 65 asks, "Do you want to transfer the currently operating screen to the main unit panel?" The inquiry part of Screen Example 65 is an example of an inquiry asking whether or not to transfer the screen. Two options are presented: "Yes" to proceed if you want to switch to the information processing device 10 and continue the operation, and "No" to proceed if you do not want to continue.

[0066] <An example of the second sequence> Figure 11 is a sequence diagram showing a second sequence of operations of the information processing system when user Z (see Figure 5) ceases to operate the information processing device 10. Figure 11 corresponds to Figure 9, which shows an example of the first sequence described above.

[0067] Here, Figure 11 is the same as in Figure 9 in that it shows the sequence when User Z (see Figure 5) is operating the operation panel 15 of the information processing device 10, while User A (see the same figure) is performing an operation on the information processing device 10 from the AR device 53. However, Figure 11 differs from Figure 9 in that it performs control according to the positional relationship between the AR device 53 used by user A and the information processing device 10.

[0068] Let's explain in more detail. Consider the case where multiple AR devices 53 are connected to the information processing device 10. The multiple AR devices 53 are at various distances from the information processing device 10. Therefore, if user Z (see Figure 5) stops using the operation panel 15 of the information processing device 10, it becomes difficult to ensure smooth sharing of the operation panel 15 when someone located far from the information processing device 10 tries to use it next. Therefore, in this embodiment, a control system is employed that determines the next person to use the operation panel 15, taking into account its position relative to the information processing device 10. The second sequence will be explained below.

[0069] Although user Z (see Figure 5) was using the information processing device 10, the information processing device 10 detected that user Z had finished using it and was in an inactive state (step 301).

[0070] Next, the information processing device 10 determines whether user A's (see Figure 5) AR device 53 is in a predetermined positional relationship with the information processing device 10. Here, we assume that the determination result is that it is not in a predetermined positional relationship (step 302). If User A's (see Figure 5) AR device 53 is in a predetermined positional relationship with the information processing device 10, the process from step 202 onwards in Figure 9 described above will be performed, so the explanation will be omitted.

[0071] The information processing device 10, in response to the detection of a non-operational state (step 201) and the determination of the positional relationship, notifies the AR device 53 being used by user A (see Figure 5) (step 303). In other words, it notifies that the device is in a non-operational state and is a short distance away from the information processing device 10.

[0072] Upon receiving notification from the information processing device 10, the AR device 53 displays a notification screen indicating that it is located a short distance away from the information processing device 10 (step 304). When user A performs an action on an image such as "OK" on the notification screen, the AR device 53 sends a signal to the information processing device 10 to that effect (step 305). In response, the information processing device 10 instructs the AR device 53 to clear the notification screen (step 306). The AR device 53 clears the notification screen (step 307).

[0073] If the AR device 53 is in a predetermined positional relationship with the information processing device 10, the information processing device 10 performs the processing from step 202 onwards in Figure 9 described above. Therefore, the explanation is omitted.

[0074] The second sequence shown in Figure 11 describes the case where user A (see Figure 5) is in the closest position to the information processing device 10, but is not limited to this case. For example, the information processing device 10 identifies users in predetermined positional relationships with the information processing device 10 and recognizes the waiting order of multiple users. The information processing device 10 then notifies the next user to use the operation panel 15 according to the order (step 303).

[0075] Figure 12 further illustrates the second sequence, where (a) is a diagram illustrating an example of a predetermined positional relationship with respect to the information processing device 10, and (b) is a diagram illustrating an example screen 66 of the AR glasses 51. As shown in Figure 12(a), in the second sequence, the positional relationship with respect to the information processing device 10 is divided into three parts. Specifically, this positional relationship is divided into a first region 71 which includes the installation position of the information processing device 10, a second region 72 which surrounds the first region 71, and a third region 73 which is excluding the first region 71 and the second region 72.

[0076] For user 71a in the first area 71, when it is their turn, a "confirmation screen for handing over operation" is displayed on their AR glasses 51. This display corresponds to the display of the positional relationship notification screen in step 304 described above. Specifically, it is the screen example 65 (see Figure 10) described above.

[0077] For user 72a in the second area 72, when it is their turn, a message is displayed on their AR glasses 51 indicating that they are a short distance away from the information processing device 10. This indicates that user 72a needs to move closer to the information processing device 10. When user 72a approaches the information processing device 10 and moves to the first area 71, the above-mentioned notification screen for user 71a is displayed on the AR glasses 51.

[0078] As an example of the notification screen display, the example screen 66 shown in Figure 12(b) can be cited. The example screen 66 includes a notification that "The main unit panel is available" and advice that "You are far away from the main unit panel. If you want to use the main unit panel, move closer to it."

[0079] For user 73a in the third area 73, no indication of positional relationships is displayed on their AR glasses 51. User 73a can use their AR glasses 51 to instruct the information processing device 10 to execute jobs, etc.

[0080] Furthermore, depending on the locations of users 71a and 72a who are waiting in line, it is possible to either not distinguish the connection order (prioritizing users closer to the information processing device 10) or to distinguish the connection order (prioritizing users with longer waiting times). The connection order referred to here is the order in which each user connected their AR glasses 51 to the information processing device 10. More specifically, one possible control method is to display a confirmation screen on each user's AR glasses 51 in the order they connected to the information processing device 10, without distinguishing between users 71a and 72a who are waiting in line, regardless of whether they are in the first area 71 or the second area 72. Another possible control method is to prioritize user 71a, who is closer to the information processing device 10, even if user 72a connects to the information processing device 10 before user 71a.

[0081] Naturally, in either case, if there are multiple users in the first area 71, the connection order will be determined among those users. Similarly, if there are multiple users in the second area 72, the connection order will be determined among those users. Furthermore, in cases where multiple users are connected to the information processing device 10 and waiting in line, a modified version is also conceivable in which priority is assigned based on the user's attribute information, regardless of their location.

[0082] <Second Embodiment> Next, a second embodiment will be described. The relationship between the operation panel 15 of the information processing device 10 and the AR glasses 51 in the second embodiment will be explained. Figure 13 illustrates an example screen 41 on the operation panel 15 of the information processing device 10 and an example screen 67 on the AR glasses 57. Figure 13 corresponds to Figure 5 in the first embodiment. As shown in Figure 13, the operation panel 15 of the information processing device 10 is operated by user A, and user A is wearing AR glasses 51. User B is waiting near the information processing device 10 for user A to finish operating the operation panel 15. User B is not wearing AR glasses 51.

[0083] The operation panel 15 shown in Figure 13 displays a screen example 57. According to screen example 57, user A is causing the information processing device 10 to execute a scan job by operating the operation panel 15.

[0084] Here, User A is attempting to switch the means of operation for the information processing device 10 from the operation panel 15 to their own AR glasses 51. User A recognizes that User B is waiting in line and is attempting to switch to their own AR glasses 51. In other words, User A is attempting to yield to User B so that User B can operate the operation panel 15 by performing the subsequent operations with their own AR glasses 51.

[0085] Therefore, screen example 57 displays a button 57a ​​that instructs the user to "hand over operation." In addition, user A's AR glasses 51 display screen example 67, which has the same design as screen example 57. Screen example 67 is an example of a screen during job execution. Note that while button 57a ​​from screen example 57 is not displayed in screen example 67, a modified version in which the button is also displayed in screen example 67 is also conceivable. An operation on the button 57a ​​of the control panel 15 is an example of an operation that instructs the control panel 15 to transfer the display of the control panel 15 to the AR glasses 51, which are connected to the information processing device 10 in a communicative manner.

[0086] <An example of the third sequence> Figure 14 is a sequence diagram showing a third sequence, which is an example of the operation of the information processing system in the second embodiment. In the example shown in Figure 14, the information processing device 10 executes a job by operating the operation panel 15 (see Figure 13) (step 401). Therefore, the operation panel 15 displays a job execution screen (step 402).

[0087] The job being executed was initiated by user A (see Figure 13) operating the control panel 15 (see the same figure). User A operates their AR device 53 to initiate a connection to the information processing device 10 (step 403). The information processing device 10 responds by notifying User A of the connection confirmation (step 404).

[0088] When the information processing device 10 recognizes that user A is operating the operation panel 15 and has started connecting the AR device 53, it displays an operation handover button on the operation panel 15 (step 405). More specifically, while operating the control panel 15 of the information processing device 10, user A began connecting their AR device 53 to the information processing device 10. This indicates that user A had finished operating on the information processing device 10 and was attempting to perform subsequent operations using the AR device 53. Therefore, the information processing device 10 confirms whether user A intends to transfer operations on the operation panel 15 to its own AR device 53 by displaying an operation transfer button. Furthermore, it is conceivable that the operation handover button be displayed on the AR glasses 51 of the AR device 53 (see Figure 5), either in place of the operation panel 15 or together with the operation panel 15.

[0089] When an operation is performed on the operation handover button on the operation panel 15 (step 406), the information processing device 10 transmits the data of the job execution screen displayed on the operation panel 15 to the AR device 53 (step 407). As a result, the job execution screen is displayed on the AR glasses 51 (see Figure 5) of the AR device 53 (step 408).

[0090] Next, the information processing device 10 displays an initial screen on the operation panel 15 (step 409) and transitions to an inactive state where no user operations are being performed (step 410). Furthermore, when the information processing device 10 receives an instruction from the AR device 53, it executes the corresponding processing.

[0091] Here, an example of a predetermined positional relationship with respect to the information processing device 10 has been explained using Figure 12(a), but other examples are also possible. Figure 15 illustrates another example of a predetermined positional relationship with respect to the information processing device 10, and is another example to the example in Figure 12(a) described above. As shown in Figure 15, in another example of a positional relationship, the positional relationship with respect to the information processing device 10 is divided into four parts. Specifically, this positional relationship is divided into a first area 81 which includes the installation location of the information processing device 10, a second area 82 which surrounds the first area 81, a third area 83 which further surrounds the second area 82, and a fourth area 84 which is excluding the first area 81, the second area 82, and the third area 83. In this way, the information processing device 10 monitors the distance to the AR device 53 (see, for example, Figure 8) in four stages.

[0092] For user 81a in area 1 81, the operation will be handed over. For user 82a in area 2 82, when it is their turn, a "confirmation screen asking if they want to hand over the operation" will be displayed on their AR glasses 51. For user 83a in the third area 83, when it is their turn, a message is displayed on their AR glasses 51 indicating that they are a short distance away from the information processing device 10. For user 84a in the fourth area 84, no indication of their position is displayed on their AR glasses 51.

[0093] Let me give you a specific example. One specific example is that a button 57a ​​(see Figure 13) indicating "handing over operation" is displayed on the control panel 15 operated by user A (see Figure 13). When button 57a ​​is pressed, the display switches to a list of terminals communicating with the information processing device 10. When user A selects their AR glasses 51 from the list, the above-mentioned display is shown according to their position relative to the information processing device 10. The list display could be sorted by signal strength or by SSID (Service Set Identifier) ​​name. The signal strength sorting here roughly corresponds to the positional relationship with the information processing device 10.

[0094] Let's explain another specific example. A button (see button 57a ​​in Figure 13) that instructs "hand over operation" is displayed on the AR glasses 51 operated by user A (see Figure 5). When such a button is pressed, the display switches to a list of terminals communicating with the information processing device 10. When user A selects their AR glasses 51 from the list, the above-mentioned display corresponding to their position relative to the information processing device 10 is shown.

[0095] Although embodiments of the present invention have been described above, the technical scope of the present invention is not limited to the embodiments described above. It is clear from the claims that various modifications or improvements to the above embodiments are also included in the technical scope of the present invention. This invention can also be applied to programs and program products.

[0096] <Note> (((1))) An information processing device comprising: an operation unit in which a user performs operations in response to a display; and a processor that receives such operations to the operation unit and controls the display output to the operation unit in response to the received operations, The aforementioned processor, When a user terminal is connected to the information processing device in a communication-enabled manner, the display is output to the user terminal, and the operation performed on the user terminal is accepted. When predetermined conditions are met, the system switches the display output to either the control unit or the user terminal, and the acceptance of the operation to either one, to the other, and continues control. Information processing device. (((2))) The aforementioned predetermined conditions indicate that when the operating unit is in use and therefore unavailable to other users, it becomes available for use by those other users. The other one is the user terminal, The other party is the operation unit of the information processing device, The information processing apparatus according to (((1))), characterized in that (((3))) The condition indicating that the aforementioned possible situation has occurred is the detection of a state in which it is highly likely that the operating unit is not in use. The information processing apparatus according to (((2))), characterized in that (((4))) The condition indicating that the aforementioned possible situation has been reached is that an operation indicating the end of use of the operating unit has been performed. The information processing apparatus according to (((2))), characterized in that (((5))) In a case where one of the above is the user terminal and the other is the operation unit of the information processing device, The user terminal is in a predetermined positional relationship with respect to the information processing device. The information processing apparatus according to (((1))), characterized in that (((6))) When multiple user terminals are in the predetermined positional relationship, The user terminal mentioned above is the user terminal closest to the information processing device among the plurality of user terminals. The information processing apparatus according to (((5))), characterized in that (((7))) When multiple user terminals are in the predetermined positional relationship, The user terminal that is one of the aforementioned users is the user terminal that has been connected to the information processing device for the longest time. The information processing apparatus according to (((5))), characterized in that (((8))) The user terminal, which is one of the aforementioned devices, displays a notification that allows the user terminal's display to be transferred to the operation unit, and a query asking whether to perform the transfer. An information processing device according to any one of (((5))) to (((7))) characterized by the above. (((9))) Information is displayed indicating that, in order to transfer data to a user terminal located farther from the information processing device than the predetermined positional relationship, it is necessary to approach the information processing device. An information processing device according to any one of (((5))) to (((7))) characterized by the above. (((10))) The aforementioned predetermined condition is that a predetermined operation is performed. The other of the above is the operation unit of the information processing device, The other party is the user terminal. The information processing apparatus according to (((1))), characterized in that (((11))) The aforementioned predetermined operation is an operation that instructs the operation unit to transfer the display of the operation unit to the user terminal which is communicably connected to the information processing device. The information processing apparatus according to (((10))), characterized in that (((12))) The display output to the operation unit and the display output to the user terminal connected to the information processing device in a communicative manner have the same display format. An information processing device according to any one of (((1))) to (((11))) characterized by the above. (((13))) The information processing device indicates that an item currently being executed by either the user terminal or the operation unit is unavailable on the display of the other unit. The information processing apparatus according to (((12))), characterized in that (((14))) An information processing device comprising: an operation unit in which a user performs operations in response to a display; and a processor that receives such operations to the operation unit and controls the display output to the operation unit in response to the received operations; When a user terminal is connected to the information processing device in a communication-enabled manner, the device has a function to output the display to the user terminal and to accept the operation performed on the user terminal. A function that, when predetermined conditions are met, switches the display output to either the control unit or the user terminal and the acceptance of the operation to that one to the other, thereby continuing control. A program that makes this possible.

[0097] According to the invention of (((1))), compared to a configuration in which an operation performed on one side is not carried over to the other side when an operation is performed on the other side before an operation on the other side is completed between the operation unit of the information processing device and the user terminal connected to the information processing device, it is possible to reduce the operational burden on the user. According to the invention of (((2))), the predetermined conditions are not conditions indicating that the operating unit is in use and therefore unavailable to other users, but have become available to other users. In this case, the handover from the user terminal to the operating unit can be carried out more smoothly than when one is a user terminal and the other is an operating unit of an information processing device. According to the invention of (((3))), the condition indicating that the operating unit has become usable is the detection of a state in which it is highly likely that the operating unit is not in use. Compared to a configuration that does not employ this, the burden on the user using the operating unit can be reduced. According to the invention of (((4))), the condition indicating that the operating unit is ready for use is that an operation indicating the end of use of the operating unit has been performed. Compared to a configuration that does not employ this, it is possible to reliably determine when the user of the operating unit has finished using it. According to the invention of (((5))), when one is a user terminal and the other is an operating unit of an information processing device, the user experience can be improved compared to a configuration in which the user terminal is not in a predetermined positional relationship with the information processing device. According to the invention of (((6))), when multiple user terminals are in a predetermined positional relationship, one of the user terminals can resume operations after the handover more quickly than in a configuration where the user terminal closest to the information processing device is not adopted. According to the invention of (((7))), when multiple user terminals are in a predetermined positional relationship, one of the user terminals is the user terminal that has been connected to the information processing device for the longest time. This configuration prevents prolonged waiting times for users. According to the invention of (((8))), one user terminal can confirm the user's intention regarding the handover, compared to a configuration that does not employ a notification that the user terminal's display can be transferred to the operation unit and a query that asks whether the handover should be performed. According to the invention of (((9))), the handover can be made smoother compared to not employing a configuration that indicates that a user terminal located farther from the information processing device than a predetermined positional relationship needs to approach the information processing device in order to perform the handover. According to the invention of (((10))), in cases where a predetermined condition is not adopted in which a predetermined operation is performed, it is possible to allow other users to use the operation unit compared to a case where one is an operation unit of an information processing device and the other is a user terminal. According to the invention of (((11))), compared to a configuration in which a predetermined operation is an operation that instructs the operation unit to transfer the display of the operation unit to a user terminal that is communicably connected to the information processing device, it is possible to confirm the user's intention regarding the transfer. According to the invention of (((12))), the display output to the operation unit and the display output to the user terminal connected to the information processing device have the same display mode, which prevents the user from feeling unfamiliar with the display on the receiving end. According to the invention of (((13))), compared to a configuration in which the information processing device does not employ a configuration in which an item being executed by either the user terminal or the operation unit is shown as unoperable on the display of the other, it is possible to prevent conflicts in operations between the user terminal and the operation unit. According to the invention of (((14))), compared to a configuration in which an operation performed on one side is not carried over to the other side when an operation is performed on the other side before an operation on the other side is completed between the operation unit of the information processing device and the user terminal connected to the information processing device, it is possible to reduce the operational burden on the user. [Explanation of Symbols]

[0098] 10...Information processing device, 11...Processor, 15...Operation panel, 20...User terminal, 41, 57, 61, 65, 66, 67...Screen examples, 51...AR glasses, 52...Controller, 53...AR device

Claims

1. An information processing device comprising: an operation unit in which a user performs operations in response to a display; and a processor that receives such operations to the operation unit and controls the display output to the operation unit in response to the received operations, The aforementioned processor, When a user terminal is connected to the information processing device in a communication-enabled manner, the display is output to the user terminal, and the operation performed on the user terminal is accepted. When predetermined conditions are met, the system switches the display output to either the control unit or the user terminal, and the acceptance of the operation to either one, to the other, and continues control. Information processing device.

2. The aforementioned predetermined conditions indicate that when the operating unit is in use and therefore unavailable to other users, it becomes available for use by those other users. The other one is the user terminal, The other party is the operation unit of the information processing device, The information processing apparatus according to feature 1.

3. The condition indicating that the aforementioned possible situation has occurred is the detection of a state in which it is highly likely that the operating unit is not in use. The information processing apparatus according to feature 2.

4. The condition indicating that the aforementioned possible situation has been reached is that an operation indicating the end of use of the operating unit has been performed. The information processing apparatus according to feature 2.

5. One of the above is the user terminal, In the case where the other party is the operation unit of the information processing device, The user terminal is in a predetermined positional relationship with respect to the information processing device. The information processing apparatus according to feature 1.

6. When multiple user terminals are in the predetermined positional relationship, The user terminal mentioned above is the user terminal closest to the information processing device among the plurality of user terminals. The information processing apparatus according to feature 5.

7. When multiple user terminals are in the predetermined positional relationship, The user terminal that is one of the aforementioned users is the user terminal that has been connected to the information processing device for the longest time. The information processing apparatus according to feature 5.

8. The user terminal, which is one of the aforementioned devices, displays a notification that allows the user terminal's display to be transferred to the operation unit, and a query asking whether to perform the transfer. The information processing apparatus according to feature 5.

9. Information is displayed indicating that, in order to transfer data to a user terminal located farther from the information processing device than the predetermined positional relationship, it is necessary to approach the information processing device. The information processing apparatus according to feature 5.

10. The aforementioned predetermined condition is that a predetermined operation is performed. The other of the above is the operation unit of the information processing device, The other party is the user terminal. The information processing apparatus according to feature 1.

11. The aforementioned predetermined operation is an operation that instructs the operation unit to transfer the display of the operation unit to the user terminal which is communicably connected to the information processing device. The information processing apparatus according to feature 10.

12. The display output to the operation unit and the display output to the user terminal connected to the information processing device in a communicative manner have the same display format. The information processing apparatus according to feature 1.

13. The information processing device, among the items it can execute, indicates on the other display that an item currently being executed by either the user terminal or the operation unit is not operable. The information processing apparatus according to feature 12.

14. An information processing device comprising: an operation unit in which a user performs operations in response to a display; and a processor that receives such operations to the operation unit and controls the display output to the operation unit in response to the received operations; When a user terminal is connected to the information processing device in a communication-enabled manner, the device has a function to output the display to the user terminal and to accept the operation performed on the user terminal. A function that, when predetermined conditions are met, switches the display output to either the control unit or the user terminal and the acceptance of the operation to that one to the other, thereby continuing control. A program that makes this possible.

Images