Information processing device, information processing system, and program

Abstract

To increase the operability of an input operation on an object that has been made visible by a display device more than when an input operation is performed by a gesture.SOLUTION: An HMD 30 as an information processor has a controller 31. In the controller 31, a display controller 101 performs a control to make an object and an input device which allow reception of an input operation of information visible by a display device. A contact detection unit 107 detects a contact of the object and the input device on the basis of information on the positions of the object and the input device made visible by the display device. Then, an input information reception unit 108 receives information input in the input device as information input in the object.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing system, and a program. [Background technology]

[0002] For example, AR (Augmented Reality) and VR (Virtual Reality) technologies that allow objects displayed on a display device such as an HMD (Head Mounted Display) to be operated by gestures have the advantage of allowing users to view 3D models with intuitive operations. For example, Patent Document 1 discloses a technology that allows a user wearing an HMD to perform input operations such as tapping and pinching by placing a smartphone over one or more virtual objects using gestures. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6346585 Summary of the Invention [Problem to be solved by the invention]

[0004] However, many users find that performing input operations on objects displayed on an HMD using gestures is more difficult than inputting data into general-purpose devices such as personal computers, smartphones, and tablets.

[0005] An object of the present invention is to improve the operability of input operations for objects that are visible via a display device compared to input operations using gestures. [Means for solving the problem]

[0006] The invention described in claim 1 includes a processor, wherein the processor controls an object and an input device that can accept an information input operation to be visible via a display device, and detects contact between the object and the input device based on information relating to the respective positions of the object and the input device that have become visible via the display device; If the detected contact is an operation of moving the object and contacting the input device by a user gesture, the object and the input device are linked as being based on an intentional operation by the user, and the The information entered into the input device The relevant The information processing device is characterized in that it accepts the information as input to the object. The invention described in claim 2 is the information processing device described in claim 1, characterized in that the processor further detects the state of the object that has come into contact with the input device, and determines whether to link the object with the input device depending on the state of the object. The invention described in claim 3 is the information processing device described in claim 2, characterized in that if the state of the object is such that writing is prohibited or input is not possible, the processor repels the object without coordination when the object contacts the input device. Claim 4 The invention described in claim 1 is characterized in that the processor performs control to make the input device present in real space visible via the display device. or any of the three The information processing device is described in Claim 5 The invention described in is characterized in that the processor is configured to select one of the input devices present in the real space, The aforementioned The present invention is characterized in that it performs control to identify an object to be input by the user and make it visible via the display device. 4 The information processing device is described in Claim 6 The invention described in claim 1 is characterized in that the processor identifies the input device that is the target of the user's input operation based on the behavior of the input device estimated from one or more of data provided from the input device, image data of the input device, and sensing data targeting the input device, and performs control to make the input device visible via the display device. 5 The information processing device is described in Claim 7 The invention described in claim 1 is characterized in that the processor performs control to identify the input device that is the target of the user's input operation based on the behavior of the input device estimated from the result of matching inertial measurement data (IMU data) of the input device with three-dimensional behavior data, and make the identified input device visible via the display device. 6The information processing device is described in Claim 8 The invention described in The aforementioned 1. The display device is attached to a user, and the input device is controlled to be visible through the display device attached to the user. or any of the three The information processing device is described in Claim 9 The invention described in claim 1 is characterized in that the processor performs control to make the input device visible via a head-mounted display as the display device worn on the user's head. 8 The information processing device is described in Claim 10 The invention described in claim 1 is characterized in that the processor controls the object and the input device, which can accept the input operation of one or more types of information among text information, audio information, and image information, to be visible via the display device. or any of the three The information processing device is described in Claim 11 The invention described in claim 1 is characterized in that the processor controls the object and the input device that can accept the input operation of the one or more types of information to be visible via the display device, and also controls other objects that support the input operation to the input device to be further visible. 10 The information processing device is described in Claim 12 The invention described in claim 1 is characterized in that the processor performs control to make the other object visible in the vicinity of the input device or so as to be superimposed on at least a part of the input device. 11 The information processing device is described in Claim 13 The invention described in claim 1 is characterized in that the processor detects contact between the object and the input device based on information indicating the outer edges of the object and the input device that are visible through the display device as the information about the position. or any of the threeThe information processing device is described in Claim 14 The invention described in claim 1 is characterized in that the processor detects contact between outer edges of virtual regions surrounding the outer edge, which are specified from information indicating the outer edges of the object and the input device, as contact between the object and the input device. 13 The information processing device is described in Claim 15 The invention described in claim 1 is characterized in that the outer edge of the virtual area is specified based on a predetermined value as a distance value from the outer edge of each of the object and the input device toward the outside. 14 The information processing device is described in Claim 16 The invention described in claim 1 is characterized in that the processor accepts information input to the input device by an input function for performing the input operation, which is specified based on a combination of the type of the object and the function of the input device, as information input to the object. or any of the three The information processing device is described in Claim 17 The invention described in claim 1 is characterized in that the processor identifies the function of the input device based on one or more of data provided from the input device, image data of the input device, and sensing data of the input device. 16 The information processing device is described in Claim 18 The invention described in claim 1 is characterized in that the processor accepts information input to the input device using the input function specified by the combination stored in the database as information input to the object. 16 The information processing device is described in Claim 19The invention described in claim 1 is characterized in that the processor further detects a separation between the object and the input device that are in contact with each other based on information about the positions of the object and the input device, and when the processor detects the separation, even if information is input to the input device, the processor does not accept the information as information input to the object. or any of the three The information processing device is described in Claim 20 The invention described in claim 1 is characterized in that, when the detected separation is based on an intentional operation by a user, even if information is input to the input device, the processor does not accept the information as information input to the object. 19 The information processing device is described in Claim 21 The invention described in claim 1 is characterized in that the processor detects the contact when registration of a collaboration candidate between the object and the input device has been completed. or any of the three The information processing device is described in Claim 22 The invention described in claim 2 is characterized in that the processor detects the contact when registration of a candidate for a collaboration between the object and the predetermined input device or the input device on which predetermined application software for connection is installed has been completed. 21 The information processing device is described in Claim 23 The invention described in the above includes a display control means for controlling an object and an input device that can receive an information input operation to be visible through a display device, and a detection means for detecting contact between the object and the input device based on information relating to the respective positions of the object and the input device that have become visible through the display device; If the detected contact is an operation of moving the object and contacting the input device by a user gesture, the object and the input device are linked as being based on an intentional operation by the user, and the The information entered into the input device The relevant and a receiving unit that receives the information as input to the object. Claim 24The invention described in the above includes a function of controlling an object and an input device that can receive an input operation of information to be visible through a display device, and a function of detecting contact between the object and the input device based on information about the respective positions of the object and the input device that have become visible through the display device; If the detected contact is an operation of moving the object and contacting the input device by a user gesture, the object and the input device are linked as being based on an intentional operation by the user, and the The information entered into the input device The relevant It is a program that realizes the function of accepting information as input to the object. [Effects of the Invention]

[0007] According to the present invention of claim 1, input operations to an input device are directly linked as input operations to an object, so that the operability of input operations to an object that is visible through a display device can be improved compared to when input operations are performed using gestures. Furthermore, by treating the operation of moving an object and touching it with the input device as an intentional operation by the user, it is possible to prevent the occurrence of unintended actions due to unintended contact by the user. According to the present invention of claim 2, it is possible to determine whether or not to link the object and the input device depending on the state of the object that has come into contact with the input device. According to the present invention of claim 3, the user can intuitively know that writing is prohibited and that input to the object is not possible. Claim 4 According to the present invention, an input device existing in real space can be made visible via a display device, and an input operation on the input device can be treated as an input operation on an object. Claim 5 According to the present invention, the input device that is the target of the user's input operation in real space can be made visible via the display device, and the user's input operation on that input device can be treated as an input operation on an object. Claim 6 According to the present invention, it is possible to identify whether an input device is a target for a user's input operation in real space based on the behavior of the input device. Claim 7 According to the present invention, it is possible to estimate whether an input device is the target of a user's input operation in real space based on the results of matching the inertial measurement data (IMU data) of the input device with three-dimensional behavior data. Claim 8According to the present invention, the object and the input device can be made visible via a display device worn by the user, and the user's input operation on the input device can be treated as an input operation on the object. Claim 9 According to the present invention, the input device can be made visible through a head-mounted display worn on the user's head, and the user's input operation on the input device can be treated as an input operation on an object. Claim 10 According to the present invention, an input operation of one or more types of information selected from text information, audio information, and image information to an input device that is visible through a display device can be treated as an input operation to an object. Claim 11 According to the present invention, it is possible to improve the operability of input operations to an input device that is visible via a display device. Claim 12 According to the present invention, it is possible to improve the operability of input operations to an input device that is visible via a display device. Claim 13 According to the present invention, it is possible to detect whether or not an object is in contact with an input device from information indicating the outer edges of the object and the input device that are visible via the display device. Claim 14 According to the present invention, user operability is improved by relaxing contact determination so that contact is detected before the outer edges of the object visible through the display device and the input device come into complete contact with each other. Claim 15 According to the present invention, the degree of relaxation of the contact determination can be set arbitrarily, which further improves the operability for the user. Claim 16 According to the present invention, it is possible to perform an input operation using an input function specified by a combination of the type of object and the function of the input device. Claim 17According to the present invention, it is possible to identify the input function of an input device from any of data provided by the input device, image data captured of the input device, and sensing data targeted at the input device. Claim 18 According to the present invention, it is possible to identify an input function from a combination of the type of object stored in the database and the function of the input device, so that it is possible to accommodate a variety of combinations. Claim 19 According to the present invention, the user can release the linkage of input operations between the object and the input device by performing an operation to separate the object and the input device that are in contact with each other. Claim 20 According to the present invention, it is possible to prevent unintended actions from occurring due to the user's unintended separation. Claim 21 According to the present invention, by requiring that registration of candidate collaboration destinations between an object and an input device be completed in advance, it is possible to prevent connection to an unintended input device. Claim 22 According to the present invention, by requiring that registration of candidate connection destinations between an object and a predetermined input device or an input device on which predetermined application software for connection is installed be completed in advance, it is possible to prevent connection to an unintended input device. Claim 23 According to the present invention, it is possible to provide an information processing system that improves the operability of input operations for objects that are visible via a display device compared to input operations using gestures. Furthermore, by treating the operation of moving an object and touching it with the input device as an intentional operation by the user, it is possible to prevent the occurrence of unintended actions due to unintended contact by the user. Claim 24 According to the present invention, it is possible to provide a program that improves the operability of input operations for objects that are visible via a display device compared to input operations using gestures. Furthermore, by treating the operation of moving an object and touching it with the input device as an intentional operation by the user, it is possible to prevent the occurrence of unintended actions due to unintended contact by the user. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram showing the overall configuration of an information processing system to which an embodiment of the present invention is applied; [Figure 2] FIG. 2 is a diagram illustrating the hardware configuration of an HMD. [Figure 3] FIG. 2 is a diagram illustrating a functional configuration of a control unit of the HMD. [Figure 4] 10 is a flowchart showing the flow of a pairing process. [Figure 5] 10 is a flowchart showing the flow of processing performed by the HMD. [Figure 6] FIG. 10 is a diagram showing an example of an object and an input device that can be viewed by a user through an HMD, in which the object is an electronic document that can accept text input, and the input device is a tablet terminal. [Figure 7] FIG. 10 is a diagram showing a specific example of an object and an input device that can be viewed by a user through an HMD, in which the object is a PDF (Portable Document Format) document and the input device is a pen tablet without a display function. [Figure 8] This figure shows an example of an object and input device that can be viewed by a user through an HMD, in which the object can accept input operations for drawing with a stylus and can be shared with other users, and the input device is a tablet terminal. [Figure 9] FIG. 10 is a diagram showing an example of an object and an input device that can be viewed by a user through an HMD, in which the object is a software keyboard and the input device is a tablet terminal. [Figure 10] This figure shows an example of an object and input device that can be viewed by a user through an HMD, in which the object is a three-dimensional model that can accept input for editing using three-dimensional model design software, and the input device is a notebook personal computer. [Figure 11]FIG. 10 is a diagram showing an example of an object that can be viewed by a user through an HMD and an input device, in which the object is a speech bubble that can accept text input, and the input device is a tablet terminal. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. (Configuration of information processing system) FIG. 1 is a diagram showing the overall configuration of an information processing system 1 to which this embodiment is applied. The information processing system 1 is configured by connecting an HMD (head mounted display) 30 having an information processing function and an input device 50 via a network 90 or a communication method such as infrared communication, visible light communication, close proximity wireless communication, Bluetooth (registered trademark), RFID (registered trademark), or UWB (Ultra Wide Band). The network 90 is, for example, a LAN (Local Area Network), the Internet, or the like.

[0010] The HMD 30 performs control to display an object and control to make the input device 50 present in the real space visible to the user U. The HMD 30 also detects contact between the displayed object and the visible input device 50. If the contact between the object and the input device 50 is based on an intentional operation by the user U, the HMD 30 links the object with the input device 50, and thereafter accepts information input to the input device 50 as information input to the object.

[0011] The HMD 30 is a head-mounted information processing device having a display for displaying image information. The HMD 30 is available in a see-through type and a non-see-through type. The see-through HMD 30 displays objects on the display and makes the input device 50 present in real space visible through the display. The non-see-through HMD 30 displays the objects and the input device 50 on the display in real time.

[0012] A user U wearing a see-through HMD 30 can simultaneously view an object displayed on the HMD 30 and an input device 50 present in real space through the HMD 30. A user U wearing a non-see-through HMD 30 can simultaneously view an object displayed on the HMD 30 and the input device 50. In this embodiment, a see-through HMD 30 is employed. Examples of objects displayed on the HMD 30 include document data, a whiteboard, a virtual keyboard, a three-dimensional model, an image, a video, and an audio file.

[0013] The input device 50 is an information processing device that exists in the real space together with the user U and has a function of accepting input operations from the user U. Examples of the input device 50 include a smartphone, a personal computer (desktop type, notebook type), a tablet terminal, as well as a so-called touch sensor, a pen tablet, an electronic pen, a smart pen, and the like.

[0014] (HMD hardware configuration) FIG. 2 is a diagram showing the hardware configuration of the HMD 30. As shown in FIG. The HMD 30 has a control unit 31, a memory 32, a storage unit 33, a communication unit 34, a display unit 35, a sensor unit 36, and an imaging unit 37. These units are connected to each other via a data bus, an address bus, a PCI (Peripheral Component Interconnect) bus, etc.

[0015] The control unit 31 is a processor that controls the operation of the device itself through the execution of various software such as an OS (operating system) and application software. The control unit 31 is configured, for example, by a CPU (Central Processing Unit). The memory 32 is a storage area that stores various software and data used for executing the software, and is used as a working area for calculations. The memory 32 is configured, for example, by a RAM (Random Access Memory).

[0016] The storage unit 33 is a storage area for storing input data for various software programs and output data from various software programs, and stores a database for storing various information. The storage unit 33 is configured with, for example, an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a semiconductor memory used to store programs and various setting data. The communication unit 34 transmits and receives data via the network 90 or by a communication method such as infrared communication. The communication unit 34 transmits and receives data to and from the input device 50 and external devices.

[0017] The display unit 35 presents images to the user using a liquid crystal display or an organic EL display, and is available in two types: a non-transparent type similar to a normal display, and a transparent type that allows the user to see ahead through the display. The former is usually called a virtual reality HMD (VR-HMD), and the latter is called an augmented reality HMD (AR-HMD) or mixed reality HMD (MR-HMD).

[0018] The sensor unit 36 ​​is composed of various sensors such as an optical sensor such as a LiDAR (Laser Imaging Detection and Ranging) sensor, an acceleration sensor, etc. The imaging unit 37 is composed of a camera, etc. These units are connected by a data bus, an address bus, a PCI bus, etc.

[0019] (Hardware configuration of input device) The hardware configuration of the input device 50 varies depending on the type of input device 50. For example, if the input device 50 is a smartphone, a personal computer (desktop type, notebook type), or a tablet terminal, it will have an operation unit in addition to the hardware configuration of the HMD 30 shown in FIG. 2. The operation unit is composed of, for example, a keyboard, a mouse, mechanical buttons, and switches, and accepts input operations. The operation unit also includes a touch sensor that forms a touch panel integrally with the display unit. Also, for example, if the input device 50 does not have a screen, such as a so-called touch sensor, pen tablet, electronic pen, or smart pen, it will have the same configuration as the hardware configuration of the HMD 30 shown in FIG. 2 except for the display unit 35.

[0020] (Functional configuration of the HMD control unit) FIG. 3 is a diagram showing the functional configuration of the control unit 31 of the HMD 30. The control unit 31 of the HMD 30 functions as a display control unit 101, a data acquisition unit 102, a registration determination unit 103, a behavior estimation unit 104, a device identification unit 105, a position acquisition unit 106, a contact detection unit 107, an input information reception unit 108, and an input control unit 109.

[0021] The display control unit 101 performs control to make visible, via the device itself, objects that can accept information input operations and the input device 50. Specifically, the display control unit 101 performs control to display, on the display unit 35, objects that can accept information input operations, and performs control to make the input device 50 that exists in the real space visible via the display unit 35.

[0022] The data acquisition unit 102 acquires sensing data from the various sensors of the sensor unit 36 ​​and data on images (still images and moving images) captured by the imaging unit 37. The sensing data from the various sensors of the sensor unit 36 ​​includes sensing data on the movements of both hands and eyes of the user U. Furthermore, the image data captured by the imaging unit 37 includes data on images of the user U and data on images of objects and the input device 50 present in an area visible via the display unit 35.

[0023] The registration determination unit 103 determines the registration status of a candidate for collaboration between the HMD 30 and the input device 50. The "registration status of a candidate for collaboration" refers to a state indicating whether a combination of multiple devices is registered as candidates for collaboration with each other and is in a state of waiting for collaboration. This state is also referred to as a "paired" state. Once pairing is performed, pairing is not required the second or subsequent time a connection is made. The registration determination unit 103 determines whether the HMD 30 and the input device 50 are paired and in a state of waiting for collaboration based on information held by the HMD 30 itself and information obtained from each of the input devices 50.

[0024] Here, the pairing between the HMD 30 and the input device 50 will be described. Pairing between the HMD 30 and the input device 50 is performed, for example, by the following procedure. That is, the pairing process may be started by a voice input from the user U. In this case, for example, when the user utters "pair," the voice may be detected and a voice guide regarding pairing may be output from the HMD 30. Also, for example, the HMD 30 may output a guide such as "press the pairing start button in the application software of the device you want to pair," and application software for connection pre-installed in the input device 50 may be launched to start the pairing process.

[0025] In this case, an open connection environment may be used in which the pairing process is initiated for any input device 50 on which connection application software is installed. Alternatively, when a user U wearing the HMD 30 on his / her head views a QR code (registered trademark) displayed on the input device 50 through the HMD 30, the HMD 30 may detect the QR code (registered trademark) and automatically initiate the pairing process. Alternatively, from the standpoint of preventing pairing with an unintended input device 50, a closed connection environment may be used in which the input devices 50 that can be paired with the HMD 30 are limited and paired in advance.

[0026] The behavior estimation unit 104 estimates the behavior of each of the user U and the input device 50 based on the data acquired by the data acquisition unit 102 and the data provided from the input device 50. Among these, the behavior of the user U refers to the movements of both hands and eyes of the user U. Furthermore, the behavior of the input device 50 refers to the movement of the input device 50 in real space. The data acquired by the data acquisition unit 102 includes, for example, data indicating the pairing status with the input device 50, sensing data of various sensors of the sensor unit 36, and data of images (still images and moving images) captured by the imaging unit 37.

[0027] The data provided from the input device 50 includes data indicating the pairing status with the HMD 30, data input to the input device 50, inertial measurement data (IMU data) of the input device 50, three-dimensional behavior data of the input device 50, data related to the position of the input device 50, etc. Specifically, for example, the behavior estimation unit 104 estimates the behavior of the input device 50 from the result of matching the inertial measurement data (IMU data) of the input device 50 with the three-dimensional behavior data.

[0028] There is no particular limitation on how the three-dimensional data of the external shape of the input device 50 is acquired. For example, the data may be acquired separately based on the type of input device pre-stored in the database of the storage unit 13. Alternatively, for example, the user U wearing the HMD 30 may look at the input device 50 from all directions, and the three-dimensional data may be generated from data of a plurality of captured images obtained by the HMD 30 capturing images of the input device 50 at the same time. In this case, audio guidance on the procedure for looking at the input device 50 may be output from the HMD 30 to the user U.

[0029] The device identification unit 105 identifies the input device 50 that is the target of the user U's input operation based on the data acquired by the data acquisition unit 102, the data provided from the input device 50, the results of estimation by the behavior estimation unit 104, and a combination of the input device type and function pre-stored in the database of the storage unit 13. Identifying the input device 50 includes identifying the function of the input device 50. Examples of the data acquired by the data acquisition unit 102 include sensing data and image data. Examples of the data provided by the input device 50 include data indicating a pairing status, input data, inertial measurement data (IMU data), three-dimensional behavior data, and positional data. Specifically, for example, the device identification unit 105 identifies the input device 50 based on image data indicating an area visible to the user U through the input device 50, among the data acquired by the data acquisition unit 102.

[0030] The position acquisition unit 106 acquires information relating to the positions of each of the objects and the input device 50 in an area visible to the user U through the display unit 35. Specifically, the position acquisition unit 106 acquires information relating to the positions of each of the objects and the input device 50 in an area visible to the user U through the display unit 35 from image data indicating an area visible to the user U through the display unit 35, among the image data acquired by the data acquisition unit 102.

[0031] The contact detection unit 107 detects contact between an object and the input device 50. Specifically, the contact detection unit 107 detects contact between the object and the input device 50 in an area visible via the display unit 35, based on information about the positions of the object and the input device 50 acquired by the position acquisition unit 106. Contact between the object and the input device 50 may be detected when the outer edges of the object and the input device 50 come into contact with each other, or when the outer edges of virtual areas surrounding the outer edges of the input device 50 and the object come into contact with each other. In this case, the "virtual area surrounding the outer edge" is specified based on a predetermined value as a distance value from the outer edges of the object and the input device 50 outward.

[0032] Furthermore, the contact detection unit 107 detects whether contact between an object and the input device 50 is based on an intentional operation by the user U. When the contact detection unit 107 detects contact between an object and the input device 50 and the contact is based on an intentional operation by the user U, the object and the input device 50 are linked. For example, when a gesture by the user U wearing the HMD 30 is performed by grabbing an object displayed on the HMD 30 and touching it with the input device 50, the contact detection unit 107 detects that the contact is based on an intentional operation by the user U. Furthermore, when it is detected that the position of the user U's gaze at the time when the object and the input device 50 come into contact with each other is at or near the point of contact between the object and the input device 50, the contact detection unit 107 may detect that the contact between the object and the input device 50 is based on an intentional operation by the user U.

[0033] Furthermore, when the contact detection unit 107 detects contact between an object and the input device 50, it detects the state of the object. If the state of the object detected by the contact detection unit 107 indicates that writing is prohibited and input is not possible, or if input to the object is not possible using the input function of the paired input device 50, the object is repelled without any linkage even if it comes into contact with the input device 50. In this manner, the user U can intuitively know that writing is prohibited. Furthermore, for example, the user U may be informed visually, audibly, and tactilely that input to the object is prohibited by displaying a color or a pop-up message on the HMD 30 indicating that input to the object is prohibited, outputting a sound, or generating a vibration.

[0034] Furthermore, the contact detection unit 107 further detects separation of the object in contact with the input device based on information about the respective positions of the object and the input device. As a result, the link between the object and the input device 50 is released. The contact detection unit 107 also detects whether the separation of the object in contact with the input device 50 is based on an intentional operation by the user U. When the contact detection unit 107 detects separation of the object from the input device 50 and the contact is based on an intentional operation by the user U, the link between the object and the input device 50 is released.

[0035] For example, when a gesture of the user U wearing the HMD 30 is performed in which the user grabs an object visible through the HMD 30 and moves it away from the input device 50, the contact detection unit 107 detects that the separation of the object from the input device 50 is based on an intentional operation by the user U. Also, for example, when it is detected that the position of the user U's line of sight at the time when the object and the input device 50 separate is at or near the contact point between the object and the input device 50, the contact detection unit 107 may detect that the separation of the object and the input device 50 is based on an intentional operation by the user U.

[0036] When the contact detection unit 107 detects contact between the object and the input device 50 and the object and the input device 50 are linked, the input information receiving unit 108 receives the information input to the input device 50 as information input to the object. In other words, once the object and the input device 50 are linked, the user U can concentrate on the input operation without particularly worrying about the positional relationship between the object and the input device 50.

[0037] Specifically, the input information receiving unit 108 receives, as information input to an object, information input by an input function for performing an input operation specified by a combination of the type of object stored in the database of the storage unit 13 and the function of the input device 50. Examples of the input function for performing an input operation include a touch panel, a smart pen, etc. when the input device 50 is a smartphone or tablet terminal, and a keyboard, a mouse, etc. when the input device 50 is a personal computer.

[0038] Furthermore, when the contact detection unit 107 detects that an object in contact and linked state has separated from the input device 50, and the link between the object and the input device 50 is released, even if information is input to the input device 50, the input information receiving unit 108 does not receive this information as information input to the object. However, when the contact detection unit 107 detects that the object has separated from the input device 50, if the cause is not a gesture by the user U, it is detected that the separation is not based on an intentional operation by the user U, and the link between the object and the input device 50 continues without being released. Therefore, the information input to the input device 50 continues to be received as information input to the object.

[0039] The input control unit 109 controls input of input information accepted by the input information accepting unit 108 into an object. For example, if the object is an electronic document capable of accepting text input and the input device 50 is a tablet terminal, the input control unit 109 controls input of text information input into the tablet terminal into the electronic document. For example, if the object is a PDF document and the input device 50 is a pen tablet, the input control unit 109 controls input of handwriting data, which is information about a drawing input into the pen tablet with a stylus, into the electronic document. For example, if the object is a three-dimensional model and the input device 50 is a personal computer, the input control unit 109 controls input of information input to edit the three-dimensional model into the three-dimensional model.

[0040] (Pairing process) Fig. 4 is a diagram showing the flow of the pairing process between the HMD 30 and the input device 50. Fig. 4 shows, as an example of the pairing process, an example in which the pairing process is started by a voice input by the user.

[0041] When the HMD 30 and the input device 50 are not paired, if a voice input is made to instruct the start of pairing processing (YES in step 201), a voice guide regarding pairing is output from the HMD 30 (step 202). On the other hand, if a voice input is not made to instruct the start of pairing processing (NO in step 201), the processing of step 201 is repeated until a voice input is made to instruct the start of pairing processing.

[0042] When an input operation to start the pairing process is performed on the input device 50 in accordance with the guidance from the HMD 30 (YES in step 203), a pairing start notification is transmitted from the input device 50 to surrounding devices (step 204). On the other hand, if an input operation to start the pairing process is not performed on the input device 50 (NO in step 203), the process of step 203 is repeated until an input operation to start the pairing process is performed on the input device 50.

[0043] When the pairing start notification transmitted from the input device 50 is received by the HMD 30 within a predetermined time (YES in step 205), the HMD 30 notifies the user U of a guide regarding an input operation for finalizing the pairing (step 206). For example, the guide may include the content, "Please enter a passcode and press the pairing confirmation button." On the other hand, when the transmitted pairing start notification is not received by the HMD 30 within a predetermined time (NO in step 205), the processing ends without pairing being performed.

[0044] Here, when the user U performs the operation of the notified content (YES in step 207), a pairing confirmation notification including a passcode is transmitted from the input device 50 to the surrounding devices (step 208). On the other hand, when the user U does not perform the operation of the notified content (NO in step 207), the processing of step 207 is repeated until the user U performs the operation of the notified content.

[0045] If the transmitted pairing confirmation notification is received by the HMD 30 within a predetermined time (YES in step 209), the HMD 30 requests the input device 50 to transmit data capable of identifying the input device 50 and data relating to functions that the input device 50 can provide (step 210). Examples of data capable of identifying the input device 50 include the Wi-Fi address of the input device 50. On the other hand, if the transmitted pairing confirmation notification is not received by the HMD 30 within a predetermined time (NO in step 209), the processing ends without pairing being performed.

[0046] When the input device 50 transmits the data requested by the HMD 30 to the HMD 30 (YES in step 211), the HMD 30 receives and stores the data transmitted from the input device 50 (step 212). On the other hand, when the input device 50 has not transmitted the data requested by the HMD 30 to the HMD 30 (NO in step 211), the processing of step 211 is repeated until the input device 50 transmits the data requested by the HMD 30 to the HMD 30.

[0047] Then, the HMD 30 issues guidance to the user U for acquiring three-dimensional data on the external shape of the input device 50 (step 213). For example, audio guidance such as "Hold the device (input device 50) in your hand and look at the device from various angles" is issued. When the user U then holds the input device 50 and looks at it from all directions by turning it around or changing his / her grip while gazing at it (YES in step 214), the HMD 30 stores the recognition results of the external shape characteristics of the input device 50 as the external shape characteristics of the paired input device 50 (step 215).

[0048] Specifically, the HMD 30 estimates the three-dimensional shape of the input device 50 held by the user U from the vicinity of the point at which the user U is gazing, recognizes the external characteristics of the input device 50 from various angles, and stores the recognition results as the external characteristics of the paired input device 50. This completes pairing between the HMD 30 and the input device 50, and the HMD 30 enters a state of waiting for cooperation. On the other hand, if the user U does not hold the input device 50 and perform an action of looking at it from all directions by changing the orientation or holding it differently while gazing at it (NO in step 214), the processing of step 214 is repeated until such an action is performed.

[0049] (HMD processing) FIG. 5 is a flowchart showing the flow of the collaboration process by the HMD 30. The HMD 30 performs control to make the object and real space visible through the HMD 30 (step 401). Specifically, control is performed to display the object on the display unit 35 of the HMD 30. Note that in this embodiment, since a see-through HMD 30 is used, the input device 50 is visible through the display unit 35.

[0050] The HMD 30 recognizes objects present in the user's field of view from the image data captured by the imaging unit 37 (step 402). If, as a result of identifying the input device 50, a paired input device 50 is present in the field of view of the user U (YES in step 403), the HMD 30 enters a state of waiting for cooperation (step 404). The HMD 30 estimates the behavior of each of the user U and the input device 50 based on the data acquired by the data acquisition unit 102 and the data acquired from each of the input devices 50 (step 405). On the other hand, if no paired input device is present in the user's field of view (NO in step 403), the process returns to step 402.

[0051] The HMD 30 acquires information about the respective positions of the object and the input device 50 from image data showing the area visible to the user U through the HMD 30 (step 406). When the HMD 30 detects contact between the object and the input device 50 in the area visible through the HMD 30 based on the acquired information about the respective positions of the object and the input device 50 (YES in step 407), the HMD 30 determines an input method and a display method based on a combination of the type and function of the input device 50 pre-stored in the database of the storage unit 13 (step 408). The HMD 30 then accepts information input to the input device 50 as information input to the object (step 409) and performs control to input the accepted input information to the object (step 410). On the other hand, if contact between the object and the input device 50 has not been detected (NO in step 407), the HMD 30 repeats the process of step 407 until contact between the object and the input device 50 is detected.

[0052] (Example) 6 to 11 are diagrams showing specific examples of objects that the user U can view through the HMD 30 and the input device 50. It is assumed that the HMD 30 and the input device 50 have been paired in advance. 6A to 6C show an example in which an object V1 visible through the HMD 30 is an electronic document capable of accepting text input, and the input device 50 is a tablet terminal. FIG. 6A shows a state in which the object V1 visible through the HMD 30 and the input device 50 are separated and not linked. Here, when a gesture by the user U is made to grab the object V1 and move the object V1 to bring it into contact with the input device 50, contact between the object V1 and the input device 50 is detected.

[0053] When contact between the object V1 and the input device 50 is detected as a result of the user U's gesture, the object V1 and the input device 50 enter a state of contact, i.e., a linked state, as shown in Fig. 6(B). Then, a software keyboard is displayed so as to be superimposed on the input device 50, which is visible through the HMD 30. This allows the user U to perform input operations using the displayed software keyboard.

[0054] When the user U performs an input operation on the input device 50 present in real space while looking at the software keyboard displayed on the HMD 30, the input operation is linked to an input operation on the software keyboard. As a result, the information input to the input device 50 is accepted as information input to the object V1 and reflected on the object V1 in a visible manner. Note that, in FIG. 6(B), the input operation is performed on a software keyboard displayed so as to be superimposed on the input device 50 that is visible via the HMD 30, but the input operation may also be performed on a software keyboard displayed on the display unit of the input device 50.

[0055] When the user U completes the input operation on the input device 50, the user U uses a gesture to press the end button B1 shown in FIG. 6(B). Then, as shown in FIG. 6(C), the object V1 and the input device 50, which have been in contact and linked, are separated from each other. As a result, the object V1 and the input device 50 are no longer linked, and the software keyboard that was displayed superimposed on the input device 50 disappears. The input information is reflected on the object V1 and is displayed in a visible manner.

[0056] The method of separating the object V1 and the input device 50 from each other and putting them into an unlinked state is not limited to pressing the end button B1. For example, the object V1 can be put into an unlinked state by the user U's gesture of grabbing the object V1 that is in contact with the input device 50 and is linked, and then moving the object V1 away from the input device 50.

[0057] 7A to 7C show an example in which object V2 visible through HMD 30 is a PDF document and input device 50 is a pen tablet without a display function. FIG. 7A shows a state in which object V2 visible through HMD 30 and input device 50 are separated and not linked. Here, when user U performs a gesture of grabbing object V2 and moving object V2 to bring it into contact with input device 50, contact between object V2 and input device 50 is detected.

[0058] When contact between the object V2 and the input device 50 is detected as a result of the user U's gesture, the object V2 and the input device 50 enter a state of contact, i.e., a linked state, as shown in FIG. 7B. Then, the portion of the PDF document to be edited is enlarged and displayed so as to be superimposed on the input device 50, which is visible through the HMD 30. This allows the user U to draw by hand with a stylus as an input operation on the enlarged portion.

[0059] Then, when the user U performs handwriting drawing with a stylus as an input operation to the input device 50 existing in real space while looking at the enlarged area superimposed on the input device 50, the input operation is linked to an input operation to the object V2. As a result, the information input to the input device 50 is accepted as information input to the object V2 and is reflected in the object V2 in a visible manner.

[0060] When the user U completes the input operation on the input device 50, the user U uses a gesture to press the end button B2 shown in Fig. 7(B) with the stylus. Then, as shown in Fig. 7(C), the object V2 and the input device 50, which have been in contact and linked, are separated from each other. As a result, the object V2 and the input device 50 are no longer linked, and the area that was enlarged and superimposed on the input device 50 disappears. The input information is also displayed on the object V2 in a visible manner.

[0061] The method for separating the object V2 and the input device 50 from each other and disengaging them from each other is not limited to pressing the end button B2. For example, the object V2 can be disengaged from the input device 50 by the user U's gesture, by grabbing the object V2 that is in contact with the input device 50 and disengaging it from the input device 50, and then moving the object V2 away from the input device 50.

[0062] 8A to 8C show an example in which an object V3 visible through the HMD 30 is an object (hereinafter referred to as a "whiteboard") that can receive an input operation for drawing with a stylus and can be shared with another user U, and the input device 50 is a tablet terminal. FIG. 8A shows a state in which the object V3 visible through the HMD 30 and the input device 50 are separated and not linked. Here, when the user U performs a gesture in which the pointer L displayed by a pinching operation indicates the object V3, and then moves the object V3 to bring it into contact with the input device 50, contact between the object V3 and the input device 50 is detected.

[0063] When contact between the object V3 and the input device 50 is detected as a result of the user U's gesture, the object V3 returns to its original position, and the object V3 and the input device 50 enter a linked state, as shown in FIG. 8(B). Then, an image of the object V3 (whiteboard image) is displayed superimposed on the input device 50 in a state visible through the HMD 30. This allows the user U to draw by hand with a stylus as an input operation on the displayed image of the object V3.

[0064] Then, when the user U performs handwriting drawing with a stylus as an input operation to the input device 50 existing in real space while looking at the image of the object V3 displayed so as to be superimposed on the input device 50, the input operation is linked to the input operation to the object V3. As a result, the information input to the input device 50 is accepted as information input to the object V3 and is reflected in the object V3 in a visible manner.

[0065] When the user U completes the input operation on the input device 50, the user U uses a gesture to press the end button B3 shown in Fig. 8(B) with the stylus. Then, the object V3 and the input device 50 are no longer linked, and the image of the object V3 (whiteboard image) that was displayed superimposed on the input device 50 disappears, as shown in Fig. 8(C). The input information is also displayed on the object V3 in a visible manner.

[0066] The method of separating the object V3 and the input device 50 from each other and disengaging them from each other is not limited to pressing the end button B3. For example, the object V3 can be disengaged from the input device 50 by a gesture of the user U, by pointing the pointer L at the object V3 that is in contact with the input device 50 and disengaging it from the input device 50.

[0067] 9(A) to 9(C) show an example in which the object V4 visible through the HMD 30 is a software keyboard and the input device 50 is a tablet terminal. FIG. 9(A) shows a state in which the object V4 visible through the HMD 30 and the input device 50 are separated and not linked. In FIG. 9(A), an object V5 that reflects information input to the object V4 is visible through the HMD 30. Here, when the user U performs a gesture of grabbing the object V4 and moving the object V4 to bring it into contact with the input device 50, contact between the object V4 and the input device 50 is detected.

[0068] When contact between the object V4 and the input device 50 is detected as a result of the user U's gesture, as shown in FIG. 9(B), the object V4 as a software keyboard disappears, and a software keyboard is displayed superimposed on the input device 50, which is visible through the HMD 30. In addition, an object V6 as another object appears so as to be in contact with the input device 50. An input line and conversion candidates are displayed on the object V6. This allows the user U to perform an input operation on the software keyboard displayed superimposed on the input device 50 while looking at the input line and conversion candidates displayed on the object V6.

[0069] When the user U performs an input operation on the software keyboard while viewing the input line and conversion candidates displayed on the object V6, the input information is accepted as information input to the object V5. The information is then reflected in the object V5 in a visible manner. Note that, although FIG. 9(B) shows a configuration in which an input operation is performed on a software keyboard displayed so as to be superimposed on the input device 50, the input operation may also be performed on a software keyboard actually displayed on the input device 50.

[0070] When the user U completes the input operation on the input device 50, the user U uses a gesture to press the end button B4 shown in Fig. 9(B). Then, as shown in Fig. 9(C), the object V6 disappears, the object V6 and the input device 50 are no longer linked, and the software keyboard that was displayed superimposed on the input device 50 disappears. In addition, the information input via the software keyboard is displayed on the object V5 in a visible manner.

[0071] The method for separating the object V6 and the input device 50 from each other and disengaging them from each other is not limited to pressing the end button B4. For example, the object V4 can be disengaged by the user U's gesture, such as grabbing the object V4 that is in contact with the input device 50 and disengaging it from the input device 50, and then moving the object V4 away from the input device 50.

[0072] 10(A) to 10(C) show an example in which an object V7 viewable through the HMD 30 is a three-dimensional model that can accept input for editing by three-dimensional model design software, and the input device 50 is a laptop personal computer. FIG. 10(A) shows a state in which the object V7 viewable through the HMD 30 and the input device 50 are separated and not linked. Here, when a gesture by the user U is made to grab the object V7 and move the object V7 to bring it into contact with the input device 50, contact between the object V7 and the input device 50 is detected.

[0073] When contact between the object V7 and the input device 50 is detected as a result of the gesture of the user U, the object V7 returns to its original position, and an editing screen for the three-dimensional model is displayed on the display unit of the input device 50, which is viewable through the HMD 30, as shown in Fig. 10(B). This allows the user U to perform input operations for editing the object V7, which is a three-dimensional model, while viewing the editing screen for the three-dimensional model, which is viewable through the HMD 30. When the user U performs input operations for editing the three-dimensional model while viewing the editing screen for the three-dimensional model, a three-dimensional model reflecting the input information is displayed as the object V7.

[0074] When the user U completes the input operation on the input device 50, the user U uses a gesture to press the end button B5 shown in Fig. 10(B). Then, as shown in Fig. 10(C), the object V7 and the input device 50 are no longer linked, and the editing screen for the three-dimensional model that was displayed on the display unit of the input device 50 disappears. Furthermore, the three-dimensional model that reflects the information input to edit the three-dimensional model is displayed in a visually recognizable manner as the object V7.

[0075] 11(A) to 11(C) show an example in which an object V8 visible through the HMD 30 is a speech bubble that can accept text input, and the input device 50 is a tablet terminal. The speech bubbles in FIGS. 11(A) to 11(C) are displayed in contact with or near inspection equipment in real space that is visible through the HMD 30. The speech bubbles display information about the inspection equipment itself, information about the inspection details, information obtained from the inspection equipment, and the like, in an editable manner. FIG. 11(A) shows a state in which the object V8 visible through the HMD 30 and the input device 50 are separated and not linked. Here, when a user U performs a gesture of grabbing the object V8 and moving the object V8 to contact the input device 50, contact between the object V8 and the input device 50 is detected.

[0076] When contact between the object V8 and the input device 50 is detected as a result of the user U's gesture, as shown in FIG. 11(B), the object V8 returns to its original position, and a software keyboard is displayed superimposed on the input device 50, which is visible through the HMD 30. In addition, another object, an object V9, appears so as to be in contact with the input device 50. An input line and conversion candidates are displayed on the object V9. This allows the user U to perform an input operation on the software keyboard displayed superimposed on the input device 50 while looking at the input line and conversion candidates displayed on the object V9.

[0077] When the user U performs an input operation on the software keyboard while viewing the input line and conversion candidates displayed on the object V9, the input information is accepted as information input to the object V9 and is reflected in the object V8 in a visible manner. Note that, in Fig. 11(B), the input operation is performed on a software keyboard displayed so as to be superimposed on the input device 50, but the input operation may be performed on a software keyboard actually displayed on the input device 50.

[0078] When the user U completes the input operation on the input device 50, the user U uses a gesture to press the end button B6 shown in Fig. 11(B). Then, as shown in Fig. 11(C), the object V9 disappears, the object V9 and the input device 50 are no longer linked, and the software keyboard that was displayed superimposed on the input device 50 disappears. In addition, the information input via the software keyboard is displayed on the object V8 in a visible manner.

[0079] The method for disabling the linked object V8 and the input device 50 is not limited to pressing the end button B6. For example, the object V8 and the input device 50 can be disengaged by the user U's gesture of grabbing the object V9 that is in contact with the input device 50 and moving the object V9 away from the input device 50.

[0080] Although the present embodiment has been described above, the present invention is not limited to the above-described embodiment. Furthermore, the effects of the present invention are not limited to those described in the above-described embodiment. For example, the system configuration shown in FIG. 1 and the hardware configuration shown in FIG. 2 are merely examples for achieving the object of the present invention and are not particularly limited. Furthermore, the functional configuration shown in FIG. 3 is also merely an example and is not particularly limited. It is sufficient that the information processing system 1 of FIG. 1 is provided with a function that can execute the above-described processing as a whole, and the functional configuration used to realize this function is not limited to the example of FIG. 3.

[0081] 4 and 5 are merely examples and are not particularly limited. The steps do not necessarily have to be performed in chronological order, but may be performed in parallel or individually. The specific examples shown in FIGS. 6 to 11 are also merely examples and are not particularly limited.

[0082] 6 to 11, for example, the user U's gesture is configured to touch the input device 50 by grasping or pointing at the object, but this is not limiting. As long as the object and the input device 50 come into contact with each other, the user U's gesture may be configured to move the input device 50 in real space and touch the object. Specifically, the user U moves the input device 50 held in his / her hand to touch the object displayed on the HMD 30. In this case, the contact detection unit 107 (see FIG. 3) of the HMD 30 detects that the contact between the object and the input device 50 is based on an intentional operation (a gesture in this case) by the user U.

[0083] Furthermore, for example, although the above-described embodiment employs a see-through HMD 30, it is also possible to employ a non-see-through HMD 30. That is, in the present invention, it is sufficient for the user to be able to visually recognize the object and the input device via the display device, so for example, the captured image of the input device may be displayed in real time on the non-see-through HMD. [Explanation of symbols]

[0084] 1...information processing system, 30...HMD, 31...control unit, 50...input device, 90...network, 101...display control unit, 102...data acquisition unit, 103...registration determination unit, 104...behavior estimation unit, 105...device identification unit, 106...position acquisition unit, 107...contact detection unit, 108...input information reception unit, 109...input control unit

Claims

1. a processor; The processor: performing control to make the object and the input device that can accept the information input operation visible via the display device; detecting contact between the object and the input device based on information relating to the respective positions of the object and the input device that are visible through the display device; When the detected contact is an operation of moving the object and contacting the input device by a user gesture, the object and the input device are linked as being based on an intentional operation by the user, and information input to the input device is accepted as information input to the object. Information processing device.

2. The processor further detects a state of the object that has come into contact with the input device, and determines whether to link the object with the input device depending on the state of the object. The information processing device according to claim 1 .

3. The processor is characterized in that if the state of the object is such that writing is prohibited or input is not possible, when the object comes into contact with the input device, the processor repels the object without coordination. The information processing device according to claim 2 .

4. the processor performs control to make the input device present in real space visible via the display device, 4. The information processing device according to claim 1.

5. the processor is characterized in that it identifies an input device that is a target of an input operation by the user among the input devices present in the real space and performs control to make the input device visible via the display device. The information processing device according to claim 4 .

6. The processor is characterized in that it identifies the input device that is the target of the user's input operation based on the behavior of the input device estimated from one or more of data provided from the input device, image data of the input device, and sensing data targeted at the input device, and performs control to identify the input device that is the target of the user's input operation and make it visible via the display device. The information processing device according to claim 5 .

7. The processor is characterized in that it identifies the input device that is the target of the user's input operation based on the behavior of the input device estimated from the matching result between inertial measurement data (IMU data) of the input device and three-dimensional behavior data, and controls to make the input device visible via the display device. The information processing device according to claim 6 .

8. The processor performs control to make the input device visible through the display device worn by the user.

4. The information processing device according to claim 1.

9. The processor controls the input device to be visible via a head-mounted display as the display device worn on the user's head. The information processing device according to claim 8 .

10. the processor controls the object and the input device, which can accept the input operation of one or more types of information selected from text information, audio information, and image information, to be visible via the display device.

4. The information processing device according to claim 1.

11. the processor controls the object and the input device that can accept the input operation of the one or more types of information to be visible via the display device, and controls the processor to further make visible other objects that support the input operation to the input device. The information processing device according to claim 10.

12. the processor performs control to make the other object visible near the input device or so as to be superimposed on at least a part of the input device. The information processing device according to claim 11.

13. the processor detects contact between the object and the input device based on information indicating outer edges of the object and the input device that are visible through the display device as the information about the position.

4. The information processing device according to claim 1.

14. the processor detects contact between outer edges of virtual regions surrounding the outer edge, which are specified from information indicating the outer edges of the object and the input device, as contact between the object and the input device. The information processing device according to claim 13.

15. The outer edge of the virtual area is specified based on a predetermined value as a distance value from the outer edge of each of the object and the input device toward the outside. The information processing device according to claim 14.

16. the processor accepts, as information input to the object, information input to the input device by an input function for performing the input operation that is specified based on a combination of the type of the object and a function of the input device.

4. The information processing device according to claim 1.

17. The processor identifies a function of the input device based on one or more of data provided from the input device, image data of the input device, and sensing data of the input device. The information processing device according to claim 16.

18. The processor accepts information input to the input device using the input function identified by the combination stored in a database as information input to the object. The information processing device according to claim 16.

19. The processor: further detecting a separation between the object and the input device in contact with each other based on information about the positions of the object and the input device; When the separation is detected, even if information is input to the input device, the information is not accepted as information input to the object.

4. The information processing device according to claim 1.

20. When the detected separation is based on an intentional operation by a user, the processor does not accept information input to the input device as information input to the object, even if the information is input to the input device. The information processing device according to claim 19.

21. The processor detects the contact when registration of a collaboration candidate between the object and the input device has been completed.

4. The information processing device according to claim 1.

22. The processor detects the contact when registration of a candidate for a collaboration between the object and a predetermined input device or an input device on which predetermined application software for connection is installed has been completed. The information processing device according to claim 21.

23. a display control means for controlling the display device to make visible the object and the input device that can accept an information input operation; a detection means for detecting contact between the object and the input device based on information about the positions of the object and the input device that are visible through the display device; a receiving means for linking the object with the input device as an intentional operation of the user when the detected contact is an operation of moving the object and contacting the input device by a user gesture, and receiving information input to the input device as information input to the object; An information processing system comprising:

24. a function of controlling the display device to make visible the object and the input device that can accept the input operation of information; a function of detecting contact between the object and the input device based on information regarding the respective positions of the object and the input device that are visible through the display device; a function of linking the object with the input device as a result of an intentional operation by the user when the detected contact is an operation of moving the object and contacting the input device by a user gesture, and accepting information input to the input device as information input to the object; A program to achieve this.

Images