Information processing system, information processing method, and program

Abstract

Provided are an information processing system, an information processing method, and a program with which it is possible to impart appropriate advice to a user and which present, to the user, apparatus-related information that makes it possible to more accurately imagine an installation location where an apparatus is to be actually installed. An information processing system (100) comprises: an MR device (1) worn by a user A; a storage device in which installation information necessary for installing an apparatus is stored in advance; a computation device for displaying an image on the MR device (1); and a terminal device (10) for displaying the image displayed on the MR device (1). The computation device creates a first image showing a three-dimensional image of the apparatus, generates a second image showing a space necessary for installation of the apparatus on the basis of the installation information stored in the storage device, and displays the second image on the MR device (1) and the terminal device (10) by superimposing the second image on the first image.

Description

Information processing system, information processing method, and program

[0001] The present disclosure relates to an information processing system, an information processing method, and a program that present information about a device to a user.

[0002] When negotiating sales of large equipment such as measuring devices such as gas chromatograph mass spectrometers and medical devices such as X-ray CT scanners, it has been necessary to help customers visualize the equipment by presenting them with a catalog of the equipment, etc. Therefore, in order to help customers visualize the size of the equipment, Japanese Patent Laid-Open Publication No. 2009-238228 (Patent Document 1) proposes an installation simulation system that inputs the size of the installation space for a medical image diagnostic device such as an MRI device, and generates and displays on a display an installation image of the medical image diagnostic device when virtually installed within the installation space.

[0003] JP 2009-238228 A

[0004] However, when a customer and a sales representative are negotiating a business deal, while the customer is viewing the installation image as a user of the installation simulation system, the sales representative cannot simultaneously view the installation image that the customer is viewing. As a result, the sales representative cannot provide appropriate advice to the user. Furthermore, even if the installation simulation system can provide an image of the size and installation location of the equipment, when actually installing the equipment, it is necessary to consider the space required for maintenance work, the space required for connecting piping and power, and so on, and there is a risk that the installation location used in the simulation will differ from the actual installation location of the equipment.

[0005] The present disclosure has been made to solve such problems, and aims to provide an information processing system, an information processing method, and a program that can provide appropriate advice to users and present information about devices to users so that they can more accurately imagine where to actually install the devices.

[0006] The information processing system disclosed herein is an information processing system that presents device information to a user. The information processing system includes an MR device worn by a user, a storage device that stores in advance installation information required for installing the device, a calculation device that displays an image on the MR device, and a terminal device that displays the image displayed on the MR device. The calculation device creates a first image showing a three-dimensional image of the device, generates a second image showing the space required for installing the device based on the installation information stored in the storage device, and displays the second image superimposed on the first image on the MR device and the terminal device.

[0007] The information processing method disclosed herein is an information processing method for presenting device information to a user in an information processing system. The information processing system includes an MR device worn by a user, a storage device in which installation information necessary for installing the device is pre-stored, a computing device that displays an image on the MR device, and a terminal device that displays the image displayed on the MR device. The information processing method includes the steps of creating a first image showing a three-dimensional image of the device, generating a second image showing the space required for installing the device based on the installation information stored in the storage device, and superimposing the second image on the first image and displaying it on the MR device and the terminal device.

[0008] The program disclosed herein is a program executed by the arithmetic unit to perform information processing for presenting information about the device to a user in an information processing system including an MR device worn by a user, a storage device in which installation information required to install the device is stored in advance, a processing unit that displays an image on the MR device, and a terminal device that displays the image displayed on the MR device. The program includes the steps of creating a first image showing a three-dimensional image of the device, generating a second image showing the space required to install the device based on the installation information stored in the storage device, and superimposing the second image on the first image and displaying it on the MR device and the terminal device.

[0009] In the present disclosure, a first image showing a three-dimensional image of the device is superimposed with a second image showing the space required to install the device and displayed on the MR device and terminal device, thereby making it possible to provide appropriate advice to users of the MR device and present users with information about the device that allows them to more accurately imagine the installation location where the device will actually be installed.

[0010] FIG. 1 is a schematic diagram of an information processing system according to an embodiment. FIG. 2 is a diagram showing an example of an image displayed on an MR device. FIG. 3 is a schematic diagram of a terminal device according to an embodiment. FIG. 4 is a flowchart for explaining an example of information processing in the terminal device according to an embodiment. FIG. 5 is a diagram showing an example of displaying an image displayed on an MR device in the terminal device according to an embodiment. FIG. 6 is a diagram showing an example of a display when an optional device is selected in the terminal device according to an embodiment. FIG. 7 is a diagram showing an example of displaying an estimated price for the device in the terminal device according to an embodiment. FIG. 8 is a diagram showing an example of displaying an explanation of the device in an image displayed on the MR device. FIG. 9 is a diagram showing an example of displaying an explanation of the device in the terminal device according to an embodiment. FIG. 10 is a diagram showing an example of a display that allows direct comparison between an existing device and a device to be newly installed.

[0011] Hereinafter, embodiments will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference numerals and description thereof will not be repeated.

[0012] [Information Processing System] In this embodiment, an example of an information processing system is described in which, when a customer and a sales representative are negotiating a business, the customer (user) is presented with information about a device using an MR (Mixed Reality) device. The information processing system according to this embodiment may also be used in situations other than business negotiations, such as explaining how to operate a device or confirming the installation route when installing the device. In the following embodiment, a gas chromatograph mass spectrometer is used as an example of the device, but the device is not limited to this and may be any large device such as a measuring device, an inspection device, a medical device, a large home appliance, or an automobile. FIG. 1 is a schematic diagram of an information processing system 100 according to an embodiment.

[0013] The information processing system 100 includes an MR device 1 worn by a user A (e.g., a customer) and a terminal device 10 operated by an operator B (e.g., a sales representative). The information processing system 100 shown in FIG. 1 will be described by way of example in which the storage of the terminal device 10 functions as a storage device in which installation information necessary for installing the device is pre-stored, and the processor of the terminal device 10 functions as a computing device that displays images on the MR device 1. However, the configuration of the storage device and computing device is not limited to this. The storage device and computing device may be configured to be implemented by a memory and a processor within the MR device 1, or may be implemented by an external device connected to the MR device 1. Furthermore, the configuration in which the storage device and computing device are implemented may be implemented by a cloud server or the like.

[0014] The MR device 1 is, for example, a head-mounted device. The MR device 1 includes a processor, a memory, a camera, multiple sensors, and a projection device. The MR device 1 has at least one visor or lens so that user A can view the real world. Furthermore, the MR device 1 has a projection device so that an image generated by a computing device can be displayed to user A on the visor or lens. Therefore, the MR device 1 can simultaneously display an image generated by the computing device overlaid on the real world seen by user A using the visor or lens. Furthermore, the MR device 1 can determine the orientation of user A, for example, the direction in which user A's head and / or eyes are facing, based on information from multiple sensors, and display an image corresponding to the determined orientation of user A.

[0015] Specifically, when the MR device 1 reads the two-dimensional code 2 placed on the table 20 with a camera, it displays an image showing a three-dimensional image of the device at the position where the two-dimensional code 2 is placed. FIG. 2 is a diagram showing an example of an image displayed on the MR device 1. In FIG. 1 , the two-dimensional code 2 is simply placed on the table 20 in the real world, but user A wearing the MR device 1 can see an image 3 (first image) showing a three-dimensional image of the device on the table 20, as shown in FIG. 2 . This allows user A to specifically imagine the size of the device to be installed on the table 20. Note that the position where the image 3 (first image) showing the three-dimensional image of the device is displayed is not limited to the position of the two-dimensional code 2, and may be a predetermined position or a position arbitrarily designated by user A or operator B.

[0016] The terminal device 10 operated by operator B can simultaneously display on the display unit 14 the image that user A is viewing using the MR device 1. The terminal device 10 can transmit and receive data to and from the MR device 1 via wireless communication, such as a short-range connection (e.g., Bluetooth® connection or near-field communication (NFC) connection) or an internet connection (e.g., Wi-Fi® or a mobile network). Therefore, the terminal device 10 can transmit an image 3 representing a generated three-dimensional image of the device to the MR device 1 and receive information such as the size and orientation of the image 3 representing the three-dimensional image of the device displayed on the MR device 1 from the MR device 1. This allows operator B (salesperson) to simultaneously view the image of the device that user A (customer) is viewing on the terminal device 10, enabling him or her to provide appropriate advice to user A.

[0017] 3 is a schematic diagram of a terminal device 10 according to an embodiment. By executing a program, the terminal device 10 can perform processes such as generating an image 3 showing a three-dimensional image of the device to be displayed on the MR device 1 and displaying the image 3 showing the three-dimensional image of the device on a display unit 14. The terminal device 10 includes a processor 11, a main memory 12, an input unit 13, a display unit 14, a storage 15, an external drive 16, and a communication unit 17. These components are connected via a processor bus 18.

[0018] The processor 11 may be configured, for example, by a central processing unit (CPU), a field programmable gate array (FPGA), a graphics processing unit (GPU), or a multi-processing unit (MPU). The processor 11 may also be configured by a processing circuit. The processor 11 can read programs (e.g., an OS 120 and a program 121) stored in the storage 15, deploy them in the main memory 12, and execute them. The processor 11 executes the program 121, which generates an image 3 representing a three-dimensional image of the device based on device information 122 stored in the storage 15 and performs processing such as displaying the image 3 on the MR device 1. The processor 11 may generate the image 3 representing the three-dimensional image of the device based on image data representing the three-dimensional image of the device previously included in the device information 122, or may generate the image 3 representing the three-dimensional image of the device based on information such as the size and appearance of the device included in the device information 122.

[0019] The main memory 12 is configured with a volatile storage device such as a dynamic random access memory (DRAM) or a static random access memory (SRAM). The storage 15 is configured with a nonvolatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD). In addition to an OS 120 for realizing basic functions, the storage 15 stores programs 121, device information 122, installation information 123, and the like.

[0020] The input unit 13 may be not only a switch provided on the terminal device 10 shown in Figure 1, but also a touch panel, microphone, camera, etc. provided on the surface of the display unit 14, and is a device for accepting operations and input data from operator B.

[0021] The display unit 14 is configured with a flat panel display such as an LCD (Liquid Crystal Display) or an organic EL (Electroluminescence) display. The external drive 16 accepts a storage medium such as a removable disk and acquires programs and data stored in the storage medium. The program 121 may be stored in the storage medium and read from the storage medium by the external drive 16.

[0022] The communication unit 17 transmits and receives data to and from other devices by performing wired or wireless communication. For example, by communicating with the MR device 1, the communication unit 17 can transmit an image 3 showing a three-dimensional image of the device to the MR device 1 and cause the MR device 1 to display the image 3 showing the three-dimensional image of the device. In addition, the communication unit 17 may be used to download the program 121, device information 122, and installation information 123 to the storage 15.

[0023] The processor 11 not only generates an image 3 showing a three-dimensional image of the device based on the device information 122 stored in the storage 15, but also generates an image 4 (second image) showing the space required to install the device based on the installation information 123. A measuring device such as a gas chromatograph mass spectrometer cannot be used for measurements permanently by simply being installed on the table 20; maintenance work is performed as needed. Therefore, during maintenance work on the device, it is necessary to ensure space for opening and closing the device's casing. By using the MR device 1 to display an image 4 showing the space required to install the device superimposed on the image 3 showing the three-dimensional image of the device, as shown in FIG. 2, the user A can more accurately imagine the installation location where the device will actually be installed.

[0024] The installation information 123 includes information on the size of the apparatus and the space required for maintenance work on the apparatus. Specifically, the installation information 123 includes information that, based on the installation specifications of the apparatus, a maintenance work space of, for example, 300 mm is required on the left side of the apparatus. In addition to the information on the space required for maintenance work on the apparatus, the installation information 123 may also include information on the space required for exhaust from the apparatus, the space required to connect the piping and power supply required to operate the apparatus, and the like. Specifically, the installation information 123 includes information that, based on the installation specifications of the apparatus, an exhaust duct protruding 180 mm in the depth direction from the rear surface is required. Furthermore, the installation information 123 includes information such as the space required to connect the piping for the carrier gas supplied to the apparatus and the space required to install a transformer for converting the voltage of the power supply supplied to the apparatus. Note that, in this embodiment, the apparatus information 122 and the installation information 123 are described as being stored separately in the storage 15, but the installation information 123 may also include the apparatus information 122.

[0025] [Information Processing for Presenting Device Information] Next, a process for presenting device information to a user in information processing system 100 will be described. In the configuration according to the present embodiment, this information processing is performed by processor 11 of terminal device 10. FIG. 4 is a flowchart for explaining an example of information processing in terminal device 10 according to the embodiment. First, processor 11 reads device information 122 stored in storage 15 (step S101). If device information 122 cannot be read (NO in step S101), processor 11 returns the process to step S101.

[0026] If the device information 122 has been read (YES in step S101), the processor 11 generates an image 3 (first image) showing a three-dimensional image of the device based on the device information 122 (step S102). Next, the processor 11 reads the installation information 123 stored in the storage 15 (step S103). If the installation information 123 has not been read (NO in step S103), the processor 11 returns the process to step S103.

[0027] If the installation information 123 has been read (YES in step S103), the processor 11 generates an image 4 (second image) indicating the space required for installing the device based on the installation information 123 (step S104).

[0028] The processor 11 superimposes an image 4 (second image) showing the space required for installing the device on an image 3 (first image) showing a three-dimensional image of the device, and outputs the superimposed image to the MR device 1 (step S105). The MR device 1 displays the image at the position of the two-dimensional code 2 provided on the table 20.

[0029] The processor 11 displays an image on the display unit 14 of the terminal device 10, in which an image 4 (second image) showing the space required to install the device is superimposed on an image 3 (first image) showing a three-dimensional image of the device (step S106). This allows the operator B (sales representative) to simultaneously view the image of the device that the user A (customer) is looking at on the terminal device 10, and can provide appropriate advice.

[0030] The MR device 1 is provided with a plurality of sensors, and therefore is able to display an image according to the direction of the user A. Therefore, the terminal device 10 can receive information such as the image being displayed on the MR device 1 or the size and orientation of the image being displayed on the MR device 1, and thereby display on the display unit 14 the same image as the image being viewed by the user A using the MR device 1 at almost the same time.

[0031] On the other hand, the image displayed on the MR device 1 may be operated by operating the image displayed on the terminal device 10. Fig. 5 is a diagram showing an example of displaying an image displayed on the MR device 1 in the terminal device 10 according to the embodiment. The display on the display unit 14 shown in Fig. 5 displays an image in which an image 4 (second image) showing the space required for installing the device is superimposed on an image 3 (first image) showing a three-dimensional image of the device, and also displays an operation button 153 for accepting an operation on the image. Note that the display unit 14 is provided with a touch panel, and when the operator B presses the operation button 153, the processor 11 can accept operation information of the pressing of the operation button 153 via the touch panel.

[0032] When the operator B operates the operation button 153, the processor 11 performs processing such as moving and rotating the image displayed on the display unit 14 in response to the operation of the operation button 153, thereby updating the display. Furthermore, the processor 11 transmits operation information corresponding to the operation of the operation button 153 to the MR device 1, and performs processing such as moving and rotating the image displayed on the MR device 1 to update the display. In other words, the terminal device 10 has the operation button 153 that can accept operation information for operating the image displayed on the MR device 1. Note that the configuration that can accept operations on the image displayed on the MR device 1 is not limited to the operation button 153, and may be an input device such as a mouse or a controller connected to the terminal device 10.

[0033] For devices such as measuring devices, testing devices, and medical devices, various optional devices can be selected and added. For example, for a gas chromatograph mass spectrometer, various optional devices such as an autosampler and an autoinjector can be selected. The space required to install the device varies depending on the type and number of optional devices selected. Therefore, a configuration will be described in which operator B selects additional devices to connect to the device from a menu of optional devices, and an image of the device including the selected additional devices is displayed on the MR device 1 and the terminal device 10.

[0034] 6 shows an example of a display when an optional device is selected on terminal device 10 according to the embodiment. Storage 15 further stores option information including at least information on the size of each of multiple optional devices connectable to the device and the space required for maintenance work on each of the multiple optional devices. Note that the option information may be included in device information 122 or installation information 123.

[0035] As shown in FIG. 6 , the terminal device 10 displays an image 154 on the display unit 14, in which an image 3 (first image) showing a three-dimensional image of the device is superimposed with an image 4 (second image) showing the space required to install the device. The terminal device 10 also displays an image 154 including a menu for operator B to select optional devices. The terminal device 10 can accept selection information from operator B to select an additional device to connect to the device from multiple optional devices. The processor 11 generates an image including additional devices that can be connected to the device based on the selection information accepted by the terminal device 10. Specifically, the processor 11 generates an image in which an image 3a (first image) showing a three-dimensional image of the device including the additional device is superimposed with an image 4a (second image) showing the space required to install the device including the additional device, as shown in FIG. 6 .

[0036] The processor 11 outputs the generated image to the MR device 1. The MR device 1 displays an image in which an image 3a (first image) showing a three-dimensional image of the device including the additional device is superimposed on an image 4a (second image) showing the space required to install the device including the additional device. This allows user A to more accurately imagine the installation location where the device, including the additional device, will actually be installed. In addition, the processor 11 displays an image on the display unit 14 in which an image 3a (first image) showing a three-dimensional image of the device including the additional device is superimposed on an image 4a (second image) showing the space required to install the device including the additional device.

[0037] Although it has been explained that the image 154 including the menu is displayed on the display unit 14 and the operator B selects an optional device, the image 154 including the menu may be displayed on the MR device 1 and the user A may select an optional device.

[0038] By storing in storage 15 the price information of the device in device information 122, the price information of optional devices in option information, and the information on the installation cost of the device in installation information 123, processor 11 can calculate an estimate for the device including the additional devices and display it on MR device 1 and terminal device 10. Fig. 7 is a diagram showing an example of displaying the estimate for the device on terminal device 10 according to the embodiment.

[0039] As shown in FIG. 7 , the terminal device 10 displays an image in which an image 4a (second image) showing the space required to install the device including the additional device is superimposed on an image 3a (first image) showing a three-dimensional image of the device including the additional device, and also displays an image 155 including an estimated cost of the device including the additional device to be presented to user A on the display unit 14. In FIG. 7 , the image 155 including the estimated cost of the device including the additional device is displayed on the display unit 14 of the terminal device 10, but it is also displayed on the MR device 1. Note that the image 155 including the estimated cost displayed on the display unit 14 of the terminal device 10 may display information (e.g., cost rate, profit margin, etc.) different from that displayed on the MR device 1. This allows the sales representative using the terminal device 10 to be provided with information useful for sales activities.

[0040] The MR device 1 may further display an image explaining the device in addition to an image in which an image 4 (second image) showing the space required for installing the device is superimposed on an image 3 (first image) showing a three-dimensional image of the device. FIG. 8 is a diagram showing an example in which an explanation of the device is displayed in an image displayed on the MR device 1. The storage 15 further stores explanatory data regarding the device. The explanatory data may be included in the device information 122 or the installation information 123. The explanatory data may also include data explaining the features of the device, data explaining improvements from the previous device, data explaining how to operate the device, data explaining maintenance work for the device, etc. The explanatory data may be not only text data or still image data, but also moving image data.

[0041] The processor 11 generates an image in which an image 4 (second image) showing the space required for installing the device is superimposed on an image 3 (first image) showing a three-dimensional image of the device, and further generates an image 5 (third image) based on the explanatory data. The processor 11 outputs the generated image to the MR device 1. As shown in FIG. 8 , the MR device 1 displays an image in which an image 4 (second image) showing the space required for installing the device is superimposed on an image 3 (first image) showing a three-dimensional image of the device, and also displays an image 5 (third image) based on the explanatory data. Therefore, by viewing the image of the device on the MR device 1, user A can not only more accurately imagine the actual installation location for the device, but also easily understand the explanation of the product's features, etc.

[0042] Meanwhile, terminal device 10 also displays image 5 based on the explanatory data on display unit 14. FIG. 9 is a diagram illustrating an example of a device description displayed on terminal device 10 according to an embodiment. As shown in FIG. 9 , display unit 14 displays image 5 (third image) based on the explanatory data in addition to an image in which image 3 (first image) showing a three-dimensional image of the device is superimposed with image 4 (second image) showing the space required for installing the device. Furthermore, since terminal device 10 is used by sales representatives, image 157 (third image) containing information to support sales activities is also displayed on display unit 14. This allows sales representatives to provide consistent explanations regardless of their individual abilities by looking at image 157 containing information to support sales activities while explaining the product, thereby reducing variations in sales activities. The information to support sales activities may include customer information, information on other companies' products, and the like, in addition to a detailed product description.

[0043] The explanation data stored in storage 15 includes first information that can be viewed only by operator B who operates terminal device 10, and second information that can be viewed by user A and operator B. The first information includes, for example, information for supporting the sales activities described above, and may also include information such as procedures for explaining the operation of the device to user A and procedures for explaining maintenance work for the device to user A. The second information includes information for explaining the features of the device, information for explaining improvements from the old device, information for explaining the operation of the device, information for explaining maintenance work for the device, etc.

[0044] When a customer purchases a product, they may purchase it to replace an existing device and want to directly compare the size of the new product with that of the existing device. Therefore, the MR device 1 may display an image 3 (first image) showing a three-dimensional image of the device superimposed on the existing device. Figure 10 shows an example of a display that allows a direct comparison between the existing device and the new device.

[0045] 10 , the MR device 1 displays an image in which an image 3 (first image) showing a three-dimensional image of the device is superimposed with an image 4 (second image) showing the space required for installing the device, superimposed on the existing device 6 placed on the table 20. Therefore, by viewing the image of the device on the MR device 1, user A can more accurately imagine the size and installation location of the newly purchased device compared to the existing device 6. Note that the terminal device 10 may also display an image on the display unit 14 in which an image 3 (first image) showing a three-dimensional image of the device is superimposed with an image 4 (second image) showing the space required for installing the device, superimposed on the existing device 6.

[0046] Furthermore, the processor 11 may generate an image (first image) showing a three-dimensional image with part of the device made transparent, and display the image on the display unit 14 of the MR device 1 and the terminal device 10. Specifically, the MR device 1 may make part of the device transparent in image 3 showing the three-dimensional image of the device shown in Fig. 2. By making part of the device transparent and showing the internal structure of the device, it is possible to easily explain the features and operation method of the device.

[0047] In the above-described embodiment, the MR device 1 has been described as an example of a device used by a user, but the device used by a user may be a head-mounted display such as the MR device 1 or a device such as smart glasses. Furthermore, the device used by a user is not limited to the MR device 1 and may be, for example, a smartphone or a PC. When the device used by a user is a smartphone or a PC, the user can use an AR app to photograph the space with a camera built into the smartphone or PC, and check the size and installation location of the device.

[0048] In the above-described embodiment, an example of use at the installation location of the device has been described, but in addition to this, it can also be used to check the delivery route to the installation location. Specifically, the user walks along the route to the room where the device is to be installed while wearing the MR device 1, and checks whether the delivery entrance through which the device will be delivered has sufficient width and height, whether there are any obstacles, etc. In this case, the image 3 showing a three-dimensional image of the device or an image showing a three-dimensional image of the device in its packaged state may be superimposed on the image of the delivery route.

[0049] Furthermore, the information processing system 100 may warn the user by using the MR device 1 or the like when the width of the carry-in route is too narrow for the device to be carried in. For example, when the distance between the image 3 showing the three-dimensional image of the device and an obstacle such as a wall becomes equal to or less than a predetermined length (e.g., 10 cm), the color of the image 3 changes to yellow, and when the device collides with an obstacle, the color of the image 3 changes to red. The warning may be given by display, sound, or vibration.

[0050] Furthermore, when checking the delivery route to the installation location, the orientation and angle of image 3 showing the three-dimensional image of the device can be adjusted as needed, just as when checking the installation location. However, the movable range of image 3 may be limited. This is effective when there are restrictions on handling during delivery. For example, if the device is upside down, the user cannot turn image 3 showing the three-dimensional image of the device upside down. Furthermore, if the device has a precise configuration and tilting it is not desirable, image 3 showing the three-dimensional image of the device cannot be tilted beyond a predetermined angle.

[0051] Aspects It will be understood by those skilled in the art that the above-described embodiments are specific examples of the following aspects.

[0052] (Item 1) An information processing system according to one embodiment is an information processing system that presents information about a device to a user, and includes an MR device worn by the user, a storage device in which installation information required to install the device is stored in advance, an arithmetic unit that displays an image on the MR device, and a terminal device that displays the image displayed on the MR device, wherein the arithmetic unit creates a first image showing a three-dimensional image of the device, generates a second image showing the space required to install the device based on the installation information stored in the storage device, and superimposes the second image on the first image and displays it on the MR device and the terminal device.

[0053] According to the information processing system described in paragraph 1, a second image showing the space required to install the device is superimposed on a first image showing a three-dimensional image of the device and displayed on the MR device and terminal device, thereby making it possible to provide appropriate advice to users of the MR device and to present users with information about the device that allows them to more accurately imagine the installation location where the device will actually be installed.

[0054] (2) In the information processing system according to the first aspect, the installation information includes at least information on the size of the device and the space required for maintenance work on the device.

[0055] (Clause 3) In the information processing system described in clause 2, the installation information further includes at least one of information on the space required for exhausting air from the device, and information on the space required for connecting the piping and power source required to operate the device.

[0056] (4) In the information processing system described in any one of paragraphs 1 to 3, the terminal device can accept operation information for performing operations on the first image and the second image displayed on the MR device.

[0057] (Clause 5) An information processing system described in any one of clauses 1 to 4, wherein the storage device further stores option information including at least information on the size of each of a plurality of optional devices connectable to the device and the space required for maintenance work on each of the plurality of optional devices, the terminal device accepts selection information for selecting an additional device connectable to the device from the plurality of optional devices, and the calculation device generates a first image and a second image including the selected additional device based on the selection information accepted by the terminal device, and displays them on the MR device and the terminal device.

[0058] (Item 6) In the information processing system according to item 5, the calculation device calculates an estimated price for the devices, including the additional devices, based on price information included in the option information, and displays the calculated price on the MR device and the terminal device.

[0059] (Clause 7) An information processing system described in any one of clauses 1 to 6, wherein the storage device further stores explanatory data relating to the device, and the calculation device creates a third image based on the explanatory data, and displays the created third image on the MR device and terminal device by superimposing the first image and the second image on the created third image.

[0060] (Clause 8) An information processing system as described in Clause 7, wherein the explanatory data includes first information that can only be viewed by an operator operating the terminal device and second information that can be viewed by the user and the operator, and the calculation device includes the second information in a third image displayed on the MR device, and includes the first information and the second information in the third image displayed on the terminal device.

[0061] (Item 9) In the information processing system according to any one of items 1 to 8, the arithmetic device generates a first image in which part of the device is made transparent.

[0062] (10) In the information processing system according to any one of the above items 1 to 9, the calculation device superimposes the first image and the second image on the existing device and displays them on the MR device and the terminal device.

[0063] (Clause 11) An information processing method according to one aspect is an information processing method for presenting information about a device to a user in an information processing system, the information processing system comprising an MR device worn by the user, a storage device in which installation information required to install the device is pre-stored, a computing device that displays an image on the MR device, and a terminal device that displays the image displayed on the MR device, and includes the steps of: creating a first image that shows a three-dimensional image of the device; generating a second image that shows the space required to install the device based on the installation information stored in the storage device; and superimposing the second image on the first image and displaying it on the MR device and the terminal device.

[0064] (Clause 12) A program according to one aspect is a program executed by the arithmetic device to perform information processing to present information about the device to a user in an information processing system including an MR device worn by a user, a storage device in which installation information required to install the device is pre-stored, an arithmetic device that displays an image on the MR device, and a terminal device that displays the image displayed on the MR device, the program including the steps of: creating a first image showing a three-dimensional image of the device; generating a second image showing the space required to install the device based on the installation information stored in the storage device; and superimposing the second image on the first image and displaying it on the MR device and the terminal device.

[0065] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the description of the above embodiments, and is intended to include all modifications within the meaning and scope of the claims.

[0066] 1 MR device, 10 terminal device, 11 processor, 12 main memory, 13 input unit, 14 display unit, 15 storage, 16 external drive, 17 communication unit, 18 processor bus, 20 table, 100 information processing system, 121 program, 122 device information, 123 installation information, 153 operation button.

Claims

1. An information processing system that presents information about a device to a user, comprising: an MR device worn by the user; a storage device in which installation information required to install the device is stored in advance; a calculation device that displays an image on the MR device; and a terminal device that displays the image displayed on the MR device, wherein the calculation device creates a first image that shows a three-dimensional image of the device, generates a second image that shows the space required to install the device based on the installation information stored in the storage device, and superimposes the second image on the first image and displays it on the MR device and the terminal device.

2. The information processing system according to claim 1, wherein the installation information includes at least information about the size of the device and the space required for maintenance work on the device.

3. The information processing system according to claim 2, wherein the installation information further includes at least one of information on the space required for exhausting air from the device, and information on the space required for connecting the piping and power source required to operate the device.

4. An information processing system according to any one of claims 1 to 3, wherein the terminal device is capable of accepting operation information for performing operations on the first image and the second image displayed on the MR device.

5. An information processing system as described in any one of claims 1 to 3, wherein the storage device further stores option information including at least information on the size of each of a plurality of optional devices connectable to the device and the space required for maintenance work on each of the plurality of optional devices, the terminal device accepts selection information for selecting an additional device connectable to the device from the plurality of optional devices, and the calculation device generates the first image and the second image including the selected additional device based on the selection information accepted by the terminal device, and displays them on the MR device and the terminal device.

6. The information processing system according to claim 5, wherein the computing device calculates an estimated price for the equipment including the additional equipment based on price information included in the option information, and displays the calculated price on the MR device and the terminal device.

7. An information processing system according to any one of claims 1 to 3, wherein the storage device further stores explanatory data relating to the device, and the computing device creates a third image based on the explanatory data, and displays the created third image on the MR device and the terminal device by superimposing the first image and the second image on the created third image.

8. The information processing system described in claim 7, wherein the explanatory data includes first information that can only be viewed by the operator operating the terminal device and second information that can be viewed by the user and the operator, and the computing device includes the second information in the third image displayed on the MR device and includes the first information and the second information in the third image displayed on the terminal device.

9. The information processing system according to any one of claims 1 to 3, wherein the computing device generates the first image by transmitting a part of the device.

10. An information processing system according to any one of claims 1 to 3, wherein the computing device superimposes the first image and the second image on an existing device and displays them on the MR device and the terminal device.

11. An information processing method for presenting information about a device to a user in an information processing system, the information processing system comprising an MR device worn by the user, a storage device in which installation information required to install the device is pre-stored, a computing device for displaying an image on the MR device, and a terminal device for displaying the image displayed on the MR device, the information processing method including the steps of: creating a first image showing a three-dimensional image of the device; generating a second image showing the space required to install the device based on the installation information stored in the storage device; and superimposing the second image on the first image and displaying it on the MR device and the terminal device.

12. In an information processing system comprising an MR device worn by a user, a storage device in which installation information required to install the device is pre-stored, an arithmetic device that displays an image on the MR device, and a terminal device that displays the image displayed on the MR device, a program executed by the arithmetic device to perform information processing to present information about the device to a user, the program including the steps of: creating a first image showing a three-dimensional image of the device; generating a second image showing the space required to install the device based on the installation information stored in the storage device; and superimposing the second image on the first image and displaying it on the MR device and the terminal device.

Images