Simulation program, simulation device, simulation system, and display method

Abstract

This simulation program causes a computer to function as: a three-dimensional model display unit (103) for displaying a three-dimensional model that reproduces a real space in which equipment is placed; a request accepting unit (104) for accepting, from a user, a request for equipment included in the three-dimensional model; and a function identifying unit (105) for identifying a function related to the equipment satisfying the request, by interpreting the request accepted by the request accepting unit (104) using an interactive AI function. The simulation program then causes the computer to function such that the three-dimensional model display unit (103) displays, in the three-dimensional model, information relating to the function identified by the function identifying unit (105).

Description

Simulation program, simulation device, simulation system, and display method

[0001] The present disclosure relates to a simulation program, a simulation device, a simulation system, and a display method.

[0002] Digital twin technology is known, which recreates real-world space as a three-dimensional model in a digital space. Digital twin technology makes it possible to display various information within the three-dimensional model in a digital space or within an image of the real space, depending on the user's request.

[0003] Meanwhile, there is known a technique for overlaying information on a real-space image in response to a user's request. For example, Patent Literature 1 discloses a technique for overlaying information acquired using a remote expert system on a real-space image displayed on a display terminal used by a worker in response to a request from the worker in the real space.

[0004] Special Publication No. 2003-515294

[0005] However, with the technology using the expert system described above, if the user makes a request that is not included in the set of pre-prepared rules, it is not possible to provide information that corresponds to the user's request. Because digital twin technology has a wide range of uses and is used by a variety of users who do not have specialized knowledge, there is a demand for it to display information that corresponds to requests made in the user's natural language.

[0006] The present disclosure has been made in consideration of the above circumstances, and aims to provide a simulation program, a simulation device, a simulation system, and a display method that are capable of displaying appropriate information that meets the user's requirements in a three-dimensional model that reproduces real space.

[0007] In order to achieve the above object, the simulation program of the present disclosure causes a computer to function as: a three-dimensional model display means that displays a three-dimensional model that reproduces a real space in which equipment is placed; a request receiving means that receives requests for equipment included in the three-dimensional model; and a function identification means that interprets the requests received by the request receiving means using an interactive AI function to identify functions related to the equipment that satisfy the requests, and the three-dimensional model display means displays information on the functions identified by the function identification means in the three-dimensional model.

[0008] According to the present disclosure, it is possible to provide a simulation program, a simulation device, a simulation system, and a display method that are capable of displaying appropriate information that meets the user's requirements in a three-dimensional model that reproduces real space.

[0009] FIG. 1 is a diagram showing a simulation system according to an embodiment; FIG. 2 is a block diagram showing the hardware configuration of an information processing device according to an embodiment; FIG. 3 is a diagram showing the functional configuration of a simulation device and a display terminal according to an embodiment; FIG. 4 is a diagram showing a state in which a widget is displayed on the screen of a simulation device according to an embodiment; FIG. 5 is a diagram showing a state in which a widget is displayed on the screen of a simulation device according to an embodiment; FIG. 6 is a diagram showing a state in which a widget is displayed on the screen of a simulation device according to an embodiment; FIG. 7 is a diagram showing a state in which a widget is displayed on the screen of a display device according to an embodiment;

[0010] (Embodiment) A simulation system according to an embodiment is a system that provides a digital twin environment for a real space in which facility equipment is arranged.

[0011] 1 shows a simulation system 1 of this embodiment. The simulation system 1 includes a simulation device 100 and a display terminal 200. The simulation device 100 is communicatively connected to the display terminal 200 via a wired or wireless communication network 500. The simulation device 100 and the display terminal 200 are communicatively connected to one or more pieces of equipment 300 arranged in a real space 400 via the communication network 500.

[0012] In the following, an example will be described in which the simulation system 1 is applied to a factory automation (FA) system installed in a factory. For example, the real space 400 is the space of the factory, and the equipment 300 is a plurality of FA devices installed in the factory. The FA devices include, for example, a programmable logic controller, a programmable display, a motion controller, a servo amplifier, an inverter, and a robot.

[0013] The simulation device 100 is a device that reproduces, in a digital space, a real space 400 in which facility equipment 300 is arranged. The simulation device 100 represents the real space 400 as a three-dimensional model. For example, the simulation device 100 generates a three-dimensional model of a factory space in which FA equipment is arranged. The simulation device 100 is generally placed in a location different from the factory, and is used, for example, by someone who designs the factory line layout, someone who designs the three-dimensional model, a factory site manager, etc. Hereinafter, a user who uses the simulation device 100 will be referred to as a "first user."

[0014] A simulation application 100-1 and an engineering tool 100-2 are installed in the simulation device 100.

[0015] The simulation application 100-1 displays a three-dimensional model that reproduces a real space 400, collects data from equipment 300 arranged in the real space 400, and reflects the collected data in the three-dimensional model. The simulation application 100-1 also has an interactive AI function, and uses the interactive AI function to output text or voice input from a user. Here, the interactive AI function is a generative AI function that uses language model technology, such as a large-scale language model, and is a function that appropriately interprets input data that includes ambiguous natural language requests and generates a natural language response.

[0016] The engineering tool 100-2 has a function of controlling the equipment 300 arranged in the real space 400, and, for example, collects data from the equipment 300 and performs settings on the equipment 300. Note that the number of engineering tools installed in the simulation device 100 is not limited to one, and multiple tools may be installed.

[0017] The display terminal 200 is a device capable of AR (Augmented Reality) display, which displays a virtual object superimposed on an image of the real space 400 captured by a camera. The display terminal 200 can also utilize the functions of the simulation application 100-1 and the engineering tool 100-2 in the simulation device 100. The virtual object is generated, for example, based on information received from the simulation device 100. The display terminal 200 is carried and used, for example, by a worker on a factory floor. Hereinafter, a user who uses the display terminal 200 will be referred to as a "second user."

[0018] An AR display application 200-1 is installed on the display terminal 200. The AR display application 200-1 transmits a request for the facility equipment 300 in the real space 400 to the simulation device 100. Then, the AR display application 200-1 displays information received from the simulation device 100 as a response to the request, superimposed on an image of the real space 400 captured by a camera.

[0019] FIG. 2 shows an example of the hardware configuration of the information processing device 10 in which the simulation device 100 and the display terminal 200 are realized.

[0020] The information processing device 10 has a processor 11 that executes various processes, a main memory unit 12 used as a work area for the processor 11, an auxiliary memory unit 13 that stores various data used in the processes of the processor 11, a communication unit 14 for communicating with external devices, an input unit 15 that acquires input information, and an output unit 16 that presents various information. The main memory unit 12, the auxiliary memory unit 13, the communication unit 14, the input unit 15, and the output unit 16 are all connected to the processor 11 via a bus 17.

[0021] The processor 11 includes a CPU (Central Processing Unit). The processor 11 executes programs stored in the auxiliary storage unit 13 to realize various functions of the information processing device 10.

[0022] The main memory unit 12 includes a RAM (Random Access Memory). Programs are loaded into the main memory unit 12 from the auxiliary memory unit 13. The main memory unit 12 is used as a working area for the processor 11.

[0023] The auxiliary storage unit 13 includes a non-volatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). In addition to programs, the auxiliary storage unit 13 stores various data used in the processing of the processor 11. In accordance with instructions from the processor 11, the auxiliary storage unit 13 supplies the processor 11 with data used by the processor 11 and stores the data supplied from the processor 11.

[0024] The communication unit 14 includes a network interface circuit for communicating with an external device. The communication unit 14 receives a signal from the external device and outputs data indicated by the signal to the processor 11. The communication unit 14 also transmits a signal indicating the data output from the processor 11 to the external device.

[0025] The input unit 15 includes input devices such as input keys, a pointing device, a microphone, a camera, etc. The input unit 15 acquires information input by the user of the information processing device 10 and notifies the processor 11 of the acquired information.

[0026] The output unit 16 includes output devices such as an LCD (Liquid Crystal Display) and a speaker. The output unit 16 may be configured as a touch screen integrally formed with a pointing device constituting the input unit 15. The output unit 16 presents various information to the user in accordance with instructions from the processor 11.

[0027] FIG. 3 shows the functional configuration of the simulation device 100 and the display terminal 200 according to this embodiment.

[0028] 3 functionally includes a three-dimensional model management unit 101 that manages three-dimensional models, a widget management unit 102 that manages widgets, a three-dimensional model display unit 103 that displays the three-dimensional models, a request receiving unit 104 that receives requests for the facility equipment 300, a function identification unit 105 that identifies a function related to the facility equipment 300 based on the request, a position identification unit 106 that identifies a position in the three-dimensional model where information on the identified function is to be placed, an analysis unit 107 that performs analysis to detect or predict a failure of the facility equipment 300, a response sending unit 108 that sends a response to a request from the display terminal 200, and a widget function unit 109 that represents a function indicated by a widget. The three-dimensional model management unit 101, the widget management unit 102, the three-dimensional model display unit 103, the request receiving unit 104, the function identification unit 105, the position identification unit 106, the analysis unit 107, and the response sending unit 108 are functions of the simulation application 100-1. The widget function unit 109 is a function that the engineering tool 100-2 has.

[0029] The three-dimensional model management unit 101 has a three-dimensional model that reproduces the real space 400 in which the equipment 300 is placed, collects data from the equipment 300, and reflects the collected data in the three-dimensional model. By reflecting the data in the three-dimensional model, the operating status of the equipment 300 in the real space 400 can be instantly reproduced on the three-dimensional model. The three-dimensional model management unit 101 is realized by the processor 11, the auxiliary storage unit 13, and the communication unit 14.

[0030] The widget management unit 102 imports widget functions, which are functions of the engineering tool 100-2, into the simulation application 100-1 and manages the widget functions so that they can be used on the three-dimensional model of the simulation application 100-1. The widget management unit 102 is realized by the processor 11, the auxiliary storage unit 13, and the input unit 15.

[0031] A widget is a graphical user interface displayed on a three-dimensional model. A widget function is a function that the engineering tool 100-2 has and that can be displayed on a three-dimensional model as a widget.

[0032] The widget management unit 102 receives, from the user, a designation of one or more engineering tools having widget functions to be imported. Upon receiving the designation of the engineering tools, the widget management unit 102 starts importing the widget functions of the designated engineering tools and manages the imported widget functions so that they can be used in the simulation application 100-1.

[0033] For example, when the widget management unit 102 receives a designation of the engineering tool 100-2 from the user, the widget management unit 102 imports the widget functions of the widget function unit 109 of the engineering tool 100-2 and manages the imported widget functions.

[0034] Note that importing of widget functions may be started automatically rather than in response to a user's designation of an engineering tool. Specifically, the simulation application 100-1 searches for engineering tools installed in the simulation device 100, and when an engineering tool is found, imports the widget functions of the engineering tool. If multiple engineering tools are installed in the simulation device 100, the search for the engineering tools and the import of widget functions may be repeated. Furthermore, the widget management unit 102 may display a list of importable widget functions on a screen and allow the user to select the widget functions they wish to import.

[0035] The three-dimensional model display unit 103 displays a three-dimensional model that reproduces the real space 400 in which the equipment 300 is placed. The three-dimensional model display unit 103 also updates the display of the three-dimensional model based on an operation by the first user. The three-dimensional model display unit 103 is realized by the processor 11, the communication unit 14, the input unit 15, and the output unit 16. The three-dimensional model display unit 103 is an example of a three-dimensional model display means.

[0036] For example, when the simulation application 100 - 1 is started, the three-dimensional model display unit 103 displays the three-dimensional model managed by the three-dimensional model management unit 101 on the screen of the simulation device 100 .

[0037] 4 shows how a three-dimensional model is displayed on the screen of simulation device 100. Simulation application 100-1 is displayed on the screen of simulation device 100, and the three-dimensional model is displayed in window 100-11 of simulation application 100-1, and a widget 601, which will be described later, is further displayed in window 100-11. The display of the three-dimensional model in window 100-11 is updated based on an operation on simulation application 100-1 by first user 701.

[0038] The request receiving unit 104 receives a request for the equipment 300 included in the three-dimensional model. The request receiving unit 104 is realized by the processor 11, the communication unit 14, and the input unit 15. The request receiving unit 104 is an example of a request receiving means.

[0039] A request to the facility device 300 is, for example, confirmation of information or status of the facility device 300, setting or operation of the facility device 300, or the like.

[0040] For example, when a first user 701 of the simulation device 100 inputs the voice "I would like to diagnose a geared motor" to the simulation application 100-1 as shown in FIG. 4, the request receiving unit 104 receives the request "I would like to diagnose a geared motor."

[0041] The function identifying unit 105 uses an interactive AI function to interpret the request received by the request receiving unit 104, and identifies a function related to the facility device 300 that satisfies the request. The function identifying unit 105 is realized by the processor 11 and the output unit 16. The function identifying unit 105 is an example of a function identifying means.

[0042] The function related to the equipment 300 is, for example, a function possessed by the engineering tool 100-2 that is communicatively connected to the equipment 300 placed in the real space 400, i.e., a widget function managed by the widget management unit 102.

[0043] Specifically, the function identification unit 105 acquires a list of widget functions imported from the engineering tool 100-2 from the widget management unit 102. Next, the function identification unit 105 interprets the request accepted by the request acceptance unit 104 using the interactive AI function, and identifies a function that satisfies the request from the acquired list of widget functions.

[0044] For example, for a request such as "I want to diagnose a geared motor," the function identification unit 105 uses the interactive AI function to interpret the content of the request and identify a widget function that can diagnose a geared motor from the list. For example, the function identification unit 105 identifies a widget function that performs "estimate gear backlash" as a widget function that satisfies the request such as "I want to diagnose a geared motor."

[0045] The position specifying unit 106 specifies the position, size, and orientation of the information on the function specified by the function specifying unit 105 in the three-dimensional model. The position specifying unit 106 is realized by the processor 11. The position specifying unit 106 is an example of a position specifying means.

[0046] Specifically, the position identification unit 106 uses a trained model that infers the position, size, and orientation of a widget of a widget function to be placed on a three-dimensional model based on the widget function and the area in which the three-dimensional model is displayed in the window. This trained model is generated by learning, for example, information on the position, size, and orientation in which a user has previously placed a widget on a three-dimensional model, and information on the area in which the three-dimensional model in which the user has placed the widget is displayed in the window, as training data.

[0047] For example, the position identification unit 106 identifies the position, size, and orientation of the “Gear Backlash Estimation” widget to be placed on the three-dimensional model using a trained model that infers the position, size, and orientation of the widget to be placed on the three-dimensional model.

[0048] The three-dimensional model display unit 103 displays the information on the function identified by the function identifying unit 105 in the three-dimensional model, arranging it at the position, size, and orientation identified by the position identifying unit 106 .

[0049] 4, the three-dimensional model display unit 103 places and displays the widget 601 for "Estimating gear backlash" based on the position, size, and orientation identified by the position identification unit 106. Then, the function identification unit 105 references the position where the widget is placed, generates a response such as "A widget for estimating gear backlash has been displayed next to device A," and outputs the generated response by voice.

[0050] Although the above describes the case where a widget is added to and displayed on a three-dimensional model, the deletion or update of a widget may also be performed based on a request from the first user. For example, if the request received by the request receiving unit 104 is a request to delete or update a widget, the function identifying unit 105 identifies a widget function that satisfies the request from among the widget functions of the widgets displayed in the three-dimensional model, and the three-dimensional model display unit 103 deletes or updates the widget of the identified widget function and displays it.

[0051] Here, if the function identification unit 105 cannot identify a function based on the request received by the request receiving unit 104, it uses an interactive AI function to generate questions to narrow down the functions to be identified from the functions related to the equipment 300 placed in the real space 400 reproduced by the three-dimensional model, based on the request received by the request receiving unit 104.

[0052] For example, as shown in FIG. 5 , when the request receiving unit 104 receives a request from the first user 701 via voice input, such as "I would like to adjust the servo used in the conveyor," the function identifying unit 105 uses the interactive AI function to interpret the content of the request received by the request receiving unit 104 as servo adjustment. However, since the list includes multiple available widget functions for adjusting the servo, the function identifying unit 105 is unable to identify a widget function that satisfies the request. In this case, the function identifying unit 105 uses the interactive AI function to generate a question, such as "What adjustment function do you want to use?" and then, by referring to the list of widget functions, generates an answer, such as "For the servo used in the conveyor, functions such as one-touch adjustment and manual adjustment are available," and outputs the question and answer by voice.

[0053] In response to the output question and answer, the first user 701 inputs, for example, a voice message such as "I would like to use the one-touch adjustment function" to the simulation application 100-1. The request receiving unit 104 then receives the request, "I would like to use the one-touch adjustment function." The function identifying unit 105 then identifies the widget function that satisfies the request as the one-touch adjustment function.

[0054] The three-dimensional model display unit 103 then arranges and displays the widget 602 having the widget function that performs the one-touch adjustment function, as shown in Fig. 5, based on the position, size, and orientation identified by the position identification unit 106. The function identification unit 105 then references the position where the widget is arranged, generates a response such as "The widget with the one-touch adjustment function has been displayed above the conveyor belt," and outputs the generated response by voice.

[0055] In addition, if the function identification unit 105 cannot identify a function based on the request accepted by the request accepting unit 104, it may identify a function related to the equipment 300 included in the three-dimensional model based on the results of the analysis by the analysis unit 107 and the request accepted by the request accepting unit 104.

[0056] For example, when the request receiving unit 104 receives a request such as "Please tell me the cause of the shutdown of device A," the function identifying unit 105 uses the interactive AI function to interpret the content of the request received by the request receiving unit 104 as identifying the cause of the shutdown of device A. If the list contains multiple widget functions related to identifying the cause of the shutdown of device A, the function identifying unit 105 determines that it has not been able to identify a widget function that satisfies the request. In this case, the function identifying unit 105 causes the analysis unit 107 to start analysis.

[0057] The analysis unit 107 performs analysis to detect failures of the facility equipment 300 based on the data collected from the facility equipment 300. The analysis unit 107 also performs analysis to estimate the degree of deterioration or wear of the facility equipment 300, predict failures, etc. based on the data collected from the facility equipment 300. The analysis unit 107 is realized by the processor 11. The analysis unit 107 is an example of an analysis means.

[0058] For example, the analysis unit 107 analyzes data on the facility equipment used in device A from among the data collected from the facility equipment 300, and obtains an analysis result that an alarm has occurred in the servo used in device A. Note that the analysis unit 107 may analyze the data collected from the facility equipment 300 using an analytical AI function that detects or analyzes the cause of a failure in the facility equipment 300. It is assumed that the analytical AI function is one in which an AI model is selected and parameters are adjusted in advance.

[0059] The function identification unit 105 identifies a widget function that executes "display servo alarm information" from a list of widget functions based on the analysis result that an alarm has occurred in the servo and the request "Please tell me the cause of the shutdown of device A." As shown in FIG. 6 , the three-dimensional model display unit 103 displays a widget 603 that displays the servo alarm information at the position, size, and orientation identified by the position identification unit 106. Then, the function identification unit 105 references the position where the widget is placed and generates a response such as "An alarm has occurred in the servo of device A. A widget for the alarm information has been displayed next to device A," and outputs the generated response by voice. By checking the widget 603, the first user 701 can recognize that a drop in the battery voltage of the servo is the cause.

[0060] Furthermore, the three-dimensional model display unit 103 immediately reflects the data of the facility equipment 300 collected by the engineering tool 100-2 in the information of the function identified by the function identifying unit 105 and displays it.

[0061] Specifically, when a widget displayed on the screen contains data that requires continuous and immediate display updates, such as a motor rotation speed, the three-dimensional model display unit 103 repeatedly executes an update process at predetermined intervals to update the display of the widget. First, the three-dimensional model display unit 103 requests the widget management unit 102 to update the data acquired by the widget function of the widget displayed on the screen. The widget management unit 102 then requests the engineering tool 100-2 to update the data acquired by the widget function that received the update request. Upon receiving the data update request, the widget function unit 109 of the engineering tool 100-2 communicates with the equipment 300 that has the data related to the request and acquires the data from the equipment 300. After acquiring the data from the equipment 300, the widget function unit 109 sends the collected data to the simulation application 100-1 as a response to the update request. Upon receiving a response from the engineering tool 100-2, the widget management unit 102 updates the data of the widget function, and the three-dimensional model display unit 103 displays the widget including the updated data.

[0062] For example, the widget 604 displayed in the window 100-11 in FIG. 7 is a widget for a widget function that acquires the rotation speed of a motor, and the three-dimensional model display unit 103 displays the value of the motor rotation speed in a field 604-1. The three-dimensional model display unit 103 also requests the widget management unit 102 to update the value of the motor rotation speed acquired by the widget function of the widget 604. The widget management unit 102 then requests the engineering tool 100-2 to update the value of the motor rotation speed acquired by the widget function of the widget 604. The widget function unit 109 of the engineering tool 100-2 communicates with a motor control device disposed in the real space 400 to acquire the value of the motor rotation speed, and sends information about the acquired value to the simulation application 100-1 as a response to the update request. Upon receiving a response from the engineering tool 100-2, the widget management unit 102 updates the data of the widget function that acquires the motor rotation speed, and the three-dimensional model display unit 103 displays the widget 604 including the updated data. In this way, the value of the motor rotation speed displayed in the column 604-1 is updated continuously and immediately, such as "100 [rpm]", "101 [rpm]", "102 [rpm]", . . .

[0063] Depending on the data to be displayed in the widget, communication from the engineering tool 100-2 to the facility device 300 may be performed multiple times based on a predefined sequence. Furthermore, when acquiring data from the facility device 300, data conversion, data division, etc. may be performed. For example, hexadecimal may be converted to decimal, or acquired data "0xFF00" may be divided into "0xFF" and "0x00." Furthermore, if the widget function is an offline function that does not involve communication with the facility device 300, such as a parameter converter function, the engineering tool 100-2 does not communicate with the facility device 300.

[0064] Furthermore, the information on the function identified by the function identification unit 105 includes a control that enables the setting or operation of the facility device 300 .

[0065] A control is something that allows settings or operations to be made on the facility device 300, and is represented by a button, edit box, spreadsheet, or the like included in a widget.

[0066] 8 shows a widget 605 including a control displayed in window 100-11. Widget 605 is a widget with a widget function that allows the user to set the notch frequency of the notch filter of device B reproduced in the three-dimensional model. Widget 605 includes an edit box 605-1, which is a control for inputting a character string, and a button 605-2, which is a control associated with a process. When button 605-2 is selected, the notch frequency of device B is set to the value input in edit box 605-1.

[0067] When the three-dimensional model display unit 103 receives an operation on a control for inputting a character string, it displays the input character string in a control of a widget in the three-dimensional model. Furthermore, when the three-dimensional model display unit 103 receives an operation on a control other than a control for inputting a character string, i.e., a control associated with a process, it requests the widget management unit 102 to execute the process associated with the control. Upon receiving the request, the widget management unit 102 requests the engineering tool 100-2 to execute the process associated with the control. Upon receiving the request, the widget function unit 109 of the engineering tool 100-2 executes the process associated with the control.

[0068] For example, when the first user 701 operates the mouse 1002 to select the edit box 605-1 and inputs the value "1000" using the keyboard 1001, the three-dimensional model display unit 103 determines that an operation on the character string input control has been accepted and displays "1000" in the edit box 605-1. Next, when the first user 701 operates the mouse 1002 to click the button 605-2, the three-dimensional model display unit 103 determines that an operation on the control associated with the process has been accepted and requests the widget management unit 102 to set the notch frequency of the notch filter of device B to "1000" [Hz]. The widget management unit 102 requests the engineering tool 100-2 to set the notch frequency of the notch filter of device B to "1000" [Hz], and the widget function of the engineering tool 100-2 sets the notch frequency of the notch filter of device B to "1000" [Hz].

[0069] In addition, the request receiving unit 104 receives a request for equipment 300 placed in the real space 400 from a display terminal 200 capable of displaying an image of the real space 400 as a request for equipment 300 included in the three-dimensional model.

[0070] 9, when the second user 702 of the display terminal 200 inputs the voice "I would like to diagnose the geared motor" to the AR display application 200-1 of the display terminal 200, the display terminal 200 transmits a request of the content indicated by the input voice to the simulation device 100. When the request receiving unit 104 of the simulation device 100 receives the request of the content "I would like to diagnose the geared motor" from the display terminal 200, the request receiving unit 104 receives the received request as a request for the equipment 300 included in the three-dimensional model.

[0071] That is, when the request receiving unit 104 receives a request from the display terminal 200, the function identification unit 105 identifies a function related to the equipment 300 that satisfies the request, just as when the request receiving unit 104 receives a request for the simulation application 100-1, and the position identification unit 106 identifies the position, size, and orientation in which to place the information of the identified function in the three-dimensional model.

[0072] When the request receiving unit 104 receives a request from the display terminal 200 for the equipment 300 arranged in the real space 400, the response sending unit 108 sends to the display terminal 200 a response including information on the function identified by the function identifying unit 105 and information on the position, size, and orientation identified by the position identifying unit 106. The response sending unit 108 is realized by the processor 11 and the communication unit 14. The response sending unit 108 is an example of a response sending means.

[0073] For example, in response to a request such as "I would like to diagnose a geared motor," the response sending unit 108 sends to the display terminal 200 a response including information identifying the widget function of "gear backlash estimation," information on the widget's position, size, and orientation, and information on the answer generated by the function identification unit 105, "A widget for estimating gear backlash has been displayed next to device A."

[0074] The widget function unit 109 is one or more functions of the engineering tool 100-2. The widget function of the widget function unit 109 is provided to the simulation application 100-1 as a usable module such as a dynamic link library (DLL) or as a service such as an application programming interface (API). This allows the simulation application 100-1 to use some or all of the functions of the engineering tool 100-2.

[0075] The display terminal 200 in Figure 3 functionally comprises a request sending unit 201 that sends a request to the simulation device 100, a response receiving unit 202 that receives a response from the simulation device 100, and an AR display unit 203 that displays information by superimposing it on an image of the real space 400.

[0076] The request sending unit 201 sends a request for the facility equipment 300 arranged in the real space 400 to the simulation device 100. The request sending unit 201 is realized by the processor 11, the communication unit 14, and the input unit 15. The request sending unit 201 is an example of a request sending means.

[0077] For example, when the second user 702 of the display terminal 200 inputs the voice, "I would like to diagnose the geared motor," to the AR display application 200-1 of the display terminal 200, as shown in FIG. 9, the request sending unit 201 sends a request to the simulation device 100 with the content indicated by the input voice.

[0078] The response receiving unit 202 receives a response from the simulation device 100. The response receiving unit 202 is realized by the processor 11 and the communication unit 14. The response receiving unit 202 is an example of a response receiving means.

[0079] For example, the response receiving unit 202 receives a response from the simulation device 100 that includes information identifying the widget function of “Gear backlash estimation,” information on the widget’s position, size, and orientation, and information on the answer “The widget for estimating gear backlash has been displayed next to device A.”

[0080] The AR display unit 203 displays information about a function included in the response received by the response receiving unit 202, superimposed on an image of the real space 400, based on position information included in the response received by the response receiving unit 202. That is, the AR display unit 203 of the display terminal 200 displays information about a function identified by the function identifying unit 105 of the simulation device 100, superimposed on an image of the real space 400, based on information about the position, size, and orientation identified by the position identifying unit 106 of the simulation device 100. The AR display unit 203 is realized by the processor 11 and the output unit 16. The AR display unit 203 is an example of an AR display means.

[0081] 9 shows how an image of the real space 400 is displayed on the screen of the display terminal 200. The AR display application 200-1 is displayed on the screen of the display terminal 200, and the image of the real space 400 is displayed in a window 200-11 of the AR display application 200-1, and further, a widget 606, which will be described later, is displayed in the window 200-11. Based on an operation on the AR display application 200-1 by the second user 702, the display of the image of the real space 400 in the window 200-11 is updated.

[0082] For example, the AR display unit 203 identifies the widget to be displayed in the window 200-11 as a "gear backlash estimation" widget based on the received information identifying the widget function. Furthermore, the AR display unit 203 identifies the position, size, and orientation at which the "gear backlash estimation" widget is to be displayed in the image of the real space 400 based on the received information on the position, size, and orientation of the widget. As shown in FIG. 9 , the AR display unit 203 displays the "gear backlash estimation" widget 606 by arranging it based on the identified position, size, and orientation. Then, the AR display unit 203 outputs a response by voice, such as, "The gear backlash estimation widget has been displayed next to device A."

[0083] Although the above describes the case where a widget is added to and displayed in the image of the real space 400, the deletion or update of the widget may also be performed based on a request from the second user. For example, when the request received by the request receiving unit 104 is a request to delete or update a widget, the function identifying unit 105 identifies a widget function that satisfies the request from among the widget functions of the widget displayed in the three-dimensional model, and the AR display unit 203 deletes or updates the widget of the identified widget function and displays it.

[0084] Furthermore, when the request receiving unit 104 cannot identify a function based on the request received from the display terminal 200, the function identifying unit 105 uses the interactive AI function to generate a question for narrowing down the functions to be identified from among the functions related to the equipment 300 placed in the real space 400 reproduced by the three-dimensional model, based on the request received by the request receiving unit 104. Then, the response sending unit 108 sends a response including the generated question to the display terminal 200.

[0085] Furthermore, if the function identification unit 105 cannot identify a function based on the request received by the request receiving unit 104 from the display terminal 200, the analysis unit 107 may perform analysis, and identify a function related to the facility device 300 based on the result of the analysis by the analysis unit 107 and the request received by the request receiving unit 104. Then, the response sending unit 108 sends a response including information on the identified function to the display terminal 200.

[0086] Furthermore, the AR display unit 203 may display data of the facility equipment 300 collected by the engineering tool 100-2 of the simulation apparatus 100, by instantly reflecting it in the information of the function identified by the function identification unit 105. Specifically, when a widget displayed on the screen contains data that requires continuous and immediate display updates, such as motor rotation speed, the AR display unit 203 requests the simulation apparatus 100 to repeatedly execute data update processing at regular intervals. Similar to updating data in a three-dimensional model, the simulation apparatus 100 sends an update request to the engineering tool 100-2 and receives a response to the request, i.e., a response including the updated data, from the engineering tool 100-2. Upon receiving the response from the engineering tool 100-2, the widget management unit 102 updates the data of the widget function, and the response sending unit 108 sends a response including the updated data to the display terminal 200. The AR display unit 203 displays the widget including the updated data.

[0087] Furthermore, the information on the function identified by the function identification unit 105 that the AR display unit 203 displays in the image of the real space 400 may include a control that enables setting or operation of the facility equipment 300 .

[0088] 10 shows a state in which a widget 607 including a control is displayed in the window 200-11. The widget 607 is a widget with a widget function that can set the notch frequency of the notch filter of device B placed in the real space 400. The widget 607 includes an edit box 607-1 that is a control for inputting a character string, and a button 607-2 that is a control associated with a process. When the button 607-2 is selected, the notch frequency of device B is set to the value input in the edit box 607-1.

[0089] When the AR display unit 203 receives an operation on a control for inputting a character string, it displays the input character string in the control of the widget in the image of the real space 400. Furthermore, when the AR display unit 203 receives an operation on a control associated with a process, the request sending unit 201 sends a request to execute the process associated with the control to the simulation device 100. When the request receiving unit 104 of the simulation device 100 receives the request to execute the process associated with the control, it requests the widget management unit 102 to execute the process associated with the control. When the widget management unit 102 receives the request, it requests the engineering tool 100-2 to execute the process associated with the control. Then, when the widget function unit 109 of the engineering tool 100-2 receives the request, it executes the process associated with the control.

[0090] For example, when the second user 702 selects the edit box 607-1 by gesture and inputs the value "1000" by voice, the AR display unit 203 determines that an operation on the character string input control has been accepted, and displays "1000" in the edit box 607-1. Next, when the second user 702 clicks the button 607-2 by gesture, the AR display unit 203 determines that an operation on the control associated with the process has been accepted, and the request sending unit 201 sends a request to the simulation device 100 to set the notch frequency of the notch filter of device B to "1000" [Hz]. When the request receiving unit 104 of the simulation device 100 receives a request from the display terminal 200 to set the notch frequency of the notch filter of device B to "1000" [Hz], the request receiving unit 104 requests the widget management unit 102 to set the notch frequency of the notch filter of device B to "1000" [Hz]. The widget management unit 102 requests the engineering tool 100-2 to set the notch frequency of the notch filter of device B to "1000" [Hz], and the widget function of the engineering tool 100-2 sets the notch frequency of the notch filter of device B to "1000" [Hz].

[0091] Next, a widget display process that the simulation device 100 according to this embodiment executes in response to a request to the simulation device 100 will be described with reference to the flowchart of Fig. 11. The display process of Fig. 11 is executed, for example, when the request receiving unit 104 receives a request from a first user of the simulation device 100. Note that the display process of Fig. 11 may be executed each time a request is received, thereby displaying multiple widgets on the simulation device 100.

[0092] The function identification unit 105 acquires a list of widget functions imported from the engineering tool 100-2 from the widget management unit 102 (step S101). Next, the function identification unit 105 interprets the request accepted by the request acceptance unit 104 using an interactive AI function and identifies a function that satisfies the request from the acquired list of widget functions (step S102). Then, the function identification unit 105 determines whether the function identification was successful (step S103). If the function identification unit 105 determines that the function identification was successful (step S103; YES), the position identification unit 106 identifies the position, size, and orientation at which information about the function identified by the function identification unit 105 is to be placed in the three-dimensional model (step S104). The three-dimensional model display unit 103 arranges and displays the information about the function identified by the function identification unit 105 in the three-dimensional model at the position, size, and orientation identified by the position identification unit 106 (step S105). Furthermore, the function specifying unit 105 generates a response to the request and outputs the generated response (step S106). On the other hand, if the function specifying unit 105 determines that the function specification has not been successful (step S103; NO), the process proceeds to step S107.

[0093] For example, as shown in FIG. 4 , when the request receiving unit 104 receives a voice request from the first user 701 saying, "I want to diagnose a geared motor," the function identifying unit 105 interprets the content of the request using the interactive AI function and identifies a widget function that performs "gear backlash estimation" from the list as a widget function that satisfies the request "I want to diagnose a geared motor." The position identifying unit 106 identifies the position, size, and orientation in which the "gear backlash estimation" widget is to be placed in the three-dimensional model. Then, as shown in FIG. 4 , the three-dimensional model display unit 103 places and displays the "gear backlash estimation" widget 601 based on the position, size, and orientation identified by the position identifying unit 106. Furthermore, the function identifying unit 105 references the position in which the widget is placed and generates a response such as, "The widget that estimates gear backlash has been displayed next to device A," and outputs the generated response by voice. On the other hand, if the request receiving unit 104 receives a request such as "I would like to adjust the servo used in the conveyor" or "Please tell me the cause of the stoppage of device A" and the function identifying unit 105 determines that it has not been able to identify the function, the process proceeds to step S107.

[0094] The analysis unit 107 performs an analysis to detect a failure of the equipment 300 based on the data collected from the equipment 300 (step S107). The function identification unit 105 identifies functions related to the equipment 300 included in the three-dimensional model based on the analysis results by the analysis unit 107 and the request accepted by the request acceptance unit 104 (step S108). The function identification unit 105 then determines whether the function identification was successful (step S109). If the function identification unit 105 determines that the function identification was successful (step S109; YES), the process proceeds to step S104. On the other hand, if the function identification unit 105 determines that the function identification was not successful (step S109; NO), the process uses an interactive AI function to generate and output a question for narrowing down the functions to be identified from among the functions related to the equipment 300 arranged in the real space 400 reproduced by the three-dimensional model, based on the request accepted by the request acceptance unit 104 (step S110).

[0095] For example, when the request receiving unit 104 receives a request such as "Please tell me the cause of the stop of device A," the analysis unit 107 analyzes data of the facility equipment used in device A from among the data collected from the facility equipment 300, and obtains an analysis result indicating that an alarm has occurred in the servo used in device A. Then, based on the analysis result indicating that an alarm has occurred in the servo and the request such as "Please tell me the cause of the stop of device A," the analysis unit 107 identifies a widget function that executes "display servo alarm information" from the list of widget functions. On the other hand, when the request receiving unit 104 receives a request such as "I would like to adjust the servo used in a conveyor," the function identification unit 105 cannot identify a function even using the analysis result. In this case, the function identification unit 105 uses its interactive AI function to generate a question such as "What adjustment function do you want to use?" and an answer such as "For the servo used in the conveyor, functions such as one-touch adjustment function and manual adjustment function are available," and outputs the question and answer.

[0096] Next, a widget update process executed by the simulation device 100 according to this embodiment will be described with reference to the flowchart in Fig. 12. The update process in Fig. 12 is repeatedly executed at predetermined intervals, for example, when a widget displayed on the screen contains data that requires continuous and immediate display updates.

[0097] The three-dimensional model display unit 103 requests the widget management unit 102 to update the data acquired by the widget function of the widget displayed on the screen (step S201). The widget management unit 102 then requests the engineering tool 100-2 to update the data acquired by the widget function that received the update request (step S202). Upon receiving the data update request, the widget function unit 109 of the engineering tool 100-2 communicates with the facility equipment 300 that has the data related to the request and acquires the data from the facility equipment 300 (step S203). Upon acquiring the data from the facility equipment 300, the widget function unit 109 sends the collected data to the simulation application 100-1 as a response to the update request (step S204). Upon receiving a response from the engineering tool 100-2, the widget management unit 102 updates the data of the widget function (step S205). The three-dimensional model display unit 103 displays the widget including the updated data (step S206).

[0098] For example, the three-dimensional model display unit 103 requests the widget management unit 102 to update the value of the motor rotation speed acquired by the widget function of the widget 604 in Fig. 7. The widget management unit 102 requests the engineering tool 100-2 to update the value of the motor rotation speed acquired by the widget function of the widget 604. The widget function unit 109 of the engineering tool 100-2 communicates with a motor control device arranged in the real space 400 to acquire the value of the motor rotation speed, and sends information about the acquired value to the simulation application 100-1 as a response to the update request. Upon receiving the response from the engineering tool 100-2, the widget management unit 102 updates the data of the widget function that acquires the motor rotation speed, and the three-dimensional model display unit 103 displays a widget including the updated data.

[0099] Next, a control process using a widget executed by the simulation device 100 according to this embodiment will be described with reference to the flowchart in Fig. 13. The control process in Fig. 13 is executed, for example, when a widget displayed on the screen includes a control and an operation on the control is received.

[0100] The three-dimensional model display unit 103 determines whether an operation on a control for inputting a character string has been received (step S301). If the three-dimensional model display unit 103 determines that an operation on a control for inputting a character string has been received (step S301; YES), the three-dimensional model display unit 103 displays the input character string in the control of the widget in the three-dimensional model (step S302). On the other hand, if the three-dimensional model display unit 103 determines that an operation on a control for inputting a character string has not been received (step S301; NO), the three-dimensional model display unit 103 requests the widget management unit 102 to execute a process associated with the control (step S303). If the widget management unit 102 receives the request, it requests the engineering tool 100-2 to execute the process associated with the control (step S304). If the widget function unit 109 of the engineering tool 100-2 receives the request, it executes the process associated with the control (step S305).

[0101] 8 using the mouse 1002 and inputs the value "1000" using the keyboard 1001, the three-dimensional model display unit 103 determines that an operation on a control for inputting a character string has been accepted, and displays "1000" in the edit box 605-1. Next, when the first user 701 operates the mouse 1002 to click the button 605-2, the three-dimensional model display unit 103 determines that an operation on a control associated with a process has been accepted, and requests the widget management unit 102 to set the notch frequency of the notch filter of device B to "1000" [Hz]. The widget management unit 102 requests the engineering tool 100-2 to set the notch frequency of the notch filter of device B to "1000" [Hz], and the widget function of the engineering tool 100-2 sets the notch frequency of the notch filter of device B to "1000" [Hz].

[0102] Next, a widget display process that the simulation device 100 according to this embodiment executes in response to a request received from the display terminal 200 will be described with reference to the flowchart of FIG. 14. The display process of FIG. 14 is executed, for example, when the request receiving unit 104 receives a request from a second user of the display terminal 200. Note that the display process of FIG. 14 may be executed each time a request is received, thereby displaying multiple widgets on the display terminal 200. The processes of steps S401 to S404 and steps S407 to S409 in FIG. 14 are the same as steps S101 to S104 and steps S107 to S109 in FIG. 11.

[0103] When the position, size, and orientation to be placed in the three-dimensional model are identified in step S404, the function identification unit 105 generates a response to the request using the interactive AI function (step S405).The response sending unit 108 then transmits to the display terminal 200 a response including information on the function identified by the function identification unit 105, information on the position, size, and orientation identified by the position identification unit 106, and information on the generated response (step S406).

[0104] For example, the function identification unit 105 generates a response saying, "A widget for estimating gear backlash has been displayed next to device A." The response sending unit 108 sends a response to the display terminal 200, which includes information identifying the widget function of "estimating gear backlash," information on the position, size, and orientation of the widget, and information on the response saying, "A widget for estimating gear backlash has been displayed next to device A."

[0105] In step S409, if the function identification unit 105 determines that the function identification was not successful (step S409; NO), the function identification unit 105 generates a question using the interactive AI function to narrow down the functions to be identified from the functions related to the equipment 300 placed in the real space 400 reproduced by the three-dimensional model, and the response sending unit 108 sends a response including the generated question to the display terminal 200 (step S410).

[0106] For example, the function identification unit 105 uses the interactive AI function to generate a question such as "What adjustment function would you like to use?" and a response such as "For servos used in conveyors, functions such as one-touch adjustment function and manual adjustment function are available." Then, the response sending unit 108 sends a response including information on the generated question and answer to the display terminal 200.

[0107] According to this embodiment, the simulation device analyzes requests input by a user using an interactive AI function, so even if the request is written in natural language that contains ambiguity, it is possible to display information on the functions of appropriate equipment that meets the user's request in a three-dimensional model.

[0108] Furthermore, according to this embodiment, the interactive AI function of the simulation device is used to identify functions that satisfy requests from users of display terminals, so that appropriate information that satisfies the user's request can be displayed even if the request from the user of the display terminal is in natural language that contains ambiguity. As a result, information on the appropriate facility equipment functions can be superimposed on an image of real space on an AR-capable terminal used by on-site workers.

[0109] Furthermore, according to this embodiment, even if the requirements from the user of the simulation device or display terminal are vague and it is difficult to identify a function that satisfies the requirements, the interactive AI function can be used to generate questions for identifying functions and prompt the user to submit further requirements, thereby making it possible to identify an appropriate function that satisfies the requirements.

[0110] Furthermore, according to this embodiment, even if the requirements from the user of the simulation device or display terminal are vague and it is difficult to identify a function that satisfies the requirements, by using the interactive AI function and the analysis function that detects failures, even a user without specialized knowledge can use an advanced analysis function such as the analytical AI function. This makes it possible to easily detect the cause of a failure in equipment or a production line on which the equipment is installed and present information on the cause of the failure.

[0111] Furthermore, according to this embodiment, the functions of an engineering tool can be easily displayed as widgets on a simulation device or a display terminal, which allows the functions of the engineering tool to be easily used by operating the widgets, without having to operate an engineering tool that requires complex pre-settings.

[0112] (Modifications) Although the embodiments of the present disclosure have been described above, various modifications and applications are possible when implementing the present disclosure.

[0113] 1 is an example and is not limiting. For example, there may be a plurality of simulation devices 100 or a plurality of display terminals 200. Furthermore, the simulation system 1 may be capable of communicating with the facility devices 300 located in a plurality of different real spaces 400.

[0114] In the above embodiment, the engineering tool 100-2 is installed in the simulation device 100. However, the present invention is not limited to this. For example, the engineering tool may be installed in another device that is communicably connected to the simulation device 100.

[0115] Furthermore, in the above embodiment, the display terminal 200 is described as a device capable of AR display in which a virtual object is superimposed on an image of the real space 400. However, the display terminal 200 is not limited to this. For example, the display terminal 200 may be a device capable of MR (Mixed Reality) display using a translucent head-up display. In the case of a display terminal 200 capable of MR display, the input unit 15 of the display terminal 200 is a positioning device such as a three-dimensional scanner, and the output unit 16 is a translucent image output device such as a head-up display. The positioning device identifies the position and orientation of the display terminal 200, and the virtual object is composited with the real space 400 displayed on the translucent screen. The display terminal 200 may be a glasses-type, a mobile terminal-type, or the like, and may have any shape.

[0116] In the above embodiment, requests to the simulation device 100 and the display terminal 200 are made by voice input, but this is not limiting. Requests may also be made by text input using the keyboard 1001 of the simulation device 100 or a keyboard externally attached to the display terminal 200.

[0117] Furthermore, in the above embodiment, an example has been shown in which a widget is displayed on the simulation device 100 based on a request from the first user 701 of the simulation device 100, and an example has been shown in which a widget is displayed on the display terminal 200 based on a request from the second user 702 of the display terminal 200. However, this is not limiting. For example, a widget may be displayed on the simulation device 100 and the display terminal 200 simultaneously based on a request received from the first user 701 of the simulation device 100. Alternatively, a widget may be displayed on the simulation device 100 and the display terminal 200 simultaneously based on a request received from the second user 702 of the display terminal 200. Alternatively, the simulation device 100 may display a widget on the simulation device 100 and the display terminal 200 based on a request from a user who uses a device different from the simulation device 100 and the display terminal 200.

[0118] Furthermore, in the above embodiment, the simulation device 100 has an interactive AI function, but it does not have to have an interactive AI function. For example, the simulation device 100 may request the engineering tool 100-2 to update the data displayed in the specified widget function without using the interactive AI function. Then, the simulation device 100 or the display terminal 200 may display a widget including the updated data.

[0119] In the above embodiment, the flowcharts of the widget display process executed by the simulation device 100 are shown in Fig. 11 and Fig. 14, but the present invention is not limited to these. For example, the processes of steps S107 to S109 in Fig. 11 may be omitted, or the processes of steps S103 and S107 to S110 may be omitted. Furthermore, the processes of steps S407 to S409 in Fig. 14 may be omitted, or the processes of steps S403 and S407 to S410 may be omitted.

[0120] In the above embodiment, the simulation system 1 is applied to an FA system installed in a factory, but the applicable system is not limited to this. For example, the simulation system 1 can be applied to a power system, a water treatment system, a railway system, a building facility management system, an air conditioning equipment management system, etc.

[0121] Furthermore, by applying an operating program that defines the operation of the simulation device 100 according to the above embodiment to an existing personal computer or information terminal device, it is also possible to make the personal computer or information terminal device function as the simulation device 100 according to the embodiment.

[0122] Furthermore, the method of distribution of such a program is arbitrary; for example, it may be stored on a computer-readable recording medium such as a CD-ROM (Compact Disk Read-Only Memory), a DVD (Digital Versatile Disk), or a memory card and distributed, or it may be distributed via a communication network such as the Internet.

[0123] The present disclosure allows various embodiments and modifications without departing from the broad spirit and scope of the present disclosure. Furthermore, the above-described embodiments are intended to explain the present disclosure and do not limit the scope of the present disclosure. That is, the scope of the present disclosure is defined by the claims, not the embodiments. Various modifications made within the scope of the claims and the meaning of equivalent disclosures are considered to be within the scope of the present disclosure.

[0124] According to the present disclosure, it is possible to provide a simulation program, a simulation device, a simulation system, and a display method that are capable of displaying appropriate information that meets the user's requirements in a three-dimensional model that reproduces real space.

[0125] 1 Simulation system, 10 Information processing device, 11 Processor, 12 Main memory unit, 13 Auxiliary memory unit, 14 Communication unit, 15 Input unit, 16 Output unit, 17 Bus, 100 Simulation device, 100-1 Simulation application, 100-2 Engineering tool, 100-11 Window, 101 Three-dimensional model management unit, 102 Widget management unit, 103 Three-dimensional model display unit, 104 Request reception unit, 105 Function identification unit, 106 Position identification unit, 107 Analysis unit, 108 Response sending unit, 109 Widget function unit, 200 Display terminal, 200-1 AR display application, 200-11 Window, 201 Request sending unit, 202 Response reception unit, 203 AR display unit, 300 Equipment, 400 Real space, 500 Communication network, 601 to 607 Widget, 604-1 column, 605-1, 607-1 edit boxes, 605-2, 607-2 buttons, 701 first user, 702 second user, 1001 keyboard, 1002 mouse.

Claims

1. A simulation program that causes a computer to function as: a three-dimensional model display means that displays a three-dimensional model that reproduces the real space in which equipment is placed; a request receiving means that receives requests for equipment included in the three-dimensional model; and a function identification means that interprets the requests received by the request receiving means using an interactive AI function to identify functions related to the equipment that satisfy the requests, and the three-dimensional model display means displays information on the functions identified by the function identification means in the three-dimensional model.

2. The simulation program of claim 1, wherein the computer is further made to function as a position specifying means for specifying the position, size and orientation at which information on the function specified by the function specifying means is to be placed in the three-dimensional model, and the three-dimensional model display means displays the information on the function specified by the function specifying means by placing it in the three-dimensional model at the position, size and orientation specified by the position specifying means.

3. The simulation program according to claim 2, wherein the request receiving means receives a request for equipment placed in the real space from a display terminal capable of displaying an image of the real space and connected to the computer so as to be communicatively connected, as a request for equipment included in the three-dimensional model; and causes the computer to function as a response sending means that, when the request receiving means receives a request for equipment placed in the real space from the display terminal, sends to the display terminal a response including information on the function identified by the function identification means and information on the position, size, and orientation identified by the position identification means; and the display terminal displays the function information included in the response superimposed on the image of the real space based on the information on the position, size, and orientation included in the response.

4. The simulation program according to any one of claims 1 to 3, wherein, when the function identification means is unable to identify the function based on the request accepted by the request acceptance means, the function identification means uses the interactive AI function to generate questions to narrow down the functions to be identified from among functions related to equipment placed in the real space reproduced by the three-dimensional model, based on the request accepted by the request acceptance means.

5. A simulation program according to any one of claims 1 to 4, which causes the computer to function as an analysis means that performs analysis to detect failures in the equipment based on data collected from the equipment, and wherein the function identification means, when it is not possible to identify the function based on the request accepted by the request acceptance means, identifies a function related to the equipment included in the three-dimensional model based on the result of the analysis by the analysis means and the request accepted by the request acceptance means.

6. A simulation program according to any one of claims 1 to 5, wherein the information on the function identified by the function identification means includes a control that enables setting or operation of the facility device.

7. A simulation program according to any one of claims 1 to 6, wherein the function related to the facility equipment is a function possessed by an engineering tool that is communicatively connected to the facility equipment arranged in the real space.

8. The simulation program according to claim 7, wherein the three-dimensional model display means displays the data of the facility equipment collected by the engineering tool by immediately reflecting it in the information of the function identified by the function identification means.

9. A simulation device comprising: a three-dimensional model display means for displaying a three-dimensional model that reproduces a real space in which equipment is placed; a request receiving means for receiving requests for equipment included in the three-dimensional model; and a function identification means for interpreting the requests received by the request receiving means using an interactive AI function to identify functions related to the equipment that satisfy the requests, wherein the three-dimensional model display means displays information on the functions identified by the function identification means in the three-dimensional model.

10. A simulation system in which a simulation device that reproduces in a digital space a real space in which equipment is placed and a display terminal capable of displaying an image of the real space are communicatively connected, wherein the simulation device comprises: a three-dimensional model display means that displays a three-dimensional model that reproduces the real space in which the equipment is placed; a request receiving means that receives a request for the equipment included in the three-dimensional model; a function identification means that uses an interactive AI function to interpret the request received by the request receiving means and identify a function related to the equipment that satisfies the request; and a position identification means that identifies a position, size, and orientation in which information on the function identified by the function identification means is to be placed in the three-dimensional model, wherein the three-dimensional model display means displays the information on the function identified by the function identification means by placing it in the position, size, and orientation specified by the position identification means, and the request receiving means receives a request for the equipment placed in the real space received from the display terminal as a request for the equipment included in the three-dimensional model, the simulation system further comprising a response sending means for, when the request receiving means receives a request from the display terminal for equipment located in the real space, sending to the display terminal a response including information on the function identified by the function identifying means and information on the position, size, and orientation identified by the position identifying means, wherein the display terminal comprises: a request sending means for sending a request for equipment located in the real space to the simulation device; a response receiving means for receiving the response from the simulation device; and an AR display means for displaying the function information included in the response received by the response receiving means by superimposing it on an image of the real space, based on the information on the position, size, and orientation included in the response received by the response receiving means.

11. A display method executed by a simulation device, wherein the simulation device displays a three-dimensional model that reproduces a real space in which equipment is placed, accepts requests for the equipment included in the three-dimensional model, interprets the accepted requests using an interactive AI function to identify functions related to the equipment that satisfy the requests, and displays information on the identified functions in the three-dimensional model.

Images