Device, method and program for reproducing line operations

Abstract

To reproduce a line without delay in a virtual space even when a control program references a large number of variables.SOLUTION: A device includes: a communication unit that communicates with a control device configured to control a line by executing a control program; a memory unit that stores data on a plurality of objects for reproducing the line in a virtual space; and a control unit that reproduces the operation of the line in the virtual space. At least some of the plurality of objects are associated with one or more variables contained in the control program. The control unit generates a request including information 1220 on one or more variables associated with selected one or more objects 1201, 1202 based on the selection of one or more objects 1201, 1202 associated with one or more variables from among a plurality of objects. The communication unit transmits the request to the control device and receives values of one or more variables from the control device. The control unit operates the one or more objects 1201, 1202 in the virtual space based on the received values of the one or more variables.SELECTED DRAWING: Figure 12

Description

[Technical field]

[0001] The present disclosure relates to an apparatus for replicating line operation, and more particularly to a technique for selecting communication data. [Background technology]

[0002] In recent years, technology related to digital twins has become widespread. Digital twins are a technology that reproduces an environment similar to the real world in a virtual space (3D (dimensional) space) based on various information collected from the real world. Digital twins are sometimes used to reproduce the operation of industrial lines, etc., in virtual space. Applying digital twins to industrial lines makes it easier to analyze manufacturing defects in products at production sites and to improve the manufacturing process of products.

[0003] Regarding technology for reproducing the operation of a line, for example, JP 2022-035310 A (Patent Document 1) discloses an information processing device that "includes a design acquisition unit that acquires design information including digitized information about the equipment used, a specification acquisition unit that acquires equipment specifications corresponding to the equipment included in the design information, and an association unit that converts the equipment specifications and the design information into associated data that associates them with each other, and generates a virtual model using the associated data" (see [Abstract]). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2022-035310 Summary of the Invention [Problem to be solved by the invention]

[0005] An industrial line comprises a large number of machines, robots, and the like. A device that reproduces the operation of the line (hereinafter, sometimes referred to as a "reproduction device") requires variables of a control program linked to each component in order to reproduce the operation of each component. Therefore, the reproduction device must periodically obtain the values ​​of the variables from a control device that controls the line. However, a line may include a large number of machines, robots, and the like. In this case, the reproduction device must obtain the values ​​of a huge number of variables from the control device in order to reproduce the operation of the line. Therefore, depending on the communication state between the reproduction device and the control device, the reception process of the variable values ​​by the reproduction device may be delayed. As a result, the reproduction process of the line by the reproduction device may also be delayed.

[0006] According to the technology disclosed in Patent Document 1, when a control program references a large number of variables, the reproduction device may experience delays in receiving the variable values. This may cause delays in the virtual model generation process. Therefore, there is a need for a technology that can reproduce lines in a virtual space without delays, even when a control program references a large number of variables.

[0007] The present disclosure has been made in consideration of the above-described background, and an object in one aspect is to provide a technology for reproducing lines in a virtual space without delay, even when a large number of variables are referenced in a control program. [Means for solving the problem]

[0008] According to an embodiment, there is provided an apparatus for acquiring data for reproducing a line in a real space from a control device of a line. The apparatus includes a communication unit for communicating with a control device configured to control the line by executing a control program, a storage unit for storing data of a plurality of objects for reproducing the line in the virtual space, and a control unit for reproducing the operation of the line in the virtual space. At least a part of the plurality of objects is associated with one or more variables included in the control program. The control unit generates a request including information of one or more variables associated with the selected one or more objects based on the selection of one or more objects associated with the one or more variables from the plurality of objects. The communication unit transmits the request to the control device and receives values ​​of the one or more variables from the control device. The control unit operates the one or more objects in the virtual space based on the received values ​​of the one or more variables.

[0009] According to the above disclosure, the device may request only one or more variable values ​​associated with one or more selected objects from among a plurality of objects for reproducing a line in a virtual space from the control device. This reduces the amount of communication between the device and the control device, and may suppress the occurrence of communication delays. As a result, the device may reproduce a line in the real space in the virtual space without delay.

[0010] In one aspect, the control unit installs a virtual camera in the virtual space and detects one or more objects included in a shooting range of the virtual camera. Generating the request includes generating a request including information of one or more variables associated with the one or more objects included in the shooting range of the virtual camera.

[0011] According to the above disclosure, the device can request only one or more variable values ​​associated with one or more objects included in the shooting range of the virtual camera from the control device, thereby further reducing the amount of communication between the device and the control device.

[0012] In one aspect, the control unit accepts an input operation to move the virtual camera, and dynamically redetects one or more objects included in the virtual camera's shooting range based on the movement of the virtual camera, and changes variable information to be included in the request based on the results of the redetection.

[0013] According to the above disclosure, the device can update the variables requested from the control device every time the installation position of the virtual camera is changed. As a result, even if the user dynamically switches the position of the virtual camera, the device can request only the values ​​of one or more variables associated with one or more objects included in the shooting range of the virtual camera after the movement from the control device.

[0014] In one aspect, the control unit accepts a setting operation for one or more installation positions of the virtual camera, detects one or more objects included in the shooting range of the virtual camera at each of the one or more installation positions based on the setting operation, creates a list including one or more variables corresponding to the one or more objects included in the shooting range of the virtual camera for each of the one or more installation positions, and based on accepting an operation to select an installation position of the virtual camera, changes variable information to be included in the request to variable information included in the list corresponding to the selected installation position.

[0015] According to the above disclosure, the device may receive a setting input for the installation position of the virtual camera from a user in advance. The device may also create a list including one or more variables associated with one or more objects included in the shooting range of the virtual camera for each installation position. In this way, when the device receives an input for changing the installation position of the camera, the device may request the control device for the variables included in the list corresponding to the changed installation position.

[0016] In one aspect, the control unit does not generate the request based on one or more objects associated with the one or more variables not being selected from among the plurality of objects.

[0017] According to the above disclosure, the device can prevent unnecessary transmission of requests to the control device when there is no object to be reproduced in the virtual space.

[0018] In one aspect, generating a request including information of one or more variables associated with one or more selected objects includes excluding from the request information of variables associated with objects that are set to hidden and excluding from the request information of variables associated with objects that are set to transparent.

[0019] According to the above disclosure, the device may exclude variables associated with objects that are set to hidden or transparent from the request.

[0020] According to another embodiment, there is provided a method for obtaining data for reproducing a line in a real space from a line control device, the method including: generating a request including information of one or more variables associated with the one or more selected objects based on selection of one or more objects associated with one or more variables from among a plurality of objects for reproducing a line in the virtual space, transmitting the request to the control device, receiving values ​​of the one or more variables from the control device, and operating the one or more objects in the virtual space based on the received values ​​of the one or more variables.

[0021] According to the above disclosure, the method may request the control device to provide only one or more variable values ​​associated with one or more selected objects among a plurality of objects for reproducing a line in a virtual space. This reduces the amount of communication between the line reproduction device and the control device, and suppresses communication delays. As a result, the line reproduction device may reproduce a line in a real space in a virtual space without delay.

[0022] According to yet another embodiment, there is provided a program executed by a computer, the program causing the computer to generate a request including information of one or more variables associated with the one or more selected objects based on selection of one or more objects associated with one or more variables from among a plurality of objects for reproducing a line in a virtual space, transmit the request to a control device, receive values ​​of the one or more variables from the control device, and operate the one or more objects in the virtual space based on the received values ​​of the one or more variables.

[0023] According to the above disclosure, the program may request the control device to provide only one or more variable values ​​associated with one or more selected objects among a plurality of objects for reproducing a line in a virtual space. This reduces the amount of communication between the line reproduction device and the control device, and may suppress communication delays. As a result, the line reproduction device may reproduce a real-space line in a virtual space without delay. Effect of the Invention

[0024] According to an embodiment, even if a control program references a large number of variables, the lines can be reproduced in a virtual space without delay.

[0025] The above and other objects, features, aspects and advantages of the present disclosure will become apparent from the following detailed description of the disclosure taken in conjunction with the accompanying drawings. [Brief description of the drawings]

[0026] [Figure 1] FIG. 1 is a diagram illustrating an example of a system 10 to which the technology of the present disclosure can be applied. [Diagram 2] FIG. 13 is a diagram showing an example of one or more variables associated with each object constituting a line in a virtual space. [Diagram 3] FIG. 1 is a diagram showing an example of the configuration of a device 100 for reproducing the operation of a line according to the present embodiment. [Figure 4] FIG. 2 is a diagram illustrating an example of a relationship between objects and variables. [Diagram 5] FIG. 2 is a diagram illustrating an example of a configuration of an object. [Figure 6] FIG. 13 is a diagram showing an example of a setting screen for parameters of an object. [Figure 7] FIG. 11 is a diagram showing a first example of a method for selecting a variable to be requested from the control device 110. [Figure 8] FIG. 11 is a diagram showing a second example of a method for selecting a variable to be requested from the control device 110. [Figure 9] FIG. 11 is a diagram showing an example of a method for selecting a variable to be requested of the control device 110. [Figure 10] FIG. 13 is a diagram showing a fourth example of a method for selecting a variable to be requested of the control device 110. [Figure 11] 1 is a diagram showing an example of a procedure for selecting a variable that the device 100 requests from the control device 110. FIG. [Figure 12] FIG. 12 is a diagram showing an example of an acquired variable list 1220 selected by the procedure in FIG. [Figure 13] FIG. 13 is a diagram showing an example of a procedure in which the device 100 communicates with the control device 110 to reproduce a line. [Figure 14] 1 is a diagram showing an example of a communication procedure between the device 100 and the control device 110. FIG. [Figure 15] 11 is a diagram showing an example of a procedure for changing a variable that the device 100 requests the control device 110 to change. [Figure 16] 13 is a diagram showing an example of how variables to be included in an acquired variable list are extracted from a list of variables referenced by a control program; FIG. [Figure 17] 1A to 1C are diagrams illustrating an example of a variation of the system configuration according to the present embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0027] Hereinafter, embodiments of the technical idea according to the present disclosure will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. Also, each embodiment and each modification example etc. may be selectively combined as appropriate.

[0028] <A. Application Example>

[0029] (a. System Overview)

[0030] FIG. 1 is a diagram showing an example of a system 10 to which the technology of the present disclosure is applicable. The system 10 reproduces the real-world line 130 and its operation in a virtual space. The virtual line 120 reproduced in the virtual space operates in conjunction with the real-world line 130. That is, the virtual line 120 in the virtual space reproduces the operation of the real-world line 130 within the virtual space. The system 10 includes an apparatus 100 and a control apparatus 110.

[0031] The control apparatus 110 controls the real-world line 130. The line 130 may include, as components, sensors, machines, robots, inoperative objects such as bases, and HMIs (Human Machine Interfaces) etc. The robot may include a vertically articulated robot, a horizontally articulated robot, a robot having a parallel link, or any other arbitrary robot. The control apparatus 110 acquires information from various sensors on the line 130 and controls various machines and robots etc. by executing a control program. The control program refers to various variables. The variables include variables that store the position information of each component on the line 130. The control apparatus 110 stores the values of the respective variables in a memory. As an example, the control apparatus 110 stores in the memory a plurality of variables associated with the position information of each axis of a machine having three-axis joints. Also, as another example, the control apparatus 110 stores in the memory a plurality of variables associated with the respective angles of each joint of a multi-articulated robot.

[0032] The device 100 is a reproduction device for reproducing a line 130 in the real world as a virtual line 120 in a virtual space. The device 100 periodically receives the values ​​of variables associated with each configuration on the line 130 from the control device 110. The control device 110 then operates each configuration on the virtual line 120 in the virtual space based on the received values ​​of each variable. For example, it is assumed that the device 100 receives the values ​​of a plurality of variables associated with the angles of each joint of a multi-joint robot from the control device 110. In this case, the device 100 reproduces the movement of the multi-joint robot in the virtual space based on the values ​​of each variable.

[0033] In one aspect, the device 100 may display the virtual line 120 in the virtual space on a display connected to the device 100. In another aspect, the device 100 may transmit information about the virtual line 120 in the virtual space to another device. In this case, the other device may display the virtual line 120 in the virtual space on a display connected to the other device. In another aspect, the device 100 may display the virtual line 120 on a display connected to the device 100 and transmit information about the virtual line 120 to the other device. The user may monitor the line 130 virtually remotely by referring to the virtual line 120 in the virtual space via a display connected to the device 100 or the like.

[0034] In order to reproduce the line 130 in the virtual space (to move the virtual line 120), the device 100 needs to periodically receive the values ​​of many variables from the control device 110. However, when the line 130 includes many components, the device 100 needs to receive the values ​​of an enormous number of variables. Therefore, depending on the state of the network, the reception process of the variable values ​​by the device 100 may be delayed. As a result, the device 100 may not be able to reproduce the operation of the line 130 in the virtual space. Therefore, the device 100 reduces the number of variable values ​​received from the control device 110 by a procedure described with reference to Figs. 7 to 10. In this way, the device 100 can prevent delays in the reception process of the variable values ​​and stably reproduce the operation of the line 130 in the virtual space.

[0035] (b. Variables)

[0036] FIG. 2 is a diagram showing an example of one or more variables linked to each object constituting a line in a virtual space. The virtual line 200 is a reproduction of a real line in a virtual space. The virtual line 200 is composed of a large number of objects (machines, robots, sensors, etc.). Some of the objects are linked to variables. The variable list 210 is a list including all variables linked to objects in the virtual line 200. The variable list 210 may also include variables that are not related to the physical behavior of the objects. The variable list 210 may also include variables linked to objects outside the display range viewed by the user.

[0037] Conventional devices for realizing digital twins had to periodically receive the values ​​of all variables included in the variable list 210 from the control device when operating the virtual line 200 in a virtual space. If the virtual line 200 becomes large, the variables included in the variable list 210 also increase dramatically. Therefore, when reproducing a large line in a virtual space, the conventional device had to receive an enormous number of variable values ​​from the control device. If the number of variable values ​​to be transmitted and received increases, delays in transmitting and receiving the variable values ​​may occur depending on the state of the network.

[0038] (c.Term)

[0039] In this specification, the term "system" includes a configuration consisting of one or more devices, a server. The system also includes a virtual machine or a container built in a cloud environment, or a configuration consisting of at least a part of these. When the system consists of one device, the system may be read as a device. According to one embodiment, the system may be connected to input / output devices such as a display and a keyboard and used by a user. According to another embodiment, the system may provide various functions to a user as a service or web application via a network. In this case, the user may use the functions of the system via a browser or client software installed on his / her terminal.

[0040] In addition, in this specification, the "apparatus or reproduction apparatus" may be any information processing apparatus such as a personal computer, a workstation, a server apparatus, a tablet, or a smartphone. The apparatus may also be a combination of these.

[0041] In this specification, a "control device" is a device that controls various components included in a line. The control device includes a PLC (Programmable Logic Controller). The control device may also be a robot integrated controller that controls various robots in addition to machines and sensors.

[0042] In addition, in this specification, a "line" refers to equipment constructed for product manufacturing, assembly, inspection, packaging, any other process, and any combination thereof. A line is used in a factory, etc. A line is composed of machines, robots, sensors, a base, an HMI, and any other equipment. Robots may include vertical articulated robots, horizontal articulated robots, robots with parallel links, or any other robots. In this specification, a line reproduced in a virtual space may be referred to as a "virtual line."

[0043] In addition, in this specification, "configuration" means any thing included in the actual line. For example, the configuration includes machines, robots, sensors, bases, HMIs, and any other things that make up the actual line. For example, the machines and robots included in line 130 are configurations.

[0044] In addition, in this specification, "object" means any thing included in the virtual line. For example, the object includes machines, robots, sensors, bases, HMIs, and any other things that make up the virtual line. For example, the machines and robots included in virtual line 120 are objects. Since the object is an object in the virtual space, it may be represented as a 3D object.

[0045] In addition, in this specification, "variable" is a variable referenced in the control program executed by the control device 110. Some variables are associated with objects. The variable can store information indicating the position of each of the multiple configurations that build the line. As an example, the variable can store information indicating the position of a machine. As another example, the variable can store information indicating the angle of a joint of a robot. Further, as another example, the variable can store information (such as speed or acceleration) for calculating the position of each configuration.

[0046] In addition, in this specification, "transmission and reception of variable values" includes transmitting and receiving variable values, and transmitting and receiving combinations of variable names and variable values. Also, transmitting and receiving variable values may simply be expressed as transmitting and receiving variables. As an example, assume that device 100 receives variables A, B, and C from control device 110. In this case, device 100 receives the values of variables A, B, and C from control device 110. Or, device 100 receives the combination of each variable name and value of variables A, B, and C from control device 110. Also, in this specification, "requesting a variable value" may be expressed as requesting a variable.

[0047] <B. Device Configuration>

[0048] Fig. 3 is a diagram showing an example of the configuration of device 100 that reproduces the operation of a line according to the present embodiment. In one aspect, device 100 can realize various processes described in this specification by executing a program on the hardware shown in Fig. 3.

[0049] The device 100 includes a processor 301, a memory 302, a storage 303, an external device IF (Interface) 304, an input IF 305, an output IF 306, a communication IF 307, and a bus 308. Each component is configured to be able to communicate with each other via the bus 308. The storage 303 stores an object 310, a variable 311, and a visualizer 312.

[0050] The processor 301 may execute a program for implementing various functions of the device 100. The processor 301 may be configured, for example, with at least one integrated circuit. According to an embodiment, the integrated circuit may include at least one central processing unit (CPU), at least one graphics processing unit (GPU), at least one field programmable gate array (FPGA), at least one application specific integrated circuit (ASIC), at least one artificial intelligence (AI) chip, or a combination thereof.

[0051] The memory 302 stores programs executed by the processor 301 and data referenced by the processor 301. In one aspect, the memory 302 can be realized by a dynamic random access memory (DRAM), a static random access memory (SRAM), or the like.

[0052] Storage 303 is a non-volatile memory, and stores programs executed by processor 301 and data referenced by processor 301. In this case, processor 301 executes programs read from storage 303 to memory 302, and references data read from storage 303 to memory 302. In one aspect, storage 303 can be realized by a hard disk drive (HDD), a solid state drive (SSD), an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM), a flash memory, or the like.

[0053] The storage 303 stores one or more objects 310. The object 310 is data for reproducing a virtual line. For example, the object 310 is 3D data created by CAD (Computer Aided Design) software or the like.

[0054] The storage 303 stores one or more variables 311. At least a portion of the one or more variables 311 is associated with any of the one or more objects 310. Each of the one or more variables 311 is identical to each of the one or more variables 332 in the control program 331 executed by the control device 110. Furthermore, each of the one or more variables 311 may store some value. As an example, each of the one or more variables 311 may store a value indicating the position of some object.

[0055] The storage 303 stores the visualizer 312. The visualizer 312 is a program for realizing a digital twin. The visualizer 312 reproduces a line in a virtual space based on one or more objects 310. The visualizer 312 also moves each object constituting the line in the virtual space based on each value of one or more variables 311. Each value of the one or more variables 311 is updated every time the device 100 receives a value of one or more variables 332 from the control device 110. That is, the current value of each of the one or more variables 311 is overwritten with the newly received value of each of the one or more variables 332.

[0056] The external device IF 304 can be connected to any external device such as a printer, a scanner, an external HDD, etc. In one aspect, the external device IF 304 can be realized by a USB (Universal Serial Bus) terminal or the like.

[0057] The input IF 305 may be connected to any input device such as a keyboard, a mouse, a touch pad, a game pad, etc. In one aspect, the input IF 305 may be realized by a USB terminal, a PS / 2 terminal, a Bluetooth (registered trademark) module, or the like.

[0058] The output IF 306 can be connected to any output device such as a cathode ray tube display, a liquid crystal display, an organic EL (Electro-Luminescence) display, etc. In one aspect, the output IF 306 can be realized by a USB terminal, a D-sub terminal, a DVI (Digital Visual Interface) terminal, an HDMI (High-Definition Multimedia Interface) terminal, a DisplayPort terminal, etc.

[0059] The communication IF 307 is connected to other devices via a wired network or a wireless network. In one aspect, the communication IF 307 can be implemented by a wired LAN (Local Area Network) port, a Wi-Fi (registered trademark) (Wireless Fidelity) module, etc. In other aspects, the communication IF 307 can transmit and receive data using communication protocols such as TCP / IP (Transmission Control Protocol / Internet Protocol) and UDP (User Datagram Protocol). Furthermore, in other aspects, the communication IF 307 can be implemented by a communication module compliant with any other communication protocol.

[0060] The device 100 communicates with the control device 110 via the communication IF 307. More specifically, the device 100 transmits a variable request to the control device 110 via the communication IF 307. In response to the variable request, the control device 110 transmits the value of the variable requested by the device 100. In one aspect, the control device 110 may transmit the value of one or more variables requested by the device 100 only once in response to the variable request. In other aspects, the control device 110 may transmit the value of one or more variables requested by the device 100 periodically in response to the variable request. The device 100 receives the variable value from the control device 110 via the communication IF 307.

[0061] The control device 110 controls the line by executing the control unit program 331. Also, the control device 110 stores one or more variables 332 referred to and / or used within the control program 331 in the memory within the control device 110. The variable 332 may include information obtained from the sensors 340 and the cameras 350 on the line.

[0062] <Relationship between Objects and Variables>

[0063] Fig. 4 is a diagram showing an example of a relationship between objects and variables. An object constituting a virtual line may be linked to one or more variables. An object 410 shown in Fig. 4 is a machine equipped with a three-axis drive mechanism. The object 410 is equipped with a three-axis drive mechanism of an X stage, a Y stage, and a Z stage. An object equipped with moving parts such as the object 410 may be linked to a variable indicating the position of each moving part.

[0064] The screen 420 is a screen for displaying one or more variables associated with any object. The visualizer 312 provides the screen 420 to the user. According to the screen 420, the variables 430 include a variable corresponding to the X stage, a variable corresponding to the Y stage, and a variable corresponding to the Z stage. For example, suppose that there is a real line L including a configuration M corresponding to the object 410. Then, suppose that the device 100 constructs a virtual line VL corresponding to the real line L in the virtual space. In this case, the device 100 periodically receives the value of the variable 430 from the control device 110. Then, the device 100 operates the object 410 on the virtual line VL based on the received value of the variable 430 (the position of the X, Y, and Z stages). In this way, the device 100 can reproduce the operation of the configuration M in the virtual space.

[0065] FIG. 5 is a diagram showing an example of an object configuration. An object may be a collection of one or more objects. For example, an object 500 shown in FIG. 5 is an assembly made up of objects 510, 520, 530, and 540, which are parts. Similarly, the object 410 described in FIG. 4 can also be said to be an assembly made up of objects at least on the X, Y, and Z stages.

[0066] In a certain situation, a variable may be associated with each object constituting the assembly. For example, a variable may be associated with each of the objects 510, 520, 530, 540. In other situations, a variable may be associated with an assembly consisting of a plurality of objects. For example, variables corresponding to each of the objects 510, 520, 530, 540 may be associated with the object 500 which is an assembly.

[0067] FIG. 6 is a diagram showing an example of a setting screen for parameters of an object. The visualizer 312 provides a screen 600 for associating an object with a variable. As an example, the user can define the movement of the object on the screen 600. Also, the user can define the relationship between the object and the variable on the screen 600.

[0068] In the example of FIG. 6, the user defines the operation of a machine moving in the X direction. Also, the user associates the variable "MC_Axis009" with the machine moving in the X direction. Similarly, the user defines the operation of a machine moving in the Y direction. Also, the user associates the variable "MC_Axis010" with the machine moving in the Y direction. Further, the user defines the operation of a rotating arm. Also, the user associates the variable "MC_Axis011" with the rotating arm.

[0069] The screen 610 is a screen for displaying variables associated with an object via the screen 600. The visualizer 312 provides the screen 610 to the user. In the example of FIG. 6, the variable 620 includes "MC_Axis009, MC_Axis010, MC_Axis011". In a certain situation, each variable may be associated with an object which is an assembly via the screen 600. In other situations, each variable may be associated with an individual object which is a component constituting the assembly via the screen 600.

[0070] <D. Method for Selecting Variables Required by the Control Device>

[0071] Next, a procedure for reducing the number of variables that device 100 requests control device 110 will be described with reference to Fig. 7 to Fig. 10. Device 100 reduces the number of variables that it requests control device 110 by using at least one of the methods shown in Fig. 7 to Fig. 10.

[0072] FIG. 7 is a diagram showing a first example of a method for selecting a variable to be requested from the control device 110. The device 100 selects a variable to be transmitted to the control device 110 according to a predetermined procedure as follows. As an example, the device 100 determines whether or not each object constituting a virtual line is linked to a variable. The device 100 may select a variable linked to any object constituting the virtual line as a variable to be transmitted to the control device 110. In a certain aspect, the device 100 may select all objects linked to variables from the virtual line. In this case, the device 100 may select a variable linked to the selected object as a variable to be transmitted to the control device 110. Some of the variables referenced by the control program may not be linked to an object. The device 100 may select only variables linked to objects from among all the variables.

[0073] 7 shows a machine object 710 equipped with a three-axis drive mechanism and a base object 720. The machine object 710, which physically moves, is linked to variables indicating at least the positions of the X, Y, and Z stages. Conversely, no variables are linked to the base object 720, which does not move. In this case, the device 100 may select one or more variables linked to the object 710 as variables to be sent to the control device 110.

[0074] Furthermore, the device 100 may select variables to be transmitted to the control device 110 based on preset selection criteria in addition to the above method. For example, the device 100 may exclude variables associated with objects that do not involve physical operations, such as an HMI, from variables to be transmitted to the control device 110. The visualizer 312 may provide the user with a screen for setting definitions of variables to be excluded from variables to be transmitted to the control device 110. The user may create preset selection criteria via the screen. The device 100 transmits a request for variables selected based on the preset selection criteria to the control device 110. The device 100 operates each object on the virtual line based on the value of the variable included in the response to the request.

[0075] That is, the device 100 includes a communication IF 307 (communication unit) that communicates with the control device 110 configured to control the line by executing a control program. The device 100 also includes a storage 303 (storage unit) that stores data of a plurality of objects for reproducing the line in a virtual space. The device 100 also includes a processor 301 (control unit) that reproduces the operation of the line in the virtual space. At least a portion of the plurality of objects is associated with one or more variables included in the control program. For example, if the line includes objects 710 and 720, the object 710 is linked to one or more variables (variables indicating the positions of the X, Y, and Z stages). The processor 301 (control unit) generates a request including information of one or more variables associated with the selected one or more objects based on the selection of one or more objects associated with the one or more variables from among the plurality of objects. The communication IF 307 (communication unit) transmits the request to the control device 110. The communication IF 307 (communication unit) receives values ​​of one or more variables from the control device 110. The processor 301 (control unit) operates one or more objects in the virtual space based on the values ​​of the one or more received variables. The selection of one or more objects associated with the one or more variables may include the method described with reference to FIG. 7 as well as the method described with reference to FIGS. 8 to 10 described later. Furthermore, the output IF 306 and / or the communication IF 307 (output unit) may output an image of the virtual space. The output IF 306 displays the image on a display. The output IF 306 may include a display or may be connected to a display. The communication IF 307 transmits the image of the virtual space to another device.

[0076] FIG. 8 is a diagram showing a second example of a method for selecting a variable to be requested from the control device 110. The device 100 (visualizer 312) presents a virtual line to a user using a virtual camera. More specifically, a virtual camera is placed in a virtual space. A part of the virtual space is photographed by the virtual camera. The device 100 outputs a video or image photographed by the virtual camera. In one aspect, the device 100 (visualizer 312) may display the video or image photographed by the virtual camera on a display of the device 100. In another aspect, the device 100 (visualizer 312) may transmit the video or image photographed by the virtual camera to another device. Furthermore, in another aspect, the device 100 (visualizer 312) may display the video or image photographed by the virtual camera on a display and transmit it to another device.

[0077] For the user, it is not necessary for the entire line in the virtual space to move, but it is sufficient to be able to confirm the movement of the line within the range captured by the virtual camera. Therefore, the device 100 may select a variable associated with an object within the range captured by the virtual camera as a variable to be transmitted to the control device 110. In the example of FIG. 8, a virtual line 800 exists in the virtual space. Also, a part of the virtual line 800 is captured in a range 810 captured by the virtual camera. In this case, the device 100 selects an object included in the range 810. Then, the device 100 extracts a variable associated with an object included in the range 810, and selects the extracted variable as a variable to be transmitted to the control device 110.

[0078] The virtual camera is configured to be movable within the virtual space. In one aspect, the virtual camera may be configured to be freely movable within the virtual space. In another aspect, the virtual camera may be configured to be settable to any one of several predetermined installation positions (shooting positions). In either case, device 100 may reselect the variables to be transmitted to control device 110 based on detection of a change in the position of the virtual camera.

[0079] As described above, the processor 301 (controller) of the device 100 may set up a virtual camera in the virtual space. The processor 301 (controller) also detects one or more objects included in the shooting range of the virtual camera. Furthermore, generating a request to be transmitted to the control device 110 includes generating a request including information of one or more variables associated with one or more objects included in the shooting range of the virtual camera. In the example of FIG. 8, the device 100 generates a request including information of one or more variables associated with one or more objects included in the shooting range of the range 810.

[0080] Also, the virtual camera is configured to be able to dynamically switch its position. That is, the processor 301 (controller) can receive an input operation to move the virtual camera. Then, the processor 301 (controller) dynamically moves the virtual camera based on the operation input. The processor 301 (controller) dynamically redetects one or more objects included in the shooting range of the virtual camera based on the movement of the virtual camera. The processor 301 (controller) changes the information of the variables to be included in the request based on the result of the redetection. In the example of FIG. 8, when the position of the virtual camera is changed, the objects reflected in the range 810 also change. Therefore, the device 100 regenerates a request including information of one or more variables associated with one or more objects included in the shooting range of the range 810 based on the change in the position of the virtual camera.

[0081] Furthermore, the position of the virtual camera is configured to be changeable to any one of several predetermined installation positions. That is, the processor 301 (control unit) may receive a setting operation of one or more installation positions of the virtual camera. The processor 301 (control unit) detects one or more objects included in the shooting range of the virtual camera at each of the one or more installation positions based on the setting operation. The processor 301 (control unit) creates a list including one or more variables associated with one or more objects included in the shooting range of the virtual camera for each of the one or more installation positions. For example, it is assumed that three installation positions (shooting positions) are set. In this case, the processor 301 (control unit) creates a list including one or more variables associated with one or more objects included in the shooting range of the virtual camera for each installation position. That is, three lists are created. Based on receiving a selection operation of the installation position of the virtual camera, the processor 301 (control unit) changes the information of the variable included in the request to the information of the variable included in the list corresponding to the selected installation position. For example, it is assumed that the position of the virtual camera is changed from the first installation position to the second installation position. In this case, the processor 301 (control unit) changes the information included in the request from the information of the variable corresponding to the first installation position to the information of the variable corresponding to the second installation position.

[0082] FIG. 9 is a diagram showing a third example of a method for selecting a variable to be requested of the control device 110. The visualizer 312 has a function and a screen for setting each object in the virtual space to be displayed or hidden. An object set to be displayed is displayed on the screen viewed by the user. Conversely, an object set to be hidden is not displayed on the screen viewed by the user. The screen 920 is a screen for switching between displaying and hiding an object. The setting 930A is a setting for displaying an object on the screen. The setting 930B is a setting for not displaying an object on the screen. The screens 900A and 900B are screens for displaying the shooting range of the virtual camera, and display a part of the virtual line. The screen 900A is a screen corresponding to the setting 930A (display setting). Moreover, the screen 900B is a screen corresponding to the setting 930B (hidden setting). The object 910A is displayed on the screen 900A. Conversely, the object 910A is not displayed on the screen 900B.

[0083] For the user, it is sufficient to be able to confirm the operation of the object set to display. Therefore, the processor 301 (control unit) excludes information on variables associated with the object set to hidden from the request. In the example of FIG. 9, when the setting 930B is set in the visualizer 312, the object 910A is hidden on the screen. In this case, the processor 301 (control unit) excludes information on variables associated with the object set to hidden 910A from the request to be sent to the control device 110.

[0084] FIG. 10 is a diagram showing a fourth example of a method for selecting a variable to be requested of the control device 110. The visualizer 312 has a function and a screen for setting transparency for each object in the virtual space. An object with a transparent setting is displayed in a transparent state (semi-transparent) on the screen viewed by the user. The screen 1020 is a screen for switching the transparency setting of an object. The setting 1030A is a setting for not setting the object to a transparent state (semi-transparent). The setting 1030B is a setting for setting the object to a transparent state (semi-transparent). The screens 1000A and 1000B are screens for displaying the shooting range of the virtual camera, and display a part of the virtual line. The screen 1000A is a screen corresponding to the setting 1030A (non-transparent setting). The screen 1000B is a screen corresponding to the setting 1030B (transparent setting). The screen 1000A shows an object 1010A that is not in a transparent state. The screen 1000B shows an object 1010B in a transparent state.

[0085] The transparent setting is a setting for making an object hidden behind another object visible. Therefore, the movement of the transparent object is likely not important to the user. Therefore, the processor 301 (control unit) excludes information of variables associated with the transparent object from the request. In the example of FIG. 10, when the setting 1030B is set in the visualizer 312, the transparent object 1010B is displayed on the screen. In this case, the processor 301 (control unit) excludes information of variables associated with the transparent object 1010B from the request to be sent to the control device 110.

[0086] In one aspect, the device 100 may select variables to be included in a request to be sent to the control device 110 using one of the variable selection methods described with reference to Figures 7 to 10. In another aspect, the device 100 may select variables to be included in a request to be sent to the control device 110 by combining two or more of the variable selection methods described with reference to Figures 7 to 10.

[0087] As an example, the device 100 may use only the procedure of Fig. 7. In this case, the device 100 may select only variables associated with any object as variables to be included in the request. Also, the device 100 may select only variables that match any predetermined setting (such as being associated with an object that performs a physical action) as variables to be included in the request.

[0088] As another example, the apparatus 100 may combine the procedures of Figures 7 and 8. In this case, the apparatus 100 may select only variables associated with objects within the field of view of the virtual camera as variables to include in the request.

[0089] As another example, the apparatus 100 may combine the procedures of Fig. 7, Fig. 8, and Fig. 9. In this case, the apparatus 100 may select only variables associated with objects within the shooting range of the virtual camera as variables to be included in the request. Furthermore, the apparatus 100 may exclude variables associated with objects set to hidden from the variables to be included in the request.

[0090] As another example, the apparatus 100 may combine the procedures of Fig. 7, Fig. 8, and Fig. 10. In this case, the apparatus 100 may select only variables associated with objects within the shooting range of the virtual camera as variables to be included in the request. Furthermore, the apparatus 100 may exclude variables associated with objects set to transparent from variables to be included in the request.

[0091] As another example, the device 100 may combine all of the procedures in Fig. 7 to Fig. 10. In this case, the device 100 may select only variables associated with objects within the shooting range of the virtual camera as variables to be included in the request. Furthermore, the device 100 may exclude variables associated with objects set to hidden or transparent from variables to be included in the request.

[0092] Note that when none of the variables are selected, the apparatus 100 may not generate a request to be sent to the control apparatus 110. In this case, the apparatus 100 may not send a request to the control apparatus 110. The case where none of the variables are selected is, for example, a case where there is no object physically operating within the shooting range of the virtual camera. That is, the processor 301 (control unit) may not generate a request based on the fact that none of the one or more objects associated with one or more variables are selected from among the plurality of objects.

[0093] <E. Flow of processing executed by apparatus 100>

[0094] FIG. 11 is a diagram showing an example of a procedure for the apparatus 100 to select variables to be requested from the control apparatus 110. In a certain aspect, the processor 301 may read a program for performing the processing of FIG. 11 from the storage 303 into the memory 302 and execute the program. In other aspects, part or all of the processing may also be realized as a combination of circuit elements configured to execute the processing.

[0095] In step S1110, the processor 301 extracts an object that uses a variable from the virtual line. The processing of this step corresponds to the processing of FIG. 7. For example, assume that the base 1101 is included in the virtual line. No variable is associated with the base 1101. Therefore, the object (base 1101) is excluded from the extraction target. In a certain aspect, the processor 301 may exclude a specific object from the extraction target based on a predetermined setting. For example, the user may include an object that does not require operation confirmation in the exclusion list. In this case, the processor 301 may exclude the object included in the exclusion list from the extraction target.

[0096] In step S1120, the processor 301 extracts objects within the 3D display screen 1102. The 3D display screen 1102 is the range captured by the virtual camera. That is, the processor 301 extracts objects within the range captured by the virtual camera. The processing in this step corresponds to the processing in FIG. 8. The processor 301 extracts only objects within the 3D display screen 1102 from the objects extracted in step S1110. In other words, the processor 301 excludes objects outside the 3D display screen 1102 from the objects to be extracted.

[0097] In step S1130, processor 301 checks the settings of each object. The settings of each object are, for example, display settings and transparency settings. Processor 301 executes the processes from steps S1130 to S1170 for each individual object. For example, assume that 100 objects are extracted in steps S1110 and S1120. In this case, processor 301 repeats the processes from steps S1130 to S1170 100 times.

[0098] In step S1140, processor 301 determines whether the object is set to be displayed. If processor 301 determines that the object is set to be displayed (YES in step S1140), processor 301 transfers control to step S1150. If not (if the object is set to be hidden) (NO in step S1140), processor 301 transfers control to step S1170. That is, processor 301 excludes hidden objects from the objects to be extracted. The process of step S1140 corresponds to the process of FIG. 9. For example, suppose that object 1103 is set to be hidden. In this case, processor 301 excludes object 1103 from the objects to be extracted.

[0099] In step S1150, processor 301 determines whether the object is set to non-transparent. If processor 301 determines that the object is set to non-transparent (YES in step S1150), processor 301 transfers control to step S1160. If not (transparent) (NO in step S1150), processor 301 transfers control to step S1170. That is, processor 301 excludes transparent objects from the extraction target. The process of step S1150 corresponds to the process of FIG. 10. For example, assume that object 1104 is set to transparent. In this case, processor 301 excludes object 1104 from the extraction target.

[0100] In step S1160, processor 301 stores variables associated with each of the one or more extracted objects in the acquired variable list. Processor 301 stores objects with display settings and non-transparent settings in the acquired variable list. In step S1170, processor 301 determines whether or not there is an object whose setting has not been confirmed. If processor 301 determines that there is an object whose setting has not been confirmed (YES in step S1170), it transfers control to step S1130. If not (NO in step S1170), processor 301 transfers control to step S1180. In step S1180, processor 301 confirms the acquired variable list. The acquired variable list is a list including variables requested of control device 110.

[0101] In one aspect, the processor 301 may execute any one of the processes of S1110, S1120, S1140, and S1150. In another aspect, the processor 301 may execute an appropriate combination of the processes of S1110, S1120, S1140, and S1150. As an example, the processor 301 may execute only step S1110. In this case, the acquired variable list includes variables linked to objects constituting the virtual line. Alternatively, the acquired variable list may include only variables related to the operation of each object based on a predetermined setting. As another example, the processor 301 may execute only steps S1110 and S1120. In this case, the acquired variable list includes variables linked to objects captured by the virtual camera. As another example, the processor 301 may execute only steps S1110, S1120, and S1140. In this case, the acquired variable list includes variables linked to objects captured by the virtual camera and having a display setting. As another example, the processor 301 may execute only steps S1110, S1120, and S1150. In this case, the acquired variable list includes variables associated with objects that are captured by the virtual camera and are set to non-transparent.

[0102] Fig. 12 is a diagram showing an example of an acquired variable list 1220 selected by the procedure of Fig. 11. A virtual line 1200 is a line constructed in a virtual space. A list 1210 is a list of variables referenced by a control program. At least a part of the variables in the list 1210 is linked to any of the objects constituting the virtual line 1200.

[0103] 11 on the virtual line 1200. For example, the processor 301 extracts objects 1201 and 1202 from the virtual line 1200 according to the procedure of FIG. 11. In this case, the processor 301 extracts only the variable 1212 linked to the object 1201 and the variable 1211 linked to the object 1202 from the list 1210. The processor 301 creates an acquired variable list 1220 including the extracted variables 1211 and 1212.

[0104] Fig. 13 is a diagram showing an example of a procedure in which device 100 communicates with control device 110 to reproduce a line. In one aspect, processor 301 may load a program for performing the processing of Fig. 13 from storage 303 into memory 302 and execute the program. In another aspect, a part or all of the processing may be realized as a combination of circuit elements configured to perform the processing.

[0105] In step S1310, the processor 301 requests the control device 110 for the values ​​of the variables. More specifically, the processor 301 transmits to the control device 110 a request message 1300 including the acquired variable list created in the process of Fig. 11. Note that the information included in the request does not need to be in list format, and may be in any data format. The request transmitted to the control device 110 only needs to include information for identifying each variable included in the acquired variable list.

[0106] In step S1320, the processor 301 obtains the value of each variable from the control device 110. More specifically, the processor receives a response message 1310 from the control device 110. The response message 1310 includes the value of each variable requested by the request message 1300. For example, assume that the request message 1300 includes a request for variables A, B, and C. In this case, the response message 1310 includes the value of variable A, the value of variable B, and the value of variable C. In one aspect, the response message 1310 may include a combination of the variable name and value. In another aspect, the response message 1310 may include only the value of the variable.

[0107] In step S1330, processor 301 calculates the display position of each object based on the received value of each variable. In step S1340, processor 301 updates the display position of each object in the virtual space based on the calculation result in step S1330.

[0108] 14 is a diagram showing an example of a communication procedure between the device 100 and the control device 110. There are mainly two types of communication procedures between the device 100 and the control device 110. The device 100 and the control device 110 can communicate with each other using any of the following communication methods.

[0109] In the first communication procedure, the device 100 notifies the control device 110 of a change only when a requested variable has been changed. For example, the device 100 transmits a variable update request message (Update3DVariables) 1410 to the control device 110. The control device 110 returns an acknowledgement (Updated) 1420 to the device 100. When the device 100 requests a new variable from the control device 110, it transmits a variable request message (Get3DVariables) to the control device 110. After receiving the variable update request message 1410, the control device 110 continues to periodically transmit a message 1440 including the variable value to the device 100.

[0110] In the second communication procedure, the device 100 periodically transmits a variable request message (Get3DVariables) 1450 to the control device 110. In response to the variable request message 1450, the control device 110 replies to the device 100 with a message 1460 including the variable value. That is, every time the control device 110 receives the request message 1450, it replies with the message 1460 including the variable value to the device 100.

[0111] Fig. 15 is a diagram showing an example of a procedure for changing a variable that device 100 requests control device 110 to change. In one aspect, processor 301 may load a program for performing the processing of Fig. 15 from storage 303 into memory 302 and execute the program. In another aspect, a part or all of the processing may be realized as a combination of circuit elements configured to perform the processing.

[0112] The user may change the position of the virtual camera or the display or transparency settings of each object. In such a case, the device 100 updates the acquired variable list. The device 100 also requests the control device 110 to newly include variables in the updated acquired variable list.

[0113] In step S1510, the processor 301 checks whether or not the viewpoint of the 3D screen has been changed (the position of the virtual camera has been changed), and whether or not the display settings and / or transparency settings of each object have been changed.

[0114] In step S1520, the processor 301 updates the acquired variable list based on the change detected in step S1520. The processor 301 may update the acquired variable list by executing the processes described with reference to Figs. 8 to 10. For example, when the processor 301 detects a change in the position of the virtual camera, the processor 301 may update the acquired variable list by executing again the processes described with reference to Fig. 8. Similarly, when the processor 301 detects a change in the display setting, the processor 301 may update the acquired variable list by executing again the processes described with reference to Fig. 9. Moreover, when the processor 301 detects a change in the transparency setting, the processor 301 may update the acquired variable list by executing again the processes described with reference to Fig. 10.

[0115] In step S1530, the processor 301 transmits the updated acquired variable list to the control device 110. More specifically, the processor 301 transmits a request message (Update3DVariables) 1510 including the updated acquired variable list to the control device 110. Note that the request message 1510 does not need to include data in list format. The request message 1510 only needs to include information for identifying each variable that is the subject of the request. The control device 110 returns an acknowledgement (Updated) 1520 to the device 100. Thereafter, the control device 110 periodically transmits the values ​​of each variable included in the updated acquired variable list to the device 100. Note that, if none of the settings have been changed in step S1510, the processor 301 does not need to execute the processes of steps S1520 and S1530.

[0116] 16 is a diagram showing an example of how variables to be included in an acquired variable list are extracted from a list of variables referenced by a control program. In the following example, the device 100 stores a list 1600 of all variables referenced by the control program in the storage 303. The list 1600 also includes variables 1 to 99.

[0117] In step S1610, the apparatus 100 extracts variables associated with objects from the list 1600. The apparatus 100 may also exclude specific variables from the extraction results by referring to the settings 1610. The settings 1610 include predetermined settings and / or settings input by a user. As an example, the apparatus 100 may exclude variables that are not related to the operation of an object from the extraction results. As another example, the apparatus 100 may exclude variables of an object designated by a user from the extraction results. Here, the apparatus 100 creates an extraction list 1620 including variables 1 to 80. The excluded variables 81 to 99 are not associated with any object or are excluded from the extraction list 1620 based on the settings 1610.

[0118] In step S1620, the apparatus 100 analyzes the image captured by the virtual camera and creates a list 1630 of objects shown in the image. Next, the apparatus 100 extracts only the variables associated with the objects included in the list 1630 from the extraction list 1620. The apparatus 100 creates an extraction list 1640 that includes the extracted variables. Here, the apparatus 100 creates an extraction list 1640 that includes variables 1 to 20. The excluded variables 21 to 80 are variables not associated with the objects within the shooting range of the virtual camera.

[0119] In step S1630, the apparatus 100 refers to the settings 1650 of individual objects and extracts the variables associated with the objects of the display settings and the objects of the non-transparency settings (not having the transparency setting) from the extraction list 1640. Next, the apparatus 100 creates an acquired variable list 1660 that includes the extracted variables. Here, the apparatus 100 creates an acquired variable list 1660 that includes variables 1 to 15. The excluded variables 16 to 20 are variables associated with the objects of the non-display settings or the objects of the transparency settings (having the transparency setting).

[0120] In step S1640, the apparatus 100 requests the control device 110 for the values of the variables included in the acquired variable list 1660. After receiving the request, the control device 110 periodically transmits the values of the respective variables to the apparatus 100.

[0121] <F. Modification Example>

[0122] FIG. 17 is a diagram showing an example of a variation of the system configuration according to the present embodiment. The systems 20, 30 further include an HMI 1700 as compared with the system 10. The HMI 1700 may include at least a part of the hardware configuration of the apparatus 100 shown in FIG. 3.

[0123] In the system 20, the device 100 executes the process up to updating the display of the virtual line, and transmits the generated screen to the HMI 1700. More specifically, the device 100 transmits a variable acquisition request message 1710 to the control device 110. The control device 110 periodically transmits a message 1720 including the values ​​of each requested variable to the device 100. The device 100 operates the virtual line according to the received values ​​of each variable. The device 100 also transmits a message 1730 including an image of the virtual line captured by the virtual camera to the HMI 1700. The HMI 1700 displays the received image on the display.

[0124] In the system 30, the device 100 performs operations up to a request for variable values ​​to the control device 110. The HMI 1700 operates the virtual line based on the value of each variable received from the control device 110. More specifically, the device 100 transmits a variable acquisition request message 1710 to the control device 110. The control device 110 periodically transmits a message 1720 including the value of each requested variable to the HMI 1700. The HMI 1700 operates the virtual line according to the received value of each variable. The HMI 1700 also displays an image of the virtual line captured by a virtual camera on the display.

[0125] As described above, the system according to the present embodiment reproduces the operation of a line in a virtual space. The system according to the present embodiment also generates an acquired variable list including variables associated with a selected object. Furthermore, the system according to the present embodiment reduces the amount of communication between the device 100 and the control device 110 by requesting only the values ​​of each variable included in the acquired variable list from the control device 110. In this way, the system according to the present embodiment can suppress the occurrence of communication delays and reproduce and display the operation of a virtual line without delay.

[0126] Also, the system according to this embodiment can select variables to be included in the acquisition variable list based on predetermined settings or based on whether each variable is associated with an object. Further, the system according to this embodiment can select variables to be included in the acquisition variable list based on whether each variable is associated with an object within the shooting range of the camera. Furthermore, the system according to this embodiment can select variables to be included in the acquisition variable list based on display settings and / or transparency settings. In this way, the system according to this embodiment can significantly reduce the number of variables required for the control device 110 by appropriately combining various conditions.

[0127] <Appendix G>

[0128] As described above, this embodiment includes the following disclosures. [Configuration 1] A communication unit (307) that communicates with a control device (110) configured to control a line by executing a control program, A storage unit (303) that stores data of a plurality of objects for reproducing the line in a virtual space, A control unit (301) that reproduces the operation of the line in the virtual space, and At least a part of the plurality of objects is associated with one or more variables included in the control program, Based on the selection of one or more objects associated with the one or more variables from among the plurality of objects, the control unit (301) generates a request including information on the one or more variables associated with the selected one or more objects, The communication unit (307) Sends the request to the control device (110), Receives the values of the one or more variables from the control device (110), and Based on the received values of the one or more variables, the control unit (301) operates the one or more objects in the virtual space, device (100).

[0129] [Configuration 2] The control unit (301) A virtual camera is placed in the virtual space. Detecting one or more objects within a range of the virtual camera; The apparatus (100) of configuration 1, wherein generating the request includes generating the request including information of one or more variables associated with one or more objects included in the field of view of the virtual camera.

[0130] [Configuration 3] The control unit (301) Accepting an input operation to move the virtual camera; dynamically redetecting one or more objects within a field of view of the virtual camera based on the movement of the virtual camera; The apparatus (100) according to configuration 2, further comprising: changing variable information included in the request based on a result of the redetection.

[0131] [Configuration 4] The control unit (301) Accepting a setting operation for one or more installation positions of the virtual camera, detecting one or more objects included in a shooting range of the virtual camera at each of the one or more installation positions based on the setting operation; creating a list including one or more variables corresponding to one or more objects included in a shooting range of the virtual camera for each of the one or more installation positions; The device (100) according to configuration 2, wherein, based on receiving a selection operation of an installation position of the virtual camera, information of a variable to be included in the request is changed to information of a variable included in the list corresponding to the selected installation position.

[0132] [Configuration 5] The device (100) according to any one of configurations 1 to 4, wherein the control unit (301) does not generate the request based on the fact that one or more objects associated with the one or more variables have not been selected from the plurality of objects.

[0133] [Configuration 6] generating a request including information of the one or more variables associated with the one or more selected objects; omitting information about the variables associated with the hidden object from the request; and The apparatus (100) according to any one of configurations 1 to 4, further comprising: excluding, from the request, information of the variable associated with the object with a transparent setting.

[0134] [Configuration 7] A method for acquiring data for reproducing a line in a real space in a virtual space from a control device (110) of the line, comprising: generating a request including information of the one or more variables associated with the one or more selected objects based on the one or more objects being selected from among a plurality of objects for reproducing the line in the virtual space; Sending the request to the control device (110); receiving values ​​of the one or more variables from the control device (110); and manipulating the one or more objects in the virtual space based on the received values ​​of the one or more variables.

[0135] [Configuration 8] A program executed by a computer, the program comprising: generating a request including information of the one or more variables associated with the one or more selected objects based on the one or more objects being selected from among a plurality of objects for reproducing a line in a virtual space; Sending the request to a control device (110); receiving values ​​of the one or more variables from the control device (110); and operating the one or more objects in the virtual space based on the received values ​​of the one or more variables.

[0136] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present disclosure is indicated by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. In addition, the disclosure contents described in the embodiments and each modified example are intended to be implemented, as far as possible, either alone or in combination. [Explanation of symbols]

[0137] 10,20,30 system,100 device,110 control device,120,200,800,1200,VL virtual line,130 line,210 variable list,301 processor,302 memory,303 storage,304 external device IF,305 input IF,306 output IF,307 communication IF,308 bus,310,410,500,510,520,530,540,710,720,910A,1010A,1010B,1103,1104,1201,1202 object,312 visualizer,331 control unit program,340 sensor,350 Camera, 420, 600, 610, 900A, 900B, 920, 1000A, 1000B, 1020 screen, 810 range, 930A, 930B, 1030A, 1030B, 1610, 1650 settings, 1101 base, 1102 display screen, 1210, 1600, 1630 list, 1220, 1660 acquisition variable list, 1300, 1450, 1510 request message, 1310 response message, 1410 update request message, 1440, 1460, 1720, 1730 message, 1620, 1640 extraction list, 1710 acquisition request message.

Claims

1. a communication unit for communicating with a control device configured to control the line by executing a control program; a storage unit for storing data of a plurality of objects for reproducing the line in a virtual space; a control unit that reproduces the operation of the line in the virtual space, At least some of the objects are associated with one or more variables included in the control program; the control unit generates a request including information of the one or more variables associated with the one or more selected objects based on the one or more objects associated with the one or more variables being selected from the plurality of objects; The communication unit is Sending the request to the control device; receiving values ​​of the one or more variables from the controller; The control unit operates the one or more objects in the virtual space based on the received values ​​of the one or more variables.

2. The control unit is A virtual camera is installed in the virtual space, Detecting one or more objects included in a field of view of the virtual camera; The apparatus of claim 1 , wherein generating the request comprises generating the request including information of one or more variables associated with one or more objects within a field of view of the virtual camera.

3. The control unit is accepting an input operation to move the virtual camera; dynamically redetecting one or more objects included in a field of view of the virtual camera based on the movement of the virtual camera; The apparatus of claim 2 , further comprising: modifying variable information included in the request based on the results of the redetection.

4. The control unit is Accepting a setting operation for one or more installation positions of the virtual camera; detecting one or more objects included in a shooting range of the virtual camera at each of the one or more installation positions based on the setting operation; creating a list including one or more variables associated with one or more objects included in a shooting range of the virtual camera for each of the one or more installation positions; The device according to claim 2 , wherein, based on receipt of a selection operation of an installation position of the virtual camera, information of a variable to be included in the request is changed to information of a variable included in the list corresponding to the selected installation position.

5. The apparatus according to any one of claims 1 to 4, wherein the control unit does not generate the request based on one or more objects associated with the one or more variables not being selected from the plurality of objects.

6. Generating a request including information of the one or more variables associated with the one or more selected objects includes: excluding information about the variables associated with hidden objects from the request; and excluding from said request information of said variables associated with an object having a transparent setting.

7. A method for acquiring data for reproducing a line in a real space in a virtual space from a control device of the line, comprising: generating a request including information of the one or more variables associated with the one or more selected objects based on the one or more objects being selected from among a plurality of objects for reproducing the line in the virtual space; sending the request to the control device; receiving values ​​of the one or more variables from the controller; and manipulating the one or more objects in the virtual space based on the received values ​​of the one or more variables.

8. A program executed by a computer, the program comprising: generating a request including information of the one or more variables associated with the one or more selected objects based on the one or more objects being selected from among a plurality of objects for reproducing a line in a virtual space; sending said request to a control device; receiving values ​​of the one or more variables from the controller; and operating the one or more objects in the virtual space based on the received values ​​of the one or more variables.

Images