Control device, control system, and storage medium

Abstract

Provided is a structure capable of implementing complex control with a simpler structure and capable of making program development easy. A control device for controlling a control target includes a PLC engine that cyclically executes a program including a sequence command, a robot control engine for controlling a robot, an image processing engine that performs image processing on an image from a camera, and a simulation module that simulates at least a part of the control target, the robot, and the camera in accordance with a user setting.

Description

TECHNICAL FIELD

[0001] The present application relates to a control device for controlling a control object, a control system for controlling a control object, and a program for realizing the control device. BACKGROUND

[0002] In the field of FA (Factory Automation), a system is realized that detects an object by a vision sensor and controls a motor or a robot or the like based on a detection result thereof.

[0003] For example, Japanese Patent Application Publication No. 2019-215635 (Patent Literature 1) discloses a control system capable of positioning an object with high precision. The control system includes a motion controller constituted by a PLC (Programmable Logic Controller) or the like and a vision sensor.

[0004] PRIOR ART DOCUMENT

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent Application Publication No. 2019-215635 SUMMARY

[0007] PROBLEMS TO BE SOLVED BY THE INVENTION

[0008] In the above-described prior art document, the motion controller and the vision sensor are independent structures from each other, and exchange data via an arbitrary interface. In a structure in which a plurality of devices exchange data via an interface, transmission delay and the like are relatively large, and it can hinder realization of high-speed control.

[0009] An object of the present application is to provide a structure capable of realizing complex control with a simpler structure and capable of facilitating program development, in view of the above-described prior art structure.

[0010] MEANS FOR SOLVING THE PROBLEMS

[0011] According to a certain embodiment of the present application, a control device for controlling a control object is provided. The control device includes a PLC engine that cyclically executes a program including a sequence command, a robot control engine for controlling a robot, an image processing engine that performs image processing on an image from a camera, and a simulation module that simulates at least a part of the control object, the robot, and the camera in accordance with a user setting.

[0012] According to this structure, even in the absence of the control object, by applying the simulation module, it is possible to confirm and verify the processing of the PLC engine, the robot control engine, and the image processing engine. Thereby, it is possible to facilitate program development by the control device.

[0013] In addition, according to this structure, the control target can be controlled by processing of arbitrary combination of the PLC engine, the robot control engine, and the image processing engine.

[0014] The control device can further include a visualizer that visualizes a state of the control target in a virtual space based on processing results of the PLC engine, the robot control engine, and the image processing engine. According to this structure, even when a part or all of the control target is not present, the state of the control target can be confirmed.

[0015] The simulation module can also be implemented using a result of collision detection in the virtual space visualized by the visualizer. According to this structure, an arbitrary control target can be simulated using a state reproduced in the virtual space.

[0016] The simulation module can also be implemented using a simulator that simulates behavior of a device or equipment included in the control target. According to this structure, by employing the simulator, behavior of an arbitrary device or equipment can be simulated.

[0017] The simulation module can also be implemented using a simulator that simulates physical action of an object included in the control target. According to this structure, by using the simulator, a simulation module reflecting physical action of an object included in the control target can be provided.

[0018] The control device further includes a user interface that arbitrarily selects a device associated with an arbitrary engine among the PLC engine, the robot control engine, and the image processing engine from among candidates including a real device and a virtual device, and a construction unit that constructs a simulation module corresponding to the selected virtual device when the virtual device is selected. According to this structure, a user can arbitrarily select a device associated with the control device.

[0019] The PLC engine can also repeatedly execute a motion control command that controls one or more motors that drive the robot, and the robot control engine can sequentially generate a command for controlling the robot in accordance with a robot program. According to this structure, a robot of another control mode can be controlled in addition to the robot controlled by the robot control engine.

[0020] The control device can further include a virtual machine monitor that manages a common hardware resource. The virtual machine monitor can also provide an execution environment of a real-time OS and a general-purpose OS. The PLC engine can also execute on the real-time OS. The robot control engine and the image processing engine can also execute on the general-purpose OS. According to this structure, a plurality of processes can be combined to be implemented using the common hardware resource.

[0021] According to another embodiment of the present application, there is provided a control system for controlling a control object. The control system includes: a control device including a PLC engine that cyclically executes a program including sequence commands, a robot control engine for controlling a robot, and an image processing engine that performs image processing on an image from a camera; and a support device that constructs a simulation module that simulates at least a part of the control object, the robot, and the camera in accordance with a user setting, and provides the constructed simulation module to the control device.

[0022] According to still another embodiment of the present application, there is provided a program for a control device for controlling a control object. The program causes a computer to function as: a PLC engine that cyclically executes a program including sequence commands; a robot control engine for controlling a robot; an image processing engine that performs image processing on an image from a camera; and a simulation module that constructs a simulation of at least a part of the control object, the robot, and the camera in accordance with a user setting.

[0023] Effects of the Invention

[0024] According to the present application, it is possible to provide a structure that enables complex control with a simpler structure, and that enables easy program development. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a schematic diagram showing an application example of the control device of the present embodiment.

[0026] Figure 2 is a schematic diagram showing a structure example of the control system of the present embodiment.

[0027] Figure 3 is a schematic diagram showing a hardware structure example of the control device that constitutes the control system of the present embodiment.

[0028] Figure 4 is a schematic diagram showing a hardware structure example of the support device that constitutes the control system of the present embodiment.

[0029] Figure 5 is a schematic diagram showing a software structure example implemented by the control device that constitutes the control system of the present embodiment.

[0030] Figure 6 is a schematic diagram showing a main part of the software structure example implemented by the control device that constitutes the control system of the present embodiment.

[0031] Figure 7 is a schematic diagram showing an example of the control object of the control system of the present embodiment.

[0032] Figure 8 is a schematic diagram showingFigure 7 Flowchart of the processing steps in the conveyor system shown.

[0033] Figure 9 is a schematic diagram showing a software configuration example for realizing Figure 7 Schematic diagram showing a software configuration example realized by the control device of the processing in the conveyor system shown.

[0034] Figure 10 is a schematic diagram showing a more detailed configuration of a visualizer installed by the control system of the present embodiment.

[0035] Figure 11 is a schematic diagram showing an example of a reproduced image provided by the visualizer installed by the control system of the present embodiment.

[0036] Figure 12 is a schematic diagram showing a software configuration example for realizing Figure 7 Schematic diagram showing a software configuration example for executing the processing in the control device even in the case where the conveyor system shown does not actually exist.

[0037] Figure 13 is a schematic diagram showing a setting example of a virtual object for collision detection by the visualizer shown. Figure 12

[0038] Figure 14 is a diagram for explaining the case where the user program is executed in the state where the industrial robot is not configured in the conveyor system shown. Figure 7

[0039] Figure 15 is a schematic diagram showing a software configuration example for realizing Figure 14 the state shown.

[0040] Figure 16 is a diagram showing an example of a setting screen involved in the user program provided in the control system of the present embodiment.

[0041] Figure 17 is a flowchart of the outline processing of the user program development processing in the control system of the present embodiment. DETAILED DESCRIPTION

[0042] Embodiments of the present application will be explained in detail with reference to the drawings. Note that the same or corresponding portions in the drawings are denoted by the same reference numerals and will not be explained repeatedly.

[0043] <A. Application Example>

[0044] First, an example of a scenario where the present application is applied will be explained. The control device of the present embodiment can combine various processing logics to control various devices. ​​

[0045] Figure 1 is a schematic diagram showing an application example of the control device 100 of the present embodiment. Referring to Figure 1 , the control device 100 controls an arbitrary control target 4.

[0046] In the present specification, the "control target" is not limited to a device, equipment, etc. for producing certain articles, and also includes an arbitrary device that provides arbitrary information to the control device 100, and an arbitrary device that is an output destination of an instruction from the control device 100. That is, the "control target" can include an arbitrary device related to a control operation in the control device 100.

[0047] As shown in (A) of Figure 1 , the control device 100 includes a PLC engine 136, a robot control engine 1420, and an image processing engine 1422.

[0048] The PLC engine 136 is a module that provides typical functions as a PLC, and cyclically executes a program including sequence commands.

[0049] The robot control engine 1420 is a module that provides functions as a robot controller, and executes processing for controlling a robot.

[0050] The image processing engine 1422 is a module that provides functions as an image processing device, and executes image processing on an image from a camera.

[0051] The PLC engine 136, the robot control engine 1420, and the image processing engine 1422 can exchange data.

[0052] Figure 1 (A) of shows a state in which the control device 100 is controlling a real control target 4. The PLC engine 136, the robot control engine 1420, and the image processing engine 1422 exchange data as needed, and control the control target 4.

[0053] In this way, the control device 100 of the present embodiment has a plurality of control functions, and can realize control with a simpler structure.

[0054] In addition, there can be a state in which a part of the control target 4 cannot be prepared at a stage in the process of constructing a device or a machine controlled by the control device 100. Even in such a case, the control device 100 of the present embodiment can execute processing in the same state as in the case where a real control target exists, by utilizing the simulation module 148.

[0055] In Figure 1A structure example in which a simulation module 148 is installed in place of a part of the control target 4 is shown in (B). The simulation module 148 simulates at least a part of the control target 4, the robot, and the camera. The simulation module 148 is arbitrarily constructed in accordance with a user setting. The object simulated by the simulation module 148 can be arbitrary.

[0056] By arbitrarily installing such a simulation module 148, even if it is not possible to utilize all of the control targets 4, it is possible to confirm the behavior of the program executed by the control device 100 and the like. By such a function, it is possible to make the development of the program executed by the control device 100 easy.

[0057] <B. Structure Example of Control System>

[0058] Figure 2 is a schematic diagram showing a structure example of the control system 1 of the present embodiment. Referring to Figure 2 , a structure example of the control system 1 of the present embodiment will be described.

[0059] The control system 1 is a comprehensive system that controls various devices by one control device. More specifically, the control system 1 includes a control device 100 for realizing comprehensive control. The control device 100 is a device for controlling a control target, and is typically realized using hardware that follows a general architecture (for example, an industrial personal computer).

[0060] The control system 1 also includes one or more devices connected to the control device 100 via a field network 14 that is an industrial network. As an example of a protocol of the field network 14, EtherCAT (registered trademark) can be used.

[0061] Figure 2 The control system 1 shown includes, as examples of devices, a servo driver 500 that constitutes a custom robot 550, a robot controller 600 for controlling an industrial robot 650, a safety controller 700, and an IO unit 800.

[0062] The safety controller 700 is electrically connected to a safety device 750 such as a light curtain, and the IO unit 800 is electrically connected to an arbitrary sensor and / or an arbitrary actuator.

[0063] The control device 100 is connected to a support device 200, a display device 300, and a server device 400 via an upper network 12. The upper network 12 can also realize a topology branched using a network hub 10. As an example of a protocol of the upper network 12, an industrial Ethernet (registered trademark) such as EtherNet / IP can be used.

[0064] The control device 100 is connected to the camera 20 for image processing. As described later, the control device 100 can also have a function of processing an image captured by the camera 20.

[0065] Thus, according to the present embodiment, it is possible to provide a comprehensive control system 1 that controls various devices by one control device 100.

[0066] <C. Hardware configuration example>

[0067] Next, an example of the hardware configuration of each device constituting the control system 1 of the present embodiment will be described.

[0068] (c1: control device 100)

[0069] Figure 3 is a schematic diagram showing an example of the hardware configuration of the control device 100 constituting the control system 1 of the present embodiment. Referring to Figure 3 , the control device 100 includes a processor 102, a main memory 104, an input section 106, a display section 108, a storage 110, communication controllers 112, 114, an optical drive 116, a memory card interface 120, a USB controller 124, and a camera interface 126. These components are connected via a processor bus 128.

[0070] The processor 102 corresponds to an operation processing section that performs control operations, and is constituted by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like. Specifically, the processor 102 reads out a program saved in the storage 110 and expands execution in the main memory 104, thereby realizing control operations corresponding to control objects and various processes described later.

[0071] The main memory 104 is constituted by a volatile storage device such as a DRAM (Dynamic Random Access Memory), an SRAM (Static Random Access Memory), or the like. The storage 110 is constituted by a non-volatile storage device such as an SSD (Solid State Drive), an HDD (Hard Disk Drive), or the like, for example.

[0072] The input section 106 is constituted by a keyboard, a mouse, or the like, and receives user operations.

[0073] The display section 108 is constituted by a display, various indicators, or the like, and outputs processing results from the processor 102.

[0074] The storage 110 holds therein an OS (Operating System) 1102 for implementing basic functions, a PLC engine program 1104, an application engine program 1106, a user program 1108, and a program execution setting 1110. The OS 1102 can include a hypervisor for implementing a virtual environment described later, an RTOS (Real Time Operating System), and a general-purpose OS. The PLC engine program 1104 provides an execution environment equivalent to a PLC. The application engine program 1106 provides an environment for executing an arbitrary application program.

[0075] The OS 1102, the PLC engine program 1104, and the application engine program 1106 are equivalent to main programs for implementing the control device 100.

[0076] The user program 1108 can also include an application program for implementing robot control, processing related to a vision sensor, and a PLC program including sequence commands and / or motion control commands. The PLC program can also be described in a form compliant with IEC61131-3, and thus, the program executed by the PLC engine will also be referred to as an "IEC program" hereinafter. Note that, in this specification, an "IEC program" can also include a program described in a form not compliant with IEC61131-3.

[0077] The program execution setting 1110 includes setting values that specify an environment for executing the user program 1108 and the like. As described later, the program execution setting 1110 specifies activation / deactivation of a simulation module 148 that simulates a part or all of a control object and connection relationships with each engine and the like.

[0078] The communication controllers 112 and 114 exchange data between an arbitrary information processing device and an arbitrary network. In Figure 2 In the control system 1, the communication controller 112 is responsible for data communication via the upper network 12, and the communication controller 114 is responsible for data communication via the field network 14.

[0079] The optical drive 116 is capable of reading arbitrary data from and writing arbitrary data to the storage medium 118 (e.g., an optical storage medium such as a DVD (Digital Versatile Disc)) that non-transitorily stores a computer-readable program.

[0080] The memory card interface 120 accepts a memory card 122 as an example of a detachable storage medium. The memory card interface 120 is capable of reading arbitrary data from and writing arbitrary data to the memory card 122.

[0081] The USB controller 124 exchanges data with any information processing device via a USB connection.

[0082] (c2: support device 200)

[0083] Figure 4 is a schematic diagram showing a hardware configuration example of the support device 200 that constitutes the control system 1 of the present embodiment. Referring to Figure 4 , the support device 200 is implemented using hardware that complies with a general architecture (for example, a general personal computer), for example.

[0084] The support device 200 provides a comprehensive development environment that enables comprehensive setting for each device included in the control system 1 and creation of programs executed by each device. In the comprehensive development environment, setting, program creation, and debugging, and the like for the control device 100, the display device 300, the servo driver 500, the robot controller 600, the safety controller 700, and the like can be performed.

[0085] Referring to Figure 4 , the support device 200 includes a processor 202, a main memory 204, an input section 206, a display section 208, a storage 210, a communication controller 212, an optical drive 216, a USB controller 224. These components are connected via a processor bus 228.

[0086] The processor 202 is constituted by a CPU or a GPU, or the like, reads out a program (OS 2102 and development program 2104, for example) saved in the storage 210, and expands execution in the main memory 204, thereby realizing various processes described later.

[0087] The main memory 204 is constituted by a volatile storage device such as a DRAM, an SRAM, or the like. The storage 210 is constituted by a non-volatile storage device such as an HDD, an SSD, or the like, for example.

[0088] In the storage 210, an OS 2102 for realizing a basic function, a development program 2104 for realizing a comprehensive development environment, and the like are saved.

[0089] The development program 2104 provides a comprehensive development environment by being executed by the processor 202. The development program 2104 includes a build module 2105 that builds a simulation module 148 corresponding to a selected virtual device. The process involved in the building of the simulation module 148 will be described later.

[0090] The input section 206 is constituted by a keyboard, a mouse, or the like, and accepts user operations.

[0091] The display section 208 is constituted by a display, various indicators, or the like, and outputs a processing result from the processor 202.

[0092] The communication controller 212 exchanges data between the support device 200 and any information processing device via any network. Figure 2 In the control system 1, the communication controller 212 is responsible for data communication via the upper network 12.

[0093] The optical drive 216 is capable of reading any data from a storage medium 218 (e.g., an optical storage medium such as a DVD) that non-transitorily stores a computer-readable program, and writing any data to the storage medium 218.

[0094] The USB controller 224 exchanges data between the support device 200 and any information processing device via a USB connection.

[0095] The programs stored in the storage medium 218 can also be read from the storage medium 218 that non-transitorily stores a computer-readable program and installed in the storage 210 or the like. Alternatively, the various programs executed by the support device 200 can also be installed in the form of being downloaded from a server device or the like on a network. Note that the functions provided by the support device 200 of the present embodiment are sometimes realized in the form of a part of a module provided by an OS.

[0096] Further, the support device 200 can also be detached from the control device 100 during operation of the control system 1.

[0097] (c3: Display device 300)

[0098] The display device 300 that constitutes the control system 1 of the present embodiment is also referred to as an HMI (Human Machine Interface) or a PT (Programmable Terminal), and provides a monitoring operation screen with reference to the information held by the control device 100, and sends an instruction corresponding to a user operation to the control device 100.

[0099] As an example, the display device 300 is realized using hardware that complies with a general architecture (e.g., a general personal computer). The basic hardware structure example is the same as that of the support device 200 shown in Figure 4 , and thus detailed explanation will not be given here.

[0100] (c4: Server device 400)

[0101] The server device 400 that constitutes the control system 1 of the present embodiment functions as a file server, a manufacturing execution system (MES: Manufacturing Execution System), a production management system, or the like, for example.

[0102] As an example, the server device 400 is implemented using hardware that follows a general architecture (e.g., a general personal computer). The basic hardware structure example is the same as that of the support device 200 shown in FIG. 1, and thus detailed description thereof will not be given here. Figure 4 The basic hardware structure example of the server device 400 is the same as that of the support device 200 shown in FIG. 1, and thus detailed description thereof will not be given here.

[0103] (c5: Servo driver 500)

[0104] The servo driver 500 that constitutes the control system 1 of the present embodiment drives the electrically connected servo motor 530. Each servo motor 530 is mechanically coupled to a movement mechanism of the custom robot 550. The servo driver 500 is an example of a motor driver, and a motor driver different from the servo driver 500 can also be used. Similarly, the servo motor 530 is an example of a motor, and a motor different from the servo motor 530 (e.g., an induction motor, a linear motor, etc.) can also be used. As the motor driver, a structure corresponding to the driven motor can be used.

[0105] (c6: Robot controller 600)

[0106] The robot controller 600 that constitutes the control system 1 of the present embodiment drives the industrial robot 650 in accordance with an instruction from the control device 100. The industrial robot 650 can use, for example, any general robot such as a vertical multi-joint robot, a horizontal multi-joint (SCARA) robot, a parallel robot, an orthogonal robot, etc.

[0107] (c7: Safety controller 700)

[0108] The safety controller 700 that constitutes the control system 1 of the present embodiment performs safety control. The safety controller 700 typically includes a processing unit that performs a control operation involved in the safety control, and one or more safety extension units that are electrically connected to the safety device 750.

[0109] (c8: IO unit 800)

[0110] The IO unit 800 that constitutes the control system 1 of the present embodiment receives a signal from any sensor and outputs an instruction to any actuator. That is, the IO unit 800 is electrically connected to any IO device.

[0111] (c9: Other modes)

[0112] The above-described apparatus can adopt an installation form in which the required functions are provided by execution of a program by one or a plurality of processors, or can adopt an installation form in which a dedicated hardware circuit (for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), or the like) is used for a part or all of the required functions.

[0113] <D. Software Configuration Example>

[0114] Next, a software configuration example implemented by the control device 100 that constitutes the control system 1 of the present embodiment will be described.

[0115] Figure 5 is a diagram that shows a software configuration example implemented by the control device 100 that constitutes the control system 1 of the present embodiment. Referring to Figure 5 , the control device 100 provides a virtual environment in which common hardware resources 130 can be shared among a plurality of systems.

[0116] More specifically, in the control device 100, a virtual machine monitor 132 that manages the common hardware resources 130, an RTOS 134, a PLC engine 136, an IEC program 138, a general-purpose OS 140, an application engine 142, and an application program 144 are implemented.

[0117] The virtual machine monitor 132 provides an execution environment for the RTOS 134 and the general-purpose OS 140. The virtual machine monitor 132, the RTOS 134, and the general-purpose OS 140 are implemented by executing the code contained in the OS 1102( Figure 3 ).

[0118] The PLC engine 136 is implemented by executing the code contained in the PLC engine program 1104( Figure 3 ). The PLC engine 136 is executed on the RTOS 134.

[0119] The application engine 142 is implemented by executing the code contained in the application engine program 1106( Figure 3 ). The application engine 142 is executed on the general-purpose OS 140.

[0120] The IEC program 138 and the application program 144 can also be contained in the user program 1108( Figure 3 ).

[0121] The PLC engine 136 of the control device 100 provides an environment capable of cyclically executing the IEC program 138 with a prescribed control cycle. Also, the application engine 142 of the control device 100 provides an environment capable of executing an arbitrary application program 144 executable by a general-purpose OS.

[0122] Thus, in the control device 100, both the functions provided by the PLC and the functions provided by the general-purpose personal computer are realized.

[0123] Figure 6 is a diagram showing the main part of the software structure realized by the control device 100 constituting the control system 1 of the present embodiment. Referring to Figure 6 , the virtual machine monitor 132 of the control device 100 provides a scheduler 1320 and a variable manager 1322. The scheduler 1320 manages the execution timing and the like of each program in the control device 100. The variable manager 1322 holds and updates variables referable by each program executed by the control device 100. The values of the variables managed by the variable manager 1322 are transmitted to the corresponding devices via the communication controller 114, and the values of the variables managed by the variable manager 1322 are updated by the values received from the devices via the communication controller 114.

[0124] The PLC engine 136 refers to the values of the variables managed by the variable manager 1322, and cyclically executes the sequence command 1382 included in the IEC program 138. The values of the variables calculated by the execution of the sequence command 1382 are reflected to the variable manager 1322.

[0125] In addition, the PLC engine 136 refers to the values of the variables managed by the variable manager 1322, and cyclically executes the motion control command 1384 included in the IEC program 138. The values of the variables calculated by the execution of the motion control command 1384 are reflected to the variable manager 1322. The motion control command 1384 typically includes one or a plurality of commands for controlling the servo driver 500 (refer to Figure 2 ) constituting the custom robot 550. The motion control command 1384 controls the driving of one or a plurality of motors that drive the custom robot 550, and is cyclically executed.

[0126] On the other hand, the application engine 142 includes, for example, a robot control engine 1420, an image processing engine 1422, an emulator 1424, a physical simulator 1426, and a visualizer 1428. These programs are appropriately executed depending on the computing resources.

[0127] The robot control engine 1420 interprets the robot program 1442 included in the application program 144, and sequentially generates commands for controlling the industrial robot 650 (refer to Figure 2). The robot program 1442 can also be described in a robot control dedicated programming language such as V+ language, a programming language related to NC control such as G code, and the like. In this way, the robot control engine 1420 generates commands for controlling the industrial robot 650 in accordance with the robot program 1442.

[0128] The image processing engine 1422 performs image processing on an image captured by the camera 20 in accordance with the image processing setting 1444. The result of the image processing performed by the image processing engine 1422 is reflected to the variable manager 1322. The image processing performed by the image processing engine 1422 includes, for example, defect inspection, object detection, object recognition, character recognition, and the like.

[0129] The simulator 1424 simulates behavior of a control target (an apparatus or a device, or the like) controlled by the control device 100. The simulator 1424 refers to the values of the variables managed by the variable manager 1322, and outputs a result in accordance with a predefined model.

[0130] The physics simulator 1426 simulates physical motion of an object included in a control target controlled by the control device 100. The physics simulator 1426 refers to the values of the variables managed by the variable manager 1322, and outputs a result in accordance with a predefined physical model.

[0131] Further, it is also possible not to distinguish the simulator 1424 and the physics simulator 1426.

[0132] The visualizer 1428 visualizes a state of a control target (an apparatus, a device, or the like) controlled by the control device 100 in a virtual space. The visualizer 1428 visualizes the state of the control target in the virtual space based on the processing results of the PLC engine 136, the robot control engine 1420, and the image processing engine 1422. More specifically, the visualizer 1428 refers to the values of the variables managed by the variable manager 1322, and generates data necessary for visualization reflecting the state at each time in accordance with a predefined model and a setting.

[0133] As described later, the control system 1 of the present embodiment is capable of controlling a real control target, and is also capable of performing processing of the entire control system 1 even when a part or all of the control target does not actually exist.

[0134] <E. Processing Example>

[0135] Next, a processing example in the control system 1 of the present embodiment will be described.

[0136] Figure 7 is a schematic view showing an example of a control target of the control system 1 of the present embodiment. Referring to Figure 7As the controlled object of control system 1, imagine a conveyor system 2 in which workpiece 8 is sequentially conveyed by conveyor 6.

[0137] A custom robot 550, positioned upstream of conveyor 6, places workpiece 8 onto conveyor 6. Workpiece 8, transported downstream by conveyor 6, is tracked based on images captured by camera 20. Based on the tracking results, industrial robot 650 picks up workpiece 8. The range within which industrial robot 650 picks up workpiece 8 (the picking range) is predetermined. A photoelectric sensor 30 is installed downstream of the picking range to detect unpicked workpiece 8.

[0138] exist Figure 7 In the conveyor system 2 shown, the conveyor 6 can rotate at any speed. An encoder 32 is installed on the conveyor 6, and the number of pulses output from the encoder 32 (or the rotational speed calculated from the number of pulses) is input to the control device 100. A conveyor coordinate system is defined for the conveyor 6. The position of the workpiece 8 on the conveyor 6 is managed based on the conveyor coordinate system.

[0139] Customized robot 550 moves in sync with the rotational speed of conveyor 6, ensuring no speed difference when placing workpiece 8 on conveyor 6. Similarly, industrial robot 650 moves in sync with the rotational speed of conveyor 6, ensuring no speed difference when picking up workpiece 8 from conveyor 6.

[0140] A camera coordinate system is defined for camera 20. The camera coordinate system is used to manage the position of workpiece 8 appearing in the image captured by camera 20.

[0141] In addition, the custom robot 550 and the industrial robot 650 can each have their own independent coordinate system.

[0142] Define a world coordinate system for the entire conveyor system 2, and prepare transformation matrices in advance for mutual position transformation between the world coordinate system and the conveyor coordinate system, as well as between the world coordinate system and the camera coordinate system.

[0143] Figure 8 It means Figure 7 A flowchart of the processing steps in conveyor system 2 is shown. (Refer to...) Figure 8 ,right Figure 7 The control of the conveyor system 2 shown is typically achieved through processing performed by the PLC engine 136, robot control engine 1420, and image processing engine 1422 of the control device 100, respectively.

[0144] The PLC engine 136 performs input / output refresh processing (step S100). The input / output refresh processing includes processing of outputting the command value calculated in the previous control cycle to the devices, and processing of acquiring input data from the devices. The input data includes the rotation speed of the conveyer 6, the state value (position, speed, acceleration, etc.) of each servo motor 530 that constitutes the custom robot 550, the current position of the industrial robot 650, and the like.

[0145] The PLC engine 136 updates the tracking database that manages the current positions of each workpiece 8 on the conveyer 6, based on the rotation speed of the conveyer 6 or the number of pulses of the encoder 32 (step S102). The tracking database includes the current positions of each workpiece 8 on the conveyer 6 in each control cycle. The PLC engine 136 calculates the movement distance (difference from the previous control cycle) of the workpiece 8 based on the rotation speed of the conveyer 6 and the length of the control cycle, and shifts the current positions of each workpiece 8 included in the tracking database by the calculated movement distance.

[0146] The PLC engine 136 determines whether or not a predetermined time has elapsed since the last transmission of the photograph trigger to the image processing engine 1422 (step S104). If the predetermined time has elapsed since the last transmission of the photograph trigger to the image processing engine 1422 (the case where "Yes" in step S104), the PLC engine 136 transmits the photograph trigger to the image processing engine 1422 (step S106).

[0147] If the predetermined time has not elapsed since the last transmission of the photograph trigger to the image processing engine 1422 (the case where "No" in step S104), the processing of step S106 is skipped.

[0148] The processing of step S104 and step S106 corresponds to processing of deciding the timing at which the camera 20 photographs the photograph range on the conveyer 6. The timing at which photographing is performed is controlled in such a manner that the same workpiece 8 is not repeatedly included as a subject in a plurality of images photographed by the camera 20.

[0149] The PLC engine 136 determines whether or not the position of the workpiece 8 recognized based on the image photographed by the camera 20 is received from the image processing engine 1422 (step S108). If the position of the workpiece 8 recognized based on the image photographed by the camera 20 is received from the image processing engine 1422 (the case where "Yes" in step S108), the PLC engine 136 appends the position of the workpiece 8 received from the image processing engine 1422 to the tracking database (step S110).

[0150] If the position of the workpiece 8 recognized based on the image captured by the camera 20 is not received from the image processing engine 1422 (the case where "No" in step S108), the processing of step S110 is skipped.

[0151] It is determined whether or not a signal indicating that the workpiece 8 is detected by the photoelectric sensor 30 is input (step S112). When the signal indicating that the workpiece 8 is detected by the photoelectric sensor 30 is input (the case where "Yes" in step S112), the PLC engine 136 deletes the position of the corresponding workpiece 8 from the tracking database (step S114).

[0152] If the signal indicating that the workpiece 8 is detected by the photoelectric sensor 30 is not input (the case where "No" in step S112), the processing of step S114 is skipped.

[0153] The PLC engine 136 refers to the tracking database, and determines whether or not a condition for placing the workpiece 8 on the conveyor 6 (placement condition) is satisfied (step S116). If the placement condition is satisfied (the case where "Yes" in step S116), the PLC engine 136 sets the action flag of the custom robot 550 to ON (step S118).

[0154] If the placement condition is not satisfied (the case where "No" in step S116), the processing of step S118 is skipped.

[0155] The PLC engine 136 refers to the tracking database, and determines whether or not a condition for picking up the workpiece 8 from the conveyor 6 (pick-up condition) is satisfied (step S120). If the pick-up condition is satisfied (the case where "Yes" in step S120), the PLC engine 136 sends a control instruction including the position information of the workpiece 8 to be picked up to the robot control engine 1420 (step S122). Then, the PLC engine 136 deletes the position of the workpiece 8 to be picked up from the tracking database (step S124).

[0156] If the pick-up condition is not satisfied (the case where "No" in step S120), the processing of steps S122 and S124 is skipped.

[0157] In addition, the processing of steps S102 to S124 is mainly described in the sequence command 1382 included in the IEC program 138.

[0158] Next, the PLC engine 136 determines whether the action flag of the custom robot 550 is set to on (step S150). If the action flag of the custom robot 550 is set to on (the case where "Yes" in step S150), the PLC engine 136 calculates, based on the state values of the respective servo motors 530, the command values for driving the respective servo motors 530 that constitute the custom robot 550 in accordance with the motion control command 1384 included in the IEC program 138 (step S152).

[0159] The PLC engine 136 determines whether the execution of the motion control command 1384 included in the IEC program 138 has been completed (step S154). If the execution of the motion control command 1384 included in the IEC program 138 is completed (the case where "Yes" in step S154), the PLC engine 136 sets the action flag to off (step S156).

[0160] If the execution of the motion control command 1384 included in the IEC program 138 is not completed (the case where "No" in step S154), the processing of step S156 is skipped.

[0161] If the action flag of the custom robot 550 is not set to on (the case where "No" in step S150), the processing of steps S150 to S156 is skipped.

[0162] In addition, the processing of steps S150 to S156 is mainly described in the motion control command 1384 included in the IEC program 138.

[0163] The image processing engine 1422 determines whether a photographing trigger is received from the PLC engine 136 (step S200). If the photographing trigger is not received from the PLC engine 136 (the case where "No" in step S200), the processing of step S200 is repeated.

[0164] If the photographing trigger is received from the PLC engine 136 (the case where "Yes" in step S200), the image processing engine 1422 acquires an image photographed by the camera 20 (step S202), and recognizes the position of the workpiece 8 included in the acquired image (step S204). The recognized position of the workpiece 8 is a position defined in accordance with the camera coordinate system.

[0165] The image processing engine 1422 transforms the recognized position of the workpiece 8 into a position in the world coordinate system (step S206), and outputs the transformed position of the workpiece 8 to the PLC engine 136 (step S208). Then, the processing of steps S200 and subsequent steps is repeated.

[0166] The robot control engine 1420 judges whether or not a new control instruction is received from the PLC engine 136 (step S300). If a new control instruction is not received from the PLC engine 136 (in the case of "No" in step S300), the process of step S300 is repeated.

[0167] If a new control instruction is received from the PLC engine 136 (in the case of "Yes" in step S300), the robot control engine 1420 interprets the object part in the robot program 1442 in accordance with the received control instruction, and successively starts generating commands for controlling the industrial robot 650 (step S302). Then, the robot control engine 1420 sends the successively generated commands to the robot controller 600 (step S304).

[0168] The robot control engine 1420 acquires the current position of the industrial robot 650 from the robot controller 600 (step S306). The acquired current position of the industrial robot 650 is a position defined in accordance with the camera coordinate system. The robot control engine 1420 converts the acquired current position of the industrial robot 650 into a position in the world coordinate system (step S308), and outputs the converted current position of the industrial robot 650 to the PLC engine 136 (step S310).

[0169] The robot control engine 1420 judges whether or not the generation of the commands for controlling the industrial robot 650 is to be continued (step S312). If the generation of the commands for controlling the industrial robot 650 is to be continued (in the case of "Yes" in step S312), the process from step S304 is repeated.

[0170] If the generation of the commands for controlling the industrial robot 650 is not to be continued (in the case of "No" in step S312), the process from step S300 is repeated.

[0171] Figure 9 indicates a software structure example realized by the control device 100 that controls the process in the conveyor system 2 shown in Fig. 1. Referring to Fig. 10, the PLC engine 136, the robot control engine 1420, and the image processing engine 1422 exchange data with each other, and each engine exchanges data or signals with a control object. Figure 7 Figure 9 indicates a software structure example realized by the control device 100 that controls the process in the conveyor system 2 shown in Fig. 1. Referring to Fig. 10, the PLC engine 136, the robot control engine 1420, and the image processing engine 1422 exchange data with each other, and each engine exchanges data or signals with a control object.

[0172] More specifically, the PLC engine 136 acquires a signal of a rotational speed or the like from the encoder 32 included in the conveyor system 2, and acquires a signal of detecting the workpiece 8 from the photoelectric sensor 30. Also, the PLC engine 136 gives an instruction value to the servo driver 500, and acquires a state value from the servo driver 500. ​

[0173] Further, the PLC engine 136 gives a control instruction to the robot control engine 1420, and acquires the current position of the industrial robot 650 from the robot control engine 1420. In addition, the PLC engine 136 gives a photograph trigger to the image processing engine 1422, and acquires the position of the workpiece 8 detected by the image processing engine 1422 from the image processing engine 1422.

[0174] The robot control engine 1420 gives a command for controlling the industrial robot 650 to the robot controller 600, and acquires the current position of the industrial robot 650 from the robot controller 600.

[0175] The image processing engine 1422 acquires an image photographed by the camera 20. In addition, a photograph instruction can also be given to the camera 20 from the image processing engine 1422.

[0176] As shown in FIG. 17, the PLC engine 136, the robot control engine 1420, and the image processing engine 1422 included in the control device 100 control the conveyor system 2 by exchanging data or signals with the real control objects. Figure 9

[0177] <F. Visualizer>

[0178] Next, the visualizer 1428 installed in the control system 1 of the present embodiment will be described. The visualizer 1428 visualizes the state of the control objects (devices, equipment, etc.) controlled by the control device 100 in a virtual space.

[0179] Figure 10 is a schematic view showing a more detailed structure of the visualizer 1428 installed in the control system 1 of the present embodiment. Referring to Figure 10 , the visualizer 1428 generates a reproduction image 1430 in which the control objects are reproduced in a virtual space, with reference to the values of the variables managed by the variable manager 1322. The reproduction image 1430 can be a dynamic image, or a static image.

[0180] More specifically, the visualizer 1428 includes a conveyor model 14281 simulating the conveyor 6, a custom robot model 14282 simulating the custom robot 550, an industrial robot model 14283 simulating the industrial robot 650, and an object library 14284.

[0181] The object library 14284 includes image data and the like used for synthesizing the reproduction image 1430.

[0182] Figure 11 is a schematic view showing an example of the reproduction image 1430 provided by the visualizer 1428 installed in the control system 1 of the present embodiment. As shown in​Figure 11 The reproduced image 1430 shown is displayed on the display section 108 of the control device 100 or the like.

[0183] The user can perform action confirmation, defect discovery, or the like of the user program 1108 while viewing the displayed reproduced image 1430.

[0184] In the case where the control object actually exists, the reproduced image 1430 provided by the visualizer 1428 directly reflects the behavior of the real control object. On the other hand, even if all or a part of the control object does not actually exist, each engine of the control device 100 can perform processing, and the visualizer 1428 provides the reproduced image 1430 as if the real control object exists based on the processing result thereof.

[0185] In the control system 1 of the present embodiment, even if a part or all of the control object does not exist, the action of the user program 1108 can be confirmed as if the real control object exists. Hereinafter, the function of the user program provided by the control device 100 that can be flexibly developed will be described.

[0186] <G. Environment for Flexible Development>

[0187] In the control system 1 of the present embodiment, a mechanism (simulation module) that freely simulates a part or all of the control object is prepared.

[0188] Figure 12 is a schematic diagram showing a software configuration example for performing processing in the control device 100 even in the case where the conveyor system 2 does not actually exist. Hereinafter, the Figure 7 Figure 9 and Figure 12 will be described while being compared.

[0189] The control object that exchanges data or signals between the control device 100 shown in Figure 9 is simulated by at least one of the simulator 1424, the physical simulator 1426, and the visualizer 1428 installed in the control device 100. That is, the simulation module can also be implemented by at least one of the simulator 1424, the physical simulator 1426, and the visualizer 1428. By such simulation of the control object, the user program 1108 can be executed in the control device 100 as in the case where the control object actually exists.

[0190] (g1: encoder 32)

[0191] ​The encoder 32 of the conveyor system 2 can be simulated by setting the conveyor settings 14241 in the simulator 1424. The simulator 1424 provides the PLC engine 136 with information simulating the rotational speed from the encoder 32 according to the conveyor settings 14241.

[0192] Thus, the simulation module for encoder 32 is implemented using simulator 1424, which simulates the behavior of the devices or equipment contained in the controlled object.

[0193] (g2: Photoelectric sensor 30)

[0194] The photoelectric sensor 30 of the conveyor system 2 can be simulated by setting a collision detection setting 14285 in the visualizer 1428. The visualizer 1428 performs collision detection on virtual objects in virtual space corresponding to the detection range of the photoelectric sensor 30 and the workpiece 8 according to the collision detection setting 14285, thereby providing a signal indicating that workpiece 8 has been detected to the PLC engine 136. Collision detection is a process that determines whether objects collide based on their proximity.

[0195] Figure 13 It indicates that it is used for passing through Figure 12 This diagram illustrates an example of setting up a virtual object for collision detection using the visualizer 1428. (Refer to...) Figure 13 In the virtual space, a virtual object 14286 is set corresponding to the detection range of the photoelectric sensor 30. The position of the workpiece 8 on the conveyor 6 is updated according to the tracking database managed by the PLC engine 136. The visual device 1428 determines whether any workpiece 8 collides with the virtual object 14286. When it is determined that the workpiece 8 collides with the virtual object 14286, a signal indicating that the photoelectric sensor 30 has detected the workpiece 8 is generated.

[0196] Thus, the analog module for the photoelectric sensor 30 utilizes the results of collision detection in a virtual space visualized by the visualizer 1428.

[0197] (g3: Custom Robot 550)

[0198] The custom robot 550 of the conveyor system 2 can be simulated by setting the servo drive model 14242 in the simulator 1424 and the custom robot model 14261 in the physical simulator 1426.

[0199] The servo driver model 14242 contains parameters reflecting response characteristics and the like of the servo driver 500. The simulator 1424 calculates a state value of the servo driver in accordance with the command value from the PLC engine 136 in accordance with the servo driver model 14242, and supplies the calculated state value to the PLC engine 136.

[0200] The custom robot model 14261 contains parameters reflecting kinematics and the like of the servo motor 530 and the custom robot 550. The physics simulator 1426 calculates a state value and a current position and the like of the custom robot model 14261 in accordance with the state value calculated by the simulator 1424 in accordance with the custom robot model 14261.

[0201] In addition, Figure 12 The structure of the custom robot 550 including the servo driver 500 and the servo motor 530 is simulated by the simulator 1424 and the physics simulator 1426 as shown in FIG. 12, but a single simulator 1424 can also be used for simulation.

[0202] Thus, the simulation module for the custom robot 550 is implemented using the simulator 1424 that simulates the behavior of the devices or equipment included in the control object, and using the physics simulator 1426 that simulates the physical movement of the objects included in the control object.

[0203] (g4: industrial robot 650)

[0204] The industrial robot 650 of the conveyor system 2 can be simulated by setting the robot controller model 14243 in the simulator 1424, and setting the industrial robot model 14262 in the physics simulator 1426.

[0205] The robot controller model 14243 contains parameters reflecting response characteristics and the like of the robot controller 600. The simulator 1424 generates an internal command for driving the industrial robot 650 in accordance with the command from the robot control engine 1420 in accordance with the robot controller model 14243. The simulator 1424 supplies the current position and the like of the industrial robot 650 calculated by the physics simulator 1426 to the robot control engine 1420.

[0206] The industrial robot model 14262 contains parameters reflecting response characteristics and kinematics and the like of the industrial robot 650. The physics simulator 1426 calculates the speed, acceleration, current position and the like of the industrial robot 650 in accordance with the internal command calculated by the simulator 1424 in accordance with the industrial robot model 14262.

[0207] In addition, Figure 12The structure example in which the robot controller 600 and the industrial robot 650 are simulated by the simulator 1424 and the physics simulator 1426 is shown in FIG. 14, but a single simulator 1424 can also be used for simulation.

[0208] Thus, the simulation module regarding the industrial robot 650 is implemented using the simulator 1424 that simulates the behavior of the devices or equipment included in the control object, and using the physics simulator 1426 that simulates the physical movement of the objects included in the control object.

[0209] (g5: camera 20)

[0210] The camera 20 of the conveyor system 2 can be simulated by setting a simulation image group 14244 in the simulator 1424. The simulation image group 14244 is constituted by one or more images corresponding to a predetermined placement pattern of the workpieces 8. The simulator 1424 sequentially provides the images included in the simulation image group 14244 to the image processing engine 1422 according to a shooting trigger provided via the image processing engine 1422.

[0211] It is also possible to simulate the state in which the workpieces 8 are sequentially placed on the conveyed state of the conveyor system 2 in the virtual space, and to generate the simulation image group 14244 using a virtual camera configured at a position corresponding to the camera 20 by virtually shooting with the virtual camera.

[0212] Thus, the simulation module regarding the camera 20 is implemented using the simulator 1424 that simulates the behavior of the devices or equipment included in the control object.

[0213] Also, instead of preparing the simulation image group 14244, it is possible to set a virtual camera in the virtual space provided by the visualizer 1428, and to sequentially provide the images generated by virtually shooting with the virtual camera to the image processing engine 1422. That is, instead of using the simulator 1424 and the simulation image group 14244, it is possible to simulate the camera 20 of the conveyor system 2 by setting a virtual camera in the visualizer 1428.

[0214] In the above description, a case in which the user program 1108 is executed in the control device 100 in a state in which all of the control objects are simulated is exemplified, but it is not limited thereto, and it is possible to simulate any part of the control objects.

[0215] Figure 14 is a diagram for explaining a case in which the user program is executed in a state in which the industrial robot 650 is not configured in the conveyor system shown in FIG. 13. Figure 7 (A) of FIG. 14 shows a case in which the user program is executed in a state in which the industrial robot 650 is not configured in the conveyor system shown in FIG. 13. Figure 14 (A) of FIG. 14 shows a case in which the user program is executed in a state in which the industrial robot 650 is not configured in the conveyor system shown in FIG. 13. Figure 7The state where the industrial robot 650 is not configured in the conveyor system 2 shown. In Figure 14 In the state shown in (A) of Figure 14 , by validating the structure that simulates the industrial robot 650 as described above, even if the industrial robot 650 does not actually exist, it is possible to display

[0216] a reproduction image 1430 as shown in (B) of Figure 14 . The user can perform actions such as validating the user program 1108 and detecting defective conditions while observing

[0217] Figure 15 is a schematic diagram showing an example of a software structure for realizing Figure 14 the state shown. In the control device 100 shown in Figure 15 , the simulator 1424 (with the robot controller model 14243 set) that simulates the robot controller 600 and the industrial robot 650 and the physical simulator 1426 (with the industrial robot model 14262 set) are validated. Through such a simulation module (implemented using the simulator 1424 and the physical simulator 1426) that simulates the robot controller 600 and the industrial robot 650, the control device 100 (PLC engine 136, robot control engine 1420, and image processing engine 1422) can execute the user program 1108 in the same way as in the case where the conveyor system 2 actually exists.

[0218] In addition, Figure 14 and Figure 15 the example of the software structure shown is an example, and it is possible to validate any part or all of the simulation modules of the control objects controlled by the simulation control device 100.

[0219] By providing the environment as described above, the user can flexibly develop the user program 1108.

[0220] <H. Settings in the control device 100 / support device 200>

[0221] The user program 1108 executed by the control device 100 and the program execution setting 1110 that defines the environment for executing the user program 1108, etc. are created by the user on the integrated development environment. The integrated development environment can also be implemented by the support device 200 executing the development program 2104 (refer to Figure 4 ). However, it can also be that the control device 100 executes the development program 2104, and thereby the control device 100 provides the integrated development environment.

[0222] In the control device 100 of this embodiment, it is also possible to simulate and perform processing on any part of the controlled object. Program execution setting 1110 (see...) Figure 3 This includes settings for whether to simulate any part of such a controlled object.

[0223] Figure 16 This diagram shows an example of the setting screen of the user program 1108 provided in the control system 1 of this embodiment. Figure 16 In the setup screen 250 shown, the control object device associated with the user program 1108 can be selected and set.

[0224] In addition to physical devices, the controlled objects also include virtual devices (devices that simulate physical devices).

[0225] More specifically, the setup screen 250 includes a device registration field 252 for registering devices associated with the user program 1108, and a device attribute field 254 for registering the attributes of each device.

[0226] When the user clicks on the device registration field 252, a device selection screen 260 is displayed for selecting devices. The device selection screen 260 includes a list of real devices 262 and a list of virtual devices 264.

[0227] Users can freely select any device from the real and virtual devices in the device selection screen 260. The user's selection result is reflected in the program execution settings 1110. Furthermore, the user program 1108 executed by the control device 100 is associated with the selected device.

[0228] Thus, the support device 200 (or control device 100) provides, for example... Figure 16 The setup screen 250 shown is an example of a user interface for arbitrarily selecting a device associated with any engine among the PLC engine 136, robot control engine 1420, and image processing engine 1422 from a pool of candidates including real and virtual devices.

[0229] Figure 17 This is a flowchart illustrating the general process of developing the user program 1108 in the control system 1 of this embodiment. Figure 17 The steps shown are typically implemented by executing the development program 2104 through the processor 202 of the support device 200.

[0230] Reference Figure 17 The support device 200 generates the source code of the user program 1108 according to the user's operation (step S400).

[0231] The support device 200 accepts selection of a device associated with the user program 1108 in accordance with the user's operation (step S402). The support device 200 judges the kind of the selected device (step S404). If the selected device is a real device (in the case of "real device" in step S404), the support device 200 associates specific information such as the network address of the selected device with the user program 1108 (step S406).

[0232] If the selected device is a virtual device (in the case of "virtual device" in step S404), the support device 200 constructs a simulation module corresponding to the selected virtual device (step S408). More specifically, the support device 200 constructs a simulation module that simulates the selected device by reflecting the settings of the selected device on one or more of the simulator 1424, the physical simulator 1426, and the visualizer 1428. Next, the support device 200 associates the simulation module corresponding to the constructed virtual device with the user program 1108 (step S410).

[0233] In this way, the support device 200 (or the control device 100) constructs a simulation module corresponding to a selected virtual device when the virtual device is selected. That is, the support device 200 (or the control device 100) can construct a simulation module that simulates at least a part of the control object, the robot, and the camera in accordance with the user's settings.

[0234] The support device 200 judges whether or not the selection of the device has ended (step S412). If the selection of the device has not ended (in the case of "No" in step S412), the process from step S402 is repeated.

[0235] On the other hand, if the selection of the device has ended (in the case of "Yes" in step S412), the support device 200 generates an execution-form program from the source code of the user program 1108 (step S414), and generates a program execution setting 1110 that reflects the result of the selection of the device (step S416).

[0236] Finally, the support device 200 transmits the generated user program 1108 (execution form) and the program execution setting 1110 to the control device 100 (step S416). Then, the process ends. In this way, the simulation module constructed by the support device 200 can also be provided to the control device 100.

[0237] [I. Modification]

[0238] In the above description, a configuration example in which the control device 100 is separated from the support device 200 is shown, but a part or all of the functions provided by the support device 200 can be incorporated into the control device 100. In this case, the user can develop the user program 1108 by using the comprehensive development environment provided by the control device 100.

[0239]

[0240] The present embodiment as described above includes the following technical idea.

[0241] [Structure 1]

[0242] A control device (100) for controlling a control target (4), wherein the control device (100) has:

[0243] a PLC engine (136) that cyclically executes a program including sequence commands (1382);

[0244] a robot control engine (1420) for controlling a robot (600, 650);

[0245] an image processing engine (1422) that performs image processing on an image from a camera (20); and

[0246] a simulation module (148) that constructs, in accordance with a user setting, a simulation of at least a part of the control target, the robot, and the camera.

[0247] [Structure 2]

[0248] In the control device described in Structure 1,

[0249] the control device further has a visualizer (1428) that visualizes a state of the control target in a virtual space based on processing results of the PLC engine, the robot control engine, and the image processing engine.

[0250] [Structure 3]

[0251] In the control device described in Structure 2,

[0252] the simulation module is implemented using a result of collision detection in the virtual space visualized by the visualizer.

[0253] [Structure 4]

[0254] In the control device described in any one of Structures 1 to 3,

[0255] ​The simulation module is implemented using a simulator (1424) that simulates behavior of a device or equipment included in the control object.

[0256] [Structure 5]

[0257] In the control device described in any one of Structures 1 to 4,

[0258] The simulation module is implemented using a simulator (1426) that simulates physical action of an object included in the control object.

[0259] [Structure 6]

[0260] In the control device described in any one of Structures 1 to 5, the control device further has:

[0261] a user interface (250) that arbitrarily selects a device associated with any of the PLC engine, the robot control engine, and the image processing engine from among candidates including a real device and a virtual device; and

[0262] a construction unit (2105) that, when the virtual device is selected, constructs the simulation module corresponding to the selected virtual device.

[0263] [Structure 7]

[0264] In the control device described in any one of Structures 1 to 6,

[0265] The PLC engine repeatedly executes a motion control command (1384) that controls one or more motors (530) of a robot (550),

[0266] The robot control engine sequentially generates commands for controlling the robot in accordance with a robot program.

[0267] [Structure 8]

[0268] In the control device described in any one of Structures 1 to 7,

[0269] The control device further has a virtual machine monitor (132) that manages a common hardware resource (130),

[0270] The virtual machine monitor provides an execution environment for a real-time OS (134) and a general-purpose OS (140),

[0271] The PLC engine executes on the real-time OS,

[0272] The robot control engine and the image processing engine execute on the general-purpose OS.

[0273] [Structure 9]

[0274] A control system (1) for controlling a control target (4), wherein the control system (1) has:

[0275] A control device (100) including a PLC engine (136) that cyclically executes a program including sequence commands (1382), a robot control engine (1420) for controlling a robot (600, 650), and an image processing engine (1422) that performs image processing on an image from a camera (20); and

[0276] A support device (200) that constructs a simulation module (148) that simulates at least one of the control target, the robot, and the camera in accordance with a user setting, and provides the constructed simulation module to the control device.

[0277] [Structure 10]

[0278] A program (1102, 1104, 1106) for realizing a control device (100) for controlling a control target (4), wherein the program causes a computer (100) to function as:

[0279] A PLC engine (136) that cyclically executes a program including sequence commands (1382);

[0280] A robot control engine (1420) for controlling a robot;

[0281] An image processing engine (1422) that performs image processing on an image from a camera; and

[0282] A simulation module (148) that is constructed in accordance with a user setting, and simulates at least one of the control target, the robot, and the camera.

[0283] [K. Advantages]

[0284] The control system 1 according to the present embodiment can realize complex control with a simpler structure, and can make program development easy.

[0285] It should be understood that the embodiments disclosed this time are illustrative and non-restrictive in all aspects. The scope of the present invention is indicated by the claims, rather than the explanation above, and is intended to include all modifications equivalent in meaning and scope to the claims.

[0286] Label Explanation

[0287] 1: control system; 2: conveyor system; 4: control object; 6: conveyor; 8: workpiece; 10: network hub; 12: upper network; 14: field network; 20: camera; 30: photoelectric sensor; 32: encoder; 100: control device; 102, 202: processor; 104, 204: main memory; 106, 206: input section; 108, 208: display section; 110, 210: storage; 112, 114, 212: communication controller; 116, 216: optical drive section; 118, 218: storage medium; 120: memory card interface; 122: memory card; 124, 224: USB controller; 128, 228: processor bus; 130: hardware resource; 132: virtual machine monitor; 134: RTOS; 136: PLC engine; 138: IEC program; 140: general-purpose OS; 142: application engine; 144: application program; 148: simulation module; 200: support device; 250: setting screen; 252: device registration column; 254: device attribute column; 260: device selection screen; 262: list of real devices; 264: list of virtual devices; 300: display device; 400: server device; 500: servo driver; 530: servo motor; 550: custom robot; 600: robot controller; 650: industrial robot; 700: safety controller; 750: safety device; 800: IO unit; 1102: OS; 1104: PLC engine program; 1106: application engine program; 1108: user program; 1110: program execution setting; 1320: scheduler; 1322: variable manager; 1382: sequence command; 1384: motion control command; 1420: robot control engine; 1422: image processing engine; 1424: simulator; 1426: physics simulator; 1428: visualizer; 1430: reproduced image; 1442: robot program; 1444: image processing setting; 2104: development program; 2105: build module; 14241: conveyor setting; 14242: servo driver model; 14243: robot controller model; 14244: simulation image group; 14261, 14282: custom robot model; 14262, 14283: industrial robot model; 14281: conveyor model; 14284: object library; 14285: collision detection setting; 14286: virtual object.

Claims

1. A control device for controlling a controlled object, wherein, The control device has: A PLC engine executes a program containing a sequence of commands in a loop. Robot control engine, used to control the robot; An image processing engine that performs image processing on images from a camera; The user interface accepts user settings for arbitrarily selecting, from a pool of candidates including real and virtual devices, devices associated with any of the PLC engine, the robot control engine, and the image processing engine. as well as The unit that associates specific information of the selected real-world device with any one of the corresponding PLC engine, robot control engine, and image processing engine according to the user settings; as well as The construction unit constructs a simulation module corresponding to the selected virtual device according to the user settings. The simulation module simulates any one of the controlled object, the robot, and the camera. When the constructed simulation module simulates the controlled object, the associated unit associates the simulation module with the PLC engine. When the constructed simulation module simulates the robot, the associated unit associates the simulation module with the robot control engine. When the constructed simulation module simulates the camera, the associated unit associates the simulation module with the image processing engine.

2. The control device according to claim 1, wherein, The control device also has a visualizer that visualizes the state of the controlled object in virtual space based on the processing results of the PLC engine, the robot control engine, and the image processing engine.

3. The control device according to claim 2, wherein, The simulation module utilizes the results of collision detection in a virtual space visualized by the visualizer.

4. The control device according to any one of claims 1 to 3, wherein, The simulation module is implemented using a simulator that simulates the behavior of the devices or equipment contained in the controlled object.

5. The control device according to any one of claims 1 to 3, wherein, The simulation module is implemented using a simulator that simulates the physical actions of the objects contained within the controlled object.

6. The control device according to any one of claims 1 to 3, wherein, The PLC engine cyclically executes motion control commands to control one or more motors driving the robot. The robot control engine generates commands for controlling the robot sequentially according to the robot program.

7. The control device according to any one of claims 1 to 3, wherein, The control unit also features a virtual machine monitor that manages shared hardware resources. The virtual machine monitor provides execution environments for both real-time and general-purpose OS. The PLC engine executes on the real-time OS. The robot control engine and the image processing engine are executed on the general-purpose OS.

8. A control system for controlling a controlled object, wherein, The control system has the following features: The control unit includes a PLC engine that cyclically executes a program containing sequential commands, a robot control engine for controlling the robot, and an image processing engine that performs image processing on images from a camera; and Support device, which is connected to the control device, The support device includes: The user interface accepts user settings for arbitrarily selecting, from a pool of candidates including real and virtual devices, devices associated with any of the PLC engine, the robot control engine, and the image processing engine. The unit that associates specific information of the selected real-world device with any one of the corresponding PLC engine, robot control engine, and image processing engine according to the user settings; A unit that constructs a simulation module corresponding to the selected virtual device according to the user settings, wherein the simulation module simulates any one of the controlled object, the robot, and the camera; as well as The constructed simulation module is provided to the unit of the control device. When the constructed simulation module simulates the controlled object, the associated unit associates the simulation module with the PLC engine. When the constructed simulation module simulates the robot, the associated unit associates the simulation module with the robot control engine. When the constructed simulation module simulates the camera, the associated unit associates the simulation module with the image processing engine.

9. A storage medium storing a program for implementing a control device for controlling a controlled object, wherein... The program enables the computer to function as a component of: A PLC engine executes a program containing a sequence of commands in a loop. Robot control engine, used to control the robot; An image processing engine that performs image processing on images from a camera; A unit that provides a user interface for accepting user settings, the user settings being used to arbitrarily select from a pool of candidates including real devices and virtual devices, each device associated with any engine among the PLC engine, the robot control engine, and the image processing engine; The unit that associates specific information of the selected real-world device with any one of the corresponding PLC engine, robot control engine, and image processing engine according to the user settings; as well as A unit that constructs a simulation module corresponding to the selected virtual device according to the user settings. The simulation module simulates any one of the controlled object, the robot, and the camera. When the constructed simulation module simulates the controlled object, the associated unit associates the simulation module with the PLC engine. When the constructed simulation module simulates the robot, the associated unit associates the simulation module with the robot control engine. When the constructed simulation module simulates the camera, the associated unit associates the simulation module with the image processing engine.

Images