Systems, methods for manufacturing articles, information processing devices, information processing methods, programs, and recording media

Abstract

To improve the convenience of programming.SOLUTION: A system includes a first processing unit for executing processing on the basis of a first command, a second processing unit for executing processing on the basis of a second command having a specification different from the first command, and an information processing unit for displaying a user interface image that can accept selections of a first module and a second module in a display unit, acquiring the first command on the basis of the first module in accepting the selection of the first module, and acquiring the second command on the basis of the second module in accepting the selection of the second module.SELECTED DRAWING: Figure 1

Description

Technical Field

[0004]

[0001] The present invention relates to information processing technology.

Background Art

[0002] In realizing factory automation (FA), it is necessary to configure a system using a plurality of FA devices such as robots, cameras, image processing devices, and sensor devices. In such a system, the data communication standards and programming languages are different for each FA device. Also, the terminals connected to each FA device are different for each device. Therefore, users such as developers had to acquire different programming languages for each device in order to operate the system.

[0003] On the other hand, Patent Document 1 discloses that different programming languages for each control device that controls FA devices are integrated so that the entire system can be operated unifiedly with one operation terminal.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, even with the method disclosed in Patent Document 1 above, users still had to acquire a programming language such as a ladder program when programming the system, and further convenience was demanded.

[0006] An object of the present invention is to improve the convenience of programming.

Means for Solving the Problems

[0007] This DisclosureFirst form Mr. / Ms. Based on the First Directive Controlling the first controlled object The first processing unit and, based on a second command having specifications different from the first command... Controlling a second control target that is different from the first control target. The second processing unit and Corresponding to the first control target Module 1 and Corresponding to the second control target A user interface image that allows selection of the second module is displayed on the display unit. The first module has an information processing unit, and the first module has a first parameter setting unit where a first parameter is set, a first execution setting unit where a first command is generated from the first parameter and output to the first processing unit, and a first result setting unit where a first execution result is obtained when control of the first controlled object based on the first command is executed. The second module has a second parameter setting unit where a second parameter is set, a second execution setting unit where a second command is generated from the second parameter and output to the second processing unit, and a second result setting unit where a second execution result is obtained when control of the second controlled object based on the second command is executed. The user interface image is from the user. The first module Or run either of the config2 modules. Accepting your selection It is possible, and the user interface image is a system capable of accepting input of parameters set in the first parameter setting unit or the second parameter setting unit. .

[0008] Book Disclosure Second form Mr. / Ms. , Corresponding to the first controlled object Module 1 、 and Corresponding to a second control target different from the first control target. A user interface image that allows selection of the second module is displayed on the display unit. do Information processing The device comprises a first module having a first parameter setting unit for setting a first parameter, a first execution setting unit for generating a first command from the first parameter and outputting it to a first processing unit, and a first result setting unit for acquiring a first execution result obtained when control of the first controlled object based on the first command is executed; the second module having a second parameter setting unit for setting a second parameter, a second execution setting unit for generating a second command from the second parameter and outputting it to a second processing unit, and a second result setting unit for acquiring a second execution result obtained when control of the second controlled object based on the second command is executed; the user interface image is capable of accepting a user's selection to execute either the first module or the second module; and the user interface image is capable of accepting input of parameters to be set in the first parameter setting unit or the second parameter setting unit.

[0009] Book Disclosure Third aspect teeth , Corresponding to the first controlled object Module 1 、 and Corresponding to a second control target different from the first control target. A user interface image that allows selection of a second module is displayed on the display unit. When the selection of the first module is accepted, a first command is obtained to be executed by the first processing unit based on the first module. When the selection of the second module is accepted, a second command with specifications different from the first command is obtained to be executed by the second processing unit based on the second module. death , The user interface image is an information processing method that accepts input for selecting the execution of the first module or the second module, and for setting parameters in the first module or the second module. . [Effects of the Invention]

[0010] According to the present invention, the convenience of programming is improved. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing a robot system, which is an example of a system according to the embodiment. [Figure 2] This is a control block diagram of the robot system according to the embodiment. [Figure 3] It is a schematic diagram for explaining a method of creating a module according to an embodiment. [Figure 4] It is a schematic diagram showing an example of a user interface image according to an embodiment. [Figure 5] It is a schematic diagram showing an example of a user interface image according to an embodiment. [Figure 6] It is an explanatory diagram showing the correspondence relationship between a template and a setting screen according to an embodiment. [Figure 7] It is an explanatory diagram showing an example of a control flow program according to an embodiment. [Figure 8] (a) to (d) are explanatory diagrams showing examples of setting windows for parameter setting according to an embodiment. [Figure 9] (a) to (c) are explanatory diagrams showing examples of setting windows for parameter setting according to an embodiment. [Figure 10] It is a schematic diagram showing an example of a user interface image according to an embodiment.

Mode for Carrying Out the Invention

[0012] Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings. FIG. 1 is a schematic diagram showing a robot system 1 which is an example of a system according to an embodiment. The robot system 1 includes an information processing apparatus 100 and a plurality of FA devices used for automatic production. The robot system 1 is used for manufacturing articles.

[0013] As FA devices, the robot system 1 includes, for example, a programmable logic controller (PLC) 110, a camera 111, a suction hand 112, a sensor device 113, a robot 114, an image processing apparatus 115, an AI (Artificial Intelligence) apparatus 116, and controllers 121 to 124.

[0014] Robot 114 is, for example, an industrial robot arm. Suction hand 112 is an example of an end effector and is attached to robot 114. Suction hand 112 is configured to hold a workpiece. Sensor device 113 is built into suction hand 112. Camera 111 is an example of an imaging device and is used to image a workpiece, for example, a component.

[0015] The PLC110 controls the execution timing of each device in order to coordinate the camera controller 121, hand controller 122, sensor controller 123, robot controller 124, image processing device 115, and AI device 116.

[0016] The camera controller 121 controls the camera 111. The hand controller 122 controls the suction hand 112. The sensor controller 123 controls the sensor device 113. The robot controller 124 controls the robot 114. The image processing device 115 and the AI ​​device 116 are each composed of computers.

[0017] The PLC 110, camera controller 121, hand controller 122, sensor controller 123, robot controller 124, image processing device 115, and AI device 116 are connected to the information processing device 100 using a connection method compliant with a predetermined standard.

[0018] In the robot system 1 configured as described above, control for picking a workpiece (part) by the robot 114 during the manufacturing of an item will be explained. The camera controller 121 causes the camera 111 to capture an image of the workpiece (part) and outputs the captured image to the image processing device 115. The image processing device 115 obtains the position information of the part from the captured image. The robot controller 124 obtains the position information of the part from the image processing device 115 and moves the robot 114 to the position of the part. The hand controller 122 operates the suction hand 112 to have the suction hand 112 hold the part. At that time, the sensor controller 123 determines whether the suction hand 112 has held the part based on the sensing result of the sensor device 113.

[0019] The information processing device 100 is composed of a computer. The information processing device 100 includes a CPU (Central Processing Unit) 151, which is an example of a processor. The CPU 151 functions as an information processing unit by executing a program 161. The information processing device 100 also includes a ROM (Read Only Memory) 152, a RAM (Random Access Memory) 153, and an SSD (Solid State Drive) 154 as storage units. The information processing device 100 also includes an I / O 155, which is an input / output interface. Furthermore, the information processing device 100 has a display 156, which is an example of a display unit, and a keyboard 158 and a mouse 157, which are examples of input units. The CPU 151, ROM 152, RAM 153, SSD 154, I / O 155, display 156, keyboard 158, and mouse 157 are connected to each other via a bus so that they can communicate data with one another.

[0020] ROM152 stores the basic programs related to the operation of the computer. RAM153 is a memory device that temporarily stores various data, such as the results of calculations performed by CPU151. SSD154 stores the results of calculations performed by CPU151 and various data acquired from external sources, as well as programs 161 that cause CPU151 to execute various processes. Program 161 is application software that CPU151 can execute.

[0021] I / O155 is an interface for external devices. Removable memory 162, such as recording disks or storage devices, can be connected to I / O155, and I / O155 can read various data and programs recorded in the removable memory 162.

[0022] The information processing device 100 includes an SSD 154 as a storage device, but is not limited to this. The storage device included in the information processing device 100 may be, for example, an HDD.

[0023] Furthermore, in this embodiment, the non-temporary recording medium readable by the computer is the SSD 154, and the program 161 is recorded on the SSD 154, but this is not limited to this. The program 161 may be recorded on any recording medium that is a non-temporary recording medium readable by the computer. As a recording medium for supplying the program 161 to the computer, removable memory 162 such as a flexible disk, hard disk, optical disk, magneto-optical disk, magnetic tape, non-volatile memory, etc., can be used.

[0024] In this embodiment, the information processing device 100 is used by a user, such as an end user, to program the system control of the robot system 1.

[0025] Figure 2 is a control block diagram of the robot system 1 according to an embodiment. The CPU 151 of the information processing device 100 functions as an information processing unit 50 by executing the program 161. The information processing unit 50 includes conversion units 102 and 104.

[0026] Each of the PLC110, camera controller121, hand controller122, sensor controller123, robot controller124, image processing device115, and AI device116 is a processing unit that executes processing based on commands with different specifications from each other.

[0027] For example, PLC110 has software 130 installed that executes ladder programs. PLC110 performs system-wide control processing based on commands COM0 created in a dedicated programming language.

[0028] The camera controller 121 has dedicated camera control software 131 installed. The camera controller 121 executes control processing for the camera 111 based on the command COM1 created in a dedicated programming language. Alternatively, as an example of camera 111 control processing, the camera 111 may be controlled by simple commands such as I / O commands using the dedicated programming language.

[0029] The hand controller 122 has dedicated hand control software 132 installed. The hand controller 122 executes control processing for the suction hand 112 based on commands COM2 created in a dedicated programming language.

[0030] The sensor controller 123 has dedicated sensor control software 133 installed. The sensor controller 123 executes control processing for the sensor device 113 based on commands COM3 created in a dedicated programming language.

[0031] The robot controller 124 has dedicated robot control software 134 installed. The robot controller 124 executes control processing for the robot 114 based on commands COM4 created in a dedicated programming language.

[0032] The image processing device 115 has an image processing library 135 installed that is specifically for image processing. The image processing device 115 executes image processing based on a command COM5 created in a dedicated programming language. Image processing may include, for example, binarization or pattern matching.

[0033] The AI ​​device 116 has an AI library 136 installed that is specifically for AI. The AI ​​device 116 executes AI processing based on commands COM6 created in a dedicated programming language. AI processing is, for example, the process of outputting output data for input data using a trained model.

[0034] Each software package, from 130 to 136, is comprised of, for example, a Software Development Kit (SDK).

[0035] A memory device, such as an SSD 154, stores a module set 103. The module set 103 includes multiple modules. The modules included in the module set 103 are data used by the information processing unit 50 to generate commands to be output to the corresponding devices.

[0036] Module set 103 includes at least one PLC module 140, which is an example of a module. It also includes at least one camera control module 141, which is an example of a module. It also includes at least one hand control module 142, which is an example of a module. It also includes at least one sensor control module 143, which is an example of a module. It also includes at least one robot control module 144, which is an example of a module. It also includes at least one image processing module 145, which is an example of a module. It also includes at least one AI module 146, which is an example of a module.

[0037] The conversion unit 102 handles the transfer of information between modules. The conversion unit 104 has the function of converting modules into commands. For example, the conversion unit 104 converts the PLC module 140 into command COM0. Also, for example, the conversion unit 104 converts the camera control module 141 into command COM1. Also, for example, the conversion unit 104 converts the hand control module 142 into command COM2. Also, for example, the conversion unit 104 converts the sensor control module 143 into command COM3. Also, for example, the conversion unit 104 converts the robot control module 144 into command COM4. Also, for example, the conversion unit 104 converts the image processing module 145 into command COM5. Also, for example, the conversion unit 104 converts the AI ​​module 146 into command COM6.

[0038] The conversion unit 104 communicates data with the devices 110, 121, 122, 123, 124, 115, and 116 in Figure 2 via the communication management unit 105 using the corresponding communication standard. The conversion unit 104 creates each command COM0 to COM6 in a format that allows communication with the corresponding device using the corresponding communication standard.

[0039] For example, the communication standard between the conversion unit 104 and the PLC 110 is TCP / IP. Also, for example, the communication standard between the conversion unit 104 and the camera controller 121 is USB3. Also, for example, the communication standard between the conversion unit 104 and the hand controller 122 is Ethernet. Also, for example, the communication standard between the conversion unit 104 and the sensor controller 123 is CC-Link. Also, for example, the communication standard between the conversion unit 104 and the robot controller 124 is TCP / IP.

[0040] While an example of a communication standard has been described, the communication standard is not limited to these. For example, the communication interface of the sensor controller 123 is a CC-Link interface, but it could also be an I / O interface or another fieldbus.

[0041] Furthermore, the sensor controller 123 can be directly connected to the information processing device 100, or it may be connected to the information processing device 100 via, for example, a PLC 110.

[0042] Furthermore, the modules included in module set 103 are not limited to the examples described above. For example, module set 103 may include control modules for other FA equipment. Also, for example, module set 103 may include modules that perform arithmetic operations, logical operations, and matrix operations. Also, for example, module set 103 may include modules that acquire PC environment information. Also, for example, module set 103 may include modules that save various data, such as image data, as logs.

[0043] The control flow program 101 is a program for comprehensively controlling each component of the robot system 1, and is created by the end user via a graphical user interface (GUI). The control flow program 101 is a program that causes the information processing unit 50 to read one module or a combination of two or more modules from the module set 103, and to generate the corresponding command.

[0044] The modules included in module set 103 are created according to a common specification template. Figure 3 is a schematic diagram illustrating the method for creating modules according to the embodiment.

[0045] Template 200 includes a setting section 201 where parameters are set, and setting sections 202, 203, 204 where programs are set. Setting section 201 includes setting sections 202, 20 3 The parameters to be passed to the source program of 204 are set. The setting unit 202 is set with a source program that specifies what kind of processing to execute. The setting unit 203 is set with a source program that specifies what kind of output to produce when the target device executes the processing according to the instructions of the setting unit 202. The setting unit 204 is set with a source program that specifies what kind of image to display on the display 156. In this way, the template 200 is divided into setting units 201 to 204 according to function.

[0046] In the setting units 202-204, a source program written in a predetermined programming language, such as C#, is set. The parameters set in the setting unit 201 are called by the source program.

[0047] The setting unit 201 is assigned an identifier such as "PARAM". The setting unit 202 is assigned an identifier such as "RUN". The setting unit 203 is assigned an identifier such as "RESULT". The setting unit 204 is assigned an identifier such as "DISPLAY".

[0048] For example, let's describe the camera control module 141. The setting unit 202, which is assigned the identifier "RUN", is set with a source program that causes the camera 111 to take images. R The setting unit 201, which is assigned the identifier "AM", is set with parameters such as exposure time and gain that determine the imaging conditions. The setting unit 203, which is assigned the identifier "RESULT", is set with a source program that stores the captured image, which is the result (output) of the imaging process, in the SSD 154. The setting unit 204, which is assigned the identifier "DISPLAY", is set with a source program that displays the captured image on the display 156.

[0049] Here, the parameters in the setting unit 201 of each module are set by the designer or the end user. In other words, the parameters can also be set by the end user. On the other hand, the source programs in the setting units 202 to 204 of each module are set by the designer. In other words, the source programs in the setting units 202 to 204 of each module are prepared in advance, and the end user does not need to set these source programs.

[0050] Each software 131-136 has various functions. Therefore, it is possible to create multiple modules with different functions that correspond to one of the software 131-136. For example, it is possible to create two image processing modules 1451 and 1452 with different functions that correspond to the image processing library 135.

[0051] Figure 4 is a schematic diagram showing an example of a user interface (UI) image UI1 according to the embodiment. Figure 4 illustrates the main screen 300 in the user interface image UI1. User operations in the user interface image UI1 are configured to be performed using a pointing device such as a mouse 157 or a keyboard 158.

[0052] The main screen 300 includes a list display unit 301 and transition buttons 302. The information processing unit 50 displays information about the modules included in the user-created control flow program 101 in a list format on the list display unit 301. Figure 4 illustrates the state after the control flow program 101 has already been created and control has been executed according to the control flow program 101.

[0053] The list display unit 301 displays as many display windows as there are modules included in the control flow program 101. For example, in the example in Figure 4, the control flow program 101 is assumed to include module information such as information for the camera control module 141, information for the image processing module 1451, information for the robot control module 144, and information for the hand control module 142. The list display unit 301 displays display windows 310, 314, 316, and 318 corresponding to modules 141, 1451, 144, and 142, respectively.

[0054] For example, the display window 310 is a display window corresponding to the camera control module 141. The display window 310 includes a display area 311 for displaying the acquired image, a display area 312 for displaying the name assigned to the module, and a display area 313 for displaying the processing time.

[0055] For example, the display window 314 is a display window corresponding to the image processing module 1451. The display window 314 includes a display area 315 that displays the contours and number of parts detected by image recognition.

[0056] For example, the display window 316 is a display window corresponding to the robot control module 144. The display window 316 includes a display area 317 that displays the coordinates of the robot 114's destination.

[0057] For example, the display window 318 is a display window corresponding to the hand control module 142. The display window 318 includes a display area 319 that displays the positional relationship between the suction hand 112 and the detected component, as well as information on when the suction hand 112 is turned ON.

[0058] Here, each module included in module set 103 is created according to template 200. Therefore, it is possible to display information consistently across multiple display windows 310, 314, 316, and 318, such as displaying the module name in display area 312 and the time required for module processing in display area 312. Furthermore, the results of executing the module can be viewed as images in display areas 311 and 317.

[0059] When the transition button 302 is operated, the information processing unit 50 transitions from the main screen 300 to the settings screen 400 and module list screen 450 shown in Figure 5 in the user interface image UI1. Figure 5 is a schematic diagram showing an example of the user interface image UI1 according to the embodiment. Figure 5 illustrates the settings screen 400 and module list screen 450 in the user interface image UI1.

[0060] The module list screen 450 is an area that displays multiple image elements A1 to A10. Each of the multiple image elements A1 to A10 corresponds to multiple modules included in module set 103. The module list screen 450 is an example of the first area.

[0061] The settings screen 400 includes a flowchart setting window 401, a parameter setting window 402, a processing result confirmation window 403, and an image confirmation window 404. The settings screen 400 also includes operation buttons 440 and 441. When operation button 440 is pressed once, the information processing unit 50 executes the multiple modules specified in the control flow program 101 in accordance with the flow for one cycle. When operation button 441 is pressed, the information processing unit 50 executes the multiple modules specified in the control flow program 101 one step at a time in accordance with the flow.

[0062] The user can verify the operation of the robot system 1 throughout the entire control flow program 101 by operating the operation button 440. Furthermore, the user can verify the operation of the robot system 1 step by step during the intermediate stages of the control flow program 101 by operating the operation button 441.

[0063] The flowchart setting window 401 is an area for creating the control flow program 101. The flowchart setting window 401 is an example of a second area.

[0064] The information processing unit 50 accepts the selection of an image element from among multiple image elements A1 to A10, each corresponding to one of the multiple modules displayed on the module list screen 450. The information processing unit 50 accepts the selection of a module corresponding to an image element when the image element from the module list screen 450 is dragged and dropped into the flowchart setting window 401 by the user's operation 451.

[0065] Furthermore, the information processing unit 50 accepts connection operations between image elements representing modules in the flowchart setting window 401, and accepts the selection of conditional branching modules corresponding to image elements A9 and A10 when drag-and-drop operations 451 are performed on image elements A9 and A10.

[0066] The information processing unit 50 accepts these operations and displays the dragged-and-dropped image elements in flowchart format in the flowchart setting window 401, arranging them in the order of control. The flowchart created in the flowchart setting window 401 is associated with the flow in the control flow program 101. Therefore, the user can program the control flow program 101 by combining image elements in flowchart format in the flowchart setting window 401.

[0067] Here, the information processing unit 50 displays information about the camera control module 141 in the parameter setting window 402, the processing result confirmation window 403, and the image confirmation window 404, for example, when an image element A1 corresponding to the camera control module 141 is selected by clicking. This allows the user to check, for example, the settings of the parameters of the camera control module 141, the processing results, and the captured image.

[0068] The information processing unit 50 displays setting windows 420 and 421 in the parameter setting window 402, which accept parameter settings for the module selected by the user. Setting window 420 is, for example, a window for accepting exposure time settings. Setting window 421 is, for example, a window for accepting gain settings.

[0069] The number of setting windows displayed corresponds to the number of parameters set in the module. In the camera control module 141, there are two parameters to be set, and the parameter setting window 402 displays two setting windows 420 and 421. This configuration allows the user to set the parameters for the selected module via the GUI.

[0070] In this embodiment, the modules included in the module set 103 are created according to the template 200. In this embodiment, the information processing unit 50 displays a parameter setting window 402 on the setting screen 400 that accepts parameters to be set in the module setting unit 201. Therefore, regardless of which module is selected, the information processing unit 50 can display setting windows that accept parameter settings, such as setting windows 420 and 421, with a consistent appearance and operability. The information processing unit 50 may also display the name assigned to the module in the display area 422 of the parameter setting window 402.

[0071] Furthermore, when the operation button 440 or 441 is operated, the information processing unit 50 executes the source program set in the module setting unit 202 to perform a predetermined process, and then executes the source program set in the module setting unit 203 to obtain the processing result. At that time, the information processing unit 50 displays the processing result in the processing result confirmation window 403.

[0072] For example, the information processing unit 50 displays the processing time required for the module's processing in area 432 of the processing result confirmation window 403, and displays the determination result of whether the module operated correctly in area 430 of the processing result confirmation window 403. These processing times and determination results are common processing results that can be obtained for any module in the module set 103. In other words, the information processing unit 50 displays common processing results for all modules in areas 432 and 430.

[0073] Meanwhile, the information processing unit 50 displays the processing results specific to the module in the area 431 included in the processing result confirmation window 403. For example, if the module is a camera control module 141, the information processing unit 50 displays a thumbnail of the captured image, which is the processing result, in the area 431. Note that the processing results displayed in the processing result confirmation window 403 include data other than numerical data and character data, such as image data.

[0074] Figure 6 is an explanatory diagram showing the correspondence between the template 200 and the settings screen 400 according to the embodiment. R The setting unit 201, which is assigned the identifier "AM", is associated with the parameter setting window 402. The setting unit 202, which is assigned the identifier "RUN", is associated with the operation buttons 440 and 441. The setting unit 203, which is assigned the identifier "RESULT", is associated with the processing result confirmation window 403. The setting unit 204, which is assigned the identifier "DISPLAY", is associated with the image confirmation window 404.

[0075] Figure 7 is an explanatory diagram showing an example of a control flow program 101 according to the embodiment. Figure 7 illustrates an example of a control flow program 101 that can be constructed by a user. The created control flow program 101 is stored, for example, in SSD 154. SSD 154 also stores global variables 600 that can be referenced in the control flow program 101.

[0076] Global variables 600 include, for example, numeric variables 500, decision variables 510, string variables 520, selection variables 530, image variables 540, region variables 550, POSE variables 560, and so on.

[0077] When the information processing unit 50 starts control according to the control flow program 101, it first operates the robot control module 610. Specifically, the information processing unit 50 generates a command COM4 corresponding to the source program set in the setting unit 202, which is assigned the identifier "RUN" and is included in the robot control module 610. The information processing unit 50 then sends the command COM4 to the robot controller 124, causing the robot controller 124 to execute the command COM4. The source program set in the setting unit 202 contains a command to move the robot 114 to the desired location.

[0078] In the robot control module 610, "PA R The setting unit 201, which is assigned the identifier "AM", has parameters 611 for the movement position and 612 for the movement mode set in it, and the information processing unit 50 refers to these parameters 611 and 612 when generating the command COM4 based on the source program of the setting unit 202. For example, parameter 611 is associated with the origin position variable 561 in the POSE type variable 560 of the global variable 600. In other words, the robot controller 124 controls the robot 114 to move to the origin position.

[0079] Although the example given shows two parameters, 611 and 612, set in the setting unit 201 of the robot control module 610, multiple parameters can be set, such as parameters that determine the operating speed and parameters that indicate waypoints. In other words, the parameters set in the robot control module 610 are merely examples and are not limited to these. The same applies to the parameters set in the other modules 620, 630, 640, 650, and 660, which are described below.

[0080] Next, the information processing unit 50 operates the camera control module 620. Specifically, the information processing unit 50 generates a command COM1 corresponding to the source program set in the setting unit 202, which is assigned the identifier "RUN" and is included in the camera control module 620. The information processing unit 50 then sends the command COM1 to the camera controller 121, causing the camera controller 121 to execute the command COM1. The source program set in the setting unit 202 contains a command to cause the camera 111 to take an image. The captured image is obtained as the processing result 622 from the imaging process of the camera 111.

[0081] In the camera control module 620, "PA R The setting unit 201, which is assigned the identifier "AM", has an exposure time parameter 621 set in it, and the information processing unit 50 refers to this parameter 621 when generating the command COM1 based on the source program of the setting unit 202. For example, the exposure time variable 501 in the numeric variable 500 of the global variable 600 is associated with parameter 621. In other words, the camera controller 121 controls the camera 111 to take an image with the exposure time set in the exposure time variable 501. The captured image is passed to the captured image variable 541 in the image type variable 540.

[0082] Next, the information processing unit 50 operates the component detection module 630. Specifically, the information processing unit 50 generates a command COM5 corresponding to the source program set in the setting unit 202, which is assigned the identifier "RUN" and is included in the component detection module 630. The information processing unit 50 then sends the command COM5 to the image processing unit 115, causing the image processing unit 115 to execute the command COM5. The command COM5 is, for example, a command to perform image processing to detect (recognize) the position of a component from the captured image. The recognized position of the component is obtained as a POSE-type processing result 633 from this image processing.

[0083] In the component detection module 630, "PA R The setting unit 201, which is assigned the identifier "AM", has a parameter 631 that indicates the captured image used for image processing and a parameter 632 that indicates a threshold value set. The information processing unit 50 refers to these parameters 631 and 632 when generating the command COM5 based on the source program of the setting unit 202. For example, parameter 631 is associated with the captured image variable 541 in the image type variable 540 of the global variable 600. In other words, the image processing device 115 receives the input of the captured image set in the captured image variable 541 and performs image processing on the captured image. Although the case in which the captured image is registered in the captured image variable 541 has been described, the image processing device 115 may also receive the input of the captured image directly from the camera controller 121.

[0084] Furthermore, the image processing device 115 outputs a judgment result based on image processing as a judgment-type processing result 634. For example, the image processing device 115 outputs the judgment-type processing result 634 as a value indicating "TRUE" or a value indicating "FALSE". If the detection of the part's position is successful, a value indicating "TRUE" is output, and if the detection of the part's position fails, a value indicating "FALSE" is output.

[0085] Next, the information processing unit 50 operates the conditional branching module 640. That is, the information processing unit 50 executes a command corresponding to the source program set in the setting unit 202, which is assigned the identifier "RUN" and is included in the part detection module 630, and performs a determination process. As part of the determination process, if the value of parameter 641 indicates "TRUE", the information processing unit 50 proceeds to the robot control module 650 which moves the robot 114 to the part removal position, and if the value of parameter 641 indicates "FALSE", it terminates there.

[0086] In the conditional branching module 640, "PA R The setting unit 201, which is assigned the identifier "AM", is configured to use the determination-type processing result 634 as parameter 641.

[0087] The information processing unit 50 operates the robot control module 650 if the value of parameter 641 is "TRUE". Specifically, the information processing unit 50 generates a command COM4 corresponding to the source program set in the setting unit 202, which is assigned the identifier "RUN" and included in the robot control module 650. The information processing unit 50 then sends the command COM4 to the robot controller 124, causing the robot controller 124 to execute the command COM4. The source program set in the setting unit 202 contains a command to move the robot 114 to the designated location.

[0088] In the robot control module 650, "PA R The setting unit 201, which is assigned the identifier "AM", has a parameter 651 for the movement position set in it, and the information processing unit 50 refers to this parameter 651 when generating the command COM4 based on the source program of the setting unit 202. For example, parameter 651 is set to be the recognized position, which is the processing result 633. In other words, the robot controller 124 controls the robot 114 to move to the recognized position.

[0089] Next, the information processing unit 50 operates the robot control module 660. Specifically, the information processing unit 50 generates a command COM4 corresponding to the source program set in the setting unit 202, which is assigned the identifier "RUN" and is included in the robot control module 660. The information processing unit 50 then sends the command COM4 to the robot controller 124, causing the robot controller 124 to execute the command COM4. The source program set in the setting unit 202 contains a command to move the robot 114 to the designated location.

[0090] In the robot control module 660, "PA R The setting unit 201, which is assigned the identifier "AM", has parameters 661 for the movement position and 662 for the movement mode set in it, and the information processing unit 50 refers to these parameters 661 and 662 when generating the command COM4 based on the source program of the setting unit 202. For example, parameter 661 is associated with the origin position variable 561 in the POSE type variable 560 of the global variable 600. In other words, the robot controller 124 controls the robot 114 to move to the origin position.

[0091] As described above, the parameters set in a module may be obtained via the global variable 600, or they may be obtained from the processing results of other modules. When passing parameters between modules, it is preferable that the parameters are of the same type as the processing results. For example, if a parameter is of type POSE, it is preferable that the processing result that receives the input as a parameter is also of type POSE.

[0092] The settings windows, which are the UI for setting parameters, such as the settings windows 420 and 421 shown in Figure 5, are prepared according to the type of parameter. Through the settings windows for parameter setting, users can directly set constants, configure the system to receive data from the processing results of other modules, or configure it to receive data from the global variable 600.

[0093] Figures 8(a) to 9(c) are explanatory diagrams showing examples of setting windows for parameter setting according to the embodiment. Parameters to be set in the setting unit 201 of a module included in module set 103 can be input through any of the setting windows 700, 710, 720, 730, or 740, as shown in Figures 8(a) to 9(c), for example. Each of the setting windows 700, 710, 720, 730, or 740 is displayed in the parameter setting window 402 by the information processing unit 50.

[0094] Figures 8(a) and 8(b) show the setting window 700 for setting the exposure time. UI701 in the setting window 700 displays the parameter type, and in the examples of Figures 8(a) and 8(b), the text "Numeric Type" is displayed. UI702 in the setting window 700 displays the parameter name, and in the examples of Figures 8(a) and 8(b), the text "Exposure Time" is displayed. This allows the user to understand that the setting window 700 is for setting the exposure time as a numeric type parameter. These UI701 and 702 are common display areas for all parameters, regardless of the parameter type.

[0095] UI703 in the settings window 700 is a UI for selecting a constant or a reference. In the example in Figure 8(a), "constant" is selected. When "constant" is selected in UI703, the information processing unit 50 displays UI704, which receives input for the exposure time parameter (numerical value), and UI705, which accepts increases or decreases in the numerical value. As the exposure time parameter to be input, the information processing unit 50 accepts the selection of one parameter from a group of candidate parameters in UI705. The user can set the exposure time parameter by operating UI705 to increase or decrease the numerical value in UI704. Alternatively, a numerical value can be directly entered into UI704. With UI704 and 705, it is easy to set numerical parameters such as exposure time.

[0096] In the example in Figure 8(b), "Global Variables" are selected as the reference target. Note that the user can also configure the reference to be something other than "Global Variables" by manipulating UI703. In the example in Figure 8(b), it is configured to reference global variable 600 from Figure 7.

[0097] The method for setting up data reference from a reference source is shown in Figure 8(b). For example, when "Global Variable" is selected, the information processing unit 50 displays UI 706 for selecting the reference source variable. In the example in Figure 8(b), the reference source is of "numeric type," so the information processing unit 50 only accepts the selection of variables belonging to the numeric type variable 500 in the global variable 600. The numeric type variable 500 includes a group of parameters consisting of multiple candidate parameters. In the example in Figure 8(b), the exposure time variable 501 from Figure 7 is selected from the group of parameters. This UI 706 makes it easy to set numeric type parameters such as exposure time.

[0098] Even if the user selects "constant" in UI703, the instructions for the parameter will differ depending on the parameter type. Figure 8(c) shows the setting window 710 for setting the judgment value. Figure 8(c) illustrates the case where the parameter type is a judgment type. For example, in the case of a judgment type parameter for indicating ON / OFF or TRUE / FALSE, the information processing unit 50 displays UI711 for setting the judgment value, as shown in Figure 8(c).

[0099] In some cases, the processing results of other modules may be referenced. Figure 8(d) shows a setting window 710 for setting the judgment value. As shown in Figure 8(d), if "flow" is selected as the reference destination in UI703, the information processing unit 50 displays UI712, which accepts the specification of the reference destination module. The module is specified by the user in UI712.

[0100] However, since multiple processing results may be output when the module is executed, the information processing unit 50 displays a UI 713 that accepts the selection of which processing result to refer to. In the example of Figure 8(d), the parameter refers to, for example, the judgment-type processing result 634 output by the component detection module 630 belonging to the control flow program 101. The judgment-type processing result 634 is a judgment result.

[0101] Furthermore, when a module is executed, multiple processing results may be output, but UI713 only displays the processing result of the decision type. This mechanism is the same for types other than the decision type.

[0102] Figure 9(a) shows an example of a setting window 720 for setting a string type parameter. In the case of a string type parameter, the information processing unit 50 displays a UI 721 for entering a string, as shown in Figure 9(a). The user can set a string as a parameter by entering a string into the UI 721.

[0103] Figure 9(b) illustrates an example of a setting window 730 for setting a selection-type parameter. In the case of a selection-type parameter, the information processing unit 50 displays a UI 731 that accepts selections from the options, as shown in Figure 9(b). The user can set the parameter by selecting one of the multiple options displayed in the UI 731.

[0104] Figure 9(c) shows an example of a setting window 740 for setting image type parameters. In the case of an image parameter, the information processing unit 50 displays a UI 741 that accepts an image from the global variable 600, as shown in Figure 9(c). The user can then select an image to set as a parameter from the displayed images.

[0105] It is also possible to register variables in global variable 600. Figure 10 is a schematic diagram showing an example of user interface image UI1 according to the embodiment. Figure 10 illustrates the settings screen 400 in user interface image UI1. A method for storing the data obtained as a result of module processing in global variable 600 will be described.

[0106] The information processing unit 50 displays a setting window 800 in the processing result confirmation window 403 that accepts the selection of the processing result to be stored in the global variable 600. In the setting window 800, the information processing unit 50 displays a UI 801 that accepts the selection of the output destination and a UI 802 that accepts the variable number, and accepts the specification of the location where the data will be stored via UI 801 and 802. If the processing result is of image type, the captured image is stored, for example, in the captured image variable 541 belonging to the image type variable 540 in the global variable 600.

[0107] As described above, according to this embodiment, even if the end user, who is the system developer, is not an engineer, they can create a control flow program 101 that enables integrated control of FA equipment by performing simple operations such as selecting each item displayed on the settings screen 400.

[0108] In the following explanation, for example, the first controlled object will be a camera 111 and the second controlled object will be a robot 114. The camera controller 121 is configured to execute processing based on command COM1. The robot controller 124 is configured to execute processing based on command COM4, ​​which has different specifications from command COM1. The camera controller 121 is, for example, an example of a first processing unit. The robot controller 124 is, for example, an example of a second processing unit. Command COM1 is, for example, an example of a first command. Command COM4 is, for example, an example of a second command.

[0109] The camera controller 121 controls the camera 111 based on command COM1. The robot controller 124 controls the robot 114 based on command COM4.

[0110] The information processing unit 50 displays a user interface image UI1 on the display 156 that can accept the selection of the camera control module 141 and the robot control module 144. The camera control module 141 is an example of a first module, and the robot control module 144 is an example of a second module. When the information processing unit 50 accepts the selection of the camera control module 141, it obtains a command COM1 based on the camera control module 141. Also, when the information processing unit 50 accepts the selection of the robot control module 144, it obtains a command COM4 based on the robot control module 144.

[0111] With the above configuration, users can program simply by selecting the camera control module 141 and the robot control module 144 on the user interface image UI1, thereby improving the convenience of programming.

[0112] Here, command COM1 is a command that conforms to the specifications of the programming language of software 131, which is dedicated to camera control. Command COM4 is a command that conforms to the specifications of the programming language of software 134, which is dedicated to robot control and differs from the programming language of software 131. The programming language used in software 131 is an example of a first programming language. The programming language used in software 134 is an example of a second programming language. The software 134 dedicated to robot control is, for example, a robot program. Users can easily program without having to learn these programming languages, further improving the convenience of programming.

[0113] The above examples show cases where there is a controlled object, but there are also cases where there is no controlled object, such as in the image processing device 115 and the AI ​​device 116. Even in these cases, programming can be easily performed, improving the convenience of programming.

[0114] As shown in Figure 3, the camera control module 141 includes a source program PR1 written in a predetermined programming language, such as C#. Source program PR1 is an example of a first source program. Source program PR1 is set in the setting units 202 to 204 of the camera control module 141. Similarly, the robot control module 144 includes a source program PR2 written in a predetermined programming language, such as C#. Source program PR2 is an example of a second source program. Source program PR2 is set in the setting units 202 to 204 of the robot control module 144.

[0115] When the information processing unit 50 receives a selection of the camera control module 141, it converts the source program PR1 of the setting unit 202 from a predetermined programming language to the programming language of the camera control software 131 and outputs it to the camera controller 121 as command COM1.

[0116] Furthermore, when the information processing unit 50 receives a selection of the robot control module 144, it converts the source program PR2 of the setting unit 202 from a predetermined programming language to the programming language of the robot control software 134, and outputs it to the robot controller 124 as a command COM4.

[0117] Source programs PR1 and PR2 are pre-created by a designer proficient in a predetermined programming language, such as C#, and the user does not need to create source programs PR1 and PR2. In this way, the user can program the control flow program 101 by combining modules without having the knowledge to create source programs PR1 and PR2, thereby improving the convenience of programming. Furthermore, by unifying source programs PR1 and PR2 in a predetermined programming language, the burden on the designer is also reduced. In this embodiment, source programs PR1 and PR2 are written in a programming language different from both the programming language of software 131 and the programming language of software 134, such as C#.

[0118] The source programs PR1 and PR2 may be in text format or binary format. Furthermore, the source programs PR1 and PR2 may be written in different programming languages. In this case, it is preferable that source program PR1 is written in the programming language of the camera control software 131, and that source program PR2 is written in the programming language of the robot control software 134.

[0119] The camera control module 141 includes parameters that are passed to source program PR1, i.e., one of the setting units 202 to 204. These parameters are set in setting unit 201. Separating the parameters from source program PR1, i.e., setting units 202 to 204, simplifies the design of the camera control module 141. The same applies to the other modules in module set 103.

[0120] The information processing unit 50 can accept parameter inputs through the parameter setting window 402 shown in Figure 5. For example, it can accept inputs for the exposure time setting window 420 and the gain setting window 421. The parameters indicating exposure time and gain are examples of first parameters. In this way, the user can set parameters as appropriate, further improving the convenience of programming.

[0121] Furthermore, as shown in Figure 5, the user interface image UI1 includes a module list screen 450 and a flowchart setting window 401. The information processing unit 50 displays a list of image elements A1 to A10, including image element A1 corresponding to the camera control module 141 and image element A4 corresponding to the robot control module 144, on the module list screen 450. Image element A1 is an example of a first image element, and image element A4 is an example of a second image element.

[0122] The information processing unit 50 accepts the selection of the camera control module 141 when image element A1 is dragged and dropped into the flowchart setting window 401. Similarly, the information processing unit 50 accepts the selection of the robot control module 144 when image element A4 is dragged and dropped into the flowchart setting window 401. This allows, for example, to operate the camera 111, the end user simply drags and drops image element A1 from the module list screen 450 to the flowchart setting window 401. Similarly, to operate the robot 114, the end user simply drags and drops image element A4 from the module list screen 450 to the flowchart setting window 401.

[0123] In this way, a control flow program 101 can be easily created by combining multiple image elements. This improves the convenience of integrating and programming the operation of the entire robot system 1.

[0124] Furthermore, the information processing unit 50 displays the drag-and-dropped image elements in a flowchart format in the flowchart setting window 401, arranging them in the order of control. This makes the control flow visual, further improving the convenience of programming.

[0125] Furthermore, as shown in Figure 5, the information processing unit 50 displays an operation button 440 in the user interface image UI1. The operation button 440 is a button that receives a command to execute multiple modules in control order when multiple modules are selected, that is, when multiple image elements corresponding to multiple modules are arranged in the flowchart setting window 401. The operation button 440 is an example of a first button. When the user operates the operation button 440, one cycle of control according to the control flow program 101 is executed. This allows the user to confirm the operation of one cycle from "START" to "END" of the control flow program 101.

[0126] Furthermore, as shown in Figure 5, the information processing unit 50 displays an operation button 441 in the user interface image UI1. The operation button 441 is a button that receives a command to execute one of the multiple modules when multiple modules have been selected. The operation button 441 is an example of a second button. When the user operates the operation button 441, one step of the module in the control flow program 101 is executed. This allows the user to confirm the operation of one step of the control flow program 101.

[0127] In this process, the information processing unit 50 displays an image showing the processing result in the image confirmation window 404, as shown in Figure 5. For example, if an image is obtained as a processing result from the execution of the camera control module 141, the information processing unit 50 displays the image obtained by the camera 111 in the image confirmation window 404. By displaying the processing result as an image in the image confirmation window 404, the user can confirm the processing result, i.e., the image captured.

[0128] Furthermore, as shown in Figures 8(a) to 9(c), setting windows 700, 710, 720, 730, and 740 for setting parameters are provided according to the data type of the parameter. In each of these setting windows 700, 710, 720, 730, and 740, parameters can be set by direct data input or selection. To add data types, setting windows similar to these 700 to 740 should be provided.

[0129] After creating the control flow program 101, the user can start the processing of the control flow program 101 by operating the operation buttons 440 and 441. For example, when executing the robot control module 144 shown in Figure 5, the information processing unit 50 reads the contents of the setting unit 202, which is assigned the identifier "RUN" and included in the robot control module 144, generates a command COM4 from the setting unit 202, and sends the command COM4 to the robot controller 124. As a result, the robot controller 124 moves the robot 114 to the instructed position.

[0130] The information processing unit 50 incorporates a conversion program in the conversion unit 104 that corresponds to each software 130-136. Therefore, the source program of the setting unit 202 is converted into a command COM4 according to the parameters set in the setting unit 201. The command COM4 is then passed to the robot controller 124 via the communication management unit 105. The robot controller 124 operates the robot 114 according to the command COM4. Note that the target of the command transmission is not limited to the controller; it may also be a computer on which software such as an image processing library or an AI library is installed.

[0131] When the robot controller 124 operates the robot 114, the information processing unit 50 obtains the processing result from the robot controller 124 via the communication management unit 105.

[0132] The information processing unit 50 executes the source program of the setting unit 203, which is assigned the identifier "RESULT" for the robot control module 144, and stores data indicating the processing result in the SSD 154. The information processing unit 50 also executes the source program of the setting unit 204, which is assigned the identifier "DISPLAY" for the robot control module 144, and displays the display content corresponding to the processing result as a still image or video on the display 156.

[0133] Furthermore, as shown in Figure 6, the information processing unit 50 links the functions of each setting unit 201 to 204 in the template 200 to the items displayed on the setting screen 400. Therefore, the user can check the processing results of the operation based on the robot control module 144 through the processing result confirmation window 403 and the image confirmation window 404.

[0134] The display format of the user interface image UI1 is not limited to the format shown in Figure 5; for example, it may be in a list format as shown in Figure 4, such as the format displayed on the list display unit 301.

[0135] Furthermore, since each module included in module set 103 is created according to template 200, the information of each module can be displayed with a consistent appearance.

[0136] Furthermore, it is easy to add modules to the module set 103, and in the conversion unit 104, it is only necessary to add a conversion program that converts the added modules into the programming language of the corresponding software, thus offering excellent expandability.

[0137] The above explanation used the operation of robot 114 as an example, but we will now explain the case where multiple FA devices are linked together as an example. For example, we will explain the operation of a part of the sequence from the component detection module 630 to the robot control module 650 shown in Figure 6.

[0138] The control flow program 101 proceeds as follows: First, the part detection module 630 detects the position of the part based on the captured image. Then, the conditional branching module 640 branches the process according to the judgment result of the part detection module 630. If the part detection module 630 successfully detects the part, the control process transitions to the robot control module 650; if detection fails, the control process transitions to "END".

[0139] To construct this flow, in Figure 5, the end user drags and drops image elements A5, A9, and A4 corresponding to the three modules mentioned above from the module list screen 450 to the flowchart setting window 401. Pusu Operation 451 expands the flowchart into the flowchart setting window 401. The user then connects the modules in the flowchart setting window 401 to form a flowchart.

[0140] Next, in order to set the parameters for the three modules mentioned above, the user displays the parameter setting window 402 by clicking on each image element A5, A9, and A4 on the module list screen 450, and sets the parameters in the setting section 201 of the module corresponding to the clicked image element.

[0141] The user selects the part detection module 630 and, in the setting window 740 shown in Figure 9(c), selects the captured image which is the processing result 622 of the camera control module 620 shown in Figure 7, thereby setting the captured image which is the processing result 622 of the camera control module 620 to parameter 631.

[0142] The component detection module 630 also has a threshold parameter 632. The user sets the threshold as parameter 632 in the setting window 700 as shown in Figure 8(a).

[0143] The parameter 641 of the conditional branching module 640 uses the result 634 of the judgment type processing of the component detection module 630. Therefore, the user sets the parameter 641 by selecting it from the setting window 710 shown in Figure 8(d). This parameter 641 allows the conditional branching module 640 to branch the transition destination.

[0144] Since the robot control module 650 uses the recognition position, which is the POSE type processing result 633 of the part detection module 630, the user sets the parameters through the settings window, just like with other data types.

[0145] In summary, even when linking multiple FA devices, users no longer need to program using the programming language of each individual FA device. Instead, they can easily configure data transfer between modules simply by inputting or selecting options through the UI.

[0146] When the user operates the operation button 440 shown in Figure 5, the created control flow program 101 is executed. This automatically transfers data between modules and performs control transitions, making it possible to link multiple FA devices together.

[0147] As described above, according to this embodiment, the user does not need to program in a programming language, and can easily create a control flow program 101 that integrally controls the entire robot system 1.

[0148] Therefore, users can easily program the control flow program 101 even without being proficient in programming languages, unlike non-engineers. This improves programming efficiency.

[0149] The present invention is not limited to the embodiments described above, and many modifications are possible within the technical concept of the present invention. Furthermore, the effects described in the embodiments are merely a list of the most preferred effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0150] In the above-described embodiment, the case in which one of the multiple FA devices in the system is a robot 114 was explained as an example, but the present invention is not limited to this, and can be applied even if the system does not have a robot 114. Furthermore, the multiple FA devices may include devices that can receive communication commands from an external source, such as conveyors, linear actuators, air cylinders, and devices capable of 3D distance measurement.

[0151] Furthermore, although the above-described embodiment uses the case where the first processing unit is a camera controller 121 and the second processing unit is a robot controller 124 as an example, it is not limited to this. The first processing unit can be any of the multiple devices such as the PLC 110, camera controller 121, hand controller 122, sensor controller 123, robot controller 124, image processing device 115, and AI device 116. The second processing unit can be a different device from the first processing unit among the multiple devices.

[0152] Furthermore, although the above-described embodiment described a case in which the information processing device 100, the image processing device 115, and the AI ​​device 116 are configured as separate computers, the invention is not limited to this. The information processing device 100, the image processing device 115, and the AI ​​device 116 may be configured as one or more computers.

[0153] Furthermore, although the above-described embodiment described a case in which the display unit, such as the display 156, and the input unit, such as the mouse 157 and keyboard 158, are configured separately, the invention is not limited to this, and the input unit and the display unit may be configured as an integrated unit, for example, as in a touch panel display.

[0154] (Other examples) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0155] The above disclosure of embodiments includes the following sections.

[0156] (Section 1) A first processing unit that executes processing based on the first command, A second processing unit that executes processing based on a second command having specifications different from the first command, The system includes an information processing unit which displays a user interface image on a display unit that can accept the selection of a first module and a second module, and which, when the selection of the first module is accepted, acquires a first command based on the first module, and when the selection of the second module is accepted, acquires a second command based on the second module. A system characterized by the following features.

[0157] (Section 2) The system further comprises a first controlled object and a second controlled object, The first processing unit executes a process to control the first controlled object based on the first command, The second processing unit executes a process to control the second controlled object based on the second command. The system according to item 1, characterized in that

[0158] (Section 3) The aforementioned first directive is a directive that conforms to the specifications of a first programming language, The second directive is a directive that conforms to the specifications of a second programming language different from the first programming language. The system according to item 1 or 2, characterized in that

[0159] (Section 4) The first module includes a first source program written in a predetermined programming language, The system according to any one of items 1 to 3, characterized in that

[0160] (Section 5) The second module includes a second source program written in the predetermined programming language, The system according to item 4, characterized by the features described herein.

[0161] (Section 6) The aforementioned first directive is a directive that conforms to the specifications of a first programming language, The aforementioned predetermined programming language is a programming language different from the aforementioned first programming language. The system according to item 4 or 5, characterized in that it is a system according to item 4 or 5.

[0162] (Section 7) The first module includes a first parameter that is passed to the first source program, The system according to any one of items 4 to 6, characterized in that it is a system according to any one of items 4 to 6.

[0163] (Section 8) The information processing unit is capable of receiving input for the first parameter. The system according to item 7, characterized in that

[0164] (Section 9) The aforementioned information processing unit, As the first parameter mentioned above, it is possible to accept input from the parameter group. The system according to item 8, characterized in that

[0165] (Section 10) The user interface image includes a first region and a second region, The aforementioned information processing unit, A first image element corresponding to the first module and a second image element corresponding to the second module are displayed in the first region. The selection of the first module is accepted when the first image element is dragged and dropped into the second region. The selection of the second module is accepted when the second image element is dragged and dropped into the second region. A system according to any one of items 1 to 9, characterized in that it is a system according to any one of items 1 to 9.

[0166] (Section 11) The aforementioned information processing unit, In the second region, the drag-and-dropped image elements are arranged and displayed in a flowchart format according to their control order. The system according to item 10, characterized in that

[0167] (Section 12) When the information processing unit receives a selection of multiple modules, it displays a first button in the user interface image that accepts a command to execute the multiple modules in the control order. The system according to any one of items 1 to 11, characterized in that

[0168] (Section 13) When the information processing unit receives a selection of multiple modules, it displays a second button in the user interface image that accepts a command to execute one of the multiple modules. The system according to any one of items 1 to 11, characterized in that

[0169] (Section 14) The first controlled object is an imaging device. The system according to item 2, characterized in that

[0170] (Section 15) The second controlled object is a robot. The system according to item 2 or 14, characterized in that it is the system described in item 2 or 14.

[0171] (Section 16) The information processing unit displays the captured image obtained by the imaging device in the user interface image. The system according to item 14, characterized in that

[0172] (Section 17) The first module and the second module are created according to a common specification template. The system according to any one of items 1 to 16, characterized in that

[0173] (Section 18) A method for manufacturing articles, characterized by manufacturing articles using the system described in any one of paragraphs 1 to 17.

[0174] (Section 19) The system includes an information processing unit that displays a user interface image on the display unit that allows the user to select a first module and a second module, and when the user selects the first module, it obtains a first command to be executed by the first processing unit based on the first module, and when the user selects the second module, it obtains a second command, which has different specifications from the first command, to be executed by the second processing unit based on the second module. An information processing device characterized by the following:

[0175] (Section 20) A user interface image that allows selection of the first module and the second module is displayed on the display unit. When the selection of the first module is received, a first command is obtained to be executed by the first processing unit based on the first module. When the selection of the second module is accepted, a second command with specifications different from the first command is obtained, which is to be executed by the second processing unit based on the second module. An information processing method characterized by the following:

[0176] (Section 21) A program for causing a computer to execute the information processing method described in item 20.

[0177] (Section 22) A computer-readable recording medium containing the program described in item 21. [Explanation of symbols]

[0178] 1…Robot system (system), 50…Information processing unit, 121…Camera controller (first processing unit), 124…Robot controller (second processing unit), 141…Camera control module (first module), 144…Robot control module (second module), COM1…Command (first command), COM4…Command (second command)

Claims

1. A first processing unit that controls a first controlled object based on a first command, A second processing unit controls a second control target different from the first control target based on a second command having specifications different from the first command, The system includes an information processing unit that displays a user interface image on a display unit that can accept the selection of a first module corresponding to the first control target and a second module corresponding to the second control target, The first module includes a first parameter setting unit for setting first parameters, a first execution setting unit for generating first commands from the first parameters and outputting them to the first processing unit, and a first result setting unit for acquiring first execution results obtained when control of the first controlled object based on the first command is performed. The second module includes a second parameter setting unit for setting a second parameter, a second execution setting unit for generating a second command from the second parameter and outputting it to the second processing unit, and a second result setting unit for obtaining a second execution result obtained when the control of the second controlled object based on the second command is executed. The user interface image can accept a selection from the user to execute either the first module or the second module. The user interface image is a system capable of receiving input of parameters set in the first parameter setting unit or the second parameter setting unit.

2. The first directive is a directive that conforms to the specifications of a first programming language, The second directive is a directive that conforms to the specifications of a second programming language different from the first programming language. The system according to claim 1.

3. The first execution setting unit includes a first source program written in a predetermined programming language, The system according to claim 1.

4. The second execution setting unit includes a second source program written in a predetermined programming language, The system according to claim 1.

5. The first directive is a directive that conforms to the specifications of a first programming language, The predetermined programming language is a programming language different from the first programming language. The system according to claim 3.

6. The first parameter is passed to the first source program, The system according to claim 3.

7. The second parameter is passed to the second source program, The system according to claim 4.

8. The aforementioned information processing unit, As the first parameter, it is possible to accept input from the parameter group. The system according to claim 1.

9. The user interface image includes a first region and a second region, The aforementioned information processing unit, A first image element corresponding to the first module and a second image element corresponding to the second module are displayed in the first region. The selection of the first module is accepted when the first image element is dragged and dropped into the second region. The selection of the second module is accepted when the second image element is dragged and dropped into the second region. The system according to claim 1.

10. The aforementioned information processing unit, In the second region, the drag-and-dropped image elements are arranged and displayed in a flowchart format according to their control order. The system according to claim 9.

11. When the information processing unit receives a selection of multiple modules, it displays a first button in the user interface image that accepts a command to execute the multiple modules in the control order. The system according to claim 1.

12. When the information processing unit receives a selection of multiple modules, it displays a second button in the user interface image that accepts a command to execute one of the multiple modules. The system according to claim 1.

13. The first controlled object is an imaging device. The system according to claim 1.

14. The second controlled object is a robot. The system according to claim 1.

15. The information processing unit displays the captured image obtained by the imaging device in the user interface image. The system according to claim 13.

16. The first module and the second module are created according to a common specification template. The system according to claim 1.

17. The first module has a first display setting unit that displays and outputs the contents of the first execution result in the user interface image, The second module has a second display setting unit that displays and outputs the contents of the second execution result in the user interface image. The system according to claim 1.

18. A method for manufacturing an article, characterized by manufacturing the article using the system according to any one of claims 1 to 17.

19. An information processing apparatus that displays a user interface image on a display unit that can accept the selection of a first module corresponding to a first control target and a second module corresponding to a second control target different from the first control target, The first module includes a first parameter setting unit for setting first parameters, a first execution setting unit for generating first commands from the first parameters and outputting them to a first processing unit, and a first result setting unit for acquiring first execution results obtained when control of the first controlled object based on the first command is performed. The second module includes a second parameter setting unit for setting a second parameter, a second execution setting unit for generating a second command from the second parameter and outputting it to a second processing unit, and a second result setting unit for obtaining a second execution result obtained when the control of the second controlled object based on the second command is executed. The user interface image can accept a selection from the user to execute either the first module or the second module. The user interface image is an information processing device capable of receiving input of parameters set in the first parameter setting unit or the second parameter setting unit.

20. A user interface image capable of accepting the selection of a first module corresponding to a first control target and a second module corresponding to a second control target different from the first control target is displayed on the display unit, When the selection of the first module is received, a first command to be executed by the first processing unit based on the first module is obtained, When the selection of the second module is accepted, a second command with specifications different from the first command is obtained, which is to be executed by the second processing unit based on the second module. An information processing method that accepts input for selecting the execution of the first module or the second module, and for setting parameters in the first module or the second module, in the user interface image.

21. A program for causing a computer to execute the information processing method described in claim 20.

22. A computer-readable recording medium storing the program described in claim 21.

Images