Information processing device, information processing method, control system, manufacturing method of article, program, and recording medium

Abstract

To improve convenience of operation confirmation.SOLUTION: An information processing device includes a control part capable of executing at least two processing units (S210 to S216) respectively corresponding to pieces of processing related to operation of a control target, in a set order. The control part receives an operation input of designating a start position (202) and an end position (203) for setting a processing unit to be subjected to sub-routine among at least two processing units (S210 to S216).SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, a control system, an article manufacturing method, a program, and a recording medium. [Background technology]

[0002] For example, an information processing device that controls the operation of a control target such as a robot controls the control target by executing a program that processes multiple processing units (hereinafter referred to as "modules") in a preset order. When the program is executed, the order in which the multiple modules are executed is sometimes displayed on a display device as a flowchart so that the user can easily understand it (see Patent Document 1). The device in Patent Document 1 is configured to visually display the executed paths in the flowchart when the program is executed. In this way, the device in Patent Document 1 aims to improve the convenience of operation confirmation (debug) by making it easy to visually understand which paths in the flowchart have been executed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 9,910,761 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology of Patent Document 1, even if it is possible to determine in which path of the flowchart a problem (such as an error) has occurred, when, for example, the problem in that path is corrected and operation is checked, execution begins with the first module in that path. As a result, modules unrelated to the part whose operation is to be checked must be executed each time the operation is checked, and further improvements in convenience are desired.

[0005] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an information processing device, an information processing method, a control system, an article manufacturing method, a program, and a recording medium that can improve the convenience of operation checks. [Means for solving the problem]

[0006] One aspect of the present invention is an information processing device that includes a control unit that can execute at least two processing units, each corresponding to processing related to the operation of a control object, in a set order, and that accepts operational input that specifies a start position and an end position for setting one of the at least two processing units to be subroutineed.

[0007] One aspect of the present invention is an information processing method that includes a control unit capable of executing at least two processing units, each corresponding to processing related to the operation of a control object, in a set order, and that processes information using the control unit, characterized in that the information processing method includes: a reception step in which the control unit receives an operation input that specifies a start position and an end position for setting one of the at least two processing units to be subroutineed; and a setting step in which the control unit sets the subroutine based on the start position and the end position. [Effects of the Invention]

[0008] According to the present invention, it is possible to improve the convenience of checking the operation. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first embodiment. [Figure 2] 5 is a flowchart showing a procedure for subroutine creation according to the first embodiment. [Figure 3] FIG. 10 is a diagram showing a display screen of a flowchart in a state where a subroutine according to the first embodiment has been designated. [Figure 4]FIG. 10 is a diagram showing a display screen of a flowchart in a state where a subroutine including a conditional branch according to the first embodiment has been designated. [Figure 5] FIG. 10 is a diagram showing a display screen of a flowchart on which a subroutine according to the second embodiment can be specified. [Figure 6] FIG. 11 is a diagram showing a display screen of a module list that allows subroutines to be specified according to the third embodiment. [Figure 7] FIG. 10 is a diagram illustrating a configuration of a robot system according to a fourth embodiment. [Figure 8] FIG. 10 is a block diagram showing the configuration of a robot controller according to a fourth embodiment. [Figure 9] FIG. 11 is a diagram showing a display screen including a display section for a flowchart on which a subroutine can be specified according to the fourth embodiment, a display section for displaying the execution results of the subroutine, and a display section for parameter setting. [Figure 10] FIG. 10 is a diagram showing a case where the designation of the data reference destination is changed to local on the display screen of FIG. 9. DETAILED DESCRIPTION OF THE INVENTION

[0010] First Embodiment A first embodiment for carrying out the present invention will be described below with reference to Figs. 1 to 4. Fig. 1 is a block diagram showing the configuration of an information processing device according to the first embodiment. Fig. 2 is a flowchart showing the procedure for creating a subroutine according to the first embodiment. Fig. 3 is a diagram showing a display screen of a flowchart on which a subroutine according to the first embodiment can be specified. Fig. 4 is a diagram showing a display screen of a flowchart in a state in which a subroutine including a conditional branch according to the first embodiment has been specified.

[0011] [Configuration of information processing device] First, the configuration of an information processing device 801 according to the first embodiment will be described with reference to FIG. 1. As shown in FIG. 1, the information processing device 801 includes a CPU (Central Processing Unit) 802, which is an example of a processor. The CPU 802 is an example of a control unit. The information processing device 801 also includes a ROM (Read Only Memory) 803, a RAM (Random Access Memory) 804, and an HDD (Hard Disk Drive) 805, which serve as storage units. The information processing device 801 also includes a recording disk drive 806 and an interface 809, which is an example of an input / output unit. The information processing device 801 is connected via the interface 809 to external devices, such as a display 901, which serves as a display screen for displaying images, and a keyboard 902 and a mouse 903, which serve as input devices for accepting operation inputs. The information processing device 801 is also connected via the interface 809 to, for example, a robot device 100, which serves as a control target (or may be connected to, for example, a robot controller 701, which will be described in the fifth embodiment). The CPU 802, ROM 803, RAM 804, HDD 805, recording disk drive 806, and interface 809 are connected by a bus so as to be able to communicate information with one another. The display 901, keyboard 902, and mouse 903 are also connected to each part of the information processing device 801 via the interface 809 so as to be able to communicate information with one another.

[0012] The ROM 803 stores a basic program related to the operation of the computer. The RAM 804 is a storage device that temporarily stores various data, such as the results of calculations performed by the CPU 802. The HDD 805 stores the results of calculations performed by the CPU 802 and various data acquired from external sources, as well as a program 807 for executing various processes, which will be described later. The program 807 is application software that describes information on multiple processing units (hereinafter referred to as modules) used by the CPU 802 to execute processes related to the operation of a control object and sets information on the order in which the modules are executed. In other words, by executing the program 807 stored in the HDD 805, the CPU 802 executes processes related to the operation of the control object, such as the operation of the control object itself, calculation of the trajectory and position and orientation for the operation, and calculation of the position and orientation of a workpiece. The HDD 805 also has an area for recording information required for executing subroutines, as will be described later. The recording disk drive 806 can read various data and programs stored on a recording disk 850. Note that a subroutine may also be simply referred to as a routine. This subroutine setting allows you to group together (execute together) the processing units you want to execute repeatedly. Note that grouping includes the case where you want to execute at least one processing unit repeatedly, and also includes the case where you want to repeat one processing unit.

[0013] In this embodiment, the non-transitory computer-readable recording medium is the HDD 805, and the program 807 is recorded on the HDD 805, but this is not limiting. The program 807 may be recorded on any non-transitory computer-readable recording medium. Examples of recording media that can be used to provide the program 807 to a computer include a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a magnetic tape, and a non-volatile memory.

[0014] As described above, the robot device 100 as a control target is connected to the information processing device 801. As will be described in detail later, the information processing device 801 transmits command signals to the robot device 100 based on the processing results obtained by executing a plurality of modules, and receives signals from sensors of the robot device 100 and cameras serving as image capturing devices. In this way, the information processing device 801 executes processing related to the operation of the robot device 100.

[0015] [Outline of the subroutine creation procedure] Next, an outline of the procedure for creating a subroutine in a flowchart according to the first embodiment will be explained using Fig. 2. That is, in the first embodiment, a plurality of modules that perform processing related to the operation of the robot device 100 are generated as a flowchart based on a set execution order, and a subroutine is generated and executed in the flowchart. Note that the procedure shown in Fig. 2 shows the procedure of the work that a user (operator) performs when generating a subroutine.

[0016] As shown in Fig. 2, a user first determines the order in which multiple modules will be executed, and then creates a flowchart in which the multiple modules are arranged in that order (S101). This flowchart is generated, for example, by running a flowchart generation application on the information processing device 801 and arranging the multiple modules in the order in which they will be executed on the display screen of the display 901. This flowchart is a series of operation processes that start and complete, for example, one or more tasks (e.g., assembly work, bonding work, inspection work, photography work, transport work, disassembly work, cutting work, welding work, etc.) by a control object (robot, camera, machine tool, inspection equipment, etc.). In other words, this generated flowchart is an operation control program (hereinafter simply referred to as "program 807") that controls the operation of the control object.

[0017] Next, the user determines a module for which the operation is to be checked from the flowchart created on the display screen (S102). Note that this module for which the operation is to be checked is assumed to be a series of multiple consecutive modules, but it may also be a single module. Then, the user sets an entry point on the display screen upstream of the determined (target) module (i.e., the module for which the operation check is to start), in other words, performs an operation input to select and specify the module from which to start (S103). The entry point can also be described as the starting point for starting a subroutine. The user also sets an endpoint downstream of the determined module (i.e., the module for which the operation check is to end), in other words, performs an operation input to select and specify the module from which to end (S104). The endpoint can also be described as the ending point for ending the subroutine.

[0018] When the entry point and endpoint are set in the flowchart in this way, the CPU 802 sets the modules sandwiched between them as subroutines, that is, performs processing to make the selected modules into subroutines (S105). Note that this subroutine processing may be performed when the entry point and endpoint are set, or may be performed when a play button 204 (see FIG. 3) for starting an operation check displayed on the display screen, which will be described in detail later, is operated. Then, when the user operates the play button 204, the CPU 802 executes the modules sandwiched between the entry point and endpoint as subroutine processing (S106).

[0019] [Subroutine details] Next, the details of the process of specifying a subroutine from the flowchart according to the first embodiment and making it a subroutine will be explained using the display screen 200 that displays the flowchart shown in FIG. 3 as an example.

[0020] The CPU 802 of the information processing device 801 according to the first embodiment is capable of constructing a program 807 that controls the operation of the robotic device 100 to perform a series of tasks as described above. That is, the CPU 802 of the information processing device 801 generates and displays a display screen 200 on the display 901 as a GUI (Graphical User Interface). Then, on the display screen 200, a plurality of modules that cause the robotic device 100 to operate in response to user operation inputs using a keyboard 902, a mouse 903, etc. are arranged in a desired execution order and constructed as a flowchart. In this way, the program 807 expressed as a flowchart is constructed.

[0021] 3, for example, the user arranges modules A to C, a conditional branch module α, and a module D in this order on the display screen 200 controlled by the CPU 802. Then, the CPU 802 displays a flowchart on the display screen 200 in the flowchart display section 210, in which the modules are arranged in the set order. In the example shown in FIG. 3, the process for starting the flowchart is arranged as step S210, the process for module A as step S211, and the process for module B as step S212. In addition, the process for module C is arranged as step S213, the process for conditional branch module α as step S214, the process for module D as step S215, and the process for terminating the flowchart as step S216.

[0022] (When the subroutine section 211 is set) Subsequently, when the flowchart is displayed on the flowchart display section 210, the CPU 802 displays an entry / endpoint display section 201 as a reception setting section arranged in parallel to the flowchart on the display screen 200. This entry / endpoint display section 201 displays a plurality of arrows (i.e., a plurality of selection positions) pointing between all of the modules in the plurality of modules.

[0023] Here, for example, when the user performs an operation input as a first operation input to select any one of the multiple arrows in the entry / end point display unit 201, the CPU 802 sets the selected position as the entry point 202. The example shown in Figure 3 shows a case where the user selects an arrow upstream of module B (S212) and sets that position as the entry point 202.

[0024] Furthermore, for example, when the user performs an operation input as a second operation input to select any other one of the multiple arrows in the entry / endpoint display unit 201, the CPU 802 sets the selected position as the endpoint 203. The example shown in Fig. 3 shows a case where the user selects an arrow downstream of module C (S213) and sets that position as the endpoint 203.

[0025] In this way, the user selects two points from the multiple arrows in the entry / endpoint display section 201 (i.e., the reception process). Then, the one that comes first (upstream) is set as the entry point 202, and the one that comes later (downstream) is set as the endpoint 203. In other words, the entry point 202 is a start point set between modules, and the endpoint 203 is an end point set between modules. Explaining using the example of FIG. 3, a start processing point similar to start (S210) is placed between module A and module B, and an end processing point similar to end (S216) is placed between module C and module α.

[0026] Once the entry point 202 and the end point 203 have been set in this way, the CPU 802 sets the modules between them as subroutine section 211 (i.e., the setting process). In other words, the modules between them are separated from the other modules in the order of execution, and are made into subroutines. This subroutine may be executed when a play button 204, which will be described later, is pressed. In this case, the subroutine section 211 simply indicates the selection made by the user, and serves as a temporary selection range before being made into a subroutine.

[0027] A subroutine can also be defined as a single flowchart with its own start and end. In the example of display screen 200, the user creates a subroutine to check whether the operations of module B (S212) and module C (S213) are performed as intended (designed purpose). For this reason, the user sets a subroutine entry point 202 upstream of module B (S212) and a subroutine end point 203 downstream of module C (S213).

[0028] When the subroutine section 211 is set, the CPU 802 generates an image on the display screen 200 that allows the user to identify the subroutine section 211 from the other parts of the flowchart. Specifically, the color of the parts that display the modules that make up the subroutine section 211, that is, in the example shown in Figure 3, module B (S212) and module C (S213), is changed from the original color, that is, the subroutine section 211 is displayed in a color different from the other parts. Note that the example shown in Figure 3 shows an example in which the CPU 802 displays the modules of the subroutine section 211 in white or light gray, and displays the modules other than the subroutines in the flowchart in dark gray.

[0029] Furthermore, the CPU 802 generates and displays on the display screen 200 an image of a play button 204 that commands the start of execution of the subroutine section 211. For example, when a user presses the play button 204 on the display screen 200, module B (S212) and module C (S213) are set as the subroutine section 211, and execution of the processing thereof is started. Note that when a user presses a button as an image displayed on the display screen, this refers to, for example, performing an operation input to select the image using a mouse, a keyboard, or the like. Furthermore, without being limited to this, when the display screen is displayed on an image display device such as a touch panel, for example, the operation input is when a user presses a displayed button.

[0030] The CPU 802 also generates and displays on the display screen 200 an image of a step operation button 205 that commands the execution of each module in the subroutine section 211 one by one. For example, when a user presses the step operation button 205 on the display screen 200, first, only module B (S212) is executed, and then the execution process of the subroutine section 211 is paused. In the following description, pausing execution after executing one module in this manner is referred to as "step execution." For example, when the user presses the step operation button 205 again on the display screen 200 as the next operation, step execution is performed for module C (S213). Furthermore, if the subroutine section 211 is repeatedly executed a loop number of times, as described below, when the user next presses the step operation button 205, the process returns to the start of the subroutine section 211, and step execution is performed for module B (S212). Similarly, thereafter, step execution is performed in which only one module in the subroutine is executed each time the step operation button 205 is pressed. When step execution is performed, the module that will be processed is visually indicated to the user by changing the display color of the module. For example, the display screen 200 shows an example in which the module to be executed next in the subroutine section 211 is displayed in white, and modules in other subroutine sections are displayed in light gray.

[0031] Furthermore, the CPU 802 generates and displays, on the display screen 200, an image of a repeat button 206 that commands repeated execution of the processing of the subroutine section 211. Furthermore, the CPU 802 generates and displays, on the display screen 200, an image of a loop count display section 207 that serves as a set count display section for inputting and setting the number of loops as the set number of times the processing of the subroutine section 211 is to be repeatedly executed. For example, when a user inputs a numerical value into the loop count display section 207 on the display screen 200 to set the number of loops for repeating the execution of the subroutine section 211 and then presses the repeat button 206, the execution of the subroutine section 211 is repeated the set number of loops.

[0032] The CPU 802 then displays an error count display section 208 on the display screen 200, which indicates the number of errors that occurred in each module when the processing of the subroutine section 211 was repeatedly executed. For example, a user presses the repeat button 206 on the display screen 200 to repeatedly execute the subroutine section 211 the number of loops set in the loop count display section 207. In this case, if an error occurs in each module (if the processing of a module does not end correctly), the number of errors is counted and the number of errors that occurred is displayed in association with the module. The example of FIG. 3 indicates that five errors occurred in module C when the subroutine section 211 was repeatedly executed 30 times. In this way, by displaying the occurrence of errors during the execution of each module using the error count display section 208, it is possible to display the location of errors in the subroutine section 211 and the frequency of the errors. In this way, visualizing modules that are prone to errors improves the convenience for users when modifying a flowchart.

[0033] (When the subroutine section 212 is set) Next, a case where, for example, a user selects and sets a subroutine section 212 including a conditional branch will be described with reference to Fig. 4. As shown in Fig. 4, on the display screen 300, for example, the user selects an arrow upstream of module C (S213) and an arrow downstream of conditional branch module α (S214). Then, the entry point 202 and the end point 203 are set in the same manner as described above. As a result, the CPU 802 sets the subroutine section 212 including the conditional branch.

[0034] For example, in the flowchart on the display screen 300, the branch decision in the conditional branch module α (S214) changes depending on the processing result (output information) of the module C (S213). Therefore, by setting the entry point 202 and the end point 203 so as to include the conditional branch module α, the user can execute only the subroutine part 212 and check whether the branch in the module α has changed appropriately.

[0035] The CPU 802 generates and displays images of upper limit setting units 301 and 302 on the display screen 300, corresponding to the branch destination of the conditional branch module α. The user can set an upper limit number of times in each of the upper limit setting units 301 and 302. That is, suppose the user presses the repeat button 206 described above and the subroutine unit 212 is repeatedly executed. In this case, even if the number of times set in the loop count display unit 207 has not been reached, when the upper limit number of times set in the upper limit setting units 301 and 302 is reached, the execution of the subroutine unit 212 is terminated.

[0036] For example, in the display screen 300 shown in FIG. 4, the upper limit setting section 302 is set to "10," and the upper limit setting section 301 is not set. Furthermore, the loop count display section 207 is set to "30" as the number of loops. In this state, when the user presses the repeat button 206, the execution of the subroutine section 212 is repeated. Then, depending on the processing result (i.e., the judgment result) of module C (S213), the number of judgments for each of the cases where the result is "yes" and the case where the result is "no" in the conditional branch module α is counted. The counted number is displayed next to each of the upper limit setting sections 301 and 302, but since the display screen 300 shown in FIG. 4 shows the state before the execution of the subroutine section 212, "-" is displayed. Furthermore, when the upper limit setting section 302 is not set, such as when the result is "no" in the conditional branch module α shown in FIG. 4, it is not necessary to count the number of judgments. Therefore, if an upper limit number is set in at least one of the plurality of upper limit setting sections, the CPU 801 will count the number of determinations for at least one of the determination results.

[0037] Then, when the execution of the subroutine unit 212 is repeated, if the number of times that the conditional branch module α returns "yes" reaches "10" times before the execution of the subroutine unit 212 reaches "30" times, the execution of the subroutine unit 212 ends at that point. In short, the upper limit setting units 301 and 302 have a function of ending the repeated execution of the subroutine unit 212 if a specific judgment result is output the set upper limit number of times during the repeated execution of the subroutine unit 212. Note that if the execution of the subroutine unit 212 reaches "30" times before the number of times that the conditional branch module α returns "yes" reaches "10", the execution of the subroutine unit 212 ends on that condition as well.

[0038] Such an upper limit setting can be used, for example, when checking how many times a "no" occurs during 10 "yes" results in module α of the conditional branch. For example, module B may photograph a workpiece using a camera, module C may perform image recognition of the workpiece, and module D may perform an operation using the robot device 100 based on the workpiece being recognized. In this case, a "no" result in module α of the conditional branch indicates that the workpiece has not been recognized, and it is desirable to minimize the number of "no" results during the 10 "yes" results. In checking the operation of this subroutine unit 212, it is possible to adjust parameters (e.g., edge search parameters, exposure time, etc.) in module C or module B depending on the number of "no" results during the 10 "yes" results.

[0039] In this embodiment, for example, if an endpoint set downstream of a module is avoided by a conditional branch, the processing is executed to the end of the flowchart. Specifically, for example, in the flowchart on the display screen 300, an endpoint is set downstream of module D (S215), and if the processing result of the conditional branch module α (S214) is "no," the processing is executed to the end of the flowchart. However, this is not limiting, and if the processing result of the conditional branch module α (S214) is "yes," module D may be executed to terminate the execution of the subroutine unit 212, and if the processing result is "no," the execution of the subroutine unit 212 may be terminated.

[0040] [Summary of the first embodiment] As described above, in the information processing device 801 according to the first embodiment, the CPU 802 receives an input from the user, for example, in which the user arranges and sets the order in which modules are to be executed as a flowchart. Then, a plurality of modules, each describing processing related to the operation of the robot device 100, can be executed in the set order. Here, the CPU 802 receives an operation input specifying a module to start (e.g., module B in FIG. 3) and a module to end (e.g., module C in FIG. 3) among the plurality of modules.

[0041] Furthermore, the CPU 801 can execute a subroutine in response to, for example, an operation input from the play button 204 by the user.

[0042] The information processing device 801 also includes an interface 809 that receives information regarding the reception of operation input by the user from, for example, a keyboard 902 or a mouse 903, and outputs information about a display image on the display screen to the display 901. This allows the information processing device 801 to receive operation input from the user and display a display image regarding the processing result, thereby achieving a function as a so-called GUI.

[0043] Furthermore, the CPU 802 generates a display screen that displays a plurality of modules and displays it on the display 901 or the like. The CPU 802 then accepts an entry point setting operation as a first operation input that specifies a module to start from on the display screen, and an end point setting operation as a second operation input that specifies a module to end from on the display screen. This allows the user to easily select and set a subroutine part visually from the display screen.

[0044] Furthermore, the CPU 802 generates a flowchart display unit 210 that displays a flowchart in which multiple modules are arranged in a set order on a display screen (for example, the display screen 200 in FIG. 3), allowing the user to visually and easily recognize the existence of multiple modules and the order in which they are executed.

[0045] Furthermore, the CPU 802 generates an image that allows a subroutine section (for example, the subroutine section 211 in FIG. 3) to be identified in the flowchart display section 210. This allows the user to easily visually recognize which module has been set as the subroutine section when the user sets an entry point and an end point.

[0046] Furthermore, when the CPU 802 receives an entry point and an end point in the flowchart display unit 210, it changes the display color of the images corresponding to the multiple modules from the starting module to the ending module, thereby allowing the subroutine part to be displayed on the display screen as an image that can be identified by the user.

[0047] Furthermore, the CPU 802 generates an entry / endpoint display unit 201 that is arranged in parallel with the flowchart in the flowchart display unit 210 and that displays an image of the position where the entry point is specified (for example, the arrow in FIG. 3) when an entry point is accepted. This allows the user to easily visually recognize the entry point.

[0048] Furthermore, the CPU 802 generates an entry / endpoint display section 201 that is arranged in parallel with the flowchart in the flowchart display section 210 and that displays an image of the position where the endpoint is specified (for example, the arrow in FIG. 3) when an endpoint is received. This allows the user to easily visually recognize the entry point. Note that, although the present embodiment has been described as displaying both the entry point and the endpoint in the entry / endpoint display section 201, it may also be configured so that these are displayed in separate sections.

[0049] Furthermore, the CPU 802 can repeatedly execute the subroutine part until the set number of loops is reached, for example, the number of loops set by the user in the loop count display unit 207. This allows the user to easily set the number of loops for which the user wants to check the operation of the subroutine part.

[0050] The CPU 802 also generates an image of the loop count display section 207 that displays the loop count on the display screen, and accepts an operation input for setting the loop count, allowing the user to set the loop count while visually recognizing it.

[0051] Furthermore, when the subroutine unit 212 includes a module (e.g., module α in FIG. 4) in which a conditional branch is written that branches and proceeds in the next order depending on the judgment result, the CPU 802 counts the number of judgments of at least one of the judgment results. Then, the CPU 802 repeatedly executes the subroutine until the number of judgments reaches the upper limit. This makes it possible to check the operation of the module based on the situation in which the upper limit is reached.

[0052] Furthermore, the CPU 802 generates an image of the upper limit setting section 302 as an upper limit display section that displays the upper limit on the display screen, and accepts an operation input to set the upper limit, thereby enabling the user to visually recognize and set the upper limit.

[0053] Furthermore, when the CPU 802 repeatedly executes the subroutine, it determines whether an error has occurred in each module in the subroutine and counts the number of times the error has occurred. Then, it generates an image of the error count display section 208, which displays the number of times each error has occurred at a position corresponding to each module on the display screen (for example, the display screen 200 in FIG. 3). This allows the user to visually and easily recognize in which module an error has occurred and the number of times the error has occurred.

[0054] Furthermore, the CPU 802 generates an image of a play button 204 on the display screen as a start command display portion for commanding the start of execution of a subroutine portion, and receives an operation input commanding the start of execution by detecting an operation of pressing the play button 204. This allows the user to command the start of execution of the subroutine portion while visually recognizing it.

[0055] Furthermore, the CPU 802 generates an image of a step operation button 205 as a pause display section that commands pausing each time each module of the subroutine section is executed on a display screen (for example, the display screen 200 in FIG. 3). Then, by detecting the operation of pressing the step operation button 205, the CPU 802 accepts an operation input that commands pausing each time each module is executed. This allows the user to execute each module one by one and check their operation.

[0056] In this embodiment, the format has been described in which an arrow symbol is selected in the entry / endpoint display unit 201. However, the present invention is not limited to this, and other formats may be used, such as providing a display unit for dragging and dropping pin-shaped icons as desired, or inverting the color of a clicked module to specify a subroutine unit.

[0057] Second Embodiment Next, a second embodiment, which is a partial modification of the first embodiment, will be described with reference to Fig. 5. Fig. 5 is a diagram showing a display screen of a flowchart on which a subroutine according to the second embodiment can be specified. In the description of this second embodiment, the same reference numerals are used for parts similar to those in the first embodiment, and their description will be omitted.

[0058] In the first embodiment, the subroutine sections 211 and 212 are set by setting an entry point 202 and an endpoint 203 between modules. In contrast, in the second embodiment, the subroutine section 211 is set by setting a breakpoint in a module and setting a module to start.

[0059] 5, on the display screen 400 controlled by the CPU 802, an image of a breakpoint display section 401 as a reception setting section is generated and displayed in parallel with the flowchart display section 210 that created the flowchart. The breakpoint display section 401 displays check boxes horizontally adjacent to each module, and the user can set a breakpoint 402 by selecting any check box.

[0060] For example, on the display screen 400 of FIG. 5, when the user selects a check box near module C (S213), a breakpoint 402 (i.e., an end position) is set in module C. Note that the breakpoint 402 is basically set at the end of the processing in module C; that is, when the subroutine section 211 is executed, the processing as module C is executed until completion. In other words, a state is created similar to an end point being set downstream of module C. Also, on the display screen 400 of FIG. 5, when the user selects the box for module B (S212) itself, the module is set as the module where the subroutine section 211 starts (i.e., a start position). In other words, a state is created similar to an entry point being set upstream of module B. This sets the subroutine section 211 from module B to module C. Therefore, as in the first embodiment, when the play button 204 is pressed in this state, the subroutine section 211 is executed (repeatedly executed), and the operation of the subroutine section 211 can be checked.

[0061] If only the checkbox for module C is selected, module C may be set as the start and end points of the subroutine. Alternatively, if the checkboxes for modules B and C are configured to be selectable, and these two checkboxes are selected, module B may be set as the start point of the subroutine and module C may be set as the end point of the subroutine.

[0062] 5, the set subroutine section 211 is displayed so that the user can visually identify it by the display color of the module B in which the entry point is set and by displaying the check box of the breakpoint 402. For example, on the display screen 400, the background color of the module B selected as the entry point is changed to white, and other unselected modules are displayed with a gray background.

[0063] [Summary of the second embodiment] In the information processing device 801 according to the second embodiment described above, the CPU 802 also receives an operation input for specifying a start module (for example, module B in FIG. 5) and an end module (for example, module C in FIG. 5) from among a plurality of modules. That is, the start module is specified by directly selecting a module, and the end module is specified by selecting a breakpoint 402. In this way, the start module to the end module are set as a subroutine section (for example, subroutine section 211 in FIG. 5). Therefore, a subroutine section that executes only the module whose operation is to be checked is constructed, and modules unrelated to the part whose operation is to be checked are not executed each time the operation is checked, thereby improving the convenience of the operation check.

[0064] Other than this, the configuration, operation, and effects of the second embodiment are the same as those of the first embodiment, and therefore a description thereof will be omitted.

[0065] <Third embodiment> Next, a third embodiment, which is a partial modification of the first and second embodiments, will be described with reference to Fig. 6. Fig. 6 is a diagram showing a display screen of a module list that allows subroutines to be specified according to the third embodiment. In the description of this third embodiment, the same reference numerals are used for parts that are the same as those in the first and second embodiments, and their description will be omitted.

[0066] In the above-described first and second embodiments, each module is generated and displayed in a flowchart format in the flowchart display section 210 on the display screens 200, 300, and 400. In contrast, in the present third embodiment, the CPU 801 generates and displays an image of the list display section 510 on the display screen 500, which generates and displays each module in a list format.

[0067] In detail, each module is displayed in a list in the list display section 510, and a check box image is generated and displayed for each module. The multiple modules displayed as a list are basically arranged from top to bottom in the order in which they are to be executed.

[0068] In the list of modules shown in Figure 6, when the user presses the ellipsis on the right side of each module's column, information about the execution order of that module is displayed, allowing the user to visually recognize the execution order. In particular, for conditional branch module α, pressing the ellipsis displays that if "yes" the program will proceed to module D, and if "no" the program will proceed to the end.

[0069] Each of the listed modules displays an image of a check box that the user can select arbitrarily. For example, when the user selects an arbitrary check box, a breakpoint is set in the module where the check box is displayed, and the area around the check box is generated and displayed as an image in the breakpoint display section 501.

[0070] For example, in the display screen 500 of FIG. 6, when the user selects the check box for module C (S213), a breakpoint display section 501 is set in module C. The breakpoint display section 501 is basically set at the end of the processing in module C. In other words, when the subroutine section 511 is executed, it is executed until the processing of module C is completed. In other words, it is in a state similar to when an end point (i.e., an end position) is set downstream of module C. Also, in the display screen 500 of FIG. 6, for example, when the user selects the list of module B (S212), it is set as the module where the subroutine section 511 starts. In other words, it is in a state similar to when an entry point (i.e., a start position) is set upstream of module B. This means that the subroutine section 511 from module B to module C is set. Therefore, as in the first and second embodiments, by pressing the play button 204 in this state, the subroutine section 511 is executed (repeatedly executed), and the operation of the subroutine section 511 can be confirmed.

[0071] 6, the set subroutine section 511 is displayed so that it can be visually identified by the user by the display color of the module B in which the entry point is set and the display of the breakpoint display section 501. For example, on the display screen 500, the background color of the module B selected as the entry point is changed to white, and other unselected modules are displayed with a gray background.

[0072] [Summary of the third embodiment] In the information processing device 801 according to the third embodiment described above, the CPU 802 also receives an operation input for specifying a start module (e.g., module B in FIG. 6) and an end module (e.g., module C in FIG. 6) from among a plurality of modules. That is, the start module is specified by directly selecting a module, and the end module is specified by selecting the breakpoint display section 501. In this way, the start module to the end module are set as a subroutine section (e.g., subroutine section 511 in FIG. 6). Therefore, a subroutine section that executes only the module whose operation is to be checked is constructed, and modules unrelated to the part whose operation is to be checked are not executed each time the operation is checked, thereby improving the convenience of the operation check.

[0073] Furthermore, the CPU 802 generates a list display section 510 that displays a plurality of modules as a list on a display screen (for example, the display screen 500 in FIG. 6), thereby enabling the user to visually and easily recognize the existence of a plurality of modules and the order in which they are executed.

[0074] Furthermore, the CPU 802 generates an image that makes it possible to identify the subroutine portion 511 in the list display portion 510. This allows the user to easily visually recognize which module has been set as the subroutine portion when the user sets a start module and an end module.

[0075] Furthermore, the CPU 802 changes the display color of the image corresponding to the module to be started when an operation to select a module is received as the first operation input in the list display section 510. This allows the module from which the user starts the subroutine section to be displayed as an identifiable image on the display screen.

[0076] Furthermore, CPU 802 displays an image indicating that the selection of a breakpoint has been accepted on the image corresponding to the module to be terminated when an operation to select a breakpoint is accepted as the second operation input in list display section 150. This allows the module at which the user will terminate the subroutine section to be displayed as an identifiable image on the display screen.

[0077] Other than this, the configuration, operation, and effects of the second embodiment are the same as those of the first embodiment, and therefore a description thereof will be omitted.

[0078] <Fourth embodiment> Next, a fourth embodiment, which is a partial modification of the first embodiment, will be described with reference to Figs. 7 to 10. Fig. 7 is a diagram showing the configuration of a robot system according to the fourth embodiment. Fig. 8 is a block diagram showing the configuration of a robot controller according to the fourth embodiment. Fig. 9 is a diagram showing a display screen including a display section for a flowchart in which subroutines according to the fourth embodiment can be specified, a display section for displaying the results of execution of the subroutines, and a display section for parameter settings. Fig. 10 is a diagram showing the case in which the designation of the data reference destination has been changed to local on the display screen of Fig. 9. Note that in the description of this fourth embodiment, the same reference numerals are used for parts similar to those in the first embodiment, and their description will be omitted.

[0079] In the fourth embodiment, the information processing device 801 according to the first embodiment performs operation check of the robot device 100 as a control target. Specifically, the flowchart described in the first embodiment controls the operation of the robot device 100 to image-recognize a workpiece and grip and transport the component 10, and a subroutine unit checks part of this operation. The following description will be given assuming that the information processing device 801 is incorporated into the robot system 1 as a control system together with the robot device 100.

[0080] [Robot system configuration] First, the schematic configuration of a robot system according to the fourth embodiment will be described with reference to Fig. 7. As shown in Fig. 7, the robot system 1 is an assembly system that automatically grasps and transports a component 10, for example, as an assembly workpiece, and assembles the component 10 on a workpiece (not shown). The robot system 1 generally includes a robot device 100 as a control target, a camera 900 as a control target or an imaging device, and an information processing device 801. The robot device 100 is fixedly supported on a base 13 and includes a robot arm (manipulator) 700 as a robot, and a robot controller 701 that controls the robot arm 700.

[0081] The robot device 100 also includes a robot hand 702, which is attached to the tip of the robot arm 700 and serves as an end effector for grasping (holding) the component 10. The robot hand 702 is not particularly limited in shape or structure as long as it can hold the component 10, and may, for example, have a structure that sucks the component 10. The robot hand 702 may also be equipped with a force sensor or the like as needed.

[0082] Furthermore, a component 10 is placed on a component stand 12 installed on a platform 13, and the robot system 1 includes a camera 900 as a control target located above the component stand 12, i.e., above the component 10. The camera 900 captures an image of a capture area (capture range) including at least the component 10 and acquires the image data of the actual image. The camera 900 may be a two-dimensional camera capable of outputting two-dimensional image data, or a three-dimensional camera such as a stereo camera capable of outputting three-dimensional image data. In this embodiment, the camera 900 is described as a fixed camera installed on the ceiling of a factory, for example. However, the camera 900 may also be an on-hand camera fixed to the robot hand 702 as long as it can capture an image of the capture area including the component 10. The image data captured by the camera 900 is sent to the robot controller 701 and processed as described in detail below. The information processing here refers to the robot controller 701 calculating command values ​​(such as the trajectory of the robot arm) for controlling the robot to assemble the component 11 into the component 10.

[0083] The robot system 1 configured as described above performs an assembly operation in which the part 10 grasped by the robot hand 702 of the robot device 100 is assembled to a workpiece (not shown). In this way, the robot system 1 uses the robot device 100 to perform the assembly operation of assembling the part 10 to the workpiece, thereby manufacturing the workpiece to which the part 10 is assembled as an article. In other words, the robot system 1 uses the robot device 100 to execute a manufacturing method for manufacturing an article in which the part 10 is assembled to the workpiece.

[0084] [Robot controller configuration] Next, the configuration of the robot controller 701 will be described with reference to Fig. 8. As shown in Fig. 8, the robot controller 701 includes a CPU 704, which is an example of a processor. The CPU 704 is an example of a control unit. The robot controller 701 also includes a ROM 705, a RAM 706, and an HDD 707 as storage units. The robot controller 701 also includes a recording disk drive 708 and an interface 709, which is an input / output interface. The CPU 704, ROM 705, RAM 706, HDD 707, recording disk drive 708, and interface 709 are connected by a bus so that they can communicate with each other.

[0085] The ROM 705 stores a basic program related to the operation of the computer. The RAM 705 is a storage device that temporarily stores various data, such as the results of calculations performed by the CPU 704. The HDD 707 stores the results of calculations performed by the CPU 704 and various data acquired from the outside, as well as a program 710 that causes the CPU 704 to execute various processes related to the operation of the robot device 100. The program 710 is application software that enables the CPU 704 to perform various processes. Therefore, the CPU 704 executes the program 710 stored in the HDD 707 to perform control processing and control the operation of the robot arm 700. The recording disk drive 708 can read various data, programs, etc. stored on a recording disk 750.

[0086] In this embodiment, the non-transitory computer-readable recording medium is the HDD 707, and the program 710 is recorded on the HDD 707, but this is not limiting. The program 710 may be recorded on any non-transitory computer-readable recording medium. Examples of recording media that can be used to provide the program 710 to a computer include a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a magnetic tape, and a non-volatile memory.

[0087] The robot controller 701 is also connected to the camera 900, the robot arm 700, and the above-mentioned information processing device 801. As will be described in detail later, information relating to the operation processing of the robot device 100 (operation commands for the robot arm 700 and the camera 900) is transmitted from the information processing device 801 to the robot controller 701. The camera 900 transmits captured image data to the robot controller 701, and the image data is processed by the program 710. The processing results are transmitted to the information processing device 801.

[0088] In the fourth embodiment, the information processing device 801 (CPU 802) executes the process of calculating the operation of the robot device 100 (robot arm 700, camera 900, etc.). The description will be given assuming that the robot controller 701 (CPU 704) actually controls the operation of the robot arm 700 and camera 900 based on the information of the calculation results transmitted from the information processing device 801. However, this is not limiting, and these processes may be executed by one computer, i.e., one CPU, or may be executed by three or more computers, i.e., three or more CPUs.

[0089] [Display screen configuration] Next, a display screen 600 in which an image is generated by the CPU 802 and displayed on the display 901 will be described with reference to FIG. 9 . As shown in FIG. 9 , the display screen 600 is broadly configured to include a control content display section 601, an execution result display section 602, and a parameter setting display section 603. The control content display section 601 includes a flowchart display section 210 that displays a flowchart for controlling the robot device 100 (hereinafter referred to as a "control flowchart"), and an entry / endpoint display section 201. A user can generate a control flowchart by arranging each module in the flowchart display section 210. The execution result display section 602 displays the results of each module when the control flowchart is executed. The parameter setting display section 603 displays various parameters used in each module of the control flowchart, and the user can set the various parameters by selecting or inputting values ​​for them. Note that the parameter setting display section 603 allows various parameters used in each module of the control flowchart to be set collectively through operation input.

[0090] [Control flow chart overview] Next, an example will be described in which a user generates a flowchart for controlling the gripping operation of the robot device 100 to grip the part 10 in the information processing device 801. As shown in Fig. 9, the user generates a flowchart for controlling the robot device 100 in the flowchart display unit 210 by arranging a plurality of modules in the order of execution, for example, as shown in Fig. 9. Below, a case in which the entire flowchart is executed will be described, and a case in which a subroutine portion is set and executed will be described later.

[0091] In this flowchart, "START" (S210) is a processing module that starts control. "ROBOT CONTROL 1" (S211) is a processing module that performs an operation to move the robot arm 700 to an initial position for grasping the component 10 by the robot arm 700. Note that the component 10 may be set on the component stand 12 by the robot arm of a robot device adjacent to the robot device 100, for example, or may be set manually by an operator.

[0092] "Image capture" (S212) is a processing module for capturing an image of the component 10 on the component stand 12 using the camera 900. The CPU 802 executes "Image capture" (S212) while controlling the camera 900 using the parameters set in the parameter setting display unit 603. Information on the execution result, such as the image captured by the camera 900 and numerical values, as the execution result of this "Image capture" (S212) is displayed in the execution result display unit 602.

[0093] "Component Recognition" (S213) is a module that performs image analysis using the image captured in "Image Capture" (S212). That is, "Component Recognition" (S213) is a processing module that detects the edges of the contour of the component 10 captured in the image, matches it with the model image, and recognizes the position and posture of the component 10 to be grasped. The CPU 802 executes "Component Recognition" (S213) using parameters set in the parameter setting display unit 603. Information on the execution result of this "Component Recognition" (S213), such as an image in which the edges of the component 10 are detected and numerical values, is displayed in the execution result display unit 602.

[0094] "Mobility Determination" (S214) is a processing module that determines whether or not the robot hand 702 of the robot arm 700 can grip the part recognized in "Part Recognition" (S213). If the CPU 802 determines that the part 10 recognized in "Part Recognition" (S213) can be gripped, the CPU 802 determines "yes" and proceeds to "Robot Control 2" (S215). On the other hand, if the CPU 802 determines that the part 10 cannot be gripped, such as if the part 10 could not be recognized in "Part Recognition" (S213), the CPU 802 determines "no" and proceeds directly to "End" (S216), where execution of the control flowchart is terminated.

[0095] "Robot control 2" (S215) is a processing module that controls the operation of gripping the part 10 with the robot hand 702 when it is determined in "movability determination" (S214) that the part 10 can be gripped. The CPU 802 controls the robot arm 700 and the robot hand 702 based on the information about the position (coordinates) and posture of the part 10 recognized in "part recognition" (S213), and executes the operation of gripping the recognized part 10 on the part stand 12 with the robot hand 702. Then, when the gripping operation of the part 10 is completed, the process proceeds to "end" (S216), and execution of the control flowchart ends. Note that this control flowchart explains the flowchart up to gripping the part 10, but after this, by executing other flowcharts, the operation of assembling the part to a workpiece (not shown) is performed, and assembly work is performed.

[0096] [Subroutine setting and execution] Next, the setting and execution of the subroutine 211 for performing only a part of the operation check of the above control flowchart will be explained. The setting and execution of this subroutine 211 are the same as those in the first embodiment, so the explanation will be simplified.

[0097] When the user performs an operation input to select an entry point 202 and an endpoint 203 in the entry / endpoint display section 201, the CPU 802 sets the module between them as a subroutine section 211. In the example shown in Fig. 9, two modules, "image capture" (S212) and "component recognition" (S213), are set as the subroutine section 211. Then, when the user performs an operation input to press the play button 204, the CPU 802 repeatedly executes the subroutine section 211 for the set number of loops.

[0098] When the subroutine unit 211 is executed, the CPU 802 displays the execution results of all the subroutine modules together as images and numerical values ​​on the execution result display unit 602. In the example shown in Fig. 9, the execution result display unit 602 displays the execution results of "image capture" (S212), such as the execution time, judgment results, and captured image. The execution result display unit 602 also displays the execution results of "component recognition" (S213), such as the execution time, judgment results, recognized image, coordinates of the component 10, and a score indicating the success rate of recognition.

[0099] The parameter settings used in each module of the subroutine section 211 can be edited all at once by displaying a list in the parameter setting display section 603. Furthermore, after executing the subroutine section 211, by selecting the corresponding module in the parameter setting display section 603, it is possible to display only the results and parameters of the selected module.

[0100] 9, the exposure time and gain values ​​are displayed as configurable parameters used in "image capture" (S212) in the parameter setting display section 603. The image to be used (hereinafter referred to as "used image") and edge threshold values ​​are displayed as configurable parameters used in "component recognition" (S213) in the parameter setting display section 603.

[0101] Incidentally, in order to execute the subroutine section 211 and check its operation, even if one wishes to check the operation of only the module of the subroutine section 211, data obtained in modules upstream of the subroutine section 211 in the flowchart is required. For example, in the display screen 600 shown in Fig. 9, when "Component Recognition" (S213) is executed, the image to be used, which is the execution result of the upstream "Image Capture" (S212), is required. Therefore, even if one wishes to set the subroutine section after "Component Recognition" (S213) and check its operation without executing "Image Capture" (S212), the image to be used must be input.

[0102] Therefore, the image to be used can be specified in the image to be used setting section 604 in the parameter setting display section 603, thereby enabling the execution of "component recognition" (S213). For example, in the example shown in Fig. 9, the user can specify how to reference data in the image to be used setting section 604 in the parameter setting display section 603. First, the user selects a data reference method in the parameter selection method setting section 605.

[0103] There are three options: "module," "local," and "global." If the user selects "module," the execution result data obtained in "image capture" (S212) during the previous execution of the flowchart is referenced and used in "component recognition" (S213). When "module" is selected, the control flowchart is executed once from the beginning during the first execution after creation, and the execution results of each module at that time are stored in a storage unit such as the HDD 805. If the user selects "local," the data previously stored in a storage unit such as the HDD 805 of the information processing device 801 by the user's operation is referenced and used only in "component recognition" (S213) of the current flowchart. If the user selects "global," the data stored in a storage unit such as the HDD 805 of the information processing device 801 is referenced and can be used not only in "component recognition" (S213) of the current flowchart but also in modules of other flowcharts.

[0104] 9, the user selects "Module" and selects to refer to the execution results of the module "Image Capture" (S212) in the reference module setting section 606. Then, the data selection section 607 indicates that the data of "Captured Image" from the execution results of "Image Capture" (S212) is to be referenced.

[0105] On the other hand, when the user specifies data by selecting "local" or "global," the display in the image to be used setting unit 604 changes as shown in Fig. 10. Fig. 10 shows a state in which "local" is selected in the parameter selection method setting unit 605. In this case, the image variable to be referenced is specified as, for example, "image variable 001" in the data selection unit 607, which indicates that the data of "image variable 001" that the user has stored in advance in the memory unit of the information processing device 801 by operating input is to be referenced. In this way, by using data prepared in advance inside the information processing device 801, it is possible to perform "component recognition" (S213) without performing "image capture" (S212).

[0106] It should be noted that the setting of reference data described above is not limited to the parameters of "component recognition" (S213), i.e., the reference data is not limited to image data, but various numerical values, conditional branches and their results can also be set in the same way.

[0107] In the fourth embodiment described above, the display screen 600 has been described as an example in which a control flowchart for controlling the operation of the robot device 100 is made into a subroutine. However, the control target controlled by the information processing device 801 is not limited to the robot device 100, and any display screen having a function or process for making a subroutine and executing the operation check of the control target may be used.

[0108] [Summary of the Fourth Embodiment] In the information processing device 801 according to the fourth embodiment described above, the CPU 802 also receives an operation input specifying a module to start from among a plurality of modules (for example, image capture in FIG. 9) and a module to end (for example, image recognition in FIG. 9). This sets the modules from the start to the end as a subroutine section (for example, subroutine section 211 in FIG. 9). Therefore, a subroutine section that executes only the module whose operation is to be checked is constructed, and therefore modules unrelated to the part whose operation is to be checked are not executed each time the operation is checked, thereby improving the convenience of the operation check.

[0109] Furthermore, when executing a subroutine, the CPU 802 executes the subroutine (for example, component recognition) using as a parameter information (for example, a captured image) resulting from executing a module other than the subroutine (for example, image capture) before executing the subroutine. This makes it unnecessary to execute a module other than the subroutine (for example, image capture) every time the subroutine (for example, component recognition) is executed, and makes it possible to execute only the subroutine.

[0110] Furthermore, when executing a subroutine, the CPU 802 executes the subroutine portion using the information stored in the storage portion as a parameter. This also makes it unnecessary to execute a module other than the subroutine (e.g., image capture) every time a subroutine portion (e.g., component recognition) is executed, and makes it possible to execute only the subroutine portion.

[0111] Furthermore, the CPU 802 generates an execution result display section 602 that displays the execution result of the subroutine section on the display screen 600. This allows the user to visually check the execution result of the subroutine section and visually confirm the operation of the robot device 100.

[0112] Furthermore, when the subroutine section includes a module describing a process for executing image analysis, such as component recognition, the CPU 802 displays the execution results of the image analysis in the execution result display section 602. This allows the user to confirm how the image analysis was performed and also the reliability of the image analysis.

[0113] <Possibilities for other embodiments> In the first to fourth embodiments described above, the robot device 100 is operated as a control target, and a subroutine is used to check the partial operation of the robot device 100. However, the control target is not limited to a robot device, and may be any device.

[0114] In the first to fourth embodiments, when a subroutine is set in a flowchart or list, an entry point and an end point or a breakpoint are set. However, the present invention is not limited to these, and any method can be used as long as it can set a module that starts and a module that ends a subroutine. Furthermore, the starting module and the ending module may be the same.

[0115] Furthermore, the display screens 200, 300, 400, and 600 described in the first to fourth embodiments are merely examples, and any display format may be used. For example, while the color of modules is changed to make subroutines in a flowchart identifiable, the display screens may be changed in brightness or light. Furthermore, modules other than subroutines may be hidden or erased. Images of the play button 204, step operation button 205, repeat button 206, loop count display unit 207, and the like are merely examples, and may be in any shape. Furthermore, the layout and display format of the control content display unit 601, execution result display unit 602, and parameter setting display unit 603 may be any.

[0116] The present disclosure can also be realized by providing a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0117] [Disclosure of the Present Embodiment] [Configuration 1] a control unit that can execute at least two processing units, each corresponding to a process related to an operation of a control target, in a set order; the control unit accepts an operation input specifying a start position and an end position for setting a processing unit to be subroutine among the at least two processing units; 1. An information processing device comprising: [Configuration 2] The control unit executes the subroutine. 2. The information processing device according to configuration 1, [Configuration 3] an input / output unit that receives information regarding reception of the operation input and outputs information about a display image to a display device; 3. The information processing device according to configuration 2. [Configuration 4] The control unit generating a display image displaying the at least two processing units and displaying the image on the display device; receiving a first operation input that specifies the start position on the display image and a second operation input that specifies the end position on the display image; 4. The information processing device according to configuration 3. [Configuration 5] the control unit generates a flowchart display unit that displays, in the display image, a flowchart in which the at least two processing units are arranged in the set order. 5. The information processing device according to configuration 4. [Configuration 6] the control unit generates an image on the flowchart display unit that allows the subroutine to be identified. 6. The information processing device according to configuration 5. [Configuration 7] the control unit changes, in the flowchart display unit, a display color of images corresponding to a plurality of processing units from the start position to the end position when the first operation input and the second operation input are received. 7. The information processing device according to configuration 6. [Configuration 8] the control unit generates, in the flowchart display unit, a reception setting unit that is arranged in parallel with the flowchart and that displays an image of a position where the first operation input is designated when the first operation input is received. 8. The information processing device according to any one of configurations 5 to 7. [Configuration 9] the control unit generates, in the flowchart display unit, a reception setting unit that is arranged in parallel with the flowchart and that displays an image of a position where the second operation input is designated when the second operation input is received. 9. The information processing device according to any one of configurations 5 to 8. [Configuration 10] the control unit generates a list display portion that displays the at least two processing units as a list in the display image. 5. The information processing device according to configuration 4. [Configuration 11] the control unit generates an image in the list display unit that makes the subroutine identifiable. 11. The information processing device according to configuration 10. [Configuration 12] the control unit changes a display color of the image corresponding to the start position on the list display unit when the first operation input is received. 12. The information processing device according to configuration 10 or 11. [Configuration 13] when the second operation input is received, the control unit displays, on the list display unit, an image indicating that the second operation input has been received on the image corresponding to the end position. 13. The information processing device according to any one of configurations 10 to 12. [Configuration 14] The control unit is capable of repeatedly executing the subroutine until the number of times reaches a set number. 14. The information processing device according to any one of configurations 3 to 13. [Configuration 15] the control unit generates a set number display unit that displays the set number of times in the display image, and receives an operation input for setting the set number of times. 15. The information processing device according to configuration 14. [Configuration 16] When the subroutine includes a processing unit in which a conditional branch that branches and proceeds in the next order depending on a determination result is described, the control unit counts the number of determinations of at least one of the determination results, and repeatedly executes the subroutine until the number of determinations reaches an upper limit number. 16. The information processing device according to configuration 15. [Configuration 17] the control unit generates an upper limit number display section that displays the upper limit number of times in the display image, and receives an operation input for setting the upper limit number of times. 17. The information processing device according to configuration 16. [Configuration 18] the control unit, when repeatedly executing the subroutine, determines an error in each processing unit in the subroutine, counts the number of times an error has occurred, and generates an error count display unit that displays the number of times an error has occurred at a position corresponding to each processing unit in the display image. 18. The information processing device according to any one of configurations 14 to 17. [Configuration 19] the control unit generates a start command display portion instructing the start of execution of the subroutine in the display image, and accepts an operation input instructing the start of execution. 19. The information processing device according to any one of configurations 14 to 18. [Configuration 20] the control unit generates a pause display unit in the display image that instructs to pause each time each processing unit of the subroutine is executed, and accepts an operation input instructing to pause each time each processing unit is executed. 20. The information processing device according to configuration 19. [Configuration 21] When the control unit executes the subroutine using a parameter, the control unit executes the subroutine using, as the parameter, information on a result of executing a processing unit other than the subroutine before executing the subroutine. 21. The information processing device according to any one of configurations 3 to 20. [Configuration 22] a storage unit that stores information about the parameters; When the control unit executes the subroutine using parameters, the control unit executes the subroutine using information stored in a storage unit as the parameters. 21. The information processing device according to any one of configurations 3 to 20. [Configuration 23] the control unit generates an execution result display unit that displays an execution result of the execution of the subroutine in the display image. 23. The information processing device according to any one of configurations 3 to 22. [Configuration 24] When the subroutine includes a processing unit describing a process for executing image analysis, the control unit displays the execution result of the image analysis on the execution result display unit. 24. The information processing device according to configuration 23. [Method 25] An information processing method including a control unit capable of executing at least two processing units, each corresponding to a process related to an operation of a control target, in a set order, and processing information by the control unit, comprising: a receiving step in which the control unit receives an operation input specifying a start position and an end position for setting a processing unit to be subroutine among the at least two processing units; a setting step in which the control unit sets the subroutine based on the start position and the end position. 1. An information processing method comprising: [Configuration 26] An information processing device according to any one of configurations 1 to 24; a control object whose operation is controlled in accordance with the processes described in each of the at least two processing units; A control system comprising: [Configuration 27] the controlled object includes a robot device, 27. The control system of claim 26. [Configuration 28] The control target includes an imaging device. 28. The control system according to configuration 26 or 27. [Method 29] 29. Manufacturing an article using the control system according to any one of configurations 26 to 28. A method for manufacturing an article. [Configuration 30] A program for causing a computer to execute the information processing method described in Method 25. [Configuration 31] A computer-readable recording medium having the program according to configuration 30 recorded thereon. [Explanation of symbols]

[0118] 1...Robot system (control system) / 100...Robot device (control target) / 200...Display screen (display image) / 201...Entry / endpoint display section (reception setting section) / 202...Entry point (start position) / 203...Endpoint (end position) / 204...Play button (start command display section) / 205...Step operation button (pause display section) / 207...Loop count display section (set count display section) / 208...Error count display section / 210...Flowchart display section / 211...Subroutine section (subroutine) / 212...Subroutine section (subroutine) / 300...Display screen (display image) / 301...Upper limit setting section (Upper limit number display unit) / 302... Upper limit setting unit (Upper limit number display unit) / 400... Display screen (display image) / 401... Breakpoint display unit (reception setting unit) / 402... Breakpoint display unit (end position) / 501... Breakpoint display unit (end position) / 510... List display unit / 600... Display screen (display image) / 602... Execution result display unit / 801... Information processing device / 802... CPU (control unit) / 805... HDD (storage unit) / 807... Program / 809... Interface (input / output unit) / 900... Camera (control target, imaging device) / 901... Display (display device) / S210 to S216... Module (processing unit)

Claims

1. a control unit capable of executing at least two processing units, each corresponding to a process related to an operation of a control target, in a set order; the control unit accepts an operation input specifying a start position and an end position for setting a processing unit to be subroutine among the at least two processing units; 1. An information processing device comprising:

2. The control unit executes the subroutine.

2. The information processing apparatus according to claim 1, wherein:

3. an input / output unit that receives information regarding reception of the operation input and outputs information about a display image to a display device; 3. The information processing apparatus according to claim 2, wherein:

4. The control unit generating a display image displaying the at least two processing units and displaying the image on the display device; receiving a first operation input that specifies the start position on the display image and a second operation input that specifies the end position on the display image; 4. The information processing apparatus according to claim 3,

5. the control unit generates a flowchart display unit that displays, in the display image, a flowchart in which the at least two processing units are arranged in the set order.

5. The information processing apparatus according to claim 4,

6. the control unit generates an image on the flowchart display unit that allows the subroutine to be identified.

6. The information processing apparatus according to claim 5,

7. the control unit changes a display color of images corresponding to a plurality of processing units from the start position to the end position when the first operation input and the second operation input are received on the flowchart display unit.

7. The information processing apparatus according to claim 6,

8. the control unit generates, in the flowchart display unit, a reception setting unit that is arranged in parallel with the flowchart and that displays an image of a position where the first operation input is designated when the first operation input is received.

6. The information processing apparatus according to claim 5,

9. the control unit generates, in the flowchart display unit, a reception setting unit that is arranged in parallel with the flowchart and that displays an image of a position where the second operation input is designated when the second operation input is received.

6. The information processing apparatus according to claim 5,

10. the control unit generates a list display portion that displays the at least two processing units as a list in the display image.

5. The information processing apparatus according to claim 4,

11. the control unit generates an image in the list display unit that makes the subroutine identifiable.

11. The information processing apparatus according to claim 10,

12. the control unit changes a display color of the image corresponding to the start position on the list display unit when the first operation input is received.

11. The information processing apparatus according to claim 10,

13. the control unit, when receiving the second operation input, displays, on the list display unit, an image indicating that the second operation input has been received on the image corresponding to the end position.

11. The information processing apparatus according to claim 10,

14. The control unit is capable of repeatedly executing the subroutine until the number of times reaches a set number.

4. The information processing apparatus according to claim 3,

15. the control unit generates a set number display unit that displays the set number of times in the display image, and receives an operation input for setting the set number of times.

15. The information processing apparatus according to claim 14,

16. When the subroutine includes a processing unit in which a conditional branch that branches and proceeds in the next order depending on a determination result is described, the control unit counts the number of determinations of at least one of the determination results, and repeatedly executes the subroutine until the number of determinations reaches an upper limit number.

16. The information processing apparatus according to claim 15,

17. the control unit generates an upper limit number display section that displays the upper limit number of times in the display image, and receives an operation input for setting the upper limit number of times.

17. The information processing apparatus according to claim 16,

18. the control unit, when repeatedly executing the subroutine, determines an error in each processing unit in the subroutine, counts the number of times an error has occurred, and generates an error count display unit that displays the number of times an error has occurred at a position corresponding to each processing unit in the display image.

15. The information processing apparatus according to claim 14,

19. the control unit generates a start command display portion instructing the start of execution of the subroutine in the display image, and accepts an operation input instructing the start of execution.

15. The information processing apparatus according to claim 14,

20. the control unit generates a pause display portion in the display image that instructs to pause each time each processing unit of the subroutine is executed, and accepts an operation input instructing to pause each time each processing unit is executed.

20. The information processing apparatus according to claim 19,

21. When the control unit executes the subroutine using a parameter, the control unit executes the subroutine using, as the parameter, information on a result of executing a processing unit other than the subroutine before executing the subroutine.

4. The information processing apparatus according to claim 3,

22. a storage unit that stores information about the parameters; When the control unit executes the subroutine using parameters, the control unit executes the subroutine using information stored in a storage unit as the parameters.

4. The information processing apparatus according to claim 3,

23. the control unit generates an execution result display unit that displays an execution result of the execution of the subroutine in the display image.

4. The information processing apparatus according to claim 3,

24. When the subroutine includes a processing unit describing a process for executing image analysis, the control unit displays the execution result of the image analysis on the execution result display unit.

24. The information processing apparatus according to claim 23,

25. An information processing method including a control unit capable of executing at least two processing units, each corresponding to a process related to an operation of a control target, in a set order, and processing information by the control unit, comprising: a receiving step in which the control unit receives an operation input specifying a start position and an end position for setting a processing unit to be subroutineed among the at least two processing units; a setting step in which the control unit sets the subroutine based on the start position and the end position.

1. An information processing method comprising:

26. The information processing device according to claim 1 ; a control object whose operation is controlled in accordance with the processes described in each of the at least two processing units; A control system comprising:

27. the controlled object includes a robot device, 27. The control system of claim 26.

28. The control target includes an imaging device.

27. The control system of claim 26.

29. 27. Manufacturing an article using the control system of claim 26. A method for manufacturing an article.

30. A program for causing a computer to execute the information processing method according to claim 25.

31. A computer-readable recording medium on which the program according to claim 30 is recorded.

Images