Control device, control method, and program

Abstract

To enable a robot device to be controlled more properly.SOLUTION: A control device comprises: a motion generation unit that generates a control command for causing a reverse direction motion for changing an operation state of a robot device from a target state to an arbitrary state different from the target state; an acquisition unit that acquires measurement information obtained by using a sensor for performing measurement related to the operation state of the robot device; and a data collection unit that repeatedly collects data including a set of the control command and the measurement information during execution of the reverse direction motion in accordance with the control command. The sensor includes at least one among a visual sensor and a force sensor.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a control method, and a program for controlling a robot device. [Background technology]

[0002] Conventionally, control devices for controlling robotic devices have been widely used. In such control devices, a technique is known in which a visual sensor and a force sensor are used to measure the motion state of the robotic device and to control the motion state of the robotic device to a target state (see, for example, Patent Document 1).

[0003] Furthermore, conventionally, a teaching method has been adopted as a method for learning the movements of a robot device, in which the robot device is manually operated and a series of movements to be executed is recorded. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 7295344 Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional technologies, new teaching is required whenever the work environment and / or the target object changes even slightly, making it difficult to efficiently collect data that can be used to control the operation of the robot device.

[0006] The present disclosure provides a control device, a control method, and a program that enable more appropriate control of a robot device. [Means for solving the problem]

[0007] A control device according to a first aspect of the present disclosure includes: an operation generating unit that generates a control command to execute a reverse operation that changes the operating state of a robotic device from a target state to an arbitrary state different from the target state; an acquiring unit that acquires measurement information obtained using a sensor for measuring the operating state of the robotic device; and a data collecting unit that repeatedly collects data including a set of the control command and the measurement information while the reverse operation is being executed in accordance with the control command, wherein the sensor includes at least one of a visual sensor and a force sensor.

[0008] A control method according to a second aspect of the present disclosure includes generating a control command for executing a reverse operation that changes the operating state of a robotic device from a target state to an arbitrary state different from the target state, acquiring measurement information obtained using a sensor for measuring the operating state of the robotic device, and repeatedly collecting data including a set of the control command and the measurement information while the reverse operation is being executed in accordance with the control command, wherein the sensor includes at least one of a visual sensor and a force sensor.

[0009] A program according to a third aspect of the present disclosure causes a control device to generate a control command for executing a reverse operation that changes the operating state of a robotic device from a target state to an arbitrary state different from the target state, acquire measurement information obtained using a sensor for measuring the operating state of the robotic device, and repeatedly collect data including a set of the control command and the measurement information while the reverse operation is being executed in accordance with the control command, wherein the sensor includes at least one of a visual sensor and a force sensor. [Effects of the Invention]

[0010] According to one aspect of the present disclosure, it is possible to provide a control device, a control method, and a program that enable more appropriate control of a robot device. [Brief explanation of the drawings]

[0011] [Figure 1]1 is a diagram illustrating an example of a system configuration of a control system including a control device according to an embodiment. [Figure 2] 2 is a block diagram showing a functional block configuration of the control device according to the first embodiment. FIG. [Figure 3] FIG. 3 is a diagram for explaining an example of data collection by the control device according to the first embodiment. [Figure 4] FIG. 3 is a diagram for explaining an example of data collection by the control device according to the first embodiment. [Figure 5] FIG. 4 is a flow chart for explaining an example of data collection by the control device according to the first embodiment. [Figure 6] FIG. 2 is a diagram illustrating an example of a setting library stored in a control device according to the embodiment. [Figure 7] FIG. 10 is a block diagram showing a functional block configuration of a control device according to a second embodiment. [Figure 8] FIG. 10 is a flow chart for explaining an example of a control flow by a control device according to a second embodiment. [Figure 9] 10A and 10B are diagrams for explaining a specific example of control during the operation of "mounting components on a board" as control by the control device according to the second embodiment. [Figure 10] FIG. 10 is a block diagram showing a functional block configuration of a control device according to a third embodiment. [Figure 11] 10A and 10B are diagrams for explaining a specific example of control during the operation of "mounting components on a board" as control by the control device according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] The embodiments will be described with reference to the drawings, in which the same or similar parts are denoted by the same or similar reference numerals.

[0013] (1) Overview of the embodiment A control device according to an embodiment includes an operation generating unit that generates a control command to execute a reverse operation that changes the operating state of a robot apparatus from a target state to an arbitrary state different from the target state, an acquiring unit that acquires measurement information obtained using a sensor for measuring the operating state of the robot apparatus, and a data collecting unit that repeatedly collects data including a set of the control command and the measurement information while the reverse operation is being executed in accordance with the control command, and the sensor includes at least one of a visual sensor and a force sensor.

[0014] In this way, by repeatedly collecting data including a set of a control command and measurement information while the robot device is performing a reverse movement in accordance with the control command, it becomes possible to efficiently collect data that can be used to control the movement of the robot device during actual work (i.e., forward movement). For example, during actual work using the robot device, by generating a control command from measurement information obtained during work based on the collected data, it becomes possible to appropriately control the movement of the robot device from an arbitrary state to a target state.

[0015] The term "robot device" refers to any device that can operate in accordance with control commands output by a control device, and any robot device is applicable. For example, the robot device may be an industrial robot such as a manipulator, or may include an automatically movable mobile object. Examples of industrial robots include vertical articulated robots, SCARA robots, parallel link robots, Cartesian robots, and collaborative robots. Examples of automatically movable mobile objects include drones, autonomously driven vehicles, automated guided vehicles, and mobile robots, as well as combinations with the above industrial robots. In the embodiments described below, an example in which the robot device is a manipulator will be mainly described.

[0016] The "control command" relates to the control of the operation of the robotic device, and is, for example, a target control amount, an operation amount, etc. "Outputting a control command" may mean directly controlling the robotic device based on the control command, or, if the robotic device is equipped with a controller, outputting a control command to the controller to cause the controller to control the operation of the robotic device.

[0017] The "operational state of the robotic device" refers to the state related to the operation of a part of the configuration of the robotic device (e.g., an end effector) and / or the state related to the object of work performed using the robotic device. Furthermore, the "measurement related to the operational state of the robotic device" refers to the measurement of the state related to the operation of a part of the configuration of the robotic device (e.g., an end effector) and / or the measurement of the state related to the object of work performed using the robotic device. The "object" is an object that may be related to the operation of the robotic device, such as a workpiece. The part of the configuration of the robotic device (e.g., an end effector) may also be considered an object.

[0018] An "operation" is a job to be performed by a robotic device, and may include multiple steps. Examples of an operation include part transport, part fitting, screw driving, and processing. An operation may also be a simple task such as gripping a workpiece and releasing the workpiece. An operation may be given in advance or may be given by designation by an operator.

[0019] A "target state" is a state at the time when the purpose of a task (or process) is achieved. The target state may vary depending on the task. For example, if a robotic device is tasked with attaching an assembly part carried by an end effector to a workpiece, the target state may be a state in which the assembly part is attached to the workpiece.

[0020] The "arbitrary state" may be any state that is different from the target state, and may be, for example, a state at the start or midway of a task performed using the robot device.

[0021] The "measurement information" is not limited to the measurement data of the sensor itself, but may also be feature quantities calculated from the measurement data, etc. The "measurement information" may be, for example, information measured or calculated using an encoder and / or a servo motor that are present as a basic component of the robot.

[0022] (2) First embodiment In the first embodiment, data collection that collects data before the work to be used for control during the actual work will be mainly described. Details of the control during the actual work will be described in the second and third embodiments.

[0023] (2.1) System Configuration 1 is a diagram showing an example of the system configuration of a control system including a control device 100 according to the first embodiment. Here, the description will be focused on the hardware configuration of the control system.

[0024] In the illustrated example, the robot device 200 is a manipulator. Specifically, the robot device 200 (manipulator) is a six-axis vertical articulated industrial robot, and has a base 221 and six joints 211 to 216. Each of the joints 211 to 216 has a built-in servo motor (not shown) and is configured to be rotatable around each axis.

[0025] The first joint 211 is connected to a base 221 and rotates its distal end portion around the axis of the base. A moving mechanism capable of automatic movement (self-propelled) may be provided instead of the base 221. The second joint 212 is connected to the first joint 211 and rotates its distal end portion back and forth. The third joint 213 is connected to the second joint 212 via a link 222 and rotates its distal end portion up and down. The fourth joint 214 is connected to the third joint 213 via a link 223 and rotates its distal end portion around the axis of the link 223. The fifth joint 215 is connected to the fourth joint 214 via a link 224 and rotates its distal end portion up and down. The sixth joint 216 is connected to the fifth joint 215 via a link 225 and rotates its distal end portion around the axis of the link 225. A force sensor 320 and a gripper 226 are attached to the tip side of the sixth joint 216. The gripper 226 is an example of an end effector.

[0026] The visual sensor 310 is a sensor that performs image measurement, and is positioned so as to observe each object (gripper 226, workpiece W1, workpiece W2) present in the environment (work space) in which the robot device 200 operates. In the illustrated example, the visual sensor 310 is attached to the link 225, and the visual sensor 310 is provided integrally with the robot device 200. However, the visual sensor 310 may be fixed to a piece of equipment in the work space, and the visual sensor 310 may be provided separately from the robot device 200. For example, a camera such as a digital camera or a video camera may be used as the visual sensor 310. The measurement data (i.e., image data) of the visual sensor 310 is an example of visual measurement information.

[0027] The force sensor 320 is a sensor that measures forces and moments acting on the robot device 200 (specifically, the gripper 226). The force sensor 320 may be, for example, a six-axis force sensor that measures forces acting on the gripper 226 in the three axial directions of the X-axis, Y-axis, and Z-axis, and moments about the X-axis, Y-axis, and Z-axis. That is, the force sensor 320 can measure forces and moments generated by contact between an assembly part supported by the robot device 200 or the gripper 226 and an object. Measurement data from the force sensor 320 may be used to adjust the gripping force of the gripper 226 or to detect whether an abnormal force is acting on the gripper 226. In the following description, the term "force" is used to include the meaning of "moment."

[0028] The force sensor 320 may be realized by, for example, measuring the current value of a motor (not shown) built into each of the joints 211 to 216 to measure forces in the three axial directions of the X-axis, Y-axis, and Z-axis and moments around the X-axis, Y-axis, and Z-axis, or may be a pressure sensor provided on the surface of the robot 200 or a sensor that utilizes changes in the state of a jacket provided on the surface of the robot 200. The sensor may be one that detects changes in the flow rate of air and / or liquid, changes in capacitance, etc.

[0029] Each of the joints 211 to 216 may have an encoder (not shown) built in. The encoder is an example of a sensor. The encoder is configured to be able to measure the angle (control amount) of each of the joints 211 to 216. The measurement data of the encoder may be used to control the angle of each of the joints 211 to 216.

[0030] The control system may have a transport device 510 that transports a workpiece W2. The robot device 200 can perform a task using a gripper 226 (end effector) attached to the tip of its arm. The end effector is an external device that can be replaced depending on the application, and a welding gun, a tool, or the like may be attached instead of the gripper 226. The robot device 200 can perform a task using a force sensor 320 while tracking the workpiece W2 traveling on the transport device 510 using a visual sensor 310. In the illustrated example, the robot device 200 performs a task of fitting the workpiece W1, which is an assembly part held by the robot device 200, into a hole in the workpiece W2 (a part to be assembled, such as a circuit board) traveling on the transport device 510.

[0031] The control device 100 includes a processor 101, a memory 102, and an external interface (I / F) 103. The processor 101 includes a central processing unit (CPU). The processor 101 may further include at least one of a microprocessor, a field-programmable gate array (FPGA), and a digital signal processor (DSP). The memory 102 includes a random access memory (RAM), a read-only memory (ROM), and an auxiliary storage device (e.g., a hard disk drive, a solid-state drive). The processor 101 and the memory 102 form a computer. The control device 100 may be composed of multiple computers. Note that the control device 100 is not limited to an information processing device designed specifically for the services provided, and may be a general-purpose information processing device such as a personal computer (PC), or a controller such as a programmable logic controller (PLC).

[0032] The memory 102 stores a program executed by the processor 101. By executing the program stored in the memory 102, the processor 101, together with the memory 102, realizes the functions of each functional block described below. As will be described in detail below, the memory 102 may store, for example, a recognition library including a trained model used for image recognition to recognize an object, and a setting library including data collected by pre-task data collection and setting information acquired based on the collected data. The recognition library and the setting library may be provided for each type of task that the robot device 200 can perform. The setting information included in the setting library may include a trained model generated based on the collected data.

[0033] The external I / F 103 is, for example, a Universal Serial Bus (USB) port or a dedicated port, and is an interface for communicatively connecting to an external device. The external I / F 103 may be connected to an external device (including the robot device 200) by wire or wirelessly. The type and number of external I / Fs 103 may be appropriately selected depending on the type and number of external devices to be connected. In the illustrated example, the control device 100 is connected to the robot device 200, a visual sensor 310, and a user interface (I / F) 400 via the external I / F 103. As shown in FIG. 2, the user I / F 400 includes a display device 410 and an operation device 420. In the illustrated example, the user I / F 400 is provided separately from the control device 100, but the user I / F 400 may be provided integrally with the control device 100. The display device 410 may be a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The display device 410 may be a display equipped with a speaker. The operation device 420 is a device for performing operation input, such as a keyboard, a mouse, or a touch panel. The display device 410 and the operation device 420 may be integrated into a touch panel display. An operator can check the status of the control device 100 and operate the control device 100 by using the display device 410 and the operation device 420.

[0034] (2.2) Functional block configuration of the control device FIG. 2 is a block diagram showing a functional block configuration of the control device 100 according to the first embodiment. In the first embodiment, a functional block configuration related to data collection, which collects data to be used for control during work before the work is performed, will be mainly described. Such data collection may be performed before a user (including an operator) starts using the control device 100 (and the robot device 200). For example, data collection may be performed in advance before the control device 100 (and the robot device 200) is shipped.

[0035] The control device 100 includes an action generation unit 110, an acquisition unit 120, a data collection unit 130, a data storage unit 140, a setting acquisition unit 150, and a library storage unit 160. In this embodiment, the control device 100 may or may not include a control unit 170.

[0036] The motion generation unit 110 generates a control command for executing a forward motion that changes the motion state of the robot device 200 from an arbitrary state to a target state. The motion generation unit 110 also generates a control command for executing a reverse motion that changes the motion state of the robot device 200 from a target state to an arbitrary state different from the target state. The motion generation unit 110 outputs the generated control command to a drive unit 210 of the robot device 200. The drive unit 210 includes servo motors provided in the joints 211 to 216 of the robot device 200. The drive unit 210 may include a controller on the robot device 200 side. The drive unit 210 drives the servo motors in accordance with the control command to operate the robot device 200.

[0037] Here, the target state refers to a state that is realized when the purpose of a task (or process) is achieved, or an intermediate state of the task. In the first embodiment, a series of tasks is assumed in which the robot device 200 attaches an assembly part (workpiece W1) carried by the end effector (gripper 226) to a part to be assembled (workpiece W2), and therefore the target state is a state in which the assembly part is attached to the part to be assembled, or an intermediate state before attachment. Specifically, the target state is a state in which the workpiece W1 is fitted into a hole in the workpiece W2, or an intermediate state in which the workpiece W1 is in contact with the workpiece W2 at a position where it can be fitted into the hole.

[0038] The arbitrary state is a state different from the target state, for example, a state in which the workpiece W1 is located away from the hole in the workpiece W2. The arbitrary state may be set by operation input (user input) via the operation device 420. For example, the position of the arbitrary state may be set by operation input (user input) based on the position of the target state. Note that the term "position" may not only mean "coordinates" but also include the meaning of "posture."

[0039] The acquisition unit 120 acquires measurement information obtained using a sensor 300 for measuring the operating state of the robot device 200. In the illustrated example, the sensor 300 includes a visual sensor 310 and a force sensor 320. The sensor 300 may further include another sensor 330 such as an encoder. The acquisition unit 120 may include an image recognition unit 121 that performs image recognition on the measurement data (i.e., image data) output by the visual sensor 310. The image recognition unit 121 may recognize objects (e.g., the gripper 226, the workpiece W1, and the workpiece W2) through image recognition such as feature extraction, and acquire the positions of the objects (e.g., coordinates of the features). Such position information is an example of visual measurement information.

[0040] The data collection unit 130 repeatedly collects data including a set of a control command output by the motion generation unit 110 and measurement information acquired by the acquisition unit 120 while the robot device 200 is performing a reverse motion in accordance with the control command. This makes it possible to efficiently collect data that can be used to control the motion (forward motion) of the robot device 200 during actual work. For example, during actual work using the robot device 200, it is possible to appropriately control the motion of the robot device from an arbitrary state to a target state by generating a control command from measurement information obtained during work based on the collected data. Note that the data collection unit 130 may repeatedly collect data including a set of the control command and measurement information acquired by the acquisition unit 120 while the robot device 200 is performing a forward motion in accordance with the control command output by the motion generation unit 110.

[0041] The motion generation unit 110 may generate control commands for executing multiple patterns of reverse motion that change the motion state of the robot device 200 from a target state to multiple arbitrary states that are different from each other. The data collection unit 130 may repeatedly collect data for each of the multiple patterns of reverse motion. This allows multiple patterns of data corresponding to multiple movement paths to be collected, thereby providing versatility to the motion control of the robot device 200 during actual work. For example, even if the workpiece W2 is not fixed or moves during work, it becomes easy to attach the workpiece W1 to the workpiece W2.

[0042] The action generation unit 110 may generate a control command that increases the amount of data collected in an area closer to the target state, and may generate a control command that decreases the amount of data collected in an area farther from the target state. The closer an area is to the target state, the more precise and accurate control is required during actual work. By increasing the amount of data collected in an area closer to the target state, it becomes possible to collect a sufficient amount of data for precise and accurate control. On the other hand, in an area farther from the target state, precise and accurate control is not as necessary. Therefore, by generating a control command that decreases the amount of data collected in an area farther from the target state, unnecessary data collection can be suppressed, enabling efficient data collection.

[0043] The acquisition unit 120 may shorten the acquisition cycle (e.g., sampling frequency) so that the amount of acquired measurement information increases in an area closer to the target state, and may extend the acquisition cycle so that the amount of acquired measurement information increases in an area farther from the target state. Even with this processing, the amount of collected data can be increased in an area closer to the target state and decreased in an area farther from the target state.

[0044] The acquisition unit 120 may acquire, as relative measurement information, the difference between target measurement information obtained in the target state and current measurement information obtained during the execution of the reverse direction movement. The data collection unit 130 may repeatedly collect data including a set of the control command and the relative measurement information during the execution of the reverse direction movement in accordance with the control command. For example, the acquisition unit 120 may acquire, as relative position information, the difference between the visual measurement information (target position) of the object obtained by the visual sensor 310 in the target state and the current measurement information (current position) of the object obtained during the execution of the reverse direction movement. In the environment of FIG. 1, the acquisition unit 120 may set the position of the hole in the workpiece W2 as the target position, and each position on the movement path of the workpiece W1 as the current position, and acquire the difference between the target position and each current position as the relative position. This allows a control command to be associated with each relative positional relationship between the target position and the current position of the object. Therefore, during actual work, appropriate control commands can be generated based on the relative positional relationship between the target position and the current position of the object. Furthermore, control using relative positional relationships is applicable even when the object moves during work.

[0045] The acquisition unit 120 may acquire measurement information including a velocity-related value related to the robotic device 200 in response to the output of the sensor 300 or a control command. The velocity-related value related to the robotic device 200 may be at least one of the velocity, acceleration, and jerk of an object. The acquisition unit 120 may derive the velocity, acceleration, and jerk of an object from visual measurement information (position information of the object) obtained by the visual sensor 310. The acquisition unit 120 may derive the velocity, acceleration, and jerk of an object from a control command output to the drive unit 210. The acquisition unit 120 may derive the velocity, acceleration, and jerk of an object from measurement information obtained by another sensor 330 (e.g., an encoder). The data collection unit 130 may repeatedly collect data including a set of the control command and measurement information including the velocity-related value during the execution of a reverse movement in accordance with the control command. The velocity-related value collected in this manner can be used to set a velocity-related target value in control during actual work. Control using the velocity-related target value will be described in the third embodiment.

[0046] The data storage unit 140 stores the data collected by the data collection unit 130. The data collected by the data collection unit 130 includes multiple sets of measurement information (relative measurement information) and control commands. Each set may include measurement information (relative measurement information), a speed-related value, and a control command. The data collected by the data collection unit 130 may include more data sets in an area closer to the target state.

[0047] The setting acquisition unit 150 acquires setting information for performing a task using the robotic device 200 based on the data collected by the data collection unit 130 (specifically, the data stored in the data storage unit 140). Setting information for performing a certain task is referred to as a "setting library" for that task. For example, the setting acquisition unit 150 acquires the setting library after the data collection unit 130 has completed data collection. The library storage unit 160 stores the setting library acquired by the setting acquisition unit 150. The control unit 170 controls the robotic device 200 using the setting library during actual tasks.

[0048] The setting library includes operation parameters for each task step and information on the transition destination and transition conditions for each step (e.g., normal values, timeout values, and error values ​​for the transition conditions). The operation parameters may include correspondence information that associates measurement information (relative measurement information) with control commands, or may include correspondence information that associates measurement information (relative measurement information), a speed relationship value, and a control command. The operation parameters may include a control ratio between visual control, which is robot control based on visual measurement information obtained by the visual sensor 310, and force control, which is robot control based on force measurement information obtained by the force sensor 320. The operation parameters may include information on the type of object (e.g., metal, resin, screw, connector, etc.). The operation parameters may include information on features on the object (e.g., holes, edge surfaces, connectors, etc.). The setting acquisition unit 150 may acquire the setting library based at least in part on operation inputs made via the operation device 420.

[0049] The setting acquisition unit 150 may use the data collected by the data collection unit 130 as learning data to acquire a trained model for deriving control commands from measurement information through machine learning. In this case, the control unit 170 controls the robot device 200 during actual work based on measurement information obtained during the work using a setting library including the trained model. The type of trained model (trained model) is not particularly limited as long as it can acquire the ability to make inferences for generating control commands through machine learning. The type of machine learning is not particularly limited, but is typically supervised learning or reinforcement learning. The training model may be configured, for example, by a neural network such as a deep neural network (DNN). The training model may be configured, for example, by a value function such as a state value function or an action value function. In this way, the setting acquisition unit 150 determines a calculation for inferring an optimal value, enables inference, and generates an optimal value variable for a control command through machine learning.

[0050] The setting acquisition unit 150 may acquire a setting library that includes, for example, a control ratio between visual control and haptic control for each process. In this case, the control unit 170 uses the setting library to dynamically or stepwise change the control ratio during actual work depending on the status of the work. Details of such control will be described in the second embodiment.

[0051] The setting acquisition unit 150 may acquire a setting library that includes speed-related values ​​related to the robotic device 200, for example, for each process. In this case, the control unit 170 may derive the speed-related values ​​related to the robotic device 200 from measurement information or control commands obtained during work. Then, the control unit 170 may repeatedly perform control on the robotic device 200 to reduce the difference between the speed-related value and the speed-related target value, based on the setting library and the measurement information obtained during work. Details of such control will be described in the third embodiment.

[0052] The motion generation unit 110 may generate a control command for executing a reverse motion for each type of task performed by the robot device 200. Assuming the environment of FIG. 1, the task type is "mounting components on a circuit board." Other examples of task types include "packing food into boxes," "screw tightening," "pick and place," and "AGV (Automated Guided Vehicle) control." The data collection unit 130 may collect data for each task type by repeatedly collecting data for each task type.

[0053] In this case, the setting acquisition unit 150 can acquire a setting library for each type of work based on the data collected by the data collection unit 130 for each type of work. Then, the control unit 170 controls the robotic device 200 to perform the work using a setting library corresponding to the type of work that is actually performed. For example, the control unit 170 may display a list of setting libraries stored in the library storage unit 160 on the display device 410, and control the robotic device 200 using a setting library selected from the list using the operation device 420. Note that if the library storage unit 160 also stores a recognition library in association with the setting library, image recognition for performing the work may be performed using the recognition library corresponding to the type of work that is actually performed.

[0054] (2.3) An example of data collection 3 and 4 are diagrams for explaining an example of data collection by the control device 100 according to the first embodiment.

[0055] As described above, the motion generation unit 110 generates control commands for executing multiple patterns of reverse motions that change the motion state of the robotic device 200 from a target state to multiple arbitrary states that are different from one another. The data collection unit 130 repeatedly collects data for each of the multiple patterns of reverse motions. As shown in FIG. 3 , the motion generation unit 110 generates control commands that collect more data in areas closer to the target state and that collect less data in areas farther from the target state. In FIG. 3 , the x-axis and y-axis indicate directions that are perpendicular to each other in a horizontal plane, and the z-axis indicates the vertical direction. Also, in FIG. 3 , the amount of collected data is indicated by shading. Specifically, the greater the amount of collected data, the darker the color, and the less the amount of collected data, the lighter the color.

[0056] In the example of FIG. 4, the motion generation unit 110 sets a state in which the workpiece W1 (component) is in the hole of the workpiece W2 (circuit board) as the target state, and causes the robot device 200 to execute reverse motions to any of eight patterns P1 to P8. While the movement paths for each of the patterns P1 to P8 are illustrated as linear, each movement path does not have to be linear. For example, the movement path may be such that the workpiece W1 (component) is moved horizontally or diagonally upward after being pulled upward from the hole of the workpiece W2 (circuit board). Furthermore, among the patterns P1 to P8, the movements of P2, P4, and P8 end in region R, which is close to the target state. As a result, the closer the point to the target state, the greater the amount of data collected.

[0057] (2.4) Specific examples of the operation of the control device FIG. 5 is a flow chart for explaining an example of data collection by the control device 100 according to the first embodiment.

[0058] In step S101, a user (e.g., a worker) moves an object (e.g., a workpiece W1 grasped or supported by an end effector) to a target state. The motion generation unit 110 also acquires sequence information of a control command corresponding to any one of a plurality of reverse motion patterns (i.e., any arbitrary state) and / or information on the position of the arbitrary state. Here, the acquisition unit 120 acquires the position of the object in the target state as a target position from visual measurement information obtained using the visual sensor 310. The acquisition unit 120 acquires the force (reaction force) applied to the object in the target state as a target force (target reaction force) from force measurement information obtained using the force sensor 320.

[0059] In step S102, the action generating unit 110 outputs a control command to the robot device 200 (drive unit 210) based on the information acquired in step S101, thereby moving the target object by a predetermined movement amount.

[0060] In step S103, the acquisition unit 120 acquires current measurement information. The acquisition unit 120 acquires the current position of the object from visual measurement information obtained using the visual sensor 310. The acquisition unit 120 may acquire the difference between the current position of the object and the target position as a relative position (amount of position change). The acquisition unit 120 also acquires a current force (reaction force) from force measurement information obtained using the force sensor 320. The acquisition unit 120 may acquire the difference between the current force (reaction force) and the target force (reaction force) as a relative force (amount of force change). Furthermore, the acquisition unit 120 may acquire velocity-related values ​​(velocity, acceleration, and jerk) of the object from the current visual measurement information or the control command of step S102.

[0061] In step S104, the data collecting unit 130 collects a set of the control command in step S102 and the information (position change amount, force change amount, velocity related value) acquired in step S103, and stores the set in the data storage unit 140.

[0062] If the movement of the object to the arbitrary state corresponding to the current reverse motion pattern is not complete (step S105: NO), the process returns to step S102, and the motion generation unit 110 outputs a control command to the robot device 200 (drive unit 210) to move the object by a predetermined movement amount. Then, in step S103, the acquisition unit 120 acquires information (position change amount, force change amount, velocity-related value). In step S104, the data collection unit 130 collects a set of the control command of step S102 and the information acquired in step S103 (position change amount, force change amount, velocity-related value), and stores the set in the data storage unit 140. This process is repeated until the movement to the arbitrary state corresponding to the current reverse motion pattern is complete.

[0063] On the other hand, if the movement of the object to the arbitrary state corresponding to the current reverse motion pattern is completed (step S105: YES), the motion generation unit 110 checks whether the motions for all of the multiple reverse motion patterns are completed. If the motions for all patterns are not completed (step S106: NO), the process moves to the next reverse motion pattern (step S107) and resumes from step S101.

[0064] If all patterns have been processed (step S106: YES), in step S108, the setting acquisition unit 150 determines operation conditions (operation parameters) for performing work using the robot device 200 based on the data collected by the data collection unit 130 (specifically, the data stored in the data storage unit 140). Then, in step S109, the setting acquisition unit 150 stores setting information including the operation conditions (operation parameters) determined in step S108 in the library storage unit 160 as a setting library.

[0065] Note that the setting library for each type of work may be stored in the library storage unit 160 by executing the flow shown in FIG. 5 for each type of work.

[0066] (2.5) An example of a configuration library FIG. 6 is a diagram illustrating an example of a setting library stored in the control device 100 according to the embodiment.

[0067] In the illustrated example, the types of work include "mounting components on a circuit board," "packing food into boxes," "screw tightening," "pick and place" (also known as "picking"), and "AGV control." Each work involves multiple steps.

[0068] For example, in a task type called "mounting components on a board," the steps are performed in the following order: Step 1 "recognize component → move" → Step 2 "grasp component" → Step 3 "move component toward board" → Step 4 "approach" → Step 5 "recognize holes on board" → Step 6 "approach hole" → Step 7 "hole tracing operation" → Step 8 "insert hole" → Step 9 "hole insertion completed" → Step 10 "release grip." On the other hand, when collecting data, the control device 100 may control the operation of the robot device 200 in the reverse order. Note that in the case of "AGV control," a step of avoiding obstacles may be further included. The step of avoiding obstacles may be included in Step 4 "move object toward target object."

[0069] The setting library includes operation parameters for each work process, and information on the transition destination and transition conditions for each process (normal values, timeout values, error values, etc. of the transition conditions). The operation parameters may include correspondence information that associates measurement information (relative measurement information) with control commands, or may include correspondence information that associates measurement information (relative measurement information), speed-related values, and control commands. The operation parameters may include a control ratio between visual control and force control. The operation parameters may include information on the type of object (metal, resin, screw, connector, etc.). The operation parameters may include information on features on the object (holes, edges, connectors, etc.).

[0070] (3) Second embodiment The second embodiment will be described mainly focusing on the differences from the first embodiment. The system configuration of the second embodiment is the same as that of the first embodiment (see FIG. 1). Note that the second embodiment will be described mainly as an example in which the second embodiment is based on the first embodiment, but the second embodiment does not necessarily have to be based on at least a part of the first embodiment.

[0071] (3.1) Functional block configuration of the control device FIG. 7 is a block diagram showing a functional block configuration of a control device 100 according to the second embodiment.

[0072] The control device 100 according to the second embodiment includes an acquisition unit 120 that acquires measurement information obtained using a sensor 300 for measuring the operating state of the robotic device 200, and a control unit 170 that controls the operation of the robotic device 200 based on the measurement information. The sensor 300 includes a visual sensor 310 and a force sensor 320. The control unit 170 dynamically or stepwise changes the control ratio between visual control, which is control based on the visual measurement information obtained by the visual sensor 310, and force control, which is control based on the force measurement information obtained by the force sensor 320, depending on the status of the work using the robotic device 200. This makes it possible to take advantage of the advantages of both visual control and force control, and appropriately control the operation of the robotic device 200 by achieving both visual control and force control.

[0073] In the second embodiment, the control unit 170 includes a visual control unit 171A that refers to visual measurement information and generates first information indicating the content of control of the robot device 200, a force-sense control unit 171B that refers to force-sense measurement information and generates second information indicating the content of control of the robot device 200, a command generation unit 172 that generates a control command for the robot device 200 based on the first information and the second information, and a weighting unit 173 that changes the control ratio by performing weighting processing between the visual measurement information and the force-sense measurement information or between the first information and the second information depending on the status of the work using the robot device 200. This makes it possible to appropriately change the control ratio by the weighting processing.

[0074] The visual control performed by the visual control unit 171A may be performed based on, for example, operation parameters (such as association information) in a setting library. The visual control unit 171A identifies the relative positional relationship between the current position and the target position of the object based on visual measurement information obtained using the visual sensor 310. Then, the visual control unit 171A may generate a visual control command from the identified positional relationship using the association information to reduce the difference between the current position and the target position (i.e., move the object closer to the target position), and output this visual control command as first information. At least a part of such visual control may be performed using a trained model.

[0075] Similarly, the force-sense control performed by the force-sense control unit 171B may be performed based on, for example, operation parameters (such as association information) in a setting library. The force-sense control unit 171B may identify a difference between the current force (current reaction force) of the object and a target force (target reaction force) based on force-sense measurement information obtained using the force sensor 320, generate a force-sense control command from the identified difference to reduce the difference using the association information, and output this force-sense control command as second information. At least a portion of such force-sense control may be performed using a trained model.

[0076] The control unit 170 (weighting unit 173) may change the control ratio dynamically or stepwise as the work using the robotic device 200 progresses. As described above, the work using the robotic device 200 may include a plurality of predetermined steps. The control unit 170 may change the control ratio for each step. This allows the operation of the robotic device 200 to be controlled with an appropriate control ratio for each step.

[0077] The library storage unit 160 may store a setting library (setting information) including settings related to the control ratios of each of a plurality of processes. The control unit 170 (weighting unit 173) may change the control ratio for each process based on the setting library. This allows an appropriate control ratio to be set for each process.

[0078] When a task transitions from one step to the next, the control unit 170 (weighting unit 173) gradually changes the control ratio from the control ratio for the one step to the control ratio for the next step. This prevents abrupt fluctuations in the control ratio, enabling precise operation control and suppressing the occurrence of operation errors.

[0079] For example, if the control ratio (visual control: force sense control) for process A is set to "80:20" and the control ratio for process B, which follows process A, is set to "50:50," when a transition condition from process A to process B is satisfied, the control unit 170 (weighting unit 173) changes the control ratio in stages, such as "80:20" → "75:25" → "70:30" → "65:35" → "60:40" → "55:45" → "50:50." Alternatively, the control unit 170 (weighting unit 173) may change the control ratio continuously, such as "80:20" → "79:21" → "78:22" → "77:22" → .... Such a change in the control ratio may be performed in units of a control cycle or in units of time consisting of multiple control cycles.

[0080] The control unit 170 (weighting unit 173) may determine whether a transition condition from one process to the next process in a task is satisfied based on measurement information obtained using the sensor 300. The transition condition may be included as one of the operation parameters in the setting library. In response to the satisfaction of the transition condition, the control unit 170 (weighting unit 173) may transition from the one process to the next process and change the control ratio corresponding to the next process.

[0081] If the control in the next step does not converge after switching to the next step, the control unit 170 may return to the previous step and change the control ratio corresponding to the previous step. In this way, if the control does not converge, by returning to the previous step and repeating the operation, it is possible to expect that the control will converge when transitioning to the next step. Note that "if the control does not converge" may mean that at least one of the following conditions is met: the measurement information does not satisfy the normal value of the transition condition, a timeout has occurred, or the measurement information has become an error value.

[0082] The control unit 170 (weighting unit 173) may switch the control of the robot device 200 between a first control state (also referred to as "visual-based") in which control is performed with priority given to visual control, a second control state (also referred to as "visual+force-based") in which control is performed using visual control and force-sense control in cooperation, and a third control state (also referred to as "force-sense-based") in which control is performed with priority given to force-sense control. "Control is performed with priority given to visual control" may mean, for example, that the control ratio accounted for by visual control is approximately 65% ​​to 100%. "Control is performed using visual control and force-sense control in cooperation" may mean, for example, that the ratio of visual control to force-sense control is approximately 50:50. "Control is performed with priority given to force-sense control" may mean, for example, that the control ratio accounted for by force-sense control is approximately 65% ​​to 100%. The control unit 170 may determine whether to switch control among the first control state, the second control state, and the third control state based on measurement information obtained using the sensor 300. When the difference between the measurement information and the target value of the measurement information does not become equal to or less than a predetermined value in the first control state (this may be the case when the transition condition is not satisfied), the control unit 170 (weighting unit 173) may switch to the second control state or the third control state and refer to the force measurement information.

[0083] When an external instruction is received, the control unit 170 (weighting unit 173) may change the control ratio in accordance with the instruction. For example, when an instruction to change the control ratio is received via the operation device 420, the control unit 170 (weighting unit 173) may change the control ratio in accordance with the instruction.

[0084] The control unit 170 may further include a prediction unit 174A that predicts a control result for a subsequent control operation performed on the robotic device 200 based on the control operation performed on the robotic device 200 and the control result (sensor information) for the control operation, and a correction unit 174B that corrects the subsequent control operation based on the prediction. For example, if the robotic device 200 does not perform the operation specified in the control command due to an external factor, such as an error factor on the robotic device 200 side or an error factor on the sensor 300 side, the prediction unit 174A identifies the error (i.e., the difference between the control command content and the control result) and predicts the control result for the subsequent control operation. Then, the correction unit 174B may correct the control command generated by the command generation unit 172 based on the prediction result of the prediction unit 174A and output the corrected control command to the driver unit 210. For example, the correction unit 174B may correct the control command to cancel out the identified error. This enables more accurate robot control even when there is an external error factor.

[0085] The library storage unit 160 may store multiple setting libraries prepared for each type of work. Each of the multiple setting libraries may include setting information (operation parameters) for control ratios. Each of the multiple setting libraries includes setting information for each process in a series. The control unit 170 controls the robot device 200 using a setting library selected from the multiple setting libraries according to the type of work to be actually performed. The selection may be made by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by the visual sensor 310 observing the work site or the object. The control unit 170 may suggest the setting library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0086] The library storage unit 160 may further store multiple recognition libraries prepared for each type of work. Each of the multiple recognition libraries may include a trained model used for image recognition processing of an object. The control unit 170 performs image recognition processing using a recognition library selected from the multiple recognition libraries according to the type of work to be actually performed. The selection may be made by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by observing the work site or the object with the visual sensor 310. The control unit 170 may suggest the recognition library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0087] (3.2) Operation flow of the control device FIG. 8 is a flow diagram illustrating an example of a control flow by the control device 100 according to the second embodiment.

[0088] In step S201, the control unit 170 starts a process n. At the start of the work, the value of n is 1. At that time, the control unit 170 applies the setting information (operation parameters including the control ratio) corresponding to the process n.

[0089] In step S202, the control unit 170 controls the operation of the robot device 200 by applying the setting information (operation parameters including the control ratio) corresponding to the process n.

[0090] In step S203, the control unit 170 determines whether the transition condition from step n to the next step is satisfied (i.e., whether step n is completed). If step n is completed (step S203: YES), the process proceeds to step S204. On the other hand, if step n is not completed (step S203: NO), in step S205, the control unit 170 determines whether the control of step n has converged. If it is determined that the control of step n has converged (step S205: YES), the process returns to step S202.

[0091] If it is determined that the control of step n has not converged (step S205: NO), the control unit 170 returns to step (n-1), which is the step before step n. At this time, the control unit 170 may gradually change the control ratio from the control ratio of step n to the control ratio of step (n-1). Alternatively, instead of returning to step (n-1), the control unit 170 may change at least some of the operation parameters of step n in accordance with a predetermined rule and then resume step n. For example, if it is estimated that the speed-related target value is too high, the speed-related target value may be lowered by one level and step n may be resumed.

[0092] In step S204, the control unit 170 determines whether all steps of the work have been completed. If all steps have been completed, this flow ends. If all steps have not been completed (step S204: NO), in step S207, the control unit 170 proceeds to step (n+1), which is the step after step n. At this time, the control unit 170 may gradually change the control ratio from the control ratio for step n to the control ratio for step (n+1).

[0093] (3.3) Specific examples of the operation of the control device FIG. 9 is a diagram for explaining a specific example of control performed by the control device 100 according to the second embodiment during the operation of "mounting components on a board."

[0094] For the task type "mounting components on a board," the process is carried out in the following order: Step 1 "Recognize and move the component" → Step 2 "Grab the component" → Step 3 "Move the component toward the board" → Step 4 "Approach" → Step 5 "Recognize holes on the board" → Step 6 "Approach the hole" → Step 7 "Hole tracing operation" → Step 8 "Insert the hole" → Step 9 "Hole insertion completed" → Step 10 "Release gripping," and the operating parameters for each of these steps are included in the settings library.

[0095] Of these processes, basically, force-based control is set for processes involving contact between objects (end effector (gripper 226), component (workpiece W1), substrate (workpiece W2)), vision-based control is set for processes in which the objects (end effector, component) are moved at a certain speed or higher, and vision-plus-force-based control is set for processes in which the objects are moved at a low speed toward contact between the objects. However, it is preferable that the specific control ratio for each process be set based on data collected in the data collection according to the first embodiment.

[0096] In the illustrated example, vision-based control is applied to step 1, "Part recognition → movement." However, when the position of the part recognized by image recognition is set as the target position and the current position of the end effector approaches the vicinity of the target position, vision- and force-based control may be applied. Force-based control is applied to step 2, "Part grip."

[0097] Vision-based control is applied to process 3, "Move the component toward the board." Vision-based control is applied to process 4, "Approach." However, when the current position of the component approaches the target position, with the position of the board recognized by image recognition as the target position, vision- and force-based control may be applied. Vision-based control is applied to process 5, "Recognize holes in the board."

[0098] Vision and force-based control is applied to step 6, "Hole Approach." Force-based control is applied to steps 7, "Hole Tracking," through 10, "Grip Release."

[0099] (4) Third embodiment The third embodiment will be described mainly focusing on the differences from the first and second embodiments. The system configuration of the third embodiment is the same as that of the first embodiment (see FIG. 1). Note that the third embodiment will be described mainly as an example in which the third embodiment is based on the first and second embodiments, but the third embodiment does not necessarily have to be based on at least part of the first and second embodiments.

[0100] (4.1) Functional block configuration of the control device FIG. 10 is a block diagram showing a functional block configuration of a control device 100 according to the third embodiment.

[0101] The control device 100 according to the third embodiment includes an acquisition unit 120 that acquires measurement information obtained using a sensor 300 for measuring the motion state of the robotic device 200, and a control unit 170 that repeatedly generates control commands for the robotic device 200 for each control period based on the measurement information. The control unit 170 derives a velocity-related value for the robotic device 200 from the measurement information or the control command, and generates the control command to reduce the difference between the velocity-related value and a velocity-related target value that is variable for each control period depending on the motion state of the robotic device 200. This enables control that prevents motion errors during work using the robotic device 200. Furthermore, by varying the velocity-related target value for each control period depending on the motion state (work situation) of the robotic device 200, precise work involving nonlinear motion can be handled. As described above, the velocity-related value includes at least one of velocity, acceleration, and jerk.

[0102] In the third embodiment, the control unit 170 includes a derivation unit 175 that derives a speed-related value from measurement information or a control command for each control cycle, a target setting unit 176 that variably sets a speed-related target value for each control cycle in accordance with the operating state (work situation) of the robotic device 200, and a command generation unit 172 that generates a control command for each control cycle so as to reduce the difference between the speed-related value and the speed-related target value. The target setting unit 176 may variably set the speed-related target value in accordance with the speed-related value included in the operation parameters in the setting library.

[0103] The command generation unit 172 may incorporate the functions of the visual control unit 171A, the haptic control unit 171B, and the weighting unit 173 described in the second embodiment. Alternatively, the control unit 170 may be provided with the visual control unit 171A, the haptic control unit 171B, and the weighting unit 173 described in the second embodiment, separately from the command generation unit 172.

[0104] The control unit 170 (command generation unit 172) generates a control command to reduce the difference between the measurement information and the target measurement information (target value of the measurement information) and to reduce the difference between the speed-related value and the speed-related target value.

[0105] For example, the control unit 170 (command generating unit 172) may generate a control command to reduce the difference between the visual measurement information (current position) obtained using the visual sensor 310 and the target visual measurement information (target position), and to reduce the difference between the speed-related value and the speed-related target value. Here, the control to reduce the difference between the visual measurement information (current position) and the target visual measurement information (target position) is the same as the visual control described in the second embodiment.

[0106] The control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the force sense measurement information (current force (reaction force)) obtained using the force sensor 320 and the target force sense measurement information (target force (reaction force)), and to reduce the difference between the velocity-related value and the velocity-related target value. Here, the control to reduce the difference between the force sense measurement information (current force (reaction force)) and the target force sense measurement information (target force (reaction force)) is the same as the force sense control described in the second embodiment.

[0107] The control unit 170 (target setting unit 176) may change the speed-related target value according to the difference between the current measurement information and the target measurement information (current relative measurement information). For example, the control unit 170 (target setting unit 176) may set the speed-related value corresponding to the current relative measurement information as the speed-related target value using association information that associates the measurement information (relative measurement information) with the speed-related value, which is included in the setting library.

[0108] For example, the control unit 170 (target setting unit 176) may change the speed-related target value in accordance with the difference between the current visual measurement information and the target visual measurement information obtained using the visual sensor 310. The control unit 170 (target setting unit 176) may change the speed-related target value in accordance with the difference between the current force-sense measurement information and the target force-sense measurement information obtained using the force sensor 320.

[0109] The control unit 170 (deriving unit 175 and target setting unit 176) may select one or more values ​​to be used as the speed-related value (and speed-related target value) from among speed, acceleration, and jerk, depending on the difference between the measurement information and the target measurement information (current relative measurement information). For example, the control unit 170 (deriving unit 175 and target setting unit 176) may select speed as the speed-related value (and speed-related target value) during visual-based control. On the other hand, during force-based control, the control unit 170 may select jerk as the speed-related value (and speed-related target value). During visual-and force-based control, the control unit 170 may select acceleration as the speed-related value (and speed-related target value).

[0110] As in the second embodiment, the control unit 170 may have a prediction unit 174A that predicts the control result of a subsequent control operation performed on the robot device 200 based on the control operation performed on the robot device 200 and the control result of the control operation, and a correction unit 174B that corrects the subsequent control operation based on the prediction.

[0111] The library storage unit 160 may store multiple setting libraries prepared for each type of work. Each of the multiple setting libraries may include setting information for speed-related values ​​(speed-related target values). The control unit 170 controls the robot device 200 using a setting library selected from the multiple setting libraries according to the type of work to be actually performed. The selection may be made by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by observing the work site or the object with the visual sensor 310. The control unit 170 may suggest the setting library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0112] The library storage unit 160 may further store multiple recognition libraries prepared for each type of work. Each of the multiple recognition libraries may include a trained model used for image recognition processing of an object. The control unit 170 performs image recognition processing using a recognition library selected from the multiple recognition libraries according to the type of work to be actually performed. The selection may be made by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by observing the work site or the object with the visual sensor 310. The control unit 170 may suggest the recognition library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0113] (4.2) Specific examples of the operation of the control device FIG. 11 is a diagram for explaining a specific example of control performed by the control device 100 according to the third embodiment during the operation of "mounting components on a board."

[0114] As described above, the task type "mounting components on a board" is carried out in the following order: Step 1 "Recognize and move the component" → Step 2 "Grab the component" → Step 3 "Move the component toward the board" → Step 4 "Approach" → Step 5 "Recognize holes on the board" → Step 6 "Approach the hole" → Step 7 "Pass the hole" → Step 8 "Insert the hole" → Step 9 "Hole insertion completed" → Step 10 "Release the grip", and the operating parameters for each of these steps are included in the settings library.

[0115] In each step, the control unit 170 (command generating unit 172) generates a control command to reduce the difference between the measurement information obtained using the sensor 300 and the target measurement information (target value of the measurement information), and to reduce the difference between the speed-related value and the speed-related target value.

[0116] In the illustrated example, in step 1 "recognize component→move," "component position" may be applied as the target measurement information, and "speed" may be applied as the speed-related target value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector obtained using the visual sensor 310 and the position of the component, and to reduce the difference between the speed of the end effector and the speed target value. Here, the control unit 170 (target setting unit 176) may change the speed target value for each control cycle depending on the difference between the current position of the end effector obtained using the visual sensor 310 and the position of the component.

[0117] For step 2 "part gripping," "target reaction force" may be applied as the target measurement information, and "speed" may be applied as the speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the target reaction force and the current reaction force acting on the end effector, which is obtained using the force sensor 320, and to reduce the difference between the end effector speed and the speed target value. Here, the control unit 170 (target setting unit 176) may change the speed target value for each control cycle according to the difference between the target reaction force and the reaction force acting on the end effector, which is obtained using the force sensor 320.

[0118] In step 3, "Move the component toward the board," "board position" may be applied as the target measurement information, and "jerk" may be applied as the velocity-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board, and to reduce the difference between the jerk of the end effector (or component) and the jerk target value. Here, the control unit 170 (target setting unit 176) may change the jerk target value for each control cycle according to the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board.

[0119] For step 4 "approach," "difference in position between the end effector and the board" may be applied as target measurement information, and "acceleration" may be applied as a velocity-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board, and to reduce the difference between the acceleration of the end effector and the acceleration target value. Here, the control unit 170 (target setting unit 176) may change the acceleration target value for each control cycle according to the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board.

[0120] In step 5 “recognizing holes in board,” “holes in board” can be applied as target measurement information. In this case, the control unit 170 (command generation unit 172) uses the visual sensor 310 to recognize holes in the board.

[0121] In step 6 "Hole Approach," "Board Hole Position" and "Target Reaction Force" may be applied as target measurement information, and "Acceleration" may be applied as a velocity-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the component obtained using the vision sensor 310 and the position of the hole on the board, and to reduce the difference between the acceleration of the end effector and the target acceleration value. The control unit 170 (command generation unit 172) may also generate a control command to reduce the difference between the current reaction force acting on the end effector obtained using the force sensor 320 and the target reaction force, and to reduce the difference between the acceleration of the end effector and the target acceleration value.

[0122] For each step from step 7 "hole tracing operation" to step 10 "gripping release," a "target reaction force" for each step may be applied as target measurement information, and a "jerk" may be applied as a speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current reaction force acting on the end effector obtained using the force sensor 320 and the target reaction force for each step, and to reduce the difference between the jerk of the end effector and the target jerk value for each step. Here, the control unit 170 (target setting unit 176) may change the target jerk value for each control cycle according to the difference between the target reaction force and the reaction force acting on the end effector obtained using the force sensor 320.

[0123] (5) Other embodiments The operational flows and operational examples in the above-described embodiments do not necessarily have to be executed in chronological order according to the order shown in the flow diagrams. For example, steps in the operations may be executed in an order different from that shown in the flow diagrams, or may be executed in parallel. Furthermore, some steps in the operations may be deleted, or additional steps may be added to the process.

[0124] A program may be provided that causes a computer to execute the operations according to the above-described embodiments. The program may be recorded on a computer-readable medium. The computer-readable medium can be used to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a CD-ROM or a DVD-ROM.

[0125] As used in this disclosure, the terms "based on" and "depending on" do not mean "based only on" or "depending only on," unless expressly stated otherwise. The term "based on" means both "based only on" and "based at least in part on." Similarly, the term "depending on" means both "depending only on" and "depending at least in part on." Furthermore, the terms "include," "comprise," and variations thereof do not mean including only the listed items, but may include only the listed items, or may include additional items in addition to the listed items. Furthermore, the term "or" as used in this disclosure is not intended to mean an exclusive or. In this disclosure, when articles are added by translation, such as a, an, and the in English, these articles are intended to include the plural unless the context clearly indicates otherwise.

[0126] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention.

[0127] (6) Supplementary notes The following additional notes are about the features of the above-described embodiment.

[0128] Appendix 1 a motion generating unit that generates a control command for executing a reverse motion that changes the motion state of the robot apparatus from a target state to an arbitrary state different from the target state; an acquisition unit that acquires measurement information obtained using a sensor for measuring the operating state of the robot device; a data collection unit that repeatedly collects data including a set of the control command and the measurement information during execution of the reverse direction operation in accordance with the control command; The sensor includes at least one of a visual sensor and a force sensor. Control device.

[0129] Appendix 2 the motion generation unit generates the control command for executing a plurality of patterns of reverse motions that change the motion state of the robot apparatus from the target state to a plurality of arbitrary states different from each other; The data collection unit repeatedly collects the data for each of the plurality of patterns of reverse direction motion. 10. The control device of claim 1.

[0130] Appendix 3 The action generation unit generating the control command such that the amount of data collected increases in an area closer to the target state; The control command is generated so that the amount of data collected decreases in an area farther from the target state. 3. The control device according to claim 1 or 2.

[0131] Appendix 4 The acquisition unit shortening the acquisition cycle so that the amount of acquired measurement information increases in an area closer to the target state; The acquisition period is extended so that the amount of acquired measurement information increases in an area farther from the target state. 4. A control device according to any one of appendices 1 to 3.

[0132] Appendix 5 the acquisition unit acquires, as relative measurement information, a difference between target measurement information acquired in the target state and current measurement information acquired during execution of the reverse direction motion; The data collection unit repeatedly collects the data including a set of the control command and the relative measurement information during the execution of the reverse direction operation in accordance with the control command. 5. A control device according to any one of appendices 1 to 4.

[0133] Appendix 6 the acquisition unit acquires the measurement information including a velocity-related value related to the robot apparatus in response to the output of the sensor or the control command; The data collection unit repeatedly collects the data including a set of the control command and the measurement information including the speed-related value during the execution of the reverse direction operation in accordance with the control command. 6. A control device according to any one of appendices 1 to 5.

[0134] Appendix 7 a setting acquisition unit that acquires setting information for performing a task using the robot device based on the data collected by the data collection unit; a control unit that uses the setting information to control the robot device to perform the work. 7. A control device according to any one of appendices 1 to 6.

[0135] Appendix 8 the setting acquisition unit acquires the setting information including a trained model for deriving the control command from the measurement information by machine learning using the data collected by the data collection unit as learning data; The control unit performs the control based on the measurement information obtained during the work, using the setting information including the trained model. 8. The control device according to claim 7.

[0136] Appendix 9 the sensors include the visual sensor and the force sensor; the setting acquisition unit acquires the setting information including information for setting a control ratio between visual control, which is the control based on visual measurement information obtained using the visual sensor, and force control, which is the control based on force measurement information obtained using the force sensor; and The control unit changes the control ratio dynamically or stepwise according to the status of the work using the robot device using the setting information. 9. The control device according to claim 7 or 8.

[0137] Appendix 10 The control unit deriving a velocity-related value for the robot device from the measurement information or control commands obtained during the work; Based on the setting information and the measurement information, the robot device is repeatedly controlled to reduce the difference between the speed-related value and the speed-related target value. 10. A control device according to any one of appendices 7 to 9.

[0138] Appendix 11 the operation generation unit generates the control command for executing the reverse operation for each type of work using the robot device; The data collection unit collects the data for each of the types by repeatedly collecting the data for each of the types. 11. A control device according to any one of appendices 1 to 10.

[0139] Appendix 12 a setting acquisition unit that acquires setting information for performing work using the robot device for each type based on the data collected for each type by the data collection unit; a control unit that uses the setting information corresponding to the type of work to be actually performed to control the robot device to perform the work. 12. The control device of claim 11.

[0140] Appendix 13 generating a control command for executing a reverse movement that changes the motion state of the robot device from a target state to an arbitrary state different from the target state; acquiring measurement information obtained using a sensor for measuring an operating state of the robot device; repeatedly collecting data including a set of the control command and the measurement information during the execution of the reverse direction operation in accordance with the control command; The sensor includes at least one of a visual sensor and a force sensor. Control method.

[0141] Appendix 14 The control device generating a control command for executing a reverse movement that changes the motion state of the robot device from a target state to an arbitrary state different from the target state; acquiring measurement information obtained using a sensor for measuring an operating state of the robot device; repeatedly collecting data including a set of the control command and the measurement information during the execution of the reverse direction operation in accordance with the control command; The sensor includes at least one of a visual sensor and a force sensor. program. [Explanation of symbols]

[0142] 100: Control device 101: Processor 102: Memory 103: External I / F 110: Motion generator 120: Acquisition section 121: Image recognition unit 130: Data collection section 140: Data storage unit 150: Setting acquisition section 160: Library storage section 170: Control unit 171A: Vision control unit 171B: Force sense control unit 172: Command generation section 173: Weighting section 174A: Prediction section 174B: Correction unit 175: Derivation part 176: Goal Setting Department 200: Robotic device 210: Drive unit 211: First joint 212: Second joint 213: Third joint 214: 4th joint 215: 5th joint 216: 6th joint 221: Base 222: Link 223: Link 224: Link 225: Link 226: Gripper 300: Sensor 310: Vision sensor 320: Force sensor 330: Other sensors 400: User I / F 410:Display device 420: Operating device 510:Transportation equipment

Claims

1. a motion generating unit that generates a control command for executing a reverse motion that changes the motion state of the robot apparatus from a target state to an arbitrary state different from the target state; an acquisition unit that acquires measurement information obtained using a sensor for measuring the operating state of the robot device; a data collection unit that repeatedly collects data including a set of the control command and the measurement information during execution of the reverse direction operation in accordance with the control command; The sensor includes at least one of a visual sensor and a force sensor. Control device.

2. the motion generation unit generates the control command for executing a plurality of patterns of reverse motions that change the motion state of the robot apparatus from the target state to a plurality of arbitrary states different from each other; The data collection unit repeatedly collects the data for each of the plurality of patterns of reverse direction motion. The control device according to claim 1 .

3. The action generation unit generating the control command such that the amount of data collected increases in an area closer to the target state; The control command is generated so that the amount of data collected decreases in an area farther from the target state. The control device according to claim 1 .

4. The acquisition unit shortening the acquisition cycle so that the amount of acquired measurement information increases in an area closer to the target state; The acquisition period is extended so that the amount of acquired measurement information increases in an area farther from the target state. The control device according to claim 1 .

5. the acquisition unit acquires, as relative measurement information, a difference between target measurement information acquired in the target state and current measurement information acquired during execution of the reverse direction motion; The data collection unit repeatedly collects the data including a set of the control command and the relative measurement information during the execution of the reverse direction operation in accordance with the control command. The control device according to claim 1 .

6. the acquisition unit acquires the measurement information including a velocity-related value related to the robot apparatus in response to the output of the sensor or the control command; The data collection unit repeatedly collects the data including a set of the control command and the measurement information including the speed-related value during the execution of the reverse direction operation in accordance with the control command. The control device according to claim 1 .

7. a setting acquisition unit that acquires setting information for performing a task using the robot device based on the data collected by the data collection unit; a control unit that uses the setting information to control the robot device to perform the work. The control device according to any one of claims 1 to 6.

8. the setting acquisition unit acquires the setting information including a trained model for deriving the control command from the measurement information by machine learning using the data collected by the data collection unit as learning data; The control unit performs the control based on the measurement information obtained during the work, using the setting information including the trained model. The control device according to claim 7.

9. the sensors include the visual sensor and the force sensor; the setting acquisition unit acquires the setting information including information for setting a control ratio between visual control, which is the control based on visual measurement information obtained using the visual sensor, and force control, which is the control based on force measurement information obtained using the force sensor; and The control unit changes the control ratio dynamically or stepwise according to the status of the work using the robot device using the setting information. The control device according to claim 7.

10. The control unit deriving a velocity-related value for the robot device from the measurement information or control commands obtained during the work; Based on the setting information and the measurement information, the robot device is repeatedly controlled to reduce the difference between the speed-related value and the speed-related target value. The control device according to claim 7.

11. the operation generation unit generates the control command for executing the reverse operation for each type of work using the robot device; The data collection unit collects the data for each of the types by repeatedly collecting the data for each of the types. The control device according to claim 1 .

12. a setting acquisition unit that acquires setting information for performing work using the robot device for each type based on the data collected for each type by the data collection unit; a control unit that uses the setting information corresponding to the type of work to be actually performed to control the robot device to perform the work. The control device according to claim 11.

13. generating a control command for executing a reverse movement that changes the motion state of the robot device from a target state to an arbitrary state different from the target state; acquiring measurement information obtained using a sensor for measuring an operating state of the robot device; repeatedly collecting data including a set of the control command and the measurement information during the execution of the reverse direction operation in accordance with the control command; The sensor includes at least one of a visual sensor and a force sensor. Control method.

14. The control device generating a control command for executing a reverse movement that changes the motion state of the robot device from a target state to an arbitrary state different from the target state; acquiring measurement information obtained using a sensor for measuring an operating state of the robot device; repeatedly collecting data including a set of the control command and the measurement information during the execution of the reverse direction operation in accordance with the control command; The sensor includes at least one of a visual sensor and a force sensor. program.

Images