Control apparatus, control method, and program

Abstract

A control apparatus (100) comprises: an acquisition unit (120) that acquires measurement information obtained by using a sensor (300) for performing measurement related to the moving state of a robotics device (200); and a control unit (170) that repeatedly generates a control command for the robotics device (200) on the basis of the measurement information. The control unit (170) derives a speed relationship value pertaining to the robotics device (200) either from the measurement information or from the control command, and generates said control command so as to reduce the difference between the speed relationship value and a speed relationship target value which is variable depending on the moving state of the robotics device (200).

Description

Control device, control method, and program CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority from Japanese Patent Application No. 2024-073074 (filed April 26, 2024), the entire contents of which are incorporated herein by reference.

[0002] The present disclosure relates to a control device, a control method, and a program for controlling a robot device.

[0003] Conventionally, control devices for controlling robotic devices have been widely used. In such control devices, a technique is known in which a visual sensor and a force sensor are used to measure the motion state of the robotic device and to control the motion state of the robotic device to achieve a target motion state (see, for example, Patent Document 1).

[0004] Patent No. 7295344

[0005] It is desirable to realize control that does not cause operational errors when working with a robotic device, and also to realize control that can handle precise work involving non-linear operations, for example, to realize highly accurate position control and / or speed control even if errors and / or characteristic changes occur in the sensors, control devices, and / or drive units.

[0006] The present disclosure provides a control device, a control method, and a program that enable more appropriate control of a robot device.

[0007] A control device according to a first aspect of the present disclosure includes an acquisition unit that acquires measurement information obtained using a sensor for measuring the operating state of a robotic device, and a control unit that repeatedly generates control commands for the robotic device based on the measurement information, wherein the control unit derives a speed-related value for the robotic device from the measurement information or the control command, and generates the control command so as to reduce the difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

[0008] A control method according to a second aspect of the present disclosure includes acquiring measurement information obtained using a sensor for measuring an operating state of a robotic device, and repeatedly generating control commands for the robotic device based on the measurement information, wherein repeatedly generating the control commands includes deriving a speed-related value for the robotic device from the measurement information or the control commands, and generating the control commands so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

[0009] A program according to a third aspect of the present disclosure causes a control device to acquire measurement information obtained using a sensor for measuring an operating state of a robotic device, and repeatedly generate control commands for the robotic device based on the measurement information, wherein repeatedly generating the control commands includes deriving a speed-related value for the robotic device from the measurement information or the control commands, and generating the control commands so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

[0010] 1 is a diagram showing an example of a system configuration of a control system including a control device according to an embodiment; FIG. 2 is a block diagram showing a functional block configuration of a control device according to a first embodiment; FIG. 3 is a diagram for explaining an example of data collection by a control device according to the first embodiment; FIG. 4 is a flow chart for explaining an example of data collection by a control device according to the first embodiment; FIG. 5 is a diagram for explaining an example of a setting library stored by a control device according to an embodiment; FIG. 6 is a block diagram showing a functional block configuration of a control device according to a second embodiment; FIG. 7 is a flow chart for explaining an example of a control flow by a control device according to the second embodiment; FIG. 8 is a diagram for explaining a specific example of control during an operation of "mounting components on a board" as control by a control device according to the second embodiment; FIG. 9 is a block diagram showing a functional block configuration of a control device according to a third embodiment; FIG. 10 is a diagram for explaining a specific example of control during an operation of "mounting components on a board" as control by a control device according to the third embodiment.

[0011] The embodiments will be described with reference to the drawings, in which the same or similar parts are denoted by the same or similar reference numerals.

[0012] (1) Overview of Embodiments A control device according to an embodiment includes an acquisition unit that acquires measurement information obtained using a sensor for measuring an operating state of a robotic device, and a control unit that repeatedly generates control commands for the robotic device based on the measurement information. The control unit derives a speed-related value for the robotic device from the measurement information or the control command, and generates the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

[0013] In this way, by generating a control command to reduce the difference between the velocity-related value and the velocity-related target value for the robot device, control that does not cause operational errors during work using the robot device is possible. Furthermore, by making the velocity-related target value variable depending on the operating state of the robot device, it is possible to handle precise work involving nonlinear movement. For example, even if errors and / or characteristic changes occur in the sensors, control devices, and / or drive units, the system operates to reduce the difference between the velocity-related value and the velocity-related target value, enabling significantly more accurate position control and / or velocity control.

[0014] The term "robot device" refers to any device that can operate in accordance with control commands output by a control device, and any robot device is applicable. For example, the robot device may be an industrial robot such as a manipulator, or may include an automatically movable mobile object. Examples of industrial robots include vertical articulated robots, SCARA robots, parallel link robots, Cartesian robots, and collaborative robots. Examples of automatically movable mobile objects include drones, vehicles configured to be autonomous, automated guided vehicles, and mobile robots, as well as combinations with the above industrial robots. In the embodiments described below, an example in which the robot device is a manipulator will be mainly described.

[0015] The "control command" relates to the control of the operation of the robotic device, and is, for example, a target control amount, an operation amount, etc. "Outputting a control command" may mean directly controlling the robotic device based on the control command, or, if the robotic device is equipped with a controller, outputting a control command to the controller to cause the controller to control the operation of the robotic device.

[0016] An "operation" is a job to be performed by a robotic device, and may include multiple steps. Examples of an operation include part transport, part fitting, screw driving, and processing. An operation may also be a simple task such as gripping a workpiece and releasing the workpiece. An operation may be given in advance or may be given by designation by an operator.

[0017] The "operational state of the robotic device" refers to a state related to the operation of a part of the configuration of the robotic device (e.g., an end effector) and / or a state related to an object of work performed using the robotic device. Furthermore, the "measurement related to the operational state of the robotic device" refers to a measurement of a state related to the operation of a part of the configuration of the robotic device (e.g., an end effector) and / or a measurement of a state related to an object of work performed using the robotic device. The "object" is an object that may be related to the operation of the robotic device, such as a workpiece. The part of the configuration of the robotic device (e.g., an end effector) may also be considered an object.

[0018] The "sensor" may be any sensor capable of measuring the operating state of the robot device. In the following description of the embodiment, an example will be described in which the sensor includes at least one of a visual sensor and a force sensor.

[0019] The "measurement information" is not limited to the measurement data itself of the sensor, but may also be a feature calculated from the measurement data, etc. The "measurement information" may be, for example, information measured and / or calculated using an encoder and / or a servo motor that are present as a basic component of the robot.

[0020] In an embodiment, the control unit may repeatedly generate a control command for the robotic device for each control period based on the measurement information. Here, the control unit may generate the control command so as to reduce the difference between the speed-related target value and the speed-related value, which is variable for each control period depending on the operating state of the robotic device. The "control period" does not necessarily have to be a period of a fixed time length, but may be a period of a variable time length and / or a period depending on the load (e.g., computational load) of the control device.

[0021] (2) First Embodiment In the first embodiment, data collection for collecting data to be used for control during actual work before the work will be mainly described. Details of control during actual work will be described in the second and third embodiments.

[0022] (2.1) System Configuration Fig. 1 is a diagram showing an example of the system configuration of a control system including a control device 100 according to the first embodiment. Here, the description will be focused on the hardware configuration of the control system.

[0023] In the illustrated example, the robot device 200 is a manipulator. Specifically, the robot device 200 (manipulator) is a six-axis vertical articulated industrial robot, and includes a base 221 and six joints 211 to 216. Each of the joints 211 to 216 incorporates a servo motor (not shown) and is configured to be rotatable around each axis.

[0024] The first joint 211 is connected to a base 221 and rotates its distal end portion around the axis of the base. A moving mechanism capable of automatic movement (self-propelled) may be provided instead of the base 221. The second joint 212 is connected to the first joint 211 and rotates its distal end portion back and forth. The third joint 213 is connected to the second joint 212 via a link 222 and rotates its distal end portion up and down. The fourth joint 214 is connected to the third joint 213 via a link 223 and rotates its distal end portion around the axis of the link 223. The fifth joint 215 is connected to the fourth joint 214 via a link 224 and rotates its distal end portion up and down. The sixth joint 216 is connected to the fifth joint 215 via a link 225 and rotates its distal end portion around the axis of the link 225. A force sensor 320 and a gripper 226 are attached to the distal end side of the sixth joint 216. The gripper 226 is an example of an end effector.

[0025] The visual sensor 310 is a sensor that performs image measurement and is positioned to observe each object (gripper 226, workpiece W1, workpiece W2) present in the environment (work space) in which the robot device 200 operates. In the illustrated example, the visual sensor 310 is attached to the link 225, and the visual sensor 310 is provided integrally with the robot device 200. However, the visual sensor 310 may be fixed to equipment in the work space or the like, and the visual sensor 310 may be provided separately from the robot device 200. For example, a camera such as a digital camera or a video camera may be used as the visual sensor 310. The measurement data (i.e., image data) of the visual sensor 310 is an example of visual measurement information.

[0026] The force sensor 320 is a sensor that measures forces and moments acting on the robot device 200 (specifically, the gripper 226). The force sensor 320 may be, for example, a six-axis force sensor that measures forces acting on the gripper 226 in the three directions of the X-axis, Y-axis, and Z-axis, and moments about the X-axis, Y-axis, and Z-axis. That is, the force sensor 320 can measure forces and moments generated by contact between an assembly part supported by the robot device 200 or the gripper 226 and an object. The measurement data of the force sensor 320 may be used to adjust the gripping force of the gripper 226 or to detect whether an abnormal force is acting on the gripper 226. In the following description, the term "force" is used to include the meaning of "moment."

[0027] The force sensor 320 may be realized, for example, by measuring the current values ​​of motors (not shown) built into each of the joints 211 to 216 to measure forces in the three axial directions of the X-axis, Y-axis, and Z-axis, and moments about the X-axis, Y-axis, and Z-axis. The force sensor 320 may be, for example, a pressure sensor provided on the surface of the robot device 200 and / or a sensor that utilizes changes in the state of a jacket provided on the surface of the robot device 200. The force sensor 320 may also be a sensor that detects changes in the flow rate of air and / or liquid and / or changes in capacitance.

[0028] Each of the joints 211 to 216 may have an encoder (not shown) built in. The encoder is an example of a sensor. The encoder is configured to be able to measure the angle (control amount) of each of the joints 211 to 216. The measurement data of the encoder may be used to control the angle of each of the joints 211 to 216.

[0029] The control system may include a transport device 510 that transports the workpiece W2. The robot device 200 can perform a task using a gripper 226 (end effector) attached to the tip of its arm. The end effector is an external device that can be replaced depending on the application, and a welding gun, tool, or the like may be attached instead of the gripper 226. The robot device 200 can perform a task using a force sensor 320 while tracking the workpiece W2 traveling on the transport device 510 using a visual sensor 310. In the illustrated example, the robot device 200 performs a task of fitting the workpiece W1, which is an assembly part gripped by the robot device 200, into a hole in the workpiece W2 (a part to be assembled, such as a circuit board) traveling on the transport device 510.

[0030] The control device 100 has a processor 101, a memory 102, and an external interface (I / F) 103. The processor 101 includes a CPU (Central Processing Unit). The processor 101 may further include at least one of a microprocessor, an FPGA (Field-Programmable Gate Array), and a DSP (Digital Signal Processor). The memory 102 includes a RAM (Random Access Memory), a ROM (Read Only Memory), and an auxiliary storage device (e.g., a hard disk drive, a solid state drive). The processor 101 and the memory 102 constitute a computer. The control device 100 may be composed of multiple computers. The control device 100 is not limited to an information processing device designed specifically for the service to be provided, but may be a general-purpose information processing device such as a PC (Personal Computer), or may be a controller such as a PLC (Programmable Logic Controller).

[0031] The memory 102 stores a program executed by the processor 101. By executing the program stored in the memory 102, the processor 101, together with the memory 102, realizes the functions of each functional block described below. As will be described in detail below, the memory 102 may store, for example, a recognition library including a trained model used for image recognition to recognize an object, and a setting library including data collected by pre-task data collection and setting information acquired based on the collected data. The recognition library and the setting library may be provided for each type of task that the robot device 200 can perform. The setting information included in the setting library may include a trained model generated based on the collected data.

[0032] The external I / F 103 is, for example, a USB (Universal Serial Bus) port or a dedicated port, and is an interface for communicatively connecting to an external device. The external I / F 103 may be connected to an external device (including the robot device 200) via a wired or wireless connection. The type and number of external I / Fs 103 may be selected appropriately depending on the type and number of external devices to be connected. In the illustrated example, the control device 100 is connected to the robot device 200, a visual sensor 310, and a user interface (I / F) 400 via the external I / F 103. As shown in FIG. 2 , the user I / F 400 includes a display device 410 and an operation device 420. In the illustrated example, the user I / F 400 is provided separately from the control device 100, but the user I / F 400 may also be provided integrally with the control device 100. The display device 410 may be a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The display device 410 may also be a display equipped with a speaker. The operation device 420 is a device for performing operation input, such as a keyboard, a mouse, or a touch panel. The display device 410 and the operation device 420 may be integrated into a touch panel display. An operator can check the status of the control device 100 and operate the control device 100 by using the display device 410 and the operation device 420.

[0033] (2.2) Functional Block Configuration of Control Device FIG. 2 is a block diagram showing the functional block configuration of the control device 100 according to the first embodiment. In the first embodiment, the functional block configuration related to data collection, which collects data to be used for control during work before the work is performed, will be mainly described. Such data collection may be performed before a user (including an operator) starts using the control device 100 (and the robot device 200). For example, data collection may be performed in advance before the control device 100 (and the robot device 200) is shipped.

[0034] The control device 100 includes an action generation unit 110, an acquisition unit 120, a data collection unit 130, a data storage unit 140, a setting acquisition unit 150, and a library storage unit 160. In this embodiment, the control device 100 may or may not include a control unit 170.

[0035] The motion generation unit 110 generates a control command for executing a forward motion that changes the motion state of the robot device 200 from an arbitrary state to a target state. The motion generation unit 110 also generates a control command for executing a reverse motion that changes the motion state of the robot device 200 from a target state to an arbitrary state different from the target state. The motion generation unit 110 outputs the generated control command to a drive unit 210 of the robot device 200. The drive unit 210 includes servo motors provided in the joints 211 to 216 of the robot device 200. The drive unit 210 may include a controller on the robot device 200 side. The drive unit 210 drives the servo motors in accordance with the control command to operate the robot device 200.

[0036] Here, the target state refers to a state that is realized when the purpose of the work (or process) is achieved and / or an intermediate state of the work. In the first embodiment, a series of operations is assumed in which the robot device 200 attaches an assembly part (workpiece W1) carried by the end effector (gripper 226) to a workpiece (workpiece W2), so the target state is a state in which the assembly part is attached to the workpiece W2 and / or an intermediate state before attachment. Specifically, the target state is a state in which the workpiece W1 is fitted into a hole in the workpiece W2 and / or an intermediate state in which the workpiece W1 is in contact with the workpiece W2 at a position where it can be fitted into the hole.

[0037] The arbitrary state is a state different from the target state, for example, a state in which the workpiece W1 is located away from the hole in the workpiece W2. The arbitrary state may be set by operation input (user input) via the operation device 420. For example, the position of the arbitrary state may be set by operation input (user input) based on the position of the target state. Note that the term "position" may not only mean "coordinates" but also include the meaning of "posture."

[0038] The acquisition unit 120 acquires measurement information obtained using a sensor 300 for measuring the operating state of the robot device 200. In the illustrated example, the sensor 300 includes a visual sensor 310 and a force sensor 320. The sensor 300 may further include another sensor 330, such as an encoder. The acquisition unit 120 may also include an image recognition unit 121 that performs image recognition on the measurement data (i.e., image data) output by the visual sensor 310. The image recognition unit 121 may recognize objects (e.g., the gripper 226, the workpiece W1, and the workpiece W2) through image recognition such as feature extraction, and acquire the positions of the objects (e.g., coordinates of the feature parts). Such position information is an example of visual measurement information.

[0039] The data collection unit 130 repeatedly collects data including a set of a control command output by the motion generation unit 110 and measurement information acquired by the acquisition unit 120 while the robot device 200 is performing a reverse motion in accordance with the control command. This makes it possible to efficiently collect data that can be used to control the motion (forward motion) of the robot device 200 during actual work. For example, during actual work using the robot device 200, it is possible to appropriately control the motion of the robot device from an arbitrary state to a target state by generating a control command from measurement information obtained during work based on the collected data. Note that the data collection unit 130 may repeatedly collect data including a set of the control command and measurement information acquired by the acquisition unit 120 while the robot device 200 is performing a forward motion in accordance with the control command output by the motion generation unit 110.

[0040] The motion generation unit 110 may generate control commands for executing multiple patterns of reverse motion that change the motion state of the robot device 200 from a target state to multiple arbitrary states that are different from each other. The data collection unit 130 may repeatedly collect data for each of the multiple patterns of reverse motion. This allows multiple patterns of data corresponding to multiple movement paths to be collected, thereby providing versatility to the motion control of the robot device 200 during actual work. For example, even if the workpiece W2 is not fixed or moves during work, it becomes easy to attach the workpiece W1 to the workpiece W2.

[0041] The action generation unit 110 may generate a control command that increases the amount of data collected in an area closer to the target state, and may generate a control command that decreases the amount of data collected in an area farther from the target state. The closer an area is to the target state, the more precise and accurate control is required during actual work. By increasing the amount of data collected in an area closer to the target state, it is possible to collect a sufficient amount of data for precise and accurate control. On the other hand, in an area farther from the target state, precise and accurate control is not as necessary. Therefore, by generating a control command that decreases the amount of data collected in an area farther from the target state, unnecessary data collection is suppressed, enabling efficient data collection.

[0042] The acquisition unit 120 may shorten the acquisition cycle (e.g., sampling frequency) so that the amount of acquired measurement information increases in regions closer to the target state, and may extend the acquisition cycle so that the amount of acquired measurement information increases in regions farther from the target state. This process also makes it possible to increase the amount of collected data in regions closer to the target state and decrease the amount of collected data in regions farther from the target state.

[0043] The acquisition unit 120 may acquire, as relative measurement information, the difference between target measurement information obtained in the target state and current measurement information obtained during the execution of the reverse motion. The data collection unit 130 may repeatedly collect data including a set of the control command and relative measurement information during the execution of the reverse motion in accordance with the control command. For example, the acquisition unit 120 may acquire, as relative position information, the difference between the visual measurement information (target position) of the object obtained by the visual sensor 310 in the target state and the current measurement information (current position) of the object obtained during the execution of the reverse motion. In the environment of FIG. 1 , the acquisition unit 120 may define the position of the hole in the workpiece W2 as the target position, define each position on the movement path of the workpiece W1 as the current position, and acquire the difference between the target position and each current position as the relative position. This allows a control command to be associated with each relative positional relationship between the target position and the current position of the object. Therefore, during actual work, appropriate control commands can be generated based on the relative positional relationship between the target position and the current position of the object. Furthermore, control using relative positional relationships is applicable even when the object moves during work.

[0044] The acquisition unit 120 may acquire measurement information including a velocity-related value related to the robot device 200 in response to the output of the sensor 300 or a control command. The velocity-related value related to the robot device 200 may be at least one of the velocity, acceleration, and jerk of an object. The acquisition unit 120 may derive the velocity, acceleration, and jerk of the object from visual measurement information (position information of the object) obtained by the visual sensor 310. The acquisition unit 120 may derive the velocity, acceleration, and jerk of the object from a control command output to the drive unit 210. The acquisition unit 120 may derive the velocity, acceleration, and jerk of the object from measurement information obtained by another sensor 330 (e.g., an encoder). The data collection unit 130 may repeatedly collect data including a set of the control command and measurement information including the velocity-related value during the execution of a reverse movement in accordance with the control command. The velocity-related value collected in this manner can be used to set a velocity-related target value in control during actual work. Control using the velocity-related target value will be described in the third embodiment.

[0045] The data storage unit 140 stores the data collected by the data collection unit 130. The data collected by the data collection unit 130 includes multiple sets of measurement information (relative measurement information) and control commands. Each set may include measurement information (relative measurement information), a speed-related value, and a control command. The data collected by the data collection unit 130 may include more data sets in an area closer to the target state.

[0046] The setting acquisition unit 150 acquires setting information for performing a task using the robotic device 200 based on the data collected by the data collection unit 130 (specifically, the data stored in the data storage unit 140). Setting information for performing a certain task is referred to as a "setting library" for that task. For example, the setting acquisition unit 150 acquires the setting library after the data collection unit 130 has completed data collection. The library storage unit 160 stores the setting library acquired by the setting acquisition unit 150. The control unit 170 controls the robotic device 200 using the setting library during actual tasks.

[0047] The setting library includes operation parameters for each task step and information on the transition destination and transition conditions for each step (e.g., normal values, timeout values, and error values ​​for the transition conditions). The operation parameters may include correspondence information that associates measurement information (relative measurement information) with control commands. The operation parameters may include correspondence information that associates measurement information (relative measurement information), a speed relationship value, and a control command. The operation parameters may include a control ratio between visual control, which is robot control based on visual measurement information obtained by the visual sensor 310, and force control, which is robot control based on force measurement information obtained by the force sensor 320. The operation parameters may include information on the type of object (e.g., metal, resin, screw, connector, etc.). The operation parameters may include information on features on the object (e.g., holes, edge surfaces, connectors, etc.). The setting acquisition unit 150 may acquire the setting library based at least in part on operation input performed via the operation device 420.

[0048] The setting acquisition unit 150 may use the data collected by the data collection unit 130 as learning data to acquire a trained model for deriving control commands from measurement information through machine learning. In this case, the control unit 170 controls the robot device 200 during actual work based on measurement information obtained during work using a setting library including the trained model. The type of trained model (trained model) is not particularly limited as long as it can acquire the ability to make inferences for generating control commands through machine learning. The type of machine learning is not particularly limited, but is typically supervised learning or reinforcement learning. The training model may be configured, for example, by a neural network such as a deep neural network (DNN). The training model may be configured, for example, by a value function such as a state value function or an action value function. In this way, the setting acquisition unit 150 determines a calculation for inferring an optimal value, enables inference, and generates an optimal value variable for a control command through machine learning.

[0049] The setting acquisition unit 150 may acquire a setting library that includes, for example, a control ratio between visual control and haptic control for each process. In this case, the control unit 170 uses the setting library to dynamically or stepwise change the control ratio during actual work depending on the status of the work. Details of such control will be described in the second embodiment.

[0050] The setting acquisition unit 150 may acquire a setting library that includes speed-related values ​​related to the robotic device 200, for example, for each process. In this case, the control unit 170 may derive the speed-related values ​​related to the robotic device 200 from measurement information or control commands obtained during work. The control unit 170 may then repeatedly control the robotic device 200 to reduce the difference between the speed-related value and the speed-related target value, based on the setting library and the measurement information obtained during work. Details of such control will be described in a third embodiment.

[0051] The motion generation unit 110 may generate a control command for executing a reverse motion for each type of task performed by the robot device 200. Assuming the environment of FIG. 1 , the task type is "mounting components on a circuit board." Other examples of task types include "packing food into boxes," "tightening screws," "pick-and-place," and "AGV (Automated Guided Vehicle) control." The data collection unit 130 may collect data for each task type by repeatedly collecting data for each task type.

[0052] In this case, the setting acquisition unit 150 can acquire a setting library for each task type based on the data collected by the data collection unit 130 for each task type. The control unit 170 then controls the robotic device 200 to perform the task using the setting library corresponding to the task type that is actually performed. For example, the control unit 170 may display a list of setting libraries stored in the library storage unit 160 on the display device 410, and control the robotic device 200 using a setting library selected from the list using the operation device 420. Note that if the library storage unit 160 also stores a recognition library in association with the setting library, image recognition for performing the task may be performed using the recognition library corresponding to the task type that is actually performed.

[0053] (2.3) Example of Data Collection FIGS. 3 and 4 are diagrams for explaining an example of data collection by the control device 100 according to the first embodiment.

[0054] As described above, the motion generation unit 110 generates control commands for executing multiple patterns of reverse motions that change the motion state of the robot device 200 from a target state to multiple arbitrary states that are different from each other. The data collection unit 130 repeatedly collects data for each of the multiple patterns of reverse motions. As shown in FIG. 3 , the motion generation unit 110 generates control commands that collect more data in areas closer to the target state and that collect less data in areas farther from the target state. In FIG. 3 , the x-axis and y-axis indicate directions that are orthogonal to each other in a horizontal plane, and the z-axis indicates the vertical direction. Also, in FIG. 3 , the amount of collected data is indicated by shading. Specifically, a larger amount of collected data is indicated by a darker color, and a smaller amount of collected data is indicated by a lighter color.

[0055] In the example of FIG. 4 , the motion generation unit 110 sets a state in which the workpiece W1 (component) is in a hole in the workpiece W2 (circuit board) as the target state, and causes the robot device 200 to execute reverse motions to any of eight patterns P1 to P8. While the movement paths for each of the patterns P1 to P8 are illustrated as linear, each movement path does not have to be linear. For example, the movement path may be such that the workpiece W1 (component) is pulled upward from the hole in the workpiece W2 (circuit board) and then moved horizontally or diagonally upward. Furthermore, among the patterns P1 to P8, the movements of P2, P4, and P8 end in region R, which is close to the target state. As a result, the closer the point to the target state, the greater the amount of data collected.

[0056] (2.4) Specific Example of Operation of the Control Device FIG. 5 is a flow chart for explaining an example of data collection by the control device 100 according to the first embodiment.

[0057] In step S101, a user (e.g., a worker) moves an object (e.g., a workpiece W1 grasped or supported by an end effector) to a target state. The motion generation unit 110 also acquires sequence information of a control command corresponding to any one of a plurality of reverse motion patterns (i.e., any arbitrary state) and / or position information of the arbitrary state. Here, the acquisition unit 120 acquires the position of the object in the target state as a target position from visual measurement information obtained using the visual sensor 310. The acquisition unit 120 acquires the force (reaction force) applied to the object in the target state as a target force (target reaction force) from force measurement information obtained using the force sensor 320.

[0058] In step S102, the motion generation unit 110 outputs a control command to the robot device 200 (drive unit 210) based on the information acquired in step S101, thereby moving the object by a predetermined movement amount.

[0059] In step S103, the acquisition unit 120 acquires current measurement information. The acquisition unit 120 acquires the current position of the object from visual measurement information obtained using the visual sensor 310. The acquisition unit 120 may acquire the difference between the current position of the object and the target position as a relative position (position change amount). The acquisition unit 120 also acquires a current force (reaction force) from force measurement information obtained using the force sensor 320. The acquisition unit 120 may acquire the difference between the current force (reaction force) and the target force (reaction force) as a relative force (force change amount). Furthermore, the acquisition unit 120 may acquire velocity-related values ​​(velocity, acceleration, and jerk) of the object from the current visual measurement information or the control command of step S102.

[0060] In step S104, the data collection unit 130 collects a set of the control command in step S102 and the information (position change amount, force change amount, velocity related value) acquired in step S103, and stores the set in the data storage unit 140.

[0061] If the movement of the object to the arbitrary state corresponding to the current reverse motion pattern has not been completed (step S105: NO), the process returns to step S102, and the motion generation unit 110 outputs a control command to the robot device 200 (drive unit 210) to move the object by a predetermined movement amount. Then, in step S103, the acquisition unit 120 acquires information (position change amount, force change amount, velocity-related value). In step S104, the data collection unit 130 collects a set of the control command of step S102 and the information acquired in step S103 (position change amount, force change amount, velocity-related value), and stores the set in the data storage unit 140. This process is repeated until the movement to the arbitrary state corresponding to the current reverse motion pattern is completed.

[0062] On the other hand, if the movement of the object to the arbitrary state corresponding to the current reverse motion pattern is completed (step S105: YES), the motion generation unit 110 checks whether the motions for all of the multiple reverse motion patterns are completed. If the motions for all patterns are not completed (step S106: NO), the process moves to the next reverse motion pattern (step S107) and resumes from step S101.

[0063] If the process has been completed for all patterns (step S106: YES), in step S108, the setting acquisition unit 150 determines operation conditions (operation parameters) for performing work using the robot device 200 based on the data collected by the data collection unit 130 (specifically, the data stored in the data storage unit 140). Then, in step S109, the setting acquisition unit 150 stores setting information including the operation conditions (operation parameters) determined in step S108 in the library storage unit 160 as a setting library.

[0064] Note that the flow shown in FIG. 5 may be executed for each type of work, so that a setting library for each type of work is stored in the library storage unit 160.

[0065] (2.5) Example of Setting Library FIG. 6 is a diagram for explaining an example of a setting library stored in the control device 100 according to the embodiment.

[0066] In the illustrated example, the types of work include "mounting components on a circuit board," "packing food into boxes," "screw tightening," "pick and place" (also known as "picking"), and "AGV control." Each work includes multiple steps.

[0067] For example, in a task type called "mounting a component on a board," the steps are performed in the following order: Step 1 "recognize component → move" → Step 2 "grasp component" → Step 3 "move component toward board" → Step 4 "approach" → Step 5 "recognize hole on board" → Step 6 "approach hole" → Step 7 "hole tracing operation" → Step 8 "insert hole" → Step 9 "hole insertion completed" → Step 10 "release grip." On the other hand, when collecting data, the control device 100 may control the operation of the robot device 200 in the reverse order to this order. Note that in the case of "AGV control," a step of avoiding obstacles may be further included. The step of avoiding obstacles may be included in Step 4 "move object toward target object."

[0068] The setting library includes operation parameters for each work process, and information on the transition destination and transition conditions for each process (normal values, timeout values, error values, etc. of the transition conditions). The operation parameters may include correspondence information that associates measurement information (relative measurement information) with control commands, or may include correspondence information that associates measurement information (relative measurement information), a speed-related value, and a control command. The operation parameters may include a control ratio between visual control and force control. The operation parameters may include information on the type of object (metal, resin, screw, connector, etc.). The operation parameters may include information on features on the object (holes, edges, connectors, etc.).

[0069] (3) Second Embodiment The second embodiment will be described mainly focusing on the differences from the first embodiment. The system configuration of the second embodiment is the same as that of the first embodiment (see FIG. 1). Note that the second embodiment will be described mainly as an example in which the second embodiment is based on the first embodiment, but the second embodiment does not necessarily have to be based on at least a part of the first embodiment.

[0070] (3.1) Functional Block Configuration of Control Device FIG. 7 is a block diagram showing the functional block configuration of the control device 100 according to the second embodiment.

[0071] The control device 100 according to the second embodiment includes an acquisition unit 120 that acquires measurement information obtained using a sensor 300 for measuring the operating state of the robot device 200, and a control unit 170 that controls the operation of the robot device 200 based on the measurement information. The sensor 300 includes a visual sensor 310 and a force sensor 320. The control unit 170 dynamically or stepwise changes the control ratio between visual control, which is control based on the visual measurement information obtained by the visual sensor 310, and force control, which is control based on the force measurement information obtained by the force sensor 320, depending on the situation of the work using the robot device 200. This makes it possible to take advantage of the advantages of both visual control and force control, and appropriately control the operation of the robot device 200 by achieving both visual control and force control.

[0072] In the second embodiment, the control unit 170 includes a visual control unit 171A that refers to the visual measurement information and generates first information indicating the content of control of the robot device 200, a force-sense control unit 171B that refers to the force-sense measurement information and generates second information indicating the content of control of the robot device 200, a command generation unit 172 that generates a control command for the robot device 200 based on the first information and the second information, and a weighting unit 173 that changes the control ratio by performing weighting processing between the visual measurement information and the force-sense measurement information or between the first information and the second information depending on the status of the work using the robot device 200. This makes it possible to appropriately change the control ratio by the weighting processing.

[0073] The visual control performed by the visual control unit 171A may be performed based on, for example, operation parameters (such as association information) in a setting library. The visual control unit 171A identifies the relative positional relationship between the current position and the target position of the object based on visual measurement information obtained using the visual sensor 310. The visual control unit 171A may then generate a visual control command based on the identified positional relationship and the association information to reduce the difference between the current position and the target position (i.e., move the object closer to the target position), and output this visual control command as first information. At least a portion of such visual control may be performed using a trained model.

[0074] Similarly, the force sense control performed by the force sense control unit 171B may be performed based on, for example, operation parameters (such as correspondence information) in a setting library. The force sense control unit 171B may identify a difference between the current force (current reaction force) of the object and a target force (target reaction force) based on force sense measurement information obtained using the force sensor 320, generate a force sense control command from the identified difference to reduce the difference using the correspondence information, and output this force sense control command as second information. At least a portion of such force sense control may be performed using a trained model.

[0075] The control unit 170 (weighting unit 173) may dynamically or stepwise change the control ratio as the work using the robotic device 200 progresses. As described above, the work using the robotic device 200 may include a plurality of predetermined steps. The control unit 170 may change the control ratio for each step. This allows the operation of the robotic device 200 to be controlled with an appropriate control ratio for each step.

[0076] The library storage unit 160 may store a setting library (setting information) including settings related to the control ratios of each of a plurality of processes. The control unit 170 (weighting unit 173) may change the control ratio for each process based on the setting library. This allows an appropriate control ratio to be set for each process.

[0077] When a task transitions from one process to the next, the control unit 170 (weighting unit 173) gradually changes the control ratio from the control ratio for the one process to the control ratio for the next process, thereby preventing abrupt fluctuations in the control ratio, enabling precise operation control and suppressing the occurrence of operation errors.

[0078] For example, if the control ratio (visual control: force control) for process A is set to "80:20" and the control ratio for process B, which follows process A, is set to "50:50," when a transition condition from process A to process B is satisfied, the control unit 170 (weighting unit 173) changes the control ratio in stages, such as "80:20" → "75:25" → "70:30" → "65:35" → "60:40" → "55:45" → "50:50." Alternatively, the control unit 170 (weighting unit 173) may change the control ratio continuously, such as "80:20" → "79:21" → "78:22" → "77:23" → .... Such a change in the control ratio may be performed in units of control cycles or in units of time consisting of multiple control cycles.

[0079] The control unit 170 (weighting unit 173) may determine whether a transition condition from one process to the next process in a task is satisfied based on measurement information obtained using the sensor 300. The transition condition may be included as one of the operation parameters in the setting library. In response to the satisfaction of the transition condition, the control unit 170 (weighting unit 173) may transition from the one process to the next process and change the control ratio corresponding to the next process.

[0080] If the control in the next process does not converge after switching to the next process, the control unit 170 may return to the previous process and change the control ratio to the control ratio corresponding to the previous process. In this way, if the control does not converge, the control may be expected to converge when transitioning to the next process by returning to the previous process and restarting the operation. Note that "if the control does not converge" may mean that at least one of the following conditions is met: the measurement information does not satisfy the normal value of the transition condition, a timeout has occurred, or the measurement information has become an error value.

[0081] The control unit 170 (weighting unit 173) may switch the control of the robot device 200 between a first control state (also referred to as "visual-based") in which control is performed with priority given to visual control, a second control state (also referred to as "visual-plus-force-based") in which control is performed using visual control and force-sense control in cooperation, and a third control state (also referred to as "force-sense-based") in which control is performed with priority given to force-sense control. "Performing control with priority given to visual control" may mean, for example, that the control ratio accounted for by visual control is approximately 65% ​​to 100%. "Performing control with cooperation between visual control and force-sense control" may mean, for example, that the ratio of visual control to force-sense control is approximately 50:50. "Performing control with priority given to force-sense control" may mean, for example, that the control ratio accounted for by force-sense control is approximately 65% ​​to 100%. The control unit 170 may determine whether to switch control among the first control state, the second control state, and the third control state based on measurement information obtained using the sensor 300. When the difference between the measurement information and the target value of the measurement information does not become equal to or less than a predetermined value in the first control state (this may be when the transition condition is not satisfied), the control unit 170 (weighting unit 173) may switch to the second control state or the third control state and refer to the force measurement information.

[0082] When an external instruction is received, the control unit 170 (weighting unit 173) may change the control ratio in accordance with the instruction. For example, when an instruction to change the control ratio is received via the operation device 420, the control unit 170 (weighting unit 173) may change the control ratio in accordance with the instruction.

[0083] The control unit 170 may further include a prediction unit 174A that predicts a control result for a subsequent control operation performed on the robotic device 200 based on the control operation performed on the robotic device 200 and the control result (sensor information) for the control operation, and a correction unit 174B that corrects the subsequent control operation based on the prediction. For example, if the robotic device 200 does not perform the operation specified in the control command due to an external factor, such as an error factor on the robotic device 200 side or an error factor on the sensor 300 side, the prediction unit 174A identifies the error (i.e., the difference between the control command content and the control result) and predicts the control result for the subsequent control operation. The correction unit 174B may then correct the control command generated by the command generation unit 172 based on the prediction result of the prediction unit 174A and output the corrected control command to the drive unit 210. For example, the correction unit 174B may correct the control command to cancel out the identified error. This enables more accurate robot control even when there is an external error factor.

[0084] The library storage unit 160 may store multiple setting libraries prepared for each type of work. Each of the multiple setting libraries may include setting information (operation parameters) for control ratios. Each of the multiple setting libraries includes setting information for each process in a series. The control unit 170 controls the robot device 200 using a setting library selected from the multiple setting libraries according to the type of work to be actually performed. The selection may be made by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by the visual sensor 310 observing the work site or the object. The control unit 170 may suggest the setting library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0085] The library storage unit 160 may further store multiple recognition libraries prepared for each type of work. Each of the multiple recognition libraries may include a trained model used for image recognition processing of an object. The control unit 170 performs image recognition processing using a recognition library selected from the multiple recognition libraries according to the type of work to be actually performed. The selection may be made by operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by observing the work site or the object with the visual sensor 310. The control unit 170 may suggest the recognition library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0086] (3.2) Operation Flow of the Control Device FIG. 8 is a flow diagram for explaining an example of a control flow by the control device 100 according to the second embodiment.

[0087] In step S201, the control unit 170 starts process n. At the start of the work, the value of n is 1. At this time, the control unit 170 applies the setting information (operation parameters including the control ratio) corresponding to process n.

[0088] In step S202, the control unit 170 controls the operation of the robot device 200 by applying the setting information (operation parameters including the control ratio) corresponding to the process n.

[0089] In step S203, the control unit 170 determines whether the transition condition from process n to the next process is satisfied (i.e., whether process n is completed). If process n is completed (step S203: YES), the process proceeds to step S204. On the other hand, if process n is not completed (step S203: NO), in step S205, the control unit 170 determines whether the control of process n has converged. If it is determined that the control of process n has converged (step S205: YES), the process returns to step S202.

[0090] If it is determined that the control of step n has not converged (step S205: NO), the control unit 170 returns to the step (n-1) before step n. At this time, the control unit 170 may gradually change the control ratio from the control ratio of step n to the control ratio of step (n-1). Alternatively, instead of returning to step (n-1), the control unit 170 may change at least some of the operating parameters of step n in accordance with a predetermined rule and then resume step n. For example, if it is estimated that the speed-related target value is too high, the speed-related target value may be lowered by one level and then resume step n.

[0091] In step S204, the control unit 170 determines whether all steps of the work have been completed. If all steps have been completed, this flow ends. If all steps have not been completed (step S204: NO), in step S207, the control unit 170 proceeds to step (n+1), which is the step after step n. At this time, the control unit 170 may gradually change the control ratio from the control ratio for step n to the control ratio for step (n+1).

[0092] (3.3) Specific Example of Operation of the Control Device FIG. 9 is a diagram for explaining a specific example of control performed by the control device 100 according to the second embodiment during the operation of "mounting components on a board."

[0093] For the task type "mounting a component on a board," the process is performed in the following order: Step 1 "recognize and move the component" → Step 2 "grasp the component" → Step 3 "move the component toward the board" → Step 4 "approach" → Step 5 "recognize holes on the board" → Step 6 "approach the hole" → Step 7 "hole tracing operation" → Step 8 "insert the hole" → Step 9 "hole insertion completed" → Step 10 "release the grip," and the operating parameters for each of these steps are included in the setting library.

[0094] Of these processes, basically, force-based control is set for processes involving contact between objects (end effector (gripper 226), component (workpiece W1), substrate (workpiece W2)), visual-based control is set for processes in which the objects (end effector, component) are moved at a certain speed or higher, and visual and force-based control is set for processes in which the objects are moved at a low speed toward contact between the objects. However, it is preferable that the specific control ratio for each process be set based on data collected in the data collection according to the first embodiment.

[0095] In the illustrated example, vision-based control is applied to step 1 "recognize part → move." However, when the position of the part recognized by image recognition is set as the target position and the current position of the end effector approaches the target position, vision- and force-based control may be applied. Force-based control is applied to step 2 "grasp part."

[0096] Vision-based control is applied to step 3, "Move the component toward the board." Vision-based control is applied to step 4, "Approach." However, when the current position of the component approaches the vicinity of the target position, with the position of the board recognized by image recognition as the target position, vision-plus-force-based control may be applied. Vision-based control is applied to step 5, "Recognize holes in the board."

[0097] Vision- and force-based control is applied to step 6 "approaching the hole." Force-based control is applied to steps 7 "following the hole" through 10 "releasing the grip."

[0098] (4) Third Embodiment The third embodiment will be described mainly focusing on the differences from the first and second embodiments. The system configuration of the third embodiment is the same as that of the first embodiment (see FIG. 1). Note that the third embodiment will be described mainly as an example in which the third embodiment is based on the first and second embodiments, but the third embodiment does not necessarily have to be based on at least part of the first and second embodiments.

[0099] (4.1) Functional Block Configuration of Control Device FIG. 10 is a block diagram showing the functional block configuration of a control device 100 according to the third embodiment.

[0100] The control device 100 according to the third embodiment includes an acquisition unit 120 that acquires measurement information obtained using a sensor 300 for measuring the operating state of the robotic device 200, and a control unit 170 that repeatedly generates control commands for the robotic device 200 for each control period based on the measurement information. The control unit 170 derives a velocity-related value for the robotic device 200 from the measurement information or the control command, and generates a control command to reduce the difference between the velocity-related value and a velocity-related target value that is variable for each control period depending on the operating state of the robotic device 200. This enables control that prevents operational errors during work using the robotic device 200. Furthermore, by varying the velocity-related target value for each control period depending on the operating state (work situation) of the robotic device 200, precise work involving nonlinear movements can be handled. As described above, the velocity-related value includes at least one of velocity, acceleration, and jerk.

[0101] In the third embodiment, the control unit 170 includes a derivation unit 175 that derives a speed-related value from measurement information or a control command for each control cycle, a target setting unit 176 that variably sets a speed-related target value for each control cycle in accordance with the operating state (work situation) of the robot device 200, and a command generation unit 172 that generates a control command for each control cycle so as to reduce the difference between the speed-related value and the speed-related target value. The target setting unit 176 may variably set the speed-related target value in accordance with the speed-related value included in the operation parameters in the setting library.

[0102] The command generation unit 172 may incorporate the functions of the visual control unit 171A, the haptic control unit 171B, and the weighting unit 173 described in the second embodiment. Alternatively, the control unit 170 may be provided with the visual control unit 171A, the haptic control unit 171B, and the weighting unit 173 described in the second embodiment, separately from the command generation unit 172.

[0103] The control unit 170 (command generation unit 172) generates a control command to reduce the difference between the measurement information and the target measurement information (target value of the measurement information) and to reduce the difference between the speed-related value and the speed-related target value.

[0104] For example, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the visual measurement information (current position) obtained using the visual sensor 310 and the target visual measurement information (target position), and to reduce the difference between the speed-related value and the speed-related target value. Here, the control to reduce the difference between the visual measurement information (current position) and the target visual measurement information (target position) is the same as the visual control described in the second embodiment.

[0105] The control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the force sense measurement information (current force (reaction force)) obtained using the force sensor 320 and the target force sense measurement information (target force (reaction force)), and to reduce the difference between the velocity-related value and the velocity-related target value. Here, the control to reduce the difference between the force sense measurement information (current force (reaction force)) and the target force sense measurement information (target force (reaction force)) is the same as the force sense control described in the second embodiment.

[0106] The control unit 170 (target setting unit 176) may change the speed-related target value according to the difference between the current measurement information and the target measurement information (current relative measurement information). For example, the control unit 170 (target setting unit 176) may set the speed-related target value corresponding to the current relative measurement information using association information that associates measurement information (relative measurement information) with speed-related values ​​and that is included in the setting library.

[0107] For example, the control unit 170 (target setting unit 176) may change the speed-related target value in accordance with the difference between the current visual measurement information and the target visual measurement information obtained using the visual sensor 310. The control unit 170 (target setting unit 176) may change the speed-related target value in accordance with the difference between the current force measurement information and the target force measurement information obtained using the force sensor 320.

[0108] The control unit 170 (deriving unit 175 and target setting unit 176) may select one or more values ​​to be used as the speed-related value (and speed-related target value) from among speed, acceleration, and jerk, depending on the difference between the measurement information and the target measurement information (current relative measurement information). For example, the control unit 170 (deriving unit 175 and target setting unit 176) may select speed as the speed-related value (and speed-related target value) during visual-based control. On the other hand, during force-based control, the control unit 170 (deriving unit 175 and target setting unit 176) may select jerk as the speed-related value (and speed-related target value). During visual-and force-based control, the control unit 170 (deriving unit 175 and target setting unit 176) may select acceleration as the speed-related value (and speed-related target value).

[0109] As in the second embodiment, the control unit 170 may have a prediction unit 174A that predicts the control results of subsequent control operations performed on the robot device 200 based on the control operations performed on the robot device 200 and the control results of the control operations, and a correction unit 174B that corrects the subsequent control operations based on the prediction.

[0110] The library storage unit 160 may store multiple setting libraries prepared for each type of work. Each of the multiple setting libraries may include setting information for speed-related values ​​(speed-related target values). The control unit 170 controls the robot device 200 using a setting library selected from the multiple setting libraries according to the type of work to be actually performed. The selection may be made by an operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by the visual sensor 310 observing the work site or the object. The control unit 170 may suggest the setting library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0111] The library storage unit 160 may further store multiple recognition libraries prepared for each type of work. Each of the multiple recognition libraries may include a trained model used for image recognition processing of an object. The control unit 170 performs image recognition processing using a recognition library selected from the multiple recognition libraries according to the type of work to be actually performed. The selection may be made by operation input via the operation device 420. Prior to such operation input, the acquisition unit 120 (image recognition unit 121) may estimate the work to be actually performed based on visual measurement information obtained by observing the work site or the object with the visual sensor 310. The control unit 170 may suggest the recognition library corresponding to the estimated work to the user (operator) by displaying it on the display device 410.

[0112] (4.2) Specific Example of Operation of the Control Device FIG. 11 is a diagram for explaining a specific example of control performed by the control device 100 according to the third embodiment during the operation of "mounting components on a board."

[0113] As described above, in the task type "mounting components on a board," the steps are performed in the following order: Step 1 "recognize and move components" → Step 2 "grasp components" → Step 3 "move components toward the board" → Step 4 "approach" → Step 5 "recognize holes on the board" → Step 6 "approach holes" → Step 7 "hole tracing operation" → Step 8 "insert holes" → Step 9 "hole insertion completed" → Step 10 "release gripping," and the operation parameters for each of these steps are included in the setting library.

[0114] In each step, the control unit 170 (command generating unit 172) generates a control command to reduce the difference between the measurement information obtained using the sensor 300 and the target measurement information (target value of the measurement information), and to reduce the difference between the speed-related value and the speed-related target value.

[0115] In the illustrated example, for step 1 "recognize component → move," "component position" may be applied as the target measurement information and "speed" as the speed-related target value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector obtained using the visual sensor 310 and the position of the component, and to reduce the difference between the speed of the end effector and the speed target value. Here, the control unit 170 (target setting unit 176) may change the speed target value for each control cycle depending on the difference between the current position of the end effector obtained using the visual sensor 310 and the position of the component.

[0116] For step 2 "part gripping," "target reaction force" may be applied as the target measurement information, and "speed" may be applied as the speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the target reaction force and the current reaction force acting on the end effector, which is obtained using the force sensor 320, and to reduce the difference between the end effector speed and the speed target value. Here, the control unit 170 (target setting unit 176) may change the speed target value for each control cycle in accordance with the difference between the target reaction force and the reaction force acting on the end effector, which is obtained using the force sensor 320.

[0117] In step 3, "Move the component toward the board," "board position" may be applied as the target measurement information, and "jerk" may be applied as the velocity-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board, and to reduce the difference between the jerk of the end effector (or component) and the jerk target value. Here, the control unit 170 (target setting unit 176) may change the jerk target value for each control cycle according to the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board.

[0118] For step 4 "approach," "difference in position between the end effector and the board" may be applied as target measurement information, and "acceleration" may be applied as a velocity-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board, and to reduce the difference between the acceleration of the end effector and the acceleration target value. Here, the control unit 170 (target setting unit 176) may change the acceleration target value for each control cycle in accordance with the difference between the current position of the end effector (or component) obtained using the visual sensor 310 and the position of the board.

[0119] In step 5, “recognizing holes in a board,” “holes in a board” can be applied as target measurement information. In this case, the control unit 170 (command generation unit 172) uses the visual sensor 310 to recognize holes in a board.

[0120] For step 6 "Hole Approach," "Board Hole Position" and "Target Reaction Force" may be applied as target measurement information, and "Acceleration" may be applied as a velocity-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current position of the component obtained using the visual sensor 310 and the position of the hole on the board, and to reduce the difference between the acceleration of the end effector and the target acceleration value. The control unit 170 (command generation unit 172) may also generate a control command to reduce the difference between the current reaction force acting on the end effector obtained using the force sensor 320 and the target reaction force, and to reduce the difference between the acceleration of the end effector and the target acceleration value.

[0121] For each step from step 7 "hole copying operation" to step 10 "gripping release," a "target reaction force" may be applied as target measurement information for that step, and a "jerk" may be applied as a speed-related value. In this case, the control unit 170 (command generation unit 172) may generate a control command to reduce the difference between the current reaction force acting on the end effector obtained using the force sensor 320 and the target reaction force for that step, and to reduce the difference between the jerk of the end effector and the target jerk value for that step. Here, the control unit 170 (target setting unit 176) may change the target jerk value for each control cycle according to the difference between the target reaction force and the reaction force acting on the end effector obtained using the force sensor 320.

[0122] (5) Other Embodiments The operational flows and operational examples in the above-described embodiments do not necessarily have to be executed in chronological order according to the order shown in the flow diagrams. For example, steps in the operations may be executed in an order different from that shown in the flow diagrams, or may be executed in parallel. Furthermore, some steps in the operations may be deleted, or additional steps may be added to the processing.

[0123] A program may be provided that causes a computer to execute the operations according to the above-described embodiments. The program may be recorded on a computer-readable medium. The computer-readable medium can be used to install the program on a computer. Here, the computer-readable medium on which the program is recorded may be a non-transitory storage medium. The non-transitory storage medium is not particularly limited, and may be, for example, a storage medium such as a CD-ROM or a DVD-ROM.

[0124] The functions performed by the apparatus according to the above-described embodiments may be implemented in circuitry or processing circuitry, including general-purpose processors, application-specific processors, integrated circuits, ASICs (Application Specific Integrated Circuits), a CPU (a Central Processing Unit), conventional circuits, and / or combinations thereof, programmed to perform the described functions. A processor includes transistors or other circuits and is considered to be circuitry or processing circuitry. A processor may also be a programmed processor that executes a program stored in a memory. In this disclosure, circuitry, units, and means refer to hardware that is programmed to perform the described functions or that executes the described functions. The hardware may be any hardware disclosed herein or any hardware known to be programmed or capable of performing the described functions. If the hardware is a processor, the circuitry, means, or unit is a combination of the hardware and software used to configure the hardware and / or processor.

[0125] As used in this disclosure, the terms "based on" and "depending on" do not mean "based only on" or "depending only on," unless expressly stated otherwise. The term "based on" means both "based only on" and "based at least in part on." Similarly, the term "depending on" means both "depending only on" and "depending at least in part on." Furthermore, the terms "include," "comprise," and variations thereof do not mean including only the listed items, but may include only the listed items, or may include additional items in addition to the listed items. Furthermore, the term "or" used in this disclosure is not intended to mean an exclusive or. In this disclosure, when articles are added by translation, such as a, an, and the in English, these articles are intended to include the plural unless the context clearly indicates otherwise.

[0126] The above describes the embodiments in detail with reference to the drawings, but the specific configuration is not limited to that described above, and various design changes can be made within the scope that does not deviate from the gist of the invention.

[0127] (6) Supplementary Notes The following are additional notes regarding the features of the above-described embodiment.

[0128] Supplementary Note 1: A control device comprising: an acquisition unit that acquires measurement information obtained using a sensor for measuring an operating state of a robotic device; and a control unit that repeatedly generates control commands for the robotic device based on the measurement information, wherein the control unit derives a speed-related value for the robotic device from the measurement information or the control command, and generates the control command so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

[0129] Supplementary Note 2: The control device according to Supplementary Note 1, wherein the control unit includes: a derivation unit that derives the speed-related value from the measurement information or the control command; a target setting unit that variably sets the speed-related target value depending on an operating state of the robot device; and a command generation unit that generates the control command so as to reduce a difference between the speed-related target value and the speed-related value.

[0130] Supplementary Note 3: The control device according to Supplementary Note 1 or 2, wherein the velocity-related value includes at least one of velocity, acceleration, and jerk.

[0131] Supplementary Note 4: The control device according to any one of Supplementary Notes 1 to 3, wherein the control unit generates the control command to reduce a difference between the measurement information and target measurement information and to reduce a difference between the speed-related value and the speed-related target value.

[0132] Supplementary Note 5: The control device according to Supplementary Note 4, wherein the sensor includes a visual sensor, and the control unit generates the control command to reduce a difference between visual measurement information obtained using the visual sensor and target visual measurement information, and to reduce a difference between the speed-related value and the speed-related target value.

[0133] Supplementary Note 6: The control device according to Supplementary Note 4 or 5, wherein the sensor includes a force sensor, and the control unit generates the control command to reduce a difference between force measurement information obtained using the force sensor and target force measurement information, and to reduce a difference between the velocity-related value and the velocity-related target value.

[0134] Supplementary Note 7: The control device according to any one of Supplementary Notes 1 to 6, wherein the control unit changes the speed-related target value in accordance with a difference between the measurement information and target measurement information.

[0135] Supplementary Note 8: The control device according to Supplementary Note 7, wherein the sensor includes a visual sensor, and the control unit changes the speed-related target value according to a difference between visual measurement information obtained using the visual sensor and target visual measurement information.

[0136] Supplementary Note 9: The control device according to Supplementary Note 7 or 8, wherein the sensor includes a force sensor, and the control unit changes the speed-related target value according to a difference between force measurement information obtained using the force sensor and target force measurement information.

[0137] Supplementary Note 10: The control device according to any one of Supplementary Notes 7 to 9, wherein the control unit selects one or more values ​​to be used as the speed-related value from among speed, acceleration, and jerk, depending on a difference between the measurement information and target measurement information.

[0138] Supplementary Note 11: The control device according to any one of Supplementary Notes 1 to 10, wherein the control unit includes: a prediction unit that predicts a control result for a subsequent control operation performed on the robot device, based on a control operation performed on the robot device and a control result for the control operation; and a correction unit that corrects the subsequent control operation, based on the prediction.

[0139] Supplementary Note 12: The control device according to any one of Supplementary Notes 1 to 11, further comprising a library storage unit that stores a plurality of setting libraries prepared for each type of work, each of the plurality of setting libraries including setting information related to a series of processes, and the control unit performs the control using a setting library selected from the plurality of setting libraries according to the type of work actually to be performed.

[0140] Supplementary Note 13: The control device according to Supplementary Note 12, wherein each of the plurality of setting libraries includes setting information for the speed-related target value.

[0141] Supplementary Note 14: The control device according to any one of Supplementary Notes 1 to 13, further comprising a library storage unit that stores a plurality of recognition libraries prepared for each type of work, each of the plurality of recognition libraries including a trained model used for image recognition processing of an object, and the control unit performs the image recognition processing using a recognition library selected from the plurality of recognition libraries according to the type of work actually to be performed.

[0142] Supplementary Note 15: A control method comprising: acquiring measurement information obtained using a sensor for measuring an operating state of a robotic device; and repeatedly generating control commands for the robotic device based on the measurement information, wherein repeatedly generating the control commands includes deriving a speed-related value for the robotic device from the measurement information or the control commands, and generating the control commands so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

[0143] Supplementary Note 16: A program that causes a control device to acquire measurement information obtained using a sensor for measuring an operating state of a robotic device, and repeatedly generate control commands for the robotic device based on the measurement information, wherein repeatedly generating the control commands includes deriving a speed-related value for the robotic device from the measurement information or the control commands, and generating the control commands so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

Claims

1. A control device comprising: an acquisition unit that acquires measurement information obtained using a sensor for measuring the operating state of a robot device; and a control unit that repeatedly generates control commands for the robot device based on the measurement information, wherein the control unit derives a speed-related value for the robot device from the measurement information or the control command, and generates the control command so as to reduce the difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robot device.

2. The control device according to claim 1, wherein the control unit comprises: a derivation unit that derives the speed-related value from the measurement information or the control command; a target setting unit that variably sets the speed-related target value according to the operating state of the robot device; and a command generation unit that generates the control command so as to reduce the difference between the speed-related target value and the speed-related value.

3. The control device according to claim 1, wherein the velocity-related value includes at least one of velocity, acceleration, and jerk.

4. A control device as claimed in any one of claims 1 to 3, wherein the control unit generates the control command so as to reduce the difference between the measurement information and the target measurement information and to reduce the difference between the speed-related value and the speed-related target value.

5. The control device according to claim 4, wherein the sensor includes a visual sensor, and the control unit generates the control command to reduce the difference between visual measurement information obtained using the visual sensor and target visual measurement information, and to reduce the difference between the speed-related value and the speed-related target value.

6. The control device according to claim 4, wherein the sensor includes a force sensor, and the control unit generates the control command to reduce the difference between force measurement information obtained using the force sensor and target force measurement information, and to reduce the difference between the speed-related value and the speed-related target value.

7. The control device according to any one of claims 1 to 3, wherein the control unit changes the speed-related target value in accordance with the difference between the measurement information and target measurement information.

8. The control device according to claim 7, wherein the sensor includes a visual sensor, and the control unit changes the speed-related target value according to the difference between visual measurement information obtained using the visual sensor and target visual measurement information.

9. The control device according to claim 7, wherein the sensor includes a force sensor, and the control unit changes the speed-related target value according to the difference between force measurement information obtained using the force sensor and target force measurement information.

10. The control device according to claim 7, wherein the control unit selects one or more values ​​to be used as the speed-related value from among speed, acceleration, and jerk, depending on the difference between the measurement information and target measurement information.

11. A control device according to any one of claims 1 to 3, wherein the control unit has: a prediction unit that predicts the control results of subsequent control operations performed on the robot device based on the control operations performed on the robot device and the control results of the control operations; and a correction unit that corrects the subsequent control operations based on the prediction.

12. A control device according to any one of claims 1 to 3, further comprising a library storage unit that stores a plurality of setting libraries prepared for each type of work, each of the plurality of setting libraries containing setting information relating to each of a series of processes, and the control unit performs the control using a setting library selected from the plurality of setting libraries according to the type of work actually to be performed.

13. The control device according to claim 12, wherein each of the plurality of setting libraries includes setting information for the speed-related target value.

14. A control device as claimed in any one of claims 1 to 3, further comprising a library storage unit that stores a plurality of recognition libraries prepared for each type of work, each of the plurality of recognition libraries including a trained model used for image recognition processing of an object, and the control unit that performs the image recognition processing using a recognition library selected from the plurality of recognition libraries according to the type of work actually to be performed.

15. A control method comprising: acquiring measurement information obtained using a sensor for measuring the operating state of a robotic device; and repeatedly generating control commands for the robotic device based on the measurement information, wherein repeatedly generating the control commands includes deriving a speed-related value for the robotic device from the measurement information or the control commands, and generating the control commands so as to reduce a difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robotic device.

16. A program that causes a control device to acquire measurement information obtained using a sensor for measuring the operating state of a robot device, and repeatedly generate control commands for the robot device based on the measurement information, wherein repeatedly generating the control commands includes deriving a speed-related value for the robot device from the measurement information or the control commands, and generating the control commands so as to reduce the difference between the speed-related value and a speed-related target value that is variable depending on the operating state of the robot device.

Images