Control device, control system, and program

Abstract

This control device comprises: a plurality of sensor signal acquisition units (11) that each acquire a sensor signal output from a mechanical device or from a sensor provided in the periphery of the mechanical device; a storage unit (12) that stores the plurality of sensor signals acquired by the plurality of sensor signal acquisition units (11) and time information indicating the time at which each sensor signal was acquired; a state estimation unit (13) that estimates the state of an object on the basis of two or more sensor signals designated among the plurality of stored sensor signals and the time information about each of the two or more designated sensor signals; and a control computation unit (14) that calculates, on the basis of the estimated state of the object, a command value to be sent to the mechanical device.

Description

Control device, control system, and program

[0001] The present disclosure relates to a control device, a control system, and a program for controlling a mechanical device.

[0002] When moving an object by driving a mechanical device, the control device may detect the state of the same object using multiple sensors and control the mechanical device based on information indicating the detection results from each sensor. In this case, the control device integrates the detection results from each sensor and controls the mechanical device based on the integrated information.

[0003] Patent Document 1 discloses a control system that enables high-precision position control by reducing discrepancies in the position information of multiple devices that update their position information at different cycles when controlled by a controller. The control system disclosed in Patent Document 1 includes a controller and two or more processing devices connected to the controller and performing predetermined processing related to a workpiece. In the control system disclosed in Patent Document 1, each processing device includes a timer and a position information update unit that updates its position information. In the control system disclosed in Patent Document 1, the controller includes a position conversion unit that converts the position information output from each processing device into world coordinate system position information indicating a position on the world coordinate system, an estimation unit that estimates the position of the workpiece on the world coordinate system during the control cycle of the controller, and a world coordinate system management unit that manages the world coordinate system position information and position information indicating the estimated position of the workpiece on a time axis.

[0004] Japanese Patent Application Laid-Open No. 2019-212123

[0005] According to Patent Document 1, it is possible to reduce errors caused by a time difference between position information indicating the position of a workpiece recognized by a controller and position information output from each processing device. However, when detecting the state of the same object using multiple sensors and integrating the detection results from each sensor, it is necessary to consider physical errors in the multiple sensors, such as coordinate transformation errors or modeling errors. The configuration disclosed in Patent Document 1 does not include a means for reducing such physical errors. Therefore, the conventional technology disclosed in Patent Document 1 has a problem in that it is difficult to accurately integrate information obtained by multiple sensors about the same object.

[0006] The present disclosure has been made in consideration of the above, and aims to provide a control device that can accurately integrate information acquired by each of multiple sensors about the same object.

[0007] In order to solve the above-mentioned problems and achieve the object, a control device according to the present disclosure is a control device for controlling a mechanical device that moves an object, and includes: a plurality of sensor signal acquisition units that each acquire a sensor signal output from a sensor provided on the mechanical device or in the vicinity of the mechanical device; a memory unit that stores the plurality of sensor signals acquired by the plurality of sensor signal acquisition units and time information indicating the time at which each sensor signal was acquired; a state estimation unit that estimates a state of the object based on two or more designated sensor signals from the stored plurality of sensor signals and the time information for each of the designated two or more sensor signals; and a control calculation unit that calculates a command value to be sent to the mechanical device based on the estimated state of the object.

[0008] The control device according to the present disclosure has the effect of being able to accurately integrate information acquired by each of a plurality of sensors regarding the same object.

[0009] FIG. 1 is a diagram showing a configuration example of a control device according to embodiment 1. FIG. 1 is a diagram showing a configuration example of a control system having a control device according to embodiment 1. FIG. 2 is a flowchart showing an example of an operation procedure of a control device according to embodiment 1. FIG. 2 is a diagram for explaining an example of estimating the state of an object by a state estimation unit included in a control device according to embodiment 1. FIG. 3 is a diagram for explaining inputs and outputs in a plurality of sensor signal acquisition units, a memory unit, and a state estimation unit included in a control device according to embodiment 1. FIG. 4 is a diagram for explaining calculation of a command value by a control calculation unit included in a control device according to embodiment 1. FIG. 3 is a diagram showing a configuration example of a control system according to embodiment 2. FIG. 4 is a diagram showing a configuration example of a control system according to embodiment 3. FIG. 5 is a diagram showing a configuration example of a control system according to embodiment 4. FIG. 5 is a diagram showing a configuration example of a control circuit according to embodiments 1 to 4. FIG. 6 is a diagram showing a configuration example of a hardware circuit according to embodiments 1 to 4.

[0010] A control device, a control system, and a program according to an embodiment will be described in detail below with reference to the accompanying drawings.

[0011] First Embodiment Fig. 1 is a diagram showing an example of the configuration of a control device according to the first embodiment. Fig. 1 shows a controller 1A which is the control device according to the first embodiment.

[0012] The controller 1A controls a mechanical device that moves an object. In the first embodiment, the mechanical device controlled by the controller 1A is a robot that grips and moves an object. The robot is installed near a processing machine that processes the object. The robot moves the object to a position in the processing machine where the object is to be placed. The processing machine processes the object placed in that position.

[0013] 2 is a diagram showing an example of the configuration of a control system 10A having a control device according to embodiment 1. The control system 10A includes a controller 1A, a robot 2 connected to the controller 1A, and a sensor connected to the controller 1A. The robot 2 is a robot controlled by the controller 1A. The robot 2 is assumed to be a robot having a robot arm.

[0014] The cameras 31, 32, and force sensor 4 are sensors connected to the controller 1A. The cameras 31, 32, and force sensor 4 detect the state of an object. The cameras 31, 32 are provided around the robot 2. The cameras 31, 32 are vision sensors that detect the position of the object by photographing the object. The cameras 31, 32 each output an image signal to the controller 1A. The force sensor 4 is provided on a robot arm of the robot 2. The force sensor 4 detects a force acting on the object by detecting a force acting on the robot arm of the robot 2. The force sensor 4 outputs a signal indicating the result of the force detection to the controller 1A.

[0015] The control system 10A includes a plurality of controllers 51, 52, 53, and 54 connected to a controller 1A. Hereinafter, the term "controller 5" will be used to refer to the controllers 51, 52, 53, and 54 without making a distinction between them.

[0016] A camera 6 is connected to the controller 51. A tactile sensor 7 is connected to the controller 52. The camera 6 and the tactile sensor 7 are each a sensor that detects the state of an object. The camera 6 is provided in the periphery of the robot 2. The camera 6 is a vision sensor that detects the position of an object by photographing the object. The camera 6 outputs an image signal to the controller 51. The controller 51 sends the image signal acquired from the camera 6 to the controller 1A. The tactile sensor 7 is provided in the hand of the robot 2. The tactile sensor 7 detects the state in which the object is being grasped by the hand. The tactile sensor 7 outputs a signal indicating the result of detecting the state in which the object is being grasped to the controller 52. The controller 52 sends the signal acquired from the tactile sensor 7 to the controller 1A.

[0017] The controller 53 is connected to the robot 8. The robot 8 is a robot that operates in the vicinity of the robot 2. The controller 53 controls the robot 8. The controller 53 sends information indicating the operating state of the robot 8 to the controller 1A.

[0018] The controller 54 is connected to the processing machine 9. The processing machine 9 processes an object carried to the placement position by the robot 2. The controller 54 controls the processing machine 9. The controller 54 sends information indicating the operating state of the processing machine 9 to the controller 1A.

[0019] Controller 1A acquires signals output from each of camera 31, camera 32, and force sensor 4 connected to controller 1A. Controller 1A acquires signals output from each of camera 6 and tactile sensor 7 by receiving signals sent from each of controllers 51 and 52. Controller 1A acquires information indicating the operating state of robot 8 by receiving information sent from controller 53. Controller 1A acquires information indicating the operating state of processing machine 9 by receiving information sent from controller 54.

[0020] Based on the sensor signals output from the sensors, the controller 1A corrects the command values ​​for controlling the robot 2. The controller 1A adjusts the drive of the robot 2 based on the state of the object recognized from the sensor signals.

[0021] The controller 1A corrects the command values ​​for controlling the robot 2 based on information indicating the states of peripheral devices such as the robot 8 or the processing machine 9. The controller 1A adjusts the driving of the robot 2 in accordance with the states of the peripheral devices by correcting the command values ​​based on the information indicating the states of the peripheral devices.

[0022] 2, the controller 1A is connected to the camera 31, the camera 32, and the force sensor 4. The controller 1A may be connected to sensors other than the cameras 31, 32 and the force sensor 4. The number of sensors connected to the controller 1A is arbitrary.

[0023] 2, the camera 6 and the tactile sensor 7 are each connected to the controller 1A via the controller 5. Sensors other than the camera 6 and the tactile sensor 7 may be connected to the controller 1A via the controller 5. Furthermore, the number of sensors connected to the controller 1A via the controller 5 is arbitrary.

[0024] 2, the robot 8 and the processing machine 9 are connected to the controller 1A via the controller 5. Peripheral devices other than the robot 8 and the processing machine 9 may be connected to the controller 1A via the controller 5. The number of peripheral devices connected to the controller 1A via the controller 5 is arbitrary. Furthermore, there may be no peripheral devices connected to the controller 1A via the controller 5. In the control system 10A, it is sufficient that there are multiple sensors connected to the controller 1A directly or via the controller 5.

[0025] Next, a configuration example of the controller 1A shown in Fig. 1 will be described. The controller 1A includes a plurality of sensor signal acquisition units 11, a storage unit 12, a state estimation unit 13, a control calculation unit 14, and a signal output unit 15.

[0026] Each of the multiple sensor signal acquirers 11 acquires a sensor signal output from a sensor provided on the mechanical device or in the vicinity of the mechanical device. In the example shown in Fig. 1 , sensor signals output from five sensors are input to the controller 1A directly or via the controller 5. The controller 1A acquires five sensor signals by each of the five sensor signal acquirers 11 acquiring a sensor signal. Two of the five sensor signal acquirers 11 are shown in Fig. 1 .

[0027] The multiple sensor signals acquired by the multiple sensor signal acquisition unit 11 may be either analog signals or digital signals. The multiple sensor signals may include both analog and digital signals. The sensor signals output from the cameras 31, 32, and 6 are image signals. The sensor signal output from the force sensor 4 is a force signal indicating the magnitude of a force or a moment signal indicating a moment. The sensor signal output from the tactile sensor 7 is a signal indicating the distribution of frictional force on the surface contacting the object, a signal indicating the amount of slippage, or a signal indicating the presence or absence of contact. The sensor signal may be a signal other than these signals. For example, the sensor signal may be an encoder signal output by an encoder in a motor that drives a robot arm of the robot 8, or a signal indicating the current value of the current flowing through the motor. Each sensor signal acquisition unit 11 outputs the acquired sensor signal to the memory unit 12.

[0028] Each sensor signal acquirer 11 converts the acquired sensor signal into information indicating a physical quantity by executing a recognition process on the acquired sensor signal. When the sensor signal acquirer 11 acquires an image signal, it extracts a feature quantity through the recognition process. In this case, the sensor signal acquirer 11 outputs, as a recognition result, coordinates indicating the position where the feature quantity is extracted in the image, or information indicating the area of ​​the region where the feature quantity is extracted. The sensor signal acquirer 11 calculates a target physical quantity according to the content of the recognition process implemented in the sensor signal acquirer 11. Each sensor signal acquirer 11 outputs the calculated physical quantity, i.e., the recognition result, to the storage unit 12.

[0029] Each sensor signal acquirer 11 generates time information indicating the time at which the sensor signal was acquired. Each sensor signal acquirer 11 outputs the generated time information to the storage unit 12. For example, each sensor signal acquirer 11 is provided with a timer, and each sensor signal acquirer 11 generates the time information by recording the time at which the sensor signal was acquired based on the output of the timer. Each sensor signal acquirer 11 outputs the time information to the storage unit 12 together with the sensor signal and the recognition result.

[0030] The memory unit 12 stores multiple sensor signals acquired by multiple sensor signal acquisition units 11, recognition results output from each sensor signal acquisition unit 11, and time information indicating the time when the sensor signal was acquired by each sensor signal acquisition unit 11.

[0031] The state estimation unit 13 estimates the state of the object at the time the mechanical device is controlled based on two or more specified sensor signals from the multiple sensor signals stored in the memory unit 12 and time information for each of the two or more specified sensor signals. The state of the object estimated by the state estimation unit 13 includes, for example, the position of the object, the velocity or acceleration of the object's movement, the force acting on the object from the environment, or the contact state between the object and the environment. Here, the environment refers to structures present around the object. Examples of the environment include the walls or floor of a room in which the mechanical device is installed, or a partition placed in the room in which the mechanical device is installed. The state estimation unit 13 also weights each of the two or more specified sensor signals and estimates the state of the object based on the weighted two or more sensor signals.

[0032] The state estimation unit 13 may estimate the state of the mechanical device and regard the estimated state of the mechanical device as the estimated state of the object. In this case, the state of the mechanical device estimated by the state estimation unit 13 may be, for example, the position of a part of the mechanical device that moves the object, the speed or acceleration of the movement of the part, the acting force that the part receives from the environment, or the contact state between the part and the environment.

[0033] The state estimation unit 13 outputs the estimation result of the state of the object to each of the storage unit 12 and the control calculation unit 14. The storage unit 12 stores the estimation result of the state of the object.

[0034] The control calculation unit 14 calculates a command value to be sent to the mechanical device based on the estimated state of the object. The signal output unit 15 outputs a control signal indicating the command value calculated by the control calculation unit 14 to the robot 2, which is the mechanical device. The signal output unit 15 also outputs a trigger signal to the robot 2, which is the mechanical device.

[0035] The controller 1A acquires information indicating the status of peripheral devices such as the robot 8 or the processing machine 9. The control calculation unit 14 adjusts the drive of the robot 2 based on the information indicating the status of the peripheral devices. In Fig. 1, elements that acquire information indicating the status of the peripheral devices are not shown.

[0036] Next, the operation procedure of the controller 1A will be described. Fig. 3 is a flowchart showing an example of the operation procedure of the control device according to the first embodiment.

[0037] In step S1, each of the plurality of sensor signal acquirers 11 acquires a sensor signal. In step S2, each of the plurality of sensor signal acquirers 11 executes a recognition process for the sensor signal acquired in step S1. Each of the plurality of sensor signal acquirers 11 outputs the sensor signal acquired in step S1, a recognition result obtained by the recognition process, and time information to the storage unit 12. In step S3, the storage unit 12 stores the sensor signal, the recognition result, and the time information.

[0038] The state estimation unit 13 reads the sensor signal, the recognition result, and the time information stored in the storage unit 12. In step S4, the state estimation unit 13 estimates the state of the object based on the sensor signal, the recognition result, and the time information. The state estimation unit 13 outputs the estimation result of the state of the object to each of the storage unit 12 and the control calculation unit 14. The storage unit 12 stores the estimation result of the state of the object.

[0039] In step S5, the control calculation unit 14 calculates a command value based on the result of estimating the state of the object. The control calculation unit 14 outputs the calculated command value to the signal output unit 15. In step S6, the signal output unit 15 outputs the command value calculated in step S5 to the robot 2. With the above, the controller 1A ends the operation according to the procedure shown in FIG.

[0040] Next, a description will be given of an example of a method for estimating the state of an object by the state estimation unit 13. Here, a case where the state estimation unit 13 estimates the position of an object to be moved by the robot 2 will be mainly described.

[0041] 4 is a diagram for explaining an example in which the state of an object is estimated by the state estimation unit 13 included in the control device according to embodiment 1. Here, it is assumed that the sensor signals acquired by the controller 1A are sensor signals output by the cameras 3 and 3 and the force sensor 4. FIG. 4 shows the robot 2 and the cameras 3 and 3 and the force sensor 4, which are sensors.

[0042] The robot 2 has a robot arm 21 and an end effector 22 attached to the tip of the robot arm 21. Fig. 4 shows a state in which an object 23 grasped by the end effector 22 is being moved by driving the robot arm 21.

[0043] 4, cameras 31 and 32 are installed around robot 2 at positions where they can capture images of object 23. Cameras 31 and 32 capture images of object 23 from different positions. Force sensor 4 is installed on robot 2 between robot arm 21 and end effector 22. Controller 1A acquires sensor signals output from cameras 31 and 32 and force sensor 4 using multiple sensor signal acquisition units 11.

[0044] The sensor signal acquisition unit 11 acquires the image signals, which are sensor signals output from the cameras 31 and 32, and performs recognition processing on the image signals to obtain position information indicating the position of the object 23 captured in the image. The sensor signal acquisition unit 11 acquires the image signals and generates time information indicating the time when the image signals were acquired. The sensor signal acquisition unit 11 outputs the image signals, the position information, which is the recognition result, and the time information.

[0045] It is assumed that the origin serving as the reference for the position indicated in the position information is positioned in advance in the local coordinate system, which is the coordinate system of the robot 2. Specifically, the origin of the local coordinate system is adopted as the origin of the position information, and calibration is performed in advance between the robot 2 and each of the cameras 31 and 32. As a result, the sensor signal acquisition unit 11 outputs position information indicating the position in the local coordinate system.

[0046] In the first embodiment, the mechanical device is the robot 2. The mechanical device may be any device that performs the task of moving an object, and is not limited to the robot 2. The mechanical device may be a processing machine 9, a combination of multiple Cartesian robots, or a combination of an autonomous mobile robot (AMR) and the robot 2, for example.

[0047] The state estimation unit 13 reads the position information and time information acquired by the sensor signal acquisition unit 11 from the storage unit 12. The state estimation unit 13 estimates the position of the object at a certain time based on the position information and time information that are information read from the storage unit 12.

[0048] Each of the multiple sensors included in the control system 10A typically detects the state of an object at a cycle set for each sensor and outputs a sensor signal. Therefore, the controller 1A may manage the object's position information at each time by time-synchronizing the multiple sensor signals stored in the storage unit 12. However, simply integrating the position information acquired by the sensor signals from two or more sensors through time synchronization alone may result in allowing for physical errors in the two or more sensors.

[0049] Specifically, even if two cameras 31 and 32 detect the position of the same object, when attempting to combine position information acquired from the image signals from each camera 31 and 32, it may not be possible to uniquely determine the position of the object. This can be said to mean that even if each camera 31 and 32 detects the position of the same object, there are only two candidates for position information that can uniquely determine the position of the object. This phenomenon can occur due to differences in the detection capabilities of each camera 31 and 32 or physical errors such as modeling errors. In embodiment 1, when combining two or more pieces of information indicating the state of the object, the controller 1A verifies the degree of likelihood of each sensor signal and prioritizes the integration of the more likely information. In embodiment 1, the operation of verifying the degree of likelihood of each sensor signal and integrating the information by prioritizing the more likely information is referred to as a state estimation process.

[0050] In the state estimation process, first, a "certain time" is set when the position of the object at that time is estimated. The "certain time" is the time when the control of the robot 2 is executed based on the command value generated by the control calculation unit 14. The current time recognized by the control calculation unit 14 is denoted as T_ cur Then, "a certain time" is T_ cur For example, the time lag T_ between when the control signal is generated and when the control of the robot 2 is executed may be lag If there exists a "certain time" T_ cur T_ lag That is, the control command time, which is the time when the control is executed, is set to T_ cmd Then, T_ cmd =T_ cur +T_ lag can be treated as "a certain time". Hereinafter, "a certain time" is T_ cmd It is assumed that:

[0051] Next, the state estimation unit 13 performs state estimation using the sensor signal and recognition result at the current time and the past state estimation result as input. Here, the number of times control has been executed since the control calculation unit 14 started the process for control is referred to as "step." Hereinafter, T_ cmd (K) represents the control command time in the Kth step, where K is an integer equal to or greater than 2. R se _(K) is T_ cmd represents the state estimation result in (K). cur (K) represents the current time at the Kth step.

[0052] The state estimation unit 13 estimates the past state se _(K-1) and T_ cur The sensor signal and the recognition result of (K) are input, and the state estimation result for the control command time in the Kth step is R se Find (K). R se _(K-1) is the control command time T_ at the (K-1)th step. cmdThe state estimation unit 13 detects the state of T_(K-1). cmd (K) the likely sensor signals and recognition results for T cmd Based on the evaluation of the continuity of the state estimation results in (K-1), T cmd Estimate the likely state in (K).

[0053] In the state estimation process, the state estimation unit 13 weights the input information by utilizing the fact that the properties of the sensor signals differ depending on the characteristics of each sensor. Here, the properties of the sensor signals that differ depending on the characteristics of each sensor will be described.

[0054] One example of a characteristic of each sensor is the length of the sampling period. The multiple sensors provided in the control system 10A include sensors with short sampling periods and sensors with long sampling periods. One difference in the properties of the sensor signals is the frequency at which the sensor signals are output. For example, cameras 31 and 32 output sensor signals at a low frequency of 10 Hz to 100 Hz. In contrast, force sensor 4 outputs sensor signals at a high frequency of about 1000 Hz. In other words, force sensor 4 can be said to be a sensor capable of high-speed sampling.

[0055] Furthermore, the cameras 31 and 32 are susceptible to disturbances from light. As a result, the reliability of the sensor signals output from the cameras 31 and 32 as input information decreases under certain conditions. Photoelectric sensors that detect the presence or absence of an object based on the properties of the received light are also susceptible to disturbances from light, and therefore, under certain conditions, the reliability of the input information decreases. Furthermore, like the force sensor 4, the tactile sensor 7 can output sensor signals at a high frequency, and can be said to be a sensor capable of high-speed sampling.

[0056] From the sensor signal output from the force sensor 4, it is possible to recognize that the object has reached a position where it comes into contact with the environment at the timing when the object begins to receive an acting force from the environment. In other words, the sensor signal output from the force sensor 4 can be used as position information that indicates the position of the object.

[0057] The force sensor 4 is capable of high-speed sampling and can therefore detect with high precision the timing at which the force acting on the object changes. In other words, it can be said that the position information based on the sensor signals output from the force sensor 4 represents the position of the object with higher precision than the position information based on the sensor signals output from the cameras 31 and 32.

[0058] Due to such differences in the characteristics of the sensor signals output from the cameras 3 and 3 and the sensor signal output from the force sensor 4, the state estimation unit 13 executes a state estimation process that prioritizes the position information derived from the force sensor 4 over the position information derived from the cameras 3 and 3. This enables the controller 1A to estimate the state of the object with high accuracy.

[0059] Furthermore, the low resolution of each of the cameras 3 and 3 may cause errors in the position information derived from the cameras 3 and 3. In this case, the state estimation unit 13 executes a state estimation process that prioritizes the position information derived from the force sensor 4, thereby enabling the controller 1A to estimate the state of the object with high accuracy.

[0060] The state estimation unit 13 can execute a state estimation process in which the recognition result based on the sensor signal output from the force sensor 4 is prioritized over the recognition result based on the sensor signals output from the cameras 31 and 32, based on the following equation (1). se , the recognition results based on the sensor signals output from the cameras 31 and 32 are R rc 1. The recognition result based on the sensor signal output from the force sensor 4 is R rc 2. A weighting parameter representing the weighting of the recognition result based on the sensor signals output from the cameras 31 and 32 and the recognition result based on the sensor signal output from the force sensor 4 is denoted by Pm1. nc is a function representing the state estimation process. se =F nc (R rc 1. R rc 2, Pm1) ... (1)

[0061] The weighting parameter Pm1 is a function F that represents the state estimation process. nc This allows the user of the control system 10A to freely define weighting in the state estimation process.

[0062] In a state where contact between the object and the environment is recognized based on the sensor signal output from the force sensor 4, the state information of the object may be updated on the assumption that the position of the object is constrained to the position at which it contacted the environment. For example, T cur In (K-1), it is assumed that contact between the object and the environment is recognized based on the sensor signal output from the force sensor 4. cur If it is continued in (K), T_ cur Assuming that the position of the object does not change from (K-1), T_ cur State information for (K) is calculated. In this way, the state information of the object is updated. Thereafter, a conditional weighting may be defined indicating that no displacement occurs in a specific traveling direction as long as the force sensor 4 detects that a force is acting on the object. This allows the state estimation unit 13 to perform state estimation processing in which the position information derived from the force sensor 4 is prioritized over the position information derived from each of the cameras 31 and 32.

[0063] When an object is determined to be in contact with the environment, regarding the object as being in a position where it is in contact with the environment can be said to be a constraint on the positional relationship between the object and the environment. In the above, the priority of the recognition result when such a constraint can be assumed is set as a weighting parameter. Note that the weighting parameter is not limited to being set based on the priority based on such a constraint. The weighting parameter may also be variable for each control cycle. Below, an example will be described in which two or more specified sensor signals from multiple sensor signals are combined. Note that the user can specify any two or more sensor signals from the multiple sensor signals.

[0064] For example, when product assembly work using the robot 2 continues, the controller 1A generates, as the control signal for the robot 2, a control signal synchronized with the sampling period of a sensor capable of high-speed sampling, such as the force sensor 4 or the tactile sensor 7. In this case, the sensor signals input to the controller 1A from the cameras 3 and 3, which have a low sampling rate, are not updated for each control period of the robot 2. For this reason, the position information of the target object is calculated using sensor signals such as encoder signals other than the sensor signals from the cameras 3 and 3, recognition results for these other sensor signals, or speed information included in past state estimation results.

[0065] Also, for example, assume that the controller 1A recognizes the position of an object in three dimensions from two-dimensional position information acquired by each of the two cameras 31 and 32, and the robot 2 then carries the object into the processing machine 9. The three mutually perpendicular axes are the X-axis, Y-axis, and Z-axis. In this case, one of the two cameras 31 and 32 estimates the position of the object in an XZ plane including the X-axis and Z-axis, and the amount of rotation of the object within the XZ plane. Furthermore, the other of the two cameras 31 and 32 estimates the position of the object in a YZ plane including the Y-axis and Z-axis, and the amount of rotation of the object within the YZ plane. In this case, the position in the Z-axis direction is measured redundantly by the two cameras 31 and 32. However, even if position feedback control is performed based on the measurements by each camera 31 and 32, the position of the object may not converge to the desired position due to physical errors in each camera 31 and 32.

[0066] In response to this, the state estimation unit 13 performs state estimation processing to estimate the position of the object as viewed from the local coordinate system of the robot 2 or the tip position of the end effector 22, and the controller 1A can bring the position of the object closer to the target position.

[0067] For example, a recognition result obtained from a sensor signal from only one camera may contain errors due to problems such as parallax, physical errors, or occlusion. In such cases, the state estimation unit 13 performs state estimation based on sensor signals from several cameras other than the one camera, and the controller 1A controls the robot 2 based on the result of the state estimation. This allows the controller 1A to control the position of the object or the tip position of the end effector 22 to approach the target position.

[0068] The controller 1A integrates two or more sensor signals depending on the properties of the sensor signals, the situation of the robot 2 or its surroundings, or physical errors of each sensor, and estimates the state of the object or the robot 2. The controller 1A performs state estimation processing by weighting each of the two or more sensor signals according to their priority, thereby enabling the controller 1A to estimate the state of the object with high accuracy.

[0069] 5 is a diagram for explaining inputs and outputs of the plurality of sensor signal acquirers 11, the storage unit 12, and the state estimation unit 13 included in the control device according to embodiment 1. FIG. 5 shows two of the plurality of sensor signal acquirers 11 included in the controller 1A, the storage unit 12, and the state estimation unit 13.

[0070] Each sensor signal acquisition unit 11 receives a sensor signal S sc Each sensor signal acquisition unit 11 receives the sensor signal S sc and time information I indicating the time when the sensor signal was acquired. T and the sensor signal S sc The recognition result R obtained by the recognition process for rc and output to the storage unit 12.

[0071] The state estimation unit 13 receives the sensor signal S sc and time information I T and the recognition result R rc The state estimation unit 13 reads R, which is the state estimation result for the control command time in the Kth step. seWhen calculating K, the state estimation unit 13 uses R, which is the state estimation result of the previous step. se _(K-1) and the (K-1)th step sensor signal S sc (K-1) from the storage unit 12. The state estimation unit 13 uses the information read from the storage unit 12 to calculate the state estimation result R se The state estimation unit 13 calculates the state estimation result R se The control calculation unit 14 outputs the R (K) stored in the storage unit 12 to the control calculation unit 14 and the storage unit 12. se _(K) and S stored in the storage unit 12 sc The sensor signal at the Kth step, where k is the sensor signal length, is read into the state estimation unit 13 when determining a state estimation result for the control command time in the (K+1)th step. In this way, the state estimation unit 13 determines a state estimation result for the control command time using the past state estimation results and past sensor signals stored in the storage unit 12.

[0072] For example, when the time is zero, the state estimation unit 13 determines the tip position of the end effector 22 based on the angles indicated by the output signals from the encoders in each axis motor of the robot arm 21 and the link parameters used in forward kinematics, assuming that there is no model error. In this case, the state estimation unit 13 determines the tip position of the end effector 22, assuming that there is no modeling error between the cameras 3 and 3 and the local coordinate system of the robot 2. The state estimation unit 13 also determines the position of the object, assuming that there is no modeling error between the cameras 3 and 3 and the local coordinate system of the robot 2. The link parameters are mechanism parameters used to express a serial link function, which is a type of mechanism possessed by the robot arm 21. Furthermore, for example, two or more processing systems included in the control system 10A synchronize their times using a timing device such as a global timer, and the time used as the reference for such time synchronization is set to zero.

[0073] Next, a description will be given of calculation of the command value by the control calculation unit 14. Fig. 6 is a diagram for explaining calculation of the command value by the control calculation unit 14 included in the control device according to the first embodiment.

[0074] The control calculation unit 14 receives a command value P cmd is input. Here, the ideal target position is an ideal target position as viewed from the local coordinate system of the robot 2. The command value representing the ideal target position is a command value generated in advance, and is a command value generated without taking physical errors into consideration. Also, the state estimation result output from the state estimation unit 13 is input to the control calculation unit 14. Here, R, which is the state estimation result for the control command time in the Kth step, is se _(K) is input to the control calculation unit 14. The ideal target position input to the control calculation unit 14 is the ideal target position in the Kth step.

[0075] The control calculation unit 14 calculates the state estimation result R se Based on the calculated correction amount, ΔP is calculated as the correction amount used to correct the command value. The control calculation unit 14 calculates the command value P after correction using the calculated correction amount. cmd +ΔP. The controller 1A generates the command value P cmd +ΔP is output to robot 2.

[0076] As a result, the controller 1A controls the robot 2 based on the command value that reflects the result of estimating the state of the object, and the controller 1A can realize control of the robot 2 taking the state of the object into consideration.

[0077] According to the first embodiment, the control device includes a plurality of sensor signal acquisition units 11 that each acquire a sensor signal; a memory unit 12 that stores the plurality of sensor signals acquired by the plurality of sensor signal acquisition units 11 and time information indicating the time at which each sensor signal was acquired; a state estimation unit 13 that estimates a state of an object based on two or more designated sensor signals from the stored plurality of sensor signals and the time information for each of the designated two or more sensor signals; and a control calculation unit 14 that calculates a command value to be sent to a mechanical device based on the estimated state of the object. By estimating the state of the object based on the two or more designated sensor signals, the control device can integrate the detection results of each sensor while taking into account physical errors in the multiple sensors. This provides the effect of accurately integrating information acquired by each of the multiple sensors for the same object.

[0078] Furthermore, the state estimation unit 13 weights each of the two or more specified sensor signals and estimates the state of the object based on the two or more weighted sensor signals. This allows the control device to perform state estimation processing by weighting each of the two or more sensor signals according to their priority, and to estimate the state of the object with high accuracy.

[0079] Embodiment 2. Fig. 7 is a diagram showing an example of the configuration of a control system 10B according to embodiment 2. The control system 10B includes a controller 1B which is a first control device, a controller 5B which is a second control device, and an information sharing unit 33. In embodiment 2, the same components as those in embodiment 1 are denoted by the same reference numerals, and the following description will mainly focus on the configuration that differs from embodiment 1.

[0080] The controller 1B has a configuration similar to that of the controller 1A shown in Fig. 1. The controller 1B acquires sensor signals output from each of a plurality of sensors connected to the controller 1B. Hereinafter, the first sensor signal will be referred to as the sensor signal acquired by the controller 1B.

[0081] The controller 5B is a controller connected to the controller 1B. The controller 5B controls a sensor provided in the mechanical device or in the vicinity of the mechanical device. The controller 5B also acquires a sensor signal output from the sensor. Hereinafter, the second sensor signal refers to a sensor signal acquired by the controller 5B. The controller 5B is connected to the controller 1B via the information sharing unit 33.

[0082] The controller 5B includes a sensor signal acquisition unit 31 and a storage unit 32. The sensor signal acquisition unit 31 acquires a sensor signal, i.e., a second sensor signal, output from a sensor connected to the controller 5B. The sensor signal acquisition unit 31 outputs the acquired sensor signal to the storage unit 32. The sensor signal acquisition unit 31 performs a recognition process on the acquired sensor signal to convert the sensor signal into information indicating a physical quantity. The sensor signal acquisition unit 31 outputs the physical quantity obtained by converting the sensor signal, i.e., the recognition result, to the storage unit 32. The sensor signal acquisition unit 31 generates time information indicating the time when the sensor signal was acquired. The sensor signal acquisition unit 31 outputs the time information to the storage unit 32 together with the sensor signal and the recognition result. The storage unit 32 stores the sensor signal, the recognition result, and the time information output from the sensor signal acquisition unit 31.

[0083] The information sharing unit 33 exchanges information shared between the controller 1B and the controller 5B. The information sharing unit 33 may be, for example, a bus, a shared memory, Time-Sensitive Networking (TSN) using Ethernet (registered trademark), or ordinary Ethernet. In the control system 10B, information is exchanged via the information sharing unit 33 at predetermined timing. The memory unit 12 of the controller 1B stores the sensor signal, the recognition result, and the time information, which are information read from the controller 5B via the information sharing unit 33. The memory unit 32 of the controller 5B stores the sensor signal, the recognition result, and the time information, which are information read from the controller 1B via the information sharing unit 33. The predetermined timing is the timing determined for each of the configurations of the information sharing unit 33 described above as examples.

[0084] By sharing information, each memory unit 12, 32 stores the first sensor signal, the second sensor signal, the recognition results obtained from each of the first sensor signal and the second sensor signal, and time information indicating the time when each of the first sensor signal and the second sensor signal was acquired.

[0085] The state estimation unit 13 estimates the state of the object based on two or more specified sensor signals including the first sensor signal and the second sensor signal. The state estimation unit 13 outputs the estimation result of the state of the object to each of the memory unit 12 and the control and calculation unit 14. The control and calculation unit 14 calculates a command value to be sent to the mechanical device based on the estimated state of the object. The signal output unit 15 outputs a control signal indicating the command value calculated by the control and calculation unit 14 to the robot 2, which is the mechanical device.

[0086] In the control system 10B, the reliability of the information shared via the information sharing unit 33, i.e., the information stored in each of the storage units 12 and 32, may vary. For example, the TSN may use a mechanism that automatically synchronizes information acquired by the TSN. However, for information with a large amount of data transmission, such as image signals, a network other than the TSN may be used. In addition, communication on the network may be performed using TCP / IP (Transmission Control Protocol / Internet Protocol) or UDP (User Datagram Protocol).

[0087] Under such conditions, the reliability of the update frequency of each of the multiple networks or the presence or absence of information updates decreases, and fluctuations may occur in the sensor signal even when the target is moving at a constant speed. In the second embodiment, a weighting parameter is set to deal with such fluctuations. This weighting parameter is designated as Pm2.

[0088] The controller 1B is provided with a detection means for detecting information loss or information delay in each configuration used in the information sharing unit 33. The detection means is not shown. The detection means detects a malfunction such as a sensor signal loss or delay when storing the sensor signals acquired by each controller 1B, 5B in the storage unit 12. When such a malfunction is detected, the state estimation unit 13 applies the weighting parameter Pm2 to perform state estimation processing using information other than the sensor signal in which the loss or delay was detected or past state estimation results.

[0089] In the first embodiment, the state estimation unit 13 applies a weighting parameter Pm1 to perform a state estimation process that prioritizes physically reliable information. In the second embodiment, by applying Pm1 and also by incorporating information selection by applying Pm2, it is possible to perform a state estimation with higher accuracy. The controller 1B outputs a command value corrected based on the highly accurate state estimation result, thereby preventing the robot 2 from performing an oscillating operation or the like and enabling the robot 2 to quickly complete a task. This allows the application of the robot 2 to improve productivity.

[0090] According to the second embodiment, the control system 10B includes a controller 1B serving as a first control device, a controller 5B serving as a second control device, and an information sharing unit 33 through which information shared between the controllers 1B and 5B is exchanged. The storage unit 12 stores a first sensor signal, a second sensor signal, and time information indicating the time at which each of the first and second sensor signals was acquired. The state estimation unit 13 estimates the state of an object based on two or more specified sensor signals including the first and second sensor signals. The controller 1B estimates the state of the object based on the sensor signals acquired by the controllers 1B and 5B for the same object. The controller 1B can perform state estimation by taking into account loss or delay in information output when information is shared and the physical reliability of the output from each sensor. This enables the controller 1B to estimate the state of the object with high accuracy.

[0091] Embodiment 3. Fig. 8 is a diagram showing an example of the configuration of a control system 10C according to embodiment 3. The control system 10C includes a controller 1C which is a first control device, a controller 5C which is a second control device, and an information sharing unit 33. In embodiment 3, the same components as those in embodiment 1 or 2 are denoted by the same reference numerals, and the following description will mainly focus on the configuration that differs from embodiment 1 or 2.

[0092] The controller 1C has a configuration similar to that of the controller 1A shown in Fig. 1, and includes a mode transition instruction unit 16. The controller 1C acquires first sensor signals that are sensor signals output from each of a plurality of sensors connected to the controller 1C.

[0093] The controller 5C is a controller connected to the controller 1C. The controller 5C has a configuration similar to that of the controller 5B shown in FIG. 7. The controller 5C acquires a second sensor signal, which is a sensor signal output from a sensor connected to the controller 5C. The information sharing unit 33 exchanges information shared between the controller 1C and the controller 5C.

[0094] The state estimation unit 13 outputs the state estimation result to each of the storage unit 12, the control calculation unit 14, and the mode transition instruction unit 16. The mode transition instruction unit 16 instructs a transition of the control mode in the mechanical device that can be driven by each of a plurality of control modes. The mode transition instruction unit 16 instructs a transition of the control mode based on the estimation result of the state of the object by the state estimation unit 13.

[0095] Specifically, the mode transition instruction unit 16 determines whether to transition the control mode based on a change in the state indicated in the state estimation result, i.e., a change in position, velocity, acceleration, or the like. For example, a threshold for the amount of change in the state between two steps is set in advance, and when the amount of change between the state in one of the two steps and the state in the other exceeds the threshold, the mode transition instruction unit 16 determines to transition the control mode. Based on this determination, the mode transition instruction unit 16 outputs an instruction for transitioning the control mode to the control calculation unit 14. The control calculation unit 14 transitions the control mode in controlling the robot 2 in accordance with the instruction from the mode transition instruction unit 16.

[0096] For example, the mode transition instruction unit 16 determines, for each sensor signal input to the state estimation unit 13, S sc The sensor signals S at the (K) and (K-1) steps scIf the amount of change, which is the difference from (K-1), exceeds a threshold value, the mode transition instruction unit 16 determines the transition of the control mode. se _(K) and the state estimation result of the (K-1)th step R se If the amount of change, which is the difference from (K-1), exceeds a threshold, the transition of the control mode is determined.

[0097] The plurality of control modes include, for example, a normal mode in which the robot 2 is operated at a normal speed, and a low-speed mode in which the robot 2 is operated at a speed slower than the normal speed. In the normal mode, the control calculation unit 14 controls the robot 2 to operate at a predetermined target command value P cmd In the low speed mode, the control calculation unit 14 operates the robot 2 at a rate of change indicated by P cmd The robot 2 is operated at a rate of change smaller than the rate of change shown in .

[0098] The plurality of control modes may include, for example, a stop mode in which the robot 2 is stopped, or an escape mode in which the robot 2 is caused to perform an escape operation. In the stop mode, the control calculation unit 14, for example, sets the target command value to P cmd P, which is the target command value from (K) to (K-1) steps cmd In the evacuation mode, the control calculation unit 14 does not change the target command value sequence P cmd _ hm By generating a new positional relationship, the robot 2 is caused to move the end effector 22 to the determined retreat point.

[0099] Within the normal mode, several detailed operation modes may be defined in accordance with the work processes of the robot 2. For example, the normal mode includes operation modes such as an assembly operation mode, a kitting operation mode, and a transport mode. The assembly operation mode is an operation mode used when assembling a processed product using the robot 2. The kitting operation mode is an operation mode used when arranging parts for the next process. The transport mode is an operation mode used when performing transport work including unloading. In this way, the normal mode can be divided into multiple operation modes in accordance with the operations of each of the multiple work processes. The mode transition instruction unit 16 determines the control mode transition among these multiple control modes and instructs the control mode transition.

[0100] For example, the controller 1C may suspend the work of the robot 2 by transitioning the control mode from the assembly operation mode to the evacuation mode based on a sensor signal obtained from the controller 5C via the information sharing unit 33. The controller 1C may then further transition the control mode to the kitting operation mode based on a state estimation result based on the sensor signal obtained from the controller 5C and a state estimation result based on a sensor signal acquired by the controller 1C. In this way, the controller 1C may transition the control mode based on information shared between the controllers 1C and 5C.

[0101] According to the third embodiment, the controller 1C includes a mode transition instructing unit 16 that instructs a transition of the control mode of the mechanical device. The mode transition instructing unit 16 instructs a transition of the control mode based on the result of estimation of the state of the object by the state estimation unit 13. By instructing a transition of the control mode based on the result of state estimation, the controller 1C can transition the control mode of the mechanical device in accordance with the estimated state. For example, when an unexpected event occurs and it becomes difficult to control the mechanical device to the target state, the controller 1C can flexibly respond to the situation and control the mechanical device, such as by suspending the operation of the mechanical device. This makes it possible to avoid a situation in which the mechanical device is unable to terminate its operation, resulting in damage to the mechanical device or the object. As a result, the controller 1C can reduce the time that the operation of the mechanical device is suspended, thereby improving productivity through the application of the mechanical device.

[0102] Fourth Embodiment Fig. 9 is a diagram showing an example of the configuration of a control system 10D according to a fourth embodiment. The control system 10D includes a controller unit 41 and a simulator 42 that simulates the operation of the control devices. The controller unit 41 includes a controller 1D that is a first control device, a controller 5D that is a second control device, and an information sharing unit 33. In the fourth embodiment, the same components as those in the first to third embodiments are denoted by the same reference numerals, and the following description will mainly focus on the configuration that differs from the first to third embodiments.

[0103] The controller 1D has the same configuration as the controller 1A shown in Fig. 1. The controller 1D acquires sensor signals output from each of a plurality of sensors connected to the controller 1D. The controller 5D is a controller connected to the controller 1D. The controller 5D has the same configuration as the controller 5B shown in Fig. 7. The controller 5D acquires sensor signals output from the sensors connected to the controller 5D. The information sharing unit 33 exchanges information shared between the controller 1D and the controller 5D.

[0104] The simulator 42 includes a simulator 43, a comparator 44, and an analyzer 45. The simulator 43 generates a simulated signal that simulates a sensor signal to be acquired in the future based on a state estimation result and a command value calculated by inputting a sensor signal to a simulation model that simulates the controller unit 41. The comparator 44 reads a sensor signal at a step subsequent to the sensor signal input to the simulation model, and compares the read sensor signal with the simulated signal. The analyzer 45 updates the simulation model by analyzing the comparison result by the comparator 44.

[0105] Specifically, the simulator 43 inputs two or more sensor signals from the controller unit 41 into the simulation model and causes the simulation model to perform calculations to determine a state estimation result and a command value. The simulator 43 simulates the sensor signals output by each of the two or more sensors in the next step, assuming that the robot 2 is controlled based on the calculated state estimation result and command value. In this way, the simulator 43 generates simulation signals that simulate the two or more sensor signals in the next step.

[0106] The comparison unit 44 reads two or more sensor signals in the next step from the controller unit 41. The comparison unit 44 compares each of the two or more read sensor signals with the simulation signal. The comparison unit 44 outputs the result of comparing each of the two or more sensor signals with the simulation signal to the analysis unit 45.

[0107] The simulating unit 43 instructs the control calculation unit 14 of the controller 1D to control the operation of the robot 2. The comparing unit 44 reads two or more sensor signals acquired in the controller unit 41 when the control calculation unit 14 operates the robot 2 in accordance with the instruction.

[0108] The analysis unit 45 analyzes the sensor signal that has a large difference from the simulated signal among the two or more sensor signals from the comparison results input to the analysis unit 45. The analysis unit 45 extracts model parameters from the simulation model that have a high correlation with the sensor signal that has a large difference from the simulated signal. In other words, the analysis unit 45 extracts model parameters that are considered to be factors that cause the difference between the simulated signal and the sensor signal.

[0109] The analysis unit 45 outputs information indicating the extracted model parameters to the simulation unit 43. The simulation unit 43 updates the simulation model by changing the model parameters indicated in the information.

[0110] The simulator 43 calculates a target command value using the updated simulation model. The simulator 43 outputs the calculated target command value to the control calculation unit 14. As a result, the control calculation unit 14 uses the target command value calculated in advance by the simulator 43 at the time of the next control. By applying the target command value calculated using the updated simulation model, the controller 5D can reduce vibrations of the robot 2 compared to when the target command value used in the previous control is applied as is. The controller 5D can achieve high-precision control of the robot 2.

[0111] By including the simulator 42, the control system 10D can generate command values ​​that take into account modeling errors of the controller 5D when generating target command values ​​in advance. As a result, the control system 10D can cause the robot 2 to perform highly responsive operations, thereby improving productivity through application of the robot 2.

[0112] According to the fourth embodiment, the control system 10D includes a simulator 42 that simulates the operation of the control device. The simulator 42 includes a simulator 43 that generates a simulated signal that simulates a sensor signal to be acquired in the future based on a state estimation result and a command value calculated by inputting a sensor signal to a simulation model that simulates the control device, a comparator 44 that reads a sensor signal at a step subsequent to the sensor signal input to the simulation model and compares the read sensor signal with the simulated signal, and an analyzer 45 that updates the simulation model by analyzing the comparison result by the comparator 44. This allows the control system 10D to cause the machine to perform highly responsive operations, thereby improving productivity through the application of the machine.

[0113] Next, hardware for realizing the controllers 1A, 1B, 1C, and 1D according to the first to fourth embodiments will be described. The controllers 1A, 1B, 1C, and 1D are realized by using a processing circuit. The processing circuit may be a circuit in which a processor executes software, or may be a dedicated circuit.

[0114] When the processing circuit is realized by software, the processing circuit is, for example, a control circuit shown in FIG. 10 . FIG. 10 is a diagram showing an example configuration of a control circuit 50 according to embodiments 1 to 4. The control circuit 50 includes an input unit 51, a processor 52, a memory 53, and an output unit 54. The input unit 51 is an interface circuit that receives data from outside the control circuit 50 and provides the data to the processor 52. The output unit 54 is an interface circuit that sends data from the processor 52 or the memory 53 to outside the control circuit 50. The function of the signal output unit 15 is realized by using the output unit 54.

[0115] 10 , the processing functions of the sensor signal acquisition unit 11, the state estimation unit 13, the control calculation unit 14, and the mode transition instruction unit 16 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 53.

[0116] The processing circuit realizes the processing functions of the controllers 1A, 1B, 1C, and 1D by having the processor 52 read and execute programs stored in the memory 53. That is, the processing circuit includes the memory 53 for storing programs that result in the processing of the controllers 1A, 1B, 1C, and 1D. The programs stored in the memory 53 can also be said to be programs that cause a computer to execute the processing procedures and methods of the controllers 1A, 1B, 1C, and 1D. The memory 53 is also used as temporary memory when the processor 52 executes various processes.

[0117] The processor 52 is a CPU (Central Processing Unit). The processor 52 may be a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor). The memory 53 may be, for example, a non-volatile or volatile semiconductor memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), a flash memory, an EPROM (Erasable Programmable Read Only Memory), or an EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc). The functions of the storage unit 12 are realized by using the memory 53.

[0118] When the processing circuit is a dedicated circuit, the controllers 1A, 1B, 1C, and 1D are realized by, for example, the hardware circuit shown in Fig. 11. Fig. 11 is a diagram showing an example of the configuration of a hardware circuit 55 according to the first to fourth embodiments.

[0119] The processing functions of the controllers 1A, 1B, 1C, and 1D are realized by a processing circuit 56, which is a dedicated circuit. The processing circuit 56 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. The processing functions of the controllers 1A, 1B, 1C, and 1D may be realized by the processing circuit 56 on a function-by-function basis, or all functions may be realized collectively by the processing circuit 56. The processing functions of the controllers 1A, 1B, 1C, and 1D may be realized by a combination of the control circuit 50 shown in FIG. 10 and the processing circuit 56 shown in FIG. 11.

[0120] The programs according to the first to fourth embodiments may be provided by being stored on a recording medium such as a CD (Compact Disc)-ROM or a DVD-ROM. The programs according to the first to fourth embodiments may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network such as the Internet. The programs according to the first to fourth embodiments may be provided or distributed via a network such as the Internet.

[0121] Each of the controllers 5B, 5C, and 5D according to embodiments 2 to 4 and the simulator 42 according to embodiment 4 has a hardware configuration similar to the hardware configuration shown in FIG. 10 or a hardware configuration similar to the hardware configuration shown in FIG. 11.

[0122] The configurations shown in the above embodiments are examples of the contents of the present disclosure. The configurations of each embodiment can be combined with other known technologies. The configurations of each embodiment can also be combined as appropriate. Part of the configuration of each embodiment can be omitted or modified without departing from the gist of the present disclosure.

[0123] 1A, 1B, 1C, 1D, 5, 51, 52, 53, 54, 5B, 5C, 5D Controller, 2, 8 Robot, 31, 32, 6 Camera, 4 Force sensor, 7 Tactile sensor, 9 Processing machine, 10A, 10B, 10C, 10D Control system, 11, 31 Sensor signal acquisition unit, 12, 32 Memory unit, 13 State estimation unit, 14 Control calculation unit, 15 Signal output unit, 16 Mode transition instruction unit, 21 Robot arm, 22 End effector, 23 Object, 33 Information sharing unit, 41 Controller unit, 42 Simulator, 43 Simulator unit, 44 Comparison unit, 45 Analysis unit, 50 Control circuit, 51 Input unit, 52 Processor, 53 Memory, 54 Output unit, 55 Hardware circuit, 56 Processing circuit.

Claims

1. A control device for controlling a mechanical device that moves an object, comprising: a plurality of sensor signal acquisition units each acquiring a sensor signal output from a sensor provided on the mechanical device or in the vicinity of the mechanical device; a memory unit storing the plurality of sensor signals acquired by the plurality of sensor signal acquisition units and time information indicating the time at which each of the sensor signals was acquired; a state estimation unit estimating the state of the object based on two or more specified sensor signals from the stored plurality of sensor signals and the time information for each of the two or more specified sensor signals; and a control calculation unit calculating a command value to be sent to the mechanical device based on the estimated state of the object.

2. The control device according to claim 1, characterized in that the state estimation unit weights each of the two or more specified sensor signals and estimates the state of the object based on the two or more weighted sensor signals.

3. The control device described in claim 1 or 2, characterized in that the memory unit stores a first sensor signal, which is the sensor signal acquired by the sensor signal acquisition unit, and a second sensor signal output from a sensor that is a control device other than the control device and is installed on the mechanical device or in the vicinity of the mechanical device, and the state estimation unit estimates the state of the object based on two or more specified sensor signals including the first sensor signal and the second sensor signal.

4. A control device as described in any one of claims 1 to 3, characterized in that it comprises a mode transition instruction unit that instructs the transition of the control mode in the mechanical device that can be driven by each of a plurality of control modes, and the mode transition instruction unit instructs the transition of the control mode based on the result of estimation of the state of the object by the state estimation unit.

5. A control system comprising: a control device for controlling a mechanical device that moves an object, the control device comprising: a sensor signal acquisition unit that acquires sensor signals output from a plurality of sensors provided on the mechanical device or in the vicinity of the mechanical device; a memory unit that stores the acquired plurality of sensor signals and time information indicating the time at which each of the sensor signals was acquired; a state estimation unit that estimates the state of the object based on two or more specified sensor signals from the stored plurality of sensor signals and the time information for each of the two or more specified sensor signals; and a control calculation unit that calculates a command value to be sent to the mechanical device based on the estimated state of the object.

6. The control system according to claim 5, comprising: a first control device that is the control device that acquires a first sensor signal that is the sensor signal; a second control device that controls a sensor provided on the mechanical device or in the periphery of the mechanical device and acquires a second sensor signal output from the mechanical device or the sensor provided in the periphery of the mechanical device; and an information sharing unit that exchanges information shared between the first control device and the second control device, wherein the memory unit stores the first sensor signal, the second sensor signal, and the time information indicating the time when each of the first sensor signal and the second sensor signal was acquired, and the state estimation unit estimates the state of the object based on two or more specified sensor signals including the first sensor signal and the second sensor signal.

7. A control system according to claim 5, further comprising a simulator that simulates the operation of the control device, the simulator comprising: a simulator section that generates a simulated signal that simulates the sensor signal that will be acquired in the future based on the command value and a state estimation result calculated by inputting the sensor signal into a simulation model that simulates the control device; a comparison section that reads in the sensor signal at a step later than the sensor signal input into the simulation model, and compares the read sensor signal with the simulated signal; and an analysis section that updates the simulation model by analyzing the comparison result by the comparison section.

8. A program causing a computer to execute the steps of: acquiring sensor signals output from a mechanical device that moves an object or from a plurality of sensors provided in the vicinity of said mechanical device; storing the acquired plurality of sensor signals and time information indicating the time at which each of said sensor signals was acquired; estimating the state of said object based on two or more specified sensor signals from the stored plurality of sensor signals and the time information for each of said two or more specified sensor signals; and calculating a command value to be sent to said mechanical device based on the estimated state of said object.

Images