Control device, control system, and program

Abstract

The control device includes a plurality of sensor signal acquisition units (11) each acquiring a sensor signal output from a sensor provided on the mechanical device or in the vicinity of the mechanical device, a memory unit (12) storing the plurality of sensor signals acquired by the plurality of sensor signal acquisition units (11) and time information indicating the time at which each sensor signal was acquired, a state estimation unit (13) estimating the state of an object based on two or more designated sensor signals from the stored plurality of sensor signals and the time information of each of the designated two or more sensor signals, and a control calculation unit (14) calculating a command value to be sent to the mechanical device based on the estimated state of the object.

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a control system, and a program for controlling a mechanical device. [Background technology]

[0002] When moving an object by driving a mechanical device, the control device may detect the state of the same object using multiple sensors and control the mechanical device based on information indicating the detection results from each sensor. In this case, the control device integrates the detection results from each sensor and controls the mechanical device based on the integrated information.

[0003] Patent Document 1 discloses a control system that enables high-precision position control by reducing discrepancies in the position information of multiple devices that update their position information at different cycles when controlled by a controller. The control system disclosed in Patent Document 1 includes a controller and two or more processing devices connected to the controller and performing predetermined processing related to a workpiece. In the control system disclosed in Patent Document 1, each processing device includes a timer and a position information update unit that updates its position information. In the control system disclosed in Patent Document 1, the controller includes a position conversion unit that converts the position information output from each processing device into world coordinate system position information indicating a position on the world coordinate system, an estimation unit that estimates the position of the workpiece on the world coordinate system during the control cycle of the controller, and a world coordinate system management unit that manages the world coordinate system position information and position information indicating the estimated position of the workpiece on a time axis. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-212123 Summary of the Invention [Problem to be solved by the invention]

[0005] According to Patent Document 1, it is possible to reduce errors caused by a time difference between position information indicating the position of a workpiece recognized by a controller and position information output from each processing device. However, when detecting the state of the same object using multiple sensors and integrating the detection results from each sensor, it is necessary to consider physical errors in the multiple sensors, such as coordinate transformation errors or modeling errors. The configuration disclosed in Patent Document 1 does not include a means for reducing such physical errors. Therefore, the conventional technology disclosed in Patent Document 1 has a problem in that it is difficult to accurately integrate information obtained by multiple sensors about the same object.

[0006] The present disclosure has been made in consideration of the above, and aims to provide a control device that can accurately integrate information acquired by each of multiple sensors about the same object. [Means for solving the problem]

[0007] In order to solve the above-mentioned problems and achieve the object, a control device according to the present disclosure is a control device for controlling a mechanical device that moves an object. The control device according to the present disclosure includes a plurality of sensor signal acquisition units each acquiring a sensor signal output from a sensor provided on the mechanical device or in the vicinity of the mechanical device, a storage unit storing the plurality of sensor signals acquired by the plurality of sensor signal acquisition units and time information indicating the time when each sensor signal was acquired, and a storage unit storing two or more designated sensor signals from the plurality of stored sensor signals. No. and The state estimation unit estimates the state of the object by integrating the two or more designated sensor signals based on time information of each of the two or more designated sensor signals, and a control calculation unit calculates a command value to be sent to the mechanical device based on the estimated state of the object. Accuracy of detecting objects in Differences Based on , Of the two or more specified sensor signals, the sensor signal that detects the object has higher accuracy than the other sensor signals.Two or more designated sensor signals are integrated, with at least one sensor signal being prioritized over the other sensor signals. [Effects of the Invention]

[0008] The control device according to the present disclosure has an effect of being able to accurately integrate information acquired by each of a plurality of sensors regarding the same object. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram showing a configuration example of a control device according to a first embodiment; [Figure 2] FIG. 1 is a diagram showing a configuration example of a control system having a control device according to a first embodiment; [Figure 3] 1 is a flowchart showing an example of an operation procedure of the control device according to the first embodiment. [Figure 4] FIG. 1 is a diagram for explaining an example in which a state of an object is estimated by a state estimation unit included in the control device according to the first embodiment. [Figure 5] FIG. 1 is a diagram for explaining inputs and outputs of a plurality of sensor signal acquisition units, a storage unit, and a state estimation unit included in the control device according to the first embodiment. [Figure 6] FIG. 1 is a diagram for explaining calculation of a command value by a control calculation unit included in the control device according to the first embodiment. [Figure 7] FIG. 10 is a diagram showing a configuration example of a control system according to a second embodiment. [Figure 8] FIG. 10 is a diagram showing a configuration example of a control system according to a third embodiment. [Figure 9] FIG. 10 is a diagram showing a configuration example of a control system according to a fourth embodiment. [Figure 10] FIG. 1 shows an example of the configuration of a control circuit according to first to fourth embodiments. [Figure 11] FIG. 1 shows an example of the configuration of a hardware circuit according to first to fourth embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0010] A control device, a control system, and a program according to an embodiment will be described in detail below with reference to the accompanying drawings.

[0011] Embodiment 1 Fig. 1 is a diagram showing an example of the configuration of a control device according to embodiment 1. Fig. 1 shows a controller 1A which is the control device according to embodiment 1.

[0012] The controller 1A controls a mechanical device that moves an object. In the first embodiment, the mechanical device controlled by the controller 1A is a robot that grips and moves an object. The robot is installed near a processing machine that processes the object. The robot moves the object to a position in the processing machine where the object is to be placed. The processing machine processes the object placed in that position.

[0013] 2 is a diagram showing an example of the configuration of a control system 10A having a control device according to the first embodiment. The control system 10A includes a controller 1A, a robot 2 connected to the controller 1A, and a sensor connected to the controller 1A. The robot 2 is a robot controlled by the controller 1A. The robot 2 is assumed to be a robot having a robot arm.

[0014] Each of the camera 31, camera 32, and force sensor 4 is a sensor connected to the controller 1A. Each of the camera 31, camera 32, and force sensor 4 detects the state of an object. The cameras 31 and 32 are provided in the periphery of the robot 2. Each of the cameras 31 and 32 is a vision sensor that detects the position of the object by photographing the object. Each of the cameras 31 and 32 outputs an image signal to the controller 1A. The force sensor 4 is provided on a robot arm of the robot 2. The force sensor 4 detects a force acting on the object by detecting a force acting on the robot arm of the robot 2. The force sensor 4 outputs a signal indicating the result of the force detection to the controller 1A.

[0015] The control system 10A includes a plurality of controllers 51, 52, 53, and 54 connected to the controller 1A. Hereinafter, the controller 5 will be referred to without distinguishing between the controllers 51, 52, 53, and 54.

[0016] A camera 6 is connected to the controller 51. A tactile sensor 7 is connected to the controller 52. The camera 6 and the tactile sensor 7 are each a sensor that detects the state of an object. The camera 6 is provided in the periphery of the robot 2. The camera 6 is a vision sensor that detects the position of an object by photographing the object. The camera 6 outputs an image signal to the controller 51. The controller 51 sends the image signal acquired from the camera 6 to the controller 1A. The tactile sensor 7 is provided in the hand of the robot 2. The tactile sensor 7 detects the state in which the object is being grasped by the hand. The tactile sensor 7 outputs a signal indicating the result of detecting the state in which the object is being grasped to the controller 52. The controller 52 sends the signal acquired from the tactile sensor 7 to the controller 1A.

[0017] The controller 53 is connected to the robot 8. The robot 8 is a robot that operates in the vicinity of the robot 2. The controller 53 controls the robot 8. The controller 53 sends information indicating the operating state of the robot 8 to the controller 1A.

[0018] The controller 54 is connected to the processing machine 9. The processing machine 9 processes an object carried to a placement position by the robot 2. The controller 54 controls the processing machine 9. The controller 54 sends information indicating the operating state of the processing machine 9 to the controller 1A.

[0019] The controller 1A acquires signals output from the cameras 31, 32, and force sensor 4 connected to the controller 1A. The controller 1A acquires signals output from the cameras 6 and tactile sensors 7 by receiving signals sent from the controllers 51 and 52. The controller 1A acquires information indicating the operating state of the robot 8 by receiving information sent from the controller 53. The controller 1A acquires information indicating the operating state of the processing machine 9 by receiving information sent from the controller 54.

[0020] Based on the sensor signals output from the sensors, the controller 1A corrects the command values ​​for controlling the robot 2. The controller 1A adjusts the driving of the robot 2 based on the state of the object recognized from the sensor signals.

[0021] The controller 1A corrects the command value for controlling the robot 2 based on information indicating the state of peripheral devices such as the robot 8 or the processing machine 9. The controller 1A adjusts the driving of the robot 2 in accordance with the state of the peripheral devices by correcting the command value based on the information indicating the state of the peripheral devices.

[0022] 2, the camera 31, the camera 32, and the force sensor 4 are connected to the controller 1A. Sensors other than the cameras 31 and 32 and the force sensor 4 may be connected to the controller 1A. The number of sensors connected to the controller 1A is arbitrary.

[0023] In FIG. 2, the camera 6 and the tactile sensor 7 are each connected to the controller 1A via the controller 5. Sensors other than the camera 6 and the tactile sensor 7 may be connected to the controller 1A via the controller 5. Furthermore, the number of sensors connected to the controller 1A via the controller 5 is arbitrary.

[0024] In FIG. 2, a robot 8 and a processing machine 9 are connected to the controller 1A via the controller 5. Peripheral devices other than the robot 8 and the processing machine 9 may be connected to the controller 1A via the controller 5. The number of peripheral devices connected to the controller 1A via the controller 5 is arbitrary. Furthermore, there may be no peripheral devices connected to the controller 1A via the controller 5. In the control system 10A, it is sufficient that there are multiple sensors connected to the controller 1A directly or via the controller 5.

[0025] Next, a configuration example of the controller 1A shown in Fig. 1 will be described. The controller 1A includes a plurality of sensor signal acquisition units 11, a storage unit 12, a state estimation unit 13, a control calculation unit 14, and a signal output unit 15.

[0026] Each of the multiple sensor signal acquirers 11 acquires a sensor signal output from a sensor provided in or around the mechanical device. In the example shown in FIG. 1, sensor signals output from five sensors are input to the controller 1A directly or via the controller 5. The controller 1A acquires five sensor signals by each of the five sensor signal acquirers 11 acquiring a sensor signal. Two of the five sensor signal acquirers 11 are shown in FIG. 1.

[0027] The multiple sensor signals acquired by the multiple sensor signal acquisition unit 11 may be either analog signals or digital signals. The multiple sensor signals may include both analog and digital signals. The sensor signals output from the cameras 31, 32, and 6 are image signals. The sensor signal output from the force sensor 4 is a force signal indicating the magnitude of a force or a moment signal indicating a moment. The sensor signal output from the tactile sensor 7 is a signal indicating the distribution of frictional force on the surface contacting the object, a signal indicating the amount of slippage, or a signal indicating the presence or absence of contact. The sensor signal may be a signal other than these signals. For example, the sensor signal may be an encoder signal output by an encoder in a motor that drives a robot arm of the robot 8, or a signal indicating the current value of the current flowing through the motor. Each sensor signal acquisition unit 11 outputs the acquired sensor signal to the memory unit 12.

[0028] Each sensor signal acquisition unit 11 converts the acquired sensor signal into information indicating a physical quantity by executing a recognition process on the acquired sensor signal. When the sensor signal acquisition unit 11 acquires an image signal, it extracts a feature quantity through the recognition process. In this case, the sensor signal acquisition unit 11 outputs, as a recognition result, coordinates indicating the position where the feature quantity is extracted in the image, or information indicating the area of ​​the region where the feature quantity is extracted. The sensor signal acquisition unit 11 calculates a target physical quantity according to the content of the recognition process implemented in the sensor signal acquisition unit 11. Each sensor signal acquisition unit 11 outputs the calculated physical quantity, i.e., the recognition result, to the storage unit 12.

[0029] Each sensor signal acquisition unit 11 generates time information indicating the time at which the sensor signal was acquired. Each sensor signal acquisition unit 11 outputs the generated time information to the storage unit 12. For example, each sensor signal acquisition unit 11 is provided with a timer, and each sensor signal acquisition unit 11 generates the time information by recording the time at which the sensor signal was acquired based on the output of the timer. Each sensor signal acquisition unit 11 outputs the time information to the storage unit 12 together with the sensor signal and the recognition result.

[0030] The memory unit 12 stores multiple sensor signals acquired by multiple sensor signal acquisition units 11, recognition results output from each sensor signal acquisition unit 11, and time information indicating the time when the sensor signal was acquired by each sensor signal acquisition unit 11.

[0031] The state estimation unit 13 estimates the state of the object at the time when the mechanical device is controlled, based on two or more designated sensor signals from the multiple sensor signals stored in the memory unit 12 and time information for each of the designated two or more sensor signals. The state of the object estimated by the state estimation unit 13 includes, for example, the position of the object, the velocity or acceleration of the object's movement, the force acting on the object from the environment, or the contact state between the object and the environment. Here, the environment refers to structures present around the object. Examples of the environment include the walls or floor of a room in which the mechanical device is installed, or partitions placed in the room in which the mechanical device is installed. The state estimation unit 13 also weights each of the designated two or more sensor signals and estimates the state of the object based on the weighted two or more sensor signals.

[0032] The state estimation unit 13 may estimate the state of the mechanical device and regard the estimated state of the mechanical device as the estimated state of the object. In this case, the state of the mechanical device estimated by the state estimation unit 13 may be, for example, the position of a part of the mechanical device that moves the object, the speed or acceleration of the movement of the part, the acting force that the part receives from the environment, or the contact state between the part and the environment.

[0033] The state estimation unit 13 outputs the estimation result of the state of the object to each of the storage unit 12 and the control calculation unit 14. The storage unit 12 stores the estimation result of the state of the object.

[0034] The control calculation unit 14 calculates a command value to be sent to the mechanical device based on the estimated state of the object. The signal output unit 15 outputs a control signal indicating the command value calculated by the control calculation unit 14 to the robot 2, which is the mechanical device. The signal output unit 15 also outputs a trigger signal to the robot 2, which is the mechanical device.

[0035] The controller 1A acquires information indicating the status of peripheral devices such as the robot 8 or the processing machine 9. The control calculation unit 14 adjusts the driving of the robot 2 based on the information indicating the status of the peripheral devices. In FIG. 1, elements that acquire information indicating the status of the peripheral devices are not shown.

[0036] Next, the operation procedure of the controller 1A will be described below. Fig. 3 is a flowchart showing an example of the operation procedure of the control device according to the first embodiment.

[0037] In step S1, each of the plurality of sensor signal acquirers 11 acquires a sensor signal. In step S2, each of the plurality of sensor signal acquirers 11 executes a recognition process for the sensor signal acquired in step S1. Each of the plurality of sensor signal acquirers 11 outputs the sensor signal acquired in step S1, a recognition result obtained by the recognition process, and time information to the storage unit 12. In step S3, the storage unit 12 stores the sensor signal, the recognition result, and the time information.

[0038] The state estimation unit 13 reads the sensor signal, the recognition result, and the time information stored in the storage unit 12. In step S4, the state estimation unit 13 estimates the state of the object based on the sensor signal, the recognition result, and the time information. The state estimation unit 13 outputs the estimation result of the state of the object to each of the storage unit 12 and the control calculation unit 14. The storage unit 12 stores the estimation result of the state of the object.

[0039] In step S5, the control calculation unit 14 calculates a command value based on the result of estimating the state of the object. The control calculation unit 14 outputs the calculated command value to the signal output unit 15. In step S6, the signal output unit 15 outputs the command value calculated in step S5 to the robot 2. With the above, the controller 1A ends the operation according to the procedure shown in FIG.

[0040] Next, a description will be given of an example of a method for estimating the state of an object by the state estimation unit 13. Here, a case where the state estimation unit 13 estimates the position of an object to be moved by the robot 2 will be mainly described.

[0041] 4 is a diagram for explaining an example in which the state of an object is estimated by the state estimation unit 13 included in the control device according to embodiment 1. Here, it is assumed that the sensor signals acquired by the controller 1A are sensor signals output by the cameras 31 and 32 and the force sensor 4. FIG. 4 shows the robot 2 and the cameras 31 and 32 and the force sensor 4, which are sensors.

[0042] The robot 2 has a robot arm 21 and an end effector 22 attached to the tip of the robot arm 21. Figure 4 shows a state in which an object 23 grasped by the end effector 22 is being moved by driving the robot arm 21.

[0043] 4, cameras 31 and 32 are installed around robot 2 at positions where they can capture images of object 23. Cameras 31 and 32 capture images of object 23 from different positions. Force sensor 4 is installed on robot 2 between robot arm 21 and end effector 22. Controller 1A acquires sensor signals output from cameras 31 and 32 and force sensor 4 using multiple sensor signal acquisition units 11.

[0044] The sensor signal acquisition unit 11 acquires the image signals, which are sensor signals output from the cameras 31 and 32, and performs recognition processing on the image signals to obtain position information indicating the position of the object 23 captured in the image. The sensor signal acquisition unit 11 acquires the image signals and generates time information indicating the time when the image signals were acquired. The sensor signal acquisition unit 11 outputs the image signals, the position information, which is the recognition result, and the time information.

[0045] It is assumed that the origin serving as the reference for the position indicated in the position information has been positioned in advance in the local coordinate system, which is the coordinate system of the robot 2. Specifically, the origin of the local coordinate system is adopted as the origin of the position information, and calibration is performed in advance between the robot 2 and each of the cameras 31 and 32. As a result, the sensor signal acquisition unit 11 outputs position information indicating the position in the local coordinate system.

[0046] In the first embodiment, the mechanical device is a robot 2. The mechanical device may be any device that performs the task of moving an object, and is not limited to a robot 2. The mechanical device may be a processing machine 9, a combination of multiple Cartesian robots, or a combination of an autonomous mobile robot (AMR) and a robot 2, etc.

[0047] The state estimation unit 13 reads the position information and time information acquired by the sensor signal acquisition unit 11 from the storage unit 12. The state estimation unit 13 estimates the position of the object at a certain time based on the position information and time information that are information read from the storage unit 12.

[0048] Each of the multiple sensors included in the control system 10A typically detects the state of an object at a cycle set for each sensor and outputs a sensor signal. For this reason, the controller 1A may manage the position information of the object at each time by time-synchronizing the multiple sensor signals stored in the storage unit 12. However, simply integrating the position information acquired by the sensor signals from two or more sensors by time synchronization alone may allow for physical errors in the two or more sensors.

[0049] Specifically, even if two cameras 31 and 32 detect the position of the same object, when attempting to combine position information acquired from the image signals from each camera 31 and 32, it may not be possible to uniquely determine the position of the object. This can be said to mean that even if each camera 31 and 32 detects the position of the same object, there are only two candidates for position information that can uniquely determine the position of the object. This phenomenon can occur due to differences in the detection capabilities of each camera 31 and 32 or physical errors such as modeling errors. In the first embodiment, when combining two or more pieces of information indicating the state of the object, the controller 1A verifies the degree of reliability of each sensor signal and prioritizes the integration of the more reliable information. In the first embodiment, the operation of verifying the degree of reliability of each sensor signal and integrating the information by prioritizing the more reliable information is referred to as a state estimation process.

[0050] In the state estimation process, first, a "certain time" is set when the position of the object at that time is estimated. The "certain time" is the time when the control of the robot 2 is executed based on the command value generated by the control calculation unit 14. The current time recognized by the control calculation unit 14 is defined as T_ cur Then, "a certain time" is T_ cur For example, the time lag T_ lag If there is a time, then "a certain time" is T_ cur T_ than lagIn other words, the control command time, which is the time when control is executed, is set as T_ cmd Then, T_ cmd =T_ cur +T_ lag can be treated as "a certain time". Hereinafter, "a certain time" means T_ cmd It is assumed that

[0051] Next, the state estimation unit 13 performs state estimation using the sensor signal and recognition result at the current time and the past state estimation result as input. Here, the number of times control has been executed since the control calculation unit 14 started the process for control is called a step. Hereinafter, T_ cmd (K) represents the control command time at the Kth step. 2 or more Let R be the integer above. se _(K) is T_ cmd (K) represents the state estimation result. cur (K) represents the current time at the Kth step.

[0052] The state estimation unit 13 calculates the past state estimation result R se _(K-1) and T_ cur The sensor signal and recognition result of (K) are input, and the state estimation result for the control command time at the Kth step is R se Find _(K). R se _(K-1) is the control command time T_ at the (K-1)th step. cmd The state estimation unit 13 detects the state of T_(K-1). cmd (K) the likely sensor signals and recognition results, and T_ cmd Based on the evaluation of the continuity of the state estimation results in (K-1), T_ cmd Estimate the likely state in (K).

[0053] In the state estimation process, the state estimation unit 13 weights the input information by utilizing the fact that the properties of the sensor signals differ depending on the characteristics of each sensor. Here, the properties of the sensor signals that differ depending on the characteristics of each sensor will be described.

[0054] One example of a characteristic of each sensor is the length of the sampling period. The multiple sensors provided in the control system 10A include sensors with short sampling periods and sensors with long sampling periods. One difference in the properties of sensor signals is the frequency at which the sensor signals are output. For example, cameras 31 and 32 output sensor signals at a low frequency of 10 Hz to 100 Hz. In contrast, force sensor 4 outputs sensor signals at a high frequency of around 1000 Hz. In other words, force sensor 4 can be said to be a sensor capable of high-speed sampling.

[0055] Furthermore, cameras 31 and 32 are susceptible to disturbances from light. For this reason, the reliability of the sensor signals output from cameras 31 and 32 as input information decreases under certain conditions. Photoelectric sensors that detect the presence or absence of an object based on the properties of the received light are also susceptible to disturbances from light, and therefore, under certain conditions, the reliability of the input information decreases. Furthermore, like force sensor 4, tactile sensor 7 can output sensor signals at high frequencies, and can be said to be a sensor capable of high-speed sampling.

[0056] From the sensor signal output from the force sensor 4, it is possible to recognize that the object has reached a position where it comes into contact with the environment at the timing when the object begins to receive an acting force from the environment. In other words, the sensor signal output from the force sensor 4 can be used as position information that indicates the position of the object.

[0057] Because the force sensor 4 is capable of high-speed sampling, it can detect with high accuracy the timing at which the force acting on the object changes. In other words, it can be said that the position information based on the sensor signals output from the force sensor 4 represents the position of the object with higher accuracy than the position information based on the sensor signals output from each of the cameras 31 and 32.

[0058] Due to such differences in the properties of the sensor signals output from the cameras 31 and 32 and the sensor signal output from the force sensor 4, the state estimation unit 13 executes a state estimation process that prioritizes the position information derived from the force sensor 4 over the position information derived from the cameras 31 and 32. This enables the controller 1A to estimate the state of the object with high accuracy.

[0059] Furthermore, the low resolution of each of the cameras 31 and 32 may cause errors in the position information derived from the cameras 31 and 32. In this case, the state estimation unit 13 executes a state estimation process that prioritizes the position information derived from the force sensor 4, thereby enabling the controller 1A to estimate the state of the object with high accuracy.

[0060] The state estimation unit 13 can execute a state estimation process in which the recognition result based on the sensor signal output from the force sensor 4 is prioritized over the recognition result based on the sensor signals output from the cameras 31 and 32, based on the following equation (1). Note that the state estimation result obtained by the state estimation unit 13 is expressed as R se , the recognition results based on the sensor signals output from the cameras 31 and 32 are R rc 1, the recognition result based on the sensor signal output from the force sensor 4 is R rc 2. A weighting parameter representing the weighting of the recognition result based on the sensor signals output from the cameras 31 and 32 and the recognition result based on the sensor signal output from the force sensor 4 is denoted by Pm1. nc is a function representing the state estimation process. R se =F nc (R rc 1,R rc 2,Pm1) ···(1)

[0061] The weighting parameter Pm1 is a function F that represents the state estimation process. nc This allows the user of the control system 10A to freely define weighting in the state estimation process.

[0062] When contact between the object and the environment is recognized based on the sensor signal output from the force sensor 4, the state information of the object may be updated on the assumption that the position of the object is constrained to the position at which it contacted the environment. For example, T cur In (K-1), it is assumed that contact between the object and the environment is recognized based on the sensor signal output from the force sensor 4. cur If it continues in (K), T_ cur Assuming that the position of the object does not change from (K-1), T_ cur State information for (K) is calculated. In this way, the state information of the object is updated. Thereafter, a conditional weighting may be defined indicating that no displacement occurs in a specific traveling direction as long as the force sensor 4 detects that a force is acting on the object. This allows the state estimation unit 13 to perform state estimation processing in which the position information derived from the force sensor 4 is prioritized over the position information derived from each of the cameras 31 and 32.

[0063] When an object is determined to be in contact with the environment, regarding the object as being in a position where it is in contact with the environment can be said to be a constraint on the positional relationship between the object and the environment. In the above, the priority of the recognition result when such a constraint can be assumed is set as a weighting parameter. Note that the weighting parameter is not limited to being set based on the priority based on such a constraint. The weighting parameter may also be variable for each control cycle. Below, an example will be described in which two or more specified sensor signals from multiple sensor signals are combined. Note that the user can specify any two or more sensor signals from the multiple sensor signals.

[0064] For example, when product assembly work using the robot 2 continues, the controller 1A generates, as a control signal for the robot 2, a control signal synchronized with the sampling cycle of a sensor capable of high-speed sampling, such as the force sensor 4 or the tactile sensor 7. In this case, the sensor signals input to the controller 1A from the cameras 31 and 32, which have a low sampling rate, are not updated for each control cycle of the robot 2. For this reason, the position information of the target object is calculated using sensor signals such as encoder signals other than the sensor signals from the cameras 31 and 32, recognition results for the other sensor signals, or speed information included in past state estimation results.

[0065] Also, for example, assume that the controller 1A recognizes the position of an object in three dimensions from two-dimensional position information acquired by each of the two cameras 31 and 32, and the robot 2 then carries out the work of loading the object into the processing machine 9. Three mutually perpendicular axes are defined as the X-axis, Y-axis, and Z-axis. In this case, assume that one of the two cameras 31 and 32 estimates the position of the object in an XZ plane including the X-axis and Z-axis, and the amount of rotation of the object within the XZ plane. Also assume that the other of the two cameras 31 and 32 estimates the position of the object in a YZ plane including the Y-axis and Z-axis, and the amount of rotation of the object within the YZ plane. In this case, the position in the Z-axis direction is measured redundantly by the two cameras 31 and 32. However, even if position feedback control is performed based on the measurements by each camera 31 and 32, it may not be possible to converge the position of the object to a desired position due to physical errors in each camera 31 and 32.

[0066] In response to this, the state estimation unit 13 performs state estimation processing to estimate the position of the object as viewed from the local coordinate system of the robot 2 or the tip position of the end effector 22, and the controller 1A can bring the position of the object closer to the target position.

[0067] For example, the recognition result obtained from the sensor signal of only one camera may contain errors due to problems such as parallax, physical errors, or occlusion. In such cases, the state estimation unit 13 performs state estimation based on sensor signals from several cameras other than the one camera, and the controller 1A controls the robot 2 based on the result of the state estimation. This allows the controller 1A to control the position of the object or the tip position of the end effector 22 to approach the target position.

[0068] The controller 1A integrates two or more sensor signals depending on the properties of the sensor signals, the situation of the robot 2 or its surroundings, or physical errors of each sensor, and estimates the state of the object or the robot 2. The controller 1A performs state estimation processing by weighting each of the two or more sensor signals according to their priority, thereby enabling the controller 1A to estimate the state of the object with high accuracy.

[0069] Fig. 5 is a diagram for explaining inputs and outputs of a plurality of sensor signal acquisition units 11, a storage unit 12, and a state estimation unit 13 included in the control device according to embodiment 1. Fig. 5 shows two of the plurality of sensor signal acquisition units 11 included in the controller 1A, the storage unit 12, and the state estimation unit 13.

[0070] Each sensor signal acquisition unit 11 receives a sensor signal S sc Each sensor signal acquisition unit 11 receives the sensor signal S sc and time information I indicating the time when the sensor signal was acquired. T and the sensor signal S sc The recognition result R obtained by the recognition process for rc and are output to the storage unit 12.

[0071] The state estimation unit 13 receives the sensor signal S sc and time information I T and the recognition result R rcThe state estimation unit 13 reads R, which is the state estimation result for the control command time in the Kth step. se When calculating K, the state estimation unit 13 calculates the state estimation result R se _(K-1) and the sensor signal at the (K-1)th step, S sc (K-1) from the storage unit 12. The state estimation unit 13 uses the information read from the storage unit 12 to calculate the state estimation result R se The state estimation unit 13 calculates the state estimation result R se _(K) is output to the control calculation unit 14 and the storage unit 12. se _(K) and S stored in the memory unit 12 sc The sensor signal at the Kth step, where k is the sensor signal, is read into the state estimation unit 13 when obtaining a state estimation result for the control command time in the (K+1)th step. In this way, the state estimation unit 13 obtains a state estimation result for the control command time by using the past state estimation results and past sensor signals stored in the storage unit 12.

[0072] For example, when the time is zero, the state estimation unit 13 determines the tip position of the end effector 22 based on the angles indicated by the output signals from the encoders in each axis motor of the robot arm 21 and the link parameters used in forward kinematics, assuming that there is no error in the model. At this time, the state estimation unit 13 determines the tip position of the end effector 22, assuming that there is no modeling error between the cameras 31 and 32 and the local coordinate system of the robot 2. The state estimation unit 13 also determines the position of the object, assuming that there is no modeling error between the cameras 31 and 32 and the local coordinate system of the robot 2. The link parameters are mechanism parameters used to express a serial link function, which is a type of mechanism possessed by the robot arm 21. For example, two or more processing systems included in the control system 10A synchronize their times using a timing device such as a global timer, and the time used as the reference for such time synchronization is set to zero.

[0073] Next, a description will be given of calculation of the command value by the control calculation unit 14. Fig. 6 is a diagram for explaining calculation of the command value by the control calculation unit 14 included in the control device according to the first embodiment.

[0074] The control calculation unit 14 receives a command value P cmd is input. Here, the ideal target position is an ideal target position as viewed from the local coordinate system of the robot 2. The command value representing the ideal target position is a command value generated in advance, and is a command value generated without taking physical errors into consideration. Also, the state estimation result output from the state estimation unit 13 is input to the control calculation unit 14. Here, R, which is the state estimation result for the control command time in the Kth step, is se It is assumed that (K) is input to the control calculation unit 14. The ideal target position input to the control calculation unit 14 is the ideal target position in the Kth step.

[0075] The control calculation unit 14 calculates the state estimation result R se Based on (K), the control calculation unit 14 calculates a correction amount ΔP used to correct the command value. The control calculation unit 14 calculates the command value P after correction using the calculated correction amount. cmd +ΔP. The controller 1A generates the command value P cmd Output +ΔP to Robot 2.

[0076] As a result, the controller 1A controls the robot 2 based on the command value that reflects the result of estimating the state of the object, and the controller 1A can realize control of the robot 2 taking the state of the object into consideration.

[0077] According to the first embodiment, the control device includes a plurality of sensor signal acquisition units 11 each acquiring a sensor signal, a memory unit 12 storing the plurality of sensor signals acquired by the plurality of sensor signal acquisition units 11 and time information indicating the time at which each sensor signal was acquired, a state estimation unit 13 estimating a state of an object based on two or more designated sensor signals from the stored plurality of sensor signals and the time information for each of the designated two or more sensor signals, and a control calculation unit 14 calculating a command value to be sent to a mechanical device based on the estimated state of the object. By estimating the state of the object based on the two or more designated sensor signals, the control device can integrate the detection results of each sensor while taking into account physical errors in the multiple sensors. This provides the effect of accurately integrating information acquired by each of the multiple sensors for the same object.

[0078] Furthermore, the state estimation unit 13 weights each of the two or more specified sensor signals and estimates the state of the object based on the two or more weighted sensor signals. This allows the control device to perform state estimation processing by weighting each of the two or more sensor signals according to their priority, and to estimate the state of the object with high accuracy.

[0079] Embodiment 2 7 is a diagram showing an example of the configuration of a control system 10B according to embodiment 2. The control system 10B includes a controller 1B which is a first control device, a controller 5B which is a second control device, and an information sharing unit 33. In embodiment 2, the same components as those in embodiment 1 are denoted by the same reference numerals, and the following mainly describes the configuration that is different from embodiment 1.

[0080] The controller 1B has the same configuration as the controller 1A shown in Fig. 1. The controller 1B acquires sensor signals output from each of a plurality of sensors connected to the controller 1B. Hereinafter, the first sensor signal refers to the sensor signal acquired by the controller 1B.

[0081] The controller 5B is a controller connected to the controller 1B. The controller 5B controls a sensor provided in a mechanical device or in the vicinity of the mechanical device. The controller 5B also acquires a sensor signal output from the sensor. Hereinafter, the second sensor signal refers to a sensor signal acquired by the controller 5B. The controller 5B is connected to the controller 1B via the information sharing unit 33.

[0082] The controller 5B includes a sensor signal acquisition unit 31 and a storage unit 32. The sensor signal acquisition unit 31 acquires a sensor signal output from a sensor connected to the controller 5B, i.e., a second sensor signal. The sensor signal acquisition unit 31 outputs the acquired sensor signal to the storage unit 32. The sensor signal acquisition unit 31 performs a recognition process on the acquired sensor signal to convert the sensor signal into information indicating a physical quantity. The sensor signal acquisition unit 31 outputs the physical quantity obtained by converting the sensor signal, i.e., the recognition result, to the storage unit 32. The sensor signal acquisition unit 31 generates time information indicating the time when the sensor signal was acquired. The sensor signal acquisition unit 31 outputs the time information to the storage unit 32 together with the sensor signal and the recognition result. The storage unit 32 stores the sensor signal, the recognition result, and the time information output from the sensor signal acquisition unit 31.

[0083] The information sharing unit 33 exchanges information shared between the controller 1B and the controller 5B. The information sharing unit 33 is, for example, a bus, a shared memory, TSN (Time-Sensitive Networking) using Ethernet (registered trademark), or ordinary Ethernet. In the control system 10B, information is exchanged via the information sharing unit 33 at a predetermined timing. The storage unit 12 of the controller 1B stores the sensor signal, the recognition result, and the time information, which are information read from the controller 5B via the information sharing unit 33. The storage unit 32 of the controller 5B stores the sensor signal, the recognition result, and the time information, which are information read from the controller 1B via the information sharing unit 33. The predetermined timing is the timing determined for each of the configurations of the information sharing unit 33 described above as examples.

[0084] By sharing information, each memory unit 12, 32 stores a first sensor signal, a second sensor signal, recognition results obtained from each of the first sensor signal and the second sensor signal, and time information indicating the time when each of the first sensor signal and the second sensor signal was acquired.

[0085] The state estimation unit 13 estimates the state of the object based on two or more specified sensor signals including a first sensor signal and a second sensor signal. The state estimation unit 13 outputs the estimation result of the state of the object to each of the memory unit 12 and the control calculation unit 14. The control calculation unit 14 calculates a command value to be sent to the mechanical device based on the estimated state of the object. The signal output unit 15 outputs a control signal indicating the command value calculated by the control calculation unit 14 to the robot 2, which is the mechanical device.

[0086] In the control system 10B, the reliability of the information shared via the information sharing unit 33, i.e., the information stored in each of the storage units 12 and 32, may vary. For example, a mechanism for automatically synchronizing information acquired by TSN can be used with TSN. However, for information with a large amount of data transmission, such as image signals, a network other than TSN may be used. In addition, the network may use TCP / IP (Transmission Control Protocol / Internet Protocol) for communication or UDP (User Datagram Protocol) for communication.

[0087] Under such conditions, the reliability of the update frequency of each of the multiple networks or the presence or absence of information updates decreases, and fluctuations may occur in the sensor signal even when the target is moving at a constant speed. In the second embodiment, a weighting parameter is set to deal with such fluctuations. This weighting parameter is designated as Pm2.

[0088] The controller 1B is provided with a detection means for detecting information loss or information delay in each configuration used in the information sharing unit 33. The detection means is not shown. The detection means detects a malfunction such as a sensor signal loss or delay when storing the sensor signals acquired by each controller 1B, 5B in the memory unit 12. When such a malfunction is detected, the state estimation unit 13 applies a weighting parameter Pm2 to perform state estimation processing using information other than the sensor signal in which the loss or delay was detected or past state estimation results.

[0089] In the first embodiment, the state estimation unit 13 applies a weighting parameter Pm1 to perform a state estimation process that prioritizes physically reliable information. In the second embodiment, by applying Pm1 and also by incorporating information selection by applying Pm2, it is possible to perform a state estimation with higher accuracy. The controller 1B outputs a command value corrected based on the highly accurate state estimation result, thereby preventing the robot 2 from performing an oscillating operation or the like and enabling the robot 2 to quickly complete a task. This allows the application of the robot 2 to improve productivity.

[0090] According to the second embodiment, the control system 10B includes a controller 1B serving as a first control device, a controller 5B serving as a second control device, and an information sharing unit 33 through which information shared between the controllers 1B and 5B is exchanged. The storage unit 12 stores a first sensor signal, a second sensor signal, and time information indicating the time at which each of the first and second sensor signals was acquired. The state estimation unit 13 estimates the state of the object based on two or more specified sensor signals including the first and second sensor signals. The controller 1B estimates the state of the object based on the sensor signals acquired by the controllers 1B and 5B for the same object. The controller 1B can perform state estimation by taking into account loss or delay in information output when information is shared and the physical reliability of the output from each sensor. This enables the controller 1B to estimate the state of the object with high accuracy.

[0091] Embodiment 3 8 is a diagram showing an example of the configuration of a control system 10C according to embodiment 3. The control system 10C includes a controller 1C which is a first control device, a controller 5C which is a second control device, and an information sharing unit 33. In embodiment 3, the same components as those in embodiment 1 or 2 are denoted by the same reference numerals, and the following mainly describes the configuration that is different from embodiment 1 or 2.

[0092] The controller 1C has a configuration similar to that of the controller 1A shown in Fig. 1, and includes a mode transition instruction unit 16. The controller 1C acquires a first sensor signal, which is a sensor signal output from each of a plurality of sensors connected to the controller 1C.

[0093] The controller 5C is a controller connected to the controller 1C. The controller 5C has a configuration similar to that of the controller 5B shown in FIG. 7. The controller 5C acquires a second sensor signal, which is a sensor signal output from a sensor connected to the controller 5C. The information sharing unit 33 exchanges information shared between the controller 1C and the controller 5C.

[0094] The state estimation unit 13 outputs the state estimation result to each of the memory unit 12, the control calculation unit 14, and the mode transition instruction unit 16. The mode transition instruction unit 16 instructs a transition of the control mode in a mechanical device that can be driven by each of a plurality of control modes. The mode transition instruction unit 16 instructs a transition of the control mode based on the estimation result of the state of the object by the state estimation unit 13.

[0095] Specifically, the mode transition instruction unit 16 determines whether to transition the control mode based on a change in the state indicated in the state estimation result, i.e., a change in position, velocity, acceleration, or the like. For example, a threshold for the amount of change in the state between two steps is determined in advance, and when the amount of change between the state in one of the two steps and the state in the other exceeds the threshold, the mode transition instruction unit 16 determines to transition the control mode. Based on this determination, the mode transition instruction unit 16 outputs an instruction for transitioning the control mode to the control calculation unit 14. The control calculation unit 14 transitions the control mode in controlling the robot 2 in accordance with the instruction from the mode transition instruction unit 16.

[0096] For example, the mode transition instruction unit 16 may determine, for each sensor signal input to the state estimation unit 13, S sc The sensor signals at the (K) and (K-1) steps are S scIf the amount of change, which is the difference from (K-1), exceeds a threshold value, the mode transition instruction unit 16 determines the transition of the control mode. se _(K) and the state estimation result of the (K-1)th step, R se If the amount of change, which is the difference from (K-1), exceeds a threshold, the transition of the control mode is determined.

[0097] The plurality of control modes include, for example, a normal mode in which the robot 2 is operated at a normal speed, and a low-speed mode in which the robot 2 is operated at a speed slower than the normal speed. In the normal mode, the control calculation unit 14 controls the robot 2 to operate at a predetermined target command value P cmd In the low speed mode, the control calculation unit 14 operates the robot 2 at a rate of change indicated by P cmd The robot 2 is operated at a rate of change smaller than the rate of change shown in .

[0098] The plurality of control modes may include, for example, a stop mode in which the robot 2 is stopped, or an escape mode in which the robot 2 is caused to perform an escape operation. In the stop mode, the control calculation unit 14, for example, sets the target command value to P cmd The target command value P from (K) to (K-1) steps cmd In the evacuation mode, the control calculation unit 14 does not change the target command value sequence P cmd _ hm By newly generating the position information, the robot 2 is caused to move the end effector 22 to the determined retreat point.

[0099] Within the normal mode, several detailed operation modes may be defined in accordance with the work processes of the robot 2. For example, the normal mode includes operation modes such as an assembly operation mode, a kitting operation mode, and a transport mode. The assembly operation mode is an operation mode used when assembling a processed product using the robot 2. The kitting operation mode is an operation mode used when arranging parts for the next process. The transport mode is an operation mode used when performing transport work including removal. In this way, the normal mode can be divided into multiple operation modes in accordance with the operations of each of the multiple work processes. The mode transition instruction unit 16 determines the control mode transition among these multiple control modes and instructs the control mode transition.

[0100] For example, the controller 1C may suspend the work of the robot 2 by transitioning the control mode from the assembly operation mode to the evacuation mode based on a sensor signal obtained from the controller 5C through the information sharing unit 33. The controller 1C may then further transition the control mode to the kitting operation mode based on a state estimation result based on the sensor signal obtained from the controller 5C and a state estimation result based on a sensor signal acquired by the controller 1C. In this way, the controller 1C may transition the control mode based on information shared between the controllers 1C and 5C.

[0101] According to the third embodiment, the controller 1C includes a mode transition instructing unit 16 that instructs a transition of the control mode of the mechanical device. The mode transition instructing unit 16 instructs a transition of the control mode based on the result of estimation of the state of the object by the state estimation unit 13. By instructing a transition of the control mode based on the result of the state estimation, the controller 1C can transition the control mode of the mechanical device according to the estimated state. For example, when an unexpected event occurs and it becomes difficult to control the mechanical device to the target state, the controller 1C can flexibly respond to the situation and control the mechanical device, such as by suspending the operation of the mechanical device. This makes it possible to avoid a situation in which the mechanical device cannot terminate its operation and the mechanical device or the object is damaged. As a result, the controller 1C can reduce the time the mechanical device's operation is suspended, thereby improving productivity through the application of the mechanical device.

[0102] Embodiment 4 9 is a diagram showing an example of the configuration of a control system 10D according to the fourth embodiment. The control system 10D includes a controller unit 41 and a simulator 42 that simulates the operation of the control devices. The controller unit 41 includes a controller 1D that is a first control device, a controller 5D that is a second control device, and an information sharing unit 33. In the fourth embodiment, the same components as those in the first to third embodiments are denoted by the same reference numerals, and the following description will mainly focus on the configuration that differs from the first to third embodiments.

[0103] The controller 1D has the same configuration as the controller 1A shown in FIG. 1. The controller 1D acquires sensor signals output from each of a plurality of sensors connected to the controller 1D. The controller 5D is a controller connected to the controller 1D. The controller 5D has the same configuration as the controller 5B shown in FIG. 7. The controller 5D acquires sensor signals output from the sensors connected to the controller 5D. The information sharing unit 33 exchanges information shared between the controller 1D and the controller 5D.

[0104] The simulator 42 includes a simulator 43, a comparator 44, and an analyzer 45. The simulator 43 generates a simulated signal that simulates a sensor signal to be acquired in the future based on a state estimation result and a command value calculated by inputting a sensor signal into a simulation model that simulates the controller unit 41. The comparator 44 reads a sensor signal at a step later than the sensor signal input into the simulation model, and compares the read sensor signal with the simulated signal. The analyzer 45 updates the simulation model by analyzing the comparison result by the comparator 44.

[0105] Specifically, the simulator 43 inputs two or more sensor signals from the controller unit 41 into the simulation model and causes the simulation model to perform calculations to determine a state estimation result and a command value. The simulator 43 simulates the sensor signals output by each of the two or more sensors in the next step, assuming that the robot 2 is controlled based on the calculated state estimation result and command value. In this way, the simulator 43 generates simulation signals that simulate the two or more sensor signals in the next step.

[0106] The comparison unit 44 reads two or more sensor signals in the next step from the controller unit 41. The comparison unit 44 compares each of the two or more read sensor signals with the simulation signal. The comparison unit 44 outputs the result of comparing each of the two or more sensor signals with the simulation signal to the analysis unit 45.

[0107] The simulator 43 instructs the control calculation unit 14 of the controller 1D to control the operation of the robot 2. The comparator 44 reads two or more sensor signals acquired in the controller unit 41 when the control calculation unit 14 operates the robot 2 in accordance with the instruction.

[0108] The analysis unit 45 analyzes the sensor signal that has a large difference from the simulated signal among the two or more sensor signals from the comparison results input to the analysis unit 45. The analysis unit 45 extracts model parameters from the simulation model that have a high correlation with the sensor signal that has a large difference from the simulated signal. In other words, the analysis unit 45 extracts model parameters that are considered to be factors that cause the difference between the simulated signal and the sensor signal.

[0109] The analysis unit 45 outputs information indicating the extracted model parameters to the simulation unit 43. The simulation unit 43 updates the simulation model by changing the model parameters indicated in the information.

[0110] The simulator 43 calculates a target command value using the updated simulation model. The simulator 43 outputs the calculated target command value to the control calculation unit 14. As a result, the control calculation unit 14 uses the target command value calculated in advance by the simulator 43 at the time of the next control. By applying the target command value calculated using the updated simulation model, the controller 5D can reduce vibrations of the robot 2 compared to when the target command value used in the previous control is applied as is. The controller 5D can achieve high-precision control of the robot 2.

[0111] By including the simulator 42, the control system 10D can generate a command value that takes into account the modeling error of the controller 5D when generating a target command value in advance. As a result, the control system 10D can cause the robot 2 to perform an operation with high responsiveness, thereby improving productivity through application of the robot 2.

[0112] According to the fourth embodiment, the control system 10D includes a simulator 42 that simulates the operation of the control device. The simulator 42 includes a simulator 43 that generates a simulated signal that simulates a sensor signal to be acquired in the future based on a state estimation result and a command value calculated by inputting a sensor signal to a simulation model that simulates the control device, a comparator 44 that reads a sensor signal at a step after the sensor signal input to the simulation model and compares the read sensor signal with the simulated signal, and an analyzer 45 that updates the simulation model by analyzing the comparison result by the comparator 44. This allows the control system 10D to cause the machine to perform an operation with high responsiveness, thereby improving productivity through the application of the machine.

[0113] Next, hardware for realizing the controllers 1A, 1B, 1C, and 1D according to the first to fourth embodiments will be described. The controllers 1A, 1B, 1C, and 1D are realized by using a processing circuit. The processing circuit may be a circuit in which a processor executes software, or may be a dedicated circuit.

[0114] When the processing circuit is realized by software, the processing circuit is, for example, the control circuit shown in FIG. 10. FIG. 10 is a diagram showing an example configuration of a control circuit 50 according to embodiments 1 to 4. The control circuit 50 includes an input unit 51, a processor 52, a memory 53, and an output unit 54. The input unit 51 is an interface circuit that receives data from outside the control circuit 50 and provides the data to the processor 52. The output unit 54 is an interface circuit that sends data from the processor 52 or the memory 53 to outside the control circuit 50. The function of the signal output unit 15 is realized by using the output unit 54.

[0115] 10, the processing functions of the sensor signal acquisition unit 11, the state estimation unit 13, the control calculation unit 14, and the mode transition instruction unit 16 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 53.

[0116] The processing circuit realizes the processing functions of the controllers 1A, 1B, 1C, and 1D by having the processor 52 read and execute programs stored in the memory 53. That is, the processing circuit includes the memory 53 for storing programs that result in the processing of the controllers 1A, 1B, 1C, and 1D. The programs stored in the memory 53 can also be said to be programs that cause a computer to execute the processing procedures and methods of the controllers 1A, 1B, 1C, and 1D. The memory 53 is also used as a temporary memory when the processor 52 executes various processes.

[0117] The processor 52 is a CPU (Central Processing Unit). The processor 52 may be a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, processor, or DSP (Digital Signal Processor). The memory 53 may be, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a minidisk, or a DVD (Digital Versatile Disc). The functions of the storage unit 12 are realized by using the memory 53.

[0118] When the processing circuit is a dedicated circuit, the controllers 1A, 1B, 1C, and 1D are realized by, for example, a hardware circuit shown in Fig. 11. Fig. 11 is a diagram showing an example of the configuration of a hardware circuit 55 according to the first to fourth embodiments.

[0119] The processing functions of the controllers 1A, 1B, 1C, and 1D are realized by a processing circuit 56, which is a dedicated circuit. The processing circuit 56 is a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a circuit combining these. The processing functions of the controllers 1A, 1B, 1C, and 1D may be realized by the processing circuit 56 on a function-by-function basis, or all of the functions may be realized collectively by the processing circuit 56. The processing functions of the controllers 1A, 1B, 1C, and 1D may also be realized by a combination of the control circuit 50 shown in FIG. 10 and the processing circuit 56 shown in FIG. 11.

[0120] The programs according to the first to fourth embodiments may be provided by being stored on a recording medium such as a CD (Compact Disc)-ROM or a DVD-ROM. The programs according to the first to fourth embodiments may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network such as the Internet. The programs according to the first to fourth embodiments may be provided or distributed via a network such as the Internet.

[0121] Each of the controllers 5B, 5C, and 5D according to embodiments 2 to 4 and the simulator 42 according to embodiment 4 has a hardware configuration similar to the hardware configuration shown in FIG. 10 or a hardware configuration similar to the hardware configuration shown in FIG. 11.

[0122] The configurations shown in the above embodiments are examples of the contents of the present disclosure. The configurations of each embodiment can be combined with other known technologies. The configurations of each embodiment can also be combined as appropriate. Part of the configuration of each embodiment can be omitted or modified without departing from the gist of the present disclosure. [Explanation of symbols]

[0123] 1A, 1B, 1C, 1D, 5, 51, 52, 53, 54, 5B, 5C, 5D Controller, 2, 8 Robot, 31, 32, 6 Camera, 4 Force sensor, 7 Tactile sensor, 9 Processing machine, 10A, 10B, 10C, 10D Control system, 11, 31 Sensor signal acquisition unit, 12, 32 Memory unit, 13 State estimation unit, 14 Control calculation unit, 15 Signal output unit, 16 Mode transition instruction unit, 21 Robot arm, 22 End effector, 23 Object, 33 Information sharing unit, 41 Controller unit, 42 Simulator, 43 Simulator, 44 Comparison unit, 45 Analysis unit, 50 Control circuit, 51 Input unit, 52 Processor, 53 Memory, 54 Output unit, 55 Hardware circuit, 56 Processing circuit.

Claims

1. A control device that controls a mechanical device that moves an object, a plurality of sensor signal acquisition units each acquiring a sensor signal output from a sensor provided on the mechanical device or in the vicinity of the mechanical device; a storage unit configured to store the plurality of sensor signals acquired by the plurality of sensor signal acquisition units and time information indicating the time at which each of the sensor signals was acquired; a state estimation unit that estimates a state of the object by integrating two or more designated sensor signals from the stored plurality of sensor signals and the time information of each of the designated two or more sensor signals; a control calculation unit that calculates a command value to be sent to the mechanical device based on the estimated state of the object, The state estimation unit prioritizes at least one of the two or more designated sensor signals, which has a higher accuracy of detecting the object than the other sensor signals, over the other sensor signals based on a difference in accuracy of detecting the object between the two or more designated sensor signals, and integrates the two or more designated sensor signals. A control device characterized by:

2. The state estimation unit weights each of the two or more designated sensor signals according to a priority, and estimates the state of the object based on the two or more weighted sensor signals.

2. The control device according to claim 1.

3. Between a first control device which is the control device and a second control device which is a control device other than the control device, information is shared by transferring information via an information sharing unit, and the first control device includes a detection means for detecting a lack of information or a delay in information in the information sharing unit, the storage unit stores a first sensor signal, which is the sensor signal acquired by the sensor signal acquisition unit of the first control device, and a second sensor signal output from a sensor provided on the mechanical device or in the periphery of the mechanical device and connected to the second control device; The state estimation unit Estimating a state of the object based on two or more designated sensor signals including the first sensor signal and the second sensor signal; and When the detection means detects a lack or delay of the first sensor signal or the second sensor signal when storing the first sensor signal or the second sensor signal in the storage unit, the state of the object is estimated using information other than the first sensor signal or the second sensor signal whose lack or delay has been detected, or a past state estimation result.

3. The control device according to claim 1 or 2.

4. a mode transition instruction unit that instructs a transition of the control mode in the mechanical device that can be driven in each of a plurality of control modes, The mode transition instruction unit instructs a transition of the control mode based on an amount of change in the sensor signal between control steps by the control calculation unit or an amount of change in a result of estimation of the state of the object by the state estimation unit between control steps by the control calculation unit.

3. The control device according to claim 1 or 2.

5. A control device is provided to control a mechanical device that moves an object, The control device a sensor signal acquisition unit that acquires sensor signals output from a plurality of sensors provided on the mechanical device or in the vicinity of the mechanical device; a storage unit configured to store the acquired sensor signals and time information indicating the time at which each of the sensor signals was acquired; a state estimation unit that estimates a state of the object by integrating two or more designated sensor signals from the stored plurality of sensor signals and the time information of each of the designated two or more sensor signals; a control calculation unit that calculates a command value to be sent to the mechanical device based on the estimated state of the object, The state estimation unit prioritizes at least one of the two or more designated sensor signals, which has a higher accuracy of detecting the object than the other sensor signals, over the other sensor signals based on a difference in accuracy of detecting the object between the two or more designated sensor signals, and integrates the two or more designated sensor signals. A control system comprising:

6. a first control device that is the control device and that acquires a first sensor signal that is the sensor signal; a second control device connected to a sensor provided on the mechanical device or in the vicinity of the mechanical device, and configured to acquire a second sensor signal output from the connected sensor; an information sharing unit that exchanges information shared between the first control device and the second control device; the first control device includes a detection means for detecting a loss of information or a delay in information in the information sharing unit; the storage unit stores the first sensor signal, the second sensor signal, and the time information indicating a time when each of the first sensor signal and the second sensor signal was acquired; The state estimation unit Estimating a state of the object based on two or more designated sensor signals including the first sensor signal and the second sensor signal; and When the detection means detects a lack or delay of the first sensor signal or the second sensor signal when storing the first sensor signal or the second sensor signal in the storage unit, the state of the object is estimated using information other than the first sensor signal or the second sensor signal whose lack or delay has been detected, or a past state estimation result.

6. The control system of claim 5.

7. a simulator that simulates the operation of the control device; The simulator comprises: a simulator that generates a simulation signal that simulates the sensor signal to be acquired in the future, based on the state estimation result calculated by inputting the sensor signal to a simulation model that simulates the control device and the command value; a comparison unit that reads the sensor signal at a step subsequent to the step at which the sensor signal was input to the simulation model, and compares the read sensor signal with the simulation signal; an analysis unit that updates the simulation model based on an analysis of the comparison result by the comparison unit; 6. The control system of claim 5.

8. On the computer, acquiring sensor signals output from a plurality of sensors provided on or around a mechanical device that moves an object; storing the acquired plurality of sensor signals and time information indicating the time at which each of the sensor signals was acquired; a step of estimating a state of the object by integrating two or more designated sensor signals from the stored plurality of sensor signals based on the two or more designated sensor signals and the time information of each of the two or more designated sensor signals; calculating a command value to be sent to the machine device based on the estimated state of the object; In the step of estimating the state of the object, at least one of the two or more designated sensor signals, which has a higher accuracy of detecting the object than the other sensor signals, is prioritized over the other sensor signals based on a difference in accuracy of detecting the object between the two or more designated sensor signals, and the two or more designated sensor signals are integrated. A program characterized by: