Control device, control method, and control program

Abstract

To provide a control device, a control method and a control program which, when a machine performs an operation such as scooping up an object T, scoop up an object by utilizing a force generated in the machine.SOLUTION: When a robot 12, which is an example of a machine, performs an operation of scooping up an object T, a control device 14 acquires force information representing a force generated in an instrument provided in the robot 12. The control device 14 controls the operation of the robot 12 so that the mass of the object T to be scooped up becomes a target mass in accordance with the acquired force information.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a control device, a control method, and a control program. [Background technology]

[0002] Conventionally, there is known a technique for controlling the movement of a robot scooping up a granular object (see, for example, Non-Patent Document 1). This technique controls the movement of the robot by inputting a height map obtained from an image of the granular object into a convolutional neural network.

[0003] There is also a known technology for controlling the operation of an excavator when scooping up soil (see, for example, Non-Patent Document 2). This technology controls the switching of the bucket from the drag phase to the scoop phase at the correct timing when the excavator drags the bucket along the ground surface and rotates the bucket to scoop up and collect the soil. This technology uses images of the soil taken by a depth camera as input data. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] C. Schenck, J. Tompson, S. Levine, and D. Fox, “Learning robotic manipulation of granular media,” in Conference on Robot Learning, 2017, pp. 239-248. [Non-patent document 2] RJ Sandzimier and HH Asada, “A data-driven approach to pre-diction and optimal bucket-filling control for autonomous excavators,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2682-2689, 2020. Summary of the Invention [Problem to be solved by the invention]

[0005] When a machine performs an action such as scooping up an object, a force is generated at a specific location on the machine. Information about the force acting on the machine is useful, and it is thought that by utilizing this force information, it may be possible to control the operation of the machine to scoop up an object.

[0006] The technologies of Non-Patent Document 1 and Non-Patent Document 2 control the operation of a machine that scoops up an object by using an image of the object to be scooped up, and do not use the force generated by the machine. In particular, the force required to scoop up the object varies depending on the properties of the object to be scooped up (for example, viscosity), so problems may arise when controlling a machine using visual information (for example, an image). This is because the hardness or viscosity of an object can change depending on factors such as humidity, temperature, or external pressure (for example, pressure generated when soil is compacted), and such information cannot be determined from an image alone.

[0007] The present disclosure has been made in consideration of the above points, and aims to scoop up an object by utilizing the force generated in a machine when the machine performs an action such as scooping up an object. [Means for solving the problem]

[0008] In order to achieve the above object, the control device according to the present disclosure is a control device that includes: an acquisition unit that acquires force information representing the force generated in a tool that is equipped on the machine to scoop up an object and a target mass of the object to be scooped up when the machine performs an operation to scoop up an object; and a control unit that acquires the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object to be scooped up by the operation of the machine when the force information is input; and controls the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass.

[0009] Furthermore, the control method disclosed herein is a control method in which, when a machine performs an operation of scooping up an object, force information representing the force generated in a tool provided on the machine for scooping up the object and a target mass of the object to be scooped up are acquired, the force information is input to a trained model that, when the force information is input, outputs a parameter for adjusting the mass of the object scooped up by the operation of the machine to acquire the parameter, and a computer executes a process to control the operation of the machine based on the parameter so that the mass of the object to be scooped up becomes the target mass.

[0010] In addition, the control program of the present disclosure is a control program that, when a machine performs an operation of scooping up an object, acquires force information representing the force generated in a tool that the machine is equipped with to scoop up the object and a target mass of the object to be scooped up, acquires the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, and causes a computer to execute a process of controlling the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass. [Effects of the Invention]

[0011] According to the control device, control method, and control program of the present disclosure, when a machine performs an action such as scooping up an object, the force generated in the machine can be used to scoop up the object. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 2 is a diagram for explaining a control system according to the present embodiment. [Figure 2] FIG. 1 is a diagram for explaining an overview of the present embodiment. [Figure 3] FIG. 1 is a diagram for explaining a trained model for mass estimation according to this embodiment. [Figure 4] 1 is a block diagram showing a schematic configuration of a control system according to an embodiment of the present invention; [Figure 5] FIG. 2 is a block diagram showing the hardware configuration of the control device according to the present embodiment. [Figure 6] 4 is a flowchart showing the flow of a control process in the present embodiment. [Figure 7] FIG. 10 is a diagram showing the results of this example. [Figure 8] FIG. 10 is a diagram illustrating another example of a trained model according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] An example of an embodiment of the present disclosure will be described below with reference to the drawings. In this embodiment, a control system equipped with a control device according to the present disclosure will be described as an example. Note that the same reference numerals are used in the drawings to designate identical or equivalent components and parts. Furthermore, the dimensions and proportions of the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0014] FIG. 1 is a diagram illustrating this embodiment. As shown in FIG. 1, a robot 12 of this embodiment performs an action of scooping up an object T having viscosity and plasticity. The object T is, for example, an object having viscosity and plasticity, such as soil or ice cream. Specifically, as shown in FIG. 1, a spoon SP, which is an example of a tool for scooping up the object T, is attached to a gripper 12A of the robot 12, and the robot 12 moves the spoon SP to scoop up the object T. The robot 12 scoops up the object T using the spoon SP so that the mass of the object to be scooped up becomes a target mass.

[0015] As shown in FIG. 1 , force-tactile sensors 13A and 13B are attached to the gripper 12A of the robot 12, and are configured to detect the force generated in the gripper 12A of the robot 12 when the spoon SP is moved. The force-tactile sensors 13A and 13B used in this embodiment are capable of detecting three-dimensional forces acting on the surface of the gripper 12A. Such force-tactile sensors 13A and 13B are also referred to as tactile sensors. In this embodiment, the force detected by the force-tactile sensors 13A and 13B and a pre-trained mass estimation model, which is a machine learning model, are used to estimate the mass of an object scooped up by the spoon SP. When the position information of the spoon SP and the force information detected by the force-tactile sensors 13A and 13B are input, the pre-trained mass estimation model outputs an estimate of the mass of the object scooped up by the spoon SP. In this embodiment, the way to move the spoon SP is determined based on the estimated mass of the object obtained in this manner. The force-tactile sensor may be of any type. For example, the force-tactile sensor may be of an image type, a magnetic type, or a strain gauge type, and any of these types may be employed. The force-tactile sensor may be of a single-point measurement type or a distribution type, and any of these types may be employed. The force-tactile sensor may also have any number of measurement point cells.

[0016] More specifically, in this embodiment, as shown by X1 and X2 in Fig. 1, the robot 12 moves the spoon SP in various ways to scoop up an object, and the mass of the object is used as training data to generate a trained model for mass estimation. Then, as shown by Y1, Y2, and Y3 in Fig. 1, the robot 12 is controlled to move the spoon SP in accordance with the target mass of the object to be scooped up, so that the mass of the scooped up object becomes the target mass. This will be explained in detail below.

[0017] <Summary> FIG. 2 is a diagram for explaining an overview of this embodiment. In this embodiment, as shown in FIG. 2, the gripper 12A of the robot 12 to which the spoon SP is attached is moved to scoop up the object T. In FIG. 2, p1 is the position of the spoon SP (or the position of the gripper 12A) at the start, p2 is the position of the spoon SP (or the position of the gripper 12A) while the spoon SP is being inserted into the object T, p3 is the position of the spoon SP (or the position of the gripper 12A) when the spoon SP is deeply inserted into the object T, and p4 is the position of the spoon SP (or the position of the gripper 12A) corresponding to the goal. In this embodiment, the position p3 of the spoon SP is optimized based on force information obtained by the force-tactile sensors 13A and 13B from the time when the spoon SP is at position p1 to the time when the spoon SP is at position p2 (force information corresponding to time interval C in FIG. 2). Note that this embodiment will be described taking as an example a case where the spoon SP is moved in the xz plane as shown in FIG. 2. Note that a mode in which the spoon SP is moved in the y direction is also feasible.

[0018] In this embodiment, an estimated value w^ of the mass of the object T present on the spoon SP is calculated using a trained model for mass estimation, which will be described later. Then, the position p3 of the spoon SP is adjusted so that the following equation is satisfied. * In addition, w in the following formula (1) is determined. target is the target mass of the object T to be scooped up.

[0019]

number

[0020] <Pre-trained model for mass estimation> 3 is a diagram for explaining the trained model for mass estimation of this embodiment. As shown in Fig. 3, the trained model for mass estimation of this embodiment includes a model M1 for state estimation and a model M2 for estimating the mass of an object.

[0021] The state estimation model M1 includes a convolutional neural network (CNN) 30 and a transformer 31. The convolutional neural network 30 and the transformer 31 are known machine learning models.

[0022] Furthermore, model M2 for estimating the mass of an object is composed of a multilayer perceptron (MLP) 32. As shown in Fig. 3, when time-series data S1 and S2 of force information detected by force tactile sensors 13A and 13B are input to convolutional neural network 30, convolutional neural network 30 outputs feature vectors v1, v2,..., vn.

[0023] S1 is time series data of force information detected by the force-tactile sensor 13A, and S2 is time series data of force information detected by the force-tactile sensor 13B. tp1 is the force information detected by the force tactile sensors 13A and 13B when the spoon SP is at the position p1, and S tp2 is the force information detected by the force-tactile sensors 13A and 13B when the position of the spoon SP is p2. Therefore, S1 and S2 are time-series data of the force information detected by the force-tactile sensors 13A and 13B in the time interval when the position of the spoon SP is from p1 to p2.

[0024] As shown in FIG. 3, the data output from the convolutional neural network 30 is subjected to predetermined processing using positional encoding E, a known technique, to generate feature vectors v1, v2,..., vn. Note that A in FIG. 3 is a symbol known in the field of machine learning and represents the direct sum of vectors. Each of the feature vectors v1, v2,..., vn corresponds to force information at a particular time. For example, feature vector v1 is a feature vector corresponding to force information detected by force-tactile sensors 13A and 13B when the spoon SP is positioned at p1. Also, for example, feature vector vn is a feature vector corresponding to force information detected by force-tactile sensors 13A and 13B when the spoon SP is positioned at p2.

[0025] As shown in Fig. 3, each of the feature vectors v1, v2,..., vn is input to a transformer 31, which outputs each of the feature vectors v1', v2',..., vn'. Then, as shown in Fig. 3, the mean vector v of the feature vectors v1', v2',..., vn' is calculated.

[0026] This average vector v is data that reflects the time-series data of force information obtained by the force-tactile sensors 13A and 13B, and by using this data, it is possible to estimate how much of the object T can be scooped up. When the average vector v and candidate position p3 of the spoon SP are input to the multilayer perceptron 32, an estimated value w^ of the mass of the object T when the spoon SP is moved to position p3 and the object T is scooped up is output.

[0027] 3, the convolutional neural network 30 is a model capable of appropriately processing three-dimensional force information generated on a two-dimensional plane of the gripper 12A of the robot 12. The transformer 31 is a model capable of appropriately processing time-series information. Therefore, in this embodiment, by using these two models, it is possible to accurately estimate the estimated value w^ of the mass of the object T when the object T is scooped up, based on the time-series data of force information obtained by the force-tactile sensors 13A and 13B attached to the gripper 12A.

[0028] Therefore, the estimated mass w^ of the object T scooped up by the spoon SP can be expressed by the following equation: where F represents the trained model for mass estimation, and S tp1:tp2 represents time-series data of force information detected by the force tactile sensors 13A and 13B in the time interval when the position of the spoon SP is from p1 to p2.

[0029]

number

[0030] <Optimization of spoon SP position p3> As described above, once the estimated value w^ of the mass of the object T scooped up by the spoon SP is calculated, the optimal position p3 of the spoon SP can be determined using the above equation (1). * In this embodiment, the optimum spoon SP position p3 is calculated using a grid search technique. * Specifically, a plurality of candidate positions p3 of the spoon SP are set, and each of the candidate positions p3 is input to the trained model for mass estimation to calculate the estimated mass w^ of the object T to be scooped up corresponding to each of the candidate positions p3. Then, among the estimated mass w^ corresponding to each of the candidate positions p3, the target mass w target The estimated mass w^ with the smallest difference between the two is identified, and the candidate position p3, which is the input data when the estimated value w^ is output, is determined as the optimal position p3 *Identify as:

[0031] (Control System 10) FIG. 4 is a block diagram illustrating a schematic configuration of a control system 10 according to this embodiment. As illustrated in FIG. 4, the control system 10 includes force-tactile sensors 13A and 13B, a robot 12, and a control device 14. When the robot 12, which is an example of a machine, performs an operation to scoop up an object T, the control device 14 acquires force information representing a force generated in a gripper 12A, which is an example of a part of the robot 12. The control device 14 then controls the operation of the robot 12 based on the acquired force information so that the mass of the object T to be scooped up becomes a target mass. Specifically, the control device 14 acquires parameters by inputting the acquired force information into a trained model that, in response to input of force information, outputs parameters for adjusting the mass of the object T to be scooped up by the operation of the robot 12. The control device 14 then controls the operation of the robot 12 based on the acquired force information so that the mass of the object to be scooped up becomes a target mass. In this embodiment, the trained model is a trained model for mass estimation that outputs an estimated value of the mass of the object T scooped up by the spoon SP when force information and position information of the spoon SP are input. Also, the parameter is an estimated value of the mass of the object T.

[0032] Force-tactile sensors 13A and 13B are attached to gripper 12A of robot 12 and sequentially detect three-axis forces, which are force information. Then, force-tactile sensors 13A and 13B output the obtained force information to control device .

[0033] Fig. 5 is a block diagram showing the hardware configuration of the control device 14 according to this embodiment. As shown in Fig. 5, the control device 14 has a CPU (Central Processing Unit) 42, a memory 44, a storage device 46, an input / output I / F (Interface) 48, a storage medium reader 50, and a communication I / F 52. Each component is connected to each other via a bus 54 so as to be able to communicate with each other.

[0034] The storage device 46 stores control programs for executing the various processes described below. The CPU 42 is a central processing unit that executes various programs and controls the various components. That is, the CPU 42 reads the programs from the storage device 46 and executes them using the memory 44 as a work area. The CPU 42 controls the various components and performs various arithmetic operations in accordance with the programs stored in the storage device 46.

[0035] The memory 44 is made up of RAM (Random Access Memory) and serves as a working area to temporarily store programs and data. The storage device 46 is made up of ROM (Read Only Memory), HDD (Hard Disk Drive), SSD (Solid State Drive), etc., and stores various programs including the operating system and various data.

[0036] The input / output I / F 48 is an interface for inputting data from an external device and outputting data to an external device. Also, input devices for various inputs, such as a keyboard or a mouse, and output devices for various information outputs, such as a display or a printer, may be connected. A touch panel display may be used as the output device, allowing it to function as an input device.

[0037] The storage medium reader 50 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

[0038] The communication I / F 52 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0039] Next, the functional configuration of the control device 14 will be described. As shown in Fig. 4, the control device 14 functionally includes an acquisition unit 16 and a control unit 22. A data storage unit 18 and a trained model storage unit 20 are provided in a predetermined storage area of ​​the control device 14. Each functional configuration is realized by the CPU 42 reading each program stored in the storage device 46, expanding it in the memory 44, and executing it.

[0040] The data storage unit 18 stores time-series data of force information detected by the force-tactile sensors 13A and 13B. The data storage unit 18 also stores control data and the like when the robot 12 operates. The data storage unit 18 also stores a target mass w of the object T to be scooped up. target For example, the target mass is preset by the user.

[0041] The trained model storage unit 20 stores the trained model for mass estimation described above. The trained model for mass estimation is a trained model that outputs an estimated value w^ of the mass of the object T scooped up by the spoon SP when force information and position information of the spoon SP are input. Note that the trained model for mass estimation in this embodiment is generated in advance by known supervised learning or unsupervised learning, etc., based on training data collected in advance in learning phases X1 and X2 as shown in FIG. 1. Note that the training data is, for example, data in which time-series data of force information obtained by the force-tactile sensors 13A and 13B is associated with the mass of the object T scooped up by the spoon SP.

[0042] The acquisition unit 16 acquires time-series data of force information obtained by the force tactile sensors 13A and 13B from the data storage unit 18. The acquisition unit 16 also acquires a target mass w target is acquired from the data storage unit 18.

[0043] The control unit 22 reads out the trained model for mass estimation stored in the trained model storage unit 20. Then, the control unit 22 inputs the force information acquired by the acquisition unit 16 and multiple candidates for position information of the spoon SP that scoops up the object T, and acquires an estimated value w^ of the mass for each of the multiple candidates for position information.

[0044] The control unit 22 determines the target mass w^ from among the multiple estimated mass values ​​w^. target The candidate position information of the spoon SP corresponding to the estimated value that minimizes the difference between * Then, the control unit 22 determines the position p3 indicated by the candidate for the position information of the identified spoon SP. * The robot 12 is controlled to move the spoon SP in the direction indicated by the arrow.

[0045] Next, the operation of the control system 10 according to this embodiment will be described.

[0046] First, learning data is collected while the robot 12 is performing an operation of scooping up the object T, and is input to the control device 14. Then, the control unit 22 of the control device 14 generates a trained model for mass estimation based on the learning data. As a result, the trained model for mass estimation is stored in the trained model storage unit 20.

[0047] Next, the robot 12 performs an operation of actually scooping up the object T. Specifically, upon receiving a predetermined instruction signal, the control device 14 executes a control processing routine shown in FIG.

[0048] In step S100, the acquisition unit 16 acquires time-series data of force information representing the force generated in the gripping unit 12A of the robot 12 while the robot 12 is performing an action of scooping up the object T using the spoon SP. Also in step S100, the acquisition unit 16 acquires time-series data of force information representing the force generated in the gripping unit 12A of the robot 12 while the robot 12 is performing an action of scooping up the object T using the spoon SP. target Get.

[0049] In step S102, the control unit 22 inputs the time series data of the force information obtained in step S100 and multiple candidates of position information of the spoon SP (or the gripping portion 12A) that scoops up the object T into the trained model for mass estimation stored in the trained model memory unit 20, and obtains an estimated value w^ of the mass of the object for each of the multiple candidates of position information.

[0050] In step S104, the control unit 22 determines the target mass w target The candidate position information of the spoon SP corresponding to the estimated value that minimizes the difference between * Identify as:

[0051] In step S106, the control unit 22 determines the optimum position p3 of the spoon SP identified in step S104. * A control command is output to the robot 12 so that the above is realized. As a result, the object T having a mass close to the target mass is picked up.

[0052] As described above, the control device according to this embodiment acquires force information representing forces generated in parts of a robot, which is an example of a machine, while the robot is performing an action to scoop up an object. The control device controls the robot's action based on the acquired force information so that the mass of the object to be scooped up becomes a target mass. This allows the robot to scoop up the object by utilizing the force generated in the robot when performing an action such as scooping up an object.

[0053] Specifically, the control device of this embodiment is a technology in which a robot 12 holding a spoon SP scoops up a viscoplastic object, such as wet sand or ice cream, at a certain target mass. The control device of this embodiment is equipped with a force-tactile sensor or tactile sensor on the robot 12, and inputs time-series information on the force when the spoon SP pierces the object T and the hand posture of the robot 12 to learn a model that predicts the amount of the object T scooped up. Then, in this embodiment, the learned model is used to determine the hand posture for scooping up the object T of the target mass. This makes it possible to weigh a viscoplastic object at an arbitrary target mass. Furthermore, according to this embodiment, the use of the force-tactile sensor or tactile sensor makes it possible to grasp the physical properties (e.g., hardness) of the object T, enabling more accurate weighing. [Example]

[0054] Next, an example will be described. In this example, an experiment is conducted to verify the effectiveness of the proposed method. Specifically, in this example, a comparison is made between a case where the target object, viscoplastic sand (kinetic sand), is scooped up by controlling the robot's movement using the trained model F for mass estimation shown in FIG. 3 above, and a case where the target object, sand (kinetic sand), is scooped up using only model M2 shown in FIG. 3 above. FIG. 7 shows the results of this example. The horizontal axis of FIG. 7 represents the target mass of the sand to be scooped up, and the vertical axis represents the difference between the target mass and the mass of the sand actually scooped up.

[0055] As shown in Figure 7, the difference between the target mass and the actual mass of the sand scooped up tends to be smaller when using the proposed method of this embodiment (trained model F for mass estimation) compared to the conventional method (using only model M2). This shows that the method of this embodiment makes it possible to accurately and efficiently generate a trained strategy to be used when moving an object.

[0056] The technology of the present disclosure is not limited to the above-described embodiment, and various modifications and applications are possible within the scope of the gist of this disclosure.

[0057] For example, in the above embodiment, an example has been described in which the object T is scooped up by controlling the operation of the robot 12, but this is not limiting. For example, the machine to be controlled does not have to be the robot 12, and any machine capable of scooping up the object T may be used. For example, by applying this embodiment to a construction machine such as a shovel, the machine may be configured to control an operation such as scooping up soil. If the machine to be controlled is a shovel, the tool for scooping up the object is the bucket of the shovel.

[0058] In the above embodiment, the utensil used to scoop up an object is a spoon, but the present invention is not limited to this. Any utensil that can scoop up an object may be used. For example, it may be a cup.

[0059] In the above embodiment, the force-tactile sensors 13A and 13B are tactile sensors capable of detecting forces in three-dimensional directions and capable of detecting forces occurring at various locations on the two-dimensional plane of the grip portion 12A, but this is not limiting. For example, the force-tactile sensors 13A and 13B may be sensors capable of detecting forces in at least one direction of three-dimensional directions and capable of detecting forces occurring at one location on the grip portion 12A. Even if such sensors are used, it is possible to realize the control device of this embodiment.

[0060] In the above embodiment, the force-tactile sensors 13A, 13B are installed in the gripper 12A of the robot 12, but this is not limiting. The force-tactile sensors may be installed in any location as long as they can detect the force acting on the tool. Furthermore, instead of the force-tactile sensors, the force acting on the tool may be estimated from a torque sensor or a motor current value mounted on a joint of the robot 12.

[0061] In the above embodiment, four points (p1, p2, p3, p4) are used as the position of the spoon SP, but this is not limiting. More or fewer points may be used.

[0062] In the above embodiment, for example, as shown in FIG. 2, the force information from the start point of the robot 12's scooping up the object T (for example, the point in time when the spoon SP is stuck in the object T) is used, but the present invention is not limited to this. For example, the robot 12 may control the operation of the robot 12 by using force information obtained while the robot 12 is scooping up the object T (for example, force information obtained in a time period after the time when the spoon SP is stuck in the object T). Furthermore, force information obtained when the robot 12 is sticking the spoon SP into the object T may also be used.

[0063] In the above embodiment, the model shown in FIG. 8 is used as the trained model for mass estimation F, but the present invention is not limited to this. Any model may be used as the trained model for mass estimation. For example, the transformer 8 may be replaced with a known LSTM (Long Short Term Memory) model. Furthermore, the robot 12 may be controlled using a model other than the trained model for mass estimation.

[0064] For example, as shown in FIG. 8(A), a trained model for motion estimation may be used that outputs motion information representing the motion of the robot 12 when the target mass and force information of the object T are input. In this case, the control unit 22 inputs the force information and the target mass into the trained model for motion estimation, which outputs motion information of the robot 12, which is an example of a machine, when the force information and the target mass are input, thereby acquiring motion information of the robot 12 and controlling the motion of the robot 12 based on the motion information. Note that the motion information may be, for example, trajectory information of the arm or gripper 12A of the robot 12. In this case, the parameters for adjusting the mass of the object output from the trained model correspond to the motion information.

[0065] Alternatively, for example, as shown in FIG. 8(A), a trained model for motion estimation may be used that outputs motion information representing the motion of the robot 12 when a target mass of the object T and stiffness information of the object T are input. In this case, for example, the control unit 22 estimates stiffness information representing the stiffness of the object T based on force information when the spoon SP is inserted into the object T. Then, the control unit 22 inputs the stiffness information representing the stiffness of the object T and the target mass into the trained model for motion estimation, which outputs motion information of the robot 12 when the stiffness information and the target mass are input, thereby acquiring motion information of the robot 12 and controlling the motion of the robot 12 based on the motion information. In this case, the parameters for adjusting the mass of the object output from the trained model correspond to motion information.

[0066] Alternatively, for example, as shown in Fig. 8(B), a trained model for motion estimation may be used that outputs motion information of the robot 12 when force information and position information of the spoon SP are input. In this case, for example, the control unit 22 inputs force information and position information of the spoon SP from the start time of the scooping motion to the current time into the trained model for motion estimation to acquire motion information of the robot 12, and controls the motion of the robot 12 based on the motion information. Note that in this case, the parameters for adjusting the mass of the object output from the trained model correspond to motion information.

[0067] In the above embodiment, the object T is a granular object having viscosity and plasticity, but the present invention is not limited to this. Any object may be used as long as force information can be obtained from the object.

[0068] In the above embodiment, the optimum position p3 is determined using a grid search technique. * However, the present invention is not limited to this. For example, the optimum position p3 can be determined by using various methods, including those that utilize the gradient of the above formula (1) and those that do not utilize the gradient. * It is also possible to specify

[0069] Furthermore, the processes executed by the CPU after reading the software (program) in the above-described embodiments may be executed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) whose circuit configuration can be changed after fabrication, such as field-programmable gate arrays (FPGAs), and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors with circuit configurations specifically designed to execute specific processes. Each process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0070] In the above embodiment, the programs are pre-stored (installed) in a storage device, but the present invention is not limited to this. The programs may be provided in a form stored on a storage medium such as a CD-ROM, DVD-ROM, Blu-ray disc, or USB memory. The programs may also be downloaded from an external device via a network.

[0071] (Addendum) The following additional notes are provided regarding aspects of the present disclosure.

[0072] (Appendix 1) an acquisition unit that acquires, when a machine performs an operation of scooping up an object, force information representing a force generated in a tool that is provided in the machine and that scoops up the object, and a target mass of the object to be scooped up; a control unit that acquires parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, and controls the operation of the machine based on the parameters so that the mass of the object scooped up becomes a target mass; A control device including: (Appendix 2) the trained model is a trained model for mass estimation that outputs an estimated value of the mass of the object scooped up by the tool when the force information and position information of the tool are input; the parameter is an estimate of the mass of the object; The control unit obtaining an estimated value of the mass for each of the plurality of candidate position information by inputting the acquired force information and the candidate position information of the tool into the trained model for mass estimation; Identifying a candidate for position information of the tool corresponding to the estimated value that has the smallest difference from the target mass among the plurality of estimated values ​​of the mass; controlling the operation of the machine so as to move the tool to the position represented by the identified candidate for tool position information; 10. The control device of claim 1. (Appendix 3) the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and the target mass are input, the parameters are operational information of the machine; The control unit acquiring operation information of the machine by inputting the force information and the target mass into the trained model for operation estimation; Controlling the operation of the machine based on the operation information of the machine. 10. The control device of claim 1. (Appendix 4) the trained model is a trained model for motion estimation that outputs motion information of the machine when stiffness information representing the stiffness of an object and the target mass are input; the parameters are operational information of the machine; The control unit Estimating stiffness information of the object based on the force information; acquiring operation information of the machine by inputting the stiffness information and the target mass into the trained model for operation estimation; Controlling the operation of the machine based on the operation information of the machine. 10. The control device of claim 1. (Appendix 5) the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and the position information of the tool are input, the parameters are operational information of the machine; The control unit acquiring operation information of the machine by inputting the force information and position information of the tool up to the current time into the trained model for operation estimation; Controlling the operation of the machine based on the operation information of the machine. 10. The control device of claim 1. (Appendix 6) The object is a granular object having viscosity and plasticity. The control device according to any one of Supplementary notes 1 to 5. (Appendix 7) When a machine performs an operation of scooping up an object, force information representing a force generated in a tool provided in the machine for scooping up the object and a target mass of the object to be scooped up are acquired; The force information is input to a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, thereby acquiring the parameters, and the operation of the machine is controlled based on the parameters so that the mass of the object scooped up becomes a target mass. A control method for computer-implemented processing. (Appendix 8) When a machine performs an operation of scooping up an object, force information representing a force generated in a tool provided in the machine for scooping up the object and a target mass of the object to be scooped up are acquired; The force information is input to a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, thereby acquiring the parameters, and the operation of the machine is controlled based on the parameters so that the mass of the object scooped up becomes a target mass. A control program that causes a computer to execute a process. [Explanation of symbols]

[0073] 10. Control System 12. Robot 12A Grip 13A, 13B Force and tactile sensors 14 Control device 16 Acquisition Department 18 Data storage unit 20 Trained model memory 22 Control Unit

Claims

1. an acquisition unit that acquires, when a machine performs an operation of scooping up an object, force information representing a force generated in a tool that is provided in the machine and that scoops up the object, and a target mass of the object to be scooped up; a control unit that acquires parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, and controls the operation of the machine based on the parameters so that the mass of the object scooped up becomes a target mass; A control device including:

2. the trained model is a trained model for mass estimation that outputs an estimated value of the mass of the object scooped up by the tool when the force information and position information of the tool are input; the parameter is an estimate of the mass of the object; The control unit obtaining an estimated value of the mass for each of the plurality of candidate position information by inputting the acquired force information and the candidate position information of the tool into the trained model for mass estimation; Identifying a candidate for position information of the tool corresponding to the estimated value that has the smallest difference from the target mass among the plurality of estimated values ​​of the mass; controlling the operation of the machine so as to move the tool to the position represented by the identified candidate for tool position information; The control device according to claim 1 .

3. the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and the target mass are input, the parameters are operational information of the machine; The control unit acquiring operation information of the machine by inputting the force information and the target mass into the trained model for operation estimation; Controlling the operation of the machine based on the operation information of the machine. The control device according to claim 1 .

4. the trained model is a trained model for motion estimation that outputs motion information of the machine when stiffness information representing the stiffness of an object and the target mass are input; the parameters are operational information of the machine; The control unit Estimating stiffness information of the object based on the force information; acquiring operation information of the machine by inputting the stiffness information and the target mass into the trained model for operation estimation; Controlling the operation of the machine based on the operation information of the machine. The control device according to claim 1 .

5. the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and the position information of the tool are input, the parameters are operational information of the machine; The control unit acquiring operation information of the machine by inputting the force information and the position information of the tool into the trained model for operation estimation; Controlling the operation of the machine based on the operation information of the machine. The control device according to claim 1 .

6. The object is a granular object having viscosity and plasticity. The control device according to claim 1 or 2.

7. When a machine performs an operation of scooping up an object, force information representing a force generated in a tool provided in the machine for scooping up the object and a target mass of the object to be scooped up are acquired; The force information is input to a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, thereby acquiring the parameters, and the operation of the machine is controlled based on the parameters so that the mass of the object scooped up becomes a target mass. A control method for computer-implemented processing.

8. When a machine performs an operation of scooping up an object, force information representing a force generated in a tool provided in the machine for scooping up the object and a target mass of the object to be scooped up are acquired; The force information is input to a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, thereby acquiring the parameters, and the operation of the machine is controlled based on the parameters so that the mass of the object scooped up becomes a target mass. A control program that causes a computer to execute a process.

Images