Control device, control method, and control program

Abstract

When a robot, which is an example of a machine, performs an operation of scooping up an object T, this control device acquires force information representing a force generated in an instrument provided in the robot. The control device controls the operation of the robot so that the mass of the object to be scooped up becomes a target mass in accordance with the acquired force information.

Description

Control device, control method, and control program

[0001] The present disclosure relates to a control device, a control method, and a control program.

[0002] Conventionally, there is a known technique for controlling the movement of a robot scooping up granular objects (see, for example, Reference 1 (C. Schenck, J. Tompson, S. Levine, and D. Fox, “Learning robotic manipulation of granular media,” in Conference on Robot Learning, 2017, pp. 239-248.)). This technique controls the movement of the robot by inputting a height map obtained from an image containing the granular object into a convolutional neural network.

[0003] There is also a known technology for controlling the behavior of an excavator when scooping soil (see, for example, Reference 2 (RJ Sandzimier and HH Asada, “A data-driven approach to prediction and optimal bucket-filling control for autonomous excavators,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2682-2689, 2020.)). This technology controls the switching from the bucket drag phase to the scoop phase at the correct timing when the excavator drags the bucket along the ground surface and rotates the bucket to scoop up and collect the soil. This technology uses soil images taken by a depth camera as input data.

[0004] When a machine performs an action such as scooping up an object, a force is generated at a specific location on the machine. Information about the force acting on the machine is useful, and it is thought that by utilizing this force information, it may be possible to control the operation of the machine to scoop up an object.

[0005] The techniques of the above-mentioned Documents 1 and 2 control the operation of a machine that scoops up an object by using an image of the object to be scooped up, but do not use the force generated by the machine. In particular, since the force required to scoop up varies depending on the properties (e.g., viscosity) of the object to be scooped up, problems may arise when controlling a machine using visual information (e.g., an image). This is because the hardness or viscosity of an object can change depending on factors such as humidity, temperature, or external pressure (e.g., pressure generated when soil is compacted), and such information cannot be determined from an image alone.

[0006] The present disclosure has been made in consideration of the above points, and aims to scoop up an object by utilizing the force generated in a machine when the machine performs an action such as scooping up an object.

[0007] In order to achieve the above object, the control device according to the present disclosure is a control device that includes: an acquisition unit that acquires force information representing the force generated in a tool that is equipped on the machine to scoop up an object and a target mass of the object to be scooped up when the machine performs an operation to scoop up an object; and a control unit that acquires the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object to be scooped up by the operation of the machine when the force information is input; and controls the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass.

[0008] Furthermore, the control method disclosed herein is a control method in which, when a machine performs an operation of scooping up an object, force information representing the force generated in a tool provided on the machine for scooping up the object and a target mass of the object to be scooped up are acquired, the force information is input to a trained model that, when the force information is input, outputs a parameter for adjusting the mass of the object scooped up by the operation of the machine to acquire the parameter, and a computer executes a process to control the operation of the machine based on the parameter so that the mass of the object to be scooped up becomes the target mass.

[0009] In addition, the control program of the present disclosure is a control program that, when a machine performs an operation of scooping up an object, acquires force information representing the force generated in a tool that the machine is equipped with to scoop up the object and a target mass of the object to be scooped up, acquires the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, and causes a computer to execute a process of controlling the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass.

[0010] According to the control device, control method, and control program of the present disclosure, when a machine performs an action such as scooping up an object, the force generated in the machine can be used to scoop up the object.

[0011] FIG. 1 is a diagram for explaining a control system of this embodiment. FIG. 2 is a diagram for explaining an overview of this embodiment. FIG. 3 is a diagram for explaining a trained model for mass estimation of this embodiment. FIG. 4 is a block diagram showing a schematic configuration of a control system of this embodiment. FIG. 5 is a block diagram showing a hardware configuration of a control device according to this embodiment. FIG. 6 is a flowchart showing the flow of control processing in this embodiment. FIG. 7 is a diagram showing the results of this embodiment. FIG. 8 is a diagram showing another example of a trained model of this embodiment.

[0012] An example of an embodiment of the present disclosure will be described below with reference to the drawings. In this embodiment, a control system equipped with a control device according to the present disclosure will be described as an example. Note that the same reference numerals are used in the drawings to designate identical or equivalent components and parts. Furthermore, the dimensions and proportions of the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0013] FIG. 1 is a diagram illustrating this embodiment. As shown in FIG. 1, a robot 12 according to this embodiment performs an action of scooping up an object T having viscosity and plasticity. The object T is, for example, an object having viscosity and plasticity, such as soil or ice cream. Specifically, as shown in FIG. 1, a spoon SP, which is an example of a tool for scooping up the object T, is attached to a gripper 12A of the robot 12, and the robot 12 moves the spoon SP to scoop up the object T. The robot 12 uses the spoon SP to scoop up the object T so that the mass of the scooped object becomes a target mass.

[0014] As shown in FIG. 1 , force-tactile sensors 13A and 13B are attached to the gripper 12A of the robot 12, and are configured to detect the force generated in the gripper 12A of the robot 12 when the spoon SP is moved. The force-tactile sensors 13A and 13B used in this embodiment are capable of detecting three-dimensional forces acting on the surface of the gripper 12A, and such force-tactile sensors 13A and 13B are also referred to as tactile sensors. In this embodiment, the force detected by the force-tactile sensors 13A and 13B and a pre-trained machine learning model for mass estimation are used to estimate the mass of an object scooped up by the spoon SP. When the pre-trained model for mass estimation receives the position information of the spoon SP and the force information detected by the force-tactile sensors 13A and 13B as input, it outputs an estimate of the mass of the object scooped up by the spoon SP. In this embodiment, the way to move the spoon SP is determined based on the estimated mass of the object obtained in this manner. The force-tactile sensor may be of any type. For example, the force-tactile sensor may be of an image type, a magnetic type, or a strain gauge type, and any of these types may be employed. The force-tactile sensor may be of a single-point measurement type or a distribution type, and any of these types may be employed. The force-tactile sensor may also have any number of measurement point cells.

[0015] More specifically, in this embodiment, as shown by X1 and X2 in Fig. 1 , the robot 12 moves the spoon SP in various ways to scoop up an object, and the mass of the object is used as training data to generate a trained model for mass estimation. Then, as shown by Y1, Y2, and Y3 in Fig. 1 , the robot 12 is controlled to move the spoon SP in accordance with the target mass of the object to be scooped up, so that the mass of the scooped up object becomes the target mass. This will be described in detail below.

[0016] <Overview> FIG. 2 is a diagram illustrating an overview of this embodiment. In this embodiment, as shown in FIG. 2, the gripper 12A of the robot 12 to which the spoon SP is attached is moved to scoop up the object T. In FIG. 2, p1 represents the position of the spoon SP (or the position of the gripper 12A) at the start of the movement, p2 represents the position of the spoon SP (or the position of the gripper 12A) while the spoon SP is being inserted into the object T, p3 represents the position of the spoon SP (or the position of the gripper 12A) when the spoon SP is deeply inserted into the object T, and p4 represents the position of the spoon SP (or the position of the gripper 12A) corresponding to the goal. In this embodiment, the position p3 of the spoon SP is optimized based on force information obtained by the force-tactile sensors 13A and 13B from the time when the spoon SP is at position p1 to the time when the spoon SP is at position p2 (force information corresponding to time interval C in FIG. 2). Note that this embodiment will be described taking as an example a case where the spoon SP is moved in the xz plane as shown in FIG. 2. It is also possible to move the spoon SP in the y direction.

[0017] In this embodiment, an estimated value w^ of the mass of the object T present on the spoon SP is calculated using a trained model for mass estimation, which will be described later. Then, the position p3 of the spoon SP is calculated so that the following equation is satisfied. ＊ In addition, w in the following formula (1) is determined. ｔａｒｇｅｔ is the target mass of the object T to be scooped up.

[0018]

[0019] <Trained Model for Mass Estimation> Fig. 3 is a diagram for explaining the trained model for mass estimation of this embodiment. As shown in Fig. 3, the trained model for mass estimation of this embodiment is configured to include a model M1 for state estimation and a model M2 for estimating the mass of an object.

[0020] The state estimation model M1 includes a convolutional neural network (CNN) 30 and a transformer 31. The convolutional neural network 30 and the transformer 31 are known machine learning models.

[0021] Furthermore, model M2 for estimating the mass of an object is composed of a multi-layer perceptron (MLP) 32. As shown in Fig. 3, when time-series data S1 and S2 of force information detected by force tactile sensors 13A and 13B are input to convolutional neural network 30, convolutional neural network 30 outputs feature vectors v1, v2, ..., vn.

[0022] S1 is time series data of force information detected by the force-tactile sensor 13A, and S2 is time series data of force information detected by the force-tactile sensor 13B. ｔｐ１ is the force information detected by the force tactile sensors 13A and 13B when the spoon SP is at the position p1, and S ｔｐ２ is the force information detected by the force-tactile sensors 13A and 13B when the position of the spoon SP is p2. Therefore, S1 and S2 are time-series data of the force information detected by the force-tactile sensors 13A and 13B during the time interval when the position of the spoon SP is from p1 to p2.

[0023] As shown in FIG. 3 , the data output from the convolutional neural network 30 is subjected to predetermined processing using positional encoding E, a known technique, to generate feature vectors v1, v2, ..., vn. Note that A in FIG. 3 is a symbol known in the field of machine learning and represents the direct sum of vectors. Each of the feature vectors v1, v2, ..., vn corresponds to force information at a given time. For example, feature vector v1 is a feature vector corresponding to force information detected by the force-tactile sensors 13A and 13B when the spoon SP is positioned at p1. Also, for example, feature vector vn is a feature vector corresponding to force information detected by the force-tactile sensors 13A and 13B when the spoon SP is positioned at p2.

[0024] 3, each of the feature vectors v1, v2, ..., vn is input to a transformer 31, which outputs each of the feature vectors v1', v2', ..., vn'. Then, as shown in FIG. 3, a mean vector v of the feature vectors v1', v2', ..., vn' is calculated.

[0025] This mean vector v is data that reflects the time-series data of force information obtained by the force-tactile sensors 13A, 13B, and by using this data, it is possible to estimate how much of the object T can be scooped up. When the mean vector v and candidate position p3 of the spoon SP are input to the multilayer perceptron 32, an estimated value w^ of the mass of the object T when the spoon SP is moved to position p3 and the object T is scooped up is output.

[0026] 3, the convolutional neural network 30 is a model capable of appropriately processing three-dimensional force information generated on a two-dimensional plane of the gripper 12A of the robot 12. Furthermore, the transformer 31 is a model capable of appropriately processing time-series information. Therefore, in this embodiment, by utilizing these two models, it becomes possible to accurately estimate the estimated value w^ of the mass of the object T when the object T is scooped up, based on the time-series data of force information obtained by the force-tactile sensors 13A and 13B attached to the gripper 12A.

[0027] Therefore, the estimated value w^ of the mass of the object T scooped up by the spoon SP can be expressed by the following equation: where F represents the trained model for mass estimation, and S ｔｐ１：ｔｐ２ represents time-series data of force information detected by the force-tactile sensors 13A and 13B during the time period when the position of the spoon SP is from p1 to p2.

[0028]

[0029] <Optimization of spoon SP position p3> As described above, once the estimated value w^ of the mass of the object T scooped up by the spoon SP is calculated, the optimal spoon SP position p3 can be determined by the above formula (1). ＊ In this embodiment, the optimum spoon SP position p3 is calculated using a grid search technique. ＊ Specifically, a plurality of candidate positions p3 for the spoon SP are set, and each of the plurality of candidate positions p3 is input into the trained model for mass estimation to calculate the estimated value w^ of the mass of the object T to be scooped up corresponding to each of the plurality of candidate positions p3. Then, among the estimated values ​​w^ of the mass corresponding to each of the plurality of candidate positions p3, the target mass w ｔａｒｇｅｔ The estimated mass value w^ having the smallest difference between the estimated mass value w^ and the input data p3 is determined as the optimal position p3. ＊ Identify as:

[0030] (Control System 10) FIG. 4 is a block diagram showing a schematic configuration of the control system 10 of this embodiment. As shown in FIG. 4, the control system 10 includes force-tactile sensors 13A and 13B, a robot 12, and a control device 14. The control device 14 acquires force information representing a force generated in a gripper 12A, which is an example of a part of the robot 12, when the robot 12, which is an example of a machine, performs an operation to scoop up an object T. The control device 14 then controls the operation of the robot 12 in accordance with the acquired force information so that the mass of the object T to be scooped up becomes a target mass. Specifically, the control device 14 acquires parameters by inputting the acquired force information into a trained model that, in response to input force information, outputs parameters for adjusting the mass of the object T to be scooped up by the operation of the robot 12. The control device 14 then controls the operation of the robot 12 in accordance with the acquired parameters so that the mass of the object T to be scooped up becomes a target mass. In this embodiment, the trained model is a trained model for mass estimation that outputs an estimated value of the mass of the object T scooped up by the spoon SP when force information and position information of the spoon SP are input. Also, the parameter is an estimated value of the mass of the object T.

[0031] The force-tactile sensors 13A and 13B are attached to the gripper 12A of the robot 12 and sequentially detect three-axis forces, which are force information. The force-tactile sensors 13A and 13B then output the obtained force information to the control device 14.

[0032] Fig. 5 is a block diagram showing the hardware configuration of the control device 14 according to this embodiment. As shown in Fig. 5, the control device 14 includes a CPU (Central Processing Unit) 42, a memory 44, a storage device 46, an input / output I / F (Interface) 48, a storage medium reader 50, and a communication I / F 52. Each component is connected to each other via a bus 54 so as to be able to communicate with each other.

[0033] The storage device 46 stores control programs for executing the various processes described below. The CPU 42 is a central processing unit that executes various programs and controls the various components. That is, the CPU 42 reads the programs from the storage device 46 and executes them using the memory 44 as a work area. The CPU 42 controls the various components and performs various arithmetic operations in accordance with the programs stored in the storage device 46.

[0034] The memory 44 is configured with a RAM (Random Access Memory) and serves as a working area to temporarily store programs and data. The storage device 46 is configured with a ROM (Read Only Memory), an HDD (Hard Disk Drive), an SSD (Solid State Drive), etc., and stores various programs including the operating system and various data.

[0035] The input / output I / F 48 is an interface for inputting data from an external device and outputting data to an external device. Input devices for various inputs, such as a keyboard or a mouse, and output devices for various information outputs, such as a display or a printer, may also be connected. A touch panel display may be used as the output device, allowing it to function as an input device.

[0036] The storage medium reader 50 reads data stored in various storage media such as CD (Compact Disc)-ROM, DVD (Digital Versatile Disc)-ROM, Blu-ray Disc, and USB (Universal Serial Bus) memory, and writes data to the storage media.

[0037] The communication I / F 52 is an interface for communicating with other devices, and uses standards such as Ethernet (registered trademark), FDDI, and Wi-Fi (registered trademark).

[0038] Next, the functional configuration of the control device 14 will be described. As shown in Fig. 4, the control device 14 functionally includes an acquisition unit 16 and a control unit 22. A data storage unit 18 and a trained model storage unit 20 are provided in a predetermined storage area of ​​the control device 14. Each functional configuration is realized by the CPU 42 reading each program stored in the storage device 46, expanding it in the memory 44, and executing it.

[0039] The data storage unit 18 stores time-series data of force information detected by the force-tactile sensors 13A and 13B. The data storage unit 18 also stores control data and the like when the robot 12 operates. The data storage unit 18 also stores the target mass w of the object T to be scooped up. ｔａｒｇｅｔ For example, the target mass is preset by the user.

[0040] The trained model storage unit 20 stores the trained model for mass estimation described above. The trained model for mass estimation is a trained model that outputs an estimated value w^ of the mass of the object T scooped up by the spoon SP when force information and position information of the spoon SP are input. Note that the trained model for mass estimation in this embodiment is generated in advance by known supervised learning or unsupervised learning, etc., based on training data collected in advance in learning phases X1 and X2 as shown in FIG. 1. Note that the training data is, for example, data that associates time-series data of force information obtained by the force-tactile sensors 13A and 13B with the mass of the object T scooped up by the spoon SP.

[0041] The acquisition unit 16 acquires time-series data of force information obtained by the force tactile sensors 13A and 13B from the data storage unit 18. The acquisition unit 16 also acquires a target mass w ｔａｒｇｅｔ is acquired from the data storage unit 18.

[0042] The control unit 22 reads out the trained model for mass estimation stored in the trained model storage unit 20. Then, the control unit 22 inputs the force information acquired by the acquisition unit 16 and multiple candidates for position information of the spoon SP that scoops up the object T, thereby acquiring an estimated mass value w^ for each of the multiple candidates for position information.

[0043] The control unit 22 determines the target mass w from among the plurality of estimated mass values ​​w^. ｔａｒｇｅｔ The candidate position information of the spoon SP corresponding to the estimated value that minimizes the difference between ＊ Then, the control unit 22 determines the position p3 indicated by the candidate for the position information of the identified spoon SP. ＊ The operation of the robot 12 is controlled so as to move the spoon SP.

[0044] Next, the operation of the control system 10 according to this embodiment will be described.

[0045] First, learning data is collected while the robot 12 is performing an operation of scooping up the object T, and is input to the control device 14. Then, the control unit 22 of the control device 14 generates a trained model for mass estimation based on the learning data. As a result, the trained model for mass estimation is stored in the trained model storage unit 20.

[0046] Next, the robot 12 performs an operation of actually scooping up the object T. Specifically, upon receiving a predetermined instruction signal, the control device 14 executes a control processing routine shown in FIG.

[0047] In step S100, the acquisition unit 16 acquires time-series data of force information representing the force generated in the gripping unit 12A of the robot 12 while the robot 12 is scooping up the object T using the spoon SP. Also in step S100, the acquisition unit 16 acquires time-series data of force information representing the force generated in the gripping unit 12A of the robot 12 while the robot 12 is scooping up the object T using the spoon SP. ｔａｒｇｅｔ Get.

[0048] In step S102, the control unit 22 inputs the time series data of force information obtained in step S100 and multiple candidates for position information of the spoon SP (or the gripping portion 12A) that scoops up the object T into the trained model for mass estimation stored in the trained model memory unit 20, and obtains an estimated value w^ of the mass of the object for each of the multiple candidates for position information.

[0049] In step S104, the control unit 22 selects the target mass w ｔａｒｇｅｔ The candidate position information of the spoon SP corresponding to the estimated value that minimizes the difference between ＊ Identify as:

[0050] In step S106, the control unit 22 determines the optimum position p3 of the spoon SP identified in step S104. ＊ A control command is output to the robot 12 so that the above equation is realized. As a result, an object T having a mass close to the target mass is picked up.

[0051] As described above, the control device according to this embodiment acquires force information representing forces generated in parts of a robot, which is an example of a machine, while the robot is performing an action to scoop up an object. The control device controls the robot's action based on the acquired force information so that the mass of the object to be scooped up becomes a target mass. This allows the robot to scoop up the object by utilizing the force generated in the robot when performing an action such as scooping up an object.

[0052] Specifically, the control device of this embodiment is a technology in which a robot 12 holding a spoon SP scoops up a viscoplastic object, such as wet sand or ice cream, at a certain target mass. The control device of this embodiment equips the robot 12 with a force-tactile sensor or tactile sensor, and inputs time-series information on the force exerted when the spoon SP penetrates the object T and the hand posture of the robot 12 to learn a model that predicts the amount of the object T scooped up. The learned model is then used to determine the hand posture required to scoop up the object T at a target mass. This enables weighing of a viscoplastic object at an arbitrary target mass. Furthermore, according to this embodiment, the use of the force-tactile sensor or tactile sensor allows the physical properties (e.g., hardness) of the object T to be grasped, enabling more accurate weighing.

[0053] Next, an example will be described. In this example, an experiment was conducted to verify the effectiveness of the proposed method. Specifically, in this example, a comparison was made between a case where the target object, viscoplastic sand (kinetic sand), was scooped up by controlling the robot's movement using the trained model F for mass estimation shown in FIG. 3 above, and a case where the target object, sand (kinetic sand), was scooped up using only model M2 shown in FIG. 3 above. FIG. 7 shows the results of this example. The horizontal axis of FIG. 7 represents the target mass of the sand to be scooped up, and the vertical axis represents the difference between the target mass and the mass of the sand actually scooped up.

[0054] 7, the difference between the target mass and the actual mass of the scooped sand tends to be smaller when using the proposed method of this embodiment (trained model F for mass estimation) compared to the conventional method (using only model M2). Therefore, it can be seen that by using the method of this embodiment, it is possible to accurately and efficiently generate a trained policy to be used when moving an object.

[0055] The technology of the present disclosure is not limited to the above-described embodiment, and various modifications and applications are possible within the scope of the gist of this disclosure.

[0056] For example, in the above embodiment, an example has been described in which the operation of the robot 12 is controlled to scoop up the object T, but the present invention is not limited to this. For example, the machine to be controlled does not have to be the robot 12, and any machine capable of scooping up the object T may be used. For example, by applying this embodiment to a construction machine such as a shovel, the machine may be configured to control an operation such as scooping up soil. If the machine to be controlled is a shovel, the tool for scooping up the object is the bucket of the shovel.

[0057] In the above embodiment, the utensil used to scoop up an object is a spoon, but the present invention is not limited to this. Any utensil that can scoop up an object may be used. For example, it may be a cup.

[0058] In the above embodiment, the force-tactile sensors 13A, 13B are tactile sensors capable of detecting forces in three dimensions and capable of detecting forces occurring at various locations on the two-dimensional plane of the grip portion 12A, but this is not limiting. For example, the force-tactile sensors 13A, 13B may be sensors capable of detecting forces in at least one three-dimensional direction and capable of detecting forces occurring at one location on the grip portion 12A. Even if such sensors are used, the control device of this embodiment can be realized.

[0059] In the above embodiment, the force-tactile sensors 13A, 13B are installed in the gripper 12A of the robot 12, but this is not limiting. The force-tactile sensors may be installed in any location as long as they can detect the force acting on the tool. Furthermore, instead of the force-tactile sensors, the force acting on the tool may be estimated using a torque sensor or motor current value mounted on the joints of the robot 12.

[0060] In the above embodiment, four points (p1, p2, p3, p4) are used as the position of the spoon SP, but this is not limiting. More or fewer points may be used.

[0061] In the above embodiment, for example, as shown in FIG. 2 , the force information from the start point of the robot 12's scooping up the object T (e.g., the point in time when the spoon SP is stuck in the object T) is used, but the present invention is not limited to this. For example, the robot 12 may control the operation of the robot 12 using force information obtained while the robot 12 is scooping up the object T (e.g., force information obtained in a time period after the time when the spoon SP is stuck in the object T). Furthermore, force information obtained when the robot 12 sticks the spoon SP in the object T may also be used.

[0062] In the above embodiment, the model shown in FIG. 8 is used as the trained model F for mass estimation, but the present invention is not limited to this. Any model may be used as the trained model for mass estimation. For example, the transformer 8 may be replaced with a known long short term memory (LSTM) model. Furthermore, the robot 12 may be controlled using a model other than the trained model for mass estimation.

[0063] For example, as shown in FIG. 8A , a trained model for motion estimation may be used that outputs motion information representing the motion of the robot 12 when the target mass and force information of the object T are input. In this case, the control unit 22 inputs the force information and the target mass into the trained model for motion estimation, which outputs motion information of the robot 12, which is an example of a machine, when the force information and the target mass are input, thereby acquiring motion information of the robot 12 and controlling the motion of the robot 12 based on the motion information. Note that the motion information may be, for example, trajectory information of the arm or gripper 12A of the robot 12. In this case, the parameters for adjusting the mass of the object output from the trained model correspond to the motion information.

[0064] Alternatively, for example, as shown in FIG. 8A , a trained model for motion estimation may be used that outputs motion information representing the motion of the robot 12 when a target mass of the object T and stiffness information of the object T are input. In this case, for example, the control unit 22 estimates stiffness information representing the stiffness of the object T based on force information when the spoon SP is inserted into the object T. The control unit 22 then inputs the stiffness information representing the stiffness of the object T and the target mass into a trained model for motion estimation that outputs motion information of the robot 12 when the stiffness information and the target mass are input, thereby acquiring motion information for the robot 12 and controlling the motion of the robot 12 based on the motion information. In this case, the parameters for adjusting the mass of the object output from the trained model correspond to motion information.

[0065] Alternatively, for example, as shown in Fig. 8(B), a trained model for motion estimation may be used that outputs motion information of the robot 12 when force information and position information of the spoon SP are input. In this case, for example, the control unit 22 inputs force information and position information of the spoon SP from the start time of the scooping motion to the current time into the trained model for motion estimation to obtain motion information of the robot 12, and controls the motion of the robot 12 based on the motion information. Note that in this case, the parameter for adjusting the mass of the object output from the trained model corresponds to motion information.

[0066] In the above embodiment, the object T is a granular object having viscosity and plasticity, but the present invention is not limited to this. Any object may be used as long as it is an object from which force information can be obtained.

[0067] In the above embodiment, the optimum position p3 is determined using a grid search technique. ＊ However, the present invention is not limited to this. For example, the optimum position p3 may be determined by using various methods, including those that utilize the gradient of the above formula (1) and those that do not utilize the gradient. ＊ It is also possible to specify

[0068] Furthermore, various processors other than the CPU may execute the processes executed by loading software (programs) in the above embodiments. Examples of processors in this case include programmable logic devices (PLDs) (such as field-programmable gate arrays (FPGAs)) whose circuit configuration can be changed after manufacture, and dedicated electrical circuits, such as application-specific integrated circuits (ASICs), which are processors having circuit configurations designed specifically to execute specific processes. Each process may be executed by one of these various processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor elements.

[0069] In the above embodiment, the programs are pre-stored (installed) in a storage device, but this is not limiting. The programs may be provided in a form stored in a storage medium such as a CD-ROM, DVD-ROM, Blu-ray Disc, or USB memory. The programs may also be downloaded from an external device via a network.

[0070] (Supplementary Notes) The following supplementary notes are provided regarding aspects of the present disclosure.

[0071] (Supplementary Note 1) A control device including: an acquisition unit that acquires force information representing a force generated in a tool that is equipped on a machine to scoop up an object and a target mass of the object to be scooped up when the machine performs an operation to scoop up the object; and a control unit that acquires the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object to be scooped up by the operation of the machine when the force information is input, and controls the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass. (Supplementary Note 2) The control device described in Supplementary Note 1, wherein the trained model is a trained model for mass estimation that, when the force information and position information of the tool are input, outputs an estimated value of the mass of the object to be scooped up by the tool, the parameter is an estimated value of the mass of the object, and the control unit: inputs the acquired force information and multiple candidates for position information of the tool to the trained model for mass estimation, thereby obtaining an estimated value of the mass for each of the multiple candidates for position information; identifies, from the multiple estimated values ​​of mass, the candidate for position information of the tool that corresponds to the estimated value that has the smallest difference from the target mass; and controls the operation of the machine to move the tool to the position indicated by the identified candidate for position information of the tool. (Supplementary Note 3) The control device described in Supplementary Note 1, wherein the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and the target mass are input, the parameters are the motion information of the machine, and the control unit acquires the motion information of the machine by inputting the force information and the target mass into the trained model for motion estimation, and controls the motion of the machine based on the motion information of the machine.(Supplementary Note 4) The control device according to Supplementary Note 1, wherein the trained model is a trained model for motion estimation that outputs motion information of the machine when stiffness information representing the stiffness of an object and the target mass are input, the parameters are the motion information of the machine, and the control unit estimates the stiffness information of the object based on the force information, acquires the motion information of the machine by inputting the stiffness information and the target mass into the trained model for motion estimation, and controls the motion of the machine based on the motion information of the machine. (Supplementary Note 5) The control device according to Supplementary Note 1, wherein the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and position information of the tool are input, the parameters are the motion information of the machine, and the control unit acquires the motion information of the machine by inputting the force information and position information of the tool up to the current time into the trained model for motion estimation, and controls the motion of the machine based on the motion information of the machine. (Supplementary Note 6) The control device according to any one of Supplementary Notes 1 to 5, wherein the object is a granular object having viscosity and plasticity. (Supplementary Note 7) A control method in which a computer executes processes to: when a machine performs an operation to scoop up an object, acquire force information representing a force generated in a tool provided on the machine that scoops up the object and a target mass of the object to be scooped up, acquire the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, and control the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass.(Supplementary Note 8) A control program for causing a computer to execute a process of: when a machine performs an operation to scoop up an object, acquiring force information representing the force generated in a tool provided on the machine to scoop up the object, and a target mass of the object to be scooped up; acquiring the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input; and controlling the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass.

[0072] The disclosure of Japanese Patent Application No. 2024-089439, filed on May 31, 2024, is incorporated herein by reference in its entirety. All documents, patent applications, and technical standards mentioned herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A control device including: an acquisition unit that acquires force information representing the force generated in a tool that is equipped on a machine to scoop up an object and a target mass of the object to be scooped up when the machine performs an operation to scoop up the object; and a control unit that acquires the parameters by inputting the force information into a trained model that, when the force information is input, outputs parameters for adjusting the mass of the object to be scooped up by the operation of the machine, and controls the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass.

2. The control device described in claim 1, wherein the trained model is a trained model for mass estimation that, when the force information and position information of the tool are input, outputs an estimated value of the mass of the object to be scooped up by the tool, the parameter is an estimated value of the mass of the object, and the control unit: inputs the acquired force information and multiple candidate position information of the tool to the trained model for mass estimation, thereby obtaining an estimated value of the mass for each of the multiple candidate position information; identifies, from the multiple estimated values ​​of mass, the candidate position information of the tool that corresponds to the estimated value that has the smallest difference from the target mass; and controls the operation of the machine so as to move the tool to the position indicated by the identified candidate position information of the tool.

3. The control device described in claim 1, wherein the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and the target mass are input, the parameters are the motion information of the machine, and the control unit obtains the motion information of the machine by inputting the force information and the target mass into the trained model for motion estimation, and controls the motion of the machine based on the motion information of the machine.

4. The control device described in claim 1, wherein the trained model is a trained model for motion estimation that outputs motion information of the machine when stiffness information representing the stiffness of an object and the target mass are input, the parameters are the motion information of the machine, and the control unit estimates the stiffness information of the object based on the force information, obtains the motion information of the machine by inputting the stiffness information and the target mass into the trained model for motion estimation, and controls the motion of the machine based on the motion information of the machine.

5. The control device described in claim 1, wherein the trained model is a trained model for motion estimation that outputs motion information of the machine when the force information and position information of the tool are input, the parameters are motion information of the machine, and the control unit obtains the motion information of the machine by inputting the force information and position information of the tool into the trained model for motion estimation, and controls the motion of the machine based on the motion information of the machine.

6. The control device according to claim 1 or 2, wherein the object is a granular object having viscosity and plasticity.

7. A control method in which, when a machine performs an operation to scoop up an object, force information representing the force generated in a tool provided on the machine to scoop up the object and a target mass of the object to be scooped up are acquired, the force information is input into a trained model that, when the force information is input, outputs parameters for adjusting the mass of the object scooped up by the operation of the machine, thereby acquiring the parameters, and the operation of the machine is controlled based on the parameters so that the mass of the object to be scooped up becomes the target mass.

8. A control program for causing a computer to execute a process of acquiring, when a machine performs an operation to scoop up an object, force information representing the force generated in a tool provided on the machine to scoop up the object and a target mass of the object to be scooped up, acquiring the parameters by inputting the force information into a trained model that outputs parameters for adjusting the mass of the object scooped up by the operation of the machine when the force information is input, and controlling the operation of the machine based on the parameters so that the mass of the object to be scooped up becomes the target mass.

Images