Autonomous control system, autonomous control method, and program

Abstract

To provide an autonomous control system, an autonomous control method, and a program which easily achieve a target task.SOLUTION: An autonomous control system includes an acquisition part for acquiring state data of a robot, visual data of the robot, and tactile data of the robot, and a processing part for determining action of the robot capable of achieving a task given to the robot, on the basis of the state data, the tactile data, and the tactile data, wherein the processing part fuses and dimensionally compresses the tactile date and the tactile data and thereby generates first compression data of a lower dimension than the data obtained by combining the tactile data and the tactile data, dimensionally compresses the tactile data and thereby generates second compression data of a lower dimension compared to the tactile data, and determines the action on the basis of coupled state data obtained by coupling the state data, the first compression data, and the second compression data into one.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an autonomous control system, an autonomous control method, and a program. [Background technology]

[0002] Research into autonomous control of robots using machine learning is currently underway. In this context, a technique for efficiently learning neural networks is known (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-185127 Summary of the Invention [Problem to be solved by the invention]

[0004] In the past, when controlling the robot's movements to accomplish a target task while capturing images of the robot's hands with a camera, if the robot changed, moved, or manipulated an object in its hands, the object could be covered by the robot's hand, resulting in occlusion. This made it difficult to estimate the object's posture and determine the robot's behavior based on that estimation using visual information alone.

[0005] To solve this problem, research is underway to combine the robot's vision and touch using machine learning, but this has not been sufficient to accomplish the desired task.

[0006] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide an autonomous control system, an autonomous control method, and a program that make it easier to accomplish a target task. [Means for solving the problem]

[0007] The autonomous control system, the autonomous control method, and the program according to the present invention employ the following configurations. (1) A first aspect of the present invention is an autonomous control system comprising: an acquisition unit that acquires status data of a robot, visual data of the robot, and tactile data of the robot; and a processing unit that determines an action of the robot that can accomplish a task given to the robot based on the status data, the visual data, and the tactile data, wherein the processing unit combines and dimensionally compresses the visual data and the tactile data to generate first compressed data that is lower in dimension than the combined data of the visual data and the tactile data, and dimensionally compresses the tactile data to generate second compressed data that is lower in dimension than the tactile data, and determines the action based on combined status data that combines the status data, the first compressed data, and the second compressed data.

[0008] (2) A second aspect of the present invention is that in the first aspect, the acquisition unit acquires depth image data generated by a camera that captures images of the robot's body and the task target as the visual data, and acquires data that associates the contact force detected by each tactile sensor with the distribution of multiple tactile sensors arranged on the body as the tactile data, and the processing unit generates the first compressed data by fusing and dimensionally compressing the distribution of the multiple tactile sensors and the depth image data.

[0009] (3) A third aspect of the present invention is that in the second aspect, the processing unit generates the second compressed data by dimensionally compressing data in which the contact force detected by each tactile sensor is associated with the distribution of the multiple tactile sensors.

[0010] (4) A fourth aspect of the present invention is any one of the first to third aspects, wherein the processing unit generates the first compressed data from the visual data and the tactile data using a first encoder, and the first encoder is a neural network trained based on a first training data set in which the correct states of the target of the task are labeled for the visual data and the tactile data.

[0011] (5) A fifth aspect of the present invention is any one of the first to fourth aspects, wherein the processing unit generates the first compressed data from the visual data and the tactile data using a first encoder, and the first encoder is a neural network that converts input data into low-dimensional data and outputs the data, and is combined with a decoder that converts input data into high-dimensional data and outputs the data, and is trained to ensure that the data input to the first encoder matches the data output by the decoder.

[0012] (6) A sixth aspect of the present invention is any one of the first to fourth aspects, wherein the processing unit generates the second compressed data from the haptic data using a second encoder, and the second encoder is a neural network that converts input data into low-dimensional data and outputs the data, and is combined with a decoder that converts input data into high-dimensional data and outputs the data, and is trained so that the data input to the second encoder matches the data output by the decoder.

[0013] (7) A seventh aspect of the present invention is any one of the first to fifth aspects, wherein the processing unit determines the action from the connection state data using reinforcement learning.

[0014] (8) An eighth aspect of the present invention is the second or third aspect, wherein the processing unit further determines the sensitivity of the tactile sensor based on the connection state data.

[0015] (9) A ninth aspect of the present invention is that, in the second or third aspect, the processing unit further determines the angle of the camera when imaging the body and the target based on the connection state data.

[0016] (10) A tenth aspect of the present invention is an autonomous control method that acquires status data of a robot, visual data of the robot, and tactile data of the robot, determines an action of the robot that can accomplish a task given to the robot based on the status data, the visual data, and the tactile data, fuses and dimensionally compresses the visual data and the tactile data to generate first compressed data that is lower in dimension than the combined data of the visual data and the tactile data, compresses the dimensionality of the tactile data to generate second compressed data that is lower in dimension than the tactile data, and determines the action based on combined status data that combines the status data, the first compressed data, and the second compressed data.

[0017] (11) An eleventh aspect of the present invention is a program for causing a computer to execute the following steps: acquiring status data of a robot, visual data of the robot, and tactile data of the robot; determining an action of the robot that will enable the robot to accomplish a task given to the robot based on the status data, the visual data, and the tactile data; fusing and dimensionally compressing the visual data and the tactile data to generate first compressed data that is lower in dimension than the combined data of the visual data and the tactile data; compressing the dimensionality of the tactile data to generate second compressed data that is lower in dimension than the tactile data; and determining the action based on combined status data that combines the status data, the first compressed data, and the second compressed data into one. [Effects of the Invention]

[0018] According to the above aspect, the target task becomes easier to accomplish. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a diagram illustrating an example of the configuration of an autonomous control system 1 according to a first embodiment. [Figure 2] FIG. 1 is a diagram schematically illustrating the appearance of a robot 10 according to a first embodiment. [Figure 3] 1 is a configuration diagram of a robot 10 and an autonomous control device 100 according to a first embodiment. [Figure 4] 10 is a flowchart showing the flow of a series of processes by a processing unit 120 according to the first embodiment. [Figure 5] FIG. 2 is a diagram schematically illustrating a flow of a series of processes performed by a processing unit 120 according to the first embodiment. [Figure 6] FIG. 1 is a configuration diagram of a learning device 200 according to a first embodiment. [Figure 7] FIG. 10 is a diagram for explaining a learning method for the first encoder MDL1. [Figure 8] FIG. 10 is a diagram for explaining a learning method of the second encoder MDL2. [Figure 9] FIG. 10 is a diagram for explaining another learning method of the first encoder MDL1. [Figure 10] 10 is a flowchart showing the flow of a series of processes by a processing unit 120 according to the second embodiment. [Figure 11] FIG. 10 is a diagram schematically illustrating a flow of a series of processes performed by a processing unit 120 according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, an embodiment of an autonomous control system, an autonomous control method, and a program according to the present invention will be described with reference to the drawings.

[0021] First Embodiment [System Configuration] 1 is a diagram illustrating an example of the configuration of an autonomous control system 1 according to the first embodiment. The autonomous control system 1 includes, for example, an autonomous control device 100 and a learning device 200. The autonomous control device 100 and the learning device 200 are connected via a network NW. The network NW includes a LAN (Local Area Network), a WAN (Wide Area Network), etc.

[0022] The autonomous control device 100 is a device that performs autonomous control so that the robot 10 behaves autonomously. The autonomous control device 100 is typically mounted on the robot 10 and directly controls the robot 10. Alternatively, the autonomous control device 100 may be installed in a remote location far away from the robot 10 and control the robot 10 remotely via a network NW. For example, the autonomous control device 100 uses a machine learning model to determine the optimal behavior that the robot 10 should take.

[0023] The learning device 200 is a device that learns a machine learning model used by the autonomous control device 100. The learning device 200 may be a single device, or may be a system in which multiple devices connected via a network NW operate in cooperation with each other. In other words, the learning device 200 may be realized by multiple computers (processors) included in a distributed computing system or a cloud computing system.

[0024] [Robot appearance] 2 is a diagram schematically illustrating the appearance of the robot 10 according to the first embodiment. The robot 10 is typically a humanoid robot having two arms, but is not limited to this and may be a quadrupedal animal-like robot, an industrial robot, a military robot, a household cleaning robot, or any other type of robot capable of autonomous action.

[0025] The robot 10 is equipped with, for example, a visual sensor 11 for imaging the external environment as seen by the robot 10, and multiple tactile sensors 12 for reproducing the robot's sense of touch, and performs the desired task by utilizing these sensors in accordance with the actions determined by the autonomous control device 100.

[0026] For example, if a target TR is a plastic bottle, a task is to grab the plastic bottle with one hand, transfer the plastic bottle to the other hand, move the plastic bottle, or remove or put on the cap of the plastic bottle. Note that the tasks are not limited to these, and any task can be set.

[0027] The visual sensor 11 is installed on a part of the body of the robot 10 (typically, on the head). The visual sensor 11 may be, for example, a depth camera. The depth camera captures an image of the front of the robot 10 and generates a color, three-dimensional image (i.e., a six-dimensional image of width (W), height (H), red (R), green (G), blue (B), and depth (D)). Note that the visual sensor 11 is not limited to a depth camera and may be, for example, a sensor that generates an image of the external environment by emitting electromagnetic waves, such as radar or lidar. For convenience, the following description will be given assuming that the visual sensor 11 is a depth camera. If a surveillance camera 20 is present in the workspace of the robot 10 and can be used in place of a depth camera, the visual sensor 11 may be omitted.

[0028] The multiple tactile sensors 12 are, for example, distributed over a portion of the body of the robot 10 (typically the fingers or palm). Specifically, the tactile sensors 12 may be distributed over 10 regions of the fingers or palm. For example, tactile sensors 12-1 and 12-2 capable of sensing contact force at 32 points are disposed on the thumb. Contact force is, for example, a physical force such as pressure, stress, or strain. Tactile sensors 12-3 and 12-4 capable of sensing contact force at 32 points are disposed on the palm. Tactile sensor 12-5 capable of sensing contact force at 24 points and tactile sensor 12-6 capable of sensing contact force at 8 points are disposed on the remaining three fingers excluding the thumb. The multiple tactile sensors 12 distributed in this manner sense the force applied to the fingers or palm when grasping a target TR through a total of 224 channels. The number of channels is not limited to 224 and may be, for example, several tens to several hundreds.

[0029] [Configuration of robot and autonomous control device] 3 is a configuration diagram of the robot 10 and the autonomous control device 100 according to the first embodiment. In addition to the visual sensor 11 and tactile sensor 12 described above, the robot 10 further includes an actuator 13, a state sensor 14, and a drive control unit 15.

[0030] The actuators 13 drive the various parts (arms, fingers, legs, head, torso, waist, etc.) of the robot 10 under the control of the drive control unit 15. The actuators 13 include, for example, electromagnetic motors, gears, artificial muscles, etc.

[0031] The status sensor 14 is a sensor that detects the status (for example, joint angle, angular velocity, torque, etc.) of the robot 10. The status sensor 14 includes, for example, a rotary encoder that detects the degree of rotation of the joint of the robot 10, a tension sensor that detects the tension of the wire that rotates the joint, a torque sensor that detects the torque applied to the joint axis, an acceleration sensor and a gyro sensor that detect the posture of the robot 10, etc.

[0032] The drive control unit 15 controls the actuator 13 based on the control command generated by the autonomous control device 100 .

[0033] The autonomous control device 100 includes, for example, a communication interface 110, a processing unit 120, and a storage unit 130.

[0034] The communication interface 110 communicates with the learning device 200 via a network NW, and with the robot 10 via a communication line such as a bus. The communication interface 110 includes, for example, a wireless communication module including a receiver and a transmitter, and a network interface card (NIC).

[0035] The processing unit 120 includes, for example, an acquisition unit 121, a data compression unit 122, a behavior determination unit 123, a command generation unit 124, and a communication control unit 125.

[0036] The components of the processing unit 120 are realized by, for example, a central processing unit (CPU) or a graphics processing unit (GPU) executing a program stored in the storage unit 130. Some or all of these components may be realized by hardware such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), or a field-programmable gate array (FPGA), or may be realized by a combination of software and hardware.

[0037] The storage unit 130 is realized by, for example, a hard disk drive (HDD), flash memory, an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), or a random access memory (RAM). The storage unit 130 stores model data in addition to various programs such as firmware and application programs. The model data is data (programs or algorithms) that define several machine learning models for determining the behavior of the robot 10. The model data may be installed in the storage unit 130 from the learning device 200 via the network NW, or may be installed in the storage unit 130 from a portable storage medium connected to a drive device of the autonomous control device 100, for example.

[0038] [Autonomous control device processing flow] Below, a description will be given using a flowchart of each component of the processing unit 120. Fig. 4 is a flowchart showing the flow of a series of processes by the processing unit 120 according to the first embodiment.

[0039] First, the acquisition unit 121 acquires state data, depth image data, and tactile data from the robot 10 via the communication interface 110 (step S100).

[0040] The state data is a multidimensional vector whose elements include, for example, the detection values ​​of the state sensors 14 of the robot 10. Hereinafter, the state data vector will be specifically referred to as a "state vector." The state vector includes, as element values, for example, the joint angle detected by a rotary encoder, the tension of the joint wire detected by a tension sensor, the torque of the joint shaft detected by a torque sensor, the acceleration of the robot 10 detected by an acceleration sensor, and the angular velocity of the robot 10 detected by a gyro sensor.

[0041] The depth image data is, for example, a vector of a color, three-dimensional image (six-dimensional image) obtained by the visual sensor 11 of the robot 10. Hereinafter, the vector of the depth image data will be specifically referred to as an "image vector." For example, four-dimensional information in which depth (D) and RGB values ​​are associated with each pixel of an image represented by width (W) and height (H) is projected onto the world coordinate system of XYZ, and the six-dimensional vector represented by RGB-XYZ after this projection becomes the image vector. Note that if the image obtained by the visual sensor 11 is black and white, the image vector will be a four-dimensional vector in which pixel values ​​of one channel are associated with XYZ.

[0042] For example, suppose that a depth camera captures an image of the robot 10 grasping a target TR with its hand to accomplish a task. In this case, the depth image data includes one or both of (i) distance information and color information from the depth camera mounted on the robot 10 to the hand of the robot 10, and (ii) distance information and color information from the depth camera mounted on the robot 10 to the target TR.

[0043] If a surveillance camera 20 is present in the workspace of the robot 10, the acquisition unit 121 may acquire depth image data from the surveillance camera 20 in addition to or instead of acquiring depth image data from the robot 10. In this case, the depth image data will include one or both of (iii) distance information and color information from the surveillance camera 20 to the hand of the robot 10, and (iv) distance information and color information from the surveillance camera 20 to the target TR.

[0044] The tactile data is, for example, a multidimensional vector in which the contact force detected by each tactile sensor 12 is associated with the distribution (contact point group) of multiple tactile sensors 12 arranged on a portion of the body of the robot 10. Hereinafter, the vector of tactile data will be specifically referred to as a "tactile vector." For example, in the example of Figure 2, the tactile vector is a 224-dimensional vector, but as mentioned above, it may be a vector with tens to hundreds of dimensions.

[0045] Next, the data compression unit 122 combines and dimensionally compresses the depth image data (image vector) and tactile data (tactile vector) from the state data (state vector), depth image data (image vector), and tactile data (tactile vector) acquired by the acquisition unit 121, thereby generating lower-dimensional data (hereinafter referred to as first compressed data) compared to the combination of the depth image data (image vector) and the tactile data (tactile vector) (step S102).

[0046] 5 is a diagram schematically illustrating a series of processing steps performed by the processing unit 120 according to the first embodiment. In the diagram, MDL1 represents a first encoder (autoencoder) trained in advance to compress the dimension of input data, and MDL2 represents a second encoder (autoencoder) trained in advance to compress the dimension of input data, similar to the first encoder MDL1. MDL3 represents a policy network trained in advance to determine the behavior of the robot 10 from state data.

[0047] The first encoder MDL1 and / or the second encoder MDL2 may be implemented, for example, by a neural network including a convolutional layer.

[0048] The policy network MDL3 is a network that uses deep reinforcement learning. There are several types of deep reinforcement learning known, such as value-based, policy-based, actor-critic that combines values ​​and policies, and predictive model-based. Actor-critic includes, for example, twin delayed DDPG (TD3) and soft actor-critic (SAC). In this embodiment, for example, the policy-based, actor-critic, and predictive model-based can be applied.

[0049] These various models are defined by model data stored in the storage unit 130. The model data includes various information, such as connection information on how units included in each of the multiple layers constituting the neural network are connected to each other, and connection coefficients assigned to data input / output between the connected units. The connection information includes, for example, the number of units included in each layer, information specifying the type of unit to which each unit is connected, activation functions that realize each unit, and gates provided between units in the hidden layer. The activation functions that realize the units may be, for example, rectified linear functions (ReLU functions), sigmoid functions, step functions, or other functions. The gates selectively pass or weight data transmitted between units depending on, for example, the value (e.g., 1 or 0) returned by the activation function. The connection coefficients include, for example, weights assigned to output data when data is output from a unit in a layer to a unit in a deeper layer in the hidden layer of the neural network. The connection coefficients may also include bias components specific to each layer.

[0050] For example, the data compression unit 122 projects depth image data (W (image size width) × H (image size height) × D (depth)) onto XYZ world coordinates. Furthermore, the data compression unit 122 projects the contact points of the tactile sensor 12 (points among the point clouds for 224 channels that detect a contact force equal to or greater than a threshold) onto XYZ world coordinates. The data compression unit 122 then combines the point cloud derived from the depth image data projected onto the world coordinates with the contact points of the tactile sensor 12 into a single vector, and inputs this vector to the trained first encoder MDL1. In response to this, the trained first encoder MDL1 outputs state data (state vector) representing the state of the target TR, such as its position and orientation, as first compressed data.

[0051] In other words, the first encoder MDL1 is trained to supplement the determination of the position and posture of the target TR in the hand of the robot 10 from the contact point of the tactile sensor 12 using depth image data alone, even when the target TR is partially or entirely covered by the hand of the robot 10 or the like and the position and posture of the target TR cannot be determined from the viewpoint of the robot 10 (i.e., when occlusion occurs).

[0052] Returning to the explanation of the flowchart in Fig. 4, the data compression unit 122 next performs dimensional compression on the haptic data (haptic vector) among the state data (state vector), depth image data (image vector), and haptic data (haptic vector) acquired by the acquisition unit 121, thereby generating data (hereinafter referred to as second compressed data) of lower dimension than the haptic data (haptic vector) (step S104).

[0053] 5, for example, the data compression unit 122 inputs tactile data (tactile vectors) in which the contact forces detected by each tactile sensor 12 are associated with the distribution (contact point group) of the multiple tactile sensors 12 to the trained second encoder MDL2. In response to this, the trained second encoder MDL2 converts, for example, the 224-dimensional tactile vector into a tactile vector with several tens of dimensions, such as 10 or 20 dimensions, and outputs it as second compressed data.

[0054] Returning to the explanation of the flowchart in Fig. 4, the behavior determining unit 123 then generates data (hereinafter referred to as combined state data) that combines the state data (state vector), the first compressed data, and the second compressed data (step S106).

[0055] Next, the behavior decision unit 123 decides the behavior of the robot 10 from the connection state data using the policy network MDL3 (step S108).

[0056] As shown in FIG. 5, for example, the behavior decision unit 123 determines the environmental state s at a certain time t. t Observation results of tInstead of inputting the state data (state vector), depth image data (image vector), and tactile data (tactile vector) directly into the policy network MDL3, the first encoder MDL1 and the second encoder MDL2 are used to generate the observation results o t The binding state data (z in the figure) is compressed to low dimensions. t ) is input to the policy network MDL3. In response to this, the policy network MDL3 determines the environmental state s at a certain time t. t One or more actions (behavior variables) a that the robot 10 can take under t Among them, the action (action variable) a with the highest value (Q value) is t Output the action (action variable) a t The action a may be various actions such as grabbing the target TR with the hand, changing the grip, or moving the target TR. t is learned appropriately according to the task required of the robot 10.

[0057] Returning to the explanation of the flowchart in Fig. 4, the command generation unit 124 then generates the action a of the robot 10 determined using the policy network MDL3. t Based on this, a control command for controlling each actuator 13 of the robot 10 is generated (step S110).

[0058] Next, the communication control unit 125 transmits a control command to the robot 10 via the communication interface 110 (step S112). When the drive control unit 15 of the robot 10 receives the control command, it controls the actuator 13 based on the control command. This causes the robot 10 to act, and the target TR is lifted or moved, thereby changing the environmental state s surrounding the robot 10. t ga s t+1 It changes to.

[0059] Next, the acquisition unit 121 reacquires the state data, depth image data, and tactile data from the robot 10 via the communication interface 110 (step S114). That is, the acquisition unit 121 reacquires the environmental state s t+1 Observation results of t+1 The state data (state vector), depth image data (image vector), and haptic data (haptic vector) are reacquired.

[0060] The processing unit 120 processes various data (i.e., observation results) acquired from the robot 10. t+1 ) at time t+1, the processing unit 120 determines whether the target task has been accomplished (step S116). t+1 is in a desired state where the robot 10 has accomplished the target task.

[0061] If the desired task is accomplished (environment state s t+1 is in the desired state), the processing of this flowchart ends.

[0062] On the other hand, if the desired task is not achieved (environmental state s t+1 is not in the desired state), the processing unit 120 returns to the above-mentioned S102 and repeats the series of steps from S102 to S114 until the target task is accomplished, thereby completing the processing of this flowchart.

[0063] [Learning device configuration] The configuration of the learning device 200 according to the first embodiment will be described below. Fig. 6 is a configuration diagram of the learning device 200 according to the first embodiment. The learning device 200 includes, for example, a communication interface 210, a processing unit 220, and a storage unit 230.

[0064] The communication interface 210 communicates with the autonomous control device 100 via the network NW. The communication interface 210 includes, for example, a wireless communication module including a receiver and a transmitter, and an NIC.

[0065] The processing unit 220 includes, for example, an acquisition unit 221, a learning unit 222, and a communication control unit 223.

[0066] The components of the processing unit 220 are realized by, for example, a CPU or a GPU executing a program stored in the storage unit 230. Some or all of these components may be realized by hardware such as an LSI, an ASIC, or an FPGA, or may be realized by a combination of software and hardware.

[0067] The storage unit 230 is realized by, for example, a HDD, a flash memory, an EEPROM, a ROM, a RAM, etc. The storage unit 230 stores various programs such as firmware and application programs, as well as model data that defines an unlearned first encoder MDL1, an unlearned second encoder MDL2, and an unlearned policy network MDL3.

[0068] The acquisition unit 221 acquires a training data set for training the untrained first encoder MDL1.

[0069] The training data set is a data set in which state data (state vector) of the correct (ideal) target TR to be output by the first encoder MDL1 is labeled with depth image data (image vector) and haptic data (haptic vector) prepared for training. In other words, the training data set is a data set in which the depth image data (image vector) and haptic data (haptic vector) are used as input data and the state data (state vector) of the correct target TR is used as output data.

[0070] For example, the acquisition unit 221 may acquire a training dataset from another device (e.g., a data source) via the communication interface 210. Furthermore, if the training dataset is already stored in the storage unit 230, the acquisition unit 221 may read the training dataset from the storage unit 230. Furthermore, if a non-transitory storage medium (e.g., flash memory) on which the training dataset is stored is connected to the drive device of the learning device 200, the acquisition unit 221 may read the training dataset from the storage medium.

[0071] The learning unit 222 uses the training data set acquired by the acquisition unit 221 to learn the first encoder MDL1.

[0072] 7 is a diagram illustrating a learning method for the first encoder MDL1. For example, the learning unit 222 inputs depth image data (image vectors) and haptic data (haptic vectors) included as input data in a training data set to the unlearned first encoder MDL1.

[0073] In response to input of depth image data (image vectors) and haptic data (haptic vectors), the unlearned first encoder MDL1 compresses the number of dimensions of the data and outputs the compressed data as first compressed data.

[0074] The learning unit 222 calculates a difference Δ between the first compressed data output by the unlearned first encoder MDL1 and the state data (state vector) of the target TR included as output data in the training data set. Then, the learning unit 222 determines (updates) parameters of the first encoder MDL1, such as weighting coefficients and bias components, using a stochastic gradient descent method or the like, so as to reduce the difference Δ.

[0075] The learning unit 222 further learns the second encoder MDL2.

[0076] FIG. 8 is a diagram illustrating a training method for the second encoder MDL2. The second encoder MDL2 is trained using a decoder MDL4 that is functionally paired with the second encoder MDL2. As described above, the second encoder MDL2 is a neural network that converts input data into low-dimensional data and outputs the data. In contrast, the decoder MDL4 is a neural network that converts input data into high-dimensional data and outputs the data.

[0077] The learning unit 222 inputs haptic data (haptic vectors) prepared for training to the unlearned second encoder MDL2. In response to the input of the haptic data (haptic vectors), the unlearned second encoder MDL2 compresses the number of dimensions of the haptic data (haptic vectors) and outputs the compressed data as second compressed data.

[0078] The second compressed data output by the second encoder MDL2 is input to the untrained decoder MDL4. In response to the input of the second compressed data, the untrained decoder MDL4 converts the second compressed data into high-dimensional data and outputs it.

[0079] The learning unit 222 calculates the difference Δ between the haptic data (haptic vector) input to the second encoder MDL2 and the high-dimensional data output by the decoder MDL4. The learning unit 222 then determines (updates) the weighting coefficients and bias components, which are parameters of the second encoder MDL2 and the decoder MDL4, using a method such as stochastic gradient descent so as to reduce the difference Δ. In other words, the learning unit 222 learns the second encoder MDL2 and the decoder MDL4 so that the haptic data (haptic vector) input to the second encoder MDL2 matches the high-dimensional data output by the decoder MDL4.

[0080] The learning unit 222 further learns the policy network MDL3. For example, if the policy network MDL3 is policy-based, the learning unit 222 may learn the policy network MDL3 using policy gradients or the like. Furthermore, for example, if the policy network MDL3 is Actor-Critic, the learning unit 222 learns an Actor (actor) that decides the action, while simultaneously learning a Critic (evaluator) that evaluates the policy.

[0081] The communication control unit 223 transmits model data defining the first encoder MDL1, the second encoder MDL2, and the policy network MDL3 learned by the learning unit 222 to the autonomous control device 100 via the communication interface 210. This enables the autonomous control device 100 to determine the behavior of the robot 10 using each learned model.

[0082] According to the embodiment described above, the autonomous control device 100 generates lower-dimensional first compressed data by fusing and dimensionally compressing the depth image data (image vector) and haptic data (haptic vector) using the first encoder MDL1. The autonomous control device 100 generates lower-dimensional second compressed data by dimensionally compressing the haptic data (haptic vector) using the second encoder MDL2. The autonomous control device 100 generates combined state data z by combining the state data (state vector) of the robot 10, the first compressed data, and the second compressed data. t and then use the policy network MDL3 to generate the joint state data z t Actions of robot 10 that can accomplish the task from a t In this way, by using the first encoder MDL1 and the second encoder MDL2 to compress the number of dimensions of the data input to the policy network MDL3, the accuracy of the policy network MDL3 can be improved. As a result, the target task becomes easier to achieve.

[0083] Furthermore, according to the first embodiment described above, the first encoder MDL1, the second encoder MDL2, and the policy network MDL3 are separated and trained individually, thereby improving the learning efficiency and the accuracy of each model.

[0084] <Modification of the first embodiment> In the first embodiment described above, the learning unit 222 is described as learning the first encoder MDL1 using a training data set, but this is not limiting. For example, the learning unit 222 may learn the first encoder MDL1 in the same way as learning the second encoder MDL2.

[0085] 9 is a diagram illustrating another learning method for the first encoder MDL1. As with the second encoder MDL2, the first encoder MDL1 is learned using a decoder MDL5 that is functionally paired with the first encoder MDL1. As described above, the first encoder MDL1 is a neural network that converts input data into low-dimensional data and outputs the data. In contrast, the decoder MDL5 is a neural network that converts input data into high-dimensional data and outputs the data.

[0086] The learning unit 222 inputs depth image data (image vectors) and haptic data (haptic vectors) prepared for training to the unlearned first encoder MDL1. In response to the input of the depth image data (image vectors) and haptic data (haptic vectors), the unlearned first encoder MDL1 compresses the number of dimensions of the data and outputs the compressed data as first compressed data.

[0087] The first compressed data output by the first encoder MDL1 is input to the unlearned decoder MDL5. In response to the input of the first compressed data, the unlearned decoder MDL5 converts the first compressed data into high-dimensional data and outputs it.

[0088] The learning unit 222 calculates the difference Δ between the depth image data (image vectors) and haptic data (haptic vectors) input to the first encoder MDL1 and the high-dimensional data output by the decoder MDL5. Then, the learning unit 222 determines (updates) parameters of the first encoder MDL1 and the decoder MDL5, such as weighting coefficients and bias components, using a stochastic gradient descent method or the like, so as to reduce the difference Δ. In other words, the learning unit 222 learns the first encoder MDL1 and the decoder MDL5 so that the depth image data (image vectors) and haptic data (haptic vectors) input to the first encoder MDL1 match the high-dimensional data output by the decoder MDL5.

[0089] Second Embodiment The second embodiment will be described below. In the second embodiment, the policy network MDL3 determines the behavior a of the robot 10. t The second embodiment differs from the first embodiment in that it outputs a sensitivity parameter of the tactile sensor 12 in addition to the threshold value. The sensitivity parameter is a parameter for adjusting the sensitivity of the tactile sensor 12, and is, for example, a threshold value that is the boundary of the contact force between whether or not there is contact at each contact point of the tactile sensor 12. The sensitivity parameter may also be the gradient (inclination) of the contact force in addition to or instead of the threshold value. The following description will focus on differences from the first embodiment, and a description of points in common with the first embodiment will be omitted. In the description of the second embodiment, parts that are the same as those in the first embodiment will be denoted by the same reference numerals.

[0090] Fig. 10 is a flowchart showing the flow of a series of processes performed by the processing unit 120 according to the second embodiment. Fig. 11 is a diagram schematically showing the flow of a series of processes performed by the processing unit 120 according to the second embodiment.

[0091] First, the acquisition unit 121 acquires state data, depth image data, and tactile data from the robot 10 via the communication interface 110 (step S200).

[0092] Next, the data compression unit 122 generates first compressed data by fusing and dimensionally compressing the depth image data (image vector) and the tactile data (tactile vector) from the state data (state vector), depth image data (image vector), and tactile data (tactile vector) acquired by the acquisition unit 121 (step S202).

[0093] Next, the data compression unit 122 generates second compressed data by dimensionally compressing the tactile data (tactile vector) from the state data (state vector), depth image data (image vector), and tactile data (tactile vector) acquired by the acquisition unit 121 (step S204).

[0094] Next, the behavior decision unit 123 combines the state data (state vector), the first compressed data, and the second compressed data into combined state data z t (step S206), and then the combined state data z t Robot 10 behavior from a t and the sensitivity parameters of the tactile sensor 12 are determined (step S208). t It is assumed that the model has been trained in advance to output the sensitivity parameters.

[0095] Next, the command generation unit 124 generates the action a of the robot 10 determined using the policy network MDL3. t Based on this, a control command for controlling each actuator 13 of the robot 10 is generated (step S210).

[0096] Next, the communication control unit 125 transmits the control command and the sensitivity parameters of the tactile sensor 12 to the robot 10 via the communication interface 110 (step S212). Upon receiving the control command, the drive control unit 15 of the robot 10 controls the actuator 13 based on the control command. This causes the robot 10 to act and changes the environmental state s surrounding the robot 10. t ga s t+1 It changes to.

[0097] Next, the acquisition unit 121 reacquires the state data, depth image data, and tactile data from the robot 10 via the communication interface 110 (step S214). t+1 Observation results of t+1 The state data (state vector), depth image data (image vector), and haptic data (haptic vector) are reacquired.

[0098] The processing unit 120 processes various data (i.e., observation results) acquired from the robot 10. t+1 ) at time t+1, the processing unit 120 determines whether the target task has been accomplished (step S216). t+1 is in a desired state where the robot 10 has accomplished the target task.

[0099] If the desired task is accomplished (environment state s t+1 is in the desired state), the processing of this flowchart ends.

[0100] On the other hand, if the desired task is not achieved (environmental state s t+1 is not in the desired state), the drive control unit 15 of the robot 10 updates the sensitivity of the tactile sensor 12 in accordance with the sensitivity parameter of the tactile sensor 12 (step S218). Then, the processing unit 120 returns to the above-mentioned S202 and repeats the series of processes from S202 to S218 until the target task is accomplished. This ends the processing of this flowchart.

[0101] According to the second embodiment described above, the autonomous control device 100 uses the policy network MDL3 to generate the connection state data z t Actions of robot 10 that can accomplish the task from a tIn addition to determining the sensitivity parameter of the tactile sensor 12, the sensitivity parameter of the tactile sensor 12 is also determined. This adjusts the sensitivity of the tactile sensor 12 of the robot 10 so that the task can be accomplished, making it easier to accomplish the target task compared to the first embodiment.

[0102] For example, if the threshold determined as the sensitivity parameter is small and the sensitivity of the tactile sensor 12 is set to high, even a small force applied to the tactile sensor 12 can be detected with high accuracy. As a result, even if the target TR is a hairpin placed on a table, the hairpin can be found by touch alone.

[0103] On the other hand, for example, if the threshold determined as the sensitivity parameter is large and the sensitivity of the tactile sensor 12 is set to low, it can be detected with high accuracy even if a large force is applied to the tactile sensor 12. As a result, for example, even if the lid of a bottle is tightly fastened, the lid can be forcefully opened.

[0104] In the second embodiment described above, the bond state data z t Robot 10 behavior from a t In addition to determining the action a of the robot 10, the sensitivity parameter of the tactile sensor 12 is also determined. However, this is not limited to this. t And / or in addition to the sensitivity parameters of the tactile sensor 12, the angle of the visual sensor 11 (depth camera) when capturing an image of the body of the robot 10 or the target TR may also be determined.

[0105] The above-described embodiment can be expressed as follows. a storage medium for storing computer-readable instructions; a processor connected to the storage medium; The processor executes the computer-readable instructions to: acquiring state data of a robot, visual data of the robot, and tactile data of the robot; determining an action for the robot that can accomplish a task assigned to the robot based on the state data, the visual data, and the tactile data; fusing and dimensionally compressing the visual data and the tactile data to generate first compressed data having lower dimensions than the combined data of the visual data and the tactile data; generating second compressed data having a lower dimension than the haptic data by compressing the dimension of the haptic data; determining the action based on combined status data obtained by combining the status data, the first compressed data, and the second compressed data; Autonomous control system.

[0106] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0107] 1...autonomous control system, 10...robot, 11...visual sensor, 12...tactile sensor, 13...actuator, 14...status sensor, 15...drive control unit, 100...autonomous control device, 110...communication interface, 120...processing unit, 130...memory unit, 200...learning device, 210...communication interface, 220...processing unit, 230...memory unit, NW...network, MDL1...first encoder, MDL2...second encoder, MDL3...policy network

Claims

1. an acquisition unit that acquires state data of the robot, visual data of the robot, and tactile data of the robot; a processing unit that determines an action of the robot that can accomplish a task given to the robot based on the state data, the visual data, and the tactile data, The processing unit fusing and dimensionally compressing the visual data and the tactile data to generate first compressed data having lower dimensions than the combined data of the visual data and the tactile data; generating second compressed data having a lower dimension than the haptic data by compressing the dimension of the haptic data; determining the action based on combined status data obtained by combining the status data, the first compressed data, and the second compressed data; inputting the combined state data into a strategy network; generating a command for controlling an actuator of the robot based on a behavior variable that is a variable indicating the behavior output by the policy network in response to the input of the connection state data; Autonomous control system.

2. The acquisition unit acquiring, as the visual data, depth image data generated by a camera that captures an image of the robot body and the task target; acquiring, as the tactile data, data in which the contact force detected by each of the tactile sensors is associated with the distribution of the plurality of tactile sensors arranged on the body; the processing unit generates the first compressed data by fusing and dimensionally compressing the distribution of the plurality of tactile sensors and the depth image data. The autonomous control system of claim 1 .

3. The autonomous control system according to claim 2 , wherein the processing unit generates the second compressed data by dimensionally compressing data in which the contact forces detected by each of the plurality of tactile sensors are associated with a distribution of the plurality of tactile sensors.

4. the processing unit generates the first compressed data from the visual data and the haptic data using a first encoder; the first encoder is a neural network trained based on a training dataset in which target correct states of the task are labeled for the visual data and the tactile data; The autonomous control system according to any one of claims 1 to 3.

5. the processing unit generates the first compressed data from the visual data and the haptic data using a first encoder; The first encoder is a neural network that converts input data into low-dimensional data and outputs the converted data, and is combined with a decoder that converts input data into high-dimensional data and outputs the converted data, and is trained so that the data input to the first encoder and the data output by the decoder match. The autonomous control system according to any one of claims 1 to 3.

6. the processing unit generates the second compressed data from the haptic data using a second encoder; The second encoder is a neural network that converts input data into low-dimensional data and outputs the converted data, and is combined with a decoder that converts input data into high-dimensional data and outputs the converted data, and is trained so that the data input to the second encoder and the data output by the decoder match.

6. An autonomous control system according to claim 1.

7. The processing unit determines the action from the combined state data using reinforcement learning.

7. An autonomous control system according to any one of claims 1 to 6.

8. The processing unit further determines the sensitivity of the tactile sensor based on the connection state data.

4. The autonomous control system according to claim 2 or 3.

9. The processing unit further determines an angle of the camera when capturing an image of the body and the target based on the connection state data.

4. The autonomous control system according to claim 2 or 3.

10. acquiring state data of a robot, visual data of the robot, and tactile data of the robot; determining an action for the robot that can accomplish a task assigned to the robot based on the state data, the visual data, and the tactile data; fusing and dimensionally compressing the visual data and the tactile data to generate first compressed data having lower dimensions than the combined data of the visual data and the tactile data; generating second compressed data having a lower dimension than the haptic data by compressing the dimension of the haptic data; determining the action based on combined status data obtained by combining the status data, the first compressed data, and the second compressed data; inputting the combined state data into a strategy network; generating a command for controlling an actuator of the robot based on a behavior variable that is a variable indicating the behavior output by the policy network in response to the input of the connection state data; Autonomous control method.

11. On the computer, acquiring state data of a robot, visual data of the robot, and tactile data of the robot; determining an action for the robot based on the state data, the visual data, and the haptic data to enable the robot to accomplish a task assigned to the robot; fusing and dimensionally compressing the visual data and the haptic data to generate first compressed data having a lower dimension than the combined data of the visual data and the haptic data; and compressing the dimensionality of the haptic data to generate second compressed data having a lower dimension than the haptic data. determining the action based on combined status data obtained by combining the status data, the first compressed data, and the second compressed data; inputting said combined state data into a strategy network; generating a command for controlling an actuator of the robot based on a behavior variable that indicates the behavior output by the policy network in response to the input of the connection state data; A program to execute.

Images