Information processing device and program

A dual-resolution gaze-based deep imitation learning method improves the accuracy and efficiency of robot manipulation by separating high-resolution gaze images for important areas and low-resolution peripheral images, addressing the inefficiencies of conventional methods in precise tasks.

JP7584134B2Active Publication Date: 2024-11-15THE UNIV OF TOKYO
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2021012310
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2021-01-28
Publication Date
2024-11-15
Estimated Expiration
2041-01-28

AI Technical Summary

Technical Problem

Conventional deep imitation learning methods for robot manipulation are inefficient in terms of calculation costs and accuracy due to equally learning important and unimportant parts of the input image, making it difficult to apply to precise manipulation tasks.

Method used

Implementing a dual-resolution gaze-based deep imitation learning approach that separates the learning process into high-resolution gaze images for important areas and low-resolution peripheral images, using separate neural networks for fast and slow movements to improve accuracy and reduce computational load.

Benefits of technology

This method enhances the accuracy of the learning model while reducing calculation costs, enabling precise robot manipulation tasks like threading a needle and bolt picking by focusing on relevant visual information and improving generalization performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007584134000002
    Figure 0007584134000002
  • Figure 0007584134000003
    Figure 0007584134000003
  • Figure 0007584134000004
    Figure 0007584134000004
Patent Text Reader

Abstract

To improve accuracy of a learning model in a simulation learning with reducing calculation costs.SOLUTION: An information processing apparatus 30 according to the present invention has an observation image acquiring unit 101 for acquiring an observation image, a viewpoint image acquiring unit 102 for acquiring a viewpoint image, a peripheral image generating unit 103 for generating a peripheral image, a gazed image generating unit 104 for generating a gazed image, a classifying unit 105 for classifying the observation image, a teacher data generating unit 106 for generating a first teacher data including the peripheral image and a second teacher data including the gazed image, and a learning unit 107 for, if the peripheral image is input based on the first teacher data, carrying out imitation learning of a first learning model for use in output of target values of a position and a posture and for, if the gazed image is input based on the second teacher data, carrying out imitation learning of a second learning model for use in output of the target values. The information processing apparatus 30 further has an operation control unit 109 for controlling a manipulator based on the acquired target value.SELECTED DRAWING: Figure 2
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to an information processing device and a program that utilize gaze-based dual-resolution deep imitation learning for high-precision dexterous robot manipulation. [Background technology]

[0002] In traditional robot control based on imitation learning ([1], [2]), the relationship between the entire input image and the output action is learned using a deep neural network based on demonstration data by a human expert. This method does not require hard-coded robot control rules or physical models of objects, and is therefore promising for robot manipulation in everyday life, where physical models of objects are difficult to obtain. Summary of the Invention [Problem to be solved by the invention]

[0003] It is expected that imitation learning will be used to make a robot manipulator perform tasks involving precise manipulation. However, in conventional deep imitation learning methods, the entire input image is used for imitation learning calculations, so that important and unimportant parts of the input image for imitating manipulation are learned equally. This results in poor learning efficiency, and there is room for improvement in terms of both reducing computational costs and improving the accuracy of the learning model.

[0004] In view of the above, an object of the present invention is to provide an information processing device and a program capable of improving the accuracy of a learning model while reducing calculation costs in imitation learning. [Means for solving the problem]

[0005] In order to solve the above problems, an information processing device according to one aspect of the present invention comprises: An information processing device for imitation learning of a learning model for controlling a manipulator that executes a task on a target, an observation image acquisition unit that acquires an observation image of the target and the manipulator at each point in time of the task that is executed by a human operating the manipulator; a gaze point acquisition unit that acquires gaze point information indicating a gaze point at which the person is gazing in the observation image; a surrounding image generating unit that generates a surrounding image by reducing the resolution of the observation image; a gaze image generating unit that generates a gaze image of the observed image, the gaze image being configured by an image portion of a predetermined range including the gaze point in the gaze point information; a classification unit that classifies a motion of the manipulator in each of the observation images at each time point of the task into either a gaze motion that is presumed to be performed using a gaze field of view with a relatively high resolution when the task is assumed to be performed by a human, or a peripheral motion that is presumed to be performed using a peripheral field of view with a relatively low resolution when the task is assumed to be performed by a human; a teacher data generating unit that generates first teacher data including a pair of the peripheral image based on an observation image in which a motion of the manipulator is classified as the peripheral motion and values ​​indicating the position and posture of the manipulator in the peripheral motion, and second teacher data including a pair of the gaze image based on an observation image in which a motion of the manipulator is classified as the gaze motion and values ​​indicating the position and posture in the gaze motion; A learning unit that imitates and learns a first learning model that outputs target values ​​of the position and orientation when the peripheral image is input based on the first teacher data, and imitates and learns a second learning model that outputs the target values ​​when the gaze image is input based on the second teacher data; Equipped with.

[0006] An information processing device for controlling a manipulator that performs a task on a target, an observation image acquisition unit that acquires an observation image of the target and the manipulator for each time point of the task; a gaze point acquisition unit that acquires gaze point information indicating a gaze point at which a person gazes when performing the task in the observed image; a surrounding image generating unit that generates a surrounding image by reducing the resolution of the observation image; a gaze image generating unit that generates a gaze image of the observed image, the gaze image being configured by an image portion of a predetermined range including the gaze point in the gaze point information; a classification unit that classifies a motion of the manipulator in each of the observation images at each time point of the task into whether the motion corresponds to a gaze motion performed using a relatively high-resolution gaze field when the task is assumed to be performed by a human, or a peripheral motion performed using a relatively low-resolution peripheral field when the task is assumed to be performed by a human; a learning model acquisition unit that acquires a first learning model that is imitation-learned based on first teacher data including a pair of the peripheral image based on an observation image in which the movement of the manipulator is classified as the peripheral movement and values ​​indicating the position and posture of the manipulator in the peripheral movement, the first learning model outputting target values ​​of the position and posture when the peripheral image is input; and a second learning model that is imitation-learned based on second teacher data including a pair of the gaze image based on an observation image in which the movement of the manipulator is classified as the gaze movement and values ​​indicating the position and posture in the gaze movement, the second learning model outputting the target value when the gaze image is input; an operation control unit that, for an observed image classified as the peripheral operation, inputs the peripheral image into the first learning model to obtain the target value output from the first learning model, and, for an observed image classified as the gaze operation, inputs the gaze image into the second learning model to obtain the target value output from the second learning model, and controls the manipulator based on the obtained target value; Equipped with.

[0007] In addition, any combination of the above, or mutual substitution of the components or expressions of the present invention among methods, devices, programs, temporary or non-temporary storage media recording programs, systems, etc., are also valid aspects of the present invention. Effect of the Invention

[0008] According to the present invention, in imitation learning, it is possible to improve the accuracy of a learning model while reducing the calculation cost. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic configuration diagram of an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a functional block diagram of the information processing device of FIG. [Diagram 3] FIG. 13 is a diagram illustrating a method for determining a motion speed threshold for classifying motions into high speed and low speed. [Figure 4] FIG. 2 illustrates the architecture of an embodiment. [Diagram 5] FIG. 5(a) is a diagram showing an example of a neural network of a first learning model, and FIGS. 5(b) and (c) are diagrams showing examples of neural networks of a second learning model. [Figure 6] 13 is a flowchart of a process for generating a first learning model and a second learning model. [Figure 7] FIG. 1 illustrates a control algorithm for the movement of a robot manipulator. [Figure 8] 13 is a flowchart of a control process for the operation of a robot manipulator. [Figure 9] FIG. 13 is a diagram showing an evaluation table for a needle threading task according to an embodiment. [Figure 10] FIG. 13 is a diagram showing an evaluation table of learning efficiency in a needle threading task in the embodiment. [Figure 11] FIG. 13 is a diagram showing an evaluation table for a bolt picking task according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, the present invention will be described based on preferred embodiments with reference to the drawings. In the embodiments and modifications, the same or equivalent components are given the same reference numerals, and redundant descriptions are omitted as appropriate.

[0011] [Embodiment] First, an overview of the present embodiment will be described. It is desirable for a general-purpose robot manipulator to perform a high-precision operation task, such as a task of threading a needle. However, the task of threading a needle, for example, (1) requires a precise operation to thread the thread through a small gap in the eye of the needle, (2) the thread is deformed during the task, and (3) the posture of the thread and the needle differs for each attempt to pick up the thread in the task. It is difficult to establish a robot control rule taking into account the deformation of the thread and the posture of the thread and the needle that differs for each attempt. Therefore, it has been difficult to apply a conventional robot control rule-based approach to a precise task such as a task of threading a needle.

[0012] Deep imitation learning ([1], [2]) is used to train deep neural networks using demonstration data that includes human visual images and their actions. Deep imitation learning does not require hard-coded robot control rules, so it may be applicable to tasks involving precision operations in robot manipulation, such as threading a needle as mentioned above. However, for reasons (1) to (3) above, a highly accurate and complex motion control policy is required to achieve this task using deep imitation learning.

[0013] Conventional deep imitation learning-based visuomotor control methods learn action control policies using the entire visual image obtained during a task demonstration ([1], [2],

[10] ,

[11] ,

[12] ). Because the entire visual image is used, the amount of data required for learning increases, and this can take a long time. On the other hand, for example, if the resolution of the input image is reduced to reduce the amount of data, the imitation learning may not be able to distinguish precise operations in the input image, and a learning model that properly imitates precise operations may not be constructed. In addition, since the demonstration equally learns actions that contribute to the success of the task and actions that do not, the computational efficiency is poor. As such, conventional deep imitation learning methods have the problem of low generalization performance because they are difficult to apply to tasks that involve precise operations on objects.

[0014] On the other hand, humans use a dual-resolution control system with different visual resolutions when performing manipulation tasks [3]. According to anatomical studies of the human eye, the retina is divided into the fovea and the peripheral region. The fovea is located in the center of the retina with a visual field dominated by cone photoreceptors, and is relatively denser and thicker than other parts of the retina [4], providing relatively high-resolution visual information. When a human moves his or her hand toward a target, the gaze naturally focuses on the target ([5], [6], [7], [8]). The peripheral region contains a relatively sparse number of photoreceptors, especially photoreceptors dominated by rod photoreceptors [4], providing relatively low-resolution visual information.

[0015] It is also known that when a hand reaches out to a target under visual guidance, the fixation field and peripheral vision are functionally dissociated [3]. Humans first move their hand close to the target using low-resolution visual information from the peripheral vision, and then use high-resolution visual information from the fixation field to accurately guide the hand to the target at the end of the hand movement trajectory ([3], [9]).

[0016] The inventors believed that by applying this human gaze-based visual-motor control system with two different resolutions to deep imitation learning, it would be possible to realize tasks involving precision manipulation, such as threading a needle, in a robot manipulator, and thus arrived at the present invention.

[0017] 1 is a schematic configuration diagram of an information processing system 1 according to the present embodiment. The information processing system 1 includes a robot manipulator 10, an imaging device 20, an information processing device 30, a database 40, and an operation device 50. The robot manipulator 10, the imaging device 20, the information processing device 30, the database 40, and the operation device 50 are communicatively connected via a network NW.

[0018] The robot manipulator 10 operates to execute a predetermined task on the target 5 in response to an operation signal from an operating device 50 or an operation command from an operation control unit 109, which will be described later. The robot manipulator 10 includes, for example, an end effector 11, a plurality of joints 12 driven by a plurality of driving devices (not shown) such as motors, and a plurality of links 13 connecting adjacent joints (all not shown). For example, in the robot manipulator 10, the driving is controlled by controlling the driving device so that the robot manipulator 10 rotates in a rotation direction and at a rotation speed specified in an operation signal or operation command, which will be described later.

[0019] The imaging device 20 captures images of the target 5 and the robot manipulator 10 to generate their observation images. The imaging device 20 of this embodiment is a stereo camera that has a field of view on both the left and right sides and generates two observation images on the left and right sides. The generated observation images are supplied to the information processing device 30, the database 40, and the operation device 50 via the network NW.

[0020] The information processing device 30 includes a control device 31, a storage device 32, and a communication device 33. The control device 31 is configured with, for example, a CPU, a GPU, a DSP, etc. The storage device 32 is configured with, for example, various memories such as a ROM and a RAM. The storage device 32 stores various data acquired via the communication device 33. The storage device 32 also stores various programs for executing various controls in the control device 31. The communication device 33 performs data communication between the robot manipulator 10 and the database 40 via a network NW by a wired or wireless connection.

[0021] The database 40 stores observed images of the target 5 and the robot manipulator 10, motion trajectory data indicating the position and posture of the robot manipulator 10 in time series, a learning model supplied from the information processing device 30, and the like. Specifically, the database 40 stores observed images in association with motion trajectory data for the observed images. The motion trajectory data in this embodiment is time-series data on the position and posture (direction) of the tip of the end effector of the robot manipulator 10. The motion trajectory data is acquired by the human operator 2 demonstrating a task by operating the robot manipulator 10 via the operating device 50, and is stored in advance in the database 40.

[0022] The operation device 50 is a device for a human operator 2 to operate the robot manipulator 10. The operation device 50 includes a head mounted display (HMD) 51, a controller 52, and an eye gaze tracking device 53. The HMD 51 displays an observation image supplied from the imaging device 20. The controller 52 includes an input unit (not shown) that receives an operation input from the operator 2. The controller 52 controls the driving of the robot manipulator 10 by transmitting a control signal to the robot manipulator 10 in response to the operation input from the operator 2. The operator 2 can perform a demonstration of a task by operating the controller 52 based on the observation image displayed on the HMD 51. The eye gaze tracking device 53 is attached to the HMD 51, and generates gaze point information indicating the gaze point at which the operator 2 gazes by tracking the position of the gaze of the operator 2. The eye gaze tracking device 53 supplies the generated gaze point information to the database 40 via the network NW, and stores the gaze point information in the database 40 in association with the observation image at the time of generating the gaze point information.

[0023] Fig. 2 is a functional block diagram of the information processing device 30 in Fig. 1. Each functional block shown in the following diagram is realized in terms of hardware by a computer having a calculation function, a control function, a storage function, an input function, and an output function, various electronic elements, mechanical parts, etc., and in terms of software by a computer program, etc., but here, functional blocks realized by the cooperation of these are depicted. Therefore, it is understood by those skilled in the art that these functional blocks can be realized in various forms by combining hardware and software.

[0024] The information processing device 30 of Figure 2 includes an observation image acquisition unit 101, a gaze point acquisition unit 102, a surrounding image generation unit 103, a gaze image generation unit 104, a classification unit 105, a teacher data generation unit 106, a learning unit 107, a learning model acquisition unit 108, an operation control unit 109, an operation control unit 109, a judgment unit 110, and a memory unit 111.

[0025] The observation image acquisition unit 101 acquires observation images of the target 5 and the robot manipulator 10 in a given task.

[0026] The gaze point acquisition unit 102 acquires gaze point information indicating a gaze point at which a person gazes in an observed image. The gaze point here indicates the position of the human's gaze point in the observed image, and more specifically, indicates the position where the observed image intersects with the line of sight. The gaze point acquisition unit 102 includes a first gaze point acquisition unit 102a that acquires gaze point information generated by the gaze tracking device 53 and stored in the database 40, and a second gaze point acquisition unit 102b that acquires gaze point information using a gaze predictor that predicts the position of the gaze point of the operator 2 in the observed image.

[0027] The surrounding image generating unit 103 generates a surrounding image by lowering the resolution of the observed image.

[0028] The gaze image generating unit 104 generates a gaze image made up of an image portion of a predetermined range including the gaze point in the gaze point information acquired by the gaze point acquiring unit 102 from the observed image.

[0029] The classification unit 105 classifies the motion of the robot manipulator 10 in the observation image into either a peripheral motion or a gaze motion. The gaze motion is a motion performed using a relatively high-resolution gaze field when it is assumed that the task is performed by a human, and is, for example, a motion below a predetermined speed range or within a predetermined distance range from the gaze point. The peripheral motion is a motion performed using a relatively low-resolution peripheral field when it is assumed that the task is performed by a human, and is, for example, a motion above a predetermined speed range or outside a predetermined distance range from the gaze point. When learning a motion scale classifier described later, the classification unit 105 classifies the motion of the robot manipulator 10 into either a peripheral motion or a gaze motion based on a comparison between the motion speed of the robot manipulator 10 and a motion speed threshold described later. In addition, when controlling the motion of the robot manipulator 10 in a task, the classification unit 105 inputs the observation image into a motion scale classifier described later to classify the motion of the robot manipulator 10 into either a peripheral motion or a gaze motion.

[0030] Here, according to physiological research, human target reaching movements can be divided into high-speed reaching movements calculated from peripheral vision and accurate low-speed guiding movements calculated from gaze vision ([3], [9]). In this embodiment, based on the results of this research, it is determined from the observation image whether the movement of the robot manipulator 10 is a high-speed movement or a low-speed movement. The high-speed movement in this embodiment is an example of a peripheral movement, and the low-speed movement in this embodiment is an example of a gaze movement. Movements are divided into high-speed and low-speed movements according to the scale of the movement. A movement is defined by the difference in position [x;y;z] between the current step and the next step of the robot manipulator 10 during remote operation. Since rotation is not important in the needle threading task and the bolt picking task described later, the movement speed, which indicates the scale of the movement, is defined by the Euclidean norm of the following equation (1), which indicates the difference in position. In equation (1), s t represents the motion speed at time step t.

[0031]

number

[0032] In addition, for tasks where rotation during operation is an important factor, the operating speed may be defined by the difference in position [x; y; z] between the current and next steps of the robot manipulator 10 as well as the difference in Euler angles [α; β; γ].

[0033] The classification unit 105 judges whether the demonstration data of the operator 2 is a high-speed motion or a low-speed motion from a histogram of the movement amount of the end effector 11 per unit time. This judgment is made based on the motion speed of the end effector 11 and a motion speed threshold value described later. The learning unit 107 learns the binary classification problem of high-speed motion and low-speed motion by supervised learning based on the judgment result. Specifically, the learning unit 107 trains a binary classifier (motion scale classifier) ​​using the scale of the motion of the robot manipulator 10. The motion scale classifier learns whether the motion of the robot manipulator 10 in the observation image corresponds to a low-speed motion or a high-speed motion using the correct answer labels of the high-speed motion and the low-speed motion judged by the classification unit 105. After this learning, the motion of the robot manipulator 10 at the current motion step is automatically selected from either the low-speed motion or the high-speed motion using the motion scale classifier. Since the gaze image has sufficient information for classification, the gaze image was used as an input of the motion scale classifier. The trained motion scale classifier is stored in the database 40 and the storage unit 111. The motion scale classifier is an example of a motion classifier.

[0034] Please refer to Fig. 3. Fig. 3 is a diagram for explaining a method of determining a motion speed threshold for classifying a high-speed motion and a low-speed motion. A Gaussian mixture model (GMM) having two Gaussian distributions corresponding to a low-speed motion and a high-speed motion, respectively, is fitted from the motion scale in a histogram of the movement amount based on the position [x; y; z] of the robot manipulator 10. The classification unit 105 determines the intersection point of the two Gaussian distributions as the motion speed threshold for distinguishing between a high-speed motion and a low-speed motion.

[0035] The teacher data generating unit 106 generates first teacher data and second teacher data. The first teacher data is teacher data including a pair of a peripheral image based on an observation image in which the motion of the robot manipulator 10 is classified as a peripheral motion, and a position and posture of the robot manipulator 10 in the peripheral motion. The second teacher data is teacher data including a pair of a gaze image based on an observation image in which the motion of the robot manipulator 10 is classified as a gaze motion, and a position and posture in the gaze motion. The position and posture as the correct answer data in the teacher data are acquired based on the motion trajectory data stored in the database 40 in association with the observation image.

[0036] The learning unit 107 imitates and learns a first learning model based on the first teacher data, and imitates and learns a second learning model based on the second teacher data. The first learning model outputs target values ​​for the position and posture of the robot manipulator 10 when a peripheral image is input. The second learning model outputs target values ​​for the position and posture of the robot manipulator 10 when a gaze image is input. The target values ​​here indicate the subsequent position and posture of the robot manipulator 10 in the task to be executed. In this embodiment, the target values ​​indicate the target position and posture in the next operation step of the robot manipulator 10.

[0037] In this embodiment, a neural network is used as the imitation learning algorithm of the first learning model and the second learning model. In this embodiment, two neural networks, a first learning model for high-speed motion and a second learning model for low-speed motion, are trained separately. This is because if high-speed motion and low-speed motion are trained together using one neural network, the weights of the neural network are better adjusted to predict high-speed motion to reduce imitation loss, and low-speed motion cannot be learned with sufficient accuracy.

[0038] The learning model acquisition unit 108 acquires the first learning model, the second learning model, etc. The learning model acquisition unit 108 of the present embodiment acquires the first learning model, the second learning model, etc. from the database 40, but is not limited thereto, and may acquire the first learning model, the second learning model, etc. from the storage unit 111, for example.

[0039] The operation control unit 109 obtains target values ​​of position and posture output from the first learning model by inputting the peripheral image of an observed image in which the operation of the robot manipulator 10 is classified as a peripheral operation into the first learning model. Also, the operation control unit 109 obtains target values ​​of position and posture output from the second learning model by inputting the gaze image of an observed image in which the operation of the robot manipulator 10 is classified as a gaze operation into the second learning model.

[0040] Furthermore, the operation control unit 109 controls the robot manipulator 10 based on the acquired target value. Specifically, the operation control unit 109 supplies an operation command to the robot manipulator 10 for controlling the operation of the robot manipulator 10 so as to realize the acquired target value.

[0041] The determination unit 110 determines whether a recovery operation, which will be described later, is necessary and whether a task has been completed.

[0042] The memory unit 111 stores various data such as observed images, a first learning model, a second learning model, a motion scale classifier, a recovery classifier, a recovery step predictor, and a learning model for recovery, which will be described later.

[0043] Hereinafter, this embodiment will be described taking as an example a task of operating the robot manipulator 10 to pick up a thread and thread the thread through the eye of a needle (hereinafter referred to as a "needle threading task"). In the needle threading task, the thread is the target 5 when the robot manipulator 10 is not gripping the thread, and the needle is the target 5 after the thread is picked up.

[0044] In this setup, a human operator remotely controls a robot manipulator 10 (e.g., a UR5 robot) based on visual input from an HMD 51. The HMD 51 displays images from an imaging device 20 (ZED Mini stereo camera

[14] ), while an eye-tracking device 53 attached to the HMD 51 measures the gaze of the operator 2.

[0045] FIG. 4 is a diagram showing the architecture of this embodiment. Both left and right observation images (1280×720) acquired by the observation image acquisition unit 101 are processed into a peripheral image and a gaze image by the peripheral image generation unit 103 and the gaze image generation unit 104, respectively. The peripheral image of this embodiment is obtained by reducing and resizing the entire acquired left and right observation images to 128×72. The peripheral image is used to predict the position of the gaze point using high-speed motion prediction and a gaze predictor described later. The left and right observation images are cropped using the positions of the left and right gaze points acquired by the gaze point acquisition unit 102 to generate a gaze image. The gaze image of this embodiment is an image obtained by cropping a 142×120 area around the gaze point from both the acquired left and right observation images. The field of view of the ZED Mini stereo camera as the imaging device 20 is 90° (H) × 60° (V) × 100°, so that 142×120 of the image from the full image size of 1280×720 corresponds to the central part (10°) of the retina of the human eye ([3]). The generated gaze and peripheral images are used as inputs for the computation of the visuomotor policy.

[0046] Using the gaze images, the following are inferred: (1) Opening and closing operation of the gripping portion (not shown) of the end effector 11 (2) A slow movement to thread the needle (3) Slow operation for recovery when a task fails (recovery will be discussed later) (4) Classification of the scale (low speed / high speed) of the movements performed by the robot manipulator 10

[0047] Please refer to Figure 5(a) to Figure 5(c). Features of the peripheral images are extracted from Spatial Softmax ([1],

[11] ). The Spatial Softmax function represents the features as 2D coordinates (Figure 5(a)). To infer the relationship between the 3D shape of the thread and the motion of the robot manipulator 10 in the observed image, the gaze image is processed using a series of convolutional neural networks and Max Pooling layers with stride number 2 (Figure 5(b) and Figure 5(c)). The extracted features are then processed in a series of fully connected (FC) layers. Batch Norm

[16] and ReLU activation function

[17] are used between the FC layers. There are 200 nodes per FC layer. The neural network that infers the motion (slow motion, fast motion, opening and closing the grip of the end effector 11) used three FC layers, and the other neural networks used two FC layers.

[0048] Hereinafter, the flow of the process of generating the first learning model and the second learning model in this embodiment will be described with reference to FIG.

[0049] In step S101, the observation image acquisition unit 101 acquires an observation image of the target 5 and the robot manipulator 10 in the needle threading task and corresponding motion trajectory data, and the first gaze point acquisition unit 102a acquires gaze point information in the observation image. In step S101, the observation image, motion trajectory data, and gaze point information are acquired for each time point of the task. In step S101, the human operator 2 remotely operates the robot manipulator 10 and records the image and trajectory using the imaging device 20, while the gaze tracking device 53 records the movement of the operator's gaze. Specifically, the observation image in step S101 is generated by the operator 2 operating the robot manipulator 10 via the operating device 50 to perform a demonstration of the needle threading task and capturing an image of this demonstration using the imaging device 20. The motion trajectory data is generated based on the operation of the operating device 50. The gaze point information in step S101 is generated by tracking the gaze of the operator 2 during the demonstration using the gaze tracking device 53. The imaging device 20 supplies the observation images generated for each time point of the needle threading task, the operation device 50 supplies the motion trajectory data corresponding to the observation images, and the gaze tracking device 53 supplies the gaze point information corresponding to the observation images to the database 40. The database 40 stores the supplied observation images, the corresponding motion trajectory data, and the gaze point information in association with each other. The observation image acquisition unit 101 of this embodiment acquires a plurality of observation images and corresponding motion trajectory data stored in advance based on the demonstration of the operator 2 from the database 40. The first gaze point acquisition unit 102a acquires the gaze point information corresponding to the observation images from the database 40. The observation image acquisition unit 101 supplies the acquired observation images to the peripheral image generation unit 103 and the gaze image generation unit 104. The observation image acquisition unit 101 also supplies the acquired motion trajectory data to the teacher data generation unit 106. The first gaze point acquisition unit 102a supplies the acquired gaze point information to the gaze image generation unit 104.

[0050] In step S102, the peripheral image generating unit 103 generates a peripheral image, and the gaze image generating unit 104 generates a gaze image. The peripheral image generating unit 103 of this embodiment resizes the entire acquired observation image to reduce the resolution of the observation image and generate a peripheral image. The gaze image generating unit 104 of this embodiment generates a gaze image by trimming an image of a predetermined range including the gaze point in the observation image based on the gaze point information. The peripheral image generating unit 103 supplies the generated peripheral image to the teacher data generating unit 106. The gaze image generating unit 104 supplies the generated gaze image to the classification unit 105 and the teacher data generating unit 106.

[0051] In step S103, the classification unit 105 classifies the motion of the robot manipulator 10 in each of the observation images acquired at each time point of the task into either a peripheral motion or a gaze motion. The classification unit 105 of this embodiment measures the motion speed (m / step) in the previous step and the current step based on the gaze image. If the measured motion speed is greater than the motion speed threshold, the classification unit 105 classifies the current motion as a high-speed motion (peripheral motion). If the measured motion speed is equal to or less than the motion speed threshold, the classification unit 105 classifies the current motion as a low-speed motion (gaze motion). The classification unit 105 supplies the classification result of the motion of the robot manipulator 10 in each observation image to the teacher data generation unit 106.

[0052] In step S104, the teacher data generating unit 106 generates first teacher data and second teacher data. Specifically, the teacher data generating unit 106 generates the first teacher data using peripheral images and motion trajectory data of observed images in which the motion of the robot manipulator 10 is classified as peripheral motion, and generates the second teacher data using gaze images and motion trajectory data of observed images in which the motion of the robot manipulator 10 is classified as gaze motion. The teacher data generating unit 106 supplies the generated first teacher data and second teacher data to the learning unit 107.

[0053] In step S105, the learning unit 107 imitates and learns a first learning model based on the first teacher data, and imitates and learns a second learning model based on the second teacher data. The learning unit 107 supplies the constructed first learning model and second learning model to the database 40 and the storage unit 111. The database 40 and the storage unit 111 store the first learning model and the second learning model related to a predetermined task (a needle threading task in this embodiment) in association with each other. After step S105, the learning model generation process S100 ends.

[0054] 7 shows a control algorithm for the operation of the robot manipulator 10 in this embodiment. In the algorithm in FIG. fast indicates high speed operation, and π slow indicates slow operation, and π rec indicates the recovery action, and π rec indicates the opening and closing of the gripping portion of the end effector 11, θ indicates the recovery classifier, ρ indicates the gaze predictor, Ψ indicates the recovery step predictor, and φ indicates the motion scale classifier.

[0055] The control process S200 for the operation of the robot manipulator 10 in this embodiment will be described with reference to FIG.

[0056] In step S201, the observation image acquisition unit 101 acquires an observation image of the target 5 and the robot manipulator 10. In step S201, the observation image acquisition unit 101 acquires an observation image captured by the imaging device 20 from the imaging device 20. The observation image acquisition unit 101 supplies the acquired observation image to the surrounding image generation unit 103 and the gaze image generation unit 104.

[0057] In step S202, the surrounding image generating unit 103 generates a surrounding image based on the supplied observation image. The surrounding image generating unit 103 supplies the generated surrounding image to the gaze point obtaining unit 102 and the operation control unit 109.

[0058] In step S203, the second gaze point acquisition unit 102b acquires gaze point information in the acquired observation image. The second gaze point acquisition unit 102b acquires gaze point information indicating a gaze point that is estimated to be gazed at when a human is assumed to perform a task, by inputting the generated peripheral image to a gaze predictor that predicts the position of the gaze point of the operator 2 in the observation image. This gaze predictor is stored in advance in the storage unit 111, and is read out from the storage unit 111 by the second gaze point acquisition unit 102b. For example, the gaze predictor uses a mixed density network (MDN) (

[13] ,

[15] ). The second gaze point acquisition unit 102b supplies the acquired gaze point information to the gaze image generation unit 104. Note that the gaze point information may be acquired by inputting an observation image, rather than a peripheral image, to the gaze predictor.

[0059] In step S204, the gaze image generating unit 104 generates a gaze image based on the supplied observation image and gaze point information. The gaze image generating unit 104 supplies the generated gaze image to the classification unit 105 and the operation control unit 109.

[0060] In step S205, the classification unit 105 classifies the motion of the robot manipulator 10 in the supplied observation image as either a peripheral motion or a gazing motion. Specifically, the classification unit 105 reads out a trained motion scale classifier from the storage unit 111, and inputs the gazing image to the motion scale classifier, thereby performing the classification based on the classification result of the peripheral motion or the gazing motion output from the motion scale classifier. If the motion is classified as a peripheral motion in step S205, the classification unit 105 supplies the classification result to the motion control unit 109, and the process S200 proceeds to step S206. If the motion is classified as a gazing motion in step S205, the classification unit 105 supplies the classification result to the motion control unit 109, and the process S200 proceeds to step S207.

[0061] In step S206, the operation control unit 109 inputs the peripheral image into the first learning model and acquires a target value from the first learning model. Here, the operation control unit 109 reads out the first learning model stored in association with the needle threading task in the database 40 via the learning model acquisition unit 108, and uses this. After step S207, the process S200 proceeds to step S211.

[0062] In step S207, the determination unit 110 determines whether a recovery operation is necessary. Here, the recovery operation will be described. The "recovery operation" refers to an operation from a state where a task for the target 5 has failed (hereinafter referred to as a "failure state") to the next attempt of the task. In this embodiment, the "recovery operation" refers to an operation of the robot manipulator 10 from a state where the end effector 11 has failed to pick up the thread or thread the needle to the next pick-up or threading. In this embodiment, the "failure state" refers to a state where the end effector 11 has failed to grasp the thread, or the thread has passed around the needle without passing through the needle hole, or a state where a recovery operation is being performed. In the needle threading task, because the gap of the needle hole is small, even human operators frequently fail in demonstrations. Humans will retry after a failure until threading is finally successful. Therefore, it was assumed that the robot manipulator 10 should also be able to recover from failure and achieve better performance in subsequent attempts.

[0063] In this embodiment, the determination unit 110 determines whether a recovery action is required based on whether the current state is a failure state. The determination unit 110 uses a recovery classifier, which is a neural network, to recognize whether the current state is a failure state. The recovery classifier predicts whether the robot manipulator 10 has failed to thread the thread into the eye of the needle based on the gaze image. The recovery classifier is trained based on the failure state actions indexed based on the demonstration data and stored in advance in the storage unit 111 and the database 40.

[0064] In response to the gaze image being supplied from the gaze image generating unit 104, the determining unit 110 reads out the recovery classifier from the storage unit 111 via the learning model acquiring unit 108, and inputs the gaze image to the recovery classifier. When the determining unit 110 recognizes that the state is not a failure state based on the classification result of the recovery classifier, the determining unit 110 determines that a recovery operation is not necessary. Here, "not a failure state" includes, for example, not only a successful state in which the thread is threaded through the eye of the needle, but also a state in which the robot manipulator 10 has not reached a position where the thread is threaded through the eye of the needle when the current state is not in the process of performing a recovery operation. When the determining unit 110 determines that a recovery operation is not necessary (N in step S207), the determining unit 110 supplies a motion control command to the motion control unit 109, and the process S200 proceeds to step S208. On the other hand, when the determining unit 110 recognizes that the state is a failure state based on the classification result of the recovery classifier, the determining unit 110 determines that a recovery operation is necessary. In this case (Y in step S207), the determination unit 110 supplies a recovery operation control command to the operation control unit 109, and the process S200 proceeds to step S209.

[0065] In step S208, the operation control unit 109 inputs the gaze image into the second learning model and acquires a target value from the second learning model. Here, the operation control unit 109 reads out the second learning model stored in association with the needle threading task in the database 40 via the learning model acquisition unit 108, and uses this. After step S208, the process S200 proceeds to step S211.

[0066] In step S209, the motion control unit 109 obtains the number of steps required to complete the recovery motion based on the gaze image. In this embodiment, the motion control unit 109 obtains the number of steps using a recovery step predictor that outputs the number of steps required to complete the recovery when the gaze image is input.

[0067] In step S210, the motion control unit 109 inputs the acquired gaze image and the number of steps into the learning model for recovery to acquire a target value from the learning model for recovery. Here, the motion control unit 109 reads out the learning model for recovery stored in association with the needle threading task in the database 40 via the learning model acquisition unit 108, and uses it. The learning model for recovery predicts a recovery motion based on the gaze image and the number of steps. The learning model for recovery is trained based on the recovery motion indexed based on the demonstration data, and is stored in advance in the storage unit 111 and the database 40. After step S210, the process S200 proceeds to step S211.

[0068] In step S211, the operation control unit 109 controls the robot manipulator 10 to move the end effector 11 of the robot manipulator 10 based on the acquired target value. After step S211, the operation control unit 109 supplies a task completion determination command, and the process S200 proceeds to step S212.

[0069] In step S212, the determination unit 110 determines whether the task is completed. In this embodiment, the determination unit 110 determines that the task is completed if the needle threading task is successful, and determines that the task is not completed if the robot manipulator 10 has not yet reached a position where the thread can be threaded into the eye of the needle. If it is determined that the task is not completed in step S212 (N in step S212), the determination unit 110 supplies an observation image acquisition command to the observation image acquisition unit 101, and the process S200 returns to step S201. If it is determined that the task is completed in step S212 (Y in step S212), the process S200 ends.

[0070] The effects of this embodiment will be described. By using a dual-resolution control method using a peripheral image and a gaze image, precise manipulation for tasks involving environments and objects that are difficult to model can be achieved using the robot manipulator 10. For example, a task that requires precision, such as threading a needle, can be achieved by using a high-resolution gaze image.

[0071] The separated visuomotor control architecture of this embodiment has the following advantages over previous deep imitation learning methods. Using an explicit gaze mechanism to focus on important features of the entire scene requires fewer neural network parameters, improving generalization performance. In addition, since only the required image regions are kept in high resolution as gaze images, higher computational efficiency can be achieved and the learning speed can be significantly improved. Furthermore, the method of this embodiment is capable of controlling the movement according to the different postures of the thread or needle for each trial. Therefore, it is expected to be applicable to the manipulation of soft objects and food.

[0072] (Example) <Task settings> In this embodiment, the needle threading task is set so that the robot manipulator 10 (1) picks up a thread and (2) threads the thread through a needle hole randomly arranged in a 9 cm x 9 cm area on a table. In this setting, there is a difference between the needle position and the thread grip position between each attempt of threading the needle. The neural network needs to adjust the policy to accommodate such position differences.

[0073] To demonstrate that the method of the present embodiment is generally applicable to precise tasks, a bolt-picking task was set in which the robot manipulator 10 picks up M6×14 bolts in a 9 cm×9 cm area on a table. Therefore, the robot manipulator 10 needs to learn the precise position to grab the bolt while avoiding collision with the table.

[0074] In this example, the neural network was trained by the needle threading task using 1649 episodes of training data (196.8 min) and 148 episodes of validation data (17.01 min). For the bolt-picking task, 775 training episodes (39.56 min) and 102 validation episodes (5.402 min) were used. Operator demonstrations of each task were sampled at 10.0 Hz. Each neural network component was selected from the minimum validation loss over 30 epochs. A learning rate of 1e-4, the Rectified Adam (RAdam) optimizer

[18] , and the same behavior loss as in

[13] were used. Each neural network component was trained using a Xeon CPU E5-2698 v4 and an NVIDIA Tesla V100 GPU with a batch size of 64. When training was performed using 1280x720 images, a batch size of 8 was used to avoid memory issues. The UR5 robotic manipulator was tested using an Intel® CPU Core i7-8700K and one NVIDIA® GeForce GTX 1080 Ti.

[0075] <Evaluation of the method of this embodiment> Evaluation of the technique of this embodiment will be described with reference to Figs. 9 to 11. Fig. 9 is a diagram showing an evaluation table for a needle threading task according to this embodiment. Fig. 10 is a diagram showing an evaluation table for learning efficiency in a needle threading task according to this embodiment. Fig. 11 is a diagram showing an evaluation table for a bolt picking task according to this embodiment.

[0076] First, refer to Fig. 9(a) and Fig. 11(a). The method of the present embodiment achieved a success rate of 81.25% for needle threading and 68.75% for bolt picking. In this example, the robot manipulator 10 picked up the thread, recognized the failure, recovered from the failed state, and finally succeeded in threading. In another trial, the robot was able to perform the appropriate operation even when the thread was deformed by colliding with the needle.

[0077] <Evaluation of gaze images> See Fig. 9(b). To evaluate how the gaze image affects both computational cost and performance, the method of the present embodiment is compared with a method that does not use the gaze image. When the gaze image is replaced with the peripheral image (128x72), it can be seen that the success rate of needle threading is 12.5%, and the failure rate is relatively high.

[0078] See Figure 9(b) and Figure 11(b). When all images were replaced with observed images (1280x720), the neural network failed to pick up the thread. This network architecture also failed in bolt picking. Therefore, choosing images with appropriate resolution depending on the target task contributes to improving generalization.

[0079] Please refer to Fig. 10. The computational cost of the method of this embodiment was compared with that of the method using an observed image of 1280 x 720. The method of this embodiment required 1 / 8.10 multiply-accumulate operations (MAC) to calculate one step, and required approximately 1 / 27.7 of the computation time to train all 30 epochs of all neural networks.

[0080] <Evaluation of separation of high-speed and low-speed motion> See Fig. 9(c) and Fig. 11(c). When both the fast and slow motions were trained using only one neural network, the success rate of needle threading dropped to 0%. The tested thresholds were [0.0208; 0.00364; 0.0790; 0.0985], which correspond to [μ1; μ1+σ1; μ2-σ2 / 2; μ2], where μ1, μ2 and σ1, σ2 are the mean and standard deviation of each Gaussian distribution (μ1<μ2). The intersection point is 0.05685~μ1+2.315σ1~μ2-1.069σ2.

[0081] <Assessment of the separation of visual-motor control> See Fig. 9(d). The approach of this embodiment assigned the peripheral image to the fast motion and the gaze image to the slow motion. Other architectures, such as assigning only the gaze image to both the fast and slow motions, or assigning both the gaze image and the peripheral image to the fast motion, failed to pick up the yarn. Furthermore, assigning both the gaze image and the peripheral image to the slow motion reduced the success rate.

[0082] <Evaluation of recovery actions> See FIG. 9(e). When only one neural network was used to train the successful and recovery actions, the performance was degraded. This was because the recovery actions were not explicitly captured, and therefore the robot manipulator 10 could not recover after failure. In particular, the method of the present embodiment was able to recover in 12 out of 13 failures, while the neural network trained with the successful and recovery actions could only recover in 1 out of 8 failures. Furthermore, without a recovery step predictor, the success rate dropped to 31.25%. At this time, the robot manipulator 10 repeatedly switched between the successful and recovery actions due to perturbations in the output of the recovery classifier.

[0083] <Resolution evaluation> See Figure 9(f). When half-resolution images (640 × 360) were used, the success rate dropped to 50%. This indicates that high-resolution images are required to perform the needle threading task.

[0084] <Evaluation of stereoscopic vision> See Figure 9(g). Without stereoscopic vision, the success rate dropped dramatically. This shows that it is important to recognize the three-dimensional state, including the depth of the thread, and the distance / direction to the needle. Therefore, obtaining depth information using left and right observation images contributes to improving performance.

[0085] <Summary of evaluation> Conventionally, the sequence of tasks from picking up a thread to threading a needle has not been realized by a robot manipulator. According to the present embodiment, it was demonstrated that the robot manipulator 10 can perform the needle threading task with high accuracy. The approach achieved appropriate control even when the thread was deformed. Furthermore, the results obtained in the bolt-picking task reveal that the method of the present embodiment is generalizable, capable of learning complex policies, and can be applied to other tasks that require high accuracy. The results obtained by the method of the present embodiment confirm that decoupled visuomotor control improves both task performance and computational efficiency compared to processing using high-resolution visual input. Furthermore, the results are consistent with those obtained by previous studies on the role of gaze / peripheral vision in visuomotor manipulation ([3], [6], [9]). This suggests that the characteristics of human behavior must be taken into account when applying imitation learning of human-generated data to the robot manipulator 10, and that using end-to-end fitting of motor control output from input sensory data may not be sufficient.

[0086] (Modification) In the embodiment, the classification unit 105 classifies whether the movement corresponds to a peripheral movement or a gaze movement based on a comparison between the movement speed and the movement speed threshold, but is not limited thereto. For example, the classification unit 105 may classify whether the movement corresponds to a peripheral movement or a gaze movement by comparing the distance between the gaze point and the end effector 11 of the robot manipulator 10 with the distance threshold. In this case, the distance threshold may be obtained using a Gaussian mixture model having two Gaussian distributions corresponding to low-speed movement and high-speed movement, respectively, similar to the movement speed threshold. In summary, the classification unit 105 can classify whether the movement corresponds to a peripheral movement or a gaze movement based on at least one of the distance between the gaze point and the robot manipulator 10 and the movement speed of the robot manipulator 10.

[0087] In the embodiment, the first learning model, the second learning model, and the recovery learning model are read from the database 40, but they may be read from the storage unit 111.

[0088] The hardware configuration shown in Fig. 1 is an example and is not limited to this. In addition, although Fig. 1 and Fig. 2 show that the information processing device 30 can also function as a learning device, the information processing device 30 and the learning device may be realized by physically different devices.

[0089] (References) [1] T. Zhang, Z. McCarthy, O. Jow, D. Lee, X. Chen, K. Goldberg, and P. Abbeel, “Deep imitation learning for complex manipulation tasks from virtual reality teleoperation,” in International Conference on Robotics and Automation. IEEE, 2018, pp. 1-8. [2] P.-C. Yang, K. Sasaki, K. Suzuki, K. Kase, S. Sugano, and T. Ogata, “Repeatable folding task by humanoid robot worker using deep learning,” Robotics and Automation Letters, vol. 2, no. 2, pp. 397-403, 2016. [3] J. Paillard, “Fast and slow feedback loops for the visual correction of spatial errors in a pointing task: a reappraisal,” Canadian journal of physiology and pharmacology, vol. 74, no. 4, pp. 401-417, 1996. [4] H. Kolb, “Simple anatomy of the retina,” Webvision: The Organization of the Retina and Visual System [Internet]., pp. 13-36, 1995. [Online]. Available: https: / / www.ncbi.nlm.nih.gov / books / NBK11533 / [5] M. Hayhoe and D. Ballard, “Eye movements in natural behavior,” Trends in Cognitive Sciences, vol. 9, pp. 188-94, 2005. [6] A. J. de Brouwer, J. P. Gallivan, and J. R. Flanagan, “Visuomotor feedback gains are modulated by gaze position,” Journal of neurophysiology, vol. 120, no. 5, pp. 2522-2531, 2018. [7] U. Sailer, J. R. Flanagan, and R. S. Johansson, “Eye-hand coordination during learning of a novel visuomotor task,” Journal of Neuroscience, vol. 25, no. 39, pp. 8833-8842, 2005. [8] D. S¨afstr¨om, R. S. Johansson, and J. R. Flanagan, “Gaze behavior when learning to link sequential action phases in a manual task,” Journal of vision, vol. 14, no. 4, pp. 3-3, 2014. [9] F. Sarlegna, J. Blouin, J.-L. Vercher, J.-P. Bresciani, C. Bourdin, and G. M. Gauthier, “Online control of the direction of rapid reaching movements,” Experimental Brain Research, vol. 157, no. 4, pp. 468-471, 2004.

[10] V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” nature, vol. 518, no. 7540, pp. 529-533, 2015.

[11] C. Finn, X. Y. Tan, Y. Duan, T. Darrell, S. Levine, and P. Abbeel, “Deep spatial autoencoders for visuomotor learning,” in International Conference on Robotics and Automation. IEEE, 2016, pp. 512-519.

[12] S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” The Journal of Machine Learning Research, vol. 17, no. 1, pp. 1334-1373, 2016.

[13] H. Kim, Y. Ohmura, and Y. Kuniyoshi, “Using human gaze to improve robustness against irrelevant objects in robot manipulation tasks,” IEEE Robotics and Automation Letters, vol. 5, no. 3, pp. 4415-4422, 2020.

[14] D. Chotrov, Z. Uzunova, Y. Yordanov, and S. Maleshkov, “Mixedreality spatial configuration with a zed mini stereoscopic camera,” 11 2018.

[15] C. M. Bishop, “Mixture density networks,” Neural Computing Research Group, Aston University, Tech. Rep., 1994.

[16] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.

[17] X. Glorot, A. Bordes, and Y. Bengio, “Deep sparse rectifier neural networks,” in Proceedings of the fourteenth international conference on artificial intelligence and statistics, 2011, pp. 315-323.

[18] L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han, “On the variance of the adaptive learning rate and beyond,” arXiv preprint arXiv:1908.03265, 2019.

Claims

1. An information processing device for imitation learning of a learning model for controlling a manipulator that executes a task on a target, an observation image acquisition unit that acquires an observation image of the target and the manipulator at each point in time of the task that is executed by a human operating the manipulator; a gaze point acquisition unit that acquires gaze point information indicating a gaze point at which the person is gazing in the observed image; a surrounding image generating unit that generates a surrounding image by reducing the resolution of the observation image; a gaze image generating unit that generates a gaze image of the observed image, the gaze image being configured by an image portion of a predetermined range including the gaze point in the gaze point information; a classification unit that classifies a motion of the manipulator in each of the observation images at each time point of the task into whether the motion corresponds to a gaze motion performed using a relatively high-resolution gaze field when the task is assumed to be performed by a human, or a peripheral motion performed using a relatively low-resolution peripheral field when the task is assumed to be performed by a human; a teacher data generating unit that generates first teacher data including a pair of the peripheral image based on an observation image in which a motion of the manipulator is classified as the peripheral motion and values ​​indicating a position and orientation of the manipulator in the peripheral motion, and a second teacher data including a pair of the gaze image based on an observation image in which a motion of the manipulator is classified as the gaze motion and values ​​indicating the position and orientation in the gaze motion; a learning unit that imitates and learns a first learning model that outputs target values ​​of the position and orientation when the peripheral image is input based on the first teacher data, and imitates and learns a second learning model that outputs the target values ​​when the gaze image is input based on the second teacher data; An information processing device comprising:

2. An information processing device for controlling a manipulator that performs a task on a target, an observation image acquisition unit that acquires an observation image of the target and the manipulator for each time point of the task; a gaze point acquisition unit that acquires gaze point information indicating a gaze point at which a person gazes when performing the task in the observed image; a surrounding image generating unit that generates a surrounding image by reducing the resolution of the observation image; a gaze image generating unit that generates a gaze image of the observed image, the gaze image being configured by an image portion of a predetermined range including the gaze point in the gaze point information; a classification unit that classifies a motion of the manipulator in each of the observation images at each time point of the task into whether the motion corresponds to a gaze motion performed using a relatively high-resolution gaze field when the task is assumed to be performed by a human, or a peripheral motion performed using a relatively low-resolution peripheral field when the task is assumed to be performed by a human; a learning model acquisition unit that acquires a first learning model that has been imitation-learned based on first teacher data including a pair of the peripheral image based on an observation image in which the movement of the manipulator is classified as the peripheral movement and values ​​indicating the position and posture of the manipulator in the peripheral movement, the first learning model outputting target values ​​of the position and posture when the peripheral image is input; and a second learning model that has been imitation-learned based on second teacher data including a pair of the gaze image based on an observation image in which the movement of the manipulator is classified as the gaze movement and values ​​indicating the position and posture in the gaze movement, the second learning model outputting the target value when the gaze image is input; an operation control unit that, for an observed image classified as the peripheral operation, inputs the peripheral image into the first learning model to obtain the target value output from the first learning model, and, for an observed image classified as the gaze operation, inputs the gaze image into the second learning model to obtain the target value output from the second learning model, and controls the manipulator based on the obtained target value; An information processing device comprising:

3. The information processing device according to claim 1 , wherein the classification unit classifies the motion as either the peripheral motion or the gaze motion based on at least one of a distance between the gaze point and the manipulator and a motion speed of the manipulator.

4. 3. The information processing device according to claim 2, wherein the classification unit inputs the gaze image into an action classifier that outputs a classification as to whether the gaze image corresponds to the peripheral action or the gaze action when the gaze image is input, and classifies the gaze image as either the peripheral action or the gaze action based on the classification result output from the classifier.

5. a determination unit that determines whether or not a recovery operation is necessary, which is an operation from a state in which the task has failed to a state in which the task is next attempted, based on the gaze image; The information processing device described in claim 2 or 4, wherein when it is determined that the recovery action is necessary, the action control unit acquires the target value for the recovery action from the recovery learning model by inputting the gaze image into a recovery learning model that outputs the target value for the recovery action when the gaze image is input, and controls the manipulator based on the acquired target value for the recovery action.

6. When it is determined that the recovery action is necessary, the action control unit obtains the number of steps from a recovery step predictor that outputs the number of steps required to complete the recovery action when the gaze image is input; The recovery learning model can further input the number of steps, The information processing device according to claim 5 , wherein the movement control unit acquires a target value for the recovery movement from the recovery learning model by inputting the gaze image and the number of steps to the recovery learning model.

7. A program for imitating and learning a learning model for controlling a manipulator that executes a task on a target, comprising: an observation image acquisition unit that acquires an observation image of the target and the manipulator at each point in time of the task that is executed by a human operating the manipulator; a gaze point acquisition unit that acquires gaze point information indicating a gaze point at which the person is gazing in the observed image; a surrounding image generating unit that generates a surrounding image by reducing the resolution of the observation image; a gaze image generating unit that generates a gaze image of the observed image, the gaze image being configured by an image portion of a predetermined range including the gaze point in the gaze point information; a classification unit that classifies a motion of the manipulator in each of the observation images at each time point of the task into either a gaze motion that is presumed to be performed using a gaze field of view with a relatively high resolution when the task is assumed to be performed by a human, or a peripheral motion that is presumed to be performed using a peripheral field of view with a relatively low resolution when the task is assumed to be performed by a human; a teacher data generating unit that generates first teacher data including a pair of the peripheral image based on an observation image in which a motion of the manipulator is classified as the peripheral motion and values ​​indicating a position and orientation of the manipulator in the peripheral motion, and a second teacher data including a pair of the gaze image based on an observation image in which a motion of the manipulator is classified as the gaze motion and values ​​indicating the position and orientation in the gaze motion; a learning unit that imitates and learns a first learning model that outputs target values ​​of the position and orientation when the peripheral image is input based on the first teacher data, and imitates and learns a second learning model that outputs the target values ​​when the gaze image is input based on the second teacher data; A program to function as a

8. 1. A program for controlling a manipulator to perform a task on a target, comprising: an observation image acquisition unit for acquiring an observation image of the target and the manipulator; a gaze point acquisition unit that acquires gaze point information indicating a gaze point at which a person gazes when performing the task in the observed image; a surrounding image generating unit that generates a surrounding image by reducing the resolution of the observation image; a gaze image generating unit that generates a gaze image of the observed image, the gaze image being configured by an image portion of a predetermined range including the gaze point in the gaze point information; a classification unit that classifies a motion of the manipulator in each of the observation images at each time point of the task into either a gaze motion that is presumed to be performed using a gaze field of view with a relatively high resolution when the task is assumed to be performed by a human, or a peripheral motion that is presumed to be performed using a peripheral field of view with a relatively low resolution when the task is assumed to be performed by a human; a learning model acquisition unit that acquires a first learning model that has been imitation-learned based on first teacher data including a pair of the peripheral image based on an observation image in which the movement of the manipulator is classified as the peripheral movement and values ​​indicating the position and posture of the manipulator in the peripheral movement, the first learning model outputting target values ​​of the position and posture when the peripheral image is input; and a second learning model that has been imitation-learned based on second teacher data including a pair of the gaze image based on an observation image in which the movement of the manipulator is classified as the gaze movement and values ​​indicating the position and posture in the gaze movement, the second learning model outputting the target value when the gaze image is input; an operation control unit that, for an observed image classified as the peripheral operation, inputs the peripheral image into the first learning model to obtain the target value output from the first learning model, and, for an observed image classified as the gaze operation, inputs the gaze image into the second learning model to obtain the target value output from the second learning model, and controls the manipulator based on the obtained target value; A program to function as a

Citation Information

Patent Citations

  • Real-time object detection and recognition system, and computer-executable program

    JP2005122617A

  • Method and system for training robot using human assisted task demonstration

    JP2013193202A

  • Imaging control device, imaging control method, and imaging apparatus

    WO2017187811A1