Learning support system, learning support method, and learning support program

Abstract

A learning assistance system S comprises: an action control unit 111; a parameter acquisition unit 112; and a display control unit 113. By controlling a control target device on the basis of a user operation, the action control unit 111 causes the control target device to perform an action corresponding to tactile sensing in the user operation. The parameter acquisition unit 112 acquires control parameters that have been used in the control by the action control unit 111. The action control unit 111 and the display control unit 113 present the following parameters to a second user so that the parameters can be compared: a first control parameter acquired by the parameter acquisition unit 112 when the action control unit 111 controlled the action of the control target device on the basis of an operation by a first user; and a second control parameter acquired by the parameter acquisition unit 112 when the action control unit 111 controlled the action of the control target device on the basis of an operation by the second user.

Description

[Technical Field]

[0001] The present invention relates to a learning support system, a learning support method, and a learning support program. [Background technology]

[0002] Generally, when learning various movements, it is effective to observe the movements of an expert of that movement. For example, a learner who wants to learn a movement observes the movements of an expert directly or through a video, etc., and then repeatedly practices by imitating the movements of the expert. In this way, by practicing after observing the movements of an expert, the learner can learn the movement more efficiently than by simply referring to descriptions in a book, etc.

[0003] An example of such technology for supporting learners in mastering movements is disclosed in Patent Document 1. The technology disclosed in Patent Document 1 compares a video of an expert's movements with a video of a learner's movements by pattern matching. This makes it possible to objectively determine whether there is any discrepancy between the expert's movements and the learner's movements, thereby supporting the learner. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-152333 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the general technology such as that disclosed in Patent Document 1, only information (e.g., video) obtained by visually observing the movements of an expert is used to support the learner's acquisition of the movement. In this regard, it is desirable to support the learner's acquisition of the movement more efficiently, rather than simply using information obtained by visually observing an expert.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to support learners in more efficiently acquiring movements. [Means for solving the problem]

[0007] In order to solve the above problems, a learning support system according to one embodiment of the present invention comprises: an operation control means for controlling a control target device based on a user's operation, thereby causing the control target device to perform an operation corresponding to the haptic sensation of the user's operation; parameter acquisition means for acquiring control parameters used in control by the operation control means; a presentation means for presenting to the second user a first control parameter acquired by the parameter acquisition means when the operation control means controls the operation of the control target device based on an operation by a first user, and a second control parameter acquired by the parameter acquisition means when the operation control means controls the operation of the control target device based on an operation by a second user, so that the first control parameter and the second control parameter can be compared; The present invention is characterized by comprising: [Effects of the Invention]

[0008] According to the present invention, it is possible to more efficiently support a learner in mastering a movement. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a block diagram showing an example of the overall configuration of a learning support system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a schematic diagram illustrating a concept of the basic principle of controlling the operation of a controlled device. [Figure 3] This is a schematic diagram showing the concept of control when a force-tactile transmission function is defined. [Figure 4] This is a schematic diagram showing the concept of a master-slave system. [Figure 5]FIG. 10 is a schematic diagram illustrating an example of information stored as an extracted result of a motion. [Figure 6] FIG. 1 is a schematic diagram illustrating the concept of controlling the force-tactile transfer function using scaling in the frequency domain. [Figure 7] FIG. 1 is a schematic diagram showing a basic configuration of a learning assistance device. [Figure 8] FIG. 2 is a block diagram showing hardware and functional blocks for realizing the learning assistance process. [Figure 9] FIG. 10 is a schematic diagram showing an example of a display based on teacher data and execution data. [Figure 10] FIG. 10 is a schematic diagram showing another example of a display based on teacher data and execution data. [Figure 11] FIG. 10 is a schematic diagram showing yet another example of a display based on teacher data and execution data. [Figure 12] FIG. 10 is a schematic diagram showing an example of processing of training data. [Figure 13] FIG. 10 is a schematic diagram showing another example of processing of training data. [Figure 14] FIG. 10 is a schematic diagram showing an example of an auxiliary display screen using audiovisual auxiliary data. [Figure 15] 10 is a flowchart illustrating the flow of a teacher data acquisition process executed by a learning support system according to one embodiment of the present invention. [Figure 16] 1 is a flowchart illustrating a flow of a learning assistance process executed by a learning assistance system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] An example of an embodiment of the present invention will now be described with reference to the accompanying drawings.

[0011] [System Configuration] Fig. 1 is a block diagram showing the overall configuration of a learning assistance system S according to this embodiment. As shown in Fig. 1, the learning assistance system S includes a learning assistance device 1a, a learning assistance device 1b, a learning assistance device 1c, and a terminal 2. Also shown in the figure are an end effector 3, a work object 4, a virtual target point 5, a virtual directional sense 6, a virtual no-entry area 7, a virtual wall 8, a virtual wall 9, a user Ua, and a user Ub. In the following description, when the learning assistance device 1a, the learning assistance device 1b, and the learning assistance device 1c are described without distinction, the suffixes of the reference numerals will be omitted and they will simply be referred to as "learning assistance device 1."

[0012] These learning assistance devices 1 are connected to each other so that they can communicate with each other. Furthermore, the learning assistance devices 1 and the terminals 2 are also connected to each other so that they can communicate with each other. Communication between these devices may be performed in accordance with any communication method, and the communication method is not particularly limited. Furthermore, communication between these devices may be performed wirelessly, by wired communication, or by a combination of wired and wireless communication. In addition, communication between these devices may be performed directly between the devices, or may be performed via a relay device (not shown), or a network such as a LAN (Local Area Network), the Internet, or a mobile phone network.

[0013] Each learning support device 1 is a device that uses the learning support system S to support a learner who is trying to learn the movements of a predetermined action. In the following, as an example for the purpose of explanation, it is assumed that each learning support device 1 is realized by a robot manipulator that can be remotely controlled (i.e., teleoperated) by a user. It is also assumed that a learner uses the learning support system S to learn the actions involved in remotely controlling the learning support device 1. It is assumed that user Ua, who is a learner, operates learning support device 1a, user Ub, who is an expert in remote operation using learning support device 1, operates learning support device 1b, and learning support device 1c performs tasks such as machining as a robot manipulator.

[0014] The learning support devices 1 communicate with each other, and the operation control units included in each learning support device 1 control the operations, so that one device operates as a master device (e.g., a device operated by a user) and the other operates as a slave device (e.g., a device that performs work such as machining as a robot manipulator). In this case, the user operates the master device while observing the operation of the slave device directly or through a video, etc., thereby achieving remote control. During this remote control, the operation control units included in each learning support device 1 control the operations, so that the operation of the master device is transmitted to the slave device and the function of feeding back the input of reaction force from an object to the slave device to the master device (i.e., a bilateral control function) is realized. It should be noted that which of the learning support devices 1 operates as a master device or a slave device will change depending on the situation, and therefore, the following will be explained as the situation dictates.

[0015] The learning support device 1c includes an end effector 3 for performing work such as machining. The learning support device 1c performs work such as machining on a workpiece 4, which is a workpiece, using the end effector 3. The content of this work such as machining is not particularly limited, but for example, the end effector 3 functions as a cutting tool to perform cutting work on the workpiece 4.

[0016] Furthermore, when working with the learning assistance device 1c, a virtual target point 5, a virtual directional feel 6, a virtual no-entry area 7, a virtual wall 8, and a virtual wall 9 are set to assist the user Ua, who is a learner, in learning the movement. These will be described in detail later.

[0017] The terminal 2 is a device that presents various information and a user interface to the user Ua, who is a learner, to assist in the acquisition of actions. The presentation by the terminal 2 is realized, for example, by displaying on a display provided in the terminal 2, outputting sound (warning sounds, voice, etc.) from a speaker provided in the terminal 2, or by flashing a warning light provided in the terminal 2. The terminal 2 also has an input unit such as various buttons, a keyboard, or a touch panel, and receives various operation instructions from the user Ua through this input unit. Note that these processes by the terminal 2 are realized by communication between the terminal 2 and the learning assistance device 1 (here, the learning assistance device 1a) and cooperation between them.

[0018] Next, an outline of the process for supporting a learner by the learning support system S having such a configuration will be explained. First, the learning assistance system S controls (here, each learning assistance device 1) a control target device based on a user's operation (here, control using a bilateral control function), thereby making the control target device perform an action corresponding to the haptic sensation of the user's operation. The learning assistance system S also acquires control parameters used in this control. Then, the learning assistance system S presents to the second user (here, user Ub, an expert) a first control parameter acquired when the action of the control target device is controlled based on the action of the first user, and a second control parameter acquired when the action of the control target device is controlled based on the action of the second user (here, user Ua, a learner), so that the first and second control parameters can be compared.

[0019] In this way, the learning support system S acquires the control parameters used to make the controlled device perform actions corresponding to the haptics of the users' operations for each of the first and second users, and then presents the control parameters of each user to the second user so that they can be compared. This allows the second user to understand whether there are any differences between the first user and the second user in the actions accompanied by haptics, and the degree of the differences. Therefore, the learning support system S can more efficiently support the learner in learning the movements.

[0020] Furthermore, since the learning support system S provides support using control parameters related to haptics in this way, it is possible to solve the problem with the general technology mentioned above, which is that "the learner's acquisition of a movement is merely supported by using only information (e.g., video) obtained by visually observing the movement of an expert."

[0021] [Operation control for controlled devices] Next, as a premise for the operation learning support in the above-described learning support system S, the basic principle of operation control of the control target device (here, each learning support device 1) in this embodiment will be described.

[0022] Human movements (i.e., human physical actions) are composed of individual "functions" of a single joint or the like, either alone or in combination. Therefore, in the following description of the present embodiment, "motion" refers to an integrated function realized by using the individual "functions" of parts of the human body as components. For example, a motion involving operation of the learning support device 1 (e.g., the motion of moving each learning support device 1 by hand) is an integrated function that uses the functions of the fingers and wrist of the hand, and the joints of the arm and shoulder connected to them as components.

[0023] (Basic principle) The basic principle of motion control in this embodiment is that any motion can be mathematically expressed using three elements: a force source, a velocity (position) source, and a transformation that represents the motion. Therefore, by supplying control energy to the system to be controlled from an ideal force source and an ideal velocity (position) source that are in a dual relationship to a group of variables defined by the transformation and inverse transformation, the extracted motion is structured and reconstructed or expanded and amplified, thereby automatically realizing (reproducing) the motion in a reversible manner.

[0024] FIG. 2 is a schematic diagram showing the concept of the basic principle of control of the operation of a controlled device in this embodiment. The basic principle shown in Figure 2 represents the control law of an actuator (here, the actuator provided in the learning support device 1) that can be used to realize human movement, and determines the movement of the actuator by using the current position of the actuator as input and performing calculations in at least one of the areas of position (or velocity) and force. In other words, the basic principle of controlling the operation of the controlled device in this embodiment is expressed as a control law including the controlled system CS, a functional force-velocity allocation conversion block FT, at least one of an ideal force source block FC or an ideal velocity (position) source block PC, and an inverse conversion block IFT.

[0025] The controlled system CS is a robot (here, the learning assistance device 1) that operates using an actuator, and controls the actuator based on acceleration, etc. Here, the controlled system CS realizes the functions of one or more parts of the human body, but the specific configuration does not necessarily have to be a form that imitates the human body as long as a control law for realizing that function is applied. For example, the controlled system CS can be a robot that uses an actuator to make a link perform a one-dimensional sliding motion.

[0026] The functional force-velocity allocation transformation block FT defines the transformation of control energy into the velocity (position) and force domains set according to the function of the controlled system CS. Specifically, the functional force-velocity allocation transformation block FT defines a coordinate transformation that takes as input the reference value (base value) of the function of the controlled system CS and the current position of the actuator. This coordinate transformation generally converts an input vector whose elements are the base value and current velocity (position) into an output vector consisting of velocity (position) for calculating the target velocity (position) control value, and also converts an input vector whose elements are the base value and current force into an output vector consisting of force for calculating the target force control value. Specifically, the coordinate transformation in the functional force-velocity allocation transformation block FT is generalized and expressed as the following equations (1) and (2).

[0027]

number

[0028] However, in formula (1), x'1 to x' n (n is an integer greater than or equal to 1) is the velocity vector for deriving the velocity state value, and x' a ~x' m (m is an integer of 1 or more) is a vector whose elements are the reference value and the velocity based on the action of the actuator (the velocity of the actuator's moving element or the velocity of the object moved by the actuator), 1a ~h nm is an element of the transformation matrix that represents the function. Also, in equation (2), f''1~f'' n (n is an integer greater than or equal to 1) is the force vector for deriving the force state value, and f'' a ~f'' m (m is an integer of 1 or more) is a vector whose elements are a reference value and a force based on the action of the actuator (the force of the actuator's moving element or the force of the object moved by the actuator).

[0029] By setting the coordinate transformation in the functional force / velocity allocation transformation block FT according to the function to be realized, various operations can be realized and operations involving scaling can be reproduced. That is, in the basic principle of controlling the operation of the controlled device in this embodiment, the functional force-velocity allocation conversion block FT "converts" the variables of a single actuator (variables in real space) into a group of variables of the entire system (variables in virtual space) that express the functions to be realized, and allocates control energy to the control energy of velocity (position) and the control energy of force. Therefore, compared to when control is performed using the variables of a single actuator (variables in real space), it is possible to assign the control energy of velocity (position) and the control energy of force independently.

[0030] The ideal force source block FC is a block that performs calculations in the force domain according to the coordinate transformation defined by the functional force-velocity allocation transformation block FT. In the ideal force source block FC, a target value for force is set when performing calculations based on the coordinate transformation defined by the functional force-velocity allocation transformation block FT. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero can be set as the target value, or when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be reproduced can be set.

[0031] The ideal velocity (position) source block PC is a block that performs calculations in the velocity (position) domain according to the coordinate transformation defined by the functional force-velocity allocation transformation block FT. In the ideal velocity (position) source block PC, a target value for velocity (position) is set when performing calculations based on the coordinate transformation defined by the functional force-velocity allocation transformation block FT. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, when realizing a function similar to the function indicated by the reference value, zero can be set as the target value, or when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be reproduced can be set.

[0032] The inverse transformation block IFT is a block that transforms values ​​in the domain of velocity (position) and force into values ​​in the domain of input to the controlled system CS (for example, voltage values ​​or current values). Based on this basic principle, when position information of the actuator of the controlled system CS is input to the functional force-velocity allocation conversion block FT, the functional force-velocity allocation conversion block FT applies the control rules for each position and force region according to the function using the velocity (position) and force information obtained based on the position information. Then, the ideal force source block FC calculates the force according to the function, and the ideal velocity (position) source block PC calculates the velocity (position) according to the function, and the control energy is distributed to each of the force and velocity (position).

[0033] The calculation results in the ideal force source block FC and the ideal velocity (position) source block PC become information indicating the control target of the controlled system CS, and these calculation results are used as input values ​​for the actuators in the inverse transformation block IFT and input to the controlled system CS. As a result, the actuators of the controlled system CS perform operations according to the functions defined by the functional force-velocity allocation transformation block FT, and the desired robot motion is realized. That is, in this embodiment, the robot (here, each learning assistance device 1) can more appropriately realize the movements of a human being during a predetermined action.

[0034] (Example of defined functions) Next, a specific example of a function defined by the functional force / speed allocation transformation block FT will be described. The functional force / velocity allocation transformation block FT defines coordinate transformation (conversion from real space to virtual space corresponding to the function to be realized) for the velocity (position) and force obtained based on the current position of the input actuator. In the functional force-velocity allocation conversion block FT, the velocity (position) and force from the current position and the velocity (position) and force as the reference value of the function are input, and the control laws for the velocity (position) and force are applied in the acceleration dimension. In other words, the force of the actuator is expressed as the product of mass and acceleration, and the velocity (position) of the actuator is expressed as the integral of acceleration. Therefore, by controlling the velocity (position) and force through the acceleration domain, the current position of the actuator can be obtained and the desired function can be achieved.

[0035] Specific examples of the various functions will be described below. (force and tactile transmission function) Fig. 3 is a schematic diagram showing the concept of control when a force-tactile transmission function is defined in the functional force-velocity allocation conversion block FT. Fig. 4 is a schematic diagram showing the concept of a master-slave system including a master device (e.g., one of the learning assistance devices 1) and a slave device (e.g., another learning assistance device 1) to which the force-tactile transmission function is applied. As shown in Figure 4, the function defined by the functional force-velocity allocation conversion block FT is to transmit the operation of the master unit to the slave unit, and also to feed back the reaction force input from an object (e.g., work target 4) to the master unit (bilateral control function). In this case, the coordinate transformation in the functional force-speed allocation transformation block FT is expressed as the following equations (3) and (4).

[0036]

number

[0037] However, in formula (3), x' p is the velocity for deriving the velocity (position) state value, x' f is the velocity with respect to the state value of the force, and x' m is the speed (differential value of the current position of the master unit) of the reference value (input from the master unit), x' sis the current velocity of the slave unit (the differential value of the current position). p is the force related to the velocity (position) state value, f f is the force for deriving the state value of the force. m is the reference force (input from the master device), f s is the current force of the slave device.

[0038] (Action extraction function) Next, the human motion extraction function realized by this embodiment will be described in detail. By following the above-mentioned (basic principle), it is possible to construct a robot with a specific mechanism (for example, a robot with a mechanism that corresponds to the functions of the human body), have this robot follow human movements, and detect the time-series movements of this robot at that time, thereby extracting human movements.

[0039] For example, in the case of a learning support device 1 as shown in Figure 1, when a person moves their hand to operate the learning support device 1, the positions of each of the multiple actuators (shown in Figure 7 described later) equipped in the learning support device 1 are detected in time series by multiple position sensors (shown in Figure 7 described later), and these positions are stored in a storage device. In this case, instead of the detection results of the positions of the actuators, each value (for example, each value on the left side of equation (4) calculated in time series) for deriving the state value obtained as a result of coordinate transformation in the functional force-velocity allocation transformation block FT may be stored in a storage device.

[0040] FIG. 5 is a schematic diagram showing an example of information stored as an extracted result of an operation, where FIG. 5(a) shows a case where the time series positions of multiple actuators (here, actuators A1 to A4) are stored, and FIG. 5(b) shows a case where the time series coordinate transformation result of equation (4) is stored. Referring to FIG. 5(a), for example, for actuator A1, the positions are stored in time series as position p1 at time t1, position p2 at time t2, position p3 at time t3, . . . Also, referring to FIG. 5(b), for example, the coordinate transformation result x'' a1 The coordinate transformation result q1 at time t1, the coordinate transformation result q2 at time t2, the coordinate transformation result q3... at time t3 are stored as time series values. This allows a human to actually perform a predetermined action in a predetermined behavior only once, and the robot can then learn and reproduce the predetermined action, even if the human does not perform the action again.

[0041] (scaling function) In the above force-tactile transmission function, the scaling functions of position, force and time can be further realized. The scaling function is a function that expands or reduces the scale of the output position, force, or time relative to the reference control. The scaling function can, for example, reduce the magnitude of the master device's movement and reproduce it on the slave device, increase the strength (force) of the master device's movement and reproduce it on the slave device, or reduce the speed of the master device's movement and reproduce it on the slave device. Furthermore, by using the scaling function on at least one of the position and force information stored in the storage device, it is possible to, for example, reduce the magnitude of the stored movement and reproduce it on the slave device, or increase the strength (force) of the stored movement and reproduce it on the slave device. An example of a configuration for realizing the scaling function will be described below.

[0042] (Force-tactile transmission function with scaling) When a force-tactile transmission function involving scaling is realized, the coordinate transformation in the functional force-velocity allocation transformation block FT in FIG. 2 is expressed as the following equations (5) and (6).

[0043]

number

[0044] When the coordinate transformation shown in Equation (5) and Equation (6) is performed, the position of the slave device is multiplied by α (α is a positive number), and the force of the slave device is multiplied by β (β is a positive number) and transmitted to the master device. Such a scaling function is effective when performing delicate operations such as surgery and microassembly, or large-scale operations such as civil engineering work and extravehicular activities.

[0045] (Force and tactile transmission function with position limitation by scaling) When the force and tactile transmission function with position limitation by scaling is realized, the coordinate transformation in the function-specific force and speed allocation conversion block FT in FIG. 2 is represented by, for example, the following Equations (7) to (10). When realizing such a function, it is appropriate to consider the following conditions. ·Be continuous up to the velocity dimension (existence condition of the Jacobian matrix) ·The position after limitation is a monotonically increasing function of the original position (stability condition) ·x s When <a, then x s =x shat Or x s ≒x shat (x shat is a parameter included in the function-specific force and speed allocation conversion block FT in Equations (9) and (10)) (Condition for guaranteeing control performance in the safety region) ·Be a saturation function (condition for realizing position limit) As another function that satisfies these conditions, it is also possible to adopt the atan function.

[0046]

Equation

[0047] When the coordinate transformations shown in equations (7) to (10) are used, if the position of the slave device is less than a, the slave device and the master device are controlled to the same position by applying the coordinate transformations of equations (7) and (8). On the other hand, if the position of the slave device is a or greater, applying the coordinate transformations of equations (9) and (10) activates a scaling function, and the slave device is controlled so as not to exceed the position of (1 / b+a). Such a scaling function is useful, for example, in protecting organs during surgery and preventing breakage of objects.

[0048] (Force-tactile transmission function using scaling in the frequency domain) FIG. 6 is a schematic diagram showing the concept of controlling the force-tactile transfer function using scaling in the frequency domain. In FIG. 6, the outputs of the master device and the slave device are input to a functional force-speed allocation conversion block FT after passing through a high-pass filter (HPF) and a low-pass filter (LPF), respectively. The functional force / speed allocation transformation block FT applies coordinate transformation for the high frequency range to the outputs of the master and slave units that have passed through a high-pass filter, and applies coordinate transformation for the low frequency range to the outputs of the master and slave units that have passed through a low-pass filter. That is, the functional force-speed allocation transformation block FT separates the inputs from the master and slave devices into high-frequency and low-frequency signals, and applies coordinate transformations corresponding to each frequency range.

[0049] As shown in FIG. 6, when a force-tactile transmission function is realized using scaling in the frequency domain, the coordinate transformation in the functional force-velocity allocation transformation block FT is expressed as the following equations (11) to (14).

[0050]

number

[0051] When the coordinate transformations shown in equations (11) to (14) are used, in the low-frequency range, the slave unit and the master unit are controlled to be in the same position by applying the coordinate transformations of equations (11) and (12), and in the high-frequency range, the position of the slave unit is multiplied by α (α is a positive number) and the force of the slave unit is multiplied by β (β is a positive number) and transmitted to the master unit by applying the coordinate transformations of equations (13) and (14). Such a scaling function enables, for example, the slave device to transmit to the master device an enhanced sensation when an object is penetrated or broken.

[0052] (Reproduction of function using time scaling) When reproducing a function using the learning support device 1, time scaling can be achieved by thinning out or interpolating information indicating the learned and stored function (for example, time-series data representing the extracted action results shown in Figure 5) to set it to a target value. Specifically, the information indicating the learned and stored function is thinned out and then used as the target value for the calculation in the ideal force source block FC or the ideal velocity (position) source block PC, thereby enabling the stored function (operation) to be reproduced at high speed. Similarly, the information indicating the learned and stored function is interpolated and then used as the target value, thereby enabling the stored function (operation) to be reproduced at low speed. In this way, when reproducing a stored function (movement) at high speed, it is sufficient to perform the movement slowly and accurately when extracting the action, and then perform the movement quickly and accurately when reproducing it. Furthermore, when memorized functions (movements) are reproduced at a slow speed, movements performed at normal speeds can be reproduced slowly, making it possible to reproduce training movements tailored to the patient, for example, in the rehabilitation of a patient with a disability.

[0053] [Configuration of learning support device] Next, the configuration of the learning assistance device 1 will be described with reference to Fig. 7. Fig. 7 is a schematic diagram showing the basic configuration of the learning assistance device 1. As shown in FIG. 7, the learning assistance device 1 includes a control unit 10, a driver 20, an actuator 30, and a position sensor 40.

[0054] As described above with reference to FIG. 1, the learning support device 1 operates as a master device or a slave device. When operating as either a master device or a slave device, the learning support device 1 operates according to its function using the detection result of the position sensor 40 installed on the actuator 30 of the learning support device 1 operating as the other device as input. As described above, the functions implemented in the learning support device 1 can be changed in various ways by switching the coordinate transformation defined by the functional force-speed allocation transformation block FT realized by the control unit 10.

[0055] The control unit 10 controls the entire learning assistance device 1, and is configured by an information processing device including a processor such as a CPU (Central Processing Unit) and a storage device such as a memory or a hard disk. The control unit 10 has the functions of a functional force-velocity allocation conversion block FT, an ideal force source block FC, an ideal velocity (position) source block PC, and an inverse conversion block IFT shown in Figures 2 and 3. The control unit 10 uses these functions to perform control to operate as either a master device or a slave device. To this end, the control unit 10 acquires a reference value (hereinafter referred to as a "reference value") for each function of the learning assistance device 1. For example, when the learning assistance device 1 operates as one of the master and slave devices, this reference value is a time-series detection value output from a position sensor 40 installed on an actuator 30 of the learning assistance device 1 operating as the other device. When the time-series detection value is acquired in real time from the learning assistance device 1 operating as the other device to the control unit 10 as the reference value, the control unit 10 can be configured with a communication interface (communication I / F). Furthermore, when the time-series detection value of the learning assistance device 1 operating as the other device is stored and sequentially read out as the reference value by the control unit 10 to realize the above-mentioned (motion extraction function), the control unit 10 can be configured with a storage device such as a memory or a hard disk.

[0056] That is, first, the time-series detection values ​​detected by the position sensor 40 of the learning support device 1 operating as the other device are input as reference values ​​to the control unit 10. These time-series detection values ​​represent the operation of the learning support device 1 operating as the other device, and the control unit 10 applies coordinate transformation set according to the function to the speed (position) and force information derived from the input detection values ​​(position).

[0057] Then, the control unit 10 performs calculations in the velocity (position) domain on the velocity (position) to derive a velocity (position) state value obtained by the coordinate transformation. Similarly, the control unit 10 performs calculations in the force domain on the force to derive a force state value obtained by the coordinate transformation. Furthermore, the control unit 10 performs dimensional unification processing on the calculated calculation results in the velocity (position) domain and the force domain to convert them into acceleration, etc., and also applies an inverse transformation of the coordinate transformation set according to the function. As a result, the control unit 10 converts the calculated calculation results in the velocity (position) domain and the force domain into values ​​in the domain of the input to the actuator 30.

[0058] Furthermore, the control unit 10 also functions as a functional block for supporting the learner in mastering the movement. This functional block will be described later with reference to FIG.

[0059] The driver 20 converts the value of the input domain to the actuator 30, which has been inversely converted by the control unit 10, into a specific control command value (voltage value, current value, etc.) for the actuator 30, and outputs the control command value to the actuator 30. The actuator 30 is driven in accordance with the control command value input from the driver 20, and controls the position of the device to be controlled. The position sensor 40 detects the position of the device to be controlled by the actuator 30 and outputs the detected value to the control unit 10.

[0060] With this configuration, the learning assistance device 1 converts the velocity (position) and force obtained from the position of the controlled device detected by the position sensor 40 into state values ​​in the velocity (position) domain and force domain by coordinate transformation according to the function. As a result, control energy is distributed to each of the velocity (position) and force according to the function. Then, each state value is inversely converted into a control command value, and the driver 20 drives the actuator 30 according to this control command value.

[0061] Therefore, by detecting the position of the device to be controlled, the learning support device 1 can calculate the state values ​​of the speed (position) and force required to achieve the desired function, and by driving the actuator 30 based on these state values, the position and force of the device to be controlled can be controlled to the desired state.

[0062] Furthermore, the learning assistance device 1 can realize different functions by switching the coordinate transformation according to the function in the control unit 10. For example, by storing coordinate transformations according to various functions in a storage device provided in the learning assistance device 1 and selecting a coordinate transformation according to one of the functions depending on the purpose, it becomes possible to realize various functions in the learning assistance device 1.

[0063] For example, when realizing the function described above as the (force-tactile transmission function), the learning assistance device 1 can use the acquired position and force values ​​input in real time from the learning assistance device 1 operating as the other device as the reference values ​​input to the control unit 10. In this case, the learning assistance device 1 can be controlled in real time in conjunction with the operation of the learning assistance device 1 operating as the other device. That is, in this case, the control unit 10 defines the coordinate transformation expressed by equation (2), so that the difference between the position of the actuator 30 of the learning assistance device 1 operating as the master device and the position of the actuator 30 of the learning assistance device 1 operating as the slave device is controlled to be zero.

[0064] Furthermore, when the above-described function (force-tactile transmission function) is realized, the haptic sensation generated by the operation performed by the operator on the actuator 30 of the learning support device 1 operating as the master device is transmitted to the slave device, and the reaction force from an object (e.g., the work object 4) acting on the actuator 30 of the learning support device 1 operating as the slave device is fed back to the master device. This allows the operation performed on the learning support device 1 operating as the master device to be accurately reproduced by the learning support device 1 operating as the slave device, and also allows the reaction force from the object input to the learning support device 1 operating as the slave device to be accurately transmitted to the learning support device 1 operating as the master device.

[0065] Additionally, for example, when realizing the function described above as (motion extraction function), the learning assistance device 1 can use the previously acquired and stored time-series position and force values ​​of the learning assistance device 1 operating as the other device as the reference values ​​input to the control unit 10. In this case, the function of the learning assistance device 1 can be realized based on the previously prepared motion of the learning assistance device 1 operating as the other device. In other words, the intended function can be reproduced in the learning assistance device 1 in a state where there is no learning assistance device 1 operating as the other device.

[0066] In addition, for example, when realizing the above-mentioned function (scaling function), the learning support device 1 can use the scaling function to, for example, reduce the magnitude of the movement of the learning support device 1 operating as a master device and reproduce it on the learning support device 1 operating as a slave device, or increase the strength (force) of the movement of the learning support device 1 operating as a master device and reproduce it on the learning support device 1 operating as a slave device, or slow down the speed of the movement of the learning support device 1 operating as a master device and reproduce it on the learning support device 1 operating as a slave device.

[0067] In addition, for example, when both the function described above as (movement extraction function) and the function described above as (scaling function) are realized, the learning support device 1 can use the scaling function on at least one of the position and force information stored in the memory device by the movement extraction function, thereby, for example, reducing the size of the stored movement and reproducing it on the learning support device 1 operating as a slave device, or increasing the strength (force) of the stored movement and reproducing it on the learning support device 1 operating as a slave device.

[0068] As described above, the learning assistance device 1 operating as a master device or a slave device further performs a "learning assistance process." Here, the learning support process is a series of processes that support a learner who is trying to learn the movements of a predetermined action.

[0069] Fig. 8 is a block diagram showing hardware and functional blocks for realizing this learning assistance process. As shown in Fig. 8, a control unit 10 includes a processor 11, a storage unit 12, a ROM 13, a RAM 14, a communication unit 15, an input unit 16, and an output unit 17. As also shown in Fig. 7, a driver 20 and a position sensor 40 are connected to the control unit 10. These units are connected by signal lines and send and receive signals between them.

[0070] The processor 11 executes various processes according to programs recorded in the ROM 13 or programs loaded from the storage unit 12 to the RAM 14. The RAM 14 also stores data and the like required for the processor 11 to execute various processes as appropriate. The storage unit 12 is configured with a semiconductor memory such as a DRAM (Dynamic Random Access Memory) and stores various data.

[0071] The communication unit 15 controls communication between the processor 11 and other devices (e.g., other learning assistance devices 1 and the terminal 2). The input unit 16 is composed of various buttons and a touch panel, or external input devices such as a mouse and keyboard, and inputs various information in response to user instructions. The output unit 17 is composed of a display, a speaker, etc., and outputs images and sounds.

[0072] When the learning support process is realized in such a hardware configuration, an operation control unit 111, a parameter acquisition unit 112, a display control unit 113, a teacher data processing unit 114, a force-tactile support unit 115, and an audiovisual support unit 116 function in the processor 11 as shown in FIG. 8. Furthermore, when implementing the learning support process in such a hardware configuration, as shown in Figure 8, a teacher data memory unit 121, an execution data memory unit 122, and an auxiliary data memory unit 123 are set in one area of ​​the memory unit 12. Including cases not specifically mentioned below, data required to realize processing is transmitted and received between these functional blocks at appropriate times.

[0073] As described above, the movement control unit 111 controls the movement of the learning assistance device 1 operating as a master device or a slave device, applying the force-tactile transmission function. That is, the movement control unit 111 realizes the functions of the functional force-velocity allocation conversion block FT, the ideal force source block FC, the ideal velocity (position) source block PC, and the inverse conversion block IFT in Fig. 2 and Fig. 3. In this case, the movement control unit 111 defines the force-tactile transmission function in the functional force-velocity allocation conversion block FT, as described above with reference to Fig. 3, and controls the movement to which the force-tactile transmission function is applied.

[0074] In addition, as one mode of presentation in the learning support process, the movement control unit 111 controls the movement to which the force-tactile transmission function is applied based on the teacher data (corresponding to the first control parameter) and execution data (corresponding to the second control parameter) acquired by the parameter acquisition unit 112 described below. As described above with reference to FIG. 1, the presentation in the learning support process involves presenting to the learner user Ua in a comparable manner the teacher data obtained when the operation of the learning support device 1b (and the learning support device 1c) is controlled based on the operation of the expert user Ub, and the execution data obtained when the operation of the learning support device 1a (and the learning support device 1c) is controlled based on the operation of the learner user Ua.

[0075] As a comparable presentation, the operation control unit 111 corrects the execution data of the learning support device 1a operated by the user Ua, who is a learner, using the teacher data used to control the operation of the learning support device 1b operated by the expert user Ub. That is, the operation control unit 111 transmits a haptic sensation (i.e., a haptic sensation corresponding to the force applied by the expert user Ub to the actuator 30 of the learning support device 1b through operation) to the learning support device 1a operated by the user Ua using the force-haptic sensation transmission function. This allows the learner to understand whether there are any differences between themselves and the expert in terms of movements involving haptics, and the degree of those differences. Therefore, the presentation by the movement control unit 111 can more efficiently support the learner in mastering the movement.

[0076] The parameter acquisition unit 112 acquires control parameters (hereinafter referred to as "force-haptic control parameters") used by the operation control unit 111 in controlling an operation to which the force-haptic transmission function is applied. Here, these force-haptic control parameters may be any control parameters used by the operation control unit 111 in controlling an operation to which the force-haptic transmission function is applied. For example, the force-haptic control parameters may be the time-series positions of each actuator in the (operation extraction function) described above with reference to FIG. 5(a) or the coordinate transformation result of the time-series equation (4) in the (operation extraction function) described above with reference to FIG. 5(b). In the following, as an example for the purpose of explanation, it is assumed that the parameter acquisition unit 112 acquires the time-series positions of each actuator as the force-haptic control parameters.

[0077] In this case, the parameter acquiring unit 112 stores, as teacher data, in the teacher data storage unit 121, control parameters related to haptics acquired when the expert user Ub operates the learning assistance device 1b as the master device to control the operation of the learning assistance device 1c, which is the slave device. The parameter acquiring unit 112 also stores, as execution data, control parameters related to haptics acquired when the learner user Ua operates the learning assistance device 1a as the master device to control the operation of the learning assistance device 1c, which is the slave device. That is, the teacher data storage unit 121 functions as a storage unit that stores teacher data. The execution data storage unit 122 also functions as a storage unit that stores execution data.

[0078] These teacher data and execution data are used to present the teacher data and execution data to the learner who is the target of the learning assistance process so that they can be compared. In this embodiment, the learner is assumed to be the user Ua. Therefore, the presentation is performed by the learning assistance device 1a (and the terminal 2 connected thereto) used by the user Ua. Therefore, in this embodiment, the parameter acquisition unit 112 of the learning assistance device 1a acquires the teacher data and execution data. Specifically, the parameter acquisition unit 112 of the learning assistance device 1a acquires the teacher data from the operation control unit 111 included in the learning assistance device 1b. Furthermore, the parameter acquisition unit 112 of the learning assistance device 1a acquires the execution data from the operation control unit 111 included in its own device (i.e., the learning assistance device 1a). In this case, the parameter acquisition unit 112 of the learning assistance device 1a may also acquire the teacher data and execution data from the operation control unit 111 included in the learning assistance device 1c. In either case, the acquisition of teacher data and execution data may be performed in real time while the operation of each learning support device 1 is being controlled, or only temporary storage (i.e., buffering) may be performed while the operation is being controlled, and the data may be acquired all at once after the operation control has ended.

[0079] The display control unit 113 performs display based on the teacher data and execution data as one mode of presentation in the learning support process. Additionally, the display control unit 113 displays a user interface related to the learning support process. Note that the display destination by the display control unit 113 is assumed to be a display provided in the terminal 2, but is not limited to this, and the display destination may be a display provided in another device. For example, the learning support device 1 may be provided with a display, and the display may be performed on this display.

[0080] The display based on the teacher data and execution data by the display control unit 113 will be described with reference to Figures 9, 10, and 11. Figures 9, 10, and 11 are schematic diagrams each showing an example of a display based on the teacher data and execution data by the display control unit 113.

[0081] (Overlay display) For example, as shown in FIG. 9, the display control unit 113 realizes at least a part of the presentation by displaying information indicating the time-series changes in the teacher data (shown by dotted lines in the figure) and information indicating the time-series changes in the execution data (shown by solid lines in the figure) in an overlapping manner. Here, as shown in the figure, the teacher data and the execution data are shown as time-series changes in position, velocity, and force, respectively, from top to bottom. Specifically, they are shown as graphs with the vertical axis representing values ​​indicating position, velocity, and force, respectively, from top to bottom, and the horizontal axis representing time. Note that these graphs may actually show more subtle changes, but each figure shows the changes in the graphs in a simplified and schematic manner.

[0082] As described above, in this embodiment, the position of the actuator 30 along a time series is acquired as the teacher data and execution data. Also, as described above, the velocity of the actuator 30 can be expressed by the integral of the acceleration, and the force of the actuator 30 can be expressed by the product of the mass and the acceleration. Therefore, the display control unit 113 can display the changes in the position, velocity, and force along a time series, as shown in FIG. 9, based on the teacher data and execution data acquired by the parameter acquisition unit 112 and the results of calculations such as integration of these.

[0083] By referring to this display, the learner can grasp the presence or absence of differences and the degree of differences between his or her own and the expert's movements involving haptics along a timeline. For example, it becomes possible to grasp at which times the movements are not different, and at which times the movements are significantly different. Therefore, the learner can be more efficiently assisted in mastering the movement not only by the presentation by the movement control unit 111 (i.e., transmission of haptic sensations) but also by the presentation by the display control unit 113 (i.e., display in chronological order). Note that in the following description, it is assumed that both the presentation by the movement control unit 111 and the presentation by the display control unit 113 are performed, but this is not limiting, and only one of the presentations may be performed.

[0084] (axis offset) In the display control unit 113's superimposed display of time-series graphs, further processing can be performed to make it easier for the learner to compare the teacher data and the execution data. For example, it is possible to adjust the offset (i.e., the offset) between the teacher data and the execution data on the vertical and horizontal axes in response to user operations, etc., before displaying the data. For example, there may be cases where the teacher data and the execution data do not differ significantly in terms of changes in values ​​indicating position, velocity, and force (i.e., the vertical axis), or the timing of the changes (i.e., the horizontal axis) may differ. In such cases, adjusting the offset in the timing of the changes (i.e., the horizontal axis) before comparing the graphs allows for a more appropriate comparison of the differences in the changes in values ​​indicating position, velocity, and force, rather than comparing the graphs in time series as is.

[0085] Therefore, for example, when there is teacher data and execution data as shown in FIG. 9, the execution data or teacher data is shifted to the horizontal axis and displayed. For example, as shown in FIG. 10, the execution data is shifted to the horizontal axis and displayed. By displaying in this manner, it is possible to more appropriately compare the differences in changes in values ​​indicating position, velocity, and force. Based on the same idea, it is also possible to shift the execution data or teacher data to the vertical axis and display it, or to shift it to both the vertical and horizontal axes and display it.

[0086] (Quantification of comparison) Furthermore, when the display control unit 113 displays graphs overlaid in a time series, in addition to the comparative display of overlaid graphs, it may also display a comparative display in which the degree of mismatch or match between the teacher data and the execution data is quantified for an arbitrary timing and an arbitrary time span. In this case, for example, an arbitrary time span at an arbitrary timing may be selected in response to a user operation, etc. For example, the display control unit 113 may accept the selection of the time span to be quantified, as shown in the hatched area in FIG. 11 . The display control unit 113 then calculates the comparison results for the position, velocity, and force by quantifying the degree of mismatch between the teacher data and the execution data for the selected time span. The comparison results are then displayed, for example, as shown as "error values" in FIG. 11 . This allows the learner to compare the quantified results of the mismatch. Examples of methods for quantifying the degree of mismatch between the teacher data and the execution data include a method of calculating the degree of mismatch between the teacher data and the execution data using least squares error, and a method of calculating the degree of mismatch between the teacher data and the execution data based on an index such as cosine similarity.

[0087] The display control unit 113 may perform any one of the above-mentioned (superimposed display), (axis offset), and (quantification of comparison), but may also perform a combination of these depending on, for example, the operation of the user Ua, who is the learner.

[0088] Prior to presentation, the teacher data processing unit 114 processes the teacher data used in the presentation by the operation control unit 111 and the presentation by the display control unit 113 from the actual teacher data acquired by the parameter acquisition unit 112. For example, the teacher data processing unit 114 processes the teacher data when it is set to perform processing in advance or when a processing instruction operation is received from the user Ua, who is a learner, or the like. The processing of the teacher data by the teacher data processing unit 114 will be described with reference to Fig. 12 and Fig. 13. Fig. 12 and Fig. 13 are schematic diagrams each showing an example of processing of the teacher data by the teacher data processing unit 114.

[0089] (position-force scaling) The teacher data processing unit 114 processes the teacher data by, for example, scaling the teacher data so as to enlarge or reduce the position and force values ​​in the teacher data. This scaling can be realized by the teacher data processing unit 114 applying the above-mentioned force-tactile transmission function with scaling to the teacher data in response to an operation by the user Ua, who is the learner, or the like.

[0090] In this case, for example, as shown in FIG. 12, scaling is performed to enlarge the values ​​indicating the position and force in the training data. By performing such scaling, the movements corresponding to the positions and forces in the training data become rougher and more general than the delicate movements performed by an expert. In other words, from the learner's perspective, the movements are easier to learn. After the training data processing unit 114 performs such scaling, the movement control unit 111 and the display control unit 113 present the movements, allowing the learner to easily learn the movements. In this way, in the early stages of learning, the scaling ratio of the position and force values ​​is set high to lower the difficulty of learning. Then, as the learner's learning progresses, the scaling ratio of the position and force values ​​is gradually brought closer to that of the original training data, increasing the difficulty of learning. This makes it possible to more efficiently support the learner in learning the movements.

[0091] (Time scaling) The teacher data processing unit 114 processes the teacher data by, for example, scaling the teacher data so that changes in values ​​indicating position or force in the teacher data become longer or shorter over time. This scaling can be realized by the teacher data processing unit 114 applying the above-mentioned process (reproduction of a function using time scaling) to the teacher data in response to an operation by the user Ua, who is the learner, or the like.

[0092] In this case, for example, as shown in FIG. 13 , scaling is performed so that the changes in values ​​indicating position and force in the training data are lengthened over time. By performing such scaling, the movements corresponding to the positions and forces in the training data become slower and gentler than the quick movements performed by an expert. In other words, from the learner's perspective, the movements are less difficult to learn. After the training data processing unit 114 performs such scaling, the motion control unit 111 and the display control unit 113 present the movements, allowing the learner to easily learn the movements. In this way, in the early stages of learning, the degree to which the changes in values ​​indicating position and force are lengthened over time in the scaling is set large, thereby lowering the difficulty of learning. Then, as the learner's learning progresses, the degree to which the changes in values ​​indicating position and force are lengthened over time is gradually brought closer to that of the original training data, thereby increasing the difficulty of learning. This makes it possible to more efficiently support the learner in learning the movements.

[0093] (Editing teacher data) The teacher data processing unit 114 processes the teacher data by, for example, extracting portions of the teacher data that have characteristic changes in values ​​indicating position or force, and editing the extracted portions by repeating the extracted portions or changing the order of the extracted portions. This editing can be realized by the teacher data processing unit 114 editing the teacher data in response to operations by the user Ua, who is the learner, or the like.

[0094] In this case, for example, in a series of movements of an expert user Ub, position and force changes corresponding to movement parts that are particularly important to master or movement parts that are difficult to master are extracted, and the training data is edited so that this is repeated. This makes it possible to more efficiently support the learner in mastering the movement parts that are particularly important to master or movement parts that are difficult to master.

[0095] The teacher data processing unit 114 may perform any of the above-mentioned (position / force scaling), (time scaling), and (teacher data editing), but may also perform a combination of these depending on, for example, the operation of the learner user Ua, etc.

[0096] The haptic assistance unit 115 acquires haptic assistance data, which is a control parameter for controlling the learning assistance device 1a to assist the operation of the user Ua, who is a learner. The haptic assistance unit 115 then corrects the execution data of the learning assistance device 1a operated by the user Ua, who is a learner, using the haptic assistance data. In other words, the operation control unit 111 transmits haptics (i.e., haptics for controlling the learning assistance device 1a to assist the operation of the user Ua, who is a learner) to the learning assistance device 1a operated by the user Ua using the force-haptic transmission function. This allows the learner's movements to be haptically assisted, and the learner's acquisition of movements can be supported more efficiently.

[0097] For example, the haptic assistance unit 115 determines to provide haptic assistance when it is set to provide haptic assistance in advance or when a condition for providing haptic assistance is met based on the operation of the learning assistance device 1a operating as the master device in this process. The haptic assistance data acquired by the haptic assistance unit 115 is created by a user or the like and stored in the assistance data storage unit 123. That is, the assistance data storage unit 123 functions as a storage unit for storing haptic assistance data. Depending on the haptic assistance data, it may be generated based on teacher data or acquired via communication from another device. In addition, assistance using the haptic assistance data by the haptic assistance unit 115 may be provided together with the presentation by the operation control unit 111, but is not limited to this and may be provided separately from the presentation by the operation control unit 111.

[0098] (Real-time remote assistance) The haptic assistance unit 115 provides assistance using haptic assistance data, for example, assistance based on real-time remote operation by an experienced user Ub. In this case, the haptic assistance unit 115 functions so that when a learner user Ua operates the learning assistance device 1a to learn, the learning assistance device 1b operated by an expert user Ub becomes the master device, and the learning assistance device 1a becomes the slave device (however, it is the master device in relation to the learning assistance device 1c).

[0099] This allows the expert user Ub to remotely operate the learning assistance device 1a in real time while learning. Therefore, the expert user Ub can transmit haptic sensations, such as subtle movements and force adjustments, to the learner user Ua in real time. In other words, real-time haptic assistance can more efficiently support learning. In this case, by applying the above-mentioned (scaling function), it is possible to adjust the degree to which the operation of the expert user Ub is transmitted to the learner user Ua. This makes it possible to make adjustments such as increasing the degree of transmission to convey the operation clearly in the early stages of learning, and gradually decreasing the degree of transmission to convey the operation more faintly as learning progresses.

[0100] (Assistance by adsorption force) The haptic assistance unit 115 provides assistance using haptic assistance data, for example, by using an adhesive force. The following description of the haptic assistance unit 115 will refer to FIG. 1 as appropriate. In this case, the haptic assistance unit 115 sets a virtual target point 5 as shown in FIG. 1 and generates an adhesive force toward the virtual target point 5 near the virtual target point 5 to assist the user in moving to the target point. The virtual target point 5 is, for example, a target point (or a passing point leading to the target point) for processing or other operations performed on the workpiece 4 by the end effector 3 of the learning assistance device 1c. The position of the virtual target point 5 may be preset as haptic assistance data and stored in the assistance data storage unit 123, or may be generated based on training data. The adhesive force is generated when, for example, the positional relationship between the current position of the end effector 3 of the learning assistance device 1c derived from the execution data and the position of the virtual target point 5 reaches a predetermined relationship (for example, close to each other). This attraction force can be realized by correcting the execution data so that the haptic sensation of attraction to the position of the virtual target point 5 is transmitted to the learning assistance device 1a. This allows the user Ua, who is a learner, to be appropriately guided to the point that is the target of the task, and allows for more efficient support of learning.

[0101] (Directional assistance) The haptic assistance unit 115 provides assistance using haptic assistance data, for example, by providing direction. In this case, the haptic assistance unit 115 provides direction by generating a sensation similar to a ball bouncing, so that the direction to the target point can be understood. To achieve this, the haptic assistance unit 115 sets a virtual target point 5 as shown in FIG. 1 , in the same manner as in the above-described assistance using adhesive force. This virtual target point 5 may be set to correspond to a single point to be processed or other work, or multiple points may be set to correspond to a path of movement leading to the point to be processed or other work. For example, multiple virtual target points 5 may be set consecutively to correspond to the path of movement of the end effector 3 when performing work such as processing, thereby reproducing this path of movement. The positions of the multiple virtual target points 5 for reproducing such a path of movement may be set in advance as haptic assistance data and stored in the assistance data storage unit 123, or may be generated based on training data. The bouncing sensation is generated when the positional relationship between the current position of the end effector 3 of the learning support device 1c derived from the execution data and the position of a virtual target point 5 (e.g., the virtual target point 5 closest to the current position of the end effector 3 among multiple virtual target points 5 corresponding to the movement path) becomes a predetermined relationship (e.g., outside a predetermined range). This bouncing sensation is generated to present the direction to the virtual target point 5 (e.g., the direction to the virtual target point 5 closest to the current position of the end effector 3) to the learner user Ua. For example, a bouncing sensation is generated when the current position of the end effector 3 of the learning support device 1c is moving away from the virtual target point 5, and no bouncing sensation is generated when the current position of the end effector 3 is moving closer to the virtual target point 5. This bouncing sensation can be realized by correcting the execution data so that a haptic sensation corresponding to the bouncing sensation is transmitted to the learning support device 1a. In this case, the bounce cycle and strength of the bounce sensation may be varied in stages. For example, the bounce sensation may be generated in a shorter cycle and with a stronger bounce sensation as the user moves away from the virtual target point 5.Alternatively, in contrast to the above example, a bouncing sensation may be generated when the current position of the end effector 3 is moving in a direction approaching the virtual target point 5, and no bouncing sensation may be generated when the current position of the end effector 3 is moving in a direction away from the virtual target point 5. This allows the learner user Ua to be appropriately guided in the direction of the point to be worked on or the direction of the route to the point to be worked on, thereby supporting learning more efficiently.

[0102] (Assistance by repulsive force) The haptic assistance unit 115 provides assistance using haptic assistance data, for example, using a repulsive force. In this case, the haptic assistance unit 115 generates a repulsive force, similar to that generated when contacting a spring or damper near a no-entry area, to prevent the end effector 3 of the learning assistance device 1c from entering the no-entry area. To this end, the haptic assistance unit 115 sets a virtual no-entry area 7 as shown in FIG. 1. This virtual no-entry area 7 is, for example, an area where the end effector 3 of the learning assistance device 1c is prohibited from performing work such as machining on the work object 4 (i.e., the entry of the end effector 3). The virtual no-entry area 7 may be preset as haptic assistance data and stored in the assistance data storage unit 123, or may be generated based on training data. For example, when performing cutting on one layer of a two-layer plate made of different materials, the area corresponding to the other layer is set as the virtual no-entry area 7. The repulsive force is generated when, for example, the positional relationship between the current position of the end effector 3 of the learning support device 1c derived from the execution data and the area corresponding to the virtual no-entry area 7 reaches a predetermined relationship (for example, within a predetermined range). This repulsive force can be realized by correcting the execution data so that the haptic sensation corresponding to the elastic force or viscous force is transmitted to the learning support device 1a. This makes it possible to prevent the user Ua, who is a learner, from entering a no-entry area, thereby more efficiently supporting learning.

[0103] (Support by roughness, etc.) The haptic assistance unit 115 provides assistance using the haptic assistance data, for example, assistance using a tactile sensation such as a roughness. In this case, the haptic assistance unit 115 generates a tactile sensation such as a roughness near the no-entry area to notify the learner user Ua that the end effector 3 of the learning assistance device 1c is approaching the no-entry area. To achieve this, the haptic assistance unit 115 sets a virtual no-entry area 7 as shown in FIG. 1 in the same manner as described above (assistance using a repulsive force). The repulsive force is generated, for example, when the positional relationship between the current position of the end effector 3 of the learning assistance device 1c derived from the execution data and the area corresponding to the virtual no-entry area 7 reaches a predetermined relationship (for example, within a predetermined range). This roughness can be achieved by correcting the execution data so that a haptic sensation corresponding to a tactile sensation such as a roughness is transmitted to the learning assistance device 1a in the virtual no-entry area 7. This allows the user Ua, who is a learner, to be notified that he or she is approaching a no-entry area, thereby making it possible to more efficiently support learning.

[0104] When the above-mentioned (assistance by repulsive force) or (assistance by roughness, etc.) is performed, the area where the repulsive force or roughness is generated may be an area whose positional relationship with the area corresponding to the no-entry area 7 is in a predetermined relationship (for example, within a predetermined range), as described above, but is not limited to this. For example, like the virtual wall 8 and virtual wall 9 shown in FIG. 1, the repulsive force or roughness may be generated in an area such as a virtual wall that indicates a work area where the end effector 3 of the learning support device 1c should not move during work such as processing on the work object 4 (for example, a work area that interferes with another learning support device 1 or another user U, not shown).

[0105] (Aided by position scaling adjustment) The haptic assistance unit 115 performs assistance using, for example, position scaling as assistance using haptic assistance data. In this case, the haptic assistance unit 115 can prevent the slave device from moving too far from a predetermined position even if the master device is moved too far by adjusting the degree of position scaling according to the current position of the end effector 3 of the learning assistance device 1c.

[0106] To this end, the haptic assistance unit 115 sets a virtual no-entry area 7 (or a virtual no-entry position) as shown in FIG. 1 , similarly to the above-described (assistance by repulsive force). For example, it sets an area or position beyond which processing or other work cannot proceed. Furthermore, the degree of position scaling is adjusted, for example, as the current position of the end effector 3 of the learning assistance device 1c derived from the execution data approaches an area corresponding to the virtual no-entry area 7, the degree of scaling is increased so that the position of the slave device does not move even if the position of the master device moves. This scaling adjustment can be achieved, for example, by applying the scaling function as described above (force-haptic transmission function with position restriction by scaling) and controlling the slave device so that it does not exceed the (1 / b+a) position (i.e., the position corresponding to the virtual no-entry area 7 or the virtual no-entry position). This makes it possible to prevent the slave device from moving beyond a predetermined position (i.e., a position corresponding to the virtual no-entry area 7 or a virtual no-entry position) when operated by the learner user Ua, thereby enabling more efficient support for learning.

[0107] The haptic assistance unit 115 may perform any of the above-mentioned (assistance by real-time remote control), (assistance by adhesive force), (assistance by direction indication), (assistance by repulsive force), (assistance by roughness or the like), and (assistance by position scaling adjustment), or may perform a combination of these depending on, for example, the operation of the learner user Ua or the like. Furthermore, for example, the haptic assistance unit 115 may partially replace these haptic assistance methods. For example, assistance may be provided using adhesive force with the aim of preventing entry into the virtual no-entry area 7, or assistance may be provided using adhesive force, repulsive force, roughness, or the like with the aim of indicating direction.

[0108] The audiovisual assistance unit 116 presents the user Ua, who is a learner, with auxiliary information, which is information for assisting the user Ua, who is a learner, in performing operations. This presentation is realized, for example, by displaying on a display provided in the terminal 2 or the like, outputting a sound (warning sound, voice, etc.) from a speaker provided in the terminal 2 or the like, or by flashing a warning light provided in the terminal 2 or the like. This allows the learner to receive audiovisual support for the movements, thereby more efficiently supporting the learner in mastering the movements.

[0109] For example, the audiovisual assistance unit 116 determines to provide audiovisual assistance when it is set to provide audiovisual assistance in advance or when a condition for providing audiovisual assistance is met based on the operation of the learning assistance device 1a operating as the master device in this process. The audiovisual assistance data acquired by the audiovisual assistance unit 116 is created by a user or the like and stored in the assistance data storage unit 123. That is, the assistance data storage unit 123 not only stores haptic assistance data but also functions as a storage unit for storing audiovisual assistance data. Note that, depending on the audiovisual assistance data, it may be generated based on teacher data or acquired via communication from another device. It should be noted that the assistance provided by the audiovisual assistance unit 116 using the audiovisual assistance data may be provided together with the presentation by the operation control unit 111, but is not limited to this and may be provided separately from the presentation by the operation control unit 111.

[0110] Specifically, the audiovisual assistance unit 116 provides assistance using an assistance display screen, for example, as assistance using audiovisual assistance data. This assistance display screen will be described with reference to FIG. 14. FIG. 14 is a schematic diagram showing an example of an assistance display screen using audiovisual assistance data by the audiovisual assistance unit 116. As shown in FIG. 14, on the assistance display screen, an end effector 71 (corresponding to the end effector 3 in FIG. 1), an aerial portion 72 where no object exists, a cutting target portion 73 (corresponding to the cutting target portion of the work object 4 in FIG. 1), a cutting prohibited portion 74 (corresponding to the virtual no-entry area 7 of the work object 4 in FIG. 1), and an execution data display area 75 are represented and displayed using schematic computer graphics.

[0111] In this example, it is assumed that an end effector 71, which is a cutting drill, is moved linearly from left to right in the drawing, passes through an aerial portion 72, and then cuts a cutting target portion 73, but does not cut a cutting prohibited portion 74. These displays are also displayed as animations based on changes in the execution data according to the movements of the learning support device 1c. By referring to such an auxiliary screen, the user Ua, who is a learner, can visually grasp how far the end effector 71 should be moved. In addition to such a display, the audiovisual auxiliary unit 116 can also provide assistance by providing an alert such as a sound warning or a flashing warning light. For example, when the end effector 71 approaches the cutting prohibited portion 74 after entering the cutting target portion 73, an alert is provided by a sound warning or a flashing warning light.

[0112] Additionally, the audiovisual assistance unit 116 can also display, for example, changes in values ​​indicating force, etc. over time in the execution data display area 75, as described above with reference to Fig. 1. Furthermore, when an excessive force, etc. that is not suitable for a task such as machining, is applied, the audiovisual assistance unit 116 can also provide assistance such as providing an alert by sound, flashing a warning light, etc. in addition to such a display. This allows the learner to receive audiovisual support for the movements, thereby more efficiently supporting the learner in mastering the movements.

[0113] The above is a detailed description of the configuration of the learning assistance device 1. Next, the content of each process performed by the learning assistance system S including the learning assistance device 1 will be described in more detail.

[0114] [Teacher data acquisition process] First, the flow of the teacher data acquisition process executed by the learning support system S will be described with reference to Fig. 15. Fig. 15 is a flowchart illustrating the flow of the teacher data acquisition process executed by the learning support system S. The teacher data acquisition process is a process executed prior to the learning support process, and is executed in response to an instruction operation from an expert user Ub or the like to the learning support system S to start acquiring teacher data.

[0115] In step S11, the learning assistance device 1b, which operates as the master device in this process, accepts an operation from the expert user Ub.

[0116] In step S12, the motion control unit 111 of the learning support device 1b, which operates as the master device in this process, and the motion control unit 111 of the learning support device 1c, which operates as the slave device in this process, control their own motions based on the operation of the expert user Ub received in step S11. This motion control is achieved by applying the force-tactile transmission function as described above, and transmits the motion of the master device to the slave device, and also feeds back to the master device the input of a reaction force from an object (e.g., work target object 4) acting on the slave device.

[0117] In step S13, the movement control unit 111 included in the learning assistance device 1b determines whether the operation of the expert user Ub has ended. If the movement has ended, the determination in step S13 is Yes, and the process proceeds to step S14. On the other hand, if the movement has not ended, the determination in step S13 is No, and the process is repeated from step S11.

[0118] In step S14, the parameter acquisition unit 112 included in the learning assistance device 1b (or the learning assistance device 1c) acquires, as teacher data, the control parameters in chronological order that have been used in the current repetition of steps S11 to S13 and that have been buffered. Then, the parameter acquisition unit 112 included in the learning assistance device 1b (or the learning assistance device 1c) transmits the acquired teacher data to the parameter acquisition unit 112 included in the learning assistance device 1a.

[0119] In step S15, the operation control unit 111 included in the learning assistance device 1a stores the received teacher data in the teacher data storage unit 121 included in the learning assistance device 1a, thereby completing this process.

[0120] [Learning support process] Next, the flow of the learning support process executed by the learning support system S will be described with reference to Fig. 16. Fig. 16 is a flowchart illustrating the flow of the learning support process executed by the learning support system S. The learning support process is executed after the teacher data is stored by the teacher data acquisition process described above, and is executed in response to an instruction operation from the user Ua, who is a learner, to the learning support system S to start learning support.

[0121] In step S21, the display control unit 113 acquires from the teacher data storage unit 121 the teacher data to be presented this time.

[0122] In step S22, the teacher data processing unit 114 determines whether or not to process the teacher data acquired in step S21. For example, the teacher data processing unit 114 determines to process the teacher data when processing is set in advance or when a processing instruction operation is received from the learner user Ua or the like. If the teacher data is to be processed, the determination in step S22 is Yes, and the process proceeds to step S23. On the other hand, if the teacher data is not to be processed, the determination in step S22 is No, and the process proceeds to step S24. In step S23, the teacher data processing unit 114 processes the teacher data.

[0123] In step S24, the learning assistance device 1a, which operates as the master device in this process, accepts an operation from the user Ua, who is a learner.

[0124] In step S25, the motion control unit 111 included in the learning support device 1a operating as the master device in this process and the motion control unit 111 included in the learning support device 1c operating as the slave device in this process control the motion of their own devices based on the operation of the expert user Ub accepted in step S24 and the teacher data acquired in step S21. This motion control is an application of the force-tactile transmission function as described above, and applies control parameters corresponding to the teacher data as one mode of presentation, and then transmits the motion of the master device to the slave device, and also feeds back to the master device the input of a reaction force from an object (e.g., work target 4) acting on the slave device.

[0125] In step S26, the haptic assistance unit 115 determines whether or not to provide haptic assistance. For example, the haptic assistance unit 115 determines to provide haptic assistance when haptic assistance is set in advance or when a condition for providing haptic assistance is met based on the operation of the learning assistance device 1a operating as the master device in this process. If haptic assistance is to be provided, the determination in step S26 is Yes, and the process proceeds to step S27. On the other hand, if haptic assistance is not to be provided, the determination in step S26 is No, and the process proceeds to step S28. In step S27, the haptic assistance unit 115 provides haptic assistance.

[0126] In step S28, the audiovisual assistance unit 116 determines whether to provide audiovisual assistance. For example, the audiovisual assistance unit 116 determines to provide audiovisual assistance if it is set in advance to provide audiovisual assistance or if the conditions for providing audiovisual assistance are met based on the operation of the learning assistance device 1a operating as the master device in this process. If audiovisual assistance is to be provided, the determination in step S28 is Yes, and the process proceeds to step S29. On the other hand, if audiovisual assistance is not to be provided, the determination in step S28 is No, and the process proceeds to step S30. In step S28, the audiovisual assistance unit 116 provides audiovisual assistance.

[0127] In step S30, the movement control unit 111 included in the learning assistance device 1b determines whether the operation of the user Ua, who is the learner, has ended. If the movement has ended, the determination in step S30 is Yes, and the process proceeds to step S31. On the other hand, if the movement has not ended, the determination in step S30 is No, and the process is repeated from step S24.

[0128] In step S31, the parameter acquisition unit 112 of the learning assistance device 1b acquires, as execution data, control parameters in chronological order that have been used in the current repetition of steps S11 to S13 and that have been buffered. Then, the operation control unit 111 of the learning assistance device 1a stores the acquired execution data in the teacher data storage unit 121 of the learning assistance device 1a.

[0129] In step S32, the display control unit 113 determines whether or not to display the teacher data acquired in step S21 and the execution data acquired in step S31 in a comparable manner, as one mode of presentation. For example, the teacher data processing unit 114 determines to display the teacher data in a comparable manner when the teacher data processing unit 114 is set to display the teacher data in advance or when a display instruction operation is received from the user Ua, who is a learner, or the like. If a comparable display is to be performed, the determination in step S32 is Yes, and the process proceeds to step S33. On the other hand, if a comparable display is not to be performed, the determination in step S32 is No, and the process ends. Note that even in this case, since the execution data was stored in step S31, the display control unit 113 can realize display at any timing when a display instruction operation is subsequently received from the user Ua, or the like.

[0130] In step S33, as one mode of presentation, the display control unit 113 displays the teacher data acquired in step S21 and the execution data acquired in step S31 so that they can be compared on the display of the terminal 2, etc. This ends the process.

[0131] According to the learning support process described above, the teacher data and execution data, which are control parameters related to haptics, can be presented to the learner user Ua in a comparable manner by the operation control unit 111 and the display control unit 113. This allows the learner user Ua to understand whether there are any differences in the movements involving haptics between himself / herself and the expert user Ub, and the degree of the differences. Furthermore, according to the learning support process, learning support can be provided by appropriate haptic assistance or audiovisual assistance as needed. Therefore, the learning support system S can more efficiently support the learner in learning the movements.

[0132] [Variations] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments and can undergo various modifications such as omissions and substitutions without departing from the spirit of the present invention. In such cases, these embodiments and their modifications are included in the scope and spirit of the invention described in this specification, etc., and are also included in the scope of the invention described in the claims and their equivalents. As an example, the above-described embodiment of the present invention may be modified as follows.

[0133] In the above-described embodiment, it is assumed that the control unit 10 of each learning assistance device 1 has the same functional blocks. However, this is not limiting, and some functional blocks may be omitted from the control unit 10 of a learning assistance device 1 that is not used by a learner (for example, the learning assistance device 1b and the learning assistance device 1c in the above-described embodiment). For example, functional blocks other than the operation control unit 111 and the parameter acquisition unit 112 may be omitted from the control unit 10 of a learning assistance device 1 that is not used by a learner.

[0134] In the above-described embodiment, it is assumed that the learning assistance for the learner includes all of the presentation by the operation control unit 111, the presentation by the display control unit 113, the force-tactile assistance, and the audiovisual assistance. However, this is not limiting, and only some of these may be executed. In this case, the functional blocks for realizing the learning assistance that is not executed may be omitted.

[0135] Furthermore, in the above-described embodiment, it is assumed that the control unit 10 of each learning assistance device 1 includes a motion control unit 111, and that each of these motion control units 111 controls the motion to realize a force-tactile transmission function, etc. However, this is not limiting, and a device for performing motion control (e.g., terminal 2) may realize the functions of the functional force-velocity allocation conversion block FT, ideal force source block FC, ideal velocity (position) source block PC, and inverse conversion block IFT shown in FIG. 2 or 3. Then, this device for performing motion control may receive reference values, etc. from each learning assistance device 1, and realize a force-tactile transmission function, etc. in each learning assistance device 1 based on the control of this device for performing motion control.

[0136] Furthermore, in the above-described embodiment, it is assumed that the display control unit 113 displays the teacher data and the execution data for comparison after the learner has finished performing the action. However, the present invention is not limited to this, and the teacher data and the execution data up to that point may be displayed for comparison while the learner is performing the action. This allows the learner to understand in real time while performing the action whether there is a difference in the action involving haptics and the degree of the difference.

[0137] Furthermore, in the above-described embodiment, it is assumed that a task such as machining is actually performed by the learning support device 1 (for example, the learning support device 1c in the above-described embodiment) operating as a slave device. It is assumed that the operation of the master device is transmitted to the slave device, and that a reaction force input from an object (for example, work target object 4) to the slave device is fed back to the master device. However, this is not limiting, and a task such as machining may be performed on a virtual object in a virtual space. It is also possible to transmit the operation of the master device to the virtual slave device, and to feed back a reaction force input from the virtual object to the virtual slave device to the master device.

[0138] At this time, the contact of the virtual object is treated as an action on the virtual slave device, and the allocation and conversion of position and force energy is performed between variables in virtual space and variables in real space. At this time, force and tactile sensations are transmitted between the master device and the virtual object, and the force and tactile sensations received by the virtual slave device are transmitted. In this way, for example, it is possible to realize the teacher data acquisition process and the learning support process without actually performing work such as processing on the work object 4. Note that the virtual object in the virtual space can be configured as one that is virtually generated, or as one that is a reproduction in the virtual space of an object in the real space based on the properties of a material in the real space.

[0139] As described above, the learning support system S according to this embodiment includes the operation control unit 111, the parameter acquisition unit 112, the operation control unit 111, and the display control unit 113. The operation control unit 111 controls the device to be controlled based on the user's operation, thereby causing the device to perform an operation corresponding to the haptic sensation of the user's operation. The parameter acquisition unit 112 acquires the control parameters used in the control by the operation control unit 111 . The operation control unit 111 and the display control unit 113 present to the second user in a comparable manner a first control parameter acquired by the parameter acquisition unit 112 when the operation control unit 111 controls the operation of the control target device based on an operation by the first user, and a second control parameter acquired by the parameter acquisition unit 112 when the operation control unit 111 controls the operation of the control target device based on an operation by the second user. In this way, the learning support system S acquires the control parameters used to make the controlled device perform actions corresponding to the haptics of the user's operation for each of the first user (e.g., an expert) and the second user (e.g., a learner), and then presents the control parameters of each user to the second user so that they can be compared. This allows the second user to understand whether there are any differences between the first user and the second user in the actions accompanied by haptics, and the degree of the differences. Therefore, the learning support system S can more efficiently support the learner in learning the movements.

[0140] The operation control unit 111 includes a position sensor 40, a force-velocity allocation conversion block FT, an ideal force source block FC, an ideal velocity (position) source block PC, and an inverse conversion block IFT, in order to control the device to be controlled based on the user's operation. The position sensor 40 detects information relating to the position of the device to be controlled, which is generated as the device operates. The force-velocity allocation conversion block FT performs a conversion to allocate control energy to the energy of a predetermined physical quantity based on information of a predetermined physical quantity corresponding to information about the position and information that serves as a control reference. The ideal force source block FC and the ideal velocity (position) source block PC calculate the control amount of a predetermined physical quantity based on the energy of the predetermined physical quantity allocated by the force-velocity allocation conversion block FT. The inverse transformation block IFT inversely transforms the control quantities calculated by the ideal force source block FC and the ideal velocity (position) source block PC to return the outputs based on the control quantities to the controlled device, and determines the inputs to the controlled device. As a result, when each user performs an action accompanied by a haptic sensation, the action of the device to be controlled can be controlled based on the control parameters relating to the haptic sensation.

[0141] The display control unit 113 realizes at least a part of the presentation by displaying information indicating the change in the first control parameter over time and information indicating the change in the second control parameter over time in an overlapping manner. This allows the second user to easily visually grasp whether there are any differences between the second user and the first user in terms of the time series of movements involving haptics, and the degree of such differences.

[0142] The display control unit 113 realizes at least a part of the presentation by correcting the second control parameter based on the first control parameter when the control target device operates in accordance with the action of the second user. This allows the second user to easily understand, via the haptic sense, the differences along the time series between the second user and the first user in the actions accompanied by the haptic sense.

[0143] The learning support system S further includes a teacher data processing unit 114. The teacher data processing unit 114 processes the first control parameter. The operation control unit 111 and the display control unit 113 use the first control parameters processed by the teacher data processing unit 114 for presentation. This allows the second user to easily understand, through haptics, whether there are any differences between the second user and the first user along the timeline in terms of actions involving haptics, and the degree of such differences.

[0144] There are other control target devices that are different from the control target device. When the controlled device operates in response to the action of the second user, the other controlled device operates in real time as a master device used by a user other than the second user, and the controlled device operates in real time as a slave device used by the second user. The master and slave devices are communicatively coupled to send and receive control parameters. This allows other users to provide real-time support for learning while the second user is performing an action.

[0145] The learning support system S further includes a force-tactile assistance unit 115 . The force haptic assistance unit 115 corrects the second control parameter based on a third control parameter, which is a control parameter for controlling the controlled device so as to assist the action of the second user. This makes it possible to control the control target device so as to assist the action of the second user.

[0146] The learning support system S further comprises an audiovisual support unit 116 . The audiovisual assistance unit 116 presents assistance information, which is information for assisting the second user's actions, based on the second control parameter. This makes it possible to output auxiliary information, which is information for assisting the second user's action, based on the state of the second control parameter.

[0147] [Realization of functions through hardware and software] The function of executing the series of processes according to the above-described embodiment can be realized by hardware, software, or a combination of these. In other words, it is sufficient that the function of executing the series of processes described above is realized in any of the learning support systems S, and there are no particular limitations on how this function is realized.

[0148] For example, when the function of executing the above-mentioned series of processes is realized by a processor that executes arithmetic processing, the processor that executes this arithmetic processing includes processors that are composed of various processing devices alone, such as single processors, multiprocessors, and multicore processors, as well as processors that combine these various processing devices with processing circuits such as ASICs (Application Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays).

[0149] Furthermore, for example, when the function of executing the above-described series of processes is realized by software, the program constituting the software is installed on a computer via a network or a recording medium. In this case, the computer may be a computer incorporating dedicated hardware, or may be a general-purpose computer (e.g., a general electronic device such as a general-purpose personal computer) that can execute predetermined functions by installing a program. Furthermore, the steps of writing the program may include only processes that are executed chronologically according to the order, but may also include processes that are executed in parallel or individually. Furthermore, the steps of writing the program may be executed in any order within the scope of the present invention.

[0150] A recording medium on which such a program is recorded may be provided to a user by being distributed separately from a computer main body, or may be provided to a user in a state where it is pre-installed in a computer main body. In this case, the recording medium distributed separately from a computer main body may be, for example, a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. An optical disk may be, for example, a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), or a Blu-ray (registered trademark) Disc. An magneto-optical disk may be, for example, an MD (Mini Disc). Furthermore, a recording medium provided to a user in a state where it is pre-installed in a computer main body may be, for example, the storage unit 12 in FIG. 5 on which the program is recorded. [Explanation of symbols]

[0151] 1a, 1b, 1c learning support device, 2 terminal, 3 end effector, 4 work object, 5 virtual target point, 6 virtual directional sensation, 7 virtual no-entry area, 8, 9 virtual wall, 10 control unit, 11 processor, 111 operation control unit, 112 parameter acquisition unit, 113 display control unit, 114 teacher data processing unit, 115 force-tactile assistance unit, 116 audio-visual assistance unit, 12 memory unit, 121 teacher data memory unit, 122 execution data memory unit, 123 assistance data memory unit, 13 ROM, 14 RAM, 15 communication unit, 16 Input section, 17 output section, 20 driver, 30 actuator, 40 position sensor, CS controlled system, FT force-velocity allocation conversion block, FC ideal force source block, PC ideal velocity (position) source block, IFT inverse conversion block, S learning support system, Ua, Ub user

Claims

1. an operation control means for controlling a control target device based on a user's operation, thereby causing the control target device to perform an operation corresponding to the haptic sensation of the user's operation; parameter acquisition means for acquiring control parameters used in control by the operation control means; a processing means for scaling a value of a control parameter corresponding to at least one of force, position, and time included in the first control parameter acquired by the parameter acquisition means when the operation control means controls the operation of the control target device based on an operation by a first user; a presentation means for performing a process of correcting a second control parameter acquired by the parameter acquisition means, when the operation control means controls the operation of the control target device based on an operation by a second user, by the first control parameter processed by the processing means, while the operation by the second user is being performed, thereby presenting the first control parameter and the second control parameter to the second user so that they can be compared; A learning support system comprising:

2. The processing means processes the first control parameters by scaling to enlarge the values ​​of at least one of the force and position control parameters included in the first control parameters.

2. The learning support system according to claim 1.

3. The processing means processes the first control parameter by scaling the value of the control parameter corresponding to the time included in the first control parameter so that it becomes longer in time.

2. The learning support system according to claim 1.

4. A learning support system as described in any one of claims 1 to 3, characterized in that a virtual object is assumed around the controlled device, and when the controlled device operates in response to the action of the second user, the second control parameter is corrected, thereby transmitting to the second user the virtual reaction force that the controlled device receives from the virtual object.

5. The presentation means At least a part of the presentation is realized by displaying information indicating the change in the first control parameter along a time series and information indicating the change in the second control parameter along a time series in an overlapping manner, and and further displaying a quantified value of the degree of mismatch between the first control parameter and the second control parameter displayed in the superimposed manner.

3. The learning support system according to claim 1 or 2.

6. There is another control target device different from the control target device, When the control target device operates in accordance with the action of the second user, the other control target device operates in real time as a master device used by a user different from the second user, and the control target device operates in real time as a slave device used by the second user; 6. The learning support system according to claim 1, wherein the master device and the slave device are communicatively connected to transmit and receive the control parameters.

7. The learning support system described in any one of claims 1 to 6, further comprising a first auxiliary means for correcting the second control parameter based on a third control parameter which is a control parameter for controlling the controlled device to assist the action of the second user.

8. The learning support system according to any one of claims 1 to 7, further comprising a second assistance means for presenting auxiliary information, which is information for assisting the second user in performing an action, based on the second control parameter.

9. an operation control step of controlling a control target device based on a user's operation, thereby causing the control target device to perform an operation corresponding to the haptic sensation of the user's operation; a parameter acquisition step of acquiring control parameters used in the control by the operation control step; a processing step of scaling a value of a control parameter corresponding to at least one of force, position, and time included in the first control parameter acquired in the parameter acquisition step when the operation control step controls the operation of the control target device based on an operation by a first user; a presentation step of correcting the second control parameter acquired in the parameter acquisition step by the first control parameter processed in the processing step when the operation control step controls the operation of the control target device based on an operation by the second user, while the operation by the second user is being performed, thereby presenting the first control parameter and the second control parameter to the second user so that they can be compared; A learning support method comprising:

10. an operation control function that controls a control target device based on a user's operation, thereby causing the control target device to perform an operation corresponding to the haptic sensation of the user's operation; a parameter acquisition function for acquiring control parameters used in control by the operation control function; a processing function that scales the value of a control parameter corresponding to at least one of force, position, and time included in the first control parameter acquired by the parameter acquisition function when the operation control function controls the operation of the control target device based on an operation by a first user; a presentation function that performs a process of correcting a second control parameter acquired by the parameter acquisition function with the first control parameter processed by the processing function when the operation control function controls the operation of the control target device based on an operation by a second user, while the operation by the second user is being performed, thereby presenting the first control parameter and the second control parameter to the second user in a comparable manner; A learning support program characterized by realizing the above on a computer.

Images