Motion data management system, motion data management method, and program

Abstract

[Problem] To realize more appropriate management for data pertaining to tactile force senses. [Solution] A motion data management system S comprises a motion data acquisition unit 411, a tag assignment unit 413, and a database management unit 415. The motion data acquisition unit 411 acquires, as motion data, time-series control parameters that have been used for transmission of tactile force senses between devices. The tag assignment unit 413 assigns, as identification information to the motion data acquired by the motion data acquisition unit 411, information pertaining to motional substance to be realized by said motion data. The database management unit 415 performs management by accumulating therein multiple sets of motion data to which the identification information has been assigned by the tag assignment unit 413, and enabling search for a desired motion data set from among the accumulated motion data sets on the basis of the identification information.

Description

Motion data management system, motion data management method, and program

[0001] The present invention relates to a motion data management system, a motion data management method, and a program.

[0002] Conventionally, there is a technology that realizes the transmission of haptic sensations between devices by converting the haptic sensations (e.g., the hardness or softness of an object) felt when touching an object into digital data. This enables the transmission of haptic sensations in both directions, for example, between one device that accepts operations from a user and another device that performs work involving touching an object.

[0003] An example of such a technology is disclosed in Patent Document 1. In the technology disclosed in Patent Document 1, the haptic sensation when an object is grasped by a robot manipulator is converted into data, and the haptic sensation is transmitted to a multi-fingered device worn by the user.

[0004] JP 2009-289179 A

[0005] Incidentally, it is believed that the force-tactile data generated by the technology of Patent Document 1 and the like can be used for purposes other than transmitting force-tactile sensations in real time, such as user analysis and realization of actions. However, such force-tactile data is simply a list of control parameter values ​​in chronological order, with formal information such as the date and time of recording added, making it difficult for users to easily grasp its contents. Due to these circumstances, in order to utilize force-tactile data, it is necessary to manage the force-tactile data more appropriately.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to more appropriately manage data relating to haptics.

[0007] In order to solve the above problem, a motion data management system according to one embodiment of the present invention is characterized by comprising: an acquisition means for acquiring, as motion data, time-series control parameters used to transmit haptic sensations between devices; an assignment means for assigning, to the motion data acquired by the acquisition means, information relating to the content of the motion realized by the motion data as identification information; and a management means for storing a plurality of motion data to which identification information has been assigned by the assignment means, and for managing desired motion data from the plurality of stored motion data in a searchable manner based on the identification information.

[0008] According to the present invention, data relating to haptics can be managed more appropriately.

[0009] 1 is a block diagram showing the overall configuration of a motion data management system S according to the present embodiment. FIG. 1 is a schematic diagram showing a "motion data accumulation process" that classifies and accumulates motion data received from a control device 30 as content. FIG. 1 is a schematic diagram showing a "motion data search process" that searches for motion data accumulated as content. FIG. 2 is a schematic diagram showing a "motion data browsing process" that browses motion data accumulated as content. FIG. 3 is a schematic diagram showing a "motion data generation process" that generates new motion data from motion data accumulated as content, and a "motion data provision process" that provides motion data to the control device 30 as a result of these processes. FIG. 4 is a block diagram showing the hardware configurations of a first driving device 10 and a second driving device 20. FIG. 5 is a block diagram showing the hardware and functional configuration of a control device 30. FIG. 6 is a block diagram showing a control algorithm for transmitting haptics in a haptic control unit 311. FIG. 7 is a block diagram showing the hardware and functional configuration of a motion data management device 40. FIG. 8 is a flowchart explaining the flow of a motion data accumulation process executed by the motion data management system S. FIG. 9 is a flowchart explaining the flow of a motion data provision process executed by the motion data management system S. FIG. 10 is a flowchart explaining the flow of a motion data search process executed by the motion data management system S. 10 is a flowchart illustrating the flow of a motion data viewing process executed by the motion data management system S. FIG. 11 is a flowchart illustrating the flow of a motion data generating process executed by the motion data management system S. FIG.

[0010] An example of an embodiment of the present invention will now be described with reference to the accompanying drawings.

[0011] [System Configuration] Figure 1 is a block diagram showing the overall configuration of a motion data management system S according to this embodiment. As shown in Figure 1, the motion data management system S includes, as local systems, multiple sets each consisting of a first driving device 10, a second driving device 20, and a control device 30. The motion data management system S also includes, as a higher-level system, a motion data management device 40.

[0012] In each group, the first driving device 10, the second driving device 20, and the control device 30 are communicatively connected via a wired or wireless connection. The control device 30 and the motion data management device 40 are communicatively connected via a network N. This network N is realized by a wired or wireless network such as a local area network (LAN) or the Internet. However, this form of communication connection is merely an example. For example, in each group, the first driving device 10, each second driving device 20, and the control device 30 may be communicatively connected to the control device 30 via the network N or another network. Alternatively, for example, each control device 30 and the motion data management device 40 may be communicatively connected without using the network N.

[0013] The i, j, and k suffixed to the reference numerals of the first drive unit 10, the second drive unit 20, and the control unit 30 are each any integer value equal to or greater than 1, and may be the same or different. That is, each set may include all of the first drive unit 10, the second drive unit 20, and the control unit 30, or only some of them. For example, some sets may include the control unit 30 and either the first drive unit 10 or the second drive unit 20. Alternatively, some sets may include one control unit and multiple sets of first drive units 10 and second drive units 20.

[0014] In this system configuration, the first drive unit 10 operates as a master unit to drive a first mechanism that accepts operation from a user. Meanwhile, the second drive unit 20 operates as a slave unit to drive a second mechanism that performs an operation involving contact with an object. In this case, the control unit 30 performs control (bilateral control) to transmit haptic sensations between the first mechanism driven by the first drive unit 10 and the second mechanism driven by the second drive unit 20. As a result, the user's operation on the first mechanism (position and force input) is transmitted to the second mechanism, and the reaction force (position and force response) from the second mechanism is transmitted to the first mechanism.

[0015] The control device 30 also acquires, as operation data, time-series control parameters used to transmit haptic sensations between devices through such bilateral control, and transmits this operation data to the operation data management device 40.

[0016] The operation data management device 40 accumulates and manages the operation data received from the control device 30. The operation data management device also enables the managed operation data to be viewed and searched, and provides desired operation data to the control device 30 that uses the operation data.

[0017] [Processing Overview] Next, an overview of each of the multiple processes performed by the action data management system S will be described with reference to the schematic diagrams of Fig. 2 to Fig. 5. Specifically, Fig. 2 is a schematic diagram showing the "action data storage process" that classifies and stores action data received from the control device 30 as content. Fig. 3 is a schematic diagram showing the "action data search process" that searches for action data stored as content.

[0018] 4 is a schematic diagram showing a "motion data browsing process" for browsing motion data stored as content. Furthermore, FIG. 5 is a schematic diagram showing a "motion data generating process" for generating new motion data from motion data stored as content, and a "motion data providing process" for providing motion data to the control device 30 as a result of these processes.

[0019] 2, in the operation data accumulation process, predetermined operations are realized in the first drive unit 10 and the second drive unit 20 by the control parameters that the control device 30 uses to transmit haptic sensations. Here, as an example, a series of operations, such as "grab," "carry," and "place," are realized by the second drive unit 20.

[0020] The control device 30 also acquires, as operation data, the time-series control parameters used to realize this series of operations. The operation data at this point is raw data corresponding to the attributes (e.g., device type, number of axes, movable range, torque output range, and sampling period) of the device that realized the operation (e.g., the second drive device 20). The control device 30 of the local system uploads this operation data by transmitting it to the operation data management device 40 of the higher-level system. Note that, hereinafter, operation data corresponding to the attributes of the device of the local system, such as this uploaded operation data, will be referred to as "operation data of functional coordinates" as appropriate.

[0021] The operation data management device 40 breaks down the operation data uploaded by the control device 30 based on the operation content realized by the operation data. Here, as an example, the operation data is broken down by type of operation content (i.e., function).

[0022] The action data management device 40 also assigns information about the action content to each piece of decomposed action data as identification information. Here, as an example, tagging is performed by assigning the name of the type of action content (i.e., function) to the action data as a tag, which is identification information.

[0023] The decomposition and assignment of identification information may be performed manually based on user instructions, or automatically using a classifier or AI (artificial intelligence) created using machine learning or a rule base.

[0024] Furthermore, the motion data management device 40 normalizes each motion data, converting the motion data that corresponded to functional coordinates into generalized coordinates. The motion data management device 40 then accumulates this motion data by storing it in a database. Note that hereinafter, general-purpose motion data that is not specialized for the attributes of a specific device, such as this normalized motion data, will be referred to as "motion data in generalized coordinates" as appropriate.

[0025] In this way, according to the action data accumulation process, action data can be accumulated as if it were content by performing tagging, normalization, etc.

[0026] As shown in Fig. 3, in the action data search process, the action data management device 40 can search for action data using a number of methods. For example, as shown in Fig. 3(A), the action data management device 40 may allow a user to directly specify and search for action data stored in the database by inputting a file name into a search field. Here, as an example, the file name "OOOXXX.CSV" is used to directly specify and search for the corresponding action data.

[0027] Alternatively, as shown in FIG. 3B, the action data management device 40 may allow a user to select a desired tag to search for action data stored in the database based on the tag. Furthermore, the search may be further narrowed down by selecting other tags for multiple action data retrieved in this way. Here, as an example, multiple action data retrieved using the tag "grab" are further narrowed down by using the tag "low degree of freedom."

[0028] In this case, filtering may be performed by excluding from the search target any operation data that does not match the attributes of the device (e.g., the second drive device 20) that is intended to use the operation data searched by the user, or the attributes of the object that comes into contact with the operation (e.g., material, weight, size, etc.).

[0029] In this way, according to the action data search process, the user can easily search for desired action data by searching using tags or the like or by performing filtering.

[0030] 4, in the operation data browsing process, the operation data management device 40 visualizes the operation data retrieved by the operation data search process and displays it to the user. Here, as an example, the time series changes in the control parameters in the operation data are converted into a graph and displayed.

[0031] In another example, the motion data management device 40 realizes the motion of a virtual device by transmitting haptic sensations to the virtual device using the motion data. The motion data management device 40 then displays the realized motion of the virtual device. That is, the motion data management device 40 displays the results of a simulation of the motion realized by the motion data as a video. In this case, the motion data management device 40 performs the simulation using a virtual device with a general structure corresponding to the tag of the motion data. For example, if the function tagged is "grab," the simulation is performed using a virtual manipulator with a general structure equipped with a gripping mechanism for grasping.

[0032] In this way, the operation data viewing process can display operation data in a manner that is easy for the user to intuitively understand, such as operation data converted into a graph or a video of the operation simulated using a virtual device, thereby assisting the user in their search.

[0033] 5, in the action data generation process, the action data management device 40 generates one action data set containing multiple action data selected by the user by linking the multiple action data in the order set by the user in response to the search performed by the action data search process or the display performed by the action data browsing process described above. The action data management device 40 also performs denormalization on the generated action data, converting the action data in generalized coordinates into action data in functional coordinates corresponding to the local system that provides the action data.

[0034] Next, in the action data providing process, the action data management device 40 transmits the converted action data to the control device 30 in the local system for download. That is, the action data management device 40 provides the action data.

[0035] The control device 30, which has received motion data of functional coordinates corresponding to the local system by downloading it, converts this motion data into motion data of robot coordinates (e.g., joint coordinates of the second drive unit 20) corresponding to a device of this local system (e.g., the second drive unit 20). By performing this conversion, the motion data of the functional coordinates corresponding to a predetermined function (e.g., grasping) can be expressed as a matrix indicating a combination of axes in the robot coordinates. The control device 30 then realizes the motion of the function corresponding to the motion data by transmitting haptic sensations to the device (e.g., the second drive unit 20) based on the motion data of the robot coordinates after this conversion. Here, as an example, a series of motions, "grab," "carry," and "place," are realized by the second drive unit 20.

[0036] In this way, the operation data generation process can generate a single piece of continuous operation data from multiple pieces of operation data, or convert the operation data into operation data of functional coordinates that correspond to the local system that provides the operation data, depending on the user's selection. Also, the operation data provision process can provide operation data that is compatible with the destination local system to this local system, thereby realizing an operation that corresponds to the operation data.

[0037] As described above, the motion data management system S executes each process to assign information related to the motion details as identification information and manages desired motion data in a searchable manner using this identification information. This allows the user to perform a search based on easily comprehensible information related to the motion details (e.g., the motion's function, motion conditions, etc.). In other words, the motion data management system S treats each piece of motion data as content and manages this content as a library, thereby improving user convenience and enabling the use of motion data. Therefore, the motion data management system S can solve the problem of more appropriately managing haptic data.

[0038] [Device Configuration] Next, the configuration of each device included in the motion data management system S will be described. Fig. 6 is a block diagram showing the hardware configuration of the first drive device 10 and the second drive device 20. As shown in Fig. 6, the first drive device 10 includes an actuator 12 for driving the first mechanism 15, a driver 11 for driving the actuator 12, and a position sensor 13 for detecting the position of a moving object moved by the actuator 12. The second drive device 20 includes an actuator 22 for driving the second mechanism 25, a driver 21 for driving the actuator 22, and a position sensor 23 for detecting the position of a moving object moved by the actuator 22.

[0039] In this case, the position of the moving object detected by the position sensor 13 is, for example, the position of a predetermined part of the first mechanism 15 or the position of a predetermined part of the user operating the first mechanism 15. Furthermore, the position of the moving object detected by the position sensor 23 is, for example, the position of a predetermined part of the second mechanism 25 or the position of a predetermined part of an object that comes into contact with the second mechanism 25 to be grasped, processed, or moved.

[0040] However, in this embodiment, instead of detecting the positions of these moving objects, the rotational angle of the output shaft of each actuator may be detected by a rotary encoder built into each actuator. That is, in this embodiment, the concept of position is considered to include an angle (e.g., the rotational angle of the output shaft of the actuator), and information related to position is considered to include position, angle, velocity, angular velocity, acceleration, and angular acceleration. Furthermore, since position and velocity (or acceleration) or angle and angular velocity (or angular acceleration) are parameters that can be replaced by differential and integral calculations, when processing position or angle, it is possible to replace them with velocity or angular velocity, etc. as appropriate before processing.

[0041] The first mechanism 15 is a mechanism that functions as an operating tool that accepts user operation, and its shape and structure are not particularly limited. For example, the first mechanism 15 may be realized by a controller with a movable part that accepts user operation, or a device shaped like a finger worn by the user. The second mechanism 25 is a mechanism that comes into contact with an object to grasp, process, or move it, and its shape and structure are not particularly limited. For example, the second mechanism 25 may be realized by a robot manipulator that comes into contact with an object to grasp or move it, or a robot arm that is equipped with (or has attached) a tool that processes an object.

[0042] Fig. 7 is a block diagram showing the hardware configuration and functional configuration of the control device 30. As shown in Fig. 7, the control device 30 includes a processor 31, a storage unit 32, a ROM (Read Only Memory) 33, a RAM (Random Access Memory) 34, and a communication unit 35.

[0043] The processor 31 is configured by an arithmetic device such as a CPU (Central Processing Unit), and executes various processes according to a program recorded in a ROM 33 or a program loaded from the storage unit 32 to a RAM 34. The RAM 34 also stores data and the like necessary for the processor 31 to execute various processes, as appropriate.

[0044] The processor 31 is connected to the ROM 33 and the RAM 34 via a bus (not shown). This bus is further connected to the storage unit 32 and the communication unit 35. The processor 31 is also connected to the position sensor 13, the driver 11, the position sensor 23, and the driver 21, which are described above with reference to FIG. 2, via signal lines. Circuits for transmitting and receiving signals, such as a D / A (digital-to-analog) conversion circuit and an A / D (analog-to-digital) conversion circuit, as well as pulse counters, are appropriately arranged along this connection path. The transmission and reception of signals via this signal line may be achieved by parallel communication or serial communication.

[0045] The storage unit 32 is configured with a hard disk, a solid state drive (SSD), a dynamic random access memory (DRAM), etc., and stores various data. The communication unit 35 controls communication with other devices via a network.

[0046] 2 to 5, a haptic control unit 311, a motion data acquisition unit 312, and a provided data acquisition unit 313 function in the processor 31. The storage unit 32 includes a motion data storage unit 321 and a provided data storage unit 322. Data required to realize the processes is transmitted and received between these functional blocks at appropriate times, even in cases not specifically mentioned below.

[0047] In this case, the force-tactile control unit 311 may be made to function by a general-purpose processor 31 performing arithmetic processing, but the force-tactile control unit 311 may also be made to function by a dedicated integrated circuit (IC) that can perform processing to make the force-tactile control unit 311 function quickly and stably.

[0048] The haptic control unit 311 outputs control commands to the driver 11 and the driver 21 based on the position information detected by the position sensor 13 and the position sensor 23, thereby realizing control to transmit haptic sensations between the first driving unit 10, which is the master device, and the second driving unit 20, which is the slave device.

[0049] As a control method for transmitting the haptic sensation by this haptic control unit 311, two control methods are assumed: "actual action transmission control" and "data transmission control."

[0050] Here, actual movement transmission control is a control method in which both the master device (here, the first driving device 10) and the slave device (here, the second driving device 20) are actually driven, and the force haptics associated with the actual driving are transmitted based on the positions detected in real time by the position sensors (e.g., position sensor 13 and position sensor 23) of the master device and slave device that actually drive the devices, and the forces calculated from these positions.

[0051] In contrast, data transmission control is a control method in which only one of the master device (here, the first drive unit 10) and the slave device (here, the second drive unit 20) is actually driven, and haptics are transmitted based on the position detected in real time by the position sensor (e.g., either one of the position sensors 13 and 23) of either the master device or the slave device that is actually driven, and the force calculated from this position, and on the motion data of functional coordinates provided by the motion data management device 40 to realize the motion of either the master device or the slave device. Note that, hereinafter, the motion data of functional coordinates provided by the motion data management device 40 for this data transmission control will be referred to as "provided data" where appropriate.

[0052] Specifically, when performing actual movement transmission control, the haptic control unit 311 executes control to transmit haptics bidirectionally between the actuator 12 of the first drive unit 10 and the actuator 22 of the second drive unit 20. To this end, the haptic control unit 311 acquires from the position sensor 13 the position (specifically, the position or angle) of the moving object moved by the actuator 12, and acquires from the position sensor 23 the position (specifically, the position or angle) of the moving object moved by the actuator 22.

[0053] On the other hand, when performing data transmission control, the haptic control unit 311 executes control to realize an operation based on the provided data in the actuator 22 (or the actuator 12) for controlling the operation of the second drive unit 20 (or the provided data for controlling the operation of the first drive unit 10). In other words, the haptic control unit 311 executes control to transmit a haptic sensation based on the provided data to the actuator 22 (or the actuator 12).

[0054] To this end, the haptic control unit 311 acquires the position (specifically, the position or angle) of the moving object moved by the actuator 22 (or the actuator 12) from the position sensor 23 (or the position sensor 13), and also acquires provided data for controlling the operation of the second drive unit 20 (or provided data for controlling the operation of the first drive unit 10). This provided data is provided from the operation data management device 40 as described above.

[0055] In this way, in the actual movement transmission control and data transmission control, the position (angle) and provided data acquired by the haptic control unit 311 are used as reference values ​​in the control algorithm for transmitting haptics to control the operation of the first drive unit 10 and the second drive unit 20. This control algorithm for transmitting haptics will be described in detail with reference to FIG. 8.

[0056] Fig. 8 is a block diagram showing a control algorithm for transmitting haptics in the haptic control unit 311. As shown in Fig. 8, the control algorithm implemented in the haptic control unit 311 is expressed as a control law including a functional force-velocity allocation conversion block FT, at least one of an ideal force source block FC or an ideal velocity (position) source block PC, and an inverse conversion block IFT. In this embodiment, the controlled system CS is composed of a first drive unit 10 and a second drive unit 20.

[0057] The functional force / speed allocation transformation block FT defines the transformation of control energy into a velocity (position) and force range set according to the function of the controlled system CS. Specifically, the functional force / speed allocation transformation block FT defines a coordinate transformation that takes as input a reference value (reference value) for the function of the controlled system CS and the current position (or current angle) of the actuator 12 or 22. This coordinate transformation generally transforms an input vector having the reference value and the current position (current angle) as elements into an output vector consisting of a position (angle) for calculating a control target value for the position (angle), and also transforms an input vector having the reference value and the current force as elements into an output vector consisting of a force for calculating a control target value for the force.

[0058] By setting the coordinate transformation in the functional force / speed allocation transformation block FT to a content that represents the force / haptic transmission function, it is possible to realize the force / haptic transmission function between the first drive unit 10 and the second drive unit 20, or to realize the operation of transmitting the force / haptic sensation with the second drive unit 20 without using the first drive unit 10. Furthermore, by setting coefficients in the elements of the transformation matrix in the coordinate transformation in the functional force / speed allocation transformation block FT, it is possible to perform scaling of the position (angle) or force.

[0059] That is, in this embodiment, the functional force / speed allocation conversion block FT "converts" the variables of the actuator 12 or 22 alone (variables in real space) into a group of variables of the entire system (variables in space after coordinate transformation) that represent the force haptic transmission function, and allocates control energy to the control energy of position (angle) and the control energy of force. That is, the coordinate transformation set in the functional force / speed allocation conversion block FT converts real space coordinates (oblique coordinates) in which position (angle) and force are related to each other into virtual space coordinates (cartesian coordinates) in which position (angle) and force are independent of each other. Therefore, compared to when control is performed using the variables of a single actuator (variables in real space), it is possible to provide the control energy of position (angle) and the control energy of force independently, i.e., to control the position (angle) and force independently.

[0060] In this embodiment, when controlling the position (angle) and force output by the first drive unit 10, the state value in the space after coordinate transformation can be calculated under the condition that the difference in position (angle) between the position (angle) of the member moved by the actuator 12 and the force input calculated from these positions (angles) and the reference value serving as the basis for position (angle) and force control is zero and the sum of forces is zero (equal forces are output in opposite directions). In this case, the reference value serving as the basis for position (angle) and force control in the actual motion transmission control is the position (angle) of the member moved by the actuator 22 in the second drive unit 20 and the force calculated from these positions (angles). In the data transmission control, the reference value serving as the basis for position (angle) and force control is provided data provided by the motion data management device 40 for controlling the operation of the first drive unit 10. Note that in this data transmission control, when the first drive unit 10 is controlled using provided data, the second drive unit 20 does not operate.

[0061] Similarly, in this embodiment, when controlling the position (angle) and force output by the second drive unit 20, the state value in the space after coordinate transformation can be calculated under the condition that the difference in position (angle) between the position (angle) of the member moved by the actuator 22 and the force input calculated from these positions (angles) and the reference value serving as the basis for position (angle) and force control are zero and the sum of forces is zero (equal forces are output in opposite directions). In this case, in actual motion transmission control, the reference value serving as the basis for position (angle) and force control is the position (angle) of the member moved by the actuator 12 in the first drive unit 10 and the force calculated from these positions (angles). In data transmission control, the reference value serving as the basis for position (angle) and force control is provided data provided by the motion data management device 40 for controlling the operation of the second drive unit 20. Note that in data transmission control, when the second drive unit 20 is controlled using provided data, the first drive unit 10 does not operate.

[0062] The ideal force source block FC is a block that performs calculations in the force domain according to the coordinate transformation defined by the functional force-velocity allocation transformation block FT. The ideal force source block FC sets a target value for the force used in calculations based on the coordinate transformation defined by the functional force-velocity allocation transformation block FT. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, to achieve a function similar to the function indicated by the reference value, the target value can be set to zero. To perform scaling, a value obtained by enlarging or reducing the information representing the function indicated by the reference value can be set. The ideal force source block FC can also set an upper limit for the force energy determined by calculations in the force domain. Setting an upper limit for the force energy limits the contact force when the second mechanism 25 contacts an object, thereby preventing the second mechanism 25 or the object from being damaged by being pressed too hard against the object.

[0063] The ideal velocity (position) source block PC is a block that performs calculations in the position (angle) domain according to the coordinate transformation defined by the functional force-velocity allocation transformation block FT. The ideal velocity (position) source block PC sets a target value for the position (angle) when performing calculations based on the coordinate transformation defined by the functional force-velocity allocation transformation block FT. This target value is set as a fixed value or a variable value depending on the function to be realized. For example, to achieve a function similar to the function indicated by the reference value, the target value can be set to zero. To perform scaling, a value obtained by enlarging or reducing the information indicating the function to be realized can be set. The ideal velocity (position) source block PC can also set an upper limit for the energy of the position (angle) determined by calculations in the position (angle) domain. Setting an upper limit for the energy of the position (angle) limits the distance the second mechanism 25 can travel, thereby preventing excessive movement of the second mechanism 25 and preventing damage to the second mechanism 25 or the object.

[0064] The inverse transformation block IFT is a block that inversely transforms values ​​in the position (angle) and force domains into values ​​in the input domain to the controlled system CS (e.g., voltage values ​​or current values, etc.) (i.e., determines command values ​​in real space).

[0065] When actual movement transmission control is performed based on such a control algorithm, the haptic control unit 311 receives input of time-series position (angle) detection values ​​detected by the position sensors 13 and 23. These time-series position (angle) detection values ​​represent the movements of the actuators 12 and 22, and the haptic control unit 311 applies coordinate transformation for transmitting haptics to the input positions (angles) and forces derived from these positions (angles).

[0066] Furthermore, when data transmission control is performed based on such a control algorithm, the haptic control unit 311 receives input of time-series position (angle) detection values ​​detected by the position sensor 23 (or position sensor 13). These time-series position (angle) detection values ​​represent the operation of the actuator 22 (or actuator 12). In addition, the haptic control unit 311 receives input of time-series position (angle) values ​​in the provided data. These position (angle) values ​​in the provided data represent the operation of the actuator 22 (or actuator 12) in the provided data. The haptic control unit 311 applies coordinate transformation to transmit haptics to the input positions (angles) and forces derived from these positions (angles). In this case, time-series values ​​of the control parameters of position (angle) and force may be input as provided data. In this case, the force-tactile control unit 311 applies coordinate transformation to transmit force-tactile sensations based on the time series values ​​of the position (angle) and force control parameters in the provided data, the position (angle) input from the position sensor 23, and the force derived from this position (angle).

[0067] Returning to FIG. 7 , the motion data acquisition unit 312 acquires, as motion data (here, raw data of motion data of functional coordinates), the time-series control parameters (here, time-series position (angle) and force control parameters calculated to transmit the haptic sensation to the first drive unit 10 and the second drive unit 20) used by the haptic control unit 311 to transmit the haptic sensation using the control algorithm described above. The motion data acquisition unit 312 also uploads the acquired motion data by transmitting it to the motion data management device 40. Furthermore, the motion data acquisition unit 312 stores the acquired motion data in the motion data storage unit 321. In other words, the motion data storage unit 321 functions as a storage unit that stores motion data.

[0068] Note that uploading by transmitting operation data by operation data acquisition unit 312 may be performed each time sampling is performed by position sensor 13 or position sensor 23 and the associated control parameters are calculated, or the time-series control parameters for each sampling or control parameter calculation may be stored in operation data storage unit 321 and uploaded all at once when the operation is completed. Furthermore, after uploading the operation data, operation data acquisition unit 312 may secure free space in the storage capacity of operation data storage unit 321 by deleting the operation data stored in operation data storage unit 321.

[0069] The motion data acquisition unit 312 adds additional information to the acquired motion data, such as an identifier for identifying the first drive unit 10 and the second drive unit 20, attributes of the first drive unit 10 and the second drive unit 20 (e.g., the type of each device, the number of axes, the range of motion, the torque output range, the sampling period, etc.), the date and time of the motion, information about the motion, and information about objects that came into contact with the motion, as necessary, before transmitting or storing the data. This allows the motion data management device 40 to use this information for tagging, normalization, etc., eliminating the need for the user of the motion data management device 40 to input this information.

[0070] The provided data acquisition unit 313 acquires the provided data by receiving the provided data transmitted from the operation data management device 40 at the time of provision. That is, the provided data acquisition unit 313 downloads the provided data. As described above, this provided data is operation data of functional coordinates that serve as reference values ​​for controlling the operation of the actuator 22 of the second drive unit 20 (or the actuator 12 of the first drive unit 10) in the data transmission control. Also, as described above, this provided data is converted into robot coordinates for use. Therefore, the provided data acquisition unit 313 converts the acquired provided data into robot coordinates and stores the converted data in the provided data storage unit 322. That is, the provided data storage unit 322 functions as a storage unit that stores the provided data. Note that the conversion into robot coordinates may be performed in real time during the data transmission control, rather than when the data is stored in the provided data storage unit 322.

[0071] Fig. 9 is a block diagram showing the hardware and functional configuration of the action data management device 40. As shown in Fig. 9, the action data management device 40 includes a processor 41, a storage unit 42, a ROM 43, a RAM 44, a communication unit 45, an input unit 46, an output unit 47, and a drive 48.

[0072] The processor 41, storage unit 42, ROM 43, RAM 44, and communication unit 45 are the same as the hardware of the same names provided in the control device 30, and therefore redundant explanations will be omitted. These units, the input unit 46, the output unit 47, and the drive 48 are connected via a bus not shown.

[0073] The input unit 46 is made up of various buttons etc. and inputs various information in response to instruction operations. The output unit 47 is made up of a display, a speaker etc. and outputs images and sounds.

[0074] Removable media 50, such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, is appropriately loaded into the drive 48. The program read from the removable media 50 by the drive 48 is installed in the storage unit 42 as needed.

[0075] 2 to 5, the processor 41 functions as follows: a motion data acquisition unit 411, a motion decomposition unit 412, a tagging unit 413, a normalization processing unit 414, a database management unit 415, a provision request response unit 416, and a UI control unit 417. A motion data database 421 is formed in the storage unit 42. Data required to realize the processing is transmitted and received between these functional blocks at appropriate times, even in cases not specifically mentioned below.

[0076] The operation data acquisition unit 411 acquires, by receiving, operation data (here, raw data of operation data of functional coordinates) uploaded by transmission from the operation data acquisition unit 312 of the control device 30. In addition, the operation data acquisition unit 411 stores the acquired operation data in the operation data database 421.

[0077] In addition, the motion data database 421 stores various motion data in association with processing by functional blocks other than the motion data acquisition unit 411. For example, the motion data database 421 stores motion data of generalized coordinates normalized after decomposition and tagging in the motion data accumulation process described above with reference to FIG. 2. The motion data database 421 also stores motion data of generalized coordinates generated in the motion data generation process described above with reference to FIG. 5. The motion data database 421 also stores motion data of non-normalized functional coordinates in the motion data provision process described above with reference to FIG. 5. In other words, the motion data database 421 functions as a database that stores and accumulates various motion data.

[0078] When unnecessary operation data is generated as a result of processing, each functional block may delete the unnecessary operation data stored in the operation data database 421 to free up storage space in the operation data database 421. For example, when normalization is performed to generate generalized coordinate operation data, the raw operation data of the functional coordinates before normalization may be deleted. Alternatively, when denormalization is performed to generate functional coordinate provision data, the provision data that has already been provided to the local system may be deleted. Alternatively, instead of deletion, data may be compressed using a general data compression technique to free up storage space in the operation data database 421. In this case, for example, the determination of whether to delete or compress the data, or whether to use lossless or lossy compression, may be made based on criteria for the importance of the operation data set by the user.

[0079] The action decomposition unit 412 decomposes the uploaded action data acquired by the action data acquisition unit 411 based on the action content realized by the action data. For example, the action decomposition unit 412 decomposes the action data by type of action content (i.e., function). For example, if multiple functions are realized by one uploaded action data, the action data is decomposed for each function to produce multiple action data. Note that if one uploaded action data realizes only one function, no decomposition is performed.

[0080] Here, in order to perform the decomposition and tagging described below, functions must be defined in advance. In this case, the user can define the functions at any level of abstraction depending on the environment in which the action data management system S is implemented, the user's intended use, and the like. For example, the functions may be defined at levels of abstraction such as "grab," "carry," "place," and "process." Alternatively, the functions may be defined at a lower level of abstraction and more specifically. For example, instead of defining "carry," the functions may be defined more specifically, such as "carry at high speed," "carry at medium speed," and "carry at low speed." Alternatively, the functions may be defined from a different perspective, such as "carry a heavy object" or "carry a light object," or these perspectives may be combined to further define the functions more specifically.

[0081] Note that the higher the level of abstraction of a function, the greater the number of operation data included in the search results, and the lower the level of abstraction of a function, the fewer operation data included in the search results. Therefore, it is recommended that users define functions at a level of abstraction that will not result in excessively large or small search results.

[0082] The tagging unit 413 assigns information about the content of the action as identification information to each piece of action data decomposed by the action decomposition unit 412. For example, the tagging unit 413 performs tagging by assigning the name of the type of action (i.e., function) to the action data as a tag, which is identification information. In this case, the name of the function to be assigned is defined by the user according to the same criteria as those of the action decomposition unit 412.

[0083] Furthermore, tags other than the names of functions may be further assigned as tags. For example, tags such as a category to which the operation content belongs or a feature of the operation content may be further assigned. For example, for operation data whose functions are "turning", "milling", or "drilling", the name of the category "cutting", to which these functions belong, may be further assigned as a tag as a category to which the operation content belongs. In addition, tags such as feature names, such as "low frequency", "high frequency", "low degree of freedom", and "high degree of freedom", may also be assigned as tags as tags for the features of the operation content. In either case, assigning tags that allow a user performing a search to intuitively understand what content the tag indicates improves the convenience of the user during the search.

[0084] The decomposition by the action decomposition unit 412 and the tagging by the tagging unit 413 may be performed manually based on user instructions, or automatically by a classifier or AI created using machine learning or a rule base. In this case, the classifier or AI can be generated, for example, by a user generating training data to which correct answers (e.g., the names of functions that are correct answers) are assigned as labels in advance, and then performing machine learning based on this training data. Alternatively, the classifier or AI can be generated, for example, by a user analyzing action data in advance and setting parameters such as thresholds based on the analysis results. In this case, the functions may not only be defined in advance by the user, but also may be defined automatically or semi-automatically by a computer based on the results of machine learning by the classifier or AI.

[0085] The normalization processing unit 414 normalizes or denormalizes the motion data based on the motion data. Specifically, the normalization processing unit 414 normalizes the motion data of functional coordinates to convert it into motion data of generalized coordinates. This normalization can be performed, for example, based on the attributes of the device (e.g., the second actuation unit 20) that is the target of transmitting haptic sensations using the motion data in the local system from which the motion data of functional coordinates is uploaded.

[0086] The normalization processing unit 414 denormalizes the motion data in generalized coordinates to convert it into motion data in functional coordinates. This denormalization can be performed, for example, based on the attributes of the device (e.g., the second drive unit 20) that transmits the haptic sensation via the motion data in the local system to which the motion data in this functional coordinates is downloaded. As described above, the device attributes include, for example, the type of device, the number of axes, the range of motion, the torque output range, and the sampling period.

[0087] The normalization processing unit 414 can use existing methods for generating normalization or denormalization, such as minimum-maximum scale conversion, variance scale conversion, and scale conversion using quantiles, to generate normalization or denormalization. Different methods may be used for different control parameters, such as using a first method to normalize or denormalize the value of a position (angle) control parameter and a second method to normalize or denormalize the value of a force control parameter. In this case, the minimum and maximum values ​​used for normalization and denormalization may be, for example, the minimum and maximum values ​​based on the device attributes described above or the minimum and maximum values ​​of the control parameter values ​​in the operation data.

[0088] By normalizing the functional coordinates of one of the local systems and normalizing the motion data, it is possible to generate generalized motion data, which is motion data in generalized coordinates. Furthermore, by denormalizing the generalized motion data in generalized coordinates when providing the data as provided data, it is possible to provide functional coordinates motion data that corresponds to the device in the local system to which the data is provided.

[0089] That is, by performing normalization and denormalization in this manner, the normalization processing unit 414 can absorb the differences in the attributes of various devices and easily generate operation data corresponding to each of the various devices. As a result, it is possible to save the human effort of preparing operation data corresponding to each of the various devices in advance. Furthermore, it is possible to make management of the operation data easier.

[0090] The database management unit 415 manages the action data stored in the action data database 421. For example, based on this action data, the database management unit 415 realizes searching in the action data search process described above with reference to Fig. 3, browsing in the action data browsing process described above with reference to Fig. 4, and generation of action data in the action data generation process described above with reference to Fig. 5.

[0091] Specifically, in the motion data search process, when a file name or tag selection is received, the database management unit 415 searches the motion data database using the file name or tag as a search key to extract the motion data to be searched. Additionally, the database management unit 415 filters out motion data that does not match the attributes of the local system to which the searched motion data is provided, by excluding it from the search targets. The attributes of the local system are, for example, the attributes of at least one of the first drive unit 10, the second drive unit 20, and the control unit 30 in the local system. For example, if the second drive unit 20 in the local system has four axes, motion data corresponding to five or more axes of motion that cannot be achieved with four axes is filtered out. Alternatively, filtering may be performed based on the attributes of the object that the second drive unit 20 comes into contact with in the local system.

[0092] Furthermore, in the motion data browsing process, the database management unit 415 visualizes the motion data retrieved by the motion data retrieval process and displays it to the user. This display can be realized, for example, by converting the time-series changes in the control parameters in the motion data into a graph format with the horizontal axis representing time and the vertical axis representing the control parameter values ​​(for example, the value of the force control parameter and the value of the position (velocity) control parameter), and displaying the graph on a display included in the output unit 47.

[0093] Alternatively, this display can be realized by displaying the results of a simulation of the operation performed by the operation data. This simulation is performed, for example, by performing data transmission control based on the operation data on a virtual second drive unit 20 (or first drive unit 10) in a virtual space realized by computer processing. In this case, the database management unit 415 performs the simulation using a virtual device with a general structure corresponding to the tag of the operation data. For example, if the tagged function is "grab," the simulation is performed using a virtual device with a general structure equipped with a gripping mechanism, which is set by default for "grab." Alternatively, the database management unit 415 may perform the simulation using a virtual device with a general structure specified by the user, rather than the default virtual device. For example, the simulation may be performed using a virtual device with a general structure that has the same degree of freedom as the device in which the user intends to use the operation data.

[0094] Furthermore, the display of the simulation results can be realized, for example, by representing the operation of the virtual second drive unit 20 (or first drive unit 10) as a schematic computer graphic and displaying this on a display included in the output unit 47. This allows the user to intuitively grasp the operation realized by the operation data. In this case, the virtual second mechanism 25 (or virtual first mechanism 15) and the virtual contact target object may also be represented as a schematic computer graphic and included in the display. Note that it is assumed that the simulation will be performed using a virtual device with a general structure so that the user can intuitively grasp the operation. However, if the operation realized by the operation data is a special operation, the simulation may be performed using a virtual device with a special structure corresponding to the special operation.

[0095] Furthermore, in the movement data generation process, the database management unit 415 generates one movement data set having the plurality of movement data selected by the user as components, by linking the plurality of movement data sets in the order set by the user in accordance with the search performed by the movement data search process or the display performed by the movement data browsing process described above. In this case, to make the movement at the linking portion smoother (i.e., to eliminate abrupt changes in the control parameters at the linking portion), the values ​​of the control parameters before and after the linking portion of the movement data to be linked may be adjusted, or movement data for interpolation may be inserted separately between the movement data to be linked.

[0096] Furthermore, the database management unit 415 may not only link multiple pieces of data, but also adjust control parameters in the operation data, or create rule-based operation data that transitions operations according to transition conditions.

[0097] Here, when adjusting the control parameters in the operation data, for example, the database management unit 415 directly adjusts the values ​​of the position (angle) and force control parameters based on user instructions. Alternatively, when the user selects a criterion such as "shorter time" or "longer time," the database management unit 415 automatically adjusts the values ​​of the control parameters so that the operation data conforms to the criterion. For example, consider operation data corresponding to a "polishing" function, in which a robot arm equipped with a polishing tool polishes a workpiece whose position is fixed. In this case, if the user selects "shorter time," the database management unit 415 automatically adjusts the value of the position (angle) control parameter in the operation data to improve the operation speed in order to complete the polishing in a shorter time. The adjustment range (i.e., the degree of adjustment) for such automatic adjustment is verified and set in advance. For example, verification involves repeatedly operating a real device or running a virtual device to simulate the operation while adjusting the control parameters, and setting the system so that appropriate automatic adjustment is performed based on the verification results. In this way, the control parameter values ​​can be appropriately adjusted even through automatic adjustment. Furthermore, adjusting the control parameters in the operation data may not only involve adjusting a portion of the existing operation data as described above, but may also involve generating new operation data by adjusting the control parameters. For example, if a user wants to realize a desired function for a real device, new operation data may be generated to realize the desired function. In this case, the generated new operation data may have a predetermined regularity that realizes a function in a predetermined pattern, such as "moving at a constant speed" or "outputting a force with a sinusoidal waveform."

[0098] Furthermore, when rule-based operation data that transitions according to a transition condition is used, the database management unit 415 adds a transition condition to the operation data generated by linking, for example. For example, to operation data generated by linking "operation data corresponding to a first function," "operation data corresponding to a second function," and "operation data corresponding to a third function," a transition condition such as "if the operation corresponding to the operation data of the first function is successful, the operation corresponding to the operation data of the second function is executed, but if the operation corresponding to the operation data of the first function fails, the operations corresponding to the operation data of the other functions are not executed" may be added. Alternatively, a transition condition such as "if the operation corresponding to the operation data of the first function is successful, the operation corresponding to the operation data of the second function is executed and terminated, but if the operation corresponding to the operation data of the first function fails, the operation corresponding to the operation data of the third function is executed instead of the operation corresponding to the operation data of the second function and terminated" may be added. The control device 30, which has received this motion data, performs data transmission control using the motion data as provided data, and determines whether the motion corresponding to the motion data of the first function was successful based on, for example, changes in position detected in real time by the position sensor 23 (or position sensor 13) and changes in force calculated from this position (angle). The control device 30 then performs data transmission control based on the determination result and the transition condition. In this manner, motion transitions according to the transition conditions can be realized. Note that, instead of the database management unit 415 adjusting these control parameters (including generating new motion data) and generating rule-based motion data, the motion data acquisition unit 411 may acquire motion data with adjusted control parameters, newly generated motion data, or rule-based motion data from another device.

[0099] The provision request response unit 416 provides operation data to the control device 30 of the local system specified by the user in response to an instruction operation related to the provision request from the user. In order to select the operation data to be provided, the database management unit 415 performs the above-mentioned operations such as searching for and browsing the operation data. The provision request response unit 416 then transmits the operation data finally selected by the user to the control device 30 of the local system specified by the user for downloading, thereby realizing the provision of the operation data.

[0100] The UI control unit 417 controls the user interface for displaying various information to the user and accepting various instruction operations from the user. For example, the UI control unit 417 generates a user interface screen for performing search, browsing, etc. in parallel with the execution of processing such as search, browsing, etc. by each functional block, and displays the generated user interface screen on a display included in the output unit 47. The UI control unit 417 also accepts instruction operations from the user who refers to this user interface screen, and outputs the content of the instruction operation to the corresponding functional block.

[0101] Next, the details of the processing content of each process performed by the action data management system S will be described.

[0102] 10 is a flowchart illustrating the flow of the action data accumulation process executed by the action data management system S. The action data accumulation process is executed when the control device 30 starts actual action transmission control in response to an operation on the first drive device 10 by the user, and when the control device 30 subsequently receives an instruction operation from the user to start the action data accumulation process.

[0103] First, a description will be given of the processing performed by the control device 30. In step S11, the haptic control unit 311 acquires the position (specifically, the position or angle) of the moving object moved by the actuator 12 and the force calculated from this position (angle), and the position (specifically, the position or angle) of the moving object moved by the actuator 22 and the force calculated from this position (angle).

[0104] In step S12, the haptic control unit 311 converts the input vector in the real space into a vector in the virtual space. In step S13, the haptic control unit 311 performs calculations in the velocity (position) domain and in the force domain.

[0105] In step S14, the haptic control unit 311 converts the values ​​of the velocity (position) and force domains back into values ​​of the domain of input to the controlled system CS (vectors in real space). In step S15, the haptic control unit 311 outputs command values ​​for the actuators 12 and 22.

[0106] In step S16, the operation data acquisition unit 312 acquires the time-series control parameters (here, the time-series position (angle) and force control parameters calculated to transmit the force haptics to the first drive unit 10 and the second drive unit 20) used to transmit the force haptics in the processing from step S10 to step S15 by the force haptics control unit 311 (i.e., actual operation transmission control) as operation data (here, raw data of the operation data of the functional coordinates).

[0107] In step S17, the action data acquisition unit 312 determines whether the haptic control unit 311 has ended the actual action transmission control in response to the end of the user's operation on the first drive unit 10. If the actual action transmission control has ended, a YES determination is made in step S17, and the process proceeds to step S18. On the other hand, if the actual action transmission control has not ended, a NO determination is made in step S17, and the process returns to step S11 and is repeated.

[0108] In step S18, motion data acquisition unit 312 stores the motion data acquired in step S16 in motion data storage unit 321. In step S19, motion data acquisition unit 312 transmits the motion data acquired in step S16 to motion data management device 40. This completes the motion data accumulation process in control device 30.

[0109] In this flowchart, the operation data is temporarily stored each time step S16 is repeated, and all of the temporarily stored operation data is transmitted and stored together in steps S18 and S19. However, the present invention is not limited to this, and the same storage and transmission as in steps S18 and S19 may be performed each time step S16 is repeated.

[0110] Next, a description will be given of the processing contents of the operation data management device 40. In step S21, the operation data acquisition unit 411 receives and acquires the operation data (here, raw data of the operation data of the functional coordinates) uploaded by transmission from the operation data acquisition unit 312 of the control device 30.

[0111] In step S22, the action decomposition unit 412 decomposes the action data acquired in step S21 based on the action content realized by the action data. In step S23, the tagging unit 413 tags each action data decomposed in step S22 by assigning information related to the action content as identification information.

[0112] In step S24, normalization processing unit 414 normalizes the movement data based on the movement data tagged in step S23. In step S25, normalization processing unit 414 stores the movement data normalized in step S24 in movement data database 421, thereby storing the movement data in the movement data database. This completes the movement data accumulation process in movement data management device 40.

[0113] According to the above-described action data accumulation process, action data can be accumulated as if it were content by performing tagging, normalization, and the like.

[0114] 11 is a flowchart illustrating the flow of the action data providing process executed by the action data management system S. The action data providing process is executed when the action data management device 40 receives a user's operation to instruct the start of the action data providing process.

[0115] First, a description will be given of the processing content of the action data management device 40. In step S31, the provision request response unit 416 accepts an instruction operation from the user to request the provision of action data.

[0116] In step S32, the UI control unit 417 starts displaying a user interface screen for displaying various information to the user and accepting various instruction operations from the user. Thereafter, the UI control unit 417 transitions the user interface screen to one appropriate for the situation. For example, when a search is performed, the UI control unit 417 transitions to a user interface screen for search.

[0117] In step S33, the UI control unit 417 determines whether or not an instruction operation to start a search for motion data has been received. If an instruction operation to start a search has been received, the determination in step S33 is YES, and the process proceeds to step S34. On the other hand, if an instruction operation to start a search has not been received, the determination in step S33 is NO, and the process proceeds to step S35.

[0118] In step S34, the database management unit 415 etc. executes a subroutine to search for action data. Details of the search for action data will be described later with reference to FIG.

[0119] In step S35, the UI control unit 417 determines whether or not an instruction operation to start browsing of the operation data has been received. If an instruction operation to start browsing has been received, the determination in step S35 is YES, and the process proceeds to step S36. On the other hand, if an instruction operation to start browsing has not been received, the determination in step S35 is NO, and the process proceeds to step S37.

[0120] In step S36, the database management unit 415 etc. executes a subroutine, which is a processing for browsing action data. The processing for browsing action data will be described in detail later with reference to FIG.

[0121] In step S37, the UI control unit 417 determines whether an instruction to start generating operation data has been received. If an instruction to start generating operation data has been received, the determination in step S37 is YES, and the process proceeds to step S38. On the other hand, if an instruction to start generating operation has not been received, the determination in step S37 is NO, and the process proceeds to step S39.

[0122] In step S38, the database management unit 415 and the like perform a subroutine of generating motion data. Details of the generating motion data will be described later with reference to FIG. 14. The motion data generated in accordance with the execution of the generating motion data process is provided to the control device 30 as provided data.

[0123] In step S39, the UI control unit 417 determines whether to end the operation data provision process. For example, the operation data provision process ends when a user operation instructing the end of the operation data provision process is received. If the operation data provision process is to be ended, a YES determination is made in step S39, and the process ends. On the other hand, if the operation data provision process is not to be ended, a NO determination is made in step S39, and the process returns to step S33 and is repeated.

[0124] Next, the processing content of the control device 30 will be described. In step S41, the provided data acquisition unit 313 receives and acquires the provided data transmitted at the time of provision from the action data management device 40. That is, the provided data acquisition unit 313 downloads the provided data. In step S42, the provided data acquisition unit 313 converts the provided data acquired in step S41 into robot coordinates and stores the converted data in the provided data storage unit 322.

[0125] In step S43, the force-tactile control unit 311 reads the provided data for realizing the data transmission control from the provided data storage unit 322, and acquires the position (specifically, the position or angle) of the moving object moved by the actuator (actuator 12 or actuator 22) of one of the driving devices that is driven as the target of the data transmission control, and the force calculated from this position (angle).

[0126] In step S44, the haptic control unit 311 converts the input vector in the real space into a vector in the virtual space. In step S45, the haptic control unit 311 performs calculations in the velocity (position) domain and in the force domain.

[0127] In step S46, the haptic control unit 311 inversely converts the values ​​of the velocity (position) and force domains into values ​​of the domain of input to the controlled system CS (vectors in real space). In step S47, the haptic control unit 311 outputs a command value for the actuator (actuator 12 or actuator 22) of one of the drive units that is driven as the target of data transmission control. Note that in the realization process, a command value for the actuator (here, actuator 12 or actuator 22) of the other drive unit that is not the target of data transmission control is not output.

[0128] In step S48, the haptic control unit 311 determines whether the provided data used for data transmission control in this process has ended. If the provided data has ended, a YES determination is made in step S48, and this process ends. On the other hand, if the provided data has not ended, a NO determination is made in step S48, and the process returns to step S43 and is repeated.

[0129] According to the operation data provision process described above, by providing operation data that is compatible with the local system to which it is provided, it is possible to realize an operation corresponding to the operation data (here, the provided data).

[0130] 12 is a flowchart illustrating the flow of the action data search process executed by the action data management system S. The action data search process is executed when the action data management device 40 receives a user's operation to instruct the start of the action data search process during the action data provision process.

[0131] In step S51, the UI control unit 417 displays an action data search screen as a user interface screen. This action data search screen has search fields arranged so that the user can input and instruct operations such as a "file name" for searching by the file name of action data, a "tag" for tag-based search, a "system attribute" for filtering by local system attributes, and a "contact object" for filtering by contact object attributes.

[0132] In step S52, the UI control unit 417 determines whether or not a file name has been input into the search field. If a file name has been input into the search field, the determination in step S52 is YES, and the process proceeds to step S53. On the other hand, if a file name has not been input into the search field, the determination in step S52 is NO, and the process proceeds to step S55.

[0133] In step S53, the database management unit 415 extracts the action data to be searched by searching the action data database 421 using the file name entered in the search field in step S52 as a search key. In step S54, the database management unit 415 displays the action data extracted in step S53 as the search result on the action data search screen.

[0134] In step S55, the UI control unit 417 determines whether or not a tag has been input into the search field. If a tag has been input into the search field, the determination in step S55 is YES, and the process proceeds to step S56. On the other hand, if a tag has not been input into the search field, the determination in step S55 is NO, and the process proceeds to step S58.

[0135] In step S56, the database management unit 415 extracts the action data to be searched by searching the action data database 421 using the tag entered in the search field in step S55 as a search key. In step S57, the database management unit 415 displays the action data extracted in step S56 as a search result on the action data search screen.

[0136] In step S58, the UI control unit 417 determines whether input of system attributes into the search field has been accepted. As described above, these system attributes are attributes of at least one of the first drive unit 10, the second drive unit 20, and the control unit 30 in the local system to which the searched operation data is provided. If input of system attributes into the search field has been accepted, a YES determination is made in step S58, and the process proceeds to step S59. On the other hand, if input of system attributes into the search field has not been accepted, a NO determination is made in step S58, and the process proceeds to step S61.

[0137] In step S59, the database management unit 415 searches the operation data database 421 using the system attribute entered in the search field in step S58 as a search key to extract operation data to be searched. In step S60, the database management unit 415 performs filtering by treating the operation data extracted in step S59 as the search target while excluding other operation data from the search target. As a result, the operation data filtered in this step is excluded from the search target in steps S53 and S56, and therefore is also excluded from the display target in steps S54 and S57.

[0138] In step S61, the UI control unit 417 determines whether or not input of an attribute of a contacting object into the search field has been accepted. As described above, the attribute of the contacting object is the attribute of the contacting object that comes into contact with the second drive unit 20 in the local system to which the searched operation data is provided. If input of the attribute of the contacting object into the search field has been accepted, a YES determination is made in step S61, and the process proceeds to step S62. On the other hand, if input of the attribute of the contacting object into the search field has not been accepted, a NO determination is made in step S61, and the process proceeds to step S64.

[0139] In step S62, the database management unit 415 searches the action data database 421 using the attributes of the contact object entered in the search field in step S61 as a search key to extract action data to be searched. In step S63, the database management unit 415 performs filtering by treating the action data extracted in step S62 as the search target while excluding other action data from the search target. As a result, the action data to be filtered in this step is excluded from the search target in steps S53 and S56, and therefore is also excluded from the display target in steps S54 and S57.

[0140] In step S64, the UI control unit 417 determines whether or not action data has been selected from the search results by the user's selection instruction operation. If action data has been selected, the determination in step S64 is YES, and the process proceeds to step S65. On the other hand, if action data has not been selected from the search results, the determination in step S64 is NO, and the process returns to step S52 and is repeated.

[0141] In step S65, the UI control unit 417 determines whether to end the action data search process. For example, the action data search process ends when a user operation instructing the end of the action data search process is received. If the action data search process is to be ended, a YES determination is made in step S65, and the action data search process returns. That is, step S34 in FIG. 11 ends, and the process proceeds to step S35. On the other hand, if the action data search process is not to be ended, a NO determination is made in step S65, and the process returns to step S52 and is repeated.

[0142] According to the action data search process described above, the user can easily search for desired action data by searching using tags or the like or by performing filtering.

[0143] 13 is a flowchart illustrating the flow of the motion data viewing process executed by the motion data management system S. The motion data viewing process is executed in response to the motion data management device 40 receiving a user's instruction to start the motion data viewing process during the motion data providing process.

[0144] In step S71, the UI control unit 417 displays a motion data viewing screen as a user interface screen. This motion data viewing screen has a display area for displaying, for example, motion data converted into a graph format and simulation results of motions realized by the motion data.

[0145] In step S72, the UI control unit 417 acquires the action data to be displayed based on the selection instruction operation of the user. In this case, the user selects the action data to be displayed from the action data included in the search results of the action data search process.

[0146] In step S73, the UI control unit 417 determines whether or not a command operation to display a graph has been received from the user for the operation data selected in step S72. If a command operation to display a graph has been received, the determination in step S73 is YES, and the process proceeds to step S74. On the other hand, if a command operation to display a graph has not been received, the determination in step S73 is NO, and the process proceeds to step S76.

[0147] In step S74, the database management unit 415 creates a graph by converting the time series changes in the control parameters in the motion data selected in step S72 into a graph format. This graph may have time on the horizontal axis and the values ​​of the force control parameter and the position (velocity) control parameter on the vertical axis, for example. In step S75, the graph created in step S74 is displayed.

[0148] In step S76, the UI control unit 417 determines whether or not a command to display a moving image has been received from the user for the action data selected in step S72. If a command to display a moving image has been received, the determination in step S76 is YES, and the process proceeds to step S77. On the other hand, if a command to display a moving image has not been received, the determination in step S76 is NO, and the process proceeds to step S80.

[0149] In step S77, the UI control unit 417 receives a user instruction to specify a virtual device for performing the simulation. The virtual device may be a default device corresponding to the function realized by the operation data, or may be specified by the user.

[0150] In step S78, the database management unit 415 creates a moving image by simulating the actions realized by the action data selected in step S72. In step S79, the database management unit 415 displays the moving image created in step S78.

[0151] In step S80, the UI control unit 417 determines whether or not to end the motion data browsing process. For example, the motion data browsing process ends when a user operation instructing the end of the motion data browsing process is received. If the motion data browsing process ends, a YES determination is made in step S80, and the motion data browsing process returns. That is, step S36 in FIG. 11 ends, and the process proceeds to step S37. On the other hand, if the motion data browsing process does not end, a NO determination is made in step S80, and the process returns to step S72 and is repeated.

[0152] According to the above-described operation data viewing process, operation data can be displayed in a manner that is easy for the user to intuitively understand, such as operation data converted into a graph or a video of the operation simulated using a virtual device, thereby assisting the user in their search.

[0153] 14 is a flowchart illustrating the flow of the action data generation process executed by the action data management system S. The action data generation process is executed in response to the action data management device 40 receiving a user's operation to start the action data generation process during the action data provision process.

[0154] In step S91, the UI control unit 417 displays a motion data generation screen as a user interface screen. On this motion data generation screen, for example, operation buttons for selecting motion data to be generated and adjusting parameters are arranged.

[0155] In step S92, the database management unit 415 selects one or more pieces of motion data in response to a user's instruction to select the motion data. The selected one or more pieces of motion data become components of the motion data to be generated.

[0156] In step S93, the UI control unit 417 determines whether the user's instruction operation to select the action data has been completed. If the instruction operation to select the action data has been completed, the determination in step S93 is YES, and the process proceeds to step S94. On the other hand, if the selection instruction operation to select the action data has not been completed, the determination in step S93 is NO, and the process returns to step S92 and is repeated.

[0157] In step S94, the UI control unit 417 determines whether or not an instruction operation to set the order of the action data selected in step S92 has been received from the user. If an instruction operation to set the order has been received, the determination in step S94 is YES, and the process proceeds to step S95. On the other hand, if an instruction operation to set the order has not been received, the determination in step S94 is NO, and the process proceeds to step S97. Note that even if only one action data is selected in step S92, the determination in step S94 is NO, and the process proceeds to step S97.

[0158] In step S95, the database management unit 415 sets the order of the multiple action data selected in step S92 in response to the user's instruction to set the order, and by linking the multiple action data in this set order, generates one action data whose components are the multiple action data.

[0159] In step S96, the UI control unit 417 determines whether the user's instruction to set the sequential application has been completed. If the instruction to set the sequential application has been completed, the determination in step S96 is YES, and the process proceeds to step S97. On the other hand, if the instruction to set the sequential application has not been completed, the determination in step S96 is NO, and the process returns to step S95 and is repeated.

[0160] In step S97, the UI control unit 417 determines whether or not an instruction operation to set a transition condition has been received from the user for the action data selected in step S92. If an instruction operation to set a transition condition has been received, the determination in step S97 is YES, and the process proceeds to step S98. On the other hand, if an instruction operation to set a transition condition has not been received, the determination in step S97 is NO, and the process proceeds to step S100. Note that even if only one action data is selected in step S92, the determination in step S97 is NO, and the process proceeds to step S100.

[0161] In step S98, the database management unit 415 sets transition conditions for the plurality of pieces of action data selected in step S92 in response to the user's instruction to set transition conditions.

[0162] In step S99, the UI control unit 417 determines whether the user's instruction to set a transition condition has been completed. If the instruction to set a transition condition has been completed, the determination in step S99 is YES, and the process proceeds to step S100. On the other hand, if the instruction to set a transition condition has not been completed, the determination in step S99 is NO, and the process returns to step S98 and is repeated.

[0163] In step S100, the UI control unit 417 determines whether or not an instruction operation to adjust the control parameters has been received from the user for the action data selected in step S92. If an instruction operation to adjust the control parameters has been received, the determination in step S100 is YES, and the process proceeds to step S101. On the other hand, if an instruction operation to adjust the control parameters has not been received, the determination in step S100 is NO, and the process proceeds to step S103.

[0164] In step S101, the database management unit 415 adjusts the control parameters of the action data selected in step S92 in response to a user's instruction to adjust the control parameters.

[0165] In step S102, the UI control unit 417 determines whether the user has finished instructing the adjustment of the control parameters. If the user has finished instructing the adjustment of the control parameters, the determination in step S102 is YES, and the process proceeds to step S103. On the other hand, if the user has not finished instructing the adjustment of the control parameters, the determination in step S102 is NO, and the process returns to step S101 and is repeated.

[0166] In step S103, the UI control unit 417 determines whether to end the motion data generation process. For example, the motion data generation process ends when a user operation to end the motion data generation process is received. If the motion data generation process is to be ended, a YES determination is made in step S103, and the process proceeds to step S104. On the other hand, if the motion data generation process is not to be ended, a NO determination is made in step S103, and the process returns to step S92 and is repeated.

[0167] In step S104, the normalization processing unit 414 denormalizes the motion data selected in step S92, for which connection and transition conditions have been set, control parameters have been adjusted, etc. as necessary. As a result, motion data in functional coordinates is generated from the motion data in generalized coordinates.

[0168] In step S105 , the database management unit 415 stores the action data generated by the denormalization in step S104 in the action data database 421 .

[0169] In step S106, the provision request response unit 416 transmits the operation data generated by the denormalization in step S104 to the control device 30 in the local system designated by the user and downloads the data as provision data, thereby providing the operation data. This causes the operation data provision process to return. That is, step S38 in FIG. 11 ends, and the process proceeds to step S39.

[0170] According to the operation data generation process described above, it is possible to generate one continuous piece of operation data from multiple pieces of operation data, or to convert the operation data into operation data of functional coordinates corresponding to the local system that provides the operation data, depending on the user's selection.

[0171] As described above, the motion data management system S executes each process to assign information related to the motion details as identification information and manages desired motion data in a searchable manner using this identification information. This allows the user to perform a search based on easily comprehensible information related to the motion details (e.g., the motion's function, motion conditions, etc.). In other words, the motion data management system S treats each piece of motion data as content and manages this content as a library, thereby improving user convenience and enabling the use of motion data. Therefore, the motion data management system S can solve the problem of more appropriately managing haptic data.

[0172] [Modifications] While the embodiments of the present invention have been described above, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take on various other embodiments and can be modified in various ways, such as omissions and substitutions, without departing from the spirit of the present invention.

[0173] [Variation 1] In the above-described embodiment, the control device 30 and the motion data management device 40 are realized as separate devices. However, this is not limiting, and for example, the control device 30 and the motion data management device 40 may be realized as an integrated device. In other words, the control device 30 may be realized as a device that acquires motion data by performing control to transmit haptic sensations and that can manage this motion data by itself.

[0174] Alternatively, for example, the first drive unit 10, the second drive unit 20, and the control unit 30 do not necessarily have to be included in a set. For example, a set of only two units, the control unit 30 and the second drive unit 20, may exist. In this case, actual operation transmission control cannot be performed in this set of only two units. However, instead, the control unit 30 in this set of only two units can control the second drive unit 20 by performing data transmission control based on the provided data provided by the operation data management device 40.

[0175] Alternatively, for example, there may be a set including one control device 30, a plurality of first drive units 10, and a plurality of second drive units 20. In such a set, for example, the plurality of first drive units 10 operate as master units of a multi-degree-of-freedom robot arm, and the plurality of second drive units 20 operate as slave units of the multi-degree-of-freedom robot arm, and the operations of all of these are controlled by the single control device 30.

[0176] Alternatively, for example, the control device 30 may be realized as a device integrated with the first drive device 10 or the second drive device 20. Alternatively, for example, the action data management device 40 may be realized by a plurality of server devices constituting a cloud server. In other words, the system configuration in the above-described embodiment is merely an example, and various configurations can be appropriately modified.

[0177] [Variation 2] The allocation of devices that realize each functional block in the above-described embodiment may be changed. For example, an input unit and an output unit may be provided in the control device 30, and a user may perform a series of operations on the action data management device 40 via the input unit and output unit of the control device 30. Alternatively, for example, a denormalization unit that performs denormalization may be provided in the control device 30. Then, the action data management device 40 may provide the action data in generalized coordinates to the control device 30 as provided data, and the control device 30 may denormalize the provided data to generate action data in functional coordinates.

[0178] [Modification 3] Predetermined processing may be added to the above-described embodiment. For example, preprocessing for the purpose of noise removal may be performed on the raw data of the motion data acquired by the motion data acquisition unit 312 in the motion data accumulation process.

[0179] For example, a filter that passes only a predetermined frequency band and attenuates other frequency bands is used as the pre-processing. In this case, for example, a low-pass filter is used to remove high-frequency noise components from the raw motion data.

[0180] Alternatively, preprocessing may be performed to remove outliers by ignoring detected values ​​of parameters included in the raw data of motion data that exceed a predetermined threshold and replacing them with the immediately preceding detected value. Alternatively, preprocessing may be performed to saturate detected values ​​of parameters included in the raw data of motion data that exceed a predetermined threshold and replace them with the threshold. This makes it possible to appropriately remove noise included in the raw data of motion data.

[0181] [Variation 4] In the above-described embodiment, haptic sensations are transmitted based on the control algorithm described with reference to FIG. 8 . In this case, as described above, the time-series position (angle) values ​​detected in real time by the position sensor 23 (or position sensor 13) and the force values ​​calculated from these positions (angles) are used as reference values ​​in the control algorithm. This is not limited to this, and a force sensor may be used in addition to a position sensor. In this case, a force sensor is provided in addition to a position sensor in each of the first drive unit 10 and the second drive unit 20. Then, the time-series force values ​​detected in real time by this force sensor are used as the reference value in the control algorithm, rather than the force values ​​calculated from the time-series position (angle) values ​​detected in real time by the position sensor. This allows, for example, if a force sensor capable of detecting force values ​​with high accuracy is available, to transmit haptic sensations using this force sensor.

[0182] [Variation 5] In the above-described embodiment, in the action data search process, the UI control unit 417 displays an action data search screen as a user interface screen. The user then refers to this screen and searches for action data by entering a search key such as a file name or tag. Action data may be searched for by other methods. For example, the control device 30 of the local system may transmit an action data provision request to the provision request response unit 416, the request including tags corresponding to the action to be performed and information such as the system attributes of its own local system. The database management unit 415 may then search for action data corresponding to this provision request and provide it to the control device 30 as provision data. This allows the local system including the control device 30 to automatically acquire action data. It is preferable that the provision request be made via, for example, an API (Application Programming Interface) so that various local systems can issue the provision request in a general-purpose manner.

[0183] As described above, the motion data management system S according to this embodiment includes a motion data acquisition unit 411, a tagging unit 413, and a database management unit 415. The motion data acquisition unit 411 acquires, as motion data, time-series control parameters used to transmit haptic sensations between devices. The tagging unit 413 assigns, to the motion data acquired by the motion data acquisition unit 411, information regarding the motion content realized by the motion data as identification information. The database management unit 415 accumulates multiple pieces of motion data to which identification information has been assigned by the tagging unit 413 and manages desired motion data from the accumulated multiple pieces of motion data in a searchable manner based on the identification information. In this way, the motion data management system S assigns information regarding the motion content as identification information and manages desired motion data in a searchable manner using this identification information. This allows the user to perform searches based on easily comprehensible information regarding the motion content (e.g., the function of the motion, the motion conditions, etc.). That is, the motion data management system S handles each piece of motion data as content and manages this content as a library, thereby improving user convenience and enabling the use of motion data. Therefore, the motion data management system S can solve the problem of more appropriately managing data related to haptics.

[0184] The motion data management system S further includes a provision request response unit 416 and a normalization processing unit 414. The provision request response unit 416 provides the motion data searched based on the identification information to a control device that performs control for transmitting a haptic sensation. The normalization processing unit 414 normalizes the motion data acquired by the motion data acquisition unit 411 based on the attributes of the device to which the haptic sensation is to be transmitted using the motion data, and denormalizes the motion data provided by the provision request response unit 416 based on the attributes of the device to which the haptic sensation is to be transmitted using the motion data. As a result, even if the attributes of the device from which the motion data was acquired differ from those of the device that transmits a haptic sensation using the provided motion data, these differences can be absorbed to transmit a haptic sensation using the motion data.

[0185] The provision request response unit 416 links the multiple pieces of motion data retrieved based on the identification information and provides the multiple pieces of motion data as a single piece of motion data to the control device. This allows the control device to continuously transmit the haptic sensations of the multiple pieces of motion data retrieved, thereby enabling the multiple motions to be realized as a series of motions.

[0186] The motion data management system S further includes a motion decomposition unit 412. The motion decomposition unit 412 decomposes one continuous piece of motion data acquired by the motion data acquisition unit 411 into a plurality of pieces of motion data according to the contents of the plurality of motions realized by the motion data. This allows each of the plurality of continuous motions to be managed as separate motion data.

[0187] The database management unit 415 excludes some of the accumulated multiple pieces of motion data from the search targets based on at least one of the attributes of the device to which the haptic sensation is to be transmitted using the motion data searched based on the identification information, the attributes of the control device that controls the transmission of the haptic sensation to the device, and the attributes of the contact object that the device comes into contact with during the operation of the device. This makes it possible to exclude (i.e., filter) from the search targets, for example, motion data that does not match the attributes of the device, control device, and contact object to which the haptic sensation is to be transmitted.

[0188] The database management unit 415 realizes the operation of the virtual device by transmitting haptic sensations to the virtual device using the operation data searched based on the identification information, and displays the realized operation of the virtual device. This makes it possible to display to the user, for example, using computer graphics, what operation will be realized by the operation data. This allows the user to intuitively understand the operation realized by the operation data.

[0189] In a system including an actuator 12, a position sensor 13, an actuator 22, and a position sensor 23 (311), the actuator 12 moves a first moving object in association with the execution of a predetermined operation. The position sensor 13 acquires first position information relating to the position of the first moving object moved by the actuator 12. The actuator 22 moves a second moving object in association with the execution of a predetermined operation. The position sensor 23 acquires second position information relating to the position of the second moving object moved by the actuator 22. The actuator 12 controls the actuator 12 based on the first position information and the second position information serving as a reference for the operation of the actuator 12 to output a position and force corresponding to the operation represented by the second position information, while the actuator 22 controls the actuator 22 based on the second position information and the first position information serving as a reference for the operation of the actuator 22 to output a position and force corresponding to the operation represented by the first position information. The operation data acquisition unit 411 acquires, as operation data, at least a portion of the time-series control parameters used by the actuator 12 and the actuator 22 (311). This makes it possible to independently apply control energy for position (angle) and control energy for force, i.e., independently control position (angle) and force, with a simple configuration in which the position is detected by a position sensor.

[0190] [Realization of Functions by Hardware or Software] The function of executing the series of processes according to the above-described embodiment can be realized by hardware, software, or a combination of these. In other words, it is sufficient that the function of executing the series of processes described above is realized in any of the action data management systems S, and there are no particular limitations on how this function is realized.

[0191] For example, when the function of executing the above-mentioned series of processes is realized by a processor that executes arithmetic processing, the processor that executes this arithmetic processing includes processors that are configured by various types of processing devices alone, such as single processors, multiprocessors, and multicore processors, as well as processors that combine these various types of processing devices with processing circuits such as ASICs (Application Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays).

[0192] Furthermore, for example, when the function of executing the above-described series of processes is realized by software, the program constituting the software is installed on a computer via a network or a recording medium. In this case, the computer may be a computer incorporating dedicated hardware, or may be a general-purpose computer (e.g., a general electronic device such as a general-purpose personal computer) that can execute predetermined functions by installing a program. Furthermore, the steps of writing the program may include only processes that are executed chronologically according to the order, but may also include processes that are executed in parallel or individually. Furthermore, the steps of writing the program may be executed in any order within the scope of the present invention.

[0193] A recording medium on which such a program is recorded may be provided to a user by being distributed separately from the computer main unit, or may be provided to a user in a state where it is pre-installed in the computer main unit. In this case, the storage medium distributed separately from the computer main unit is, for example, removable media 50, which may be a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. An optical disk may be, for example, a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), or a Blu-ray (registered trademark) Disc. An magneto-optical disk may be, for example, an MD (Mini Disc). Furthermore, the recording medium provided to the user in a state where it is pre-installed in the computer main body is, for example, the storage unit 32 or storage unit 42 on which the program is recorded, and is configured by an HDD (hard disk drive) or an SSD (solid state drive).

[0194] 10-1, ..., 10-i First driving device, 11, 21 Driver, 12, 22 Actuator, 13, 23 Position sensor, 15 First mechanism, 20-1, ..., 20-j Second driving device, 25 Second mechanism, 30-1, ..., 30-k Control device, 31, 41 Processor, 32, 42 Storage unit, 33, 43 ROM, 34, 44 RAM, 35, 45 Communication unit, 46 Input unit, 47 Output unit, 48 Drive, 40 Motion data management device, 50 Removable media, 311 Haptic control unit, 312, 411 Motion data acquisition unit, 313 Provided data acquisition unit, 321, 421 Motion data storage unit, 322 Provided data storage unit, 412 Motion decomposition unit, 413 Tagging unit, 414 Normalization processing unit, 415 Database management unit, 416 Provision request response unit, 417 UI control unit, CS controlled system, FT force / velocity allocation conversion block, FC ideal force source block, PC ideal velocity (position) source block, IFT inverse conversion block, N network, S operation data management system

Claims

1. an acquisition means for acquiring, as operation data, time-series control parameters used in transmitting haptic sensations between the devices; an assigning means for assigning, to the action data acquired by the acquiring means, information regarding the action content realized by the action data as identification information; a management means for storing a plurality of pieces of motion data to which the identification information has been added by the adding means, and for managing a desired motion data from the plurality of stored motion data in a manner that allows the motion data to be searched for based on the identification information; An operation data management system comprising:

2. providing means for providing the motion data searched for based on the identification information to a control device that performs control for transmitting a haptic sensation; a normalization processing means for normalizing the motion data acquired by the acquisition means based on attributes of a device to which a haptic sensation is to be transmitted by the motion data, and for non-normalizing the motion data provided by the provision means based on attributes of a device to which a haptic sensation is to be transmitted by the motion data; The operation data management system according to claim 1, further comprising:

3. the providing means provides the control device with one piece of action data including the plurality of action data as components by linking the plurality of action data searched based on the identification information.

3. The operation data management system according to claim 2.

4. a decomposition means for decomposing one continuous piece of motion data acquired by the acquisition means into a plurality of pieces of motion data according to a plurality of motion contents realized by the motion data; 4. The motion data management system according to claim 1, further comprising:

5. the management means excludes a portion of the accumulated plurality of motion data from the search target based on at least one of an attribute of a device to which a haptic sensation is to be transmitted by the motion data searched based on the identification information, an attribute of a control device which controls to transmit a haptic sensation to the device, and an attribute of a contact object which the device contacts during the operation of the device.

4. The motion data management system according to claim 1, wherein the motion data management system is a system for managing motion data.

6. the management means realizes an operation of the virtual device by transmitting a haptic sensation to the virtual device based on the operation data searched based on the identification information, and displays the realized operation of the virtual device.

4. The motion data management system according to claim 1, wherein the motion data management system is a system for managing motion data.

7. a first driving means for moving a first moving object in association with execution of a predetermined operation; a first position information acquisition means for acquiring first position information relating to a position of the first moving object moved by the first driving means; a second driving means for moving a second moving object in association with the execution of the predetermined operation; a second position information acquisition means for acquiring second position information relating to a position of the second moving object moved by the second driving means; a control means for controlling the first driving means, based on the first position information and the second position information serving as a reference for the operation of the first driving means, so as to output a position and force corresponding to the operation represented by the second position information, while controlling the second driving means, based on the second position information and the first position information serving as a reference for the operation of the second driving means, so as to output a position and force corresponding to the operation represented by the first position information; If exists, the acquiring means acquires, as the operation data, at least a part of a time-series control parameter used by the control means to control the first drive means and the second drive means.

4. The action data management device according to claim 1, wherein the action data management device is a device for managing an action of a user.

8. an acquisition step of acquiring, as operation data, time-series control parameters used in transmitting haptic sensations between the devices; an assigning step of assigning, as identification information, information regarding the content of an action realized by the action data to the action data acquired in the acquiring step; a management step of storing a plurality of pieces of motion data to which the identification information has been added in the adding step, and managing the plurality of pieces of motion data stored so that a desired motion data can be searched for based on the identification information; 13. An operational data management method comprising:

9. An acquisition function for acquiring, as operation data, time-series control parameters used in transmitting haptic sensations between devices; an assignment function that assigns, to the action data acquired by the acquisition function, information regarding the action content realized by the action data as identification information; a management function that stores a plurality of pieces of motion data to which the identification information is added by the adding function, and manages the plurality of pieces of motion data stored so that a desired motion data can be searched for based on the identification information; A program characterized by causing a computer to realize the above.