Actuation system

Abstract

Provided is an actuation system for appropriately controlling the motion of an actuated body even if there is a possibility that the actuated body follows an unintended path due to a certain operation performed by a user. In order to simulate the motion of an actuated body that performs a prescribed motion in accordance with an input operation from an operation device operated by a user, this actuation system comprises: a simulation unit that simulates, in a virtual space, the positional relationship between a virtual actuated body corresponding to the actuated body and a virtual restriction unit that is virtually provided for restricting movement of the virtual actuated body; and a motion control unit that controls the motion of the actuated body in accordance with the simulated positional relationship. The simulation unit is configured to correct the direction of movement of the virtual actuated body corresponding to the input operation on the basis of the shape of the virtual restriction unit, when it is determined that the distance between the virtual actuated body and the virtual restriction unit is within a prescribed range.

Description

Actuation system

[0001] The present invention relates to an actuation system.

[0002] Conventionally, techniques have been proposed for preventing inconveniences such as interference with obstacles when a working device, such as a robot that performs work on behalf of a human, performs a work (see, for example, Patent Document 1). The interference prevention device described in Patent Document 1 is configured to calculate a motion trajectory of the robot before the robot starts to operate based on content that has been taught to the robot in advance, determine based on the calculation result whether the robot will interfere with an obstacle, and, if it is determined that the robot will interfere with an obstacle, issue a standby command to the robot, or the like.

[0003] Japanese Patent Application Publication No. 8-36410

[0004] However, the technology described in Patent Literature 1 controls movements along a predetermined trajectory based on teaching. Therefore, if a user controls a robot to move along an arbitrary trajectory via an operating device, the technology described in Patent Literature 1 may not be able to properly control the robot's movements. Specifically, if a user controls a robot to move along an arbitrary trajectory via an operating device, the robot may move along an unexpected trajectory. Therefore, the technology described in Patent Literature 1 may not be able to properly control the robot's movements, for example, the robot may interfere with an obstacle during operation.

[0005] In view of the above problems, the present invention aims to provide an actuation system that makes it possible to appropriately control the movement of an actuation body even if there is a possibility that the actuation body will move in an unexpected trajectory due to arbitrary operation by the user.

[0006] The actuation system of the present invention is an actuation system including a simulating unit that simulates, in a virtual space, a positional relationship between a virtual actuating body corresponding to the actuating body and a first virtual regulating unit that is virtually provided to regulate the movement of the virtual actuating body, in order to simulate the action of the actuating body that performs a predetermined action in response to an input operation from an operating device by a user, and an action control unit that controls the action of the actuating body in response to the positional relationship simulated by the simulating unit, wherein, when an input operation is performed on the actuating body, the simulating unit operates the virtual actuating body in the virtual space with content corresponding to the input operation in parallel with the input operation, and determines the positional relationship between the virtual actuating body and the first virtual regulating unit at predetermined time intervals, and the action control unit operates the actuating body in parallel with an action command based on the input operation, based on a result of the determination by the simulating unit, determines whether the distance between the virtual operating body and the first virtual regulating portion is within a predetermined range, and if it is determined that the distance between the virtual operating body and the first virtual regulating portion is within the predetermined range, and if the movement direction of the virtual operating body corresponding to the input operation includes a component in a direction toward the surface of the first virtual regulating portion and a component in a direction along the surface of the first virtual regulating portion, the simulator unit corrects the movement direction of the virtual operating body corresponding to the input operation based on the shape of the first virtual regulating portion, thereby (i) determining the movement direction of the virtual operating body for moving the virtual operating body along the surface of the first virtual regulating portion, and the operation control unit is configured to move the operating body in the determined movement direction, or (ii) stopping the virtual operating body against the surface of the first virtual regulating portion, and the operation control unit is configured to stop the operating body against the first virtual regulating portion.

[0007] According to the present invention, it is possible to provide an actuation system that makes it possible to appropriately control the movement of an actuation body even if there is a possibility that the actuation body will move in an unexpected trajectory due to arbitrary operation by the user.

[0008] 1 is a perspective view schematically showing an example of an actuation system according to an embodiment of the present invention. FIG. 1 is a block diagram showing the configuration of the actuation system shown in FIG. 1. FIG. 2 is a diagram showing a simulation image displayed on the display unit shown in FIG. 3. FIG. 3 is a perspective view schematically showing an example of an actuation body and a destination position of the actuation system shown in FIG. 1. FIG. 5 is a perspective view schematically showing a state in which a virtual actuation body corresponding to the actuation body shown in FIG. 5 is guided to a virtual destination position by a first virtual regulating part. FIG. 6 is a diagram showing an example of a method for determining a direction in which a virtual actuation body is to be moved when the simulating part of the actuation system shown in FIG. 1 moves the virtual actuation body along the surface of the first virtual regulating part while bringing the virtual actuation body into contact with the first virtual regulating part. FIG. 7 is a perspective view schematically showing a state in which a virtual actuation body corresponding to the actuation body shown in FIG. 5 is guided to a virtual destination position by the first virtual regulating part, in which the virtual actuation body is tilted. FIG. 8 is a perspective view schematically showing a state in which a virtual actuation body corresponding to the actuation body shown in FIG. 5 is inserted into a hole, which is the virtual destination position, in which the virtual actuation body is tilted with respect to the hole. 6 is a diagram schematically illustrating a state in which a virtual operating body corresponding to the operating body shown in FIG. 5 is inserted into a hole that is a virtual destination position, and is a perspective view of the virtual operating body being pressed along the wall surface of the hole. FIG. 7 is a diagram schematically illustrating a state in which the positional relationship between a virtual work object and the virtual destination position is displayed on a display screen. FIG. 8 is a diagram schematically illustrating a state in which the positional relationship between a tip surface of the virtual work object and the virtual destination position is displayed on a display screen. FIG. 9 is a diagram schematically illustrating a state in which the positional relationship between a tip surface of the virtual work object and the virtual destination position is displayed on a display screen. FIG. 10 is a diagram schematically illustrating a state in which the positional relationship between a tip surface of the virtual work object and the virtual destination position is displayed on a display screen. FIG. 11 is a perspective view showing a state in which a virtual operating body corresponding to the operating body shown in FIG. 5 passes through a side wall of a first virtual restricting portion from the outside to the inside. FIG. 12 is a perspective view schematically illustrating a state in which a prismatic virtual operating body is guided to the virtual destination position by the first virtual restricting portion.21 is a schematic view of an example of an actuation system according to another embodiment of the present invention, and is a perspective view showing a state in which an insertion target portion of a virtual actuating body is guided in a direction oblique to the axis of a hole, which is a virtual destination position. 17 is a perspective view showing a state in which a tip of the insertion target portion shown in FIG. 17 is inserted into a hole. 18 is a perspective view showing a state in which a first virtual restriction portion has been eliminated in the state shown in FIG. 19 is a partial cross-sectional view showing a state in which a tip of the insertion target portion is inserted into a hole in the state shown in FIG. 20 is a partial cross-sectional view showing a state in which the insertion target portion shown in FIG. 20 is in contact with the periphery of the opening of the hole. 21 is a perspective view showing a state in which first virtual restriction portions are generated so as to sandwich the insertion target portion from both radial sides. 22 is a partial cross-sectional view showing a state in which the insertion target portion shown in FIG. 22 is swung and in contact with the periphery of the opening of the hole at multiple positions. 23 is a partial cross-sectional view showing a state in which the insertion target portion shown in FIG. 23 has moved in a direction to come out of the hole. 25 is a partial cross-sectional view schematically showing a state in which the insertion target part shown in FIG. 24 has swung and is in contact with the periphery of the opening of the hole at a plurality of positions. FIG. 26 is a partial cross-sectional view schematically showing a state in which the insertable condition of the insertion target part is satisfied. FIG. 27 is a perspective view schematically showing an example of an actuation system according to another embodiment of the present invention, showing a manner in which a work object is grasped by an actuation body. FIG. 28 is a perspective view schematically showing a virtual actuation body corresponding to the actuation body shown in FIG. 27, and a virtual work object grasped by the virtual actuation body. FIG. 29 is a perspective view schematically showing a state in which the virtual actuation body shown in FIG. 28 is positioned at a target position for grasping the virtual work object. FIG. 29 is a perspective view schematically showing a state in which the virtual actuation body shown in FIG. 28 grasps the virtual work object. 1 is a diagram showing a schematic view of a positional relationship between a first virtual regulating portion and a second virtual regulating portion when a direction connecting two first virtual regulating portions does not match a direction connecting two second virtual regulating portions before the first virtual regulating portion and the second virtual regulating portion are aligned in a manner in which a work object is grasped by an acting body.

[0023] FIG. 1 is a diagram showing a schematic view of an example of an acting system according to another embodiment of the present invention, and is a perspective view showing a manner in which a virtual acting body is guided along a predetermined movement path.36 is a perspective view showing an example of an actuation system according to another embodiment of the present invention, the perspective view showing an aspect including a ring-shaped second virtual regulating portion and a linear first virtual regulating portion that guides the second virtual regulating portion. FIG. 37 is a perspective view showing a modification of the aspect shown in FIG. 33 , showing a configuration in which the second virtual regulating portion cannot rotate in a direction around the axis of the first virtual regulating portion. FIG. 38 is a perspective view showing an example of an actuation system according to another embodiment of the present invention, showing a configuration in which the virtual regulating portion is formed in a plate shape and a rod-shaped virtual operating body can enter the virtual regulating portion in a thickness direction. FIG. 39 is a perspective view showing an example of an actuation system according to another embodiment of the present invention, showing a configuration in which the virtual regulating portion is disposed inside a virtual work object. FIG. 39 is a perspective view showing a modification of the aspect shown in FIG. 36 , showing a configuration in which the virtual operating body makes surface contact with the virtual regulating portion. FIG. 39 is a perspective view showing an example of an actuation system according to another embodiment of the present invention, showing an aspect in which, when work is performed by multiple operating bodies, virtual regulating portions provided between multiple virtual operating bodies prevent the operating bodies from contacting each other. FIG. 39 is a perspective view showing a state in which the virtual regulating portion has been eliminated in the aspect shown in FIG. 40 is a perspective view showing a state in which the virtual restricting portions are in contact with each other in the state shown in FIG. 40 . FIG. 40 is a perspective view showing a state in which the virtual restricting portions are reduced in size in the state shown in FIG. 40 . FIG. 40 is a perspective view showing a state in which the virtual restricting portions have disappeared in the state shown in FIG. 40 . FIG. 40 is a perspective view showing a state in which the virtual restricting portions have disappeared in the state shown in FIG. 40 . FIG. 40 is a perspective view showing a state in which the virtual restricting portions have disappeared in the state shown in FIG. 40 . FIG. 40 is a perspective view showing a state in which the virtual restricting portions have disappeared in the state shown in FIG. 44 . FIG. 40 is a perspective view showing a state in which the virtual restricting portions have disappeared in the state shown in FIG. 44 .

[0009] An actuation system according to an embodiment of the present invention will be described below with reference to the drawings. Note that the embodiment described below is merely an example, and the actuation system of the present invention is not limited to the following embodiment.

[0010] The actuation system WS described below enables appropriate control of the action of an actuating body by simulating the action of the actuating body, which performs a predetermined action in response to an input operation from a user via an operating device. In this specification, the actuating body refers to any structure that performs a predetermined action in response to an input operation from a user via an operating device OD (see FIG. 1). In the embodiment described below, the actuating body is configured as a working device WD (see FIG. 1) such as a robot, but the actuating body is not limited to a working device such as a robot. Note that the actuating body may be configured as a single structure such as the working device WD, or may include a single structure and other elements that move in conjunction with the structure. For example, when a specific work object OB1 (see FIG. 1) is operated by the working device WD (when a part of the working device WD and the work object OB1 move integrally), the working device WD, which is a first actuating element, and the work object OB1, which is a second actuating element, are treated as a single actuating body.

[0011] The actuation system WS (see FIG. 2) according to this embodiment is a system that performs work on a work object OB1 using a work device WD, such as a robot that performs work in place of a person, as shown in FIG. 1. Specifically, for example, the work device WD is configured to perform a predetermined operation in response to an input operation by a user via an operating device OD. In this embodiment, the work device WD is configured to perform a predetermined operation in various facilities such as production facilities in response to an operation command that indicates the content of the operation to be performed by the work device WD.

[0012] In this specification, the "operation of the working device WD" is not limited to, but includes at least one of, for example, transporting the work object OB1, positioning the work object OB1, moving relative to the work object OB1, and following the work object OB1. An example of an operation that includes transporting the work object OB1 is, but is not limited to, pick-and-place (grasping the work object OB1 and transporting it to a target position). An example of an operation that includes positioning the work object OB1 is fitting (grasping the work object OB1 and fitting it into a hole). An example of an operation that includes movement relative to the work object OB1 is, but is not limited to, painting, polishing, writing, scooping gravel, etc. In this embodiment, the working device WD is configured to guide the work object OB1 to a target location, as described below. In other embodiments described below, the work device WD is configured to prevent interference with other work devices WD, surrounding structures, and the work device WD itself when performing the intended work. In still other embodiments, the work device WD is configured to control the distance of the work device WD from the work object OB1, and in still other embodiments, the work device WD is configured to follow the moving work object OB1.

[0013] In this embodiment, the work device WD is configured to be operable in manual mode. The work device WD may be a device that operates in semi-automatic mode or automatic mode in addition to manual mode. The manual mode is a mode in which the work device WD is always operated by the user. In semi-automatic mode, only some predetermined operations are performed automatically, and other operations are operated by the user. In automatic mode, all operations are performed automatically using sensors or the like. In this specification, an "operation command" is a command indicating the content of an operation to be performed by the work device WD. The operation to be performed by the work device WD is, for example, an operation that the user wants the work device WD to perform via the operation device OD, or an operation that a program in a computer (a computer for executing operations in automatic mode or semi-automatic mode) that causes the work device WD to perform an automatic operation is wanting the work device WD to perform. The operation to be performed by the work device WD can be corrected by interference prevention control and / or tracking control, which will be described later, based on the results of simulation by the simulator 1.

[0014] The working device WD is configured to operate based on operation commands input from the operating device OD. The operation commands are generated by a program that generates operation commands based on input operations from the operating device OD. The program has the function of converting operation requests into operation commands by performing calculations based on operation requests for operating the working device WD according to the content of the operation to be performed by the working device WD. The program is stored, for example, in a memory unit of a computer (a computer other than the simulator 1 described below) used to operate or run the working device WD. In the manual mode shown in FIG. 1 , the computer is provided, for example, in the operating device OD. In the automatic mode, the computer is provided, for example, in the computer that causes the working device WD to perform automatic operations. In the semi-automatic mode, the computer is provided, for example, in the computer that causes the operating device OD and the working device WD to perform automatic operations. The program is configured to perform calculations to convert an operation request (for example, an operation request by a user or a predetermined operation request in an automatic operation) for operating the work device WD according to the content of the operation that the work device WD is to perform into an operation command.

[0015] The operation request is converted into an operation command for each component of the working device WD (e.g., the arm unit WD2, etc.) based on the calculation result. The operation command is operation information for the components of the working device WD, for realizing the operation of the working device WD corresponding to the operation request. Specifically, the operation information is parameter values ​​for realizing the operation of the components of the working device WD corresponding to the operation request, such as one or more of the movement speed, acceleration / deceleration, movement direction, movement distance, joint rotation angle, rotation speed, and rotation acceleration / deceleration of the components of the working device WD. When the working device WD operates in manual mode, the operation command is, for example, operation information (parameter values) for realizing the movement of the components of the working device WD corresponding to the operation applied to the operating device OD by the user. In this case, the operating device OD converts the content of the operation applied to the operating device OD by the user into operation information for realizing the movement of the components of the working device WD. When the working device WD operates in automatic mode, the operation command is, for example, operation information for realizing the movement of the components of the working device WD, corresponding to the operation that a computer that automatically operates the working device WD intends the working device WD to perform. In this case, the computer converts the content of the operation that the computer intends the working device WD to perform into operation information for realizing the movement of the components of the working device WD. When the working device WD operates in semi-automatic mode, the operation command is a command that corresponds to both manual mode and automatic mode. As will be described later, in this embodiment, the above-mentioned calculations and converted operation commands by a program stored in a predetermined computer are transmitted to a simulator 1, which will be described later. The simulator 1 receives and uses the converted operation command, thereby eliminating the need for the simulator 1 to perform the above-mentioned calculations and conversion process.

[0016] The structure of the working device WD may be modified as appropriate depending on the type of work required, and is not limited to the structure shown in the drawings. In this embodiment, the working device WD is shown as an articulated robot, but is not limited to a robot and may be a construction machine such as a crane. The working device WD may be a device that is operated (remotely controlled) from a location distant from the working device WD, or may be a device that is not remotely controlled but is operated in the vicinity of the working device WD (for example, by on-machine operation). The following description will be given assuming that the working device WD is a robot that operates in manual mode and is remotely controlled by a user. The following description is also applicable to automatic mode and semi-automatic mode.

[0017] The working device WD shown in FIG. 1 is a device that performs work by applying a predetermined operation to a work object OB1. In the example shown in FIG. 1, the working device WD is configured to be movable in a predetermined direction in response to a user's operation and to perform a predetermined task. The working device WD includes a base unit WD1, an arm unit WD2 having multiple joints and rotatably fixed to the base unit WD1, and a tool unit WD3 provided at the tip of the arm unit WD2. The tool unit WD3 corresponds to a tool or machine tool that performs work on the work object OB1 and performs an operation on the work object OB1 according to the type of work to be performed by the working device WD. In this embodiment, the tool unit WD3 includes fingers WD31 that grip the work object OB1. In this embodiment, the working device WD is configured to be movable in a first direction D1, which is one horizontal direction (X direction), a second direction D2, which is a horizontal direction perpendicular to the first direction D1 (Y direction), and a third direction D3, which is a vertical direction. Specifically, the base unit WD1 rotates around an axis extending in the third direction D3, and the arm unit WD2 and tool unit WD3, which have multiple joints, move, thereby enabling movement in the first direction D1, the second direction D2, and the third direction D3. Note that the movement direction and movement method of the working device may be different from those of the working device WD shown in the figure. For example, the working device itself may not be fixed to the installation surface, but may move horizontally in the first direction D1 and the second direction D2 relative to the installation surface, or may be configured to move along a beam or pillar.

[0018] As shown in FIG. 1 , the working device WD may further include a measuring device WD4 for measuring the position of the work object OB1. The measuring device WD4 may include, for example, a camera WD41 for capturing an image of the work object OB1 and a measuring unit (not shown) such as a sensor for measuring the position of the work object OB1 based on the captured image. In this embodiment, the measuring device WD4 is provided on a base WD5 on which the working device WD is supported. However, the location of the measuring device WD4 is not particularly limited as long as it can measure the position of any part of the work object OB1. In this embodiment, the position (or angle or orientation) of the work object OB1 measured by the measuring device WD4 is associated with the outer shape of the work object OB1 and stored in a memory unit (not shown). This information is used when forming a virtual operating body (a portion in virtual space corresponding to an operating body formed by the working device WD and the work object OB1). The measuring device WD4 is not limited to a device having a camera, as long as it can measure the position of the work object OB1. It may also be a sensor other than a camera that can measure the position of the work object OB1. The measuring device WD4 may also be a device that can measure both the position (angle, orientation) and the external shape of the work object OB1. In this case, the position and the external shape measured by the measuring device WD4 are associated with each other and stored in a memory unit (not shown). The external shape of the work object OB1 may be stored in a predetermined memory device in advance. The virtual operating body may also be formed based on information from sources other than the measuring device WD4.

[0019] The working device WD is communicatively connected to an operating device OD that accepts user operations. The working device WD is configured to receive operation information indicating the content of the user's operation input to the operating device OD and perform an operation corresponding to the operation information. The configuration of the operating device OD is not particularly limited as long as it can instruct the working device WD to perform the content of the work, including moving the tool unit WD3. In the example shown in FIG. 1 , the operating device OD is configured to instruct the tool unit WD3 to move in any three-dimensional direction in the workspace in which the working device WD is actually installed. The operating device OD can be, for example, a 3D mouse, a joystick, a tablet, or the like.

[0020] In this embodiment, as shown in FIG. 1 , the tool unit WD3 is a gripping device that grips a transported object (described below), which is an example of the workpiece OB1. The gripping device includes multiple fingers WD31 that grip the workpiece OB1 and a drive unit (not shown), such as a motor, that drives the fingers WD31. The gripping device is configured to perform a gripping operation by narrowing the spacing between the fingers WD31 and to perform a release operation by widening the spacing between the fingers WD31. When gripping the workpiece OB1, the gripping device may be configured to adjust the orientation (inclination) of the fingers WD31 to a direction that makes gripping easier, depending on the outer shape and orientation of the workpiece OB1. In this case, the gripping device can appropriately grip the workpiece OB1 depending on the outer shape and orientation of the workpiece OB1. In the example shown in FIG. 1 , the workpiece WD performs pick-and-place of the workpiece OB1. Specifically, the work device WD performs a series of operations, including picking up (grasping and lifting) the work object OB1 at the start point of the pick-and-place, moving the work object OB1 to the end point of the pick-and-place, and releasing the work object OB1 at the end point.

[0021] 1, the work object OB1 is an object that is gripped by a tool unit (gripping device) WD3 and moved by a working device WD. In the example shown in Fig. 1, the work object OB1 is a columnar object such as a cylindrical or rectangular columnar object, but is not limited to this.

[0022] In the example shown in FIGS. 1 and 2 , the actuation system WS includes a simulator 1 that simulates, in a virtual space, the positional relationship between a virtual actuation body VW corresponding to the actuation body W and a virtual regulating unit (corresponding to a first virtual regulating unit) VB1 virtually provided to regulate the movement of the virtual actuation body VW in order to simulate the movement of an actuation body W (in this embodiment, a portion formed by the actuation body WD and the work object OB1 when the work object OB1 is gripped by the actuation body WD) that performs a predetermined action in response to an input operation from an operating device OD by a user. The simulator 1 and the action control unit 2 control the movement of the actuation body W in accordance with the positional relationship simulated by the simulator 1. Note that, in this embodiment, the simulator 1 and the action control unit 2 are shown schematically as separate units, but the simulator 1 and the action control unit 2 may be provided in the same device or in separate devices. Furthermore, the simulator 1 and the action control unit 2 may be realized as different functions of a single control unit. In this specification, the "first virtual restriction portion" is prefixed with "first" to distinguish it from the "second virtual restriction portion" described later, but even if it is described as a "first virtual restriction portion," it is not necessarily the case that a "second virtual restriction portion" is provided. Hereinafter, the first virtual restriction portion will also be referred to simply as the virtual restriction portion.

[0023] Before the working device WD is operated by the operation control unit 2, the simulator 1 simulates the positional relationship between a virtual operating body VW corresponding to the operating body W and a virtual regulating unit VB1 in the virtual space VS, as shown in FIG. 4 . More specifically, when an arbitrary operation input is made to the operation device OD, as will be described later, the simulator 1 performs a simulation to determine whether to operate the operating body as is, stop the operating body, or cause the operating body to perform a different operation, based on the operation input to the operation device OD. As will be described in detail later, the virtual regulating unit VB1 regulates the movement of the virtual operating body VW in the virtual space VS, thereby regulating the movement of the operating body W in the real space RS to a predetermined direction and range.

[0024] In this specification, a "virtual operating body" refers to a virtual object generated in a virtual space that corresponds to the operating body, specifically, the shape and size of the operating body. The virtual operating body may correspond to the entire operating body or a part of the operating body. If the working device WD is not associated with the work object OB1, the virtual operating body may be a virtual object corresponding to the working device WD (or a part thereof). If the working device WD moves together with the work object OB1, such as when gripping the work object OB1, the virtual operating body may be a virtual object corresponding to an object (or a part thereof) formed by the working device WD and the work object OB1. A "part of an object formed by the working device WD and the work object OB1" refers to a part of the object formed by the working device WD and the work object OB1 that serves as a reference for determining the positional relationship with the virtual regulating portion VB1. This part may be, for example, the work object OB1 or a part of the work object OB1. Specifically, the "part of the object in which the work device WD and the work object OB1 are integrated" is, for example, the tip of the work object OB1 (in the example shown in FIG. 1 , the tip (e.g., the tip surface) of the cylindrical work object OB1). The "positional relationship between the virtual operating body VW and the virtual regulating portion VB1" refers to the three-dimensional positional relationship between the virtual operating body VW and the virtual regulating portion VB1. More specifically, the "positional relationship between the virtual operating body VW and the virtual regulating portion VB1" refers to the positional relationship between the virtual operating body VW and the virtual regulating portion VB1 in the virtual space VS, and may be, but is not limited to, one or more of the following: the distance between the virtual operating body VW and the virtual regulating portion VB1; the orientation of the virtual operating body VW relative to the virtual regulating portion VB1; whether the virtual operating body VW is in contact with the virtual regulating portion VB1; and whether the virtual operating body VW is within a predetermined range (distance) of the virtual regulating portion VB1.

[0025] 1 and 2 , in the present embodiment, the simulator 1 is communicatively connected to the operating device OD and the operation control unit 2. The simulator 1 is configured to receive, from the operating device OD, an operation command corresponding to operation information indicating the content of a user's operation input to the operating device OD, and to simulate the content corresponding to the operation command (in the present embodiment, the content of the operation corresponding to the operation information). More specifically, the simulator 1 simulates the above-mentioned positional relationship by a computer simulation (computer simulation in a virtual space VS simulating a real space RS) based on form information of the virtual operating body VW corresponding to the form (shape and size) of the operating body W stored in a storage unit (not shown), form information (shape and size) of the virtual regulating part VB1 stored in the storage unit, relative position information relative to each other at a certain point in time, and the content of the above-mentioned operation command (in the present embodiment, the operation information, the movement of the components of the working device WD corresponding to the operation, or parameter values ​​for realizing the movement of the components).

[0026] The simulator 1 operates a virtual operating body VW (in this embodiment, a virtual object combining a virtual operating device VD and a virtual work object VOB1) corresponding to the operating body W in the virtual space VS in accordance with the operation command. Specifically, the simulator 1 operates the virtual operating device VD and the virtual work object VOB1 in accordance with the operation command in the virtual space VS. For example, when the virtual operating device VD grasps and transports the virtual work object VOB1, the virtual work object VOB1 grasped by the tool unit (gripping device) WD3 of the virtual operating device VD also moves in accordance with the movement of the tool unit WD3. Therefore, the simulator 1 simulates not only the operation of the virtual operating device VD but also the operation of the virtual work object VOB1. Note that if the work object OB1 is not present or if the work object OB1 does not move together with the operating device WD, the simulator 1 operates only the virtual operating device VD in the virtual space VS in accordance with the operation command.

[0027] When operating the virtual operating body, the simulator 1 receives an operation command from the outside (the operating device OD in the example shown in FIG. 1 ) and determines the positional relationship between the virtual operating body and the virtual regulating unit VB1 based on the received operation command. In this embodiment, the simulator 1 does not generate the operation command, but acquires it from the outside (the operating device OD in the example shown in FIG. 1 ). That is, the simulator 1 does not calculate the operation command, which is the operation information for the operation of the virtual operating device VD in the virtual space VS, but instead calculates the coordinates of the components of the virtual operating device VD based on the received calculated and converted operation command, and calculates the relative position information. Therefore, the simulator 1 does not require calculation and conversion time, enabling high-speed simulation. The simulator 1 can notify the operation control unit 2 of the simulation results. The simulator 1 can be configured, for example, using a known central processing unit (CPU) typically installed in a computer.

[0028] As shown in FIG. 3 , the simulator 1 may be communicatively connected to an operation input unit 3 and an output unit 4. The operation input unit 3 accepts user operation inputs for operating the simulator 1 and transmits the contents of the user operation inputs to the simulator 1. By including the operation input unit 3, the operating system WS can execute a simulation by the simulator 1 in response to the user's input and change the conditions of the simulation by the simulator 1. The operation input unit 3 can be embodied by known input means such as a keyboard, mouse, or controller. The output unit 4 is configured to output the results of the simulation by the simulator 1. The simulator 1 can notify the user of the results of the simulation through the output unit 4. The output unit 4 can be embodied by known data output means such as a display, printer, or speaker. Note that the simulation information by the simulator 1 does not necessarily have to be output to the output unit 4.

[0029] As shown in FIGS. 3 and 4 , the simulator 1 is capable of operating a virtual operating body VW (in this embodiment, a virtual operating device VD and / or a virtual work object VOB1) corresponding to an operating body W (in this embodiment, a working device WD and a work object OB1) in a virtual space VS in which the virtual operating body VW and a virtual regulating unit VB1 are arranged. In this specification, the term "virtual operating body VW corresponding to the operating body W" refers not only to an object having an external shape and size that is completely identical to or similar to the external shape and size of the operating body W, which is a real object, but also to an object having a shape that simplifies part of the external shape of the operating body W or an object having a slightly different size. Furthermore, in the simulator 1, the virtual operating body VW is arranged to correspond to the external shape and position of the operating body W in the real space RS. Furthermore, the virtual regulating unit VB1 in the simulator 1 is determined according to the movement path and range of the operating body W in order to regulate the movement of the operating body W. In this embodiment, a virtual regulating unit VB1 is provided to guide the effector W to the destination position OP1, and the virtual regulating unit VB1 is disposed in a predetermined positional relationship with respect to a virtual destination position VOP1 provided in the virtual space VS. The simulator 1 can cause the virtual effector to perform an action corresponding to a user's operation in the virtual space VS. In the example shown in FIG. 3 , a virtual space VS is displayed on the screen of the display serving as the output unit 4, in which the virtual effector VW (virtual work device VD and virtual work object VOB1) is disposed to correspond to the shape and position of the effector W (work device WD and work object OB1), and the virtual destination position VOP1 is disposed to correspond to the shape and position of the destination position OP1. Furthermore, in the virtual space VS, the virtual regulating unit VB1 is disposed in a predetermined positional relationship with respect to the virtual destination position VOP1. The form information (shape and size) of the virtual operating body VW and the virtual regulation portion VB1 described above, and their relative position information in the virtual space VS are stored in advance in a storage unit (not shown), and when the real operating body W moves, the position information of the virtual operating body VW is updated and stored. The position information resulting from the movement of the real operating body W is acquired by the measurement device WD4 at predetermined time intervals (for example, within 1 to 10 milliseconds), and each time new position information is acquired, the position information of the virtual operating body VW is updated and stored.

[0030] The simulator 1 receives an operation command when the operation command is given to the working device WD, and in parallel with the operation command, operates the virtual operating body VW in the virtual space VS with content corresponding to the operation command, and determines the positional relationship between the virtual operating body VW and the virtual regulating part VB1 at predetermined time intervals T. In this embodiment, when the working device WD is operated by the user, the simulator 1 operates the virtual operating body VW in the virtual space VS in parallel with the arbitrary operation in response to the arbitrary operation, and determines the positional relationship between the virtual operating body VW and the virtual regulating part VB1 at predetermined time intervals T.

[0031] The phrase "in parallel with the action command" means that when the simulator 1 receives an action command to operate the effector W (in this embodiment, when the action command is received from the operation device OD), the action of the effector W is performed in accordance with the action command from the operation device OD. In other words, the action of the virtual effector VW (and the determination of the positional relationship) can be performed substantially simultaneously with the action command, i.e., synchronously, in real time. Furthermore, the phrase "in parallel with an arbitrary operation" means that when an arbitrary operation to operate the effector W is performed, the action of the virtual effector VW is performed in accordance with the user's operation. In other words, the action of the virtual effector VW (and the determination of the positional relationship) can be performed substantially simultaneously with the user's operation, i.e., synchronously, in real time. In this embodiment, the action of the virtual effector VW corresponding to the user's operation is initiated in the virtual space VS immediately after the operation (for example, within 1 to 10 milliseconds). By operating the virtual effector VW in parallel with the user's operation in this way, the user's operation is quickly reflected in the action of the virtual effector VW. Therefore, the positional relationship between the virtual operating body VW and the virtual regulating section VB1 is determined instantly by the simulator 1, and based on the positional relationship determined by the simulator 1, the operation of the operating body W by the operation control section 2 can be executed with a short time lag.

[0032] Furthermore, "determining the positional relationship at a predetermined time interval T" means that the determination of the positional relationship is repeated multiple times at the predetermined time interval T. Therefore, for example, if the working device WD is operated continuously for a certain period of time (e.g., 10 seconds), the positional relationship between the virtual operating body VW and the virtual regulating unit VB1 is determined multiple times at each predetermined time interval T (e.g., within 1 to 10 milliseconds). For each time interval T, the positional relationship between the virtual operating body VW and the virtual regulating unit VB1 is determined in the virtual space VS. If there is no problem such as interference if the operating body VW continues to operate as is, the operation control unit 2 operates the operating body W in the real space RS. On the other hand, if the simulator 1 determines that the operating body W should be stopped or that the operating body W should perform a different operation, control is performed to stop the operating body W or to cause the operating body W to perform a different operation in accordance with the determination. Note that the "predetermined time interval T" is not particularly limited, but is a time that does not cause a delay or the like when a person operates the operating body W, for example, within 1 to 10 milliseconds.

[0033] The simulator 1 is configured to determine whether the distance between the virtual operating body VW and the virtual regulating portion VB1 is within a predetermined range. In this embodiment, the simulator 1 determines whether the virtual operating body VW and the virtual regulating portion VB1 are in contact with each other (whether the distance between the virtual operating body VW and the virtual regulating portion VB1 is zero) based on the distance between the virtual operating body VW and the virtual regulating portion VB1. If it is determined that the virtual operating body VW and the virtual regulating portion VB1 are in contact with each other and the movement direction D4 (see FIG. 7 ) of the virtual operating body VW corresponding to the input operation includes a component toward the surface of the virtual regulating portion VB1 (e.g., a component perpendicular to the surface of the virtual regulating portion VB1) and a component along the surface of the virtual regulating portion VB1 (e.g., a component parallel to the surface of the virtual regulating portion VB1), the simulator 1 is configured to correct the movement direction of the virtual operating body VW corresponding to the input operation based on the shape of the virtual regulating portion VB1.

[0034] The term "the distance is within a predetermined range" is not particularly limited, but may mean, for example, as shown in FIG. 6, a state in which the virtual actuator and the virtual regulating portion VB1 are in contact (i.e., the distance between the virtual actuator and the virtual regulating portion VB1 is 0), a state in which they are slightly spaced apart (the distance between the virtual actuator and the first virtual regulating portion VB1 is less than a predetermined value), or a state in which they are spaced apart by more than a predetermined distance (the distance is greater than a predetermined value).

[0035] In the example shown in FIG. 6, the surface of the virtual restricting portion VB1 is in contact with the virtual operating body VW (specifically, the virtual work object VOB1 corresponding to the work object OB1).

[0036] Furthermore, when it is determined that the distance between the virtual operating body VW and the virtual regulating portion VB1 is within a predetermined range, the simulator 1 may be configured to calculate a movement direction D5 of the virtual operating body VW for moving the virtual operating body VW along the surface of the virtual regulating portion VB1 (see FIG. 6 ) by correcting (correcting the trajectory) a movement direction D4 of the virtual operating body VW corresponding to the operation command (in this embodiment, operation information of the components of the working device WD for realizing the operation of the working device WD in response to an operation by the user) based on the shape of the virtual regulating portion VB1, as shown in FIG. 7 , and move the virtual operating body VW in the calculated movement direction D5. The operation control portion 2 may be configured to move the operating body W in the calculated movement direction D5.

[0037] Furthermore, when it is determined that the distance between the virtual operating body VW and the virtual regulating portion VB1 is within a predetermined range, the simulator 1 may be configured to stop the virtual operating body VW against the surface of the virtual regulating portion VB1. The operation controller 2 may be configured to stop the operating body W against the virtual regulating portion VB1. The meaning of "stopping the virtual operating body VW against the surface of the virtual regulating portion VB1" includes both the case where the virtual operating body VW is stopped in a state where it is in contact with the surface of the virtual regulating portion VB1, and the case where the virtual operating body VW is stopped in a state where it is spaced apart to a certain extent from the surface of the virtual regulating portion VB1.

[0038] Furthermore, when it is determined that the distance between the virtual operating body VW and the virtual regulating portion VB1 is within a predetermined range, the simulator 1 may be configured to repel the virtual operating body VW against the surface of the virtual regulating portion VB1. "Repelling the virtual operating body VW against the surface of the virtual regulating portion VB1" means bouncing the virtual operating body VW against the surface of the virtual regulating portion VB1. "Stopping the virtual operating body VW against the surface of the virtual regulating portion VB1" includes both cases where the virtual operating body VW is repelled after contacting the surface of the virtual regulating portion VB1, and cases where the virtual operating body VW is repelled at a position some distance away from the surface of the virtual regulating portion VB1. The angle of incidence and the angle of reflection when repelling the virtual operating body VW against the surface of the virtual regulating portion VB1 can be set to, for example, equal values. The angle of incidence here refers to the angle between the approach direction of the virtual operating body VW toward the surface of the virtual regulating portion VB1 and the normal to the surface of the virtual regulating portion VB1. The reflection angle here means the angle formed by the direction of rebound of the imaginary operating body VW relative to the surface of the imaginary restricting portion VB1 and the normal to the surface of the imaginary restricting portion VB1.

[0039] The virtual operating body VW (specifically, the virtual work object VOB1) is configured to perform a predetermined task by moving along the surface of the virtual regulating portion VB1 while contacting the virtual regulating portion VB1 or while maintaining a fixed distance from the virtual regulating portion VB1. For example, as shown in FIG. 7 , the simulator 1 corrects the trajectory of the moving direction D4 of the virtual operating body VW based on the shape information of the virtual regulating portion VB1 stored in a memory unit (not shown), the operation command received from the operating device OD (specifically, the moving vector F1 corresponding to the moving direction D4 of the virtual operating body VW), and the positional relationship information between the virtual operating body VW and the virtual regulating portion VB1. That is, until the virtual operating body VW contacts the virtual regulating portion VB1, the simulator 1 moves the virtual operating body VW in the moving direction D4 based on the input operation information, and when it is determined that the virtual operating body VW has contacted the virtual regulating portion VB1, it corrects the trajectory of the virtual operating body VW to follow the surface shape of the virtual regulating portion VB1.

[0040] More specifically, in the example shown in FIG. 7 , the movement vector F1 can be corrected based on a movement vector F1 corresponding to the movement direction D4 of the virtual operating body VW (the virtual work device VD and the virtual work object VOB1) and a normal vector N originating from a collision point P1 when the virtual operating body VW collides with the virtual regulating portion VB1 by moving the virtual operating body VW in the movement direction D4. This correction determines a corrected movement vector F2. More specifically, with the end point of the movement vector F1 aligned with the collision point P1, a straight line is drawn from the start point of the movement vector F1 parallel to the surface of the virtual regulating portion VB1 at the collision point P1, and an intersection P2 between this line and the normal vector N is determined. The vector extending from the start point of the movement vector F1 to the intersection P2 is the corrected movement vector F2. Furthermore, the direction in which the corrected movement vector F2 extends is the corrected movement direction D5 of the virtual operating body VW for moving the virtual operating body VW along the surface of the virtual regulating portion VB1. Furthermore, the dot product a (= F1·N) of the movement vector F1 and the normal vector N is calculated, and then the sum (F1+aN) of the movement vector F1 and the vector (aN) obtained by multiplying the normal vector N by the dot product a is calculated, thereby obtaining a corrected movement vector F2 (= F1+aN). The simulator 1 calculates the corrected movement vector F2 at predetermined time intervals. By moving the virtual operating body VW using the calculated movement vector F2, the virtual operating body VW can be moved along the surface of the virtual regulating portion VB1 while contacting the surface of the virtual regulating portion VB1. Note that the method for correcting (trajectory correction) the movement direction D4 of the virtual operating body VW based on the surface shape of the virtual regulating portion VB1 is not limited to the above description, and other methods may be used.

[0041] The simulator 1 moves the virtual operating body VW in the corrected movement direction D5 determined by the simulator 1, thereby moving the virtual operating body along the surface of the first virtual regulating portion VB1 while keeping it in contact with the surface. The operation control portion 2 moves the operating body W in the corrected movement direction D5 determined by the simulator 1, thereby moving the operating body W along the surface of the first virtual regulating portion VB1. The surface of the virtual regulating portion VB1 may be curved or flat. Whether the surface is curved or flat, as long as the movement direction D4 of the virtual operating body VW due to the user's operation includes a component toward the surface of the virtual regulating portion VB1 and a component along the surface of the virtual regulating portion VB1, the simulator 1 can determine the corrected movement direction D5 (in this embodiment, the movement vector F2) by performing trajectory correction (movement direction correction) as shown in FIG. 6. Therefore, the user can move the virtual operating body VW along the surface of the virtual regulating section VB1 while keeping it in contact with the surface of the virtual regulating section VB1 by simply performing a rough operation that includes a component directed toward the surface of the virtual regulating section VB1 and a component along the surface.

[0042] In the example shown in FIG. 6 , the virtual regulating portion VB1 is configured to guide a virtual operating body VW corresponding to the operating body W (see FIG. 5 ) to a virtual destination position VOP1 corresponding to a predetermined destination position OP1 (see FIG. 5 ). In this specification, the "predetermined destination position OP1" refers to a position according to the purpose of the work of the working device WD, and corresponds to the destination of the operating body W. In the example shown in FIG. 1 , the purpose of the work of the working device WD is to move a work target OB1 to a predetermined position OP1 (for example, an opening (see FIG. 5 ) in a floor surface, etc.) and place it at the predetermined position OP1. In the example shown in FIG. 6 , the virtual regulating portion VB1 has a virtual guide surface VB11 configured to narrow the movement range of the virtual operating body VW as it approaches the virtual destination position VOP1 corresponding to the predetermined destination position OP1. The virtual guide surface VB11 is configured so that the movement range of the virtual operating body VW narrows as the virtual operating body VW approaches the virtual destination position VOP1, thereby enabling the virtual operating body VW to be smoothly guided toward the virtual destination position VOP1. Note that, for convenience of explanation, FIG. 5 shows only the work object OB1 of the operating body W, and does not show the working device WD (finger WD31). Similarly, for convenience of explanation, FIGS. 6 and 8 show only the virtual work object VOB1 of the virtual operating body VW, and does not show the virtual operating device VD (virtual finger VD31). A similar correction may be made when the virtual operating device VD (virtual finger VD31) of the virtual operating body VW comes into contact with the virtual regulating portion VB1. Alternatively, the simulator 1 and the motion controller 2 may be configured so that the movement direction of only the virtual work object VOB1 of the virtual operating body VW is corrected by the virtual regulating portion VB1.

[0043] In this embodiment, the virtual guide surface VB11 is formed in a cone shape with an opening at the bottom BO (the end with a larger diameter in the axial direction of the cone) and the top TO (the end with a smaller diameter in the axial direction of the cone), such that the inner space narrows as the virtual guide surface VB11 approaches the virtual destination position VOP1. That is, the virtual regulating portion VB1 has a cone-shaped side wall SW with openings at the bottom BO and the top TO. The type of cone is not particularly limited, but in the example shown in FIG. 6 , it is a circular cone (frustum of a cone). The simulating portion 1 is configured to determine a movement direction D5 of the virtual work object VOB1 for moving the virtual operating body VW from the bottom BO side to the top TO side along the inner surface of the side wall SW of the virtual regulating portion VB1, and to move the virtual work object VOB1 in the determined movement direction D5 to eject the virtual work object VOB1 from the opening at the top TO. The motion control unit 2 is configured to move the work object OB1 (see FIG. 5 ) to the destination position OP1 by moving the work object OB1 in the determined direction D5. The opening of the top portion TO is set to a size that allows the portion of the actuator W to be placed at the destination position OP1 (the work object OB1 in the example shown in FIG. 1 ) to easily pass through. The "easily passable size" refers to a size that provides a predetermined radial clearance between the opening and the virtual work object VOB1 so that the user can easily pass through even with rough operations on the operation device OD. In the example shown in FIG. 6 , the diameter of the opening of the top portion TO is larger than the diameter of the columnar work object OB1. The virtual guide surface VB11 is preferably set to have an inclination angle (inclination angle with respect to the vertical direction) that smoothly guides the virtual work object VOB1 to the virtual destination position VOP1. Such an inclination angle is set according to the respective shapes and sizes of the virtual work object VOB1 and the bottom portion BO. If the inclination angle of the imaginary guide surface VB11 is too small (the diameter of the bottom BO becomes close to the diameter of the top TO), the opening of the bottom BO becomes small, which may make it difficult to insert the imaginary work object VOB1 into the opening of the bottom BO. If the inclination angle of the imaginary guide surface VB11 is too large (the diameter of the bottom BO is too large compared to the diameter of the top TO), the posture of the work object OB1 may become close to lying on its side.If the position of the work object OB1 becomes close to being sideways, it may not be possible to smoothly guide the work object OB1 to the destination position OP1. The side walls SW of the virtual restriction portion VB1 may have a substantially constant inclination angle (inclination angle with respect to the vertical direction) from the bottom BO to the top TO (see FIG. 6), or may be configured so that the inclination angle (inclination angle with respect to the vertical direction) gradually decreases (so as to become convex inward) as the position approaches the top TO from the bottom BO.

[0044] In the example shown in FIG. 6 , the simulator 1 determines a movement direction D5 of the virtual actuator VW (virtual work object VOB1) for moving the virtual actuator VW from the bottom BO side to the top TO side along the virtual guide surface VB11 of the virtual regulating portion VB1, and the motion control unit 2 is configured to move the actuator (work object OB1) in the determined movement direction D5. The actuator W moves along the inclination direction of the virtual guide surface VB11 in the real space RS. When the simulator 1 determines that the virtual actuator VW has reached or passed the top TO, it stops the movement of the virtual actuator VW. The motion control unit 2 stops the movement of the actuator W (work object OB1, see FIG. 5 ) based on the results of the simulation. After the actuator W has stopped, the user can perform an operation to release the work object OB1. When the user performs this operation, the simulator 1 releases the virtual work object VOB1 (see FIG. 6 ). This allows the virtual work object VOB1 to be placed at the virtual destination position VOP1. The simulator 1 may be configured to stop the movement of the virtual work object VOB1 and adjust the attitude of the virtual work object VOB1 when it determines that the virtual working body VW has reached or passed the top TO. Specifically, the simulator 1 is configured to cause the virtual work device VD to adjust the attitude of the virtual work object VOB1 so that the attitude of the virtual work object VOB1 becomes appropriate for placement at the virtual destination position VOP1. The "attitude appropriate for placement at the virtual destination position VOP1" is, for example, an attitude that allows the virtual work object VOB1 to stand on its own at the virtual destination position VOP1 (a position that does not tip over). Furthermore, if a hole (the cylindrical portion indicated by the dashed line in FIG. 6 ) is provided in the virtual destination position VOP1, the attitude is an attitude that allows the virtual work object VOB1 to be inserted into the hole. The virtual destination position VOP1 may be a hole (see FIG. 6) into which the virtual work object VOB1 is inserted, or may be a surface (for example, a plane) on which the virtual work object VOB1 is placed.

[0045] In the example shown in FIG. 6 , the actuation system WS is configured to use a virtual restricting unit VB1 to insert (including engage) a work object OB1 (see FIG. 5 ) into a hole formed at a predetermined position OP1. Specifically, the hole is formed on the top surface of a member FL placed in a real space RS (see FIG. 5 ). The measuring device WD4 measures the position of the hole in the member FL based on a captured image. The position measured by the measuring device WD4 is associated with the shape and position of the hole and stored in a memory unit (not shown). Note that the shape and position of the hole (target position OP1) formed in the member FL may be previously stored in association with the position of the working device WD. As shown in FIG. 6 , the simulator 1 places a virtual member VFL corresponding to the member FL in a virtual space VS. As shown in FIG. 6 , the simulator 1 provides the virtual restricting unit VB1 in the virtual space VS so that the position of the top TO coincides with the position of the opening of the hole (virtual target position VOP1). The simulator 1 determines a movement direction D5 of the virtual actuator VW (virtual work object VOB1) for moving the virtual actuator VW from the bottom BO side to the top TO side along the virtual guide surface VB11 of the virtual regulating portion VB1. The operation controller 2 moves the actuator W (work object OB1) in the determined movement direction D5. The actuator W moves along the virtual guide surface VB11. When the simulator 1 determines that the virtual actuator VW has reached the top TO, it stops the movement of the virtual actuator VW.

[0046] When the virtual operating body VW reaches the top TO, the center line CL2 of the virtual work object VOB1 may be inclined relative to the center line CL1 of the hole (virtual destination position OP1) (see FIG. 8 ). In such a case, the user can insert the tip of the work object OB1 into the hole and change the tilt angle of the work object OB1 (shake the work object OB1) by operating the operating device. When the user performs this operation, the simulator 1 inserts the tip of the virtual work object VOB1 into the hole based on the user's operation, and inserts the virtual work object VOB1 into the hole while changing the tilt angle of the virtual work object VOB1 (shake the virtual work object VOB1). By changing the tilt angle of the virtual work object VOB1, the posture of the virtual work object VOB1 can be changed to a posture that allows it to be inserted into the hole. Based on the results of the simulation, the operation control unit 2 inserts the tip of the workpiece OB1 into the hole, and inserts the workpiece OB1 into the hole while changing the tilt angle of the workpiece OB1 (while swinging the workpiece OB1). By changing the tilt angle of the workpiece OB1, the tilt angle of the workpiece OB1 (which may be zero or a value greater than zero) can be adjusted to an angle that allows the workpiece OB1 to be inserted into the hole.

[0047] After the tip of the work object OB1 is inserted into the hole, the user can perform an operation to insert the work object OB1 to a predetermined depth into the hole. Specifically, for example, the user can perform an operation to insert the work object OB1 to a predetermined depth into the hole (destination position OP1) while the center line of the work object OB1 is tilted relative to the center line of the hole (at an angle greater than zero) (see FIG. 9 ). When the user performs this operation, the simulator 1 inserts the virtual work object VOB1 to a predetermined depth into the hole while the virtual work object VOB1 is tilted relative to the hole. Alternatively, for example, the user can perform an operation to move the inserted portion of the work object OB1 toward one radial side of the hole (destination position OP1) and press the side of the inserted portion against the inner circumferential surface of the hole along the length of the hole (see FIG. 10 ). This allows the user to insert the work object OB1 to a predetermined depth into the hole without tilting the work object OB1 relative to the hole (the tilt angle is approximately zero). When the user performs this operation, the simulator 1 shifts the inserted portion of the virtual work object VOB1 toward one radial side of the hole and changes the posture of the virtual work object VOB1 so that the side of the inserted portion is pressed against the inner circumferential surface of the hole along the length of the hole. Furthermore, the simulator 1 inserts the virtual work object VOB1 to a predetermined depth into the hole without tilting the work object OB1 relative to the hole.

[0048] The simulator 1 may be configured to adjust the orientation of the virtual work object VOB1 so that the center line CL2 substantially coincides with the center line CL1. By adjusting the orientation of the virtual work object VOB1 by the simulator 1, the work object OB1 can be more easily inserted into the hole (target position OP1). Even if the center line CL2 of the virtual work object VOB1 is inclined with respect to the center line CL1 of the hole (virtual target position OP1), if the opening of the hole (virtual target position OP1) is large enough that the virtual work object VOB1 can be inserted as is, the simulator 1 does not need to adjust the orientation of the virtual work object VOB1.

[0049] The actuation system WS according to this embodiment simulates the positional relationship between the actuating body VW (the working device WD and the work object OB1), which performs a predetermined action in response to input operations from the operating device OD by the user, and a virtual restricting section VB1 virtually provided to restrict the movement of the actuating body VW, and corrects the movement direction of the actuating body VW based on the results of the simulation. As a result, even if the working device WD may perform an unexpected action in real space RS due to an arbitrary operation by the user, the actuating body W is guided to the destination position OP1, allowing the working device WD to smoothly perform its task (e.g., transporting the work object OB1). Furthermore, the actuating body W is stopped or repelled as necessary. Therefore, even if the user is unable to perform fine movements using the operating device OD, the user can easily perform the necessary operations on the work object OB1 using the working device WD. In particular, when a user remotely operates the work device WD while visually checking the operation of the work device WD by operating the work device WD, in the past it was necessary to do so while closely checking the positional relationship between the work device WD and the object OB from a distance, which could impose a heavy burden on the user. However, with the operating system WS of this embodiment, the operation of the work device WD can be appropriately controlled even with rough operations, so it is possible to protect equipment and perform advanced trajectory operations of the work device WD while reducing the burden on the user.

[0050] The operating system WS may be configured to display the positional relationship between the virtual operating body VW (in the example shown in FIG. 11 , a virtual work object VOB1) and a virtual destination position VOP1, which is a hole, on the screen of a display, which is the output unit 4, as shown in FIG. 11 . In the example shown in FIG. 11 , the positional relationship between the virtual work object VOB1 and the virtual destination position VOP1 is displayed on the display screen. The virtual work object VOB1 is shown moving from right to left in FIG. 11 . As the virtual work object VOB1 moves from right to left in FIG. 11 , it gradually approaches the virtual destination position VOP1. By displaying the positional relationship between the virtual operating body VW and the virtual destination position VOP1 on the display screen, the user can visually recognize the positional relationship on the screen. This allows the user to operate the operating device OD (see FIG. 1 ) based on the visually recognized positional relationship and cause the working device WD to perform a predetermined task. Therefore, compared to when the user operates the operating device OD while looking directly at the operating device WD and the work object OB1, the user can make the operating device WD perform the work more accurately and can operate it more easily. Furthermore, if the work object OB1 and the destination position OP1, as captured by a camera or the like, are displayed on a display, the view of the camera or the like of the work object OB1 and the destination position OP1 may be partially or entirely blocked by the operating device WD or the like. In this case, even if the work object OB1 and the destination position OP1 are displayed on the display, the user may not be able to fully grasp the positional relationship between the work object OB1 and the destination position OP1. In contrast, when the positional relationship between the virtual work object VOB1 and the virtual destination position VOP1 is displayed on the display screen, the virtual work object VOB1 and the virtual destination position VOP1 can be displayed so that the view of the virtual work object VOB1 and the virtual destination position VOP1 is not blocked by the virtual operating device VD or the like. This makes it easier for the user to grasp the positional relationship between the work object OB1 and the destination position OP1, thereby enabling the user to easily perform operations to make the work device WD perform work accurately.

[0051] 12 to 14, the operating system WS may be configured to display the positional relationship between a predetermined portion of the virtual work object VOB1 (in the examples shown in FIGS. 12 to 14, its tip (specifically, tip surface)) and the virtual destination position VOP1 on the screen of the display serving as the output unit 4. In the examples shown in FIGS. 12 to 14, the display shows the tip surface of the virtual work object VOB1 and the virtual destination position VOP1 as seen from the axial direction D6 (see FIG. 11) of the virtual work object VOB1. In the examples shown in FIGS. 12 to 14, in response to a user operation, the simulator 1 moves the virtual work object VOB1 so as to gradually bring the virtual work object VOB1 closer to the virtual destination position VOP1 and gradually decrease the tilt angle of the virtual work object VOB1 with respect to the virtual destination position VOP1. The display shows the movement of the virtual work object VOB1.

[0052] Furthermore, in this embodiment, the simulator 1 can be configured to determine whether the distance between a predetermined portion of the virtual operating body VW and the first virtual regulating portion VB1 is within a predetermined range (e.g., whether the predetermined portion is in contact with the first virtual regulating portion VB1), but not to determine whether the distance between another portion of the virtual operating body VW (a portion other than the predetermined portion) and the first virtual regulating portion VB1 is within the predetermined range. That is, the simulator 1 can limit the object for determining the distance to the first virtual regulating portion VB1 to a predetermined portion of the virtual operating body VW. In this case, since the “other portion” is not subject to distance determination, even if the “other portion” is in a positional relationship that causes it to come into contact with the first virtual regulating portion VB1, that positional relationship is not involved in the processing of the simulator 1 and the operation control unit 2. Therefore, the simulator 1 can omit determination of positional relationships that are less relevant to the task purpose and perform positional determination that is more relevant to the task purpose. This allows the simulator 1 to appropriately and quickly determine positional relationships according to the task purpose. The "predetermined portion of the virtual operating body VW" is a portion highly relevant to the task objective of the operating system WS. Specifically, for example, the "predetermined portion of the virtual operating body VW" is a part or all of the virtual work object VOB1 (i.e., the virtual operating device VD is not subject to position determination). In the examples shown in FIGS. 11 to 14 , the "predetermined portion of the virtual operating body VW" is the tip or tip surface of the virtual work object VOB1. The simulator 1 determines whether the distance between the tip or tip surface of the virtual work object VOB1 and the first virtual regulating portion VB1 is within a predetermined range (e.g., whether the tip or tip surface is in contact with the first virtual regulating portion VB1), but does not determine whether the distance between other portions of the virtual operating body VW (portions other than the predetermined portion) and the first virtual regulating portion VB1 is within the predetermined range. This allows for an appropriate and prompt simulation to be performed to achieve the task objective of placing the work object OB1 at the destination position OP1.That is, for example, when the virtual work device VD comes into contact with the virtual regulating part VB1, the movement of the virtual work device VD is restricted by the virtual regulating part VB1, and it is possible to prevent the inconvenience of the virtual regulating part VB1 hindering the guidance of the work object VOB1.

[0053] In this embodiment, as shown in FIG. 15 , the virtual regulating portion VB1 may be configured so that the virtual operating body (only the virtual work object VOB1 is shown in FIG. 15 ) can pass through the virtual guide surface VB11 from the side opposite the virtual guide surface VB11 to the virtual guide surface VB11 side, but cannot pass through the virtual guide surface VB11 from the virtual guide surface VB11 side to the side opposite the virtual guide surface VB11. Specifically, as shown in FIG. 15 , the virtual regulating portion VB1 may be configured so that the virtual operating body can pass through the virtual guide surface VB11 from the outside to the inside of the virtual regulating portion VB1, but cannot pass through the virtual guide surface VB11 from the inside to the outside of the virtual regulating portion VB1. With this configuration, the virtual work object VOB1 can enter the virtual regulating portion VB1 from a portion other than the bottom BO of the virtual regulating portion VB1. Therefore, the user can more easily place the virtual work object VOB1 into the virtual regulating section VB1, and once the virtual work object VOB1 enters the virtual regulating section VB1, it cannot leave the virtual regulating section VB1, thereby improving the operational efficiency when the user places the virtual work object VOB1 into the virtual regulating section VB1.

[0054] 16, the virtual regulating portion VB1 may be formed in a pyramidal shape with openings at the bottom BO and top TO. The type of pyramid of the virtual regulating portion VB1 may be set according to the shape of the virtual work object VOB1. Because the virtual work object VOB1 shown in FIG. 16 is formed in a quadrangular prism shape, the virtual regulating portion VB1 is formed in a quadrangular pyramid shape. The shapes of the openings of the bottom BO and the top TO may also be set according to the shape of the virtual work object VOB1. Because the virtual work object VOB1 shown in FIG. 16 is formed in a quadrangular prism shape, the shapes of the openings of the bottom BO and the top TO are also formed in a quadrangular shape. The opening of the top TO is set to a size that allows the portion of the operating body W to be placed at the destination position OP1 (the work object OB1 in the example shown in FIG. 1) to easily pass through. By making the shape of the sidewalls SW correspond to the shape of the virtual work object VOB1, it becomes easier to guide the virtual work object VOB1 in a stable position. Also, by making the shapes of the openings of the top and bottom parts TO and BO correspond to the shape of the virtual work object VOB1, it becomes easier to put the virtual work object VOB1 in and take it out of the virtual restriction part VB1.

[0055] Next, other embodiments of the actuation system will be described with reference to FIGS. 17 to 26 . In the embodiments shown in FIGS. 17 to 26 , the actuation system is configured to use a virtual regulating unit VB1 to perform an operation of inserting (including fitting) a work object into a hole provided at a predetermined destination position. The virtual regulating unit VB1 is configured to guide a virtual actuating body VW to a hole (hereinafter also referred to as hole VOP1) that is a virtual destination position VOP1. In this embodiment, the virtual regulating unit VB1 is configured to guide a virtual work object VOB1, which is an insertion target portion of the virtual actuating body VW, to the hole VOP1. In the example shown in FIG. 17 , the virtual work object VOB1 is a columnar object such as a cylindrical or prismatic object, but is not limited thereto. Also, in the example shown in FIG. 17 , the hole VOP1 is a hole having a columnar space such as a cylindrical or prismatic shape, but is not limited thereto. The hole VOP1 need only have a shape and size that allows the virtual work object VOB1 to be inserted therein. The diameter of the hole VOP1 is larger than the diameter of the workpiece OB1. As will be described later, in this embodiment, even if the difference in diameter between the hole VOP1 and the workpiece OB1 is small, the workpiece OB1 can be easily inserted into the hole VOP1.

[0056] In this embodiment, the virtual restricting portion VB1 has a guide restricting portion VBG (see FIGS. 17 and 18). Furthermore, in this embodiment, the virtual restricting portion VB1 further has a swing restricting portion VBS (see FIG. 22). The guide restricting portion VBG is a portion that guides the insertion target portion VOB1 of the virtual operating body VW (hereinafter also referred to as the insertion target portion VOB1) so that a portion of the tip of the insertion target portion VOB1 is inserted into the hole VOP1 and the center line CL2 of the insertion target portion VOB1 is inclined with respect to the depth direction of the hole VOP1 (see FIGS. 18 to 20). In this embodiment, the depth direction of the hole VOP1 is the direction along the center line CL1 of the hole VOP1. The depth direction of the hole VOP1 is approximately parallel to the center line CL1 of the hole VOP1. As shown in FIG. 20, "a portion of the tip of the insertion target section VOB1" refers to a portion of the end of the insertion target section VOB1 in the length direction (the direction along the center line CL2) that is closer to the hole VOP1. Specifically, for example, consider a cylindrical insertion target section VOB1 whose center line CL2 is inclined relative to the center line CL1 of the hole VOP1 (see FIG. 20). In this case, "a portion of the tip of the insertion target section VOB1" refers to a portion of the end of the insertion target section (cylinder) VOB1 that is closer to the hole VOP1 (a portion of the circumferential edge). The guide and restrictor VBG may have any configuration, as long as it guides the insertion target section VOB1 so that a portion of the tip of the insertion target section VOB1 of the virtual operator VW is inserted into the hole VOP1 and the center line CL2 of the insertion target section VOB1 is inclined relative to the depth direction of the hole VOP1. In the example shown in Figure 17, the guide restricting portion VBG is configured to have a generally V-shaped cross section formed by combining two flat portions. The intersection line between the two flat portions is inclined with respect to the depth direction of the hole VOP1 (the direction along the center line CL1). In this embodiment, the guide restricting portion VBG is configured so that a portion of the tip of the insertion target portion VOB1 is inserted inside the opening periphery PE of the hole VOP1 (see Figure 20). In this embodiment, the guide restricting portion VBG is configured to guide the insertion target portion VOB1 into the hole VOP1 so that the center line CL2 of the insertion target portion VOB1 and the center line CL1 of the hole VOP1 intersect or are close to each other.This makes it easier for the guide restricting portion VBG to insert a portion of the tip of the insertion target portion VOB1 inside the opening periphery PE of the hole VOP1. Also, in this embodiment, the simulator 1 is configured to generate the guide restricting portion VBG when the distance between the insertion target portion VOB1 and the hole VOP1 becomes equal to or less than a predetermined distance. Also, the simulator 1 is configured to eliminate the guide restricting portion VBG after a portion of the tip of the insertion target portion VOB1 is inserted inside the opening periphery PE of the hole VOP1.

[0057] The swing restriction portion VBS (see FIG. 22 ) is configured to restrict the swing direction D9 (see FIG. 23 ) of the insertion target portion VOB1, with a portion of its tip inserted into the hole VOP1, in contact with the opening periphery PE of the hole VOP1. Specifically, the swing restriction portion VBS is configured to restrict the swing direction D9 so that the inclination angle θ of the center line CL2 of the insertion target portion VOB1 relative to the depth direction of the hole VOP1 is small. In the example shown in FIGS. 21 and 23 , the swing restriction portion VBS restricts the swing direction D9 so that the inclination angle θ shown in FIG. 23 is smaller than the inclination angle θ shown in FIG. 21 . In the state shown in FIG. 21 , the side surface (outer peripheral surface) of the insertion target portion VOB1 is in contact with the opening periphery PE at one location (one contact position CP). In this state, the insertion target portion VOB1 can swing in a direction that reduces the inclination angle θ. The insertion target portion VOB1 swings in the swing direction D9, resulting in a transition to the state shown in FIG. 23. The swing regulation portion VBS is not particularly limited in configuration as long as it can regulate the swing direction D9 of the insertion target portion VOB1, centered on the contact position with the opening periphery PE, when the insertion target portion VOB1, with a portion of its tip inserted into the hole VOP1, is in contact with the opening periphery PE of the hole VOP1. In the example shown in FIG. 22, the swing regulation portion VBS is composed of two flat plate-shaped portions arranged to sandwich the insertion target portion VOB1 from both radial sides. The two flat plate-shaped portions are arranged parallel to each other. The distance between the two flat plate-shaped portions is approximately the same as the diameter of the insertion target portion VOB1. The two flat portions are arranged parallel to the center line CL1 of the hole VOP1 and the center line CL2 of the insertion target portion VOB1, so that the swinging restriction portion VBS can guide the insertion target portion VOB1 so that the center line CL2 of the insertion target portion VOB1 swings in a direction in which the center line CL2 of the insertion target portion VOB1 becomes substantially parallel to the center line CL1 of the hole VOP1 (a direction in which the tilt angle θ, described below, becomes smaller).

[0058] In the state shown in Figure 23, the insertion target portion VOB1 is in contact with the opening periphery PE at three locations (three contact positions CP). In the example shown in Figure 23, the side surface of the insertion target portion VOB1 is in contact with the opening periphery PE at one contact position CP (hereinafter also referred to as the first contact position CP1). In addition, in this embodiment, the periphery of one end of the insertion target portion VOB1 (the end on the hole portion VOP1 side) is in contact with the opening periphery PE at two contact positions CP (hereinafter also referred to as the second contact positions CP2). The two contact positions CP (two second contact positions CP2) are spaced apart from each other in the circumferential direction on the periphery of one end of the insertion target portion VOB1. Figure 23 shows the second contact position CP2, of the two second contact positions CP2, which is the second contact position CP2 on the near side in the direction perpendicular to the paper surface. When the insertion target portion VOB1 and the opening periphery PE are in contact at three points (see FIG. 23), the insertion target portion VOB1 cannot swing in a direction that reduces the inclination angle θ. Furthermore, in this state, the insertion target portion VOB1 cannot be inserted deeper into the hole VOP1. To insert the insertion target portion VOB1 deeper into the hole VOP1, the inclination angle θ of the insertion target portion VOB1 must be reduced. To reduce the inclination angle θ of the insertion target portion VOB1, the insertion target portion VOB1 is moved in the direction D10 (see FIG. 24) that pulls it out of the hole VOP1 (but does not pull it out completely). When the insertion target portion VOB1 is moved in the direction D10 that pulls it out of the hole VOP1, the insertion target portion VOB1 transitions to a state in which it contacts the opening periphery PE at one point (one contact position CP) (see FIG. 24).

[0059] In the state shown in Fig. 24, the insertion target part VOB1 can be swung in a direction in which the tilt angle θ becomes smaller (see Fig. 25). By swung the insertion target part VOB1 in the swing direction D9, it transitions to the state shown in Fig. 25. The tilt angle θ shown in Fig. 25 is smaller than the tilt angle θ shown in Fig. 23. This means that the posture of the insertion target part VOB1 shown in Fig. 25 is closer to the posture in which it can be inserted into the hole VOP1 than the posture of the insertion target part VOB1 shown in Fig. 23.

[0060] In detail, when the inclination angle θ exceeds a certain value, it becomes difficult to insert the entire tip of the insertion target portion VOB1 (for example, the entire circumferential direction of the circular edge) into the hole VOP1 while the side surface (outer peripheral surface) of the insertion target portion VOB1 is in contact with the opening periphery PE. The inclination angle at this point can be called the critical angle θa. The critical angle θa can be expressed by the following equation (1): where R is the radius of the hole VOP1, and r is the radius of the insertion target portion VOB1.

[0061] cos(θa)=r / R...(1)

[0062] When the insertion target part VOB1 is in a position where it can be inserted into the hole part VOP1 (hereinafter also referred to as the insertable position), the following formula (2) holds: The insertable position is a position where the entire tip of the insertion target part VOB1 can be inserted into the hole part VOP1 with the outer peripheral surface of the insertion target part VOB1 in contact with the opening periphery PE (see FIG. 26).

[0063] θ<θa (2)

[0064] The tilt angle θ shown in FIG. 25 is greater than the critical angle θa. However, the tilt angle θ shown in FIG. 25 is closer to the critical angle θa than the tilt angle θ shown in FIG. 23. In the state shown in FIG. 25, the insertion target portion VOB1 contacts the opening periphery PE at three locations (three contact positions CP). In FIG. 25, two second contact positions CP2 (only one second contact position CP2 is shown in FIG. 25) are spaced apart from each other in the circumferential direction on the periphery of one end of the insertion target portion VOB1. However, the distance between the first contact position CP1 and the second contact position CP2 in FIG. 25 is smaller than the distance between the first contact position CP1 and the second contact position CP2 in FIG. 23. In the state shown in FIG. 25 where the insertion target portion VOB1 and the opening periphery PE contact each other at three locations, the insertion target portion VOB1 cannot swing in a direction that reduces the tilt angle θ. Therefore, the insertion target portion VOB1 is moved again in the direction D10 to be pulled out from the hole portion VOP1.

[0065] 23 to 25 are repeated. At a certain point, the entire tip of the insertion target portion VOB1 (e.g., the entire circumferential length of the circular edge) is inserted into the hole VOP1 (see FIG. 26). In the state shown in FIG. 26, the inclination angle θ satisfies the above-described formula (2). In the state shown in FIG. 26, the side of the insertion target portion VOB1 contacts the opening periphery PE at one point (one first contact position CP1), and a portion of the tip of the insertion target portion VOB1 contacts the inner circumferential surface of the hole VOP1 at a third contact position CP3. This allows the insertion target portion VOB1 to be inserted even deeper by adjusting the orientation of the insertion target portion VOB1 within the hole VOP1. For example, the insertion target portion VOB1 is swung around the contact position (third contact position CP3 in FIG. 26) between the tip of the insertion target portion VOB1 and the inner circumferential surface of the hole VOP1 so that the center line CL2 becomes approximately parallel to the center line CL1 of the hole VOP1. This makes the center line CL2 approximately parallel to the center line CL1 of the hole VOP1, allowing the insertion target portion VOB1 to be inserted even deeper (or even completely inserted).

[0066] A user can operate the operating device to move the tip of the insertion target portion of the actuator closer to the hole in order to insert the insertion target portion into the hole. When the user performs this operation, the simulator 1 generates a guide restricting portion VBG based on the user's operation when the distance between the insertion target portion VOB1 and the hole VOP1 in the virtual actuator VW becomes equal to or less than a predetermined distance, as shown in FIG. 17 . The simulator 1 calculates a movement direction D5 (see FIG. 18 ) of the insertion target portion VOB1 for moving the insertion target portion VOB1 along the virtual guide surface VB11 of the guide restricting portion VBG in a direction approaching the hole VOP1. The operation controller 2 moves the insertion target portion VOB1 in the calculated movement direction D5 (see FIG. 18 ). Based on the results of the simulation, the operation controller 2 moves the insertion target portion of the actuator along the inclination direction of the virtual guide surface VB11. When the simulator 1 determines that a portion of the tip of the insertion target portion VOB1 has been inserted into the hole portion VOP1, it stops the movement of the virtual actuator VW (see FIG. 20). The operation controller 2 stops the movement of the actuator based on the results of the simulation. After the actuator stops, the user can perform an operation to move the insertion target portion on the inclined side of the insertion target portion in a direction approaching the opening periphery PE (movement direction D11, see FIG. 21). When the user performs this operation, the simulator 1 moves the insertion target portion VOB1 in the movement direction D11 based on the operation. This allows the side surface (outer peripheral surface) of the insertion target portion VOB1 to contact the opening periphery PE (see FIG. 21). The simulator 1 can determine that the side surface (outer peripheral surface) of the insertion target portion VOB1 has contacted the opening periphery PE based on the distance between the insertion target portion VOB1 and the opening periphery PE. 21, the first contact position CP1, the center line CL1 of the hole VOP1, and the center line CL2 of the insertion target portion VOB1 are preferably on substantially the same plane. In this case, the insertion target portion VOB1 can be easily swung in a direction in which the center line CL2 of the insertion target portion VOB1 is substantially parallel to the center line CL1 of the hole VOP1 (a direction in which the tilt angle θ, described below, decreases) (see swung direction D9 in FIG. 23).In this embodiment, the actuation system WS may include a contact detection unit (not shown) that detects contact between the insertion target portion and the periphery of the opening of the hole in real space. The configuration of the contact detection unit is not particularly limited as long as it can detect contact between the insertion target portion and the periphery of the opening of the hole. Specifically, the contact detection unit may be, for example, a force sensor or a device that detects electrical conduction between the insertion target portion and the periphery of the opening of the hole when the insertion target portion comes into contact with the periphery of the opening. The device that detects electrical conduction may be configured to detect, for example, a change in the potential difference between the insertion target portion and the periphery of the opening.

[0067] The simulator 1 stops the movement of the virtual operating body VW (insertion target body VOB1) when it determines that the insertion target body VOB1 in the virtual space VS has come into contact with the opening periphery PE (see FIG. 21), or when the contact detector detects contact between the insertion target body and the opening periphery in real space. The operation controller 2 stops the movement of the operating body (insertion target body) based on the results of the simulation.

[0068] After the movement of the actuator stops, the simulator 1 generates a swing restriction member VBS (see FIG. 22). After the swing restriction member VBS is generated, the user can perform an operation to swing the insertion target member toward the central axis of the hole. When the user performs this operation, the simulator 1 determines the movement direction (swing direction D9, see FIG. 23) of the insertion target member VOB1 for swinging the insertion target member VOB1 toward the center line CL1 of the hole VOP1 along the virtual guide surface VB11 of the swing restriction member VBS. The operation controller 2 moves (swings) the insertion target member VOB1 in the determined movement direction (swing direction D9) (see FIG. 23). Based on the results of the simulation, the operation controller 2 moves (swings) the insertion target member of the actuator toward the central axis of the hole along the virtual guide surface VB11.

[0069] When the simulator 1 determines that the tip of the insertion target portion VOB1 has come into contact with the opening periphery PE of the hole VOP1 based on the positional relationship between the tip of the insertion target portion VOB1 and the opening periphery PE of the hole VOP1, it determines that the insertion target portion VOB1 has come into contact with the opening periphery PE of the hole VOP1 at three points (see FIG. 23). When the simulator 1 determines that the insertion target portion VOB1 has come into contact with the opening periphery PE of the hole VOP1 at three points, it stops the movement (swing) of the virtual actuator VW (insertion target portion VOB1). The operation controller 2 stops the movement (swing) of the actuator (insertion target portion) based on the results of the simulation.

[0070] After the movement of the actuator stops, the user can move the insertion target portion in a direction to pull it out of the hole. When the user performs this operation, the simulator 1 moves the insertion target portion VOB1 in a direction D10 to pull it out of the hole VOP1 based on the operation (see FIG. 24). The operation controller 2 moves the insertion target portion in the direction D10 to pull it out of the hole based on the results of the simulation. As a result, the insertion target portion VOB1 transitions to a state in which it is in contact with the opening periphery PE of the hole VOP1 at one point (first contact position CP). In the state shown in FIG. 24, the insertion depth of the insertion target portion VOB1 into the hole VOP1 is shallower than the state shown in FIG. 23 (the state before movement in the pull-out direction D10). The simulator 1 stops the movement of the insertion target portion VOB1 in the pull-out direction D10 when the insertion depth of the insertion target portion VOB1 becomes shallow to a certain extent and the insertion target portion VOB1 transitions to a state in which it is in contact with the opening periphery PE at one point. The operation controller 2 stops the movement of the insertion target portion based on the results of the simulation.

[0071] After the movement of the insertion target portion has stopped, the user can again perform an operation to swing the insertion target portion toward the central axis of the hole. When the user performs this operation, the simulator 1 determines the movement direction (swing direction D9, see FIG. 25 ) of the insertion target portion VOB1 for swinging the insertion target portion VOB1 toward the center line CL1 of the hole VOP1 along the virtual guide surface VB11 of the swing regulation portion VBS. The simulator 1 moves (swings) the insertion target portion VOB1 in the determined movement direction (swing direction D9) (see FIG. 25 ). Based on the results of the simulation, the operation controller 2 moves (swings) the insertion target portion of the actuator along the virtual guide surface VB11 toward the central axis of the hole.

[0072] Thereafter, the user can repeatedly move the insertion target portion in the pull-out direction D11 and swing the insertion target portion in the swing direction D9. When the user performs these operations, the simulator 1 repeats the same procedures as those shown in FIGS. 23 to 25 based on the operations. At a certain point, the entire tip of the insertion target portion VOB1 (e.g., the entire circumferential length of the circular edge) is inserted into the hole VOP1 (see FIG. 26). In the state shown in FIG. 26, the side of the insertion target portion VOB1 contacts the opening periphery PE at one point (one first contact position CP1), and a portion of the tip of the insertion target portion VOB1 contacts the inner circumferential surface of the hole VOP1 at a third contact position CP3. In this state, the user can perform operations to adjust the orientation of the insertion target portion within the hole. For example, the user may perform an operation to swing the insertion target portion around the contact position between the tip of the insertion target portion and the inner circumferential surface of the hole so that the center line of the insertion target portion is approximately parallel to the center line of the hole. Based on this operation, the simulator 1 swings the insertion target portion VOB1 around the contact position between the tip of the insertion target portion VOB1 and the inner circumferential surface of the hole VOP1 (contact position CP on the left side in FIG. 26 ) so that the center line CL2 is approximately parallel to the center line CL1 of the hole VOP1. This makes the center line CL2 approximately parallel to the center line CL1 of the hole VOP1, allowing the insertion target portion VOB1 to be inserted even deeper (even completely). Based on the result of the judgment by the simulator, the operation controller 2 swings the insertion target portion so that the center line of the insertion target portion is approximately parallel to the center line of the hole.

[0073] In this embodiment, as described above, a portion of the tip of the insertion target portion VOB1 is inserted into the hole VOP1, and this insertion serves as an opportunity to insert the insertion target portion VOB1 deeper, so the user can easily insert the insertion target portion into the hole even if the difference in diameter between the hole and the insertion target portion is slight. Moreover, by using the above-mentioned guide and restricting portion VBG and swing restricting portion VBS, the insertion target portion is guided into the hole so that it can be easily inserted, making the insertion operation even easier.

[0074] In this embodiment, the simulator is configured to generate a virtual regulator VB1 when the distance between the virtual operator VOB1 corresponding to the operator and the destination position VOP1 falls below a predetermined distance (see FIGS. 17 and 22). The simulator is also configured to eliminate the virtual regulator VB1 when the distance between the virtual operator VOB1 corresponding to the operator and the destination position VOP1 exceeds a predetermined distance (see FIG. 19). By generating or eliminating the virtual regulator VB1 as needed in accordance with the distance between the virtual operator VOB1 and the destination position VOP1, the virtual regulator VB1 does not exist when it is not needed, preventing the virtual regulator VB1 from interfering with the operation of the operator. In other embodiments, the simulator can also generate or eliminate a virtual regulator as needed in accordance with the distance between the virtual operator VOB1 corresponding to the operator and the destination position VOP1. The attitude and trajectory of the effector determined by the operation control unit 2 may be stored in a memory unit, and the working device may be caused to operate in semi-automatic mode or automatic mode based on the stored attitude and trajectory of the effector.

[0075] Next, another embodiment of the actuation system will be described with reference to FIGS. 27 to 30 . In the embodiment shown in FIGS. 27 to 30 , the actuating body W (see FIG. 27 ) is a manipulator WD that applies a predetermined operation to a predetermined work object OB1. In the above-described embodiments (see FIGS. 1 to 26 ), the actuating body W was described as an integrated part of the manipulator WD and the work object OB1. However, in this embodiment, the actuating body W is configured to position the manipulator WD before grasping the work object OB1, and the actuating body W is composed of only the manipulator WD. In this embodiment, the manipulator WD is configured to grasp the work object OB1, as shown in FIG. 30 . In this embodiment, as shown in FIGS. 28 and 29 , a virtual manipulator VD corresponding to the manipulator WD and a virtual work object VOB1 corresponding to the work object OB1 are shown in the virtual space VS. The virtual operating body VW corresponding to the operating body W has a main body portion (virtual operating device VD in the example shown in FIG. 28) corresponding to the operating body W (see the operating device WD shown in FIG. 1) and a second virtual regulating portion VB2 provided on at least a part of the main body portion and moving relatively to the first virtual regulating portion VB1. In the example shown in FIG. 28, the main body portion (virtual operating device VD) has an outer shape and size corresponding to the outer shape and size of the operating body W. In the example shown in FIG. 28, the second virtual regulating portion VB2 is provided with a virtual tool portion VD3 corresponding to the tool portion WD3 of the operating device WD (see FIG. 1). Specifically, the second virtual regulating portion VB2 is provided in a portion (virtual finger VD31) corresponding to the finger WD31 of the tool portion WD3. The first virtual regulating portion VB1 is configured to guide the operating body W to a virtual destination position VOP2 corresponding to a predetermined destination position OP2. Specifically, the first virtual regulating portion VB1 is configured to guide the second virtual regulating portion VB2 to a virtual destination position VOP2. The first virtual regulating portion VB1 has a virtual guide surface VB11 configured so that the movement range of the virtual operating body VW becomes narrower as the first virtual regulating portion VB1 approaches the virtual destination position VOP2. In this embodiment, the destination position OP2 is a position where the operating body W should be located in order to apply a predetermined operation to the work object OB1, and can be set depending on the size, shape, position, etc. of the work object OB1.For example, the position, size, angle, attitude, etc. of the work object OB1 may be measured by the above-mentioned measuring device WD4, and a destination position OP2 may be set based on the position information and shape information of the work object OB1. The position of the virtual regulating portion VB1 is set based on a virtual destination position VOP2 that is set in advance or determined by judgment by the simulating portion 1. Note that, as long as the first virtual regulating portion VB1 and the second virtual regulating portion VB2 move relative to each other, only one of the first virtual regulating portion VB1 and the second virtual regulating portion VB2 may move (the other may not move), or both the first virtual regulating portion VB1 and the second virtual regulating portion VB2 may move.

[0076] In this embodiment, the first imaginary regulating portion VB1 has a shape similar to that of the first imaginary regulating portion VB1 shown in FIG. 6 . However, the shape of the first imaginary regulating portion VB1 is not particularly limited and may be different from the shape shown in FIG. 6 . The first imaginary regulating portion VB1 is determined based on the position of the workpiece OB1. Specifically, the first imaginary regulating portion VB1 is provided at a position facing the tool portion WD3 when the tool portion WD3 grips the workpiece OB1. The number of first imaginary regulating portions VB1 is not particularly limited as long as the tool portion WD3 is provided at a position facing the tool portion WD3 when the tool portion WD3 grips the workpiece OB1. In the example shown in FIG. 28 , the number of first imaginary regulating portions VB1 is two. However, the number of first imaginary regulating portions VB1 is not limited to two and may be one or more (e.g., three or more), and can be changed appropriately depending on the operation applied to the workpiece OB1 by the work device WD. Furthermore, the position of the first virtual regulating portion VB1 can be changed as appropriate depending on the operation applied to the work object OB1 by the working device WD. For example, in this embodiment, since the work object OB1 is gripped by the working device WD, as shown in FIG. 28 , the first virtual regulating portion VB1 is provided at positions on both sides of the virtual work object VOB1 in a direction perpendicular to the longitudinal direction of the virtual work object VOB1. In this embodiment, the number of first virtual regulating portions VB1 and the number of second virtual regulating portions VB2 are set to be the same. In the example shown in FIG. 28 , the number of first virtual regulating portions VB1 and the number of second virtual regulating portions VB2 are each two. In the example shown in FIG. 28 , the second virtual regulating portion VB2 is provided at a position corresponding to the virtual finger VD31. By providing the second imaginary regulating portion VB2 at a position corresponding to the imaginary finger VD31, the position of the finger WD31 can be reliably determined when gripping the workpiece OB1. The size of the first imaginary regulating portion VB1 may be substantially the same as that of the second imaginary regulating portion VB2, or may be larger than that of the second imaginary regulating portion VB2. For example, the first imaginary regulating portion VB1 may have side walls with substantially the same slope and a larger bottom portion BO than the second imaginary regulating portion VB2.By having a larger bottom portion BO, the first imaginary restriction portion VB1 has a larger area that can accommodate the second imaginary restriction portion VB2, making it easier for the user to insert the second imaginary restriction portion VB2 into the first imaginary restriction portion VB1.

[0077] The shape of the second imaginary restriction portion VB2 is not particularly limited as long as it can be guided to the destination position OP2 by the first imaginary restriction portion VB1. In this embodiment, the second imaginary restriction portion VB2 has substantially the same shape and size as the first imaginary restriction portion VB1. Because the first imaginary restriction portion VB1 and the second imaginary restriction portion VB2 have substantially the same shape and size, the first imaginary restriction portion VB1 can smoothly guide the second imaginary restriction portion VB2 to the destination position OP2.

[0078] The simulator 1 is configured to determine the positional relationship between the virtual operating body VW and the first virtual regulating portion VB1 by determining the positional relationship between the first virtual regulating portion VB1 and the second virtual regulating portion VB2. Specifically, the simulator 1 determines the positional relationship between the virtual operating body VW (specifically, the virtual tool portion VD3 of the virtual work device VD) and the first virtual regulating portion VB1 by determining the positional relationship between the inner surface of the first virtual regulating portion VB1 and the outer surface of the second virtual regulating portion VB2. In response to an operation command from the operating device OD, the simulator 1 moves the second virtual regulating portion VB2 to a position where the second virtual regulating portion VB2 and the first virtual regulating portion VB1 coincide with each other while bringing the outer surface of the side wall of the second virtual regulating portion VB2 into contact with the inner surface of the side wall of the first virtual regulating portion VB1 (the virtual guide surface VB11) (see FIG. 29 ). The second virtual regulating portion VB2 is guided along the virtual guide surface VB11, thereby guiding the operating body W to the destination position OP2. By guiding the second virtual regulating portion VB2 along the virtual guide surface VB11, the operating body W can be easily and reliably moved to the destination position OP2 even if the user performs a rough operation. Once the operating body W is guided to the destination position OP2, the simulator 1 causes the virtual tool portion VD31 of the virtual working device VD to grasp the virtual work object VOB1. Specifically, the simulator 1 causes the virtual finger VD31 corresponding to the finger WD31 to grasp the virtual work object VOB1. The simulator 1 then lifts the virtual work object VOB1 (see FIG. 30 ) and transports the virtual work object VOB1 to the destination location.

[0079] Here, when the number of first virtual regulating portions VB1 and the number of second virtual regulating portions VB2 are each plural (two in the example shown in Figures 28 and 29), assume a state before the first virtual regulating portions VB1 and the second virtual regulating portions VB2 are aligned. In this state, there may be cases where the direction D7 connecting one first virtual regulating portion VB1 (e.g., the first virtual regulating portion VB1 on the left in Figure 31) of the plural (e.g., two) first virtual regulating portions VB1 and another first virtual regulating portion VB1 (e.g., the first virtual regulating portion VB1 on the right in Figure 31) does not coincide with the direction D8 connecting one second virtual regulating portion VB2 (e.g., the second virtual regulating portion VB2 on the left in Figure 31) and another second virtual regulating portion VB2 (e.g., the second virtual regulating portion VB2 on the right in Figure 31) of the plural (e.g., two) second virtual regulating portions VB2 (see Figure 31). In the example shown in Figure 31, before the first virtual regulating portion VB1 and the second virtual regulating portion VB2 are aligned, the positions of one first virtual regulating portion VB1 (hereinafter also referred to as one first virtual regulating portion VB1) and one second virtual regulating portion VB2 (hereinafter also referred to as one second virtual regulating portion VB2) are approximately aligned when viewed from the vertical direction (direction D3). In contrast, the positions of the other first virtual regulating portion VB1 (hereinafter also referred to as the other first virtual regulating portion VB1) and the other second virtual regulating portion VB2 (hereinafter also referred to as the other second virtual regulating portion VB2) are offset from each other when viewed from the vertical direction (direction D3) (they are offset in directions D1 and D2). Specifically, for example, the centers of the other first virtual regulating portion VB1 and the other second virtual regulating portion VB2 are offset from each other when viewed from direction D3 (only a portion of each of them overlap). In this embodiment, the simulator 1 is configured so that, during the process of inserting one second virtual regulating portion VB2 and the other second virtual regulating portion VB2 into one first virtual regulating portion VB1 and the other first virtual regulating portion VB1, respectively, a rotational moment that rotates the other second virtual regulating portion VB2 around one second virtual regulating portion VB2 acts on the other second virtual regulating portion VB2.When the other second imaginary regulating portion VB2 moves along the inclined direction of the virtual guide surface VB11 of the other first imaginary regulating portion VB1, a rotational moment is generated on the other second imaginary regulating portion VB2 due to the force it receives from the other first imaginary regulating portion VB1. This rotational moment causes the other second imaginary regulating portion VB2 to rotate around the one second imaginary regulating portion VB2 (see the arrow in FIG. 31 ). This allows the other second imaginary regulating portion VB2 to rotate in a direction that matches the other first imaginary regulating portion VB1. This rotation allows the simulator 1 to match the one second imaginary regulating portion VB2 and the other second second imaginary regulating portion VB2 with the one first imaginary regulating portion VB1 and the other first first imaginary regulating portion VB1, respectively. The operation controller 2 rotates the operating body W (work object OB1) according to the results of the simulation, and moves the operating body W to the target position OP2.

[0080] Furthermore, in this embodiment, the position of the first virtual regulating portion VB1 in the virtual space VS is determined by the simulator 1 according to the shape and position of the work object OB1. As a result, the first virtual regulating portion VB1 is generated at an appropriate position that makes it easy to operate the work object OB1 according to the shape and position of the work object OB1. Therefore, by guiding the second virtual regulating portion VB2 by the first virtual regulating portion VB1, the working device WD can operate the work object OB1 at an appropriate position in the real space RS based on the shape and position of the work object OB1. More specifically, in this embodiment, the first virtual regulating portion VB1 is generated for the rod-shaped work object OB1 at the center of the work object OB1 in the longitudinal direction and at a position symmetrical with respect to the axis of the work object OB1. As a result, the rod-shaped work object OB1 can be grasped in a well-balanced position, enabling easy operation of the work object OB1.

[0081] In another embodiment of the actuation system shown in FIG. 32 , a working device is configured to be guided along a predetermined movement path. In this embodiment, the operating body is a working device that applies a predetermined operation to a predetermined work object, and the work object is grasped by the working device. In this embodiment, the working device is configured to transport the work object (e.g., a container such as a cup filled with liquid) along a predetermined movement path. In the example shown in FIG. 32 , a virtual working device VD corresponding to the working device is grasping a virtual work object VOB1 corresponding to the work object. A virtual operating body VW corresponding to the operating body has a main body portion (the virtual working device VD and the virtual work object VOB1 in the example shown in FIG. 32 ) corresponding to the external shape and size of the operating body, and a second virtual regulating portion VB2 provided on at least a portion of the main body portion. In the example shown in FIG. 32 , the second virtual regulating portion VB2 is provided on a virtual tool portion VD3 corresponding to the tool portion WD3 (see FIG. 1 ) of the working device WD. The first virtual regulating portion VB1 is configured to guide the operating body to a predetermined destination position. Specifically, the first imaginary regulating portion VB1 is configured to guide the second imaginary regulating portion VB2 to a predetermined destination position. The first imaginary regulating portion VB1 has an imaginary guide surface VB11 along a predetermined movement path to the destination position. The shape of the first imaginary regulating portion VB1 is not particularly limited and may be any shape suited to the purpose of the work tool. In the example shown in FIG. 32 , the first imaginary regulating portion VB1 has a shape formed by combining three flat portions VB1a, VB1b, and VB1c into a substantially U-shape.

[0082] The shape of the second virtual regulating portion VB2 is not particularly limited as long as it can be guided to the target position by the first virtual regulating portion VB1. In the example shown in FIG. 32 , the second virtual regulating portion VB2 is configured to ensure a predetermined distance between the virtual working device VD and the first virtual regulating portion VB1. Specifically, the second virtual regulating portion VB2 includes three rod-shaped portions VB21 extending in different directions. The three rod-shaped portions VB21 each have a length corresponding to the distance to be ensured between the three planar portions VB1a, VB1b, and VB1c and the virtual working device VD. The lengths of the three rod-shaped portions VB21 may be the same or different. The simulating unit 1 is configured to determine the positional relationship between the first virtual regulating portion VB1 and the second virtual regulating portion VB2, thereby determining the positional relationship between the virtual operating body VW and the first virtual regulating portion VB1. Specifically, the simulator 1 determines the positional relationship between the virtual operating body VW (specifically, the virtual tool member VD3 of the virtual working device VD) and the first virtual operating body VB1 by determining the positional relationship between the tip of the first virtual operating body VB1 and the tip of the second virtual operating body VB2. Upon receiving an operation command from the operating device OD, the simulator 1 moves the second virtual operating body VB2 and the virtual tool member VD3 to a target position while bringing the tip of the second virtual operating body VB2 into contact with the inner surface (virtual guide surface VB11) of the first virtual operating body VB1 (see FIG. 32). The second virtual operating body VB2 is guided by the virtual guide surface VB11, thereby guiding the operating body along a predetermined movement path to the target position. This allows a work object (e.g., a container such as a cup filled with liquid) to be transported along the predetermined movement path to the target position.

[0083] In the modified example shown in FIG. 33 , the first imaginary regulating portion VB1 is linear, and the second imaginary regulating portion VB2 is ring-shaped. Furthermore, the second imaginary regulating portion VB2 is configured to be movable along the first imaginary regulating portion VB1 with the first imaginary regulating portion VB1 inserted inside. The shape of the first imaginary regulating portion VB1 is not particularly limited, and may be, for example, linear or curved (see FIG. 33 ). When the first imaginary regulating portion VB1 is linear, the second imaginary regulating portion VB2 can be moved to the target position via the shortest route. When the first imaginary regulating portion VB1 is curved, the second imaginary regulating portion VB2 can be moved to the target position via a route that avoids obstacles, for example.

[0084] In another embodiment shown in FIG. 33 , the actuation system has a first virtual regulating portion VB1 configured linearly and a second virtual regulating portion VB2 configured ring-shaped in a virtual space VS. The second virtual regulating portion VB2 is configured to be movable along the first virtual regulating portion VB1 with the first virtual regulating portion VB1 inserted therethrough, thereby guiding the working device WD along a predetermined path along the extension direction of the linear first virtual regulating portion VB1. In this embodiment, the second virtual regulating portion VB2 is provided on a virtual tool portion VD3 corresponding to the tool portion WD3 (see FIG. 1 ) of the working device WD. This configuration allows the virtual tool portion VD3 to move along a linear movement path. In the example shown in FIG. 33 , the second virtual regulating portion VB2 has a ring-shaped portion VB22 and a connecting portion VB23. The ring-shaped portion VB22 is provided on the virtual tool portion VD3 via the connecting portion VB23. With this configuration, the virtual tool part VD3 can be moved while maintaining a constant distance between the virtual tool part VD3 and the first virtual regulating part VB1. The first virtual regulating part VB1 may be linear, curved, or bent. When the first virtual regulating part VB1 has a curved or bent shape, the virtual tool part VD3 can be moved along a complex movement path.

[0085] In the example shown in FIG. 33 , the cross-sectional shape of the first imaginary regulating portion VB1 is a circular shape that allows the first imaginary regulating portion VB1 to rotate about the axis relative to the second imaginary regulating portion VB2, but this is not limited to this. In the modified example shown in FIG. 34 , the cross-sectional shape of the first imaginary regulating portion VB1 has a shape that prevents the first imaginary regulating portion VB1 from rotating about the axis relative to the second imaginary regulating portion VB2. Specifically, the cross-sectional shape of the first imaginary regulating portion VB1 is angular (square in the example shown in FIG. 34 ). Furthermore, the cross-sectional shape of the second imaginary regulating portion VB2 is formed as a rectangular tube (square tube in the example shown in FIG. 34 ) that follows the outer periphery of the first imaginary regulating portion VB1. By configuring the first imaginary regulating portion VB1 so that it cannot rotate about the axis relative to the second imaginary regulating portion VB2, the posture of the imaginary operating body VW is stabilized. Therefore, for example, when transporting a container such as a cup filled with liquid, the container can be transported in a stable position so that the liquid does not spill out of the container during transport.

[0086] Furthermore, the second imaginary regulating portion VB2 may be in point contact with the first imaginary regulating portion VB1 (see FIG. 33 ), but is not limited thereto. The second imaginary regulating portion VB2 may be in line contact or surface contact with the first imaginary regulating portion VB1 (see FIG. 34 ), thereby reducing the degree of freedom of its position relative to the first imaginary regulating portion VB1. Reducing the degree of freedom of the position of the second imaginary regulating portion VB2 relative to the first imaginary regulating portion VB1 stabilizes the position of the second imaginary regulating portion VB2. Therefore, the position of the imaginary operating body can be stabilized. Specifically, in the example shown in FIG. 33 , the linear first imaginary regulating portion VB1 and the ring-shaped portion VB22 of the second imaginary regulating portion VB2 are in point contact. Therefore, the ring-shaped portion VB22 can move along the length of the first imaginary regulating portion VB1 and can rotate around the first imaginary regulating portion VB1. Therefore, in the example shown in FIG. 33, the second imaginary regulating portion VB2 has a high degree of freedom in its orientation. In contrast, in the example shown in FIG. 34, the linear first imaginary regulating portion VB1 having a rectangular cross section and the rectangular cylindrical second imaginary regulating portion VB2 are in surface contact on four surfaces. Therefore, the second imaginary regulating portion VB2 can move along the length of the first imaginary regulating portion VB1 but cannot rotate around the first imaginary regulating portion VB1. Therefore, in the example shown in FIG. 34, the second imaginary regulating portion VB2 has a low degree of freedom in its orientation.

[0087] In another embodiment shown in FIG. 35 , the virtual regulating portion VB1 is formed in a plate shape with a predetermined thickness. The actuation system of this embodiment can be used to draw letters or pictures on a sheet-like work object using an actuating body (working device WD, for example, a writing implement such as a brush). The shape of the actuating body in this embodiment is not particularly limited as long as it can draw letters or pictures on the work object. In this embodiment, the actuating body is configured in a rod shape. The virtual regulating portion VB1 is configured so that a tool portion VD3 (hereinafter also referred to as a rod-shaped portion VD3), which is part of the virtual actuating body VW, can enter the virtual regulating portion VB1 in the thickness direction from its tip. In the example shown in FIG. 35 , the tool portion VD3 is configured in a rod shape. In this embodiment, the tip of the rod-shaped actuating body is flexibly deformable in the real space RS, but in the virtual space VS, the tip of the rod-shaped virtual actuating body VW is configured to enter the virtual regulating portion VB1 without deforming. FIG. 35 shows three rod-shaped portions VD3 with different penetration depths into the virtual regulating portion VB1. The rod-shaped portion VD3 is a portion whose tip is pressed against the surface of a predetermined work object VOB1 (e.g., a sheet-like member such as paper, provided on the surface of the virtual regulating portion VB1 (the upper surface in FIG. 35 )) like a writing implement (e.g., a brush), and the thickness of the line drawn on the surface of the work object VOB1 can change depending on the pressure of the pressing force. The rod-shaped portions VD3 on the left, center, and right sides in FIG. 35 have progressively greater penetration depths in that order. The rod-shaped portion VD3 is configured to gradually become thicker from the tip side to the base end side (a cone-shaped portion in the example shown in FIG. 35 ). As the depth to which the rod-shaped portion VD3 penetrates the virtual regulating portion VB1 from the tip side to the base end side increases, the existence region RE of the rod-shaped portion VD3 on the surface of the virtual regulating portion VB1 (the region where the surface of the virtual regulating portion VB1 and the rod-shaped portion VD3 intersect; the elliptical region in FIG. 35 ) is configured to become larger (see FIG. 35 ). As the rod-shaped portion VD3 moves along the surface of the work object VOB1, the existence region RE can trace a trajectory TR corresponding to characters, etc. The top surface of the virtual work object VOB1, which corresponds to the work object such as paper, is set to coincide with the top surface of the virtual regulating portion VB1 or to a position slightly shifted in the thickness direction.In the real space RS, the thickness of the lines drawn on the paper increases as the pressure of a writing implement such as a brush against the paper increases. Therefore, by using the size of the existence region RE described above, a simulation is performed to control the penetration depth of the rod-shaped portion VD3 within a predetermined range, and based on the results of the simulation, the motion control unit 2 controls the motion of the operating body (for example, a writing implement), thereby adjusting the thickness of the lines drawn on the paper or the like.

[0088] The embodiment shown in FIG. 36 enables an operation in which a predetermined pressure is applied to a work object (e.g., an object to which a coating such as putty is to be applied) by pressing an operating body (e.g., a tool (working device) such as a spatula) against the work object by determining the positional relationship between the virtual operating body VW and the virtual work object VOB1. In this embodiment, a virtual working device VD corresponding to the working device has a virtual tool part VD3 corresponding to the tool part (e.g., a spatula). A virtual regulating part VB1 is provided along the surface of the virtual work object VOB1 corresponding to the work object to be worked by the operating body, but is located inside the virtual work object VOB1 (inside the surface of the virtual work object VOB1). The depth at which the virtual regulating part VB1 is located is set to a depth corresponding to the pressure with which the tool part is pressed against the work object. The operating body (e.g., a spatula) bends a predetermined amount when a predetermined pressure is applied, but the virtual operating body VW is configured not to bend. When the tip of the virtual operating body VW contacts the virtual regulating portion VB1 located inside the virtual work object VOB1, the operating body in the real space RS contacts the work object with a predetermined pressure. This makes it possible to control the pressure with which the operating body is pressed against the work object using the virtual regulating portion VB1 in the virtual space VS. To increase the pressure with which the tool unit presses against the work object, the depth at which the virtual regulating portion VB1 is located can be set to a larger value. The simulator 1 operates the virtual operating body VW (specifically, the virtual work device VD) so that the virtual operating body VW contacts the virtual regulating portion VB1. The operation controller 2 enables work to be performed while applying pressure to the surface of the work object OB1 by operating the virtual operating body VW. When the virtual operating body VW contacts the virtual regulating portion VB1 inside the virtual work object VOB1, the tool unit, such as a spatula, bends in the real space RS due to the pressing pressure. Therefore, the putty can be applied to the workpiece with an appropriate pressure, and damage to the spatula and the workpiece can be prevented. In the application work using a spatula, the spatula and the workpiece are in line contact, but this is not limited to this. For example, if the tool part is a grinder, the same principle can be applied to the grinding work in which the grinder makes surface contact with the workpiece (see Figure 37).

[0089] The embodiment shown in FIG. 38 suppresses contact between multiple actuating bodies (e.g., multiple work devices WD) when work is being performed by these bodies. The simulator 1 is configured to provide virtual regulating portions VB1 between multiple virtual actuating bodies VW corresponding to the multiple actuating bodies when the distance between the multiple actuating bodies becomes equal to or less than a predetermined distance. Specifically, when the distance between the multiple actuating bodies (e.g., multiple work devices) becomes equal to or less than a predetermined distance (a distance at which mutual interference may occur), the simulator 1 provides virtual regulating portions VB1 between multiple virtual work devices VD corresponding to the multiple actuating bodies (see FIG. 38). The virtual regulating portions VB1 are arranged to separate the multiple actuating bodies. Specifically, the virtual regulating portions VB1 are arranged to separate the multiple virtual work devices VD corresponding to the multiple actuating bodies (see FIG. 38). The shape and size of the virtual regulating portions VB1 are not particularly limited as long as they can prevent interference between the actuating bodies by separating the multiple actuating bodies. In the example shown in FIG. 38 , the virtual regulating portion VB1 is plate-shaped (e.g., flat), but it may also be planar (e.g., planar or curved). When the virtual working device VD comes into contact with the virtual regulating portion VB1, the simulator 1 prevents the virtual working device VD from moving beyond the virtual regulating portion VB1. This prevents interference between the virtual working devices VD. The simulator 1 may also be configured to eliminate the virtual regulating portion VB1 when the spacing between multiple acting bodies exceeds a predetermined distance (see FIG. 39 ). Specifically, the simulator 1 eliminates the virtual regulating portion VB1 when the spacing between multiple virtual acting bodies VW corresponding to multiple acting bodies exceeds a predetermined distance (a distance at which mutual interference is unlikely). This configuration prevents interference between the acting bodies and increases the degree of freedom in the work.

[0090] In another embodiment shown in FIGS. 40 to 43 , a first virtual regulating portion VB1 is provided to individually surround a portion of one of the multiple acting bodies (see FIG. 40 ). Specifically, the first virtual regulating portion VB1 is provided to individually surround a virtual tool portion VD3 corresponding to a portion (e.g., a tool portion) of one of the multiple acting bodies (e.g., work devices). Furthermore, a second virtual regulating portion VB2 is provided to individually surround a portion of another of the multiple acting bodies (see FIG. 40 ). The simulator 1 simulates the positional relationship between the first virtual regulating portion VB1 and the second virtual regulating portion VB2. The operation control portion 2 is configured to prevent the spacing between the multiple acting bodies from becoming equal to or smaller than a predetermined spacing based on the simulated positional relationship. Specifically, the simulator 1 is configured to prevent the spacing between the virtual work devices VD corresponding to the multiple acting bodies from becoming equal to or smaller than a spacing at which mutual interference may occur based on the simulated positional relationship. For example, when the distance between multiple virtual operating bodies (virtual work devices VD) reaches a predetermined distance, the simulator 1 temporarily stops the operation of both of them (see FIG. 41). With this configuration, it is possible to prevent interference between the multiple operating bodies.

[0091] In addition, the simulator 1 is preferably configured to change the size and / or shape of at least one of the first virtual regulating portion VB1 and the second virtual regulating portion VB2 depending on the positional relationship between the first virtual regulating portion VB1 and the second virtual regulating portion VB2 (see Figure 42). In the example shown in Figure 42, the simulator 1 is configured to reduce the size of at least one of the first virtual regulating portion VB1 and the second virtual regulating portion VB2 when the first virtual regulating portion VB1 and the second virtual regulating portion VB2 approach each other and the distance between the first virtual regulating portion VB1 and the second virtual regulating portion VB2 becomes a predetermined distance. In the example shown in FIG. 42 , the simulator 1 is configured to narrow the shape of at least one of the first virtual regulating portion VB1 and the second virtual regulating portion VB2 when the first virtual regulating portion VB1 and the second virtual regulating portion VB2 approach each other and the distance between the first virtual regulating portion VB1 and the second virtual regulating portion VB2 reaches a predetermined distance. Specifically, the simulator 1 is configured to reduce the size of at least one of the first virtual regulating portion VB1 and the second virtual regulating portion VB2 so that work can be performed in order starting with the operating body with the highest priority. This configuration allows work to be performed in order starting with the operating body with the highest priority. For example, in the example shown in FIG. 42 , by reducing the size of the first virtual regulating portion VB1 and the second virtual regulating portion VB2, the left virtual operating portion VW can perform work first. The simulator 1 then retracts the virtual operating portion VW from the work position after the work has been completed. Then, when the distance between the two virtual actuators VW becomes sufficiently large and there is no possibility of interference, the simulator 1 eliminates the first virtual regulating portion VB1 and the second virtual regulating portion VB2 (see Figure 43).

[0092] The simulator 1 may be configured to increase the size of at least one of the first virtual regulating portion VB1 and the second virtual regulating portion VB2 when the first virtual regulating portion VB1 and the second virtual regulating portion VB2 approach each other and the distance between the first virtual regulating portion VB1 and the second virtual regulating portion VB2 reaches a predetermined distance. In this case, interference between multiple actuating bodies can be more reliably prevented. The simulator 1 may also be configured to change the shape of at least one of the first virtual regulating portion VB1 and the second virtual regulating portion VB2 to an arbitrary shape that makes it easier for the actuating body to perform the operation when the first virtual regulating portion VB1 and the second virtual regulating portion VB2 approach each other and the distance between the first virtual regulating portion VB1 and the second virtual regulating portion VB2 reaches a predetermined distance. In addition, when changing the size and / or shape of at least one of the first virtual regulating section VB1 and the second virtual regulating section VB2, the simulating section 1 may be configured to eliminate the virtual regulating section whose size and / or shape is to be changed, and then generate a virtual regulating section whose size and / or shape has been changed.

[0093] In the embodiment shown in Figures 44 to 46, the operating body is a work device that applies a predetermined operation to a moving work object, allowing the work device to follow the moving work object. In this embodiment, as an example, the operating body is placed on a transport device such as a conveyor and follows the moving work object to perform a predetermined task. In this embodiment, the virtual regulating portion VB1 is determined based on the position of the moving work object. The virtual regulating portion VB1 may be generated so as to overlap a portion of the moving virtual work object VOB1, or may be generated at a position slightly separated from the moving virtual work object VOB1. The virtual regulating portion VB1 is generated so as to maintain a constant distance from the moving virtual work object VOB1. Therefore, the virtual regulating portion VB1 is configured to move together with the virtual work object VOB1 within the virtual space SV (see Figures 45 and 46). In this case, the working device is moved toward the moving work object, and when the virtual working device VD hits the moving virtual restricting portion VB1 in the virtual space VS, the virtual working device VD stops at the position of the moving virtual work object VOB1. This allows the working device in the real space RS to also stop at the part of the moving work object by the operation control portion 2 and perform a predetermined task on the moving work object (see FIGS. 44 to 46).

[0094] In the embodiment shown in FIG. 47 , the virtual regulating portion VB1 is configured to restrict movement of the acting body within the range inside the virtual regulating portion VB1. The shape of the virtual regulating portion VB1 is not particularly limited as long as it can restrict movement of the acting body within the range inside the virtual regulating portion VB1, and it may be, for example, frame-shaped or container-shaped. In the example shown in FIG. 47 , the virtual regulating portion VB1 is configured in a frame shape. For example, when the virtual acting body VD corresponding to the acting body hits the inner circumferential surface of the frame-shaped virtual regulating portion VB1, the simulator 1 corrects the movement direction along the first virtual regulating portion VB1. This allows movement of the acting body to be restricted within the range inside the first virtual regulating portion VB1.

[0095] The above-described embodiment is preferably applied to, for example, the following industrial robots. The industrial robot mainly processes metals or materials used in metal processing. For example, the industrial robot is a robot that performs tasks such as spraying, sealing, jet nozzle, roller brush, paintbrush, brushing, roller herring, polishing, grinding, drilling, cutting, shoveling, scooping, rake, scraping, plastering trowel, or ladle work.

[0096] In addition, the industrial robot may be a robot that performs tasks such as scarfing, processing, assembling, welding, transporting, inspecting, pressing, painting, spraying, coating, writing, cleaning, cutting, arranging, or picking.

[0097] Spraying is the process of applying a liquid or powdered solid to an object.

[0098] Sealing is the process of applying a sealant (often in paste form) to an object.

[0099] Jet nozzles are used, for example, to spray water at high pressure onto an object to clean it.

[0100] Roller brushing is the process of applying colored paint or other liquids to an object using a roller.

[0101] Brush painting is a process performed using a tool made of bundled bristles called a "brush." ​​The materials of the bristles and handle that make up the brush vary depending on the application. For example, brush painting can be performed for painting, cleaning, or applying various coating agents. Brush painting can also be used to apply paint, adhesives, or cleaning agents.

[0102] Painting is a process of applying paint or the like to an object to give it color, etc. However, painting may also be performed for purposes other than coloring, such as applying fireproofing material or coating.

[0103] Lettering is the act of forming letters (which may also include numbers, pictures, or patterns) on an object by, for example, applying paint or engraving.

[0104] Spraying is the process of depositing a gas, liquid, or gel-like substance onto an object.

[0105] Brushing is the process of pressing a metal brush against an object and moving the brush. For example, brushing is used to remove paint or rust from a surface. Brushing is also used to create lines on a surface or to process a surface.

[0106] Roller hemming is a process of hemming the edges of automobile doors or bodies while applying pressure with a roller attached to the tip of a robot.

[0107] Polishing is the process of using an abrasive or the like to scrape and smooth the surface of an object.

[0108] Grinding is a process of scraping the surface of an object using hard particles such as a grindstone or powder.

[0109] Cutting and scraping are operations in which a part of an object is removed from the object using a tool.

[0110] Drilling is the process of forming a hole in an object using a drill or the like.

[0111] Cutting is an operation of dividing an object into multiple pieces using a tool.

[0112] Gathering is the process of concentrating scattered objects in a specific location using a tool such as a rake.

[0113] Troweling is the process of applying a substance to an object using a tool known as a trowel. For example, troweling can be used for repairs, finishing, or painting large areas.

[0114] Scooping with a ladle and scooping are the actions of scooping up an object using a tool such as a ladle.

[0115] Scarfing (also called "scarf" or "scalding") uses a gas containing multiple types of gases. The gas includes oxygen. Scarfing uses a gas in which flammable gases such as hydrogen, acetylene (propane), and LPG (Liquefied Petroleum Gas) are mixed with oxygen. Note that two or more types of gases may be mixed with oxygen. The gas is used in a high-pressure state for scarfing. Specifically, scarfing is a process in which high-pressure gas is sprayed onto an object to remove defects in the object. The removal process can perform processes such as scalding, gouging, cutting, or fillet removal on the object.

[0116] Processing is the process of forming the shape of an object. For example, processing includes mechanically cutting, bending, punching, hammering, applying pressure, or putting something into a mold to create a shape. Processing also includes the process of creating or adding a shape using a 3D printer, etc. Other examples of processing include the process of changing the properties of an object. For example, processing includes heating or surface treatment. In this way, processing is the process of performing mechanical, thermal, or chemical treatment.

[0117] Assembly is the process of combining multiple parts to create a complete assembly.

[0118] Welding is a process of joining parts of an object by applying heat or pressure to them.

[0119] Transportation is the process of moving an object from its current location to another location.

[0120] Inspection is the process of determining whether the quality of an object that has been processed satisfies requirements. For example, inspection may involve checking for scratches on the surface of the object, scratches detected by ultrasonic waves, a surface profile (e.g., measured according to ISO 8503), or internal distortion.

[0121] Pressing is a process of applying pressure to an object to cause deformation or the like.

[0122] Cleaning is a process of spraying water or scraping to remove dirt and other debris from an object.

[0123] Picking is the task of taking out an object.

[0124] Sorting is the task of placing multiple objects that have been picked up by picking or other means in a specific order or in a specific location.

[0125] [Other Embodiments] The processes described above and the data used in the processes performed in this embodiment may be executed and stored by an information processing system. For example, the information processing system may execute or store the processes or data in multiple information processing devices to achieve redundant, distributed, parallel, or a combination thereof. Therefore, the present invention may be realized in devices other than those having the hardware configurations described above and in systems other than those described above.

[0126] Furthermore, the program according to the present invention is not limited to a single program, but may be a collection of multiple programs. Furthermore, the program according to the present invention is not limited to being executed by a single device, but may be executed by multiple information processing devices in a shared manner. Furthermore, the allocation of roles among the information processing devices is not limited to the above-mentioned example. In other words, some or all of the above-mentioned processes may be executed by information processing devices different from the above-mentioned information processing device.

[0127] Some or all of the means implemented by the program may be implemented in hardware such as an integrated circuit. Furthermore, the program may be provided in a form recorded on a non-transitory recording medium readable by a computer. Examples of recording media include a hard disk, an SD card (registered trademark), an optical disk such as a DVD, or a server on the Internet. Therefore, the program may be distributed via a telecommunications line such as the Internet.

[0128] Furthermore, the information processing devices that make up the information processing system may be located overseas.

[0129] The present invention is not limited to the above-described embodiments. Therefore, the present invention allows for the addition or modification of components within the scope of the technical gist. Therefore, all technical matters included in the technical concept described in the claims are subject to the present invention. In other words, the above-described embodiments are preferred specific examples for implementation. Furthermore, a person skilled in the art can realize various modifications from the disclosed content, and such modifications are included in the technical scope described in the claims.

[0130] 1 Simulator 2 Operation control unit 3 Input unit 4 Output unit CL1 Center line of virtual target position (hole) CL2 Center line of virtual work object CP Contact position CP1 First contact position CP2 Second contact position CP3 Third contact position OB1 Work object OD Operation device OP1 Target position P1 Collision point PE Opening periphery RS Real space VB1 First virtual regulating portion VB11 Virtual guide surface VB1a, VB1b, VB1c Planar portion VB2 Second virtual regulating portion VB21 Rod-shaped portion VB22 Ring-shaped portion VB23 Connecting portion VBG Guide regulating portion VBS Swing regulating portion VD Virtual work device VD3 Virtual tool portion VD31 Virtual finger VOB1 Virtual work object VOP1 Virtual target position VS Virtual space WS Operation system WD Work device WD1 Base part WD2 Arm part WD3 Tool part WD31 Finger WD4 Measuring device WD41 Camera θ Tilt angle

Claims

1. An operation system comprising: a simulating unit that simulates, in a virtual space, the positional relationship between a virtual operating body corresponding to the operating body and a first virtual regulating unit that is virtually provided to regulate the movement of the virtual operating body, in order to simulate the operation of the operating body that performs a predetermined operation in response to an input operation from an operating device by a user; and an operation control unit that controls the operation of the operating body in response to the positional relationship simulated by the simulating unit, wherein, when an input operation is performed on the operating body, the simulating unit operates the virtual operating body in the virtual space with content corresponding to the input operation in parallel with the input operation, and determines the positional relationship between the virtual operating body and the first virtual regulating unit at predetermined time intervals; and the operation control unit is configured to operate the operating body in parallel with an operation command based on the input operation, based on the result of the determination by the simulating unit; and the simulating unit determines whether the distance between the virtual operating body and the first virtual regulating unit is within a predetermined range. When the distance between the virtual operating body and the first virtual regulating portion is determined to be within the predetermined range, and the movement direction of the virtual operating body corresponding to the input operation includes a component in a direction toward the surface of the first virtual regulating portion and a component in a direction along the surface of the first virtual regulating portion, the simulator unit corrects the movement direction of the virtual operating body corresponding to the input operation based on the shape of the first virtual regulating portion, thereby: (i) determining the movement direction of the virtual operating body for moving the virtual operating body along the surface of the first virtual regulating portion, and the operation control unit is configured to move the operating body in the determined movement direction, or (ii) stopping the virtual operating body against the surface of the first virtual regulating portion, and the operation control unit is configured to stop the operating body against the first virtual regulating portion, or (iii) repelling the virtual operating body against the surface of the first virtual regulating portion, and the operation control unit is configured to repel the operating body against the first virtual regulating portion.

2. The actuation system of claim 1, wherein the simulator determines whether the virtual actuation body and the first virtual regulating portion are in contact based on the distance between the virtual actuation body and the first virtual regulating portion, and when it is determined that the virtual actuation body and the first virtual regulating portion are in contact, the simulator is configured to correct the movement direction of the virtual actuation body corresponding to the input operation based on the shape of the first virtual regulating portion.

3. An actuation system as described in claim 1, wherein the first virtual regulating section has a virtual guide surface configured to guide the actuating body to a predetermined target position, and the first virtual regulating section is configured to allow the virtual actuating body to pass through the virtual guide surface from the side opposite the virtual guide surface to the virtual guide surface side, and to prevent the virtual actuating body from passing through the virtual guide surface from the virtual guide surface side to the side opposite the virtual guide surface.

4. An actuation system as described in claim 1, wherein the first virtual regulating unit is configured to guide the actuating body to a predetermined destination position, the simulating unit is configured to generate the first virtual regulating unit when the distance between the actuating body and the destination position becomes equal to or less than a predetermined distance, and the simulating unit is configured to eliminate the first virtual regulating unit when the distance between the actuating body and the destination position exceeds the predetermined distance.

5. An actuation system as described in claim 1, wherein the first virtual regulating section is configured to guide the actuating body to a hole that is a predetermined target position, and the first virtual regulating section has a guide regulating section that guides the insertion target section so that a portion of the tip of the insertion target section of the virtual actuating body is inserted into the hole and the center line of the insertion target section is inclined relative to the depth direction of the hole.

6. An operating system as described in claim 5, further comprising a swing regulating unit that regulates the swing direction of the insertion target part about the contact position with the periphery of the opening when the insertion target part, with a portion of its tip inserted into the hole, is in contact with the periphery of the opening of the hole, and the swing regulating unit regulates the swing direction so that the inclination angle of the center line of the insertion target part relative to the depth direction of the hole is small.

7. An actuation system as described in claim 1, wherein the virtual actuating body has a main body portion corresponding to the actuating body and a second virtual regulating portion provided on at least a part of the main body portion and moving relative to the first virtual regulating portion, and the simulating portion is configured to determine the positional relationship between the virtual actuating body and the first virtual regulating portion by determining the positional relationship between the first virtual regulating portion and the second virtual regulating portion.

8. An actuation system as described in claim 7, wherein the first virtual regulating section is configured to guide the actuating body to a predetermined target position, the first virtual regulating section has a virtual guide surface configured so that the range of movement of the virtual actuating body becomes narrower as the first virtual regulating section approaches the target position, and the actuating body is guided to the target position by the second virtual regulating section being guided by the virtual guide surface.

9. An actuation system as described in claim 1, wherein the first virtual regulating portion has a predetermined thickness, and the first virtual regulating portion is configured so that a portion of the virtual operating body can enter from the tip in the thickness direction of the first virtual regulating portion, the portion is configured to gradually become thicker from the tip side toward the base end side, and the area of ​​existence of the portion on the surface of the first virtual regulating portion becomes larger as the depth to which the portion enters the first virtual regulating portion from the tip side toward the base end side increases.

10. The actuation system of claim 1, wherein the actuation body is a work device that applies a predetermined operation to a predetermined work object, the first virtual regulating unit is disposed inside the work object, the simulating unit operates the virtual actuation body so that the virtual actuation body contacts the first virtual regulating unit, and the operation control unit enables work to be performed while applying pressure to the surface of the work object by the operation of the virtual actuation body.

11. An actuation system as described in claim 8, wherein the simulating unit is configured to change the size and / or shape of at least one of the first virtual regulating unit and the second virtual regulating unit depending on the positional relationship between the first virtual regulating unit and the second virtual regulating unit.

12. The actuation system according to claim 1, wherein the actuating body is a work device that applies a predetermined operation to a moving work object, and the first virtual regulating part is determined based on the position of the moving work object and is configured to move together with the work object within the virtual space.

13. The actuation system according to claim 1, wherein the first virtual restriction portion is configured to restrict movement of the actuation body within a range inside the first virtual restriction portion.

Images