Actuation system

Abstract

Providing an operating system that appropriately controls the operation of an operating body even if the operating body may operate along an unexpected locus due to an arbitrary operation by a user. 【Solution means】In order to simulate the operation of an operating body that performs a predetermined operation in response to an input operation from an operation device by a user, in a virtual space, a virtual operating body corresponding to the operating body, and a virtual restriction unit that is virtually provided to restrict the movement of the virtual operating body A simulation unit that simulates the positional relationship between the virtual operating body and the virtual restriction unit, and an operation control unit that controls the operation of the operating body according to the simulated positional relationship. The simulation unit determines that the distance between the virtual operating body and the virtual restriction unit is within a predetermined range. In this case, the moving direction of the virtual operating body corresponding to the input operation is configured to be corrected based on the shape of the virtual restriction unit.

Description

Technical Field

[0001] The present invention relates to an operating system.

Background Art

[0002] Conventionally, in order to prevent inconveniences such as a work device such as a robot that performs work on behalf of a person from interfering with an obstacle when performing work, a technique has been proposed (see, for example, Patent Document 1). The interference prevention device described in Patent Document 1 calculates the operation trajectory of the robot before the operation of the robot based on the content pre-taught to the robot, determines whether the robot interferes with an obstacle based on the calculation result, and when it is determined that the robot interferes with an obstacle, it is configured to perform control such as issuing a standby command to the robot.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the technique described in Patent Document 1 controls the operation along a predetermined trajectory based on teaching. For this reason, if the user operates the robot along an arbitrary trajectory via the operating device, the technique described in Patent Document 1 may not be able to appropriately control the operation of the robot. Specifically, when the user operates the robot along an arbitrary trajectory via the operating device, the robot may operate on an unexpected trajectory. Therefore, with the technique described in Patent Document 1, for example, the operation of the robot may not be appropriately controlled, such as when the robot interferes with an obstacle during work.

[0005] In view of the above problems, an object of the present invention is to provide an operating system that enables appropriate control of the operation of an operating body even when the operating body may operate along an unexpected trajectory due to an arbitrary operation by a user.

Means for Solving the Problems

[0006] The operating system of the present invention includes a simulation unit that simulates the positional relationship between a virtual operating body corresponding to the operating body and a first virtual restriction unit that is virtually provided to restrict the movement of the virtual operating body in a virtual space in order to simulate the operation of the operating body that performs a predetermined operation in response to an input operation from an operating device by a user, and an operation control unit that controls the operation of the operating body according to the positional relationship simulated by the simulation unit. The simulation unit, when an input operation is performed on the operating body, operates the virtual operating body in the virtual space with the content corresponding to the input operation in parallel with the input operation, determines the positional relationship between the virtual operating body and the first virtual restriction unit at a predetermined time interval, and the operation control unit is configured to operate the operating body in parallel with the operation command based on the input operation based on the result of the determination by the simulation unit. The simulation unit determines whether the distance between the virtual operating body and the first virtual restriction unit is within a predetermined range, and when it is determined that the distance between the virtual operating body and the first virtual restriction unit is within the predetermined range, and the moving direction of the virtual operating body corresponding to the input operation includes a component in the direction toward the surface of the first virtual restriction unit and a component in the direction along the surface of the first virtual restriction unit, the simulation unit corrects the moving direction of the virtual operating body corresponding to the input operation based on the shape of the first virtual restriction unit, thereby (i) obtaining the moving direction of the virtual operating body for moving the virtual operating body along the surface of the first virtual restriction unit, and the operation control unit is configured to move the operating body in the obtained moving direction, or (ii) stopping the virtual operating body with respect to the surface of the first virtual restriction unit, and the operation control unit is configured to stop the operating body with respect to the first virtual restriction unit, or (iii) repelling the virtual operating body with respect to the surface of the first virtual restriction unit, and the operation control unit is configured to repel the operating body with respect to the first virtual restriction unit.

Effect of the Invention

[0007] According to the present invention, even if there is a possibility that the operating body operates in an unexpected trajectory due to an arbitrary operation by the user, it is possible to provide an operating system that enables appropriate control of the operation of the operating body.

Brief Explanation of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Figure 38

Figure 39

Figure 40

Figure 41

Figure 42

Figure 43

Figure 44

Figure 45

Figure 46

Figure 47

Mode for Carrying Out the Invention

[0009] Hereinafter, with reference to the drawings, an operating system according to an embodiment of the present invention will be described. Note that the following embodiments are merely examples, and the operating system of the present invention is not limited to the following embodiments.

[0010] The operation system WS described below enables appropriate control of the operation of an operating body by simulating the operation of the operating body that performs a predetermined operation in response to an input operation from an operating device by the user. In this specification, the operating body refers to any structure that performs a predetermined operation in response to an input operation from an operating device OD (see FIG. 1) by the user. In the embodiment shown below, the operating body is constituted by a working device WD (see FIG. 1) such as a robot, but the operating body is not limited to a working device such as a robot. Note that the operating body may be constituted by only one structure such as the working device WD, or may include other elements that move along with the structure in addition to one structure. For example, when a predetermined work object OB1 (see FIG. 1) is operated by the working device WD (when a part of the working device WD and the work object OB1 move integrally), the working device WD, which is the first operating element, and the work object OB1, which is the second operating element, are treated as one operating body.

[0011] The operation system WS (see FIG. 2) according to this embodiment is, for example, as shown in FIG. 1, a system that performs work on a work object OB1 using a working device WD such as a robot that performs work on behalf of a person. Specifically, for example, the working device WD is configured to perform a predetermined operation in response to an input operation from an operating device OD by the user. In this embodiment, the working device WD is configured to perform a predetermined operation in response to an operation command, which is a command indicating the content of the operation to be performed on the working device WD, in various facilities such as production facilities.

[0012] The "operation of the working device WD" in this specification includes, but is not limited to, at least any one of operations such as conveyance of the work object OB1, positioning of the work object OB1, relative movement with respect to the work object OB1, following the work object OB1, etc. The work including the conveyance of the work object OB1 includes, but is not limited to, for example, pick and place (gripping the work object OB1 and conveying it to the target position), etc. The work including the positioning of the work object OB1 includes, but is not limited to, for example, fitting (gripping the work object OB1 and fitting it into a hole), etc. The work including the relative movement with respect to the work object OB1 includes, but is not limited to, for example, painting, polishing, writing characters, scooping gravel, etc. In the present embodiment, as will be described later, the working device WD is configured to guide the work object OB1 to the target site. Also, in other embodiments described later, the working device WD is configured to be able to prevent interference with other working devices WD or surrounding structures and the working device WD itself when performing the target work. In still other embodiments, the working device WD is configured to control the distance to the work object OB1 by the working device WD, and in still other embodiments, the working device WD is configured to follow the moving work object OB1.

[0013] In this embodiment, the working device WD is configured to be operable in a manual mode. In addition to the manual mode, the working device WD may be a device that operates in a semi-automatic mode or an automatic mode. The manual mode is a mode that is always operated by the user. The semi-automatic mode is a mode in which only some predetermined operations are automatically performed and other operations are operated by the user. The automatic mode is a mode in which all operations are automatically performed using a sensor or the like. In this specification, an "operation command" is a command indicating the content of the operation to be performed on the working device WD. The operation to be performed on the working device WD is, for example, an operation that the user attempts to perform on the working device WD via the operation device OD, or an operation that a program of a computer (a computer for executing an automatic mode or semi-automatic mode operation) that causes the working device WD to perform an automatic operation attempts to perform on the working device WD. The operation to be performed on the working device WD can be corrected by interference prevention control and / or imitation control described later based on the result of simulation by the simulation unit 1.

[0014] The working device WD is configured to operate based on an operation command based on an input operation from the operation device OD. The operation command is generated by a program that generates an operation command based on the input operation from the operation device OD. The program has a function of converting an operation request into an operation command by performing an operation based on an operation request for operating the working device WD according to the content of the operation to be performed on the working device WD. The program is stored, for example, in a storage unit of a computer for operating or moving the working device WD (a computer other than the simulation unit 1 described later). In the manual mode as shown in FIG. 1, the computer is provided, for example, in the operation device OD. In the automatic mode, the computer is provided, for example, in the above-described computer that causes the working device WD to perform an automatic operation. In the semi-automatic mode, the computer is provided, for example, in the operation device OD and the above-described computer that causes the working device WD to perform an automatic operation. The program is configured to perform an operation of converting an operation request (for example, an operation request by a user's operation or a predetermined operation request in an automatic operation) for operating the working device WD according to the content of the operation to be performed on the working device WD into an operation command.

[0015] Based on the above calculation results, the operation requirements are converted into operation commands for each component of the working device WD (such as the arm part WD2, etc.). The operation command is operation information of the components of the working device WD for realizing the operation of the working device WD corresponding to the operation requirements. Specifically, the operation information is, for example, one or more of the moving speed, acceleration and deceleration, moving direction, moving distance, rotation angle of the joint, rotation speed, and rotation acceleration and deceleration of the components of the working device WD, that is, the value of the parameter for realizing the operation of the components of the working device WD corresponding to the operation requirements. When the working device WD operates in the manual mode, the operation command is, for example, operation information (parameter value) for realizing the movement of the components of the working device WD corresponding to the operation given by the user to the operation device OD. In this case, the operation device OD converts the content of the operation given by the user to the operation device OD into operation information for realizing the movement of the components of the working device WD. When the working device WD operates in the automatic mode, the operation command is, for example, operation information for realizing the movement of the components of the working device WD corresponding to the operation that the computer for automatically operating the working device WD intends to make the working device WD perform. In this case, the above computer converts the content of the operation that it intends to make the working device WD perform into operation information for realizing the movement of the components of the working device WD. When the working device WD operates in the semi-automatic mode, the operation command is a command corresponding to both the manual mode and the automatic mode. As will be described later, in this embodiment, the operation command after the above calculation and conversion by the program stored in a predetermined computer is transmitted to the simulation unit 1 described later. The simulation unit 1 receives and uses the converted operation command, so that it is not necessary for the simulation unit 1 to perform the above-described calculation and conversion processing.

[0016] Note that the structure of the working device WD can be appropriately changed according to the required work content and is not limited to the illustrated structure. In this embodiment, the working device WD is shown as an articulated robot, but it is not limited to a robot and may be a construction machine such as a crane. Further, the working device WD may be a device that is operated (remotely operated) from a location away from the working device WD, or may be a device that is operated (for example, machine-side operation) in the vicinity of the working device WD instead of remote operation. In the following description, a case where the working device WD is a working device that operates in manual mode and is a robot remotely operated by a user will be described. Note that the content described below is also applicable in automatic mode and semi-automatic mode.

[0017] The working device WD shown in FIG. 1 is a device that performs work by applying a predetermined operation to the work object OB1. In the example shown in FIG. 1, the working device WD is configured to be movable in a predetermined direction and to perform a predetermined work in response to a user's operation. The working device WD includes a pedestal portion WD1, an arm portion WD2 having a plurality of joints and rotatably fixed to the pedestal portion WD1, and a tool portion WD3 provided at the tip of the arm portion WD2. The tool portion WD3 corresponds to a tool or a machine tool that performs work on the work object OB1, and performs an operation corresponding to the type of work performed by the working device WD on the work object OB1. In the present embodiment, the tool portion WD3 includes fingers WD31 that grip the work object OB1. In the present embodiment, the working device WD is configured to be operable in a first direction D1 that is one direction (X direction) in the horizontal direction, a second direction D2 that is a direction (Y direction) perpendicular to the first direction D1 in the horizontal direction, and a third direction D3 that is the vertical direction. Specifically, the pedestal portion WD1 rotates around an axis extending in the third direction D3, and the arm portion WD2 and the tool portion WD3 having a plurality of joints operate, whereby operations in the first direction D1, the second direction D2, and the third direction D3 become possible. Note that the operation direction and operation method of the working device may be different from those of the working device WD shown in the drawing. For example, instead of the working device itself being fixed to the installation surface, it may move horizontally in the first direction D1 and the second direction D2 with respect to the installation surface, or may be configured to move along a beam or a column.

[0018] Also, as shown in FIG. 1, the working device WD may further include a measuring device WD4 that measures the position of the work object OB1. The measuring device WD4 may include, for example, a camera WD41 that captures an image of the work object OB1 and a measuring unit (not shown) such as a sensor that measures the position of the work object OB1 based on the captured image. In the present embodiment, the measuring device WD4 is provided on the base WD5 that supports the working device WD. However, the position where the measuring device WD4 is provided is not particularly limited as long as it can measure the position of any part of the work object OB1. In the present embodiment, the position (or angle, posture) of the work object OB1 measured by the measuring device WD4 is associated with the outer shape of the work object OB1 and stored in a storage unit (not shown). This information is used when forming a virtual working body (a part corresponding to the working device WD and the work object OB1 integrated in the virtual space) described later. Note that the measuring device WD4 is not limited to having a camera as long as it can measure the position of the work object OB1, and it may be a sensor other than a camera that can measure the position of the work object OB1. Further, the measuring device WD4 may be a device that can measure both the position (angle, posture) and the outer shape of the work object OB1. In this case, the position and the outer shape measured by the measuring device WD4 are associated with each other and stored in a storage unit (not shown). The outer shape of the work object OB1 may be stored in a predetermined storage device in advance. Note that the formation of the virtual working body may be based on information from sources other than the measuring device WD4.

[0019] The working device WD is communicably connected to an operating device OD that receives the user's operations. The working device WD is configured to receive operation information indicating the content of the user's operations input to the operating device OD and perform operations corresponding to the operation information. The configuration of the operating device OD is not particularly limited as long as it can instruct the working device WD about the content of the work including the movement of the tool unit WD3. In the example shown in FIG. 1, the operating device OD is configured to be able to instruct the tool unit WD3 to move three-dimensionally in any direction in the working space where the working device WD is actually provided. As the operating device OD, for example, a 3D mouse, a joystick, a tablet, or the like can be used.

[0020] In the present embodiment, as schematically shown in FIG. 1, the tool unit WD3 is a gripping device that grips a conveyance object, which is an example of the work object OB1 described later. The gripping device includes a plurality of fingers WD31 that grip the work object OB1 and a drive unit (not shown), such as a motor, that drives the fingers WD31. The gripping device is configured to perform a gripping operation by narrowing the distance between the fingers WD31 and perform a gripping release operation (release operation) by widening the distance between the fingers WD31. The gripping device may be configured to be able to adjust the orientation (tilt) of the fingers WD31 in a direction that is easy to grip according to the outer shape and posture of the work object OB1 when gripping the work object OB1. In this case, the gripping device can appropriately grip the work object OB1 according to the outer shape and posture of the work object OB1. In the example shown in FIG. 1, the working device WD performs pick-and-place of the work object OB1. Specifically, the working device WD performs a series of operations of picking up (gripping and lifting) the work object OB1 at the start point of pick-and-place, moving the work object OB1 to the end point of pick-and-place, and releasing the work object OB1 at the end point.

[0021] In the example shown in FIG. 1, the work object OB1 is an object that is gripped by a tool part (gripping device) WD3 and moved by a work device WD. In the example shown in FIG. 1, the work object OB1 is a columnar object such as a cylindrical shape or a prismatic shape, but is not limited thereto.

[0022] In the example shown in FIGS. 1 and 2, the operating system WS is an operating body W (in this embodiment, the part where the work device WD and the work object OB1 are integrated in a state where the work object OB1 is gripped by the work device WD) that performs a predetermined operation in response to an input operation from an operation device OD by a user. In order to simulate the movement of the virtual operating body VW corresponding to the operating body W and a virtual restriction unit (corresponding to the first virtual restriction unit) VB1 that is virtually provided to restrict the movement of the virtual operating body VW in a virtual space, the simulation unit 1 that simulates the positional relationship between them, and an operation control unit 2 that controls the operation of the operating body W according to the positional relationship simulated by the simulation unit 1. In this embodiment, although the simulation unit 1 and the operation control unit 2 are schematically shown separately, the simulation unit 1 and the operation control unit 2 may be provided in the same device, or may be provided in separate devices. Further, the simulation unit 1 and the operation control unit 2 may be realized as different functions of one control unit. In this specification, the "first virtual restriction unit" is given the "first" in order to distinguish it from the "second virtual restriction unit" described later, but even if it is described as the "first virtual restriction unit", it is not necessarily required to provide the "second virtual restriction unit". Hereinafter, the first virtual restriction unit will also be simply referred to as the virtual restriction unit for explanation.

[0023] Before the working device WD operates under the operation control of the operation control unit 2, as shown in FIG. 4, the simulation unit 1 simulates the positional relationship between the virtual working body VW corresponding to the working body W and the virtual restriction unit VB1 in the virtual space VS. More specifically, as will be described later, when an arbitrary operation input is made to the operation device OD, the simulation unit 1 performs a simulation to determine whether to operate the working body as it is, stop the working body, or cause the working body to perform a different operation based on the operation input to the operation device OD. Although details will be described later, the virtual restriction unit VB1 restricts the movement of the virtual working body VW in the virtual space VS, thereby restricting the movement of the working body W in the real space RS to a predetermined direction and range.

[0024] In this specification, a "virtual actuator" is a virtual object generated in a virtual space that corresponds to an actuator, specifically corresponding to the shape and size of the actuator. Note that the virtual actuator may correspond to the entire actuator or a part of the actuator. Note that when the working device WD is not associated with the work object OB1, the virtual actuator can be a virtual object corresponding to the working device WD (or a part thereof), and when the working device WD grips the work object OB1 and moves together with the work object OB1, etc., it can be a virtual object corresponding to an object (or a part thereof) in which the working device WD and the work object OB1 are integrated. "A part of the object in which the working device WD and the work object OB1 are integrated" is a part that serves as a reference when determining the positional relationship with the virtual restriction unit VB1 in the object in which the working device WD and the work object OB1 are integrated. The said part is, for example, the work object OB1 or a part of the work object OB1. Specifically, "a part of the object in which the working device WD and the work object OB1 are integrated" is, for example, the tip of the work object OB1 (in the example shown in FIG. 1, the tip (for example, the end face) of the columnar work object OB1). The "positional relationship between the virtual actuator VW and the virtual restriction unit VB1" refers to the three-dimensional positional relationship between the virtual actuator VW and the virtual restriction unit VB1. More specifically, the "positional relationship between the virtual actuator and the virtual restriction unit VB1" is the positional relationship between the virtual actuator VW in the virtual space VS and the virtual restriction unit VB1, and although not limited, for example, the distance between the virtual actuator VW and the virtual restriction unit VB1, the direction in which the virtual actuator VW is provided with respect to the virtual restriction unit VB1, whether the virtual actuator VW is in contact with the virtual restriction unit VB1, and whether the virtual actuator VW is within a predetermined range (distance) with respect to the virtual restriction unit VB1, any one or more of them can be used.

[0025] In this embodiment, as shown in FIGS. 1 and 2, the simulation unit 1 is communicably connected to the operation device OD and the operation control unit 2. The simulation unit 1 receives, from the operation device OD, an operation command corresponding to the operation information indicating the content of the user's operation input to the operation device OD, and is configured to perform a simulation of the content corresponding to the operation command (in this embodiment, the content of the operation corresponding to the operation information). More specifically, the simulation unit 1 performs a computer simulation (computer simulation in a virtual space VS imitating the real space RS) based on the form information of the virtual working body VW corresponding to the form (shape and size) of the working body W stored in the storage unit (not shown), the form (shape and size) information of the virtual regulation unit VB1 stored in the storage unit, the relative position information with respect to each other at a certain point in time, and the content of the above-described operation command (in this embodiment, the operation information, the movement of the components of the working device WD corresponding to the operation, or the value of the parameter for realizing the movement of the components) to simulate the above-described positional relationship.

[0026] In the virtual space VS, the simulation unit 1 operates the virtual working body VW (in this embodiment, a virtual object integrating the virtual working device VD and the virtual work object VOB1) corresponding to the working body W in the content corresponding to the operation command. Specifically, the simulation unit 1 operates the virtual working device VD and the virtual work object VOB1 in the virtual space VS in the content corresponding to the operation command. For example, when the virtual working device VD grips and transports the virtual work object VOB1, as the Virtual tool part (gripping device) V moves D3, Virtual the tool part V and the virtual work object VOB1 gripped by D3 also moves. Therefore, the simulation unit 1 simulates not only the operation of the virtual working device VD but also the operation of the virtual work object VOB1. When the work object OB1 does not exist, or when the work object OB1 does not move together with the working device WD, the simulation unit 1 operates only the virtual working device VD in the virtual space VS in the content corresponding to the operation command.

[0027] When operating the virtual actuator, the simulation unit 1 receives an operation command from the outside (in the example shown in FIG. 1, the operation device OD), and determines the positional relationship between the virtual actuator and the virtual restriction unit VB1 based on the received operation command. In the present embodiment, the simulation unit 1 does not generate an operation command but acquires it from the outside (the operation device OD in the example shown in FIG. 1). That is, in the simulation unit 1, it does not calculate the operation command, which is operation information for the operation of the virtual work device VD in the virtual space VS, but calculates the coordinates of the components of the virtual work device VD based on the received calculated and converted operation command, and calculates the relative position information. Therefore, the calculation and conversion time in the simulation unit 1 becomes unnecessary, and high-speed simulation becomes possible. The simulation unit 1 can notify the operation control unit 2 of the result of the simulation. The simulation unit 1 can be configured using, for example, a known central processing unit (CPU) generally installed in a computer.

[0028] As shown in FIG. 3, the simulation unit 1 may be communicably connected to the operation input unit 3 and the output unit 4. The operation input unit 3 receives a user's operation input for operating the simulation unit 1 and transmits the content of the user's operation input to the simulation unit 1. By including the operation input unit 3, the operating system WS can execute the simulation by the simulation unit 1 in response to the user's input and can also change the conditions of the simulation by the simulation unit 1. The operation input unit 3 can be embodied by known input means such as a keyboard, a mouse, and a controller. The output unit 4 is configured to output the result of the simulation by the simulation unit 1. The simulation unit 1 can notify the user of the result of the simulation through the output unit 4. The output unit 4 can be embodied by known data output means such as a display, a printer, and a speaker. Note that the simulation information by the simulation unit 1 may not be output to the output unit 4.

[0029] As shown in FIGS. 3 and 4, the simulation unit 1 can operate a virtual moving body VW (in this embodiment, a virtual working device VD and / or a virtual work object VOB1) corresponding to a moving body W (in this embodiment, a working device WD and a work object OB1) in a virtual space VS in which a virtual restriction unit VB1 is arranged. In this specification, the "virtual moving body VW corresponding to the moving body W" includes not only those that are exactly the same or similar in shape and size to the outer shape and size of the actual object moving body W, but also those with a simplified shape of a part of the outer shape of the moving body W and those with slightly different sizes. Further, in the simulation unit 1, the virtual moving body VW is arranged corresponding to the outer shape and position of the moving body W in the real space RS. Further, the virtual restriction unit VB1 in the simulation unit 1 is determined according to the movement path / range of the moving body W for restricting the movement of the moving body W. In this embodiment, a virtual restriction unit VB1 is provided to guide the moving body W to the target position OP1, and the virtual restriction unit VB1 is arranged in a predetermined positional relationship with respect to a virtual target position VOP1 provided in the virtual space VS. The simulation unit 1 can cause the virtual moving body to perform an operation corresponding to the user's operation in the virtual space VS. In the example shown in FIG. 3, on the screen of the display which is the output unit 4, a virtual space VS is displayed in which the virtual moving body VW (virtual working device VD and virtual work object VOB1) is arranged corresponding to the shape and position of the moving body W (working device WD and work object OB1), and the virtual target position VOP1 is arranged corresponding to the shape and position of the target position OP1. Further, in the virtual space VS, the virtual restriction unit VB1 is arranged in a predetermined positional relationship with respect to the virtual target position VOP1. The form information (shape and size) of the above-described virtual moving body VW and virtual restriction unit VB1 and the position information in the virtual space VS with respect to each other are stored in a storage unit (not shown) in advance, and when the actual moving body W operates, the position information of the virtual moving body VW is updated and stored. The position information due to the movement of the actual moving body W is acquired by the measuring device WD4 at a predetermined time interval (for example, within 1 to 10 milliseconds), and each time new position information is acquired, the position information of the virtual moving body VW is updated and stored.

[0030] When an operation command is issued to the working device WD, the simulation unit 1 receives the operation command. In parallel with the operation command, in the virtual space VS, the virtual actuator VW is operated with the content corresponding to the operation command, and the positional relationship between the virtual actuator VW and the virtual restriction unit VB1 is determined at a predetermined time interval T. In the present embodiment, when the working device WD is arbitrarily operated by the user, in parallel with the arbitrary operation, in the virtual space VS, the virtual actuator VW is operated corresponding to the arbitrary operation, and the positional relationship between the virtual actuator VW and the virtual restriction unit VB1 is determined at a predetermined time interval T.

[0031] "In parallel with the operation command" means that when the simulation unit 1 receives an operation command for operating the actuator W (in the present embodiment, when receiving an operation command from the operation device OD), the operation of the actuator W is performed following the operation command from the operation device OD. In other words, the operation (and determination of the positional relationship) of the virtual actuator VW can be performed almost simultaneously and in real time with the operation command, that is, synchronously. Also, "in parallel with an arbitrary operation" means that when an arbitrary operation for operating the actuator W is performed, the operation of the virtual actuator VW is performed following the user's operation. In other words, the operation (and determination of the positional relationship) of the virtual actuator VW can be performed almost simultaneously and in real time with the user's operation, that is, synchronously. In the present embodiment, the operation of the virtual actuator VW corresponding to the user's operation starts in the virtual space VS immediately after the operation (for example, within 1 to 10 milliseconds). By operating the virtual actuator VW in parallel with the user's operation in this way, the user's operation is promptly reflected in the operation of the virtual actuator VW. Therefore, the positional relationship between the virtual actuator VW and the virtual restriction unit VB1 by the simulation unit 1 is instantaneously determined, and based on the positional relationship determined by the simulation unit 1, the operation of the actuator W by the operation control unit 2 can be executed with a short time lag.

[0032] Also, "judging the positional relationship at a predetermined time interval T" means repeatedly judging the positional relationship at the predetermined time interval T a plurality of times. Therefore, for example, when the work device WD is continuously operated for a certain period of time (for example, 10 seconds), it is divided every predetermined time interval T (for example, within 1 to 10 milliseconds), and the positional relationship between the virtual moving body VW and the virtual restricting portion VB1 is judged a plurality of times. For each such time interval T, if the positional relationship between the virtual moving body VW and the virtual restricting portion VB1 is judged in the virtual space VS and there are no problems such as interference even if it operates as it is, the operating body W in the real space RS is operated by the operation control unit 2, while in the simulation unit 1, if it is determined that the operating body W should be stopped or the operating body W should be made to perform a different operation, control is performed to stop the operating body W or make the operating body W perform a different operation according to the determination. Note that the "predetermined time interval T" is not particularly limited, but is a time such that a person does not feel a delay or the like when operating, for example, within 1 to 10 milliseconds.

[0033] The simulation unit 1 is configured to judge whether the distance between the virtual moving body VW and the virtual restricting portion VB1 is within a predetermined range. In the present embodiment, the simulation unit 1 judges whether the virtual moving body VW and the virtual restricting portion VB1 are in contact with each other (whether the distance between the virtual moving body VW and the virtual restricting portion VB1 is zero) based on the distance between the virtual moving body VW and the virtual restricting portion VB1. When it is determined that the virtual moving body VW and the virtual restricting portion VB1 are in contact with each other, and the moving direction D4 (see FIG. 7) of the virtual moving body VW corresponding to the input operation includes a component in the direction toward the surface of the virtual restricting portion VB1 (for example, a component in the direction perpendicular to the surface of the virtual restricting portion VB1) and a component in the direction along the surface of the virtual restricting portion VB1 (for example, a component in the direction parallel to the surface of the virtual restricting portion VB1), the simulation unit 1 is configured to correct the moving direction of the virtual moving body VW corresponding to the input operation based on the shape of the virtual restricting portion VB1.

[0034] The phrase "within a predetermined range of distance" is not particularly limited. For example, as shown in FIG. 6, it may be a state where the virtual actuator is in contact with the virtual restricting portion VB1 (that is, the distance between the virtual actuator and the virtual restricting portion VB1 is 0), or it may be a slightly separated state (the distance between the virtual actuator and the first virtual restricting portion VB1 is equal to or less than a predetermined value). Yes.

[0035] In the example shown in FIG. 6, the surface of the virtual restricting portion VB1 is in contact with the virtual actuator VW (specifically, the virtual work object VOB1 corresponding to the work object OB1).

[0036] Furthermore, when it is determined that the distance between the virtual actuator VW and the virtual restricting portion VB1 is within a predetermined range, as shown in FIG. 7, the simulation unit 1 corrects (track correction) the moving direction D4 of the virtual actuator VW corresponding to the operation command (in this embodiment, the operation information of the components of the work device WD for realizing the operation of the work device WD corresponding to the operation by the user) based on the shape of the virtual restricting portion VB1, thereby obtaining the moving direction D5 of the virtual actuator VW for moving the virtual actuator VW along the surface of the virtual restricting portion VB1 (see FIG. 6), and the virtual actuator VW may be configured to be moved in the obtained moving direction D5. The operation control unit 2 may be configured to move the actuator W in the obtained moving direction D5.

[0037] Also, when it is determined that the distance between the virtual actuator VW and the virtual restricting portion VB1 is within a predetermined range, the simulation unit 1 may be configured to stop the virtual actuator VW with respect to the surface of the virtual restricting portion VB1. The operation control unit 2 may be configured to stop the actuator W with respect to the virtual restricting portion VB1. The meaning of "stopping the virtual actuator VW with respect to the surface of the virtual restricting portion VB1" includes both the case of stopping the virtual actuator VW in a state where the virtual actuator VW is in contact with the surface of the virtual restricting portion VB1 and the case of stopping the virtual actuator VW in a state where the virtual actuator VW is separated from the surface of the virtual restricting portion VB1 to a certain extent.

[0038] Further, when it is determined that the distance between the virtual moving body VW and the virtual restricting unit VB1 is within a predetermined range, the simulation unit 1 may be configured to repel the virtual moving body VW against the surface of the virtual restricting unit VB1. "Repelling the virtual moving body VW against the surface of the virtual restricting unit VB1" means bouncing back the virtual moving body VW against the surface of the virtual restricting unit VB1. "Causing the virtual moving body VW to Repulsion be repelled" includes both the case of repelling the virtual moving body VW after it has contacted the surface of the virtual restricting unit VB1 and the case of repelling the virtual moving body VW at a position where it is separated from the surface of the virtual restricting unit VB1 by a certain distance. When repelling the virtual moving body VW against the surface of the virtual restricting unit VB1, the incident angle and the reflection angle can be set to equal values, for example. The incident angle referred to here means the angle formed by the approaching direction of the virtual moving body VW with respect to the surface of the virtual restricting unit VB1 and the normal line with respect to the surface of the virtual restricting unit VB1. The reflection angle referred to here means the angle formed by the bouncing-back direction of the virtual moving body VW with respect to the surface of the virtual restricting unit VB1 and the normal line with respect to the surface of the virtual restricting unit VB1.

[0039] The virtual moving body VW (specifically, the virtual work object VOB1) is configured to perform a predetermined operation by moving along the surface of the virtual restricting unit VB1 while contacting the virtual restricting unit VB1 or while maintaining a certain distance from the virtual restricting unit VB1. For example, as shown in FIG. 7, the simulation unit 1 corrects the moving direction D4 of the virtual moving body VW based on the form information of the virtual restricting unit VB1 stored in a storage unit (not shown), the operation command received from the operation device OD (specifically, the moving vector F1 corresponding to the moving direction D4 of the virtual moving body VW), and the positional relationship information between the virtual moving body VW and the virtual restricting unit VB1. That is, until the virtual moving body VW contacts the virtual restricting unit VB1, the simulation unit 1 moves the virtual moving body VW in the moving direction D4 based on the input operation information, and when it is determined that the virtual moving body VW has contacted the virtual restricting unit VB1, the simulation unit 1 corrects the trajectory of the virtual moving body VW along the surface shape of the virtual restricting unit VB1.

[0040] More specifically, in the example shown in FIG. 7, based on the movement vector F1 corresponding to the movement direction D4 of the virtual moving body VW (virtual working device VD and virtual work object VOB1) and the normal vector N starting from the collision point P1 when the virtual moving body VW is moved in the movement direction D4 and collides with the virtual restriction unit VB1, the movement vector F1 can be corrected. By this correction, the corrected movement vector F2 is obtained. More specifically, with the end point of the movement vector F1 coinciding with the collision point P1, a straight line parallel to the surface of the virtual restriction unit VB1 at the collision point P1 is drawn from the start point of the movement vector F1, and the intersection point P2 of the straight line and the normal vector N is obtained. The vector extending from the start point of the movement vector F1 to the intersection point P2 is the corrected movement vector F2. Also, the direction in which the corrected movement vector F2 extends is the corrected movement direction D5 of the virtual moving body VW for moving the virtual moving body VW along the surface of the virtual restriction unit VB1. Further, the inner product a (= F1·N) of the movement vector F1 and the normal vector N is obtained, and the corrected movement vector F2 (= F1 + aN) can be obtained by obtaining the sum (F1 + aN) of the movement vector F1 and the vector (aN) obtained by multiplying the normal vector N by the inner product a. The simulation unit 1 obtains the corrected movement vector F2 at predetermined time intervals. By moving the virtual moving body VW with the obtained movement vector F2, the virtual moving body VW can be moved along the surface while being in contact with the surface of the virtual restriction unit VB1. Note that the method of correcting (trajectory correction) the movement direction D4 of the virtual moving body VW based on the surface shape of the virtual restriction unit VB1 is not limited to the above-described content and may be by other methods.

[0041] The simulation unit 1 can move the virtual actuator VW along the surface of the first virtual restriction unit VB1 while bringing the virtual actuator into contact with the surface of the first virtual restriction unit VB1 by moving the virtual actuator VW in the corrected movement direction D5 obtained by the simulation unit 1. The operation control unit 2 can move the actuator W along the surface of the first virtual restriction unit VB1 by moving the actuator W in the corrected movement direction D5 obtained by the simulation unit 1. Note that the surface of the virtual restriction unit VB1 may be a curved surface or a flat surface. Even when the surface is a curved surface or a flat surface, if the movement direction D4 of the virtual actuator VW by the user's operation includes a component in the direction toward the surface of the virtual restriction unit VB1 and a component in the direction along the surface of the virtual restriction unit VB1, the simulation unit 1 can obtain the corrected movement direction D5 (in this embodiment, the movement vector F2) by orbit correction (correction of the movement direction) as shown in FIG. 6. Therefore, the user can move the virtual actuator VW along the surface of the virtual restriction unit VB1 while bringing the virtual actuator into contact with the surface of the virtual restriction unit VB1 by simply performing a rough operation including a component toward the surface of the virtual restriction unit VB1 and a component along the surface.

[0042] In the example shown in FIG. 6, the virtual restriction unit VB1 is configured to guide a virtual actuator VW corresponding to the actuator W (see FIG. 5) to a virtual target position VOP1 corresponding to a predetermined target position OP1 (see FIG. 5). In this specification, the "predetermined target position OP1" is a position corresponding to the purpose of the operation of the working device WD and is a position corresponding to the destination of movement of the actuator W. In the example shown in FIG. 1, the purpose of the operation of the working device WD is to move the work object OB1 to a predetermined position (for example, an opening provided on the floor surface or the like (see FIG. 5)) OP1 and arrange it at the predetermined position OP1. In the example shown in FIG. 6, the virtual restriction unit VB1 has a virtual guide surface VB11 configured such that the movement range of the virtual actuator VW becomes narrower as it approaches the virtual target position VOP1 corresponding to the predetermined target position OP1. Since the virtual guide surface VB11 is configured such that the movement range of the virtual actuator VW becomes narrower as it approaches the virtual target position VOP1, the virtual actuator VW can be smoothly guided toward the virtual target position VOP1. In FIG. 5, for convenience of explanation, only the work object OB1 among the actuators W is shown, and the working device WD (finger WD31) is not shown. Similarly, in FIGS. 6 and 8, for convenience of explanation, only the virtual work object VOB1 among the virtual actuators VW is shown, and the virtual working device VD (virtual finger VD31) is not shown. Even when the virtual working device VD (virtual finger VD31) among the virtual actuators VW contacts the virtual restriction unit VB1, the same correction as described above may be performed. Alternatively, the simulation unit 1 and the operation control unit 2 may be configured such that only the virtual work object VOB1 among the virtual actuators VW has its movement direction corrected by the virtual restriction unit VB1.

[0043] In this embodiment, the virtual guide surface VB11 is formed in a conical shape with the inner space narrowing as it approaches the virtual target position VOP1, and the bottom BO (the end with the larger diameter in the axial direction of the cone) and the top TO (the end with the smaller diameter in the axial direction of the cone) are open. That is, the virtual restricting member VB1 has a conical side wall SW with the bottom BO and the top TO open. The type of the cone is not particularly limited, but in the example shown in FIG. 6, it is a circular cone (frustum of a cone). The simulation unit 1 obtains the moving direction D5 of the virtual work object VOB1 for moving the virtual moving body VW from the bottom BO side to the top TO side along the inner surface of the side wall SW of the virtual restricting unit VB1, and moves the virtual work object VOB1 in the obtained moving direction D5 so as to discharge the virtual work object VOB1 from the opening of the top TO. The operation control unit 2 is configured to move the work object OB1 (see FIG. 5) in the obtained direction D5, thereby moving the work object OB1 to the target position OP1. The opening of the top TO is set to a size that allows the portion of the moving body W that is desired to be disposed at the target position OP1 (the work object OB1 in the example shown in FIG. 1) to easily pass through. The "size that allows easy passage" is a size in which a predetermined clearance is provided in the radial direction between the virtual work object VOB1 so that the user can easily pass through even with a rough operation on the operation device OD. In the example shown in FIG. 6, the diameter of the opening of the top TO is larger than the diameter of the columnar work object OB1. The virtual guide surface VB11 is preferably set to have an inclination angle (inclination angle with respect to the vertical direction) such that the virtual work object VOB1 is smoothly guided to the virtual target position VOP1. Such an inclination angle is set according to the respective shapes and sizes of the virtual work object VOB1 and the bottom BO. If the inclination angle of the virtual guide surface VB11 is too small (the diameter of the bottom OB becomes close to the diameter of the top TO), the opening of the bottom BO becomes small, and it may be difficult to insert the virtual work object VOB1 into the opening of the bottom BO. If the inclination angle of the virtual guide surface VB11 is too large (the diameter of the bottom OB is too large with respect to the diameter of the top TO), the posture of the work object OB1 may be close to a lying-down posture.When the posture of the work object OB1 is close to a lying-down posture, it may not be possible to smoothly guide the work object OB1 to the target position OP1. Note that the side wall SW of the virtual regulating member VB1 may have a substantially constant inclination angle (the inclination angle with respect to the vertical direction) between the bottom BO and the top TO (see FIG. 6), or may be configured such that the inclination angle (the inclination angle with respect to the vertical direction) gradually decreases (convex inward) as approaching from the bottom BO to the top TO.

[0044] In the example shown in FIG. 6, the simulation unit 1 obtains the moving direction D5 of the virtual moving body VW (virtual work object VOB1) for moving the virtual moving body VW from the bottom BO side to the top TO side along the virtual guide surface VB11 of the virtual restricting unit VB1, and the operation control unit 2 is configured to move the moving body (work object OB1) in the obtained moving direction D5. The moving body W moves along the inclination direction of the virtual guide surface VB11 in the real space RS. When the simulation unit 1 determines that the virtual moving body has reached the top TO or has exceeded the top TO, the simulation unit 1 stops the movement of the virtual moving body VW. The operation control unit 2 stops the movement of the moving body W (work object OB1, see FIG. 5) based on the result of the simulation. After the moving body W stops, the user can perform an operation to release the work object OB1. When the user performs this operation, the simulation unit 1 releases the virtual work object VOB1 (see FIG. 6). Thereby, the virtual work object VOB1 can be arranged at the virtual target position VOP1. Note that when the simulation unit 1 determines that the virtual moving body VW has reached the top TO or has exceeded the top TO, the simulation unit 1 may be configured to stop the movement of the virtual work object VOB1 and adjust the posture of the virtual work object VOB1. Specifically, the simulation unit 1 is configured to cause the virtual work device VD to adjust the posture of the virtual work object VOB1 so that the posture of the virtual work object VOB1 becomes a posture suitable for arrangement at the virtual target position VOP1. The "posture suitable for arrangement at the virtual target position VOP1" is, for example, a posture in which the virtual work object VOB1 can stand independently at the virtual target position VOP1 (a posture that does not fall over). Also, when a hole (a cylindrical portion shown by a broken line in FIG. 6) is provided at the virtual target position VOP1, it is a posture in which the virtual work object VOB1 can be inserted into the hole. Note that the virtual target position VOP1 may be a hole portion (see FIG. 6) into which the virtual work object VOB1 is inserted, or may be a surface (for example, a flat surface) on which the virtual work object VOB1 is arranged.

[0045] In the example shown in FIG. 6, the operating system WS is configured to perform an operation of inserting (including fitting) the work object OB1 (see FIG. 5) into a hole provided at a predetermined position OP1 using the virtual control unit VB1. Specifically, the hole is formed on the upper surface of the member FL arranged in the real space RS (see FIG. 5). The measuring device WD4 measures the position of the hole in the member FL based on the captured image. The position measured by the measuring device WD4 is associated with the shape and position of the hole and stored in a storage unit (not shown). Note that the shape and position of the hole (target position OP1) provided in the member FL may be stored in advance in association with the position of the working device WD. As shown in FIG. 6, the simulation unit 1 arranges a virtual member VFL corresponding to the member FL in the virtual space VS. As shown in FIG. 6, the simulation unit 1 provides the virtual control unit VB1 in the virtual space VS such that the position of the top TO coincides with the position of the opening of the hole (virtual target position VOP1). The simulation unit 1 obtains a moving direction D5 of the virtual moving body VW for moving the virtual moving body VW (virtual work object VOB1) from the bottom BO side to the top TO side along the virtual guide surface VB11 of the virtual control unit VB1. The operation control unit 2 moves the moving body W (work object OB1) in the obtained moving direction D5. The moving body W moves along the virtual guide surface VB11. When the simulation unit 1 determines that the virtual moving body VW has reached the top TO, the simulation unit 1 stops the movement of the virtual moving body VW.

[0046] When the virtual moving body VW reaches the top TO, the center line CL2 of the virtual workpiece VOB1 may be inclined with respect to the center line CL1 of the hole (virtual target position VOP1) (see Fig. 8). In such a case, the user can perform an operation of inserting the tip of the workpiece OB1 into the hole and changing the inclination angle of the workpiece OB1 (rocking the workpiece OB1) by operating the operating device OD. When the user performs this operation, based on the user's operation, the simulation unit 1 inserts the tip of the virtual workpiece VOB1 into the hole and changes the inclination angle of the virtual workpiece VOB1 (rocks the virtual workpiece VOB1) while inserting the virtual workpiece VOB1 into the hole. By changing the inclination angle of the virtual workpiece VOB1, the posture of the virtual workpiece VOB1 can be changed to a posture that can be inserted into the hole. The motion control unit 2 inserts the tip of the workpiece OB1 into the hole based on the result of the simulation and changes the inclination angle of the workpiece OB1 (rocks the workpiece OB1) while inserting the workpiece OB1 into the hole. By changing the inclination angle of the workpiece OB1, the inclination angle of the workpiece OB1 (the inclination angle may be zero or a value greater than zero) can be adjusted to an angle that can be inserted into the hole.

[0047] After the tip of the work object OB1 is inserted into the hole, the user can perform an operation of inserting the work object OB1 to a predetermined depth in the hole. Specifically, for example, the user can perform an operation of inserting the work object OB1 to a predetermined depth in the hole (target position OP1) with the center line of the work object OB1 inclined with respect to the center line of the hole (the inclination angle is greater than zero) (see FIG. 9). When the user performs this operation, the simulation unit 1 inserts the virtual work object VOB1 to a predetermined depth in the hole with the virtual work object VOB1 inclined with respect to the hole. In addition, for example, the user can perform an operation of moving the inserted portion of the work object OB1 to one side in the radial direction of the hole (target position OP1) and pressing the side surface of the inserted portion against the inner peripheral surface of the hole along the length direction of the hole (see FIG. 10). Thereby, an operation of inserting the work object OB1 to a predetermined depth in the hole can be performed with the work object OB1 not inclined with respect to the hole (the inclination angle is substantially zero). When the user performs this operation, the simulation unit 1 moves the inserted portion of the virtual work object VOB1 to one side in the radial direction of the hole and changes the posture of the virtual work object VOB1 so as to press the side surface of the inserted portion against the inner peripheral surface of the hole along the length direction of the hole. Further, the simulation unit 1 inserts the virtual work object VOB1 to a predetermined depth in the hole with the work object OB1 not inclined with respect to the hole.

[0048] Note that the simulation unit 1 may be configured to adjust the orientation of the virtual work object VOB1 so that the center line CL2 substantially coincides with the center line CL1. By adjusting the orientation of the virtual work object VOB1 by the simulation unit 1, the work object OB1 can be more easily inserted into the hole (target position OP1). Note that even if the center line CL2 of the virtual work object VOB1 is inclined with respect to the center line CL1 of the hole (virtual target position VOP1), when the opening of the hole (virtual target position VOP1) is large enough to allow the virtual work object VOB1 to be inserted as it is, the simulation unit 1 may not need to adjust the orientation of the virtual work object VOB1.

[0049] The operation system WS according to this embodiment simulates the positional relationship between an operating body VW (a working device WD and a work object OB1) that performs a predetermined operation in response to an input operation from an operating device OD by a user, and a virtual restriction unit VB1 that is virtually provided to restrict the movement of the operating body VW, and corrects the moving direction of the operating body VW according to the result of the simulation. Thereby, even if there is a possibility that the working device WD may perform an unexpected operation along with an arbitrary operation by the user in the real space RS, since the operating body W is guided to the target position OP1, the work of the working device WD (for example, conveyance of the work object OB1, etc.) can be performed smoothly. Also, if necessary, a stop or a repulsion operation of the operating body W is performed. Therefore, even when the user cannot perform a fine operation with the operating device OD, the user can easily perform the necessary operation on the work object OB1 with the working device WD. In particular, when the user remotely operates the working device WD while visually recognizing the operation of the working device WD with the operating device OD, conventionally, it has been necessary to perform the operation while finely confirming the positional relationship between the working device WD and the object OB from a distance, which may impose a large burden on the user. However, in the operation system WS according to this embodiment, since the operation of the working device WD can be appropriately controlled even with a rough operation, it is possible to protect the facility and perform a sophisticated trajectory operation of the working device WD while reducing the burden on the user.

[0050] Note that, as shown in FIG. 11 for example, the operation system WS may be configured to display on the screen of the display which is the output unit 4, the positional relationship between the virtual operating body VW (the virtual object to be worked on VOB1 in the example shown in FIG. 11) and the virtual target position VOP1 which is a hole portion. In the example shown in FIG. 11, the positional relationship between the virtual object to be worked on VOB1 and the virtual target position VOP1 is displayed on the screen of the display. In FIG. 11, in order from the right side to the left side, the state of the virtual object to be worked on VOB1 moving is shown. As the virtual object to be worked on VOB1 moves from the right side to the left side in FIG. 11, it gradually approaches the virtual position VOP1. By displaying the positional relationship between the virtual operating body VW and the virtual target position VOP1 on the screen of the display, the user can visually recognize the positional relationship on the screen. Thereby, the user can operate the operation device OD (refer to FIG. 1) based on the visually recognized positional relationship, and cause the working device WD to perform a predetermined work. Therefore, compared with the case where the user operates the operation device OD while directly viewing the working device WD and the object to be worked on OB1, the user can cause the working device WD to perform accurate work and can easily perform the operation. Also, if the object to be worked on OB1 and the target position OP1 imaged by a camera or the like are displayed on the display, part or all of the visual field of the camera or the like with respect to the object to be worked on OB1 and the target position OP1 may be blocked by the working device WD or the like. In this case, even if the object to be worked on OB1 and the target position OP1 are displayed on the display, the user may not be able to sufficiently grasp the positional relationship between the object to be worked on OB1 and the target position OP1. On the other hand, when the positional relationship between the virtual object to be worked on VOB1 and the virtual target position VOP1 is displayed on the screen of the display, the virtual object to be worked on VOB1 and the virtual target position VOP1 can be displayed so that the visual field with respect to the virtual object to be worked on VOB1 and the virtual target position VOP1 is not blocked by the virtual working device VD or the like. Therefore, it becomes easier for the user to grasp the positional relationship between the object to be worked on OB1 and the target position OP1. Thereby, the user can easily perform the operation of causing the working device WD to perform accurate work.

[0051] Note that, as shown in FIGS. 12 to 14 for example, the operation system WS may be configured to display, on the screen of the display which is the output unit 4, the positional relationship between a predetermined portion of the virtual work object VOB1 (in the examples shown in FIGS. 12 to 14, the tip portion (specifically, the tip surface) and the virtual target position VOP1). In the examples shown in FIGS. 12 to 14, a state of viewing the tip surface of the virtual work object VOB1 and the virtual target position VOP1 from the axial direction D6 (see FIG. 11) of the virtual target position VOP1 is displayed. The virtual work objects VOB1 in FIGS. 12 to 14 are respectively shown in a state of viewing the virtual work objects VOB1 located on the right side, the center, and the left side in FIG. 11 from the axial direction D6 of the virtual target position VOP1. In the examples shown in FIGS. 12 to 14, by the operation of the user, the simulation unit 1 moves the virtual work object VOB1 so as to gradually reduce the inclination angle of the virtual work object VOB1 with respect to the virtual target position VOP1 while gradually approaching the virtual work object VOB1 to the virtual target position VOP1. The display displays the movement of the virtual work object VOB1.

[0052] In addition, in the present embodiment, the simulation unit 1 can be configured to determine whether the distance between a predetermined portion of the virtual actuator VW and the first virtual restriction unit VB1 is within a predetermined range (for example, whether the predetermined portion and the first virtual restriction unit VB1 are in contact), and not to determine whether the distance between another portion (a portion other than the predetermined portion) of the virtual actuator VW and the first virtual restriction unit VB1 is within a predetermined range. That is, the object for which the simulation unit 1 determines the distance from the first virtual restriction unit VB1 can be limited to a predetermined portion of the virtual actuator VW. In this case, since the "other portion" is not an object for distance determination, for example, even when the "other portion" is in a positional relationship such that it contacts the first virtual restriction unit VB1, that positional relationship does not participate in the processing of the simulation unit 1 and the operation control unit 2. Therefore, the simulation unit 1 can omit the determination of the positional relationship with low relevance to the work purpose and perform the position determination with high relevance to the work purpose. Thereby, the simulation unit 1 can appropriately and promptly determine the positional relationship according to the work purpose. The "predetermined portion of the virtual actuator VW" is a portion highly relevant to the work purpose of the operation system WS. Specifically, for example, the "predetermined portion of the virtual actuator VW" is part or all of the virtual work object VOB1 (that is, the virtual work device VD is not an object for position determination). In the examples shown in FIGS. 11 to 14, the "predetermined portion of the virtual actuator VW" is the tip or the tip surface of the virtual work object VOB1. The simulation unit 1 determines whether the distance between the tip or the tip surface of the virtual work object VOB1 and the first virtual restriction unit VB1 is within a predetermined range (for example, whether the tip or the tip surface and the first virtual restriction unit VB1 are in contact), and does not determine whether the distance between another portion (a portion other than the predetermined portion) of the virtual actuator VW and the first virtual restriction unit VB1 is within a predetermined range. Thereby, it is possible to appropriately and promptly perform the simulation for achieving the work purpose of arranging the work object OB1 at the target position OP1.That is, for example, when the virtual working device VD contacts the virtual restriction unit VB1, it is possible to prevent the situation where the movement of the virtual working device VD is restricted by the virtual restriction unit VB1 and the guidance of the virtual working object VOB1 by the virtual restriction unit VB1 is hindered.

[0053] In this embodiment, as shown in FIG. 15, the virtual restriction unit VB1 may be configured such that a virtual moving body (only the virtual working object VOB1 is shown in FIG. 15) can pass from the side opposite to the virtual guidance surface VB11 to the virtual guidance surface VB11 side of the virtual guidance surface VB11, and the virtual moving body cannot pass from the virtual guidance surface VB11 side to the side opposite to the virtual guidance surface VB11. Specifically, as shown in FIG. 15, the virtual restriction unit VB1 may be configured such that the virtual moving body can pass from the outside to the inside of the virtual restriction member VB1 through the virtual guidance surface VB11, and the virtual moving body cannot pass from the inside to the outside of the virtual restriction member VB1 through the virtual guidance surface VB11. With such a configuration, the virtual working object VOB1 can be placed into the virtual restriction unit VB1 from a portion other than the bottom BO of the virtual restriction unit VB1. Therefore, the user can more easily place the virtual working object VOB1 into the virtual restriction unit VB1, and once the virtual working object VOB1 enters the virtual restriction unit VB1, it cannot exit the virtual restriction unit VB1. Thereby, the operation efficiency when the user places the virtual working object VOB1 into the virtual restriction unit VB1 can be improved.

[0054] Also, as shown in FIG. 16, the virtual restriction part VB1 may be formed in a pyramid shape with the bottom BO and the top TO open. The type of the pyramid of the virtual restriction part VB1 can be set according to the shape of the virtual work object VOB1. Since the virtual work object VOB1 shown in FIG. 16 is formed in a quadrangular prism shape, the virtual restriction part VB1 is formed in a quadrangular pyramid shape. Also, the shape of the opening of the bottom OB and the shape of the opening of the top TO can be set according to the shape of the virtual work object VOB1. Since the virtual work object VOB1 shown in FIG. 16 is formed in a quadrangular prism shape, the shape of the opening of the bottom OB and the shape of the opening of the top TO are formed in a quadrangular shape. The opening of the top TO is set to a size that allows the part (the work object OB1 in the example shown in FIG. 1) to be arranged at the target position OP1 in the actuator W to easily pass through. By making the shape of the side wall WS conform to the shape of the virtual work object VOB1, it becomes easier to guide the virtual work object VOB1 in a stable posture. Also, by making the shapes of the openings of the top TO and the bottom BO conform to the shape of the virtual work object VOB1, it becomes easier to take the virtual work object VOB1 in and out of the virtual restriction part VB1.

[0055] Next, referring to FIGS. 17 to 26, another embodiment of the operating system will be described. In the embodiment shown in FIGS. 17 to 26, the operating system is configured to perform an operation of inserting (including fitting) a work object into a hole provided at a predetermined target position using a virtual restriction unit VB1. The virtual restriction unit VB1 is configured to guide a virtual operating body VW to a hole which is a virtual target position VOP1 (hereinafter also referred to as the hole VOP1). In the present embodiment, the virtual restriction unit VB1 is configured to guide a virtual work object VOB1, which is an insertion target part of the virtual operating body VW, to the hole VOP1. In the example shown in FIG. 17, the virtual work object VOB1 is a columnar object such as a cylindrical or prismatic object, but is not limited thereto. Also, in the example shown in FIG. 17, the hole VOP1 is a hole having a columnar space such as a cylindrical or prismatic space, but is not limited thereto. The hole VOP1 may have a shape and size into which the virtual work object VOB1 can be inserted. The diameter of the hole VOP1 is larger than the diameter of the work object OB1. As will be described later, in the present embodiment, even if the difference between the diameter of the hole VOP1 and the diameter of the work object OB1 is slight, it is possible to easily insert the work object OB1 into the hole VOP1.

[0056] In this embodiment, the virtual regulation unit VB1 has a guide regulation unit VBG (see FIGS. 17 and 18). Further, in this embodiment, the virtual regulation unit VB1 further has a swing regulation unit VBS (see FIG. 22). The guide regulation unit VBG is a part that guides the insertion target portion VOB1 of the virtual actuator VW (hereinafter also referred to as the insertion target portion VOB1) such that a part of the tip of the insertion target portion VOB1 is inserted into the hole portion VOP1 and the center line CL2 of the insertion target portion VOB1 is inclined with respect to the depth direction of the hole portion VOP1 (see FIGS. 18 to 20). In this embodiment, the depth direction of the hole portion VOP1 is the direction along the center line CL1 of the hole portion VOP1. The depth direction of the hole portion VOP1 is substantially parallel to the center line CL1 of the hole portion VOP1. The "part of the tip of the insertion target portion VOB1" is, as shown in FIG. 20, a part of the region closer to the hole portion VOP1 at the end in the length direction (the direction along the center line CL2) of the insertion target portion VOB1. Specifically, for example, assume a columnar insertion target portion VOB1 in a posture where the center line CL2 is inclined with respect to the center line CL1 of the hole portion VOP1 (see FIG. 20). In this case, the "part of the tip of the insertion target portion VOB1" is the region closer to the hole portion VOP1 (a part in the circumferential direction of the circular edge) at one end of the insertion target portion (cylindrical body) VOB1. The configuration of the guide regulation unit VBG is not particularly limited as long as it guides the insertion target portion VOB1 such that a part of the tip of the insertion target portion VOB1 of the virtual actuator VW is inserted into the hole portion VOP1 and the center line CL2 of the insertion target portion VOB1 is inclined with respect to the depth direction of the hole portion VOP1. In the example shown in FIG. 17, the guide regulation unit VBG is configured in a substantially V-shaped cross section formed by combining two flat portions. The intersection line of the two flat portions is inclined with respect to the depth direction of the hole portion VOP1 (the direction along the center line CL1). Further, in this embodiment, the guide regulation unit VBG is configured such that a part of the tip of the insertion target portion VOB1 is inserted inside the peripheral edge PE of the opening of the hole portion VOP1 (see FIG. 20). Further, in this embodiment, the guide regulation unit VBG is configured to guide the insertion target portion VOB1 to the hole portion VOP1 such that the center line CL2 of the insertion target portion VOB1 and the center line CL1 of the hole portion VOP1 intersect or are close to each other.As a result, the guide restriction part VBG makes it easier to insert a part of the tip of the insertion target part VOB1 inside the peripheral edge PE of the opening of the hole part VOP1. Further, in the present embodiment, the simulation part 1 is configured to generate the guide restriction part VBG when the distance between the insertion target part VOB1 and the hole part VOP1 becomes equal to or less than a predetermined distance. Further, the simulation part 1 is configured to eliminate the guide restriction part VBG after a part of the tip of the insertion target part VOB1 is inserted inside the peripheral edge PE of the opening of the hole part VOP1.

[0057] The swing restricting portion VBS (see FIG. 22) is configured to restrict the swing direction D9 (see FIG. 23) of the insertion target portion VOB1 around the contact position with the peripheral edge PE of the opening portion in a state where a part of the tip is inserted into the hole portion VOP1 and the insertion target portion VOB1 contacts the peripheral edge PE of the opening portion. Specifically, the swing restricting portion VBS is configured to restrict the swing direction D9 so that the inclination angle θ of the center line CL2 of the insertion target portion VOB1 with respect to the depth direction of the hole portion VOP1 becomes smaller. In the example shown in FIGS. 21 and 23, the swing restricting portion VBS restricts the swing direction D9 so that the inclination angle θ shown in FIG. 23 is smaller than the inclination angle θ shown in FIG. 21. In the state shown in FIG. 21, the side surface (outer peripheral surface) of the insertion target portion VOB1 contacts the peripheral edge PE of the opening portion at one place (one contact position CP). In this state, the insertion target portion VOB1 can swing in the direction in which the inclination angle θ becomes smaller. When the insertion target portion VOB1 swings in the swing direction D9, it transitions to the state shown in FIG. 23. The configuration of the swing restricting portion VBS is not particularly limited as long as it can restrict the swing direction D9 of the insertion target portion VOB1 around the contact position with the peripheral edge PE of the opening portion in a state where a part of the tip is inserted into the hole portion VOP1 and the insertion target portion VOB1 contacts the peripheral edge PE of the opening portion. In the example shown in FIG. 22, the swing restricting portion VBS is composed of two flat plate portions provided so as to sandwich the insertion target portion VOB1 from both sides in the radial direction. The two flat plate portions are arranged parallel to each other. The interval between the two flat plate portions is substantially the same as the diameter of the insertion target portion VOB1. The two flat plate portions are provided so as to be parallel to the center line CL1 of the hole portion VOP1 and the center line CL2 of the insertion target portion VOB1. Thereby, the swing restricting portion VBS can guide the insertion target portion VOB1 so that the insertion target portion VOB1 swings in a direction in which the center line CL2 of the insertion target portion VOB1 becomes substantially parallel to the center line CL1 of the hole portion VOP1 (the direction in which the inclination angle θ described later becomes smaller).

[0058] In the state shown in FIG. 23, the insertion target portion VOB1 is in contact with the peripheral edge PE of the opening at three locations (three contact positions CP). In the example shown in FIG. 23, the side surface of the insertion target portion VOB1 is in contact with the peripheral edge PE of the opening at one contact position CP (hereinafter also referred to as the first contact position CP1). Further, in the present embodiment, the peripheral edge of one end portion (the end portion on the hole portion VOP1 side) of the insertion target portion VOB1 is in contact with the peripheral edge PE of the opening at two contact positions CP (hereinafter also referred to as the second contact positions CP2). The two contact positions CP (the two second contact positions CP2) are spaced apart from each other in the circumferential direction at the peripheral edge of one end portion of the insertion target portion VOB1. In FIG. 23, among the two second contact positions CP2, the second contact position CP2 on the front side in the direction perpendicular to the paper surface is shown. In the state where the insertion target portion VOB1 and the peripheral edge PE of the opening are in contact at three locations (see FIG. 23), the insertion target portion VOB1 cannot swing in the direction in which the tilt angle θ becomes smaller. Further, in this state, the insertion target portion VOB1 cannot be inserted deeper into the hole portion VOP1. In order to insert the insertion target portion VOB1 deeper into the hole portion VOP1, it is necessary to make the tilt angle θ of the insertion target portion VOB1 smaller. In order to make the tilt angle θ of the insertion target portion VOB1 smaller, it may be moved in the direction D10 of pulling out from the hole portion VOP1 (see FIG. 24) (however, it is not completely pulled out). When the insertion target portion VOB1 is moved in the direction D10 of pulling out from the hole portion VOP1, the insertion target portion VOB1 transitions to a state of being in contact with the peripheral edge PE of the opening at one location (one contact position CP) (see FIG. 24).

[0059] In the state shown in FIG. 24, the insertion target portion VOB1 can swing in the direction in which the tilt angle θ becomes smaller (see FIG. 25). By swinging the insertion target portion VOB1 in the swing direction D9, the state shown in FIG. 25 is transitioned to. The tilt angle θ shown in FIG. 25 is smaller than the tilt angle θ shown in FIG. 23. This means that the posture of the insertion target portion VOB1 shown in FIG. 25 is closer to the posture that allows insertion into the hole portion VOP1 than the posture of the insertion target portion VOB1 shown in FIG. 23.

[0060] Specifically, when the inclination angle θ is an inclination angle of a certain degree or more, it becomes difficult to insert the entire tip of the insertion target portion VOB1 (for example, the entire circumferential direction of a circular edge) into the hole portion VOP1 with the side surface (outer circumferential surface) of the insertion target portion VOB1 in contact with the peripheral edge PE of the opening. The inclination angle at that time can be called the critical angle θa. The critical angle θa can be expressed by the following formula (1). Here, R is the radius of the hole portion VOP1, and r is the radius of the insertion target portion VOB1. cos(θa)=r / R ···(1)

[0061] When the insertion target portion VOB1 is in a posture in which it can be inserted into the hole portion VOP1 (hereinafter, also referred to as an insertable posture), the following formula (2) holds. The insertable posture is a posture in which the entire tip of the insertion target portion VOB1 can be inserted into the hole portion VOP1 with the outer circumferential surface of the insertion target portion VOB1 in contact with the peripheral edge PE of the opening (see Fig. 26). θ<θa ···(2)

[0062] The inclination angle θ shown in Fig. 25 is larger than the critical angle θa. However, the inclination angle θ shown in Fig. 25 is closer to the critical angle θa than the inclination angle θ shown in Fig. 23. In the state shown in Fig. 25, the insertion target portion VOB1 is in contact with the peripheral edge PE of the opening at three locations (three contact positions CP). In Fig. 25, the two second contact positions CP2 (only one second contact position CP2 is shown in Fig. 25) are separated from each other in the circumferential direction at the peripheral edge of one end of the insertion target portion VOB1. However, the distance between the first contact position CP1 and the second contact position CP2 in Fig. 25 is smaller than the distance between the first contact position CP1 and the second contact position CP2 in Fig. 23. In the state of Fig. 25 where the insertion target portion VOB1 and the peripheral edge PE of the opening are in contact at three locations, the insertion target portion VOB1 cannot swing in the direction in which the inclination angle θ decreases. Therefore, the insertion target portion VOB1 is moved again in the direction D10 of pulling it out from the hole portion VOP1.

[0063] Thereafter, repeat the same procedure as shown in FIGS. 23 to 25. Then, at a certain point in time, the entire tip of the insertion target portion VOB1 (for example, the entire circumferential direction of the circular edge) is inserted into the hole portion VOP1 (see FIG. 26). In the state shown in FIG. 26, the inclination angle θ satisfies the above formula (2). In the state shown in FIG. 26, the side surface of the insertion target portion VOB1 contacts the peripheral edge PE of the opening at one location (one first contact position CP1), and a part of the tip of the insertion target portion VOB1 contacts the inner peripheral surface of the hole portion VOP1 at the third contact position CP3. Thereby, by adjusting the posture of the insertion target portion VOB1 within the hole portion VOP1, it becomes possible to insert the insertion target portion VOB1 deeper. For example, with the contact position between the tip of the insertion target portion VOB1 and the inner peripheral surface of the hole portion VOP1 (the third contact position CP3 in FIG. 26) as the center, the insertion target portion VOB1 is swung so that the center line CL2 is substantially parallel to the center line CL1 of the hole portion VOP1. As a result, since the center line CL2 is substantially parallel to the center line CL1 of the hole portion VOP1, the insertion target portion VOB1 can be inserted deeper (it can also be inserted completely).

[0064] The user can perform an operation to bring the tip of the insertion target part closer to the hole part in order to insert the insertion target part in the operating body into the hole part by operating the operating device OD. When the user performs this operation, the simulation unit 1 generates a guide restriction unit VBG when the distance between the insertion target part VOB1 and the hole part VOP1 in the virtual operating body VW becomes equal to or less than a predetermined distance as shown in FIG. 17 based on the user's operation. The simulation unit 1 obtains the moving direction D5 (see FIG. 18) of the insertion target part VOB1 for moving the insertion target part VOB1 along the virtual guide surface VB11 of the guide restriction unit VBG in the direction approaching the hole part VOP1. The motion control unit 2 moves the insertion target part VOB1 in the obtained moving direction D5 (see FIG. 18). The motion control unit 2 moves the insertion target part in the operating body along the inclination direction of the virtual guide surface VB11 based on the result of the simulation. When the simulation unit 1 determines that a part of the tip of the insertion target part VOB1 has been inserted into the hole part VOP1, it stops the movement of the virtual operating body VW (see FIG. 20). The motion control unit 2 stops the movement of the operating body based on the result of the simulation. After the operating body stops, the user can perform an operation to move the insertion target part on the inclined side of the insertion target part and in the direction approaching the peripheral edge PE of the opening (moving direction D11, see FIG. 21). When the user performs this operation, the simulation unit 1 moves the insertion target part VOB1 in the moving direction D11 based on this operation. Thereby, the side surface (outer peripheral surface) of the insertion target part VOB1 can be brought into contact with the peripheral edge PE of the opening (see FIG. 21). The simulation unit 1 can determine that the side surface (outer peripheral surface) of the insertion target part VOB1 has come into contact with the peripheral edge PE of the opening based on the distance between the insertion target part VOB1 and the peripheral edge PE of the opening. In the example shown in FIG. 21, it is preferable that the first contact position CP1, the center line CL1 of the hole part VOP1, and the center line CL2 of the insertion target part VOB1 are on substantially the same plane. In this case, it is easy to swing the insertion target part VOB1 in the direction in which the center line CL2 of the insertion target part VOB1 becomes substantially parallel to the center line CL1 of the hole part VOP1 (the direction in which the inclination angle θ described later becomes smaller) (see the swinging direction D9 in FIG. 23).Note that in this embodiment, the operating system WS may include a contact detection unit (not shown) that detects contact between the insertion target part in the real space and the peripheral edge of the opening of the hole part. The configuration of the contact detection unit is not particularly limited as long as it can detect contact between the insertion target part and the peripheral edge of the opening of the hole part. Specifically, the contact detection unit may be, for example, a force sensor, or a device that detects energization between the insertion target part and the peripheral edge of the opening when they come into contact. The device for detecting energization can be configured to detect, for example, a change in the potential difference between the insertion target part and the peripheral edge of the opening.

[0065] When the simulation unit 1 determines that the insertion target part VOB1 and the peripheral edge of the opening PE in the virtual space VS are in contact (see FIG. 21), or when the contact detection unit detects contact between the insertion target part in the real space and the peripheral edge of the opening, the movement of the virtual actuator VW (insertion target part VOB1) is stopped. The operation control unit 2 stops the movement of the actuator (insertion target part) based on the result of the simulation.

[0066] After the movement of the actuator stops, the simulation unit 1 generates a swing restriction unit VBS (see FIG. 22). After the swing restriction unit VBS is generated, the user can perform an operation to swing the insertion target part toward the central axis side of the hole part. When the user performs this operation, the simulation unit 1 determines the moving direction (swing direction D9, see FIG. 23) of the insertion target part VOB1 for swinging the insertion target part VOB1 along the virtual guide surface VB11 of the swing restriction unit VBS toward the central axis CL1 side of the hole part VOP1. The operation control unit 2 moves (swings) the insertion target part VOB1 in the obtained moving direction (swing direction D9) (see FIG. 23). The operation control unit 2 moves (swings) the insertion target part in the actuator along the virtual guide surface VB11 toward the central axis side of the hole part based on the result of the simulation.

[0067] When the simulation unit 1 determines that the tip of the insertion target part VOB1 contacts the peripheral edge PE of the opening of the hole part VOP1 based on the positional relationship between the tip of the insertion target part VOB1 and the peripheral edge PE of the opening of the hole part VOP1, it determines that the insertion target part VOB1 contacts the peripheral edge PE of the opening of the hole part VOP1 at three positions (see FIG. 23). When the simulation unit 1 determines that the insertion target part VOB1 contacts the peripheral edge PE of the opening of the hole part VOP1 at three positions, it stops the movement (swing) of the virtual actuator VW (insertion target part VOB1). The operation control unit 2 stops the movement (swing) of the actuator (insertion target part) based on the result of the simulation.

[0068] After the movement of the actuator stops, the user can move the insertion target part in the direction of pulling it out of the hole part. When the user performs this operation, the simulation unit 1 moves the insertion target part VOB1 in the direction D10 of pulling it out of the hole part VOP1 based on this operation (see FIG. 24). The operation control unit 2 moves the insertion target part in the direction D10 of pulling it out of the hole part based on the result of the simulation. As a result, the insertion target part VOB1 transitions to a state where it contacts the peripheral edge PE of the opening of the hole part VOP1 at one position (the first contact position CP1). In the state shown in FIG. 24, the insertion depth of the insertion target part VOB1 with respect to the hole part VOP1 is shallower than the state shown in FIG. 23 (the state before moving in the pulling-out direction D10). The simulation unit 1 stops the movement of the insertion target part VOB1 in the pulling-out direction D10 in a state where the insertion depth of the insertion target part VOB1 becomes shallower to a certain extent and the insertion target part VOB1 transitions to a state where it contacts the peripheral edge PE of the opening at one position. The operation control unit 2 stops the movement of the insertion target part based on the result of the simulation.

[0069] After the movement of the insertion target part stops, the user can again perform an operation to swing the insertion target part toward the central axis side of the hole part. By the user performing this operation, the simulation unit 1 determines the moving direction (swinging direction D9, see FIG. 25) of the insertion target part VOB1 for swinging the insertion target part VOB1 along the virtual guide surface VB11 of the swing restriction part VBS toward the central axis CL1 side of the hole part VOP1. The simulation unit 1 moves (swings) the insertion target part VOB1 in the determined moving direction (swinging direction D9) (see FIG. 25). The operation control unit 2 moves (swings) the insertion target part in the operating body along the virtual guide surface VB11 toward the central axis side of the hole part based on the result of the simulation.

[0070] Thereafter, the user pulls out in the pulling direction D1 0The movement operation of the insertion target part and the swinging operation of the insertion target part in the swinging direction D9 can be repeated. When the user performs such an operation, the simulation unit 1 repeats the same procedure as the procedure shown in FIGS. 23 to 25 based on the operation. Then, at a certain point in time, the entire tip of the insertion target part VOB1 (for example, the entire circumferential direction of the circular edge) is inserted into the hole part VOP1 (see FIG. 26). In the state shown in FIG. 26, the side surface of the insertion target part VOB1 contacts the peripheral edge PE of the opening at one place (one first contact position CP1), and a part of the tip of the insertion target part VOB1 contacts the inner peripheral surface of the hole part VOP1 at the third contact position CP3. In such a state, the user can perform an operation to adjust the posture of the insertion target part within the hole part. For example, the user can perform an operation of swinging the insertion target part so that the center line of the insertion target part is substantially parallel to the center line of the hole part around the contact position between the tip of the insertion target part and the inner peripheral surface of the hole part. The simulation unit 1 swings the insertion target part VOB1 based on the operation so that the center line CL2 is substantially parallel to the center line CL1 of the hole part VOP1 around the contact position between the tip of the insertion target part VOB1 and the inner peripheral surface of the hole part VOP1 (the left contact position CP in FIG. 26). As a result, since the center line CL2 is substantially parallel to the center line CL1 of the hole part VOP1, the insertion target part VOB1 can be inserted deeper (it can also be completely inserted). The operation control unit 2 swings the insertion target part based on the result of the determination by the simulation unit 1 so that the center line of the insertion target part is substantially parallel to the center line of the hole part.

[0071] In this embodiment, as described above, since a part of the tip of the insertion target part VOB1 is inserted into the hole part VOP1 and the deep insertion of the insertion target part VOB1 is performed taking this insertion as an opportunity, the user can easily insert the insertion target part into the hole part even if the difference between the diameter of the hole part and the diameter of the insertion target part is small. Moreover, by using the above-described guide restriction part VBG and the swing restriction part VBS, the insertion target part is guided into the hole part so that it is easily inserted into the hole part, and thus the insertion operation becomes even easier.

[0072] In addition, in the present embodiment, the simulation unit 1 is configured to generate a virtual restriction unit VB1 when the distance between the virtual moving body VW corresponding to the moving body and the virtual target position VOP1 becomes equal to or less than a predetermined distance (see FIG. 1 7 references ). Further, the simulation unit 1 is configured to eliminate the virtual restriction unit VB1 when the distance between the virtual moving body VW corresponding to the moving body and the virtual target position VOP1 exceeds the predetermined distance (see FIG. 19). By appropriately generating or eliminating the virtual restriction unit VB1 according to the distance between the virtual moving body VW and the virtual target position VOP1, the virtual restriction unit VB1 does not exist when it is unnecessary, and it is possible to prevent the virtual restriction unit VB1 from interfering with the operation of the moving body. Also in other embodiments, the simulation unit 1 can appropriately generate or eliminate the virtual restriction unit according to the distance between the virtual moving body VW corresponding to the moving body and the virtual target position VOP1. Note that the attitude and trajectory of the moving body by the motion control unit 2 may be stored in the storage unit, and based on the stored attitude and trajectory of the moving body, the working device may be caused to operate in a semi-automatic mode or an automatic mode.

[0073] Next, referring to FIGS. 27 to 30, another embodiment of the operating system will be described. In the embodiment shown in FIGS. 27 to 30, the moving body W (see FIG. 27) is a working device WD that applies a predetermined operation to a predetermined work object OB1. In the above-described embodiment (see FIGS. 1 to 26), the moving body W was described as a part in which the working device WD and the work object OB1 are integrated, but in this embodiment, it is configured to position the working device WD before gripping the work object OB1, and the moving body W is composed only of the working device WD. In this embodiment, the working device WD is , makingIt is configured to grip the work object OB1. In the present embodiment, as shown in FIGS. 28 and 29, in the virtual space VS, a virtual work device VD corresponding to the work device WD and a virtual work object VOB1 corresponding to the work object OB1 are shown. The virtual actuator VW corresponding to the actuator W has a main body part (the virtual work device VD in the example shown in FIG. 28) corresponding to the actuator W (refer to the work device WD shown in FIG. 1), and a second virtual restriction part VB2 provided on at least a part of the main body part and moving relative to the first virtual restriction part VB1. In the example shown in FIG. 28, the main body part (virtual work device VD) has an outer shape and size corresponding to the outer shape and size of the actuator W. In the example shown in FIG. 28, the second virtual restriction part VB2 is provided with a virtual tool part VD3 corresponding to the tool part WD3 (refer to FIG. 1) of the work device WD. Specifically, the second virtual restriction part VB2 is provided on a part (virtual finger VD31) corresponding to the finger WD31 of the tool part WD3. The first virtual restriction part VB1 is configured to guide the actuator W to a virtual target position VOP2 corresponding to a predetermined target position OP2. Specifically, the first virtual restriction part VB1 is configured to guide the second virtual restriction part VB2 to the virtual target position VOP2. The first virtual restriction part VB1 has a virtual guide surface VB11 configured such that the movement range of the virtual actuator VW becomes narrower as it approaches the virtual target position VOP2. The target position OP2 is, in the present embodiment, the position where the actuator W should be located to apply a predetermined operation to the work object OB1, and can be set according to the size, shape, position, etc. of the work object OB1. For example, the work object OB1 can be measured for its position, size, angle, posture, etc. by the measurement device WD4 described above, and the target position OP2 can be set based on the position information and shape information of the work object OB1. The position of the virtual restriction part VB1 is set based on the virtual target position VOP2 set in advance or obtained by the determination of the simulation unit 1.In addition, if the first virtual regulation unit VB1 and the second virtual regulation unit VB2 move relatively, only one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2 may move (the other does not move), or both the first virtual regulation unit VB1 and the second virtual regulation unit VB2 may move.

[0074] In this embodiment, the first virtual restriction unit VB1 has, for example, the same shape as the first virtual restriction unit VB1 shown in FIG. 6. However, the shape of the first virtual restriction unit VB1 is not particularly limited, and it may be a shape different from the shape shown in FIG. 6. The first virtual restriction unit VB1 is determined based on the position of the work object OB1. Specifically, the first virtual restriction unit VB1 is provided at a position facing the tool unit WD3 when the tool unit WD3 grips the work object OB1. If it is provided at a position facing the tool unit WD3 when the tool unit WD3 grips the work object OB1, the number of the first virtual restriction units VB1 is not particularly limited. In the example shown in FIG. 28, the number of the first virtual restriction units VB1 is two, but the number of the virtual restriction units VB1 is not limited to two, and it may be one or more (for example, three or more), and can be appropriately changed according to the operation applied to the work object OB1 by the working device WD. Also, the position of the first virtual restriction unit VB1 can be appropriately changed according to the operation applied to the work object OB1 by the working device WD. For example, in this embodiment, since the work object OB1 is gripped by the working device WD, as shown in FIG. 28, the first virtual restriction unit VB1 is provided at positions on both sides of the virtual work object VOB1 in a direction orthogonal to the length direction of the virtual work object VOB1. In this embodiment, the number of the first virtual restriction units VB1 is set to be the same as the number of the second virtual restriction units VB2. In the example shown in FIG. 28, the number of the first virtual restriction units VB1 and the number of the second virtual restriction units VB2 are each two. Also, in the example shown in FIG. 28, the second virtual restriction unit VB2 is provided at a position corresponding to the virtual finger VD31. By providing the second virtual restriction unit VB2 at a position corresponding to the virtual finger VD31, the position of the finger WD31 when gripping the work object OB1 can be reliably positioned. The size of the first virtual restriction unit VB1 may have substantially the same size as the second virtual restriction unit VB2, or may have a size larger than the second virtual restriction unit VB2. For example, the first virtual restriction unit VB1 can have side walls with substantially the same inclination and a larger bottom BO compared to the second virtual restriction unit VB2.By having a larger bottom BO, the first virtual regulation unit VB1 increases the area capable of receiving the second virtual regulation unit VB2. Therefore, it becomes easier for the user to perform an operation of placing the second virtual regulation unit VB2 into the first virtual regulation unit VB1.

[0075] The second virtual regulation unit VB2 is guided by the first virtual regulation unit VB1 Virtual target position V As long as it can be guided to OP2, its shape is not particularly limited. In the present embodiment, the second virtual regulation unit VB2 has substantially the same shape and size as the first virtual regulation unit VB1. By having substantially the same shape and size, the first virtual regulation unit VB1 guides the second virtual regulation unit VB2 Virtual target position V to OP2 smoothly.

[0076] The simulation unit 1 is configured to determine the positional relationship between the virtual actuator VW and the first virtual restriction unit VB1 by determining the positional relationship between the first virtual restriction unit VB1 and the second virtual restriction unit VB2. Specifically, for example, the simulation unit 1 determines the positional relationship between the virtual actuator VW (specifically, the virtual tool unit VD3 of the virtual working device VD) and the first virtual restriction unit VB1 by determining the positional relationship between the inner surface of the first virtual restriction unit VB1 and the outer surface of the second virtual restriction unit VB2. The simulation unit 1 receives an operation command from the operation device OD, and moves the second virtual restriction unit VB2 to a position where the second virtual restriction unit VB2 and the first virtual restriction unit VB1 match while bringing the outer surface of the side wall of the second virtual restriction unit VB2 into contact with the inner surface of the side wall of the first virtual restriction unit VB1 (virtual guide surface VB11) (see Fig. 29). By guiding the second virtual restriction unit VB2 along the virtual guide surface VB11, the actuator W is guided to the target position OP2. Since the second virtual restriction unit VB2 is guided along the virtual guide surface VB11, even if the user's operation is a rough operation, the actuator W can be easily and surely moved to the target position OP2. When the actuator W is guided to the target position OP2, the simulation unit 1 causes the virtual tool unit VD3 of the virtual working device VD to grip the virtual work object VOB1. Specifically, the virtual work object VOB1 is gripped by the virtual finger VD31 corresponding to the finger WD31. Thereafter, the simulation unit 1 lifts the virtual work object VOB1 (see Fig. 30) and transports the virtual work object VOB1 to the target location.

[0077] Here, assuming a state before the first virtual control unit VB1 and the second virtual control unit VB2 are aligned, when the number of the first virtual control units VB1 and the number of the second virtual control units VB2 are each plural (two in the examples shown in FIGS. 28 and 29). In this state, the direction D7 connecting one first virtual control unit VB1 (for example, the left first virtual control unit VB1 in FIG. 31) and another first virtual control unit VB1 (for example, the right first virtual control unit VB1 in FIG. 31) among the plural (for example, two) first virtual control units VB1 may not match the direction D8 connecting one second virtual control unit VB2 (for example, the left second virtual control unit VB2 in FIG. 31) and another second virtual control unit VB2 (for example, the right second virtual control unit VB2 in FIG. 31) among the plural (for example, two) second virtual control units VB2 (see FIG. 31). In the example shown in FIG. 31, before the first virtual control unit VB1 and the second virtual control unit VB2 are aligned, one first virtual control unit VB1 (hereinafter also referred to as one first virtual control unit VB1) and one second virtual control unit VB2 (hereinafter also referred to as one second virtual control unit VB2) have substantially matching positions when viewed from the vertical direction (direction D3). In contrast, the other first virtual control unit VB1 (hereinafter also referred to as the other first virtual control unit VB1) and the other second virtual control unit VB2 (hereinafter also referred to as the other second virtual control unit VB2) are displaced from each other when viewed from the vertical direction (direction D3) (displaced in directions D1 and D2). Specifically, for example, the other first virtual control unit VB1 and the other second virtual control unit VB2 are displaced in their central positions when viewed from direction D3 (only a part of each other overlaps). In the present embodiment, the simulation unit 1 is configured such that a rotational moment for rotating the other second virtual control unit VB2 about the one second virtual control unit VB2 acts on the other second virtual control unit VB2 in the process of inserting the one second virtual control unit VB2 and the other second virtual control unit VB2 into the one first virtual control unit VB1 and the other first virtual control unit VB1, respectively.When the other second virtual restriction part VB2 moves along the inclination direction of the virtual guide surface VB11 of the other first virtual restriction part VB1, a rotational moment is generated on the other second virtual restriction part VB2 by the force received from the other first virtual restriction part VB1. Due to this rotational moment, the other second virtual restriction part VB2 rotates about one second virtual restriction part VB2 (refer to the arrow shown in FIG. 31). As a result, the other second virtual restriction part VB2 can rotate in a direction that coincides with the other first virtual restriction part VB1. Due to this rotation, the simulation part 1 can make the one second virtual restriction part VB2 and the other second virtual restriction part VB2 coincide with the one first virtual restriction part VB1 and the other first virtual restriction part VB1 respectively. The operation control part 2 rotates the operating body according to the result of the simulation and moves the operating body W to the target position OP2. W as Rotate and move the operating body W to the target position OP2.

[0078] Also, in this embodiment, in the virtual space VS, the position of the first virtual restriction part VB1 is determined by the simulation part 1 according to the shape and position of the work object OB1. Thereby, according to the shape and position of the work object OB1, the first virtual restriction part VB1 is generated at an appropriate position where it is easy to operate on the work object OB1. Therefore, by guiding the second virtual restriction part VB2 by the first virtual restriction part VB1, in the real space RS, the work device WD can operate on the work object OB1 at an appropriate position with respect to the shape and position of the work object OB1. More specifically, in this embodiment, for the rod-shaped work object OB1, the first virtual restriction part VB1 is generated at the central part in the length direction of the work object OB1 and at a position symmetrical to the axis of the work object OB1. Thereby, the rod-shaped work object OB1 can be gripped at a well-balanced position, and easy operation on the work object OB1 becomes possible.

[0079] The operating system of another embodiment shown in FIG. 32 is configured such that the working device is guided along a predetermined movement path. In this embodiment, the operating body is a working device that applies a predetermined operation to a predetermined work object and a work object held by the working device. In this embodiment, the working device is configured to transport a work object (for example, a container such as a cup containing liquid) along a predetermined movement path. In the example shown in FIG. 32, it is assumed that a virtual working device VD corresponding to the working device holds a virtual work object VOB1 corresponding to the work object. The virtual operating body VW corresponding to the operating body has a main body portion (the virtual working device VD and the virtual work object VOB1 in the example shown in FIG. 32) corresponding to the outer shape and size of the operating body, and a second virtual restricting portion VB2 provided on at least a part of the main body portion. In the example shown in FIG. 32, the second virtual restricting portion VB2 is provided on a virtual tool portion VD3 corresponding to the tool portion WD3 (see FIG. 1) of the working device WD. The first virtual restricting portion VB1 is configured to guide the operating body to a predetermined target position. Specifically, the first virtual restricting portion VB1 is configured to guide the second virtual restricting portion VB2 to a predetermined target position. The first virtual restricting portion VB1 has a virtual guide surface VB11 along a predetermined movement path to the target position. The shape of the first virtual restricting portion VB1 is not particularly limited and can be any shape along the purpose of the working device. In the example shown in FIG. 32, the first virtual restricting portion VB1 has a shape in which three flat portions VB1a, VB1b, and VB1c are combined in a substantially U shape.

[0080] The shape of the second virtual restriction unit VB2 is not particularly limited as long as it can be guided to the target position by the first virtual restriction unit VB1. In the example shown in FIG. 32, the second virtual restriction unit VB2 is configured to ensure a predetermined interval between the virtual working device VD and the first virtual restriction unit VB1. Specifically, the second virtual restriction unit VB2 includes three rod-shaped portions VB21 extending in different directions. Each of the three rod-shaped portions VB21 has a length corresponding to the interval to be ensured between the three planar portions VB1a, VB1b, VB1c and the virtual working device VD. The lengths of the three rod-shaped portions VB21 may be the same or different. The simulation unit 1 is configured to determine the positional relationship between the virtual moving body VW and the first virtual restriction unit VB1 by determining the positional relationship between the first virtual restriction unit VB1 and the second virtual restriction unit VB2. Specifically, the simulation unit 1 determines the positional relationship between the virtual moving body VW (specifically, the virtual tool portion VD3 of the virtual working device VD) and the first virtual restriction unit VB1, for example, by determining the positional relationship between the first virtual restriction unit VB1 and the tip of the second virtual restriction unit VB2. The simulation unit 1 receives an operation command from the operation device OD and moves the second virtual restriction unit VB2 and the virtual tool portion VD3 to the target position while bringing the tip of the second virtual restriction unit VB2 into contact with the inner surface (virtual guide surface VB11) of the first virtual restriction unit VB1 (see FIG. 32). By guiding the second virtual restriction unit VB2 along the virtual guide surface VB11, the moving body is guided along a predetermined movement path to the target position. Thereby, the work object (for example, a container such as a cup containing liquid) can be conveyed along a predetermined movement path to the target position.

[0081] In the modification shown in Fig. 33, the first virtual restriction part VB1 is configured in a linear shape, and the second virtual restriction part VB2 is configured in a ring shape. Further, the second virtual restriction part VB2 is configured to be movable along the first virtual restriction part VB1 with the first virtual restriction part VB1 inserted therein. The shape of the first virtual restriction part VB1 is not particularly limited, and may be, for example, linear or curved (see Fig. 33). When the first virtual restriction part VB1 is formed in a linear shape, the second virtual restriction part VB2 can be moved to the target position along the shortest route. When the first virtual restriction part VB1 is formed in a curved shape, the second virtual restriction part VB2 can be moved to the target position, for example, along a route that avoids obstacles.

[0082] In another embodiment shown in Fig. 33, in the virtual space VS, the operating system has the first virtual restriction part VB1 configured in a linear shape and the second virtual restriction part VB2 configured in a ring shape. The second virtual restriction part VB2 is configured to be movable along the first virtual restriction part VB1 with the first virtual restriction part VB1 inserted therein, whereby the working device WD is guided along a predetermined path along the extending direction of the linear first virtual restriction part VB1. In this embodiment, the second virtual restriction part VB2 is provided in a virtual tool part VD3 corresponding to the tool part WD3 (see Fig. 1) of the working device WD. With such a configuration, the virtual tool part VD3 can be moved along the linear movement path. In the example shown in Fig. 33, the second virtual restriction part VB2 has a ring-shaped part VB22 and a connecting part VB23. The ring-shaped part VB22 is provided in the virtual tool part VD3 via the connecting part VB23. With such a configuration, the virtual tool part VD3 can be moved while ensuring a certain interval between the virtual tool part VD3 and the first virtual restriction part VB1. The first virtual restriction part VB1 may be linear, curved, or bent. When the first virtual restriction part VB1 has a curved shape or a bent shape, the virtual tool part VD3 can be moved along a complex movement path.

[0083] In the example shown in FIG. 33, the cross-sectional shape of the first virtual regulation unit VB1 is a shape (circular shape) that allows the first virtual regulation unit VB1 to rotate around the axis with respect to the second virtual regulation unit VB2, but it is not limited thereto. In the modified example shown in FIG. 34, the cross-sectional shape of the first virtual regulation unit VB1 has a shape that does not allow the first virtual regulation unit VB1 to rotate around the axis with respect to the second virtual regulation unit VB2. Specifically, the cross-sectional shape of the first virtual regulation unit VB1 is a rectangular shape (square shape in the example shown in FIG. 34). Further, the cross-sectional shape of the second virtual regulation unit VB2 is formed in a rectangular tube shape (square tube shape in the example shown in FIG. 34) having a cross-sectional shape along the outer periphery of the first virtual regulation unit VB1. By configuring the first virtual regulation unit VB1 not to be able to rotate around the axis with respect to the second virtual regulation unit VB2, the posture of the virtual actuator VW is stabilized. Therefore, for example, when transporting a container such as a cup containing liquid, it can be transported in a stable posture so that the liquid does not spill from the container during transportation.

[0084] Further, the second virtual regulation unit VB2 may contact the first virtual regulation unit VB1 in a point contact state (see FIG. 33), but is not limited thereto. The second virtual regulation unit VB2 may be configured such that, by being in a line contact state or a surface contact state (see FIG. 34) with respect to the first virtual regulation unit VB1, the degree of freedom of the posture with respect to the first virtual regulation unit VB1 is reduced. By reducing the degree of freedom of the posture of the second virtual regulation unit VB2 with respect to the first virtual regulation unit VB1, the posture of the second virtual regulation unit VB2 is stabilized. Therefore, the posture of the virtual actuator can be stabilized. Specifically, in the example shown in FIG. 33, the linear first virtual regulation unit VB1 and the ring-shaped portion VB22 of the second virtual regulation unit VB2 are in point contact. For this reason, the ring-shaped portion VB22 is movable along the length direction of the first virtual regulation unit VB1 and can rotate around the first virtual regulation unit VB1 with the first virtual regulation unit VB1 as the center. Therefore, in the example shown in FIG. 33, the degree of freedom of the posture of the second virtual regulation unit VB2 is high. On the other hand, in the example shown in FIG. 34, the linear first virtual regulation unit VB1 having a square cross section and the square tube-shaped second virtual regulation unit VB2 are in surface contact on four surfaces. For this reason, the second virtual regulation unit VB2 is movable along the length direction of the first virtual regulation unit VB1, but cannot rotate around the first virtual regulation unit VB1 with the first virtual regulation unit VB1 as the center. Therefore, in the example shown in FIG. 34, the degree of freedom of the posture of the second virtual regulation unit VB2 is low.

[0085] In another embodiment shown in FIG. 35, the virtual regulation unit VB1 is formed in a plate shape having a predetermined thickness. The operating system of this embodiment can be used when drawing characters or pictures on a sheet-like workpiece by an operating body (working device WD, such as a writing instrument like a pen). The shape of the operating body in this embodiment is not particularly limited as long as it can draw characters or pictures on the workpiece. In this embodiment, the operating body is configured in a rod shape. The virtual regulation unit VB1 is configured such that a virtual tool part VD3 (hereinafter also referred to as a rod-shaped part VD3), which is a part of the virtual operating body VW, can enter in the thickness direction of the virtual regulation unit VB1 from the tip. In the example shown in FIG. 35, the virtual tool part VD3 is configured in a rod shape. In this embodiment, in the real space RS, the tip of the rod-shaped operating body can be flexibly deformed, but in the virtual space VS, the tip of the rod-shaped virtual operating body VW is configured to enter the virtual regulation unit VB1 without deformation. FIG. 35 shows three rod-shaped parts VD3 with different depths of entry into the virtual regulation unit VB1. The rod-shaped part VD3 is a part where the tip is pressed against the surface of a predetermined workpiece (such as a sheet-like member like paper, provided on the surface (upper surface in FIG. 35) of the virtual regulation unit VB1), and the thickness of the line represented on the surface of the workpiece can change according to the pressing pressure. The left, center, and right rod-shaped parts VD3 in FIG. 35 have gradually increasing depths of entry in this order. The rod-shaped part VD3 is configured to gradually become thicker from the tip side toward the base end side (the conical part in the example shown in FIG. 35). As the depth of entry of the rod-shaped part VD3 into the virtual regulation unit VB1 increases from the tip side toward the base end side, the existing area RE of the rod-shaped part VD3 on the surface of the virtual regulation unit VB1 (the area where the surface of the virtual regulation unit VB1 and the rod-shaped part VD3 intersect, the elliptical area in FIG. 35) is configured to become larger (see FIG. 35). By moving the rod-shaped part VD3 along the surface of the workpiece VOB1, the above-mentioned existing area RE can draw a locus TR corresponding to characters and the like. Note that the upper surface of the virtual workpiece corresponding to a workpiece such as paper coincides with the upper surface of the virtual regulation unit VB1 or is set at a position slightly shifted in the thickness direction.In the real space RS, as the pressure applied by a writing instrument such as a pen onto the paper surface increases, the thickness of the line represented on the paper surface increases. Therefore, by utilizing the size of the existence region RE described above, a simulation is performed to control the penetration depth of the rod-shaped portion VD3 within a predetermined range. Based on the result of the simulation, the operation control unit 2 controls the operation of the operating body (for example, a writing instrument), thereby enabling adjustment of the thickness of the line represented on the paper surface or the like.

[0086] In the embodiment shown in FIG. 36, when performing an operation of pressing an operating body (for example, a tool (working device) such as a spatula) against a work object (for example, an object to which a coating such as putty is applied), by determining the positional relationship between the virtual operating body VW and the virtual work object VOB1, an operation of applying a predetermined pressure to the work object becomes possible. In the present embodiment, the virtual working device VD corresponding to the working device has a virtual tool part VD3 corresponding to the tool part (such as a spatula). The virtual restriction part VB1 is provided along the surface of the virtual work object VOB1 corresponding to the work object worked by the operating body, but is arranged inside the virtual work object VOB1 (inside with respect to the surface of the virtual work object VOB1). The depth at which the virtual work object VOB1 is arranged is set to a depth corresponding to the pressure with which the tool part is pressed against the work object. The operating body (such as a spatula) bends by a predetermined amount when a predetermined pressure is applied, but the virtual operating body VW is configured not to bend. When the tip of the virtual operating body VW contacts the virtual restriction part VB1 arranged inside the virtual work object VOB1, the operating body in the real space RS contacts the work object with a predetermined pressure. Thereby, it becomes possible to control the pressure for pressing the operating body against the work object by using the virtual restriction part VB1 in the virtual space VS. When it is desired to increase the pressure with which the tool part is pressed against the work object, the depth at which the virtual work object VOB1 is arranged may be set to a large value. The simulation unit 1 operates the virtual operating body VW so that the virtual operating body (specifically, the virtual working device VD) contacts the virtual restriction part VB1. The operation control unit 2 enables performing an operation while applying pressure to the surface of the work object OB1 by the operation of the virtual operating body VW. When the virtual operating body VW contacts the virtual restriction part VB1 inside the virtual work object VOB1, in the real space RS, the tool part such as a spatula bends due to the pressing pressure. Therefore, a putty application operation or the like on the work object can be performed with an appropriate pressure, and damage to the spatula or the coating target is also suppressed. In the coating operation using a spatula, the spatula and the work object are in line contact, but it is not limited thereto. For example, when the tool part is a grinder, the same principle can be applied to a polishing operation by surface contact of the grinder with the work object (see FIG. 37).

[0087] In the embodiment shown in FIG. 38, when work is performed by a plurality of actuators (for example, a plurality of working devices WD), these contacts are suppressed. The simulation unit 1 is configured to provide a virtual restriction unit VB1 between a plurality of virtual actuators VW respectively corresponding to the plurality of actuators when the interval between the plurality of actuators becomes equal to or less than a predetermined interval. Specifically, when the interval between a plurality of actuators (for example, a plurality of working devices) becomes equal to or less than a predetermined interval (an interval in which interference with each other may occur), a virtual restriction unit VB1 is provided between a plurality of virtual working devices VD corresponding to the plurality of actuators (see FIG. 38). The virtual restriction unit VB1 is arranged so as to partition between the plurality of actuators. Specifically, the virtual restriction unit VB1 is arranged so as to partition between a plurality of virtual working devices VD corresponding to the plurality of actuators (see FIG. 38). The shape and size of the virtual restriction unit VB1 are not particularly limited as long as the interference between the actuators can be prevented by partitioning between the plurality of actuators. In the example shown in FIG. 38, the virtual restriction unit VB1 is configured in a plate shape (for example, a flat plate shape), but may be configured in a planar shape (for example, a flat planar shape or a curved planar shape). When the virtual working device VD contacts the virtual restriction unit VB1, the simulation unit 1 does not allow the virtual working device VD to move beyond the virtual restriction unit VB1. Therefore, interference between the virtual working devices VD can be prevented. Further, the simulation unit 1 may be configured to eliminate the virtual restriction unit VB1 when the interval between the plurality of actuators exceeds a predetermined interval (see FIG. 39). Specifically, the simulation unit 1 eliminates the virtual restriction unit VB1 when the interval between a plurality of virtual actuators VW corresponding to the plurality of actuators exceeds a predetermined interval (an interval in which interference with each other is unlikely to occur). With such a configuration, interference between the actuators can be prevented and the degree of freedom of work is also increased.

[0088] In other embodiments shown in FIGS. 40 to 43, the first virtual restriction unit VB1 is provided so as to individually surround a part of one of the plurality of actuators (see FIG. 40). Specifically, the first virtual restriction unit VB1 is provided so as to individually surround a virtual tool unit VD3 corresponding to a part (for example, a tool part) of one of the plurality of actuators (for example, a working device). Further, the second virtual restriction unit VB2 is provided so as to individually surround a part of another one of the plurality of actuators (see FIG. 40). Specifically, the second virtual restriction unit VB2 is provided so as to individually surround a virtual tool unit VD3 corresponding to a part (for example, a tool part) of another one of the plurality of actuators (for example, a working device). The simulation unit 1 simulates the positional relationship between the first virtual restriction unit VB1 and the second virtual restriction unit VB2. The operation control unit 2 is configured to prevent the distance between the plurality of actuators from becoming equal to or less than a predetermined distance based on the simulated positional relationship. Specifically, the simulation unit 1 is configured to prevent the distance between the virtual working devices VD corresponding to the plurality of actuators from becoming equal to or less than a distance at which mutual interference may occur based on the simulated positional relationship. For example, when the distance between the plurality of virtual actuators (virtual working devices VD) becomes a predetermined distance, the simulation unit 1 temporarily stops both operations (see FIG. 41). With such a configuration, interference between the plurality of actuators can be prevented.

[0089] Note that the simulation unit 1 is preferably configured to change the size and / or shape of at least one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2 according to the positional relationship between the first virtual regulation unit VB1 and the second virtual regulation unit VB2 (see FIG. 42). In the example shown in FIG. 42, when the first virtual regulation unit VB1 and the second virtual regulation unit VB2 approach each other and the distance between the first virtual regulation unit VB1 and the second virtual regulation unit VB2 becomes a predetermined distance, the simulation unit 1 is configured to reduce the size of at least one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2. Also, in the example shown in FIG. 42, when the first virtual regulation unit VB1 and the second virtual regulation unit VB2 approach each other and the distance between the first virtual regulation unit VB1 and the second virtual regulation unit VB2 becomes a predetermined distance, the simulation unit 1 is configured to narrow the shape of at least one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2. Specifically, the simulation unit 1 is configured to reduce the size of at least one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2 so that the work can be performed in order from the actuator with the highest work priority. With such a configuration, the work can be performed in order from the actuator with the highest work priority. For example, in the example shown in FIG. 42, by reducing the sizes of the first virtual regulation unit VB1 and the second virtual regulation unit VB2, the virtual actuator VW on the left can perform the work first. The simulation unit 1 retracts the virtual actuator VW that has completed the work backward from the work position. Then, when the distance between the two virtual actuators VW becomes sufficiently large and there is no possibility of interference, the simulation unit 1 eliminates the first virtual regulation unit VB1 and the second virtual regulation unit VB2 (see FIG. 43).

[0090] Note that the simulation unit 1 may be configured to increase the size of at least one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2 when the first virtual regulation unit VB1 and the second virtual regulation unit VB2 approach each other and the distance between the first virtual regulation unit VB1 and the second virtual regulation unit VB2 becomes a predetermined distance. In this case, interference between a plurality of actuators can be more reliably prevented. Further, the simulation unit 1 may be configured to change the shape of at least one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2 to an arbitrary shape that facilitates the operation by the actuator when the first virtual regulation unit VB1 and the second virtual regulation unit VB2 approach each other and the distance between the first virtual regulation unit VB1 and the second virtual regulation unit VB2 becomes a predetermined distance. Further, when changing the size and / or shape of at least one of the first virtual regulation unit VB1 and the second virtual regulation unit VB2, the simulation unit 1 may be configured to generate a virtual regulation unit with the changed size and / or shape after eliminating the virtual regulation unit that is the target of the size and / or shape change.

[0091] The embodiments shown in FIGS. 44 to 46 are such that the working body is a working device that applies a predetermined operation to a moving work object, and it is possible to make the working device follow the moving work object. In this embodiment, as an example, it is an operation of being arranged on a conveying device such as a conveyor and moving while following a moving work object to perform a predetermined work. In this embodiment, the virtual restriction unit VB1 is determined based on the position of the moving work object. The virtual restriction unit VB1 may be generated so as to overlap a part of the virtual work object VOB1 that moves, or may be generated at a position slightly separated from the virtual work object VOB1 that moves. The virtual restriction unit VB1 is generated so that the distance from the virtual work object VOB1 that moves is constant. Therefore, the virtual restriction unit VB1 is configured to move together with the virtual work object VOB1 within the virtual space SV (see FIGS. 45 and 46). In this case, when the working device is moved toward the moving work object and the virtual working device VD hits the virtual restriction unit VB1 that moves in the virtual space VS, the virtual working device VD stops at the position of the virtual work object VOB1 that moves. Thereby, the working device in the real space RS can also stop at the part of the moving work object by the operation control unit 2 and perform a predetermined work on the moving work object (see FIGS. 44 to 46).

[0092] In the embodiment shown in FIG. 47, the virtual restriction unit VB1 is configured to restrict the movement of the working body within the range inside the virtual restriction unit VB1. The shape of the virtual restriction unit VB1 is not particularly limited as long as it can restrict the movement of the working body within the range inside the virtual restriction unit VB1, and for example, it may be in a frame shape or a container shape. In the example shown in FIG. 47, the virtual restriction unit VB1 is configured in a frame shape. For example, when the virtual working body VW corresponding to the working body hits the inner peripheral surface of the frame-shaped virtual restriction unit VB1, the simulation unit 1 corrects the moving direction along the virtual restriction unit VB1. Thereby, the movement of the working body can be restricted within the range inside the virtual restriction unit VB1.

Explanation of Reference Numerals

[0093] 1 Simulation unit 2 Operation control unit 3 Operation Input Unit 4 Output Unit CL1 Center Line of Virtual Target Position (Hole Part) CL2 Center Line of Virtual Work Object CP Contact Position CP1 First Contact Position CP2 Second Contact Position CP3 Third Contact Position OB1 Work Object OD Operating Device OP1 Target Position P1 Collision Point PE Periphery of Opening RS Real Space VB1 First Virtual Regulation Unit VB11 Virtual Guide Surface VB1a, VB1b, VB1c Plane Parts VB2 Second Virtual Regulation Unit VB21 Rod-shaped Part VB22 Ring-shaped Part VB23 Connecting Part VBG Guide Regulation Unit VBS Swing Regulation Unit VD Virtual Working Device VD3 Virtual Tool Part VD31 Virtual Finger VOB1 Virtual Work Object VOP1 Virtual Target Position VS Virtual Space VW Virtual Actuator W Actuator WS Actuation System WD Working Device WD1 Pedestal Part WD2 Arm Part WD3 Tool Part WD31 Finger WD4 Measuring Device WD41 Camera θ Tilt Angle

Claims

1. In order to simulate the operation of an operating body that performs a predetermined operation in response to an input operation from an operating device by a user, in a virtual space, a virtual operating body corresponding to the operating body and a first virtual restricting unit that is virtually provided to restrict the movement of the virtual operating body A simulation unit that simulates the positional relationship between; An operation control unit that controls the operation of the operating body according to the positional relationship simulated by the simulation unit An operating system comprising: When an input operation is performed on the operating body, the simulation unit, in parallel with the input operation, operates the virtual operating body in the virtual space with the content corresponding to the input operation, and determines the positional relationship between the virtual operating body and the first virtual restricting unit at a predetermined time interval. The operation control unit is configured to operate the operating body in parallel with an operation command based on the input operation based on the result of the determination by the simulation unit. The simulation unit determines whether the distance between the virtual operating body and the first virtual restricting unit is within a predetermined range. When it is determined that the distance between the virtual operating body and the first virtual restricting unit is within the predetermined range, and the moving direction of the virtual operating body corresponding to the input operation includes a component in the direction toward the surface of the first virtual restricting unit and a component in the direction along the surface of the first virtual restricting unit, the simulation unit corrects the moving direction of the virtual operating body corresponding to the input operation based on the shape of the first virtual restricting unit, whereby (i) obtaining a component in the direction along the surface of the first virtual restricting unit included in the moving direction of the virtual operating body corresponding to the input operation as the corrected moving direction of the virtual operating body for moving the virtual operating body along the surface of the first virtual restricting unit, the operation control unit is configured to move the operating body in the obtained corrected moving direction, or (ii) stopping the virtual operating body with respect to the surface of the first virtual restricting unit, the operation control unit is configured to stop the operating body with respect to the first virtual restricting unit, or (iii) causing the virtual operating body to bounce back with respect to the surface of the first virtual restricting unit, the operation control unit is configured to cause the operating body to bounce back with respect to the first virtual restricting unit. An operating system.

2. The simulation unit determines whether the virtual actuator and the first virtual restriction unit are in contact based on the distance between the virtual actuator and the first virtual restriction unit. When it is determined that the virtual actuator and the first virtual restriction unit are in contact, the simulation unit is configured to correct the moving direction of the virtual actuator corresponding to the input operation based on the shape of the first virtual restriction unit. The operation system according to claim 1.

3. The first virtual restriction unit has a virtual guide surface configured to guide the actuator to a predetermined target position. The first virtual restriction unit is configured such that the virtual actuator can pass through the virtual guide surface from the side opposite to the virtual guide surface to the virtual guide surface side, and the virtual actuator cannot pass through the virtual guide surface from the virtual guide surface side to the side opposite to the virtual guide surface. The operation system according to claim 1.

4. The first virtual restriction unit is configured to guide the actuator to a predetermined target position. The simulation unit is configured to generate the first virtual restriction unit when the distance between the actuator and the target position becomes equal to or less than a predetermined distance. The simulation unit is configured to eliminate the first virtual restriction unit when the distance between the actuator and the target position exceeds the predetermined distance. The operation system according to claim 1.

5. The first virtual restriction unit is configured to guide the actuator to a hole, which is a predetermined target position. The first virtual restriction unit has a guide restriction unit that guides the insertion target part such that a part of the tip of the insertion target part of the virtual actuator is inserted into the hole and the center line of the insertion target part is inclined with respect to the depth direction of the hole. The operation system according to claim 1.

6. The first virtual restriction unit further has a swing restriction unit that restricts the swing direction of the insertion target part centered on the contact position with the peripheral edge of the opening in a state where the insertion target part with a part of its tip inserted into the hole contacts the peripheral edge of the opening. The swing restriction unit restricts the swing direction such that the inclination angle of the center line of the insertion target part with respect to the depth direction of the hole becomes smaller. The operation system according to claim 5.

7. The virtual actuator has a main body corresponding to the actuator, and a second virtual restriction portion provided on at least a part of the main body and moving relative to the first virtual restriction portion. The simulation unit is configured to determine the positional relationship between the virtual actuator and the first virtual restriction portion by determining the positional relationship between the first virtual restriction portion and the second virtual restriction portion. The operating system according to claim 1.

8. The first virtual restriction portion is configured to guide the actuator to a predetermined target position. The first virtual restriction portion has a virtual guide surface configured such that the movement range of the virtual actuator becomes narrower as it approaches the target position. The operating system according to claim 7, wherein the second virtual restriction portion is guided by the virtual guide surface, and the actuator is guided to the target position.

9. The first virtual restriction portion has a predetermined thickness, and the first virtual restriction portion is configured such that a part of the virtual actuator can enter in the thickness direction of the first virtual restriction portion from the tip. The part is configured to gradually become thicker from the tip side toward the base end side. The operating system according to claim 1, wherein as the depth at which the part enters the first virtual restriction portion increases from the tip side toward the base end side, the existence region of the part on the surface of the first virtual restriction portion becomes larger.

10. The actuator is a working device that applies a predetermined operation to a predetermined work object. The first virtual restriction portion is disposed inside the work object. The simulation unit operates the virtual actuator so that the virtual actuator contacts the first virtual restriction portion. The operation control unit enables the work to be performed while applying pressure to the surface of the work object by the operation of the virtual actuator. The operating system according to claim 1.

11. The simulation unit is configured to change at least one of the size and / or shape of the first virtual restriction portion and the second virtual restriction portion according to the positional relationship between the first virtual restriction portion and the second virtual restriction portion. The operating system according to claim 7.

12. The actuator is a working device that applies a predetermined operation to a moving work object. The first virtual restriction unit is determined based on the position of the work object to be moved and is configured to move together with the work object within the virtual space. The operating system according to claim 1.

13. The operating system according to claim 1, wherein the first virtual restriction unit is configured to restrict the movement of the operating body within a range inside the first virtual restriction unit.

Images