Robot system, robot control device, and robot control method

Abstract

To keep a workpiece disposed on the front surface of measurement means as much as possible in order to keep the workpiece within a measurement range regarding the ability of the measurement means such as a camera to directly measure the workpiece, which greatly affects the overall performance of a robot system, and retention of the workpiece within the measurement range which is a condition for enabling the measurement means such as a camera to directly measure the workpiece.SOLUTION: This robot control device is for controlling a robot (100). The robot (100) includes object measurement means (310) in a place equivalent to a fingertip. As long as there is a workpiece within the measurement range of the object measurement means (310), a limit is imposed to inhibit change of a posture of the robot (100) for an operation instruction to the robot (100).SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a robot system, a robot control device, and a robot control method. [Background technology]

[0002] When working with a robot, it is necessary to teach the robot the position and posture information required for the work accurately. This work is called teaching work, and the method mainly used is to operate a control device called a teaching box to move and record the position and posture of the hand. However, operation using a teaching box is complicated, the movement is hard to predict, and it is not easy to obtain the target position and posture.

[0003] For example, Patent Document 1 describes technology related to a direct teaching robot, which allows an operator to control the position and posture of the hand by directly applying force to the robot in the direction of the desired movement. The robot described in Patent Document 1 can be intuitively operated, which is expected to improve operability and reduce the effort required for teaching work. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 5-303425 Summary of the Invention [Problem to be solved by the invention]

[0005] A robot system that uses image processing (hereinafter referred to as a "machine vision system") measures the target workpiece using a measuring device such as a camera. At this time, the conditions for measuring the workpiece using a measuring device such as a camera are factors that greatly affect the performance of the entire machine vision system. The condition for measuring a workpiece with a measuring means such as a camera is, for example, the position of the workpiece within the measurement range, and it is desirable that the workpiece be placed in front of the measuring means as much as possible.

[0006] In a machine vision system in which a measuring device is attached to the hand of a robot, when the position and posture of the hand of the robot changes, the position of the workpiece within the measurement range changes accordingly. Although the magnitude of the change varies depending on the distance between the measuring device and the workpiece, the posture of the robot's hand in particular can significantly change the position of the workpiece within the measurement range even if the change is minute. Therefore, the posture of the robot's hand must be precisely controlled. In the conventional technology exemplified in Patent Document 1, in order to precisely control the attitude, the force applied by the operator needs to be finely adjusted, and the resulting problem of reduced operability is not described or suggested. There is therefore room for improvement in terms of reducing the effort required for teaching work. In consideration of the above-mentioned problems, the present disclosed technology aims to provide a machine vision system that can easily teach the posture of a robot's hand during measurement. [Means for solving the problem]

[0007] The robot control device according to the disclosed technology is a robot control device that controls a robot, and the robot is equipped with an object measuring means at a location equivalent to the hand, and can restrict the robot's posture from changing in response to operational instructions given to the robot only when a workpiece is present within the measurement range of the object measuring means. Effect of the Invention

[0008] Since the robot system according to the present disclosure has the above-described configuration, it is possible to easily teach the posture of the robot's hand. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing the appearance of a robot system according to the first embodiment. [Diagram 2] FIG. 2 is a block diagram showing a functional configuration of the object position acquisition unit 300 that constitutes the robot system according to the first embodiment. [Diagram 3] FIG. 3 is a block diagram illustrating a functional configuration of the robot system according to the first embodiment. As shown in FIG. [Figure 4] FIG. 4 is a block diagram showing the movement instruction calculation unit 424 in the robot control device 400 according to the first embodiment in more detailed functional blocks. [Diagram 5] FIG. 5 is an explanatory diagram showing how attitude operation limiting conditions are set based on the field of view (FoV) of the object measuring means 310. [Figure 6] FIG. 6 is a block diagram illustrating how the follower control simulation calculation unit 422 in the robot control device 400 according to the first embodiment calculates the virtual operation amount (ΔP). [Figure 7] FIG. 7 is a block diagram showing a detailed functional configuration of attitude manipulation adjustment section 424c in movement instruction calculation section 424 according to the second embodiment. [Figure 8] FIG. 8 is an explanatory diagram showing the relationship between the field of view (FoV) of the object measuring means 310 used in the robot system according to the second embodiment and the workpiece virtual position plane. [Figure 9] FIG. 9 is an explanatory diagram showing the allowable reference vector and the allowable angular range introduced in the robot system according to the second embodiment. [Figure 10] FIG. 10 is an explanatory diagram showing the allowable reference vector and the allowable angular range in the robot system according to the third embodiment. [Figure 11] FIG. 11 is an explanatory diagram showing the allowable reference vector and the allowable angle range in the robot system according to the third embodiment (in the case of a special case). [Figure 12] FIG. 12 is an explanatory diagram showing the shortest distance vector actually adopted in the special case shown in FIG. [Figure 13] FIG. 13 is an explanatory diagram showing the allowable reference vector and the allowable angular range introduced in the robot system according to the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] In the formulas appearing in this specification, vector or matrix variables are written in bold whenever possible to distinguish them from scalar variables. In this specification, the so-called absolute value symbol applied to a scalar means "absolute value," and the so-called absolute value symbol applied to a vector means "vector magnitude." Furthermore, all three-dimensional coordinate systems appearing in this specification and the drawings are right-handed.

[0011] Embodiment 1 Fig. 1 is an explanatory diagram showing the appearance of a robot system 1000 according to the first embodiment. Fig. 1 shows the elements constituting the robot system 1000, namely, a robot 100, an object position acquisition unit 300, and a robot control device 400. Although not explicitly shown in Fig. 1, the robot system 1000 according to the first embodiment also includes a force sensing unit 200 in addition to the robot 100, the object position acquisition unit 300, and the robot control device 400. Details of the functional configuration of the robot system 1000 according to the first embodiment will become clear from the description with reference to Fig. 3 described later.

[0012] "100 Robots" The robot 100 performs a task on a work object (hereinafter referred to as a "workpiece") that exists in a working space. The robot 100 may be, for example, an industrial robot equipped with a motor and an encoder for each axis. The robot 100 is equipped with a drive unit 110 as a motor for each axis.

[0013] <<Force sensing unit 200>> The force sensing unit 200 is a component that detects physical quantities such as force and moment (torque) applied to the robot 100. The force sensing unit 200 may be, for example, a force sensor provided at the hand of the robot 100, a torque sensor provided at each joint of the robot 100, etc. The physical quantity detected by the force sensing unit 200 is used to control the robot 100.

[0014] 《Object position acquisition unit 300》 The object position acquisition unit 300 is a component that acquires the position of an object that exists in the working space. The object that exists in the working space includes a workpiece. 2 is a block diagram showing a functional configuration of the object position acquisition unit 300 constituting the robot system 1000 according to embodiment 1. As shown in FIG. 2, the object position acquisition unit 300 includes an object measurement means 310 and an object position estimation unit 320.

[0015] <<Object measuring means 310 constituting object position acquisition unit 300>> The object measurement means 310 constituting the object position acquisition unit 300 is a means for measuring information such as the shape and appearance of an object existing in the working space. Specifically, the object measurement means 310 is a camera. The object measurement means 310 is preferably a camera capable of acquiring three-dimensional data by the principle of triangulation, such as a stereo camera. The object measurement means 310 may be a 3D-LiDAR. The object measurement means 310 measures the workpiece and the surroundings of the workpiece. The object measurement means 310 acquires measurement data such as still images or videos (hereinafter collectively referred to as "still and video images") of the workpiece and the surroundings of the workpiece. The measurement data acquired by the object measurement means 310 is used to estimate the position and orientation of the workpiece. Therefore, the measurement data is selected in a format and specifications suitable for estimating the position and orientation of the workpiece. Specifically, the measurement data includes visible images (color or monochrome), multispectral images, near-infrared images, distance images, point cloud data, and the like.

[0016] The robot 100 has several coordinate systems. Specifically, these include a base coordinate system, a mechanical interface coordinate system, a tool coordinate system, and the like. The base coordinate system is a coordinate system whose origin is the base mounting surface of the robot 100, and is a coordinate system that is the reference for the position and posture of the robot 100. The mechanical interface coordinate system is a coordinate system whose origin is the center of the surface on which the end effector of the robot 100 is mounted. The tool coordinate system is a coordinate system whose origin is generally set on the end effector mounted on the mechanical interface of the robot 100. The origin of the tool coordinate system is called the Tool Center Point (hereinafter simply referred to as "TCP"), and is, for example, the point where the end effector of the robot 100 touches the workpiece. The position and posture of the robot 100 refer to the position and posture of the tool coordinate system in the base coordinate system. On the other hand, the inherent coordinate system of the object measuring means 310 is called a camera coordinate system. The position and orientation of the object measuring means 310 refer to the position and orientation of the camera coordinate system in the base coordinate system. When the robot 100 performs delicate work, the object measuring means 310 is often fixed near the tip of the robot 100 (see FIG. 1). In this case, the position and orientation of the object measuring means 310 change according to the drive of the robot 100. The robot system 1000 knows the relative positional relationship between the camera coordinate system and the tool coordinate system. In other words, the camera coordinate system and the tool coordinate system can be converted into each other by simple coordinate conversion. The robot system 1000 uses the camera coordinate system and the base coordinate system appropriately depending on the situation of use.

[0017] <<Object position estimation unit 320 constituting object position acquisition unit 300>> The object position estimation unit 320 constituting the object position acquisition unit 300 is a component that estimates the position and orientation of a workpiece based on the measurement data acquired by the object measurement means 310. The object position estimation unit 320 may estimate the position and orientation of the workpiece using an algorithm related to alignment, for example, an Iterative Closest Point (ICP) matching algorithm. In this way, the object position estimation unit 320 performs image signal processing to estimate the position and orientation of the workpiece. Note that the workpiece has a workpiece coordinate system that is an inherent coordinate system, and the object position estimation unit 320 estimates the position and orientation of the workpiece coordinate system in the camera coordinate system.

[0018] <<Robot control device 400>> 3 is a block diagram showing a functional configuration of the robot system 1000 according to Embodiment 1. As described above, the robot system 1000 includes the robot 100, the force sensing unit 200, the object position acquiring unit 300, and the robot control device 400. The robot controller 400 is a controller that controls the driving unit 110 of the robot 100. As shown in FIG. The robot control device 400 may be a processing circuit using dedicated hardware, or may be a general-purpose computer that executes a program stored in a memory. It can be said that the present disclosure has special technical features, particularly in the robot control device 400.

[0019] <<Storage unit 410 constituting the robot control device 400>> The storage unit 410 constituting the robot control device 400 is a component that stores various information. Specifically, the storage unit 410 is a memory or a storage device. The various information stored in the storage unit 410 includes information that enables mutual coordinate conversion between the above-mentioned camera coordinate system and the tool coordinate system, information necessary for teaching the robot 100, and the like. In this embodiment, the teaching operation for the robot 100 is mainly shown as direct teaching.

[0020] <<Control unit 420 constituting the robot control device 400>> The control unit 420 constituting the robot control device 400 is a component that generates a control signal for controlling the driving unit 110 (motor) of the robot 100. Specifically, the control unit 420 is a controller. If the robot control device 400 is a general-purpose computer, the control unit 420 is realized by a control program. 3, the control unit 420 includes a force detection information acquisition unit 421, a follower control simulation calculation unit 422, an operation instruction acquisition unit 423, and a movement instruction calculation unit 424. The details of the processing contents performed by the control unit 420 will be made clear later. The control unit 420 also has a function of controlling the position and posture of the robot's hand and the speed at which they change, as well as the angle and angular velocity of each joint, to desired values. These functions are usually provided in a general robot control device, and are realized by, for example, a control program.

[0021] <<Control board 430 constituting robot control device 400>> The control board 430 constituting the robot controller 400 is an interface for the robot controller 400 to output a control signal. If the robot controller 400 is a general-purpose computer, the control board 430 is specifically an interface board including a power supply terminal for driving a motor. The robot controller 400 controls the drive unit 110 of the robot 100 via the control board 430.

[0022] <<Force detection information acquisition unit 421 in control unit 420>> The force detection information acquisition unit 421 in the control unit 420 is a component that acquires a signal sent from the force sense detection unit 200 as force detection information. The force detection information includes a "force instruction vector" that is a vector representation of the force and moment applied to the hand of the robot 100 by the operator of the robot 100 (hereinafter referred to as the "user") during a teaching operation, or an "external force torque vector" that is a vector representation of the torque applied to the rotation axis of each joint. The force instruction vector is represented by a symbol F with a hat (see FIG. 6A described below). The external force torque vector is represented by a symbol τ with a hat (see FIG. 6B described below). In the direct teaching method, the robot 100 is subjected to a follow-up control, which is a control for realizing a follow-up movement in response to a force applied by a user.

[0023] <<Follower control simulation calculation unit 422 in control unit 420>> The follower control simulation calculation unit 422 in the control unit 420 is a component that calculates a "virtual operation amount" based on the force detection information acquired by the force detection information acquisition unit 421. Here, the virtual operation amount means the amount of change in the position and posture of the hand of the robot 100 when the robot 100 is subjected to follower control based on the force detection information. When the robot 100 is subjected to follower control, the virtual operation amount (ΔP) is given by the following formula (see FIG. 6A). TIFF2025006026000002.tif49166 Here, the subscript "1" appearing in formula (1) is a serial number to distinguish it from other aspects and has no other meaning than as a serial number. As shown in equation (1), the force indication vector is {f x ,f y ,f z ,m x ,m y ,m z} is a vertical vector (also called a column vector). x ,f y ,f z} is the force applied to the hand of the robot 100. x ,m y ,m z} is the moment of the force applied to the hand of the robot 100. The diagonal matrix K1 in formula (1) is {f x ,f y ,f z ,m x ,m y ,m z} is a matrix having gains set for each element in the diagonal elements. In this way, in the robot 100 that is subject to follower control, the virtual operation amount (ΔP) is proportional to the force instruction vector (F with a hat).

[0024] The following information can be read from the virtual operation amount (ΔP1) given by equation (1). TIFF2025006026000003.tif32166 Here, Δt in formula (2) is the time interval. The vertical vector in formula (2) is the rate of change (velocity) of the position of the robot 100, {v 1x ,v 1y ,v 1z}, and {ω 1x ,ω 1y ,ω 1z} and are elements. As shown in formulas (1) and (2), if the force instruction vector (F with a hat) is known, the speed at which the robot 100 moves, {v 1x ,v 1y ,v 1z} and the angular velocity {ω 1x ,ω 1y ,ω 1z}, we can see that.

[0025] Equations (1) and (2) relate to a force sensor in the case where the force detection unit 200 is provided at the hand of the robot 100, and directly measures the force and moment applied to the hand of the robot 100. In the case where the force detection unit 200 is a torque sensor provided at each joint of the robot 100, and measures the torque generated at each joint due to the force and moment applied to the hand of the robot 100, the virtual operation amount (ΔP) is given by the following equation (see FIG. 6B). TIFF2025006026000004.tif35166 Here, the subscript "2" appearing in formula (3) is a serial number to distinguish it from other embodiments, similar to the subscript "1" in formula (1). The matrix J in the formula (3) is a Jacobian matrix. This Jacobian matrix (J) is obtained by partially differentiating the position and orientation vector (r) of the hand of the robot 100 with respect to the displacement vector (q) of the joint of the robot 100. As shown in equation (3), the external torque vector is {τ1, τ2, …, τ N} is a vertical vector with elements {τ1, τ2, …, τ N} is a torque applied to each rotation axis of the robot 100 by a user operation. That is, in the aspect shown in the formula (3), the robot 100 has torque sensors (force sensing units 200) at N joints. The diagonal matrix K2 in formula (3) is {τ1, τ2, …, τ N} is a matrix having the gain set for each axis in the diagonal elements.

[0026] Similar to equation (2), the following information can be read from the virtual operation amount (ΔP2) given by equation (3). TIFF2025006026000005.tif32166 As shown in the formulas (3) and (4), the follower control simulation calculation unit 422 can calculate the speed {v 2x ,v 2y ,v 2z} and angular velocity {ω 2x ,ω 2y ,ω 2z} can be calculated.

[0027] <<Operation instruction acquisition unit 423 in control unit 420>> The operation instruction acquisition unit 423 in the control unit 420 is a component that acquires a virtual operation amount (ΔP) of the position and orientation of the hand of the robot 100 as an “operation instruction” to the robot 100 by the user. The operation instruction acquisition unit 423 can acquire, as an "operation instruction", any instruction for the user to operate the robot 100. For example, the instruction may be an input signal from a control panel called a teaching box, or the amount of operation of an operating robot provided separately from the robot 100 by the user. When the robot control device 400 is a general-purpose computer that executes a program stored in a memory, the operation instruction acquisition unit 423 assigns the value of the virtual operation amount (ΔP) to a variable assigned to the "operation instruction".

[0028] <<Movement instruction calculation unit 424 in control unit 420>> Simply put, the movement instruction calculation unit 424 in the control unit 420 is a component that imposes restrictions on the “operation instruction” so that the workpiece is less likely to deviate from the measurement range of the object measuring means 310 . Fig. 4 is a block diagram showing in more detail functional blocks the movement instruction calculation unit 424 in the robot control device 400 according to embodiment 1. As shown in Fig. 4, the movement instruction calculation unit 424 can be divided into detailed level functions, that is, a work virtual movement position calculation unit 424a, an attitude operation restriction determination unit 424b, an attitude operation adjustment unit 424c, and a movement instruction output unit 424d.

[0029] <<Work virtual movement position calculation unit 424a of the movement instruction calculation unit 424>> The work virtual movement position calculation unit 424a of the movement instruction calculation unit 424 is a component that calculates the position and orientation of the work after Δt as seen from the object measurement means 310 (camera) based on the position and orientation of the work at the current time and the virtual operation amount (ΔP), which is an operation instruction. Therefore, it can be said that the work virtual movement position calculation unit 424a is a component that estimates the state of the work after Δt. Hereinafter, the position of the work after Δt is referred to as the "work virtual movement position." Here, the term "virtual" is merely a prefix that emphasizes that it is a predicted value or an estimated value of a future state. In this way, the work virtual movement position is an estimate of how the work position will move relatively after Δt has elapsed when the camera coordinate system itself moves as the TCP moves due to the virtual operation amount (ΔP), which is the operation instruction. The workpiece virtual movement position calculation unit 424a is not an essential component, but is included as a component as appropriate depending on the selection of a "workpiece evaluation position" described later.

[0030] <<Posture operation restriction determination unit 424b of movement instruction calculation unit 424>> The attitude manipulation restriction determination unit 424b of the movement instruction calculation unit 424 is a component that determines whether or not to restrict the attitude manipulation amount of the virtual manipulation amount (ΔP), which is an operation instruction, based on the "workpiece evaluation position". Here, specifically, the following two candidates are considered for the "work evaluation position". One candidate for the "work evaluation position" is the work virtual movement position calculated by the work virtual movement position calculation unit 424a. The work virtual movement position is a future predicted position that reflects the user's intention, and is therefore preferable as a value for determining whether or not to limit the amount of posture manipulation. The other candidate for the "work evaluation position" is the position and posture of the work at the current time estimated by the object position acquisition unit 300 (hereinafter referred to as "work position information"). In addition to this, the "work evaluation position" may be calculated based on the work position information and the work virtual movement position. Since the work is also a work object, the work position information may be read as work object position information.

[0031] <<Attitude Operation Adjustment Unit 424c of Movement Instruction Calculation Unit 424>> The posture operation adjustment section 424c of the movement instruction calculation section 424 is a component that calculates an adjustment coefficient defined for the virtual operation amount (ΔP), which is an operation instruction, based on the determination result of the posture operation restriction determination section 424b. The adjustment coefficient and detailed processing contents of the attitude operation adjustment unit 424c will become apparent from the explanation given later.

[0032] <<Movement instruction output unit 424d of movement instruction calculation unit 424>> The movement instruction output unit 424d of the movement instruction calculation unit 424 is a component that calculates a movement instruction value based on a virtual operation amount (ΔP) that is an operation instruction and an adjustment coefficient. Here, the movement instruction value is defined as a control amount that indicates the movement amount and direction of the TCP and the posture change amount of the robot hand. Based on the movement instruction value calculated by the movement instruction output unit 424d, the control unit 420 controls each joint, and the command value as a result of the control of each joint is sent to the driving unit 110 of the robot 100 via the control board 430 (see FIG. 3). The detailed processing contents of the movement instruction output unit 424d will become apparent from the description given later.

[0033] As described above, the control unit 420 as a whole drives the drive unit 110 in accordance with the movement instruction value, and controls the position and posture of the hand of the robot 100.

[0034] (Regarding teaching work) One aspect of the disclosed technology employs a strategy in which, when the posture of the robot 100's hand changes due to operational instructions from a user during a teaching operation, the posture of the robot 100's hand after Δt has elapsed is a posture that satisfies as closely as possible the condition that "a workpiece is present within the measurement range of the object measurement means 310." Generally speaking, articulated robots have kinematic singularities, and they cannot be controlled unless they are avoided. In addition, the working range of articulated robots is limited.

[0035] FIG. 5 is an explanatory diagram showing how to set the attitude operation limiting condition based on the field of view (FoV) of the object measurement means 310. The FoV shown in FIG. 5 may be considered to represent the angle of view (AoV) of the object measurement means 310. constrain " is an angle that represents a region in which the robot 100 being taught is restricted so that the workpiece does not leave the field of view of the object measuring means 310 after Δt has elapsed, taking into account a safety factor. That is, θ constrainThe larger θ is set, the higher the safety factor will be. A partial range included in the measurement range of the object measurement means 310 and set on the periphery of the measurement range in consideration of the safety factor will be referred to as a "partial range" in this specification. In FIG. 5, this "partial range" is the range sandwiched between the solid line and the dashed line radially extending from the camera on the left side, in other words, θ constrain The range is the range enclosed by the two arrows shown near the arrow. That is, the posture manipulation restriction determination unit 424b of the movement instruction calculation unit 424 sets a "partial range" included in the measurement range of the object measurement means 310 in consideration of a safety factor, and determines whether or not to restrict the posture manipulation amount based on whether or not the workpiece evaluation position is present in the "partial range." The aforementioned attitude control limit conditions are determined by how to set this "partial range". For example, θ constrain are stored in advance in the storage unit 410 of the robot control device 400.

[0036] The first processing step performed in the teaching operation is acquisition of workpiece position information by the object position acquisition unit 300. The process of acquiring workpiece position information by the object position acquisition unit 300 is performed continuously and repeatedly during the teaching operation. As described above, the object position acquisition unit 300 is a component that acquires the position of an object present in the working space. The object present in the working space includes a workpiece. In other words, the things that the object position acquisition unit 300 judges as objects in the working space, or the objects that the object position acquisition unit 300 judges as workpieces, may be multiple or may be various objects, for example, various jigs. In this case, the object position acquisition unit 300 acquires position information for each of the multiple and various objects. Furthermore, specific workpiece position information is output when a workpiece is present within the measurement range of the object measurement means 310. When a workpiece is not present, the object measurement means 310 outputs information indicating that a workpiece is not present (such as a command signal to set a corresponding flag).

[0037] The next processing step performed in the teaching operation is calculation of a virtual operation amount (ΔP) by the driven control simulation calculation unit 422. As described above, the driven control simulation calculation unit 422 calculates the virtual operation amount (ΔP) based on the force detection information acquired by the force detection information acquisition unit 421. Fig. 6 is a block diagram for explaining how the follower control simulation calculation unit 422 in the robot control device 400 according to the first embodiment calculates a virtual operation amount (ΔP). Fig. 6A is a block diagram corresponding to the formula (1). Fig. 6B is a block diagram corresponding to the formula (3). As described above, Δt is the time interval. The virtual manipulated variable (ΔP) can be considered as an estimate of how much the position and posture of the robot 100 will be manipulated when Δt has elapsed from the current time if follower control is performed during the teaching task. The virtual operation amount (ΔP) is sent to the control unit 420 via the operation instruction acquisition unit 423. The movement instruction calculation unit 424 refers to the workpiece position information obtained by the object position acquisition unit 300 and the attitude operation restriction conditions, and calculates a movement instruction value from the virtual operation amount (ΔP), which is an operation instruction.

[0038] The specific calculation content of the movement instruction value by movement instruction calculation unit 424 will become apparent from the following description. As described above, the work virtual movement position calculation unit 424a of the movement instruction calculation unit 424 calculates the work virtual movement position based on the work position information at the current time and the virtual operation amount (ΔP) which is the operation instruction. Specifically, the work virtual movement position is given by the following formula. TIFF2025006026000006.tif27166 As shown in formula (5), the work virtual movement position is obtained by moving the work position information by the inverse vector of the virtual movement vector. Here, the virtual movement vector is a vector that expresses only the amount of change in position with respect to the amount of change in the position and attitude of the object measuring means 310 in response to the change in the position and attitude of the hand of the robot 100 due to the virtual operation amount (ΔP) which is an operation instruction. Although it will be repeated, the work virtual movement position is the position of the work as seen from the object measuring means 310 after the change in position when only the change in position is considered, even though the position and attitude of the object measuring means 310 also change when the position and attitude of the hand of the robot 100 changes due to the virtual operation amount (ΔP) which is an operation instruction. At this time, the change in the position of the work as seen from the object measuring means 310 is relative to the change in the position of the object measuring means 310, and can be expressed by the inverse vector of the virtual movement vector. Note that the work virtual movement position is a position coordinate expressed in the camera coordinate system, and is expressed by a three-dimensional vertical vector. Also, the work position information in formula (5) is expressed in the camera coordinate system. In equation (5), the prime or dash symbol (') attached to the variables emphasizes that the variables are virtual in the camera coordinate system.

[0039] The virtual movement vector appearing in formula (5) is a vector expressed in the camera coordinate system, but the virtual movement vector calculated based on the virtual operation amount (ΔP), which is the operation instruction, is expressed in the base coordinate system. The offset of the origin can be ignored in the coordinate transformation of the vector. As a result, the virtual movement vector is given by performing coordinate transformation using the following formula. TIFF2025006026000007.tif46166 However, R in formula (6) represents a rotation matrix of size 3 × 3. For example, camera R tool is a rotation matrix, which indicates the amount of rotation required to match the camera coordinate system to the tool coordinate system. tool R fixedis a rotation matrix, which indicates the amount of rotation required to align the tool coordinate system with the base coordinate system. The upper right subscript T in formula (6) indicates a transposition operation on the matrix.

[0040] The virtual movement vector (P work The square root of the sigma (with a hat symbol attached) is given by the following formula: TIFF2025006026000008.tif19166 The initial value P(0) appearing on the right side of equation (7) is the amount of change in position of the object measuring means 310 caused by a change in the posture of the robot 100, and is given by the following equation. TIFF2025006026000009.tif22166 Here, I appearing on the right side of the upper part of the formula (8) is a unit matrix of size 3 × 3. tool q camera is a vertical vector indicating the position (offset position) of the origin of the camera coordinate system as viewed from the tool coordinate system. Furthermore, R x (θ), R y (θ), and R z (θ) are the general rotation matrices around the x-axis, y-axis, and z-axis, respectively. R x (θ), R y (θ), and R z (θ) is given by the following formula: TIFF2025006026000010.tif20166 TIFF2025006026000011.tif18166 TIFF2025006026000012.tif17166 The order of multiplication of the rotation matrices is not limited to that shown in Equation (8). The order of multiplication of the rotation matrices may be other than that shown in Equation (8) as long as it matches the order of defining the attitude (for example, the order of roll, pitch, and yaw).

[0041] In general, the position and orientation of a certain coordinate system (B-coord) viewed from another coordinate system (A-coord) can be expressed by the following enlarged system (homogeneous coordinates). TIFF2025006026000013.tif33166 Here, the equation (12) A-coord T B-coord is called the Homogeneous Transformation Matrix.

[0042] In the embodiment shown in the first embodiment, the position and orientation of the object measuring means 310 as viewed from the TCP can be specified. That is, in the embodiment shown in the first embodiment, the homogeneous transformation matrix ( tool T camera ) can be specified. Note that the homogeneous transformation matrix ( fixed T tool ) can be calculated by solving forward kinematics based on the angles of each joint of the robot 100 and the link parameters stored in the memory unit 410. In addition, although the formula (7) and the formula (8) define the virtual operation amount (ΔP) which is an operation instruction in the base coordinate system, the disclosed technology is not limited to this. The virtual operation amount (ΔP) which is an operation instruction may be defined in any coordinate system as long as it can be calculated.

[0043] The next processing step performed in the teaching operation is a determination operation by the posture manipulation restriction determination unit 424b of the movement instruction calculation unit 424 as to whether or not to impose a manipulation restriction on the posture of the hand of the robot 100. The conditions for the determination performed by the posture manipulation restriction determination unit 424b are as shown in the table below. TIFF2025006026000014.tif60166Note that the work evaluation position in Table 1 is the work virtual movement position. As described above, the FoV of the object measuring means 310 is assumed to be known as a specification and stored in the memory unit 410.

[0044] The next processing step performed in the teaching operation is the calculation of an adjustment coefficient (λ) by the attitude operation adjustment unit 424c of the movement instruction calculation unit 424. The adjustment coefficient (λ) is given by, for example, the following formula. TIFF2025006026000015.tif14166 Here, λ const is a predetermined constant parameter and is stored in the storage unit 410. const The smaller is the smaller the operation restriction will be.

[0045] The next processing step performed in the teaching operation is calculation of a movement instruction value by the movement instruction output unit 424d of the movement instruction calculation unit 424. The movement instruction value (Ref with a hat added) is specifically given by the following formula. TIFF2025006026000016.tif31166 The movement instruction value (Ref with a hat) calculated by formula (14) is a control amount indicating the amount of movement and direction of the TCP in the base coordinate system and the amount of change in the posture of the robot hand, so by determining the movement instruction value, the command value for controlling the posture of the TCP and the robot hand is determined. The control unit 420 controls the posture of the TCP and the robot hand so that they match this command value, and the result of this control is output to the driving unit 110 via the control board 430, and the position and posture of the robot 100 are controlled.

[0046] For example, when the movement command value is calculated by the formula (14), first, the posture of the TCP and the robot hand after Δt is calculated from the current value of the TCP, the posture of the robot hand, and the movement command value, and this is set as the command value of the posture of the TCP and the robot hand. Next, the angle of each joint is calculated so that the posture of the TCP and the robot hand coincides with the command value. This calculation can be performed by known inverse kinematics. Finally, the driving unit 110 is controlled so that the angle of each joint coincides with the calculated angle. This control is usually performed by a device that controls and drives a motor, called a motor driver, which is included in the control board 430 or is integrated with the driving unit 110. The above procedure realizes the control of the driving unit 110 based on the movement command value. Note that the control of the driving unit 110 is not limited to the above procedure, and it may be controlled via the angular velocity of each joint or the torque generated by the motor.

[0047] A technical feature unique to the robot system 1000 according to the first embodiment is that it employs a strategy in which, when the posture of the hand of the robot 100 changes due to an operation instruction from a user during a teaching operation, the posture of the hand of the robot 100 after Δt has elapsed is a posture that satisfies the condition that "a workpiece is present within the measurement range of the object measuring means 310" as much as possible. In order to employ this strategy, the movement instruction calculation unit 424 of the control unit 420 includes a workpiece virtual movement position calculation unit 424a that calculates the position of the workpiece after Δt as seen from the object measuring means 310 (camera), a posture operation restriction determination unit 424b that determines whether or not to restrict the posture operation amount of the virtual operation amount (ΔP) that is the operation instruction, a posture operation adjustment unit 424c that calculates an adjustment coefficient (λ) that adjusts the movement instruction value, and a movement instruction output unit 424d that calculates the movement instruction value that realizes the above strategy.

[0048] Because of the above configuration, the robot system 1000, the robot 100, and the robot control method according to the first embodiment have the advantage of controlling the posture operation during the teaching operation so that the workpiece is unlikely to deviate from the measurement range. Due to this advantage, the robot system 1000, the robot 100, and the robot control method according to the first embodiment have the advantage of being able to easily teach the posture of the hand of the robot 100 during measurement, regardless of the operation method.

[0049] The disclosed technology is also effective for a robot system 1000 in which sensors are attached to the arms, hands, and fingers of a human instructor and the instructor's movements are simulated in real time based on signals obtained from the sensors. The disclosed technology is also effective for master-slave type robot manipulators intended for robot-assisted surgery (especially endoscopic surgery). In this case, the object described as a "workpiece" is replaced with, for example, a tumor or a vital organ to be removed in surgery, and the disclosed technology is applied. Furthermore, the disclosed technology is also effective for a robot system 1000 that simulates the movements of a human instructor in real time by placing markers on the arms, hands, and fingers of the instructor and tracking them. Such a technology is known as one type of motion capture technique.

[0050] Embodiment 2 The robot system 1000, the robot 100, and the robot control method according to the second embodiment are modified examples of the robot system 1000, the robot 100, and the robot control method according to the present disclosure. In the second embodiment, the same reference symbols as those used in the first embodiment are used unless otherwise specified. In the second embodiment, descriptions that overlap with those in the first embodiment are omitted as appropriate.

[0051] Fig. 7 is a block diagram showing a detailed functional configuration of posture operation adjustment section 424c in movement instruction calculation section 424 according to embodiment 2. As shown in Fig. 7, posture operation adjustment section 424c in movement instruction calculation section 424 according to embodiment 2 includes a gaze point virtual movement direction calculation section 424c-1, a gaze point allowable movement range calculation section 424c-2, and an adjustment coefficient calculation section 424c-3. In this manner, the concept of a "gazing point" is introduced in the robot system 1000, the robot 100, and the robot control method according to the second embodiment.

[0052] The fixation point introduced in the second embodiment is a point indicating the measurement direction of the object measurement means 310 (camera). In general, there are an infinite number of points indicating the measurement direction of the object measurement means 310 (camera). Therefore, in this specification, the fixation point is defined as a point that indicates the direction of the object measurement means 310 (camera) and whose distance from the pinhole of the object measurement means 310 (camera) is w (in FIG. 8, w=Z′ work ) is defined as a point where. Details of FIG. 8 will become clear from the explanation given later.

[0053] <<The gaze point virtual movement direction calculation unit 424c-1 constituting the attitude operation adjustment unit 424c>> The gaze point virtual movement direction calculation unit 424c-1, which constitutes the posture operation adjustment unit 424c, is a component that calculates the virtual movement direction of the gaze point (hereinafter referred to as the ``gaze point virtual movement direction'') based on the virtual operation amount (ΔP), which is an operation instruction calculated by the follower control simulation calculation unit 422. As described above, the gaze point represents the orientation of the object measurement means 310 (camera) and is a point whose distance from the pinhole of the object measurement means 310 (camera) is w. Therefore, in the measurement data obtained by the object measurement means 310 (camera) (see FoV in FIG. 8), the gaze point always appears in the center. The virtual movement direction of the gaze point is the displacement caused by the virtual movement of the gaze point when it is assumed that the position and posture of the hand of the robot 100 are controlled based on a virtual operation amount (ΔP), which is an operation instruction.

[0054] <<The gaze point allowable movement range calculation unit 424c-2 constituting the attitude operation adjustment unit 424c>> The gaze point allowable movement range calculation unit 424c-2 constituting the posture operation adjustment unit 424c is a component that calculates the gaze point allowable movement range based on the work evaluation position. As described above, the work evaluation position is calculated based on the work virtual movement position calculated by the work virtual movement position calculation unit 424a, the work position information obtained by the object position acquisition unit 300, or the work position information and the work virtual movement position. The gaze point allowable movement range, as its name suggests, means a movement range allowed for the gaze point. Specifically, the gaze point allowable movement range is given as an angle range. The specific processing content for obtaining the gaze point allowable movement range will become clear from the explanation below.

[0055] <<Adjustment coefficient calculation unit 424c-3 constituting the attitude manipulation adjustment unit 424c>> The adjustment coefficient calculation unit 424c-3 constituting the posture operation adjustment unit 424c is a component that calculates an adjustment coefficient based on the gaze point virtual movement direction and the gaze point allowable movement range. The details of calculating the adjustment coefficient will become clear from the explanation given later. The adjustment coefficient calculated by the adjustment coefficient calculation unit 424c-3 is sent to the movement instruction output unit 424d.

[0056] (Details of the calculation procedure for the adjustment coefficient) 7, the first step in the process of calculating the adjustment coefficient is the calculation of the gaze point virtual movement direction by the gaze point virtual movement direction calculation unit 424c-1. The gaze point virtual movement direction is finally given in the camera coordinate system by the following procedure. In the following explanation, the camera coordinate system is defined as having an origin at the pinhole of the object measurement means 310 (camera), the Z axis being the positive direction in the measurement direction of the object measurement means 310 (camera), the X axis being the positive direction from left to right when facing the measurement direction from the object measurement means 310 (camera), and the Y axis being the positive direction from top to bottom when facing the measurement direction. That is, the explanation will be given assuming that the X-axis, Y-axis, and Z-axis arrows in the upper part of Fig. 8 coincide with the positive directions of each axis of the camera coordinate system. The amount of change in posture of the hand of the robot 100 (hereinafter referred to as "hand posture change") is given by ΔR in formula (8), that is, the rotation matrix. The amount of change in posture of the hand (ΔR) is given by the following formula in the camera coordinate system. TIFF2025006026000017.tif18166 Here, the formula (15) camera R fixed is the rotation matrix, which indicates the amount of rotation required to align the camera coordinate system with the base coordinate system. In the left side of (15), the prime or dash symbol (′) attached to the variables emphasizes that the variables are expressed in the camera coordinate system. These conventions in (15) are the same as those in (5) and (6).

[0057] Virtual movement direction of gaze point (P viewpoint Specifically, the above equation is given by the following formula: TIFF2025006026000018.tif43166 Here, w in Equation (16) is the distance from the pinhole of the object measuring means 310 (camera) to the gaze point. w may be set to 1, or Z′ work In addition, r11 to r23 in the formula (16) are the corresponding elements of the matrix ΔR′. In formula (16), the matrix multiplied from the left side of ΔR′ determines the virtual movement direction of the gaze point (P viewpointThe reason why the value of the third component of (a hat is placed on) is 0 (zero) is that the point of gaze is defined as a point whose distance from the pinhole of the object measurement means 310 (camera) is w (see paragraph

[0053] ), and the Z coordinate of the point of gaze in the camera coordinate system is fixed, which reflects the fact that the Z coordinate of the point of gaze in the camera coordinate system remains constant even if the point of gaze moves.

[0058] Although the formulas (15) and (16) are explanations using mapping from the base coordinate system to the camera coordinate system, the disclosed technology is not limited to this. Finally, the virtual moving direction of the gaze point in the camera coordinate system (P viewpoint It doesn't matter what coordinate system you start from, as long as you can get a coordinate system that gives you a coordinate system of 1000 x 1000 (with a hat added).

[0059] 8 is an explanatory diagram showing the relationship between the field of view (FoV) of the object measuring means 310 used in the robot system 1000 according to the second embodiment and a work virtual position plane. work ,Y′ work ,Z′ work 8) and perpendicular to the measurement direction of the object measuring means 310 (camera). In FIG. 8, the work virtual position plane corresponds to the plane described as the XY plane. The fixation point virtual movement direction calculated by the fixation point virtual movement direction calculation unit 424c-1 and the allowable reference vector calculated by the fixation point allowable movement range calculation unit 424c-2 are defined on this work virtual position plane. In FIG. 8, the XY plane corresponds to the plane described as the XY plane. width is the width that can be measured by the object measuring means 310 (camera) on the work virtual position plane, and Y height Similarly, (X viewpoint , Y viewpoint ) is the gaze point, which is located at the center of the FoV on the work virtual position plane. In the camera coordinate system, X viewpoint , Y viewpoint Both will be 0. In addition, the point where several lines intersect in the lower left of Fig. 8 represents the pinhole of the object measuring means 310 (camera). In addition, in this specification, the description will be given assuming the ideal pinhole camera model shown in Fig. 8.

[0060] As shown in FIG. 7, the second step in the process of calculating the adjustment coefficient is calculation of the gaze point allowable movement range by the gaze point allowable movement range calculation unit 424c-2. Specifically, the gaze point allowable movement range calculation unit 424c-2 calculates the gaze point allowable movement range by introducing an "allowable reference vector" and an "allowable angle range". Fig. 9 is an explanatory diagram showing the allowable reference vector and allowable angle range introduced in the robot system 1000 according to the embodiment 2. Specifically, Fig. 9 shows a measurement image obtained by the object measurement means 310 (camera) at a certain time. As shown in FIG. 9, the allowable reference vector is given as a position vector from the gaze point at the center of the measurement image as the origin to the workpiece. As shown in FIG. 9, the allowable angle range is an angle (θ P ) is given as θ P is a parameter that can be preset by the user and can be stored in the storage unit 410, for example.

[0061] As described above, when the position and posture of the hand of the robot 100 changes, the state of the workpiece does not actually change, but in the measurement image, the workpiece appears as if it has moved. The homogeneous transformation matrix from the coordinates in the base coordinate system to the coordinates in the camera coordinate system is obtained as follows by the relationship shown in equation (12) and the homogeneous transformation matrix from the coordinates in the camera coordinate system to the coordinates in the base coordinate system. TIFF2025006026000019.tif53166 As in Equation (6), the upper right subscript T in Equation (17) represents a transposition operation on a matrix. As shown in Equation (17), the inverse transformation of a homogeneous transformation is given by the inverse matrix of the homogeneous transformation matrix.

[0062] Ultimately, the teaching operation should bring the position of the workpiece closer to the central gaze point in the measurement image. If we consider the gaze point as the subject, the teaching operation should bring the gaze point closer to the workpiece. In other words, the gaze point virtual movement direction should roughly match the direction of the permissible reference vector. Thus, the "permissible" in the names of "permissible reference vector" and "permissible angle range" means that it is permissible with respect to the gaze point virtual movement direction.

[0063] The allowable reference vector in the camera coordinate system (P reference Specifically, the above equation is given by the following formula: TIFF2025006026000020.tif14166 X′ in formula (18) work and Y′ work are the X and Y components of the virtual workpiece movement position given by formula (5). The allowable reference vector is given as a position vector to the workpiece with the gaze point at the center of the measurement image as the origin, but since the X and Y coordinates of the gaze point are 0 in the camera coordinate system, the X and Y components of the virtual workpiece movement position become the components of the allowable reference vector as they are. The reason why the value of the third component in equation (18) is 0 (zero) is because the allowable reference vector is defined on the workpiece virtual position plane, and the measurement image obtained by the object measurement means 310 (camera) is perceived as an image on the workpiece virtual position plane.

[0064] 7, the third step in the process of calculating the adjustment coefficient is a main process in which adjustment coefficient calculation unit 424c-3 actually calculates the adjustment coefficient. The adjustment coefficient (λ) calculated by adjustment coefficient calculation unit 424c-3 is given by, for example, the following formula. TIFF2025006026000021.tif31166 However, θ appearing in the latter part of equation (19) diff is the virtual movement direction of the gaze point (P viewpoint (with a hat) and the allowable criterion vector (P reference θ is the angle between the diff Specifically, it is given by the following formula: TIFF2025006026000022.tif17166 Here, the dot in the numerator on the right side of formula (20) represents the inner product of vectors. Also, the symbol in the denominator on the right side of formula (20) represents a norm. The norm in formula (20) is the L2 norm, which is the Euclidean distance. θ diff is expressed as the arccosine (cos -1 ), so the range is 0 to π (0≦θ diff ≦π). In addition, the arccosine function (cos -1 ) is expressed as a fraction, but cases where division by zero occurs, where the denominator of this fraction is 0 (zero), correspond to cases where the work evaluation position overlaps with the gaze point, and where there is no change in posture due to rotation around the gaze point direction vector. In this case, since there is no need to add any restrictions, cases where division by zero occurs will not be a problem if appropriate processing is performed in the program.

[0065] The parameter λ newly introduced in the second embodiment permit is the virtual movement direction of the gaze point (P viewpoint (with a hat) and the allowable criterion vector (P reference The angle (θ diff ) is θ P and the virtual movement direction of the gaze point is outside the allowable angle range, the attitude control is restricted.

[0066] λ in Equation (19)const is λ in Equation (13) of the first embodiment. const The possible range is slightly different from that of λ , and includes 1. This difference in the possible range is due to the fact that λ permit This is caused by the existence of λ. const If we set =1, then λ becomes permit If it is guaranteed that the virtual workpiece movement position is within the measurement range, λ permit By itself, λ const Of course, this is because the formula for determining the adjustment coefficient (λ) is different between the second embodiment and the first embodiment, and this is limited to the formula (19) that explains the second embodiment. In the formula (13) of the first embodiment, λ const = 1. In the formula (13) of the first embodiment, λ const The adjustment coefficient (λ) for limiting operation is determined by only λ const This is because operation restrictions cannot be implemented if θ = 1. Since the virtual workpiece movement position always exists within the measurement range, various methods can be considered, such as imaginary springs and dampers for the interface with the environment, switching the stiffness of the robot 100 according to the virtual workpiece movement position, damping control, impedance control, stiffness control, etc.

[0067] A configuration feature unique to the robot system 1000 of embodiment 2 is that the posture operation adjustment unit 424c of the movement instruction calculation unit 424 includes a gaze point virtual movement direction calculation unit 424c-1, a gaze point allowable movement range calculation unit 424c-2, and an adjustment coefficient calculation unit 424c-3. The technical strategy specific to the robot system 1000 according to the second embodiment is to reinterpret the strategy adopted by the present disclosure technology by introducing the concept of a "gazing point". As described above, the strategy adopted by the present disclosure technology is that the posture of the hand of the robot 100 after Δt has elapsed is a posture that satisfies the condition that "a workpiece is present within the measurement range of the object measuring means 310" as much as possible. Therefore, in the second embodiment, it is interpreted that it is sufficient if the position of the workpiece in the measurement image approaches the central gaze point through a teaching operation. If the gaze point is considered as the main subject, it is sufficient if the gaze point approaches the workpiece through a teaching operation. The technical feature unique to the robot system 1000 according to the second embodiment is the parameter λ related to the adjustment coefficient. permit The point is that the parameter λ is newly introduced in the second embodiment. permit is the virtual movement direction of the gaze point (P viewpoint (with a hat) and the allowable criterion vector (P reference The angle (θ diff ) is θ P and the virtual movement direction of the gaze point is outside the allowable angle range, the attitude control is restricted.

[0068] Because of the above configuration, the robot system 1000, the robot 100, and the robot control method according to the second embodiment have the advantage of controlling the posture operation during the teaching operation so that the workpiece is unlikely to deviate from the measurement range. Due to this advantage, the robot system 1000, the robot 100, and the robot control method according to the second embodiment have the advantage of being able to easily teach the posture of the hand of the robot 100 during measurement, regardless of the operation method.

[0069] In the second embodiment, θ diff θ P By comparing with λ permit (For example, see Equation (19) and Equation (20)), but the disclosed technology is not limited to this. viewpoint(with a hat) and the allowable criterion vector (P reference Based on the similarity between the vectors, e.g., the cosine similarity (equal to the argument of the arccosine function in Equation (20)), λ permit may be determined.

[0070] Embodiment 3 The robot system 1000, the robot 100, and the robot control method according to the third embodiment are modified examples of the robot system 1000, the robot 100, and the robot control method according to the present disclosure. In the third embodiment, unless otherwise specified, the same reference symbols as those used in the previous embodiments are used. Furthermore, in the third embodiment, descriptions that overlap with those in the previous embodiments are omitted as appropriate.

[0071] 10 is an explanatory diagram showing an allowable reference vector and an allowable angular range in a robot system 1000 according to embodiment 3. In embodiment 3, the allowable reference vector is the same as that shown in embodiment 2. In addition, in embodiment 3, the allowable angular range is set independently for each side of the allowable reference vector.

[0072] Here, the measurement range of the object measurement means 310, that is, FoV, is the workpiece virtual position plane (Z coordinate value in the camera coordinate system is Z′ work In the xy plane fixed at , it is given by the following formula: TIFF2025006026000023.tif33166 Where X max is the maximum value of the X component, and X min is the minimum value of the X component, and Y is max is the maximum value of the Y component, and Y min In this way, the measurement range of the object measurement means 310, that is, FoV, has a width of X and Y on the work virtual position plane as shown in FIG. width , height Y height The coordinates of the gaze point located at the center of the FoV (X viewpoint, Y viewpoint ) is 0 in the camera coordinate system, and X max and X min is width X width Y max and Y min is the height Y height It is determined by half the value of .

[0073] Now, while the concept of the fixation point virtual movement direction was introduced in the second embodiment, the concept of "the shortest point on the FoV boundary from the work evaluation position" is introduced in the third embodiment. Basically, the shortest vector pair is extracted for each of the X and Y directions. Exceptionally, there are cases where a pair is extracted for only the X or Y direction. The following four directional vectors are candidates for the vector that represents the "shortest point on the FoV boundary from the work evaluation position." These four directional vectors are expressed in the camera coordinate system on the work virtual position plane. Therefore, all Z components are 0. TIFF2025006026000024.tif38166 Here, P work (t) represents the position of the reference point of the workpiece expressed in the camera coordinate system at the current time (t). The reference point of the workpiece may be the workpiece evaluation position expressed in the above-mentioned TCP. The reference point of the workpiece is found from the workpiece position information obtained by the object position acquisition unit 300, as shown in formula (5) described in the first embodiment. In formula (22), the subscript "POS" at the bottom right indicates that the direction vector is positive, and the subscript "NEG" at the bottom right indicates that the direction vector is negative. In FIG. 10, the positive direction of X is defined as the right direction, and the positive direction of Y is defined as the downward direction. In the example shown in FIG. 10, the distance from the reference point of the workpiece is d X_NEG And, d Y_NEG and are depicted with arrows.

[0074] The vector representing "the shortest point on the FoV boundary from the position of the workpiece reference point" is given by the following formula, separated into X and Y components. TIFF2025006026000025.tif14166 TIFF2025006026000026.tif13166 FIG. 11 is an explanatory diagram showing the allowable reference vector and the allowable angle range in the robot system 1000 according to the third embodiment (for a special case). FIG. 12 is an explanatory diagram showing the shortest distance vector actually adopted in the special case shown in FIG. 11. In this case, d X_POS and d X_NEG Since the magnitudes of are equal, d X_POS and d X_NEG The pair of vectors d is selected as the shortest distance vector. Y_POS and d Y_NEG If the magnitudes of are equal, then d Y_POS and d Y_NEG The pair of vectors is selected as the shortest distance vectors.

[0075] In the third embodiment, θ p The following two angles (θ1, θ2) are introduced as candidates. TIFF2025006026000027.tif14166 TIFF2025006026000028.tif13166 θ1 and θ2 are angles formed by the pair of the allowable reference vector and the shortest distance vector, respectively. θ1 is expressed as the arccosine (cos -1 ), so in general, the range is 0 to π (0≦θ1≦π). Similarly, θ2 is given by the arccosine (cos -1), so generally, the range is 0 to π (0≦θ2≦π). However, since the vector (V1, V2) that represents "the shortest point on the FoV boundary from the position of the work reference point" is selected as shown in formula (23) and formula (24), the range of θ1 is 0 to π / 2 (0≦θ1≦π / 2), and the range of θ2 is 0 to π / 2 (0≦θ2≦π / 2). In addition, the arccosine function (cos -1 Although the argument of () is expressed as a fraction, the case of division by zero, where the denominator of the fraction is 0 (zero), is not mentioned in this specification.

[0076] First, a rotation direction for selecting the permissible movement range is obtained from the cross product of the permissible reference vector and the gaze point virtual movement direction vector. Next, the rotation direction is derived from the cross product of each pair of the allowable reference vector and the shortest distance vector, and the shortest distance vector having the same rotation direction as the gaze point virtual movement direction vector is selected. Then, the allowable angle range (θp) is determined based on the selected shortest distance vector. The determination using θp is the same as that described in the second embodiment. The cross product of vectors is calculated to be the same for the same vector and the inverse vector. Since we want to restrict the attitude control only in the case of the inverse vector, when the cross product is a zero vector, θ p Set θ to π. diff is 0 in the case of the same vector and is π in the case of the opposite vector, so the desired operation can be achieved by using equation (19). In the following, the description will be divided into four quadrants, from the first quadrant to the fourth quadrant, based on the positive and negative X and Y coordinate values ​​of the workpiece reference point in the camera coordinate system. These quadrants correspond to the areas at the lower right, lower left, upper left, and upper right, respectively, as viewed from the gaze point in the measurement image obtained by the object measurement means 310 (camera). Except for the special case where the workpiece reference point is on the center line of the measurement range (the special case where it does not belong to any quadrant but is on the boundary of the quadrants), the origin can be taken at the intersection (i.e., the center) of two center lines in the measurement range, and the four cases can be considered by dividing the first quadrant into the fourth quadrant. TIFF2025006026000029.tif121166Note that in Table 2, the camera coordinate system is a right-hand screw system. The symbol "x" in Table 2 represents the cross product used for vectors. Furthermore, the matrix T in Table 2 is an operation matrix for extracting the Z component from the elements of the vertical vector. The strategies in Table 2 show additional strategies for each quadrant in addition to the strategies in the second embodiment. That is, the virtual workpiece movement position is restricted from an operation that moves the virtual workpiece movement position away from the gaze point, and at the same time, the virtual workpiece movement position is made to remain in each quadrant. In other words, the third embodiment adds a strategy that restricts an operation that moves the virtual workpiece movement position to another quadrant. The "Allowable Movement Range (θ P )" is described by distinguishing between cases depending on whether, as viewed from the allowable reference vector, the virtual workpiece movement position is a position where "the direction in which a right-handed screw advances is the positive direction of the Z axis" or the virtual workpiece movement position is a position where "the direction in which a right-handed screw advances is the negative direction of the Z axis". Note that this specification does not mention the singular case in which the two vectors have the same direction and the result of the cross product is a zero vector.

[0077] In addition, in the formula (23) and the formula (24), when the magnitude of the positive direction vector and the negative direction vector are equal, the positive direction vector is selected. However, the disclosed technology is not limited to this. When the reference point of the work does not belong to any quadrant but is on the boundary of the quadrants, a strategy may be adopted in which the virtual work movement position stays in one of the adjacent quadrants. In addition, when the reference point of the work is at the gaze point (origin), it is considered that the position and posture of the robot 100 are in a state where the purpose of the teaching work is achieved, but it is sufficient that the virtual work movement position stays in the FoV.

[0078] The technical feature unique to the robot system 1000 according to the third embodiment is the allowable movement range (θ P) is dynamically set. As described above, the strategy adopted by the disclosed technology is to set the posture of the hand of the robot 100 after Δt has elapsed to a posture that satisfies, as much as possible, the condition that "a workpiece is present within the measurement range of the object measurement means 310." Therefore, in the second embodiment, it is conceived that it is sufficient to bring the gaze point closer to the workpiece through a teaching operation. In the third embodiment, in order to adopt a strategy that satisfies the condition more strictly, the allowable movement range (θ P ) has been shown to change

[0079] Considering the work as the main subject, the direction in which the distance from the work evaluation position to the FoV boundary is the longest is the direction with the most room for movement. If we focus on a certain straight line, we can say that the direction in which the distance to the FoV boundary is the longest among the straight lines passing through the work evaluation position is the direction with the most room for movement. According to the strategy adopted by the disclosed technology, this straight line is a straight line parallel to the FoV boundary. Also, considering the gaze point as the main subject, the direction with the most room for movement is the direction in which the distance to the FoV boundary is the shortest, which is the shortest distance vector in the third embodiment.

[0080] Because of the above configuration, the robot system 1000, the robot 100, and the robot control method according to the third embodiment have the advantage of controlling the posture operation during the teaching operation so that the workpiece is unlikely to deviate from the measurement range. Due to this advantage, the robot system 1000, the robot 100, and the robot control method according to the third embodiment have the advantage of being able to easily teach the posture of the tip of the robot 100 during measurement, regardless of the operation method.

[0081] Embodiment 4 The robot system 1000, the robot 100, and the robot control method according to the fourth embodiment are modified examples of the robot system 1000, the robot 100, and the robot control method according to the present disclosure. In the fourth embodiment, unless otherwise specified, the same reference symbols as those used in the previous embodiments are used. Furthermore, in the fourth embodiment, descriptions that overlap with those in the previous embodiments are omitted as appropriate.

[0082] In the first embodiment, as shown in the formula (13), when the attitude operation restriction determination unit 424b determines that no operation restriction is to be applied, the adjustment coefficient (λ) is fixed to 1, and when the attitude operation restriction determination unit 424b determines that an operation restriction is to be applied, the adjustment coefficient (λ) is fixed to an arbitrary constant. That is, the operation feel changes suddenly in an operation that crosses the boundary between the presence and absence of an operation restriction (the boundary of a partial range). Therefore, it is desirable to change the adjustment coefficient (λ) continuously or stepwise, taking into consideration how close the work evaluation position is to the boundary of the partial range.

[0083] As an example of continuously changing the adjustment coefficient (λ) according to the distance between the work evaluation position and the boundary line of the partial range, the following formula can be considered. TIFF2025006026000030.tif13166 The adjustment coefficient λ given by formula (27) becomes smaller as the work evaluation position approaches the boundary line of the partial range. const The adjustment coefficient λ expressed by formula (27) linearly varies from 1 to λ depending on the distance from the gaze point to the boundary line of the partial range on the line connecting the work evaluation position and the gaze point. const This is an example where λ const is a value less than 1.

[0084] As described above, various methods can be considered as a means for achieving an attitude that satisfies as much as possible the condition that "the workpiece is present within the measurement range of the object measurement means 310", such as, for example, assuming a virtual spring / damper at the interface with the environment, switching the stiffness of the robot 100 according to the position of the reference point of the workpiece, damping control, impedance control, stiffness control, etc.

[0085] The robot system 1000, the robot 100, and the robot control method according to the fourth embodiment can continuously change the adjustment coefficient (λ), so that the operational feel does not change suddenly during the teaching work. That is, as the work evaluation position approaches the boundary line of the partial range, the operational feel becomes gradually more restricted, which further provides the effect that the operational feel changes smoothly as the work evaluation position moves.

[0086] In the above, when the operation restriction is performed by the attitude operation restriction determination unit 424b, the adjustment coefficient (λ) is set to a constant value λ const However, the disclosed technology is not limited to this. In the disclosed technology, even if the operation restriction is performed by the attitude operation restriction determination unit 424b, the adjustment coefficient (λ) may be changed continuously according to the position of the reference point of the work.

[0087] Embodiment 5. The robot system 1000, the robot 100, and the robot control method according to the fifth embodiment are modified examples of the robot system 1000, the robot 100, and the robot control method according to the present disclosure. In the fifth embodiment, unless otherwise specified, the same reference symbols as those used in the previous embodiments are used. Furthermore, in the fifth embodiment, descriptions that overlap with those in the previous embodiments are omitted as appropriate.

[0088] In the second and third embodiments, as shown in the formula (19), λ permit A fixed value is used. Like the continuously changing adjustment coefficient (λ) in the fourth embodiment, λ permitThe continuously changing λ may also be permit is given, for example, by the following formula: TIFF2025006026000031.tif17166 λ given by equation (28) permit θ diff The magnitude of θ P As it approaches , it approaches 0. λ shown by equation (28) permit θ diff This is an example where the value changes linearly from 1 to 0 depending on the magnitude of .

[0089] The robot system 1000, the robot 100, and the robot control method according to the fifth embodiment are permit Since the angle can be changed continuously, there is a further effect that an operation across the allowable angle range can be smoothly performed in a teaching operation.

[0090] Embodiment 6 The robot system 1000, the robot 100, and the robot control method according to the sixth embodiment are modified examples of the robot system 1000, the robot 100, and the robot control method according to the present disclosure. In the sixth embodiment, unless otherwise specified, the same reference symbols as those used in the previous embodiments are used. In the sixth embodiment, descriptions that overlap with those in the previous embodiments are omitted as appropriate.

[0091] In the fifth embodiment, the formula (28) is exemplified, and λ permit It was shown that can be varied linearly. In the sixth embodiment, λ permit By setting more finely for λ permit This shows that the can change like a stepped broken line.

[0092] FIG. 13 is an explanatory diagram showing an allowable reference vector and an allowable angular range introduced in a robot system 1000 according to the sixth embodiment. As shown in FIG. 13, in the sixth embodiment, a first angle (θ P,1 ), and a second angle (θ P,2 ) is defined.

[0093] The first angle (θ P,1 ), and a second angle (θ P,2 ), λ permit is given, for example, by the following formula: TIFF2025006026000032.tif28166 λ given by equation (29) permit θ diff The magnitude of the first angle (θ P,1 ) changes from 1 to 0.4, and θ diff The magnitude of the first angle (θ P,1 ) to the second angle (θ P,2 ), the change is from 0.4 to 0. In this specification, such a change is referred to as a "broken line change." Note that 0.4 is merely a value used as an example and has no other meaning.

[0094] The broken line change shown in equation (29) is from 1 to 0.4 and from 0.4 to 0, and there is only one connecting point (0.4) between the broken lines. However, by increasing the number of connecting points, equation (29) can be generalized. permit A generalization of is given by the following formula: TIFF2025006026000033.tif18166 Here, θ appearing in equation (30) P,n (n=1, 2, ..., N) are the n-th angles (n=1, 2, ..., N) in the allowable angle range. Also, λ n(n=1, 2, ..., N) are the adjustment coefficients (n=1, 2, ..., N) at the nth angle in the allowable angle range. P,0 = 0, and θ P,n-1 <θ P,n Each of the n-th angles is set so that the following relationship is satisfied. Similarly, λ n <λ n-1 While satisfying the relationship, 0≦λ permit The adjustment coefficients are set so that the n-th angle (n=1, 2, ..., N) related to these allowable angle ranges and the adjustment coefficients at the n-th angle (n=1, 2, ..., N) are stored in advance in storage unit 410.

[0095] The disclosed technique does not directly store the n-th angle (n=1, 2, . . . , N) related to the allowable angle range in the storage unit 410, but stores "θ P The idea of ​​dividing the n , n=1, 2, ..., N) may be stored in the storage unit 410. n ) is given by the following formula: TIFF2025006026000034.tif13166

[0096] The robot system 1000, the robot 100, and the robot control method according to the sixth embodiment are permit Since the angle can be changed continuously in a broken line, it is possible to provide a directional operation feeling, for example, by making it easier to move the point of attention in a more permissible reference vector direction within the permissible angle range.

[0097] Embodiment 7 The seventh embodiment is an aspect in which the disclosed technology is applied to a technical field that is slightly different from the technical fields described in the previous embodiments.

[0098] Incidentally, gimbals and stabilizers are known as techniques for preventing camera shake. A gimbal is a mechanism that uses a rotating platform that drives around an internal axis, and mechanical and electric gimbals are known. Electric gimbals are also used in so-called drones (see, for example, JP 2019-140536 A). The disclosed technology can also be applied to a motorized gimbal by replacing the object referred to as the workpiece in the above-described embodiments with an object to be photographed, thereby realizing a more intelligent motorized gimbal. [Industrial Applicability]

[0099] The disclosed technology can be applied to machine vision systems equipped with computer vision and capable of direct teaching, master-slave type robot manipulators, and even motorized gimbals, and has industrial applicability. [Explanation of symbols]

[0100] 100 robot, 200 force detection unit, 300 object position acquisition unit, 310 object measurement means, 320 object position estimation unit, 400 robot control device, 410 memory unit, 420 control unit, 421 force detection information acquisition unit, 422 follower control simulation calculation unit, 423 operation instruction acquisition unit, 424 movement instruction calculation unit, 424a work virtual movement position calculation unit, 424b attitude operation restriction determination unit, 424c attitude operation adjustment unit, 424c-1 gaze point virtual movement direction calculation unit, 424c-2 gaze point allowable movement range calculation unit, 424c-3 adjustment coefficient calculation unit, 424d movement instruction output unit, 430 drive unit, 1000 robot system.

Claims

1. A robot control device for controlling a robot, The robot is provided with an object measuring means at a location corresponding to a hand, Only when a workpiece is present within the measurement range of the object measurement means, a restriction may be imposed on the robot so that the posture of the robot is not changed in response to an operation instruction given to the robot. Robot control device.

2. applying a restriction so as not to change the posture of the robot according to the position of the workpiece in the measurement range; The robot control device according to claim 1 .

3. The measurement range is defined as a partial range including a boundary taking into account a safety factor; When the workpiece is present in the partial range, a restriction is imposed so as not to change the posture of the robot. The robot control device according to claim 1 or 2.

4. The restriction on not changing the posture of the robot changes stepwise or continuously depending on the position of the workpiece in the measurement range. The robot control device according to claim 2 .

5. The partial range is defined based on an angle of view of the object measuring means. The robot control device according to claim 3.

6. The apparatus further includes an object position estimation unit that estimates a position and a posture of the workpiece based on the measurement data acquired by the object measurement means. The robot control device according to claim 1 .

7. A robot and a robot control device, The robot is provided with an object measuring means at a location corresponding to a hand, the robot control device may restrict the robot from changing its posture in response to an operation instruction given to the robot only when a workpiece is present within a measurement range of the object measurement means; Robot system.

8. A robot control method for controlling a robot, comprising: The robot is provided with an object measuring means at a location corresponding to a hand, Only when a workpiece is present within the measurement range of the object measurement means, a restriction may be imposed on the robot so that the posture of the robot is not changed in response to an operation instruction given to the robot. A method for controlling a robot.

Images