Information processing device, industrial device control system and information processing method

Abstract

To provide an information processing device, an industrial device control system and an information processing method which can improve efficiency in setting information necessary for processing an industrial device such as a robot arm more than before.SOLUTION: An information processing device (200) is provided with: an imaging information obtaining unit (203) that obtains imaging information on imaged marks (C1 and C2) displayed on a surface of a target object (W1), on the basis of a signal from an imaging unit (201) that images the target object W1 that an industrial device handles; a position information obtaining unit (205) that obtains position information showing a position of a surface; a direction information obtaining unit (204) that obtains direction information showing a direction of the surface, on the basis of the imaging information obtained by the imaging information obtaining unit; and a processing information generating unit (206) that generates processing information necessary for executing predetermined processing on the target object, on the basis of the position information obtained by the position information obtaining unit and the direction information obtained by the direction information obtaining unit.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an industrial device control system, and an information processing method. [Background technology]

[0002] Generally, when an industrial device such as an industrial processing device, a transport device, or a robot arm is made to perform a predetermined process, for example, the robot arm is taught in advance and the trajectory of a predetermined part of the robot arm is stored in the robot control device. Conventionally, as a method for teaching a robot arm (arm), a method called direct teaching (lead-through teaching) in which an operator directly touches the robot arm and applies an external force to operate it has been disclosed (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-037172 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the control of a robot arm by direct teaching as described in Patent Document 1, a command value for controlling the movement of the robot arm when the robot arm performs a task is determined by the trajectory of a predetermined part when the operator operates the robot arm. For this reason, depending on the task of the robot arm, for example, polishing a workpiece that requires accuracy in the movement of the robot arm, if the direct teaching is excessively careful, the efficiency may decrease, and conversely, if the direct teaching is rough with an emphasis on efficiency, sufficient accuracy may not be obtained in the movement of the robot arm.

[0005] The present disclosure is intended to solve the above-mentioned problems, and aims to provide an information processing device, an industrial equipment control system, and an information processing method that can improve the efficiency of setting information necessary for processing industrial equipment such as a robot arm compared to conventional methods, and as a result, improve the accuracy of the operation of the industrial equipment. [Means for solving the problem]

[0006] The information processing device according to the present disclosure is characterized by including an imaging information acquisition unit that acquires imaging information of an image of a symbol displayed on a surface of an object based on a signal from an imaging unit that images the object, a position information acquisition unit that acquires position information indicating the position of the surface, a direction information acquisition unit that acquires directional information indicating the direction of the surface based on the imaging information acquired by the imaging information acquisition unit, and a processing information generation unit that generates processing information required to execute a predetermined process on the object based on the position information acquired by the position information acquisition unit and the directional information acquired by the directional information acquisition unit. Effect of the Invention

[0007] According to the present disclosure, an information processing device generates information (processing information) necessary for determining, for example, the trajectory of a specific part of a robot arm based on the position of the surface of an object and the direction of the surface, thereby improving the efficiency of setting information necessary for processing industrial equipment such as a robot arm compared to a case where a trajectory is generated based on direct instruction of the robot arm, and as a result, improving the accuracy of the operation of the industrial equipment. [Brief description of the drawings]

[0008] [Figure 1] 1 is a schematic diagram showing a robot control system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram showing a configuration of a robot control system according to a first embodiment. [Diagram 3] FIG. 2 is a diagram showing an example of a hardware configuration of the command value calculation device according to the first embodiment. [Figure 4]FIG. 2 is a diagram showing an example of a hardware configuration of the command value calculation device according to the first embodiment. [Diagram 5] 4 is a flowchart showing a process performed by a command value calculation device according to the first embodiment. [Figure 6] FIG. 13 is a diagram showing an example in which two markers are displayed on the surface of a workpiece so that the command value calculation device according to the first embodiment can obtain position information indicating the position of the surface of the workpiece and directional information indicating the direction of the surface. [Figure 7] FIG. 13 is a diagram showing an example in which four markers are displayed on the surface of a workpiece so that the command value calculation device according to the first embodiment can obtain position information indicating the position of the surface of the workpiece and directional information indicating the direction of the surface. [Figure 8] FIG. 13 is a diagram showing an example in which markers are displayed along the ridgeline of a workpiece so that the command value calculation device according to the first embodiment can obtain position information indicating the position of the face of the workpiece and directional information indicating the direction of the face. [Figure 9] FIG. 13 is a diagram showing an example in which two markers are displayed on one surface of a workpiece in order to set a range within which the command value calculation device according to the first embodiment generates a trajectory of a predetermined part of a robot arm. [Figure 10] FIG. 13 is a diagram showing an example in which four markers are displayed on one surface of a workpiece in order to set a range within which the command value calculation device according to the first embodiment generates a trajectory of a predetermined part of a robot arm. [Figure 11] FIG. 13 is a diagram showing an example in which a marker is displayed on the surface of a cylindrical workpiece in order for the command value calculation device according to the first embodiment to set a range in which a trajectory of a predetermined part of a robot arm is generated. [Figure 12] FIG. 2 is a diagram showing an example in which markers are displayed on a curved workpiece surface in order for the command value calculation device according to the first embodiment to set a range in which a trajectory of a predetermined part of a robot arm is generated. [Figure 13] FIG. 13 is a diagram showing an example in which markers are displayed on the surface of a spherical workpiece in order for the command value calculation device according to the first embodiment to set a range in which a trajectory of a predetermined part of a robot arm is generated. [Figure 14] FIG. 11 is a block diagram showing the configuration of a robot control system according to a second embodiment. [Figure 15] 10 is a flowchart showing a process performed by a command value calculation device according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present disclosure will be described in detail with reference to the drawings. Note that, although a robot control system will be described as a representative application example of the industrial device control system according to the present disclosure, the present disclosure is not limited to robots and can be widely applied to industrial devices such as a conveying device that conveys an object and a processing device that processes an object. Embodiment 1 Fig. 1 is a diagram showing an example of the configuration of a robot control system 1 according to embodiment 1. As shown in Fig. 1, the robot control system 1 includes a robot arm 100, a command value calculation device 200 as an information processing device, and a robot control device 300.

[0010] The robot arm 100 has two or more links that are movable relative to one another, and an end effector 104 attached to the tip. The links are connected to one another so that adjacent links can rotate relatively around a specific central axis. For example, the robot arm 100 is configured as a six-axis vertical articulated robot having six joints. The robot arm 100 performs work on a workpiece W1, which is a work target, by moving one or more links, using the end effector 104.

[0011] For example, the robot arm 100 has a processing tool that is an end effector 104, and a plurality of links move while rotating the processing tool to perform processing or polishing of the workpiece W1. Note that the work performed by the robot arm on the workpiece W1 may be any work that includes a step in which a specific part of the robot arm moves relative to the workpiece in at least a part of the work, and may be, for example, a work of welding the workpiece by a specific part of the robot arm, a work of applying adhesive to the workpiece by a specific part of the robot arm, a work of moving or deforming the workpiece by applying an external force to the workpiece by a specific part of the robot arm, or a work of acquiring some information from the workpiece with or without contacting the workpiece.

[0012] The command value calculation device 200 calculates a command value for the robot arm 100 to perform a task on a workpiece W1 that is a task target of the robot arm 100. For example, the command value calculation device 200 calculates a command value related to the position of a predetermined part of the robot arm 100 to move the predetermined part along a preset trajectory.

[0013] The robot control device 300 is electrically connected to the command value calculation device 200 and the robot arm 100 by wire or wirelessly, and controls the robot arm 100 based on a command value calculated by the command value calculation device 200. For example, the robot control device 300 supplies power to the robot arm 100 based on the command value calculated by the command value calculation device 200, thereby controlling the position of a predetermined part of the robot arm 100 when the robot arm 100 performs an operation on a workpiece W1.

[0014] The robot control device may be configured to control the robot arm 100 by outputting a control signal to the robot arm based on the command value calculated by the command value calculation device 200. When the robot control device is configured in this way, the robot arm may be configured to receive power via a power supply device (not shown). The robot control device is not limited to being electrically connected directly to the command value calculation device and the robot arm 100, but may be indirectly connected to them via other devices (not shown), or may be indirectly connected to the robot arm via the command value calculation device.

[0015] Fig. 2 is a block diagram showing the configuration of the robot control system 1 according to the first embodiment. As shown in Fig. 2, the robot arm 100 has a plurality of torque sensors 101 corresponding to each joint, a plurality of motors 102, and a plurality of encoders 103. The motors 102 rotate the plurality of links connected at the corresponding joints relative to each other based on the power supply from the robot control device 300. For example, the motors 102 are formed of servo motors. The encoders 103 detect the rotation angle of the corresponding joints and output a signal according to the detected rotation angle.

[0016] The torque sensor 101 detects the torque applied to the corresponding joint. In other words, the torque sensor 101 detects the torque acting on the corresponding joint. When the torque sensor 101 detects the torque, it outputs a signal according to the magnitude of the detected torque. For example, the torque sensor 101 is configured by a strain gauge type torque sensor. Note that the torque sensor may be a sensor that detects torque by other methods such as a capacitance type, a magnetostriction type, or an optical type.

[0017] The command value calculation device 200 includes an imaging unit 201, a device position acquisition unit 202, an imaging information acquisition unit 203, a marker direction acquisition unit 204, a marker position acquisition unit 205, a trajectory generation unit 206, a command value calculation unit 207, a storage unit 208, an input unit 209, and a display unit 210. When the present disclosure is applied to a robot control system, the trajectory generation unit 206 also generates a trajectory for moving a robot arm according to processing information for an object (workpiece W1), and corresponds to a processing information generation unit in the present disclosure. In addition, the above-mentioned robot control device corresponds to an industrial device control device that controls an industrial device based on the processing information generated by the information processing generation unit in the present disclosure. In addition, the imaging unit 201 of the command value calculation device 200 may be configured by a glasses-type wearable device such as smart glasses.

[0018] The imaging unit 201 captures an image of the workpiece W1 and outputs a signal corresponding to optical information obtained by the imaging. For example, the imaging unit 201 is configured with an image sensor such as a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor.

[0019] The device position acquisition unit 202 acquires information regarding the position of the imaging unit 201. For example, the device position acquisition unit 202 acquires, as the information regarding the position of the imaging unit 201, device position information indicating the position of the imaging unit 201 in a predetermined three-dimensional coordinate system and device orientation information indicating the imaging direction of the imaging unit 201 in the three-dimensional coordinate system. Specifically, the device position acquisition unit 202 acquires, as the information regarding the position of the imaging unit 201, device position information indicating the position of the imaging unit 201 in a three-dimensional coordinate system based on a fixed part (root part) of the robot arm 100 and device orientation information indicating the imaging direction of the imaging unit 201 in the three-dimensional coordinate system.

[0020] For example, the device position acquisition unit 202 acquires device position information based on a signal from the imaging unit 201 that images a predetermined part of the robot arm 100 and information indicating the position of the predetermined part of the robot arm 100 relative to the imaging unit 201, and acquires device orientation information based on a signal from the imaging unit 201 that images the predetermined part of the robot arm 100. Specifically, the device position acquisition unit 202 acquires device position information based on a signal from the imaging unit 201 that images the predetermined part of the robot arm 100 and distance measurement information from a distance measurement sensor (not shown) that measures the distance from the imaging unit 201 to the predetermined part of the robot arm 100. In addition, the device position acquisition unit 202 acquires device orientation information based on a signal from the imaging unit 201 that images the predetermined part of the robot arm 100 and the device position information.

[0021] Furthermore, when either or both of the position of the imaging unit 201 and the imaging direction of the imaging unit 201 change, the device position acquisition unit 202 acquires the amount of change in the position of the imaging unit 201 and the amount of change in the imaging direction of the imaging unit 201. For example, the device position acquisition unit 202 acquires the amount of change in the position of the imaging unit 201 and the amount of change in the imaging direction of the imaging unit 201 in a three-dimensional coordinate system based on a fixed part of the robot arm 100, based on information from an acceleration sensor (not shown) that detects the magnitude of acceleration received by the imaging unit 201. In the following description, the three-dimensional coordinate system based on a fixed part of the robot arm 100 is also referred to as a reference coordinate system.

[0022] The imaging information acquisition unit 203 acquires imaging information obtained by capturing images of the markers C1 and C2 as symbols displayed on either side of the workpiece W1 based on a signal from the imaging unit 201. For example, the imaging information acquisition unit 203 has a marker information acquisition unit 203a that acquires decoded information obtained by decoding the markers C1 and C2 including a two-dimensional code such as a QR code (registered trademark), and a marker shape acquisition unit 203b that acquires geometric information obtained from the appearance of the markers C1 and C2, and acquires imaging information including the decoded information and geometric information.

[0023] Since the markers C1 and C2 are displayed on either side of the workpiece W1, the apparent shape changes depending on the relative position between the workpiece W1 and the imaging unit 201. The marker shape acquisition unit 203b acquires geometric information including the apparent shape of the markers C1 and C2, which changes depending on the relative position between the workpiece W1 and the imaging unit 201. The markers C1 and C2 may be printed on a label attached to a predetermined surface of the workpiece W1, may be directly printed on a predetermined surface of the workpiece W1, or may be displayed by being projected by a projector or the like (not shown) on the predetermined surface of the workpiece W1. Alternatively, depending on the situation, a procedure may be used in which the markers C1 and C2 are substantially displayed by holding them over a hand or a jig.

[0024] The marker position acquisition unit 205 as a position information acquisition unit acquires position information indicating the positions of the markers C1 and C2 relative to the imaging unit 201. Specifically, the marker position acquisition unit 205 acquires position information indicating the positions of the markers C1 and C2 relative to the imaging unit 201 based on imaging information of the markers C1 and C2 and distance measurement information from a distance measurement sensor (not shown) that measures the distance from the imaging unit 201 to the markers C1 and C2. Note that since the markers C1 and C2 are displayed on one of the surfaces of the workpiece W1, the position information indicating the positions of the markers C1 and C2 can be said to be information indicating the positions of the surface of the workpiece W1 on which the markers C1 and C2 are displayed.

[0025] The marker direction acquisition unit 204 as a direction information acquisition unit acquires direction information indicating the direction of the markers C1 and C2 based on the imaging information of the markers C1 and C2 and the position information indicating the positions of the markers C1 and C2. For example, the marker direction acquisition unit 204 acquires direction information indicating the direction of the markers C1 and C2 relative to the imaging direction of the imaging unit 201 based on the geometric information obtained from the appearance of the markers C1 and C2 and the position information indicating the positions of the markers C1 and C2. Also, for example, the marker direction acquisition unit 204 acquires direction information indicating the direction of the markers C1 and C2 in the reference coordinate system based on the geometric information obtained from the appearance of the markers C1 and C2 and the position information indicating the positions of the markers C1 and C2. Note that since the markers C1 and C2 are displayed on one of the surfaces of the workpiece W1, the direction information indicating the direction of the markers C1 and C2 can be said to be information indicating the direction of the surface of the workpiece W1 on which the markers C1 and C2 are displayed.

[0026] The trajectory generating unit 206 generates a trajectory of a predetermined part of the robot arm 100 with respect to the workpiece W1 based on the position information acquired by the marker position acquiring unit 205 and the direction information acquired by the marker direction acquiring unit 204. For example, the trajectory generating unit 206 acquires information indicating the position of a predetermined surface of the workpiece W1 in the reference coordinate system based on the position information acquired by the marker position acquiring unit 205 and the device position information acquired by the device position acquiring unit 202, acquires information indicating the direction of the predetermined surface of the workpiece W1 in the reference coordinate system based on the direction information acquired by the marker direction acquiring unit 204 and the device direction information acquired by the device position acquiring unit 202, and generates a trajectory of a predetermined part of the robot arm 100 with respect to the workpiece W1 based on the acquired information. More specifically, for example, the predetermined surface is a plurality of planes, and the position and length of the intersection line between different planes are calculated as the range of a three-dimensional linear equation by solving a mathematical geometric equation based on the position information and direction information. Alternatively, the inter-surface distance between different parallel planes is calculated numerically by solving mathematical geometric equations. These calculation results are processing information required for the robot arm 100 to execute a predetermined process on the workpiece W1, and the generation of this processing information further makes it possible to generate a trajectory of a predetermined portion of the robot arm 100.

[0027] The command value calculation unit 207 calculates a command value for the robot arm 100 so that a predetermined portion of the robot arm 100 moves along the trajectory generated by the trajectory generation unit 206. For example, the command value calculation unit 207 calculates a command value for the position of the end effector 104 with respect to the robot arm 100 so that the end effector 104 of the robot arm 100 moves along the trajectory generated by the trajectory generation unit 206.

[0028] The storage unit 208 stores various types of information such as data, parameters, programs, etc., which are referenced when the command value calculation device 400 performs various processes.

[0029] The input unit 209 receives an input signal from a device external to the command value computation device 200. For example, the input unit 209 receives an input signal in response to an input operation from an operator who operates the command value computation device 200. Specifically, the input unit 209 receives an input signal from an input device (not shown), such as a button, a keyboard, a mouse, or a touch panel, in response to an input operation by the operator on the input device.

[0030] The display unit 210 displays information acquired by the command value computation device 200 performing various processes, information acquired by the command value computation device 200 from an external device, information stored in the storage unit 208, and the like. For example, the display unit 210 is configured by a head-mounted display that transmits visible light. Note that the entire configuration of the command value computation device 200 other than the display unit 210 may be configured integrally with the head-mounted display, or some of the configuration may be configured independent of the head-mounted display. Details of the processes performed by each configuration of the command value computation device 200 will be described later.

[0031] The robot controller 300 controls the robot arm 100 based on the command value calculated by the command value calculation unit 207. In other words, the robot controller 300 controls the robot arm 100 based on the trajectory generated by the trajectory generation unit 206. The robot controller 300 includes a drive control unit 301 and a storage unit 302.

[0032] The drive control unit 301 controls the operation of the motor 102 of the robot arm 100 based on the command value calculated by the command value calculation unit 207 and signals from the torque sensor 101 and the encoder 103 of the robot arm 100. For example, the drive control unit 301 controls the operation of the motor 102 by supplying power to the motor 102 of the robot arm 100 based on the command value calculated by the command value calculation unit 207 and signals from the torque sensor 101 and the encoder 103 of the robot arm 100. The storage unit 302 stores various information such as data, parameters, programs, etc. referred to when the robot control device 300 controls the robot arm 100.

[0033] Next, the hardware configuration of the command value calculation device 200 will be described with reference to Figs. 3 and 4. Fig. 3 is a block diagram showing an example of the hardware configuration of the command value calculation device 200 according to the first embodiment, and Fig. 4 is a block diagram showing an example of a hardware configuration of the command value calculation device 200 according to the first embodiment, which is different from that shown in Fig. 3. For example, as shown in Fig. 3, the command value calculation device 200 has a processor 200a, a memory 200b, and an I / O port 200c, and is configured so that the processor 200a reads and executes a program stored in the memory 200b. The memory 200b may be, for example, a non-volatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, or an EEPROM. The memory 200b may also be a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD, or the like. The memory 200b may also be an HDD or an SSD.

[0034] 4, the command value calculation device 200 has a processing circuit 200d, which is dedicated hardware, and an I / O port 200c. The processing circuit 200d is configured, for example, by a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, a system LSI (Large-Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination of these. Each function of the command value calculation device 200 is realized by the processor 200a or the processing circuit 200d, which is dedicated hardware, executing a program, which is software, firmware, or a combination of software and firmware.

[0035] Next, details of the processing performed by the command value calculation device 200 according to the first embodiment will be described with reference to Fig. 5 to Fig. 13. Fig. 5 is a flowchart showing processing related to an information processing method performed by the command value calculation device 200 according to the first embodiment.

[0036] 5, when the process starts, the command value calculation device 200 acquires device position information and device direction information (step ST1). For example, in this process, the device position acquisition unit 202 acquires, as information related to the position of the imaging unit 201, device position information indicating the position of the imaging unit 201 in the reference coordinate system and device direction information indicating the imaging direction of the imaging unit 201 in the reference coordinate system.

[0037] Specifically, in this process, the device position acquisition unit 202 acquires information on the position of the surface on which the marker is displayed relative to the imaging unit 201 by measuring the distance between the imaging unit 201 and the marker when the imaging unit 201 captures the marker, not shown, displayed on the predetermined surface of the fixed part of the robot arm 100. Also, in this process, the device position acquisition unit 202 acquires information indicating the direction of the predetermined surface of the fixed part of the robot arm 100 relative to the imaging direction of the imaging unit 201, based on a signal from the imaging unit 201 that captures the marker, not shown, displayed on the predetermined surface of the fixed part of the robot arm 100, and information on the position of the surface on which the marker is displayed relative to the imaging unit 201. Also, in this process, the device position acquisition unit 202 acquires device position information indicating the position of the imaging unit 201 in the reference coordinate system and device direction information indicating the imaging direction of the imaging unit 201 in the three-dimensional coordinate system, based on the acquired information.

[0038] Furthermore, for example, after acquiring device position information and device orientation information, the device position acquisition unit 202 acquires the amount of change in the position and imaging direction of the imaging unit 201 in the three-dimensional coordinate system at any time, thereby making it possible to acquire the position and imaging direction of the imaging unit 201 at any time.

[0039] After performing the process of step ST1, the command value calculation device 200 acquires decoded information of the marker (step ST2). In this process, the imaging information acquisition unit 203 decodes the two-dimensional code included in the marker based on the signal from the imaging unit 201 by the marker information acquisition unit 203a, and acquires the decoded information. For example, the decoded information is information for identifying which of the multiple markers the marker is, information for identifying which of the multiple surfaces of the workpiece the marker is displayed on, information for identifying which of the multiple workpieces the marker is displayed on, information for identifying between the marker displayed on the workpiece and the marker displayed on the robot arm, and the like.

[0040] After performing the process of step ST2, the command value calculation device 200 acquires geometric information of the marker (step ST3). In this process, the imaging information acquisition unit 203 acquires geometric information obtained from the appearance of the marker based on the signal from the imaging unit 201 by the marker shape acquisition unit 203b.

[0041] When the process of step ST3 is performed, the command value calculation device 200 acquires the position information and the direction information of the marker (step ST4). For example, in this process, the marker position acquisition unit 205 acquires the position information of the marker based on the measurement result of the distance from the imaging unit 201 to the marker, and the marker direction acquisition unit 204 acquires the direction information indicating the direction of the marker based on the position information of the marker and the geometric information obtained from the appearance of the marker. In other words, in this process, the marker position acquisition unit 205 acquires the position information of the surface of the workpiece W1 based on the measurement result of the distance from the imaging unit 201 to the marker, and the marker direction acquisition unit 204 acquires the direction information indicating the direction of the surface of the workpiece W1 based on the position information of the surface and the imaging information.

[0042] 6 is a diagram showing an example in which two markers C1 and C2 are displayed on the surface of the workpiece W1 so that the command value calculation device 200 can obtain position information indicating the position of the surface of the workpiece W1 and directional information indicating the direction of the surface. As shown in FIG. 6, in the workpiece W1 having a first surface and a second surface arranged adjacent to the first surface, the command value calculation device 200 obtains first position information indicating the position of the first marker C1 relative to the position of the imaging unit 201 and first directional information indicating the direction of the first surface relative to the imaging direction of the imaging unit 201 by displaying the first marker C1 as a first symbol on the first surface.

[0043] Specifically, first, the marker position acquisition unit 205 detects a first line L11, which is a predetermined outline of the rectangular first marker, and a second line L12 intersecting with the first line L11, based on the first imaging information obtained by imaging the first marker. Next, the marker position acquisition unit 205 detects a first point P1, which is an intersection point between the first line L11 and the second line L12, a second point P2, which is an end point of the first line L11 on the opposite side to the first point P1, and a third point P3, which is an end point of the second line L12 on the opposite side to the first point P1, based on the first imaging information. Next, the marker position acquisition unit 205 acquires position information indicating the positions of each of the first point P1 to the third point P3 relative to the imaging unit 201, based on the measurement results of the distances from the imaging unit 201 to each of the first point P1 to the third point P3. Note that the origin position itself of a measurable marker within the marker cannot necessarily be positioned on the outline. In such a case, it is only necessary to grasp in advance the positional relationship between the measurable origin position and the outline.

[0044] Next, the marker direction acquisition unit 204 acquires direction information indicating the direction of a plane including the first point P1 to the third point P3 with respect to the fixed part of the robot arm 100 based on the device position information of the imaging unit 201 with respect to the fixed part of the robot arm 100 acquired by the device position acquisition unit 202, the position information of each of the first point P1 to the third point P3 acquired by the marker position acquisition unit 205, and the geometric information of the first marker C1 acquired by the imaging information acquisition unit 203. In other words, the marker direction acquisition unit 204 calculates a plane including the first point P1 to the third point P3 in the reference coordinate system based on the device position information, the position information of each of the first point P1 to the third point P3, and the geometric information of the first marker C1. For example, when it is known in advance that the plane including the first point P1 to the third point P3 is a plane, the equation indicating the plane including the first point P1 to the third point P3 is expressed by the following mathematical formula (1). a 1 x+b 1 y+c 1 z+1=0 (1)

[0045] In addition, when the coordinates of the first point P1 to the third point P3 in the reference coordinate system are expressed as (x, y, z), the coordinates of the first point P1 (x 1 ,y 1 ,z 1 ), the coordinates of the second point P2 (x 2 ,y 2 ,z 2 ), the coordinates of the third point P3 (x 3 ,y 3 ,z 3 ) into formula (1), the coefficient a 1 , b 1 , c 1 In this way, it can be said that the information indicating the equation of the plane including the first point P1 to the third point P3 in the reference coordinate system is first direction information indicating the direction of the first surface with respect to the imaging direction of the imaging section 201.

[0046] Similarly, in the workpiece W1 having a first surface and a second surface arranged adjacent to the first surface, the command value calculation device 200 can obtain second position information indicating the position of the second surface and second direction information indicating the direction of the second surface based on second imaging information obtained by imaging the second marker, in this manner, the command value calculation device 200 calculates the position and direction of each surface of the workpiece W1 based on imaging information obtained by imaging the markers displayed on each surface of the workpiece W1. In other words, a coordinate system based on each surface is obtained based on position information indicating the position of the marker displayed on each surface of the workpiece W1 and posture information indicating the posture of the marker, and the position and direction of each surface are calculated based on the coordinate system.

[0047] When the process of step ST4 is performed, the command value calculation device 200 generates a trajectory of a predetermined part of the robot arm 100 relative to the workpiece W1 based on the first position information, the first direction information, the second position information indicating the position of the second surface, and the second direction information (step ST5). For example, in this process, the trajectory generation unit 206 generates either one or both of the trajectory of the end effector 104 in which the end effector 104 moves along the first surface and the trajectory of the end effector 104 in which the end effector 104 moves along the second surface. Also, for example, in this process, the trajectory generation unit 206 generates a trajectory of the end effector 104 in which the end effector 104 moves along the edge line E1 that is the intersection line between the first surface and the second surface. For example, the equation indicating the edge line E1 formed by the first surface and the second surface is calculated by the equation indicating the first surface and the equation indicating the second surface. In this way, the equation representing the first surface and the equation representing the second surface, given in the form of formula (1), are automatically obtained from the position information and direction information of the marker, and the equation representing the edge line E1, which corresponds to the processing information, which is information about the shape of the workpiece W1, is automatically calculated by solving commonly known mathematical geometric equations from these equations, thereby improving the efficiency of setting the information required for processing by the robot arm (processing information).

[0048] After performing the process of step ST5, the command value calculation device 200 causes the display unit 210 to display the trajectory generated by the trajectory generation unit 206 (step ST6). In this process, the command value calculation device 200 enables the operator to evaluate the validity of the trajectory in advance by showing the trajectory of a predetermined part of the robot arm 100 generated by the trajectory generation unit 206 to the operator before starting the work by the robot arm 100.

[0049] After performing the process of step ST6, the command value calculation device 200 calculates a command value for the robot arm 100 so that a predetermined part of the robot arm 100 moves along the trajectory generated by the trajectory generation unit 206 (step ST7). For example, in this process, the command value calculation unit 207 calculates a command value for the robot arm 100 so that the end effector 104 moves along the trajectory generated by the trajectory generation unit 206. Note that in this process, the command value calculation unit 207 may calculate a command value for the movement speed of the predetermined part, a command value for the operation speed of the end effector 104, a command value for the external force that the end effector 104 applies to the workpiece W1, etc., in addition to the command value for the predetermined part of the robot arm 100 to move along the trajectory generated by the trajectory generation unit 206.

[0050] After performing the process of step ST7, the command value calculation device 200 outputs the command value calculated by the command value calculation section 207 to the robot control device 300 (step ST8), and ends the process.

[0051] Next, a process for displaying markers on a workpiece in a manner different from that shown in FIG. 6 will be described. FIG. 7 is a diagram showing an example in which four markers are displayed on the surface of the workpiece W1 so that the command value calculation device 200 according to the first embodiment acquires position information indicating the position of the surface of the workpiece W1 and direction information indicating the direction of the surface. Specifically, a first marker C1 is displayed on the first surface of the workpiece W1, a second marker C2 is displayed on the second surface adjacent to the first surface, a third marker C3 is displayed on the third surface adjacent to the first and second surfaces, and a fourth marker C4 is displayed on the fourth surface adjacent to the first and second surfaces and opposed to the third surface.

[0052] By displaying the first marker C1 to the fourth marker C4 in this manner, the command value calculation device 200 can obtain information indicating the positions of the end points P4 and P5 of the edge line E1, and can set, for example, the positions of the start point and end point of the trajectory when a predetermined part of the robot arm 100 moves. Note that the command value calculation device 200 is not limited to obtaining information indicating the positions of both end points P4 and P5 of the edge line E1, and may obtain information indicating the position of either one of the end points P4 and P5 of the edge line E1 when only one of the third marker C3 and the fourth marker C4 is displayed.

[0053] 8 is a diagram showing an example in which markers C1 and C2 are displayed along the edge line E1 of the workpiece W1 so that the command value calculation device 200 according to the first embodiment can obtain position information indicating the position of the edge line of the workpiece W1 and directional information indicating the direction of the edge line of the workpiece W1. Specifically, the markers C1 and C2 are displayed so that the edge line E1 of the workpiece W1 coincides with the predetermined outlines of the markers C1 and C2 in order for the command value calculation device 200 to obtain position information indicating the position of the edge line of the workpiece W1 and directional information indicating the direction of the edge line of the workpiece W1. By displaying the markers on the workpiece W1 in this way, the command value calculation device 200 can directly obtain information indicating the position and direction of the edge line E1 of the workpiece W1 by the markers without performing a process of calculating two faces and then calculating the intersection line between these faces.

[0054] The command value calculation device 200 can acquire information indicating the position and direction of the ridgeline E1 not only when multiple markers are displayed so that the ridgeline E1 of the workpiece W1 coincides with the outer shape, but also when one marker is displayed so that the ridgeline E1 of the workpiece W1 coincides with the outer shape. However, when multiple markers are displayed so that the ridgeline E1 of the workpiece W1 coincides with the outer shape, the accuracy of the operation of the robot arm 100 can be further improved by adopting a representative value such as the average value or median value of the position and direction of the ridgeline E1 acquired based on each outer shape of the multiple markers.

[0055] FIG. 9 is a diagram showing an example in which two markers are displayed on one surface of the workpiece W1 in order to set a range in which the command value calculation device 200 according to the first embodiment generates a trajectory of a predetermined part of the robot arm 100. For example, as shown in FIG. 9, when a first marker C8 and a second marker C9 are displayed on any one surface of the workpiece W1, the command value calculation device 200 may be configured to generate a trajectory of a predetermined part of the robot arm 100 within a rectangular area A1 defined by a first line L8, which is a predetermined outline of the first marker C8, an end point P8 of the first line L8, and a vertex P9 of the second marker C9. Note that the origin position as a marker that can be measured within the marker is the upper right vertex position of a small upper right rectangle displayed within the marker in the example of FIG. 9 for the first marker C8.

[0056] 10 is a diagram showing an example in which four markers are displayed on one surface of the workpiece W1 in order to set a range in which the command value calculation device 200 according to the first embodiment generates a trajectory of a predetermined part of the robot arm 100. For example, as shown in FIG. 10, when a first marker C10, a second marker C11, a third marker C12, and a fourth marker C13 are displayed on any one surface of the workpiece W1, the command value calculation device 200 may be configured to generate a trajectory of a predetermined part of the robot arm 100 within an area A2 defined by a first line L10 that is a predetermined outline of the first marker C10, a second line L11 that is a predetermined outline of the second marker C11, a third line L12 that is a predetermined outline of the third marker C12, and a fourth line L13 that is a predetermined outline of the fourth marker C13.

[0057] The surface of the workpiece from which the command value calculation device 200 obtains the position information indicating the position of the surface and the direction information indicating the direction of the surface is not limited to a flat surface. For example, the surface of the workpiece may be a curved surface. FIG. 11 is a diagram showing an example in which markers are displayed on the surface of the workpiece W2 having a cylindrical surface in order to set a range in which the command value calculation device 200 according to the first embodiment generates a trajectory of a predetermined part of the robot arm 100. For example, as shown in FIG. 11, when a plurality of markers are displayed so that the ridge line E2 of the cylindrical surface of the workpiece W2 coincides with a predetermined outline or an end point of the outline of each marker, the command value calculation device 200 may be configured to generate a trajectory of a predetermined part of the robot arm 100 along the circular ridge line E2 defined by the end points P15, P16, and P17 of the outline or the outline of the plurality of markers.

[0058] 12 is a diagram showing an example in which markers are displayed on the surface of a workpiece W3 having a curved surface in order to set a range in which the command value calculation device 200 according to the first embodiment generates a trajectory of a predetermined part of the robot arm 100. For example, as shown in FIG. 12, when a sufficient number of markers are displayed such that a curved edge line E3 of the workpiece W3 coincides with a predetermined outline or an end point of the outline of each marker, the command value calculation device 200 may be configured to generate a trajectory of a predetermined part of the robot arm 100 along the curved edge line E3 defined by the outline or the end points of the outline of these multiple markers.

[0059] Fig. 13 is a diagram showing an example in which markers are displayed on the surface of a workpiece W4 having a spherical surface in order to set a range in which the command value calculation device 200 according to the first embodiment generates a trajectory of a predetermined part of the robot arm 100. For example, as shown in Fig. 13, when a sufficient number of markers are displayed so that the curved surface of the workpiece W4 and a predetermined vertex (end point of the outline) of each marker coincide with each other, the command value calculation device 200 may be configured to generate a trajectory of a predetermined part of the robot arm 100 along a curved surface F1 defined by these multiple vertices. When the curved surface F1 is a spherical surface, the curved surface F1 can be specified by displaying four markers.

[0060] As described above, the robot control system 1 according to the first embodiment includes an imaging information acquisition unit 203 that acquires imaging information of a marker displayed on the surface of the workpiece W1 based on a signal from an imaging unit 201 that images the workpiece W1, a marker position acquisition unit 205 that acquires position information indicating the position of the surface, a marker direction acquisition unit 204 that acquires directional information indicating the direction of the surface based on the imaging information acquired by the imaging information acquisition unit 203, and a trajectory generation unit 206 that generates a trajectory of a specific portion of the robot arm 100 relative to the workpiece W1 based on the position information acquired by the marker position acquisition unit 205 and the directional information acquired by the marker direction acquisition unit 204.

[0061] By being configured in this manner, the robot control system 1 generates a trajectory for a specific part of the robot arm 100 based on the position of the surface of the workpiece W1 and the direction of that surface, thereby improving the accuracy of the operation of the robot arm compared to when a trajectory is generated based on direct instruction to the robot arm.

[0062] The command value calculation device 200 according to the first embodiment is configured to generate a trajectory of a predetermined part of the robot arm 100 relative to the workpiece based on the position information acquired by the marker position acquisition unit 205 and the direction information acquired by the marker direction acquisition unit 204, but may be configured to generate the trajectory of the predetermined part of the robot arm 100 based on other information. For example, the command value calculation device may be configured to generate the trajectory of the predetermined part of the robot arm 100 based on other information such as various pre-set conditions, numerical input by an operator, parameters acquired from an external device, and programs acquired from an external device in addition to the position information acquired by the marker position acquisition unit 205 and the direction information acquired by the marker direction acquisition unit 204, or may be configured to generate a part of the trajectory of the predetermined part of the robot arm 100 based on direct instruction by an operator. In addition, the command value calculation device is equipped with a geometric information acquisition unit that acquires geometric information indicating the standard or fundamental dimensions, position, shape, etc. of the workpiece through an external device not shown in the figure or an input operation by an operator to the input unit 209, and the trajectory generation unit may be configured to generate a trajectory based on the position information acquired by the position information acquisition unit, the directional information acquired by the directional information acquisition unit, and the geometric information of the workpiece acquired by the geometric information acquisition unit, or may be configured to generate a part of the trajectory based on the position information acquired by the position information acquisition unit and the directional information acquired by the directional information acquisition unit, and generate another part of the trajectory based on the geometric information of the workpiece acquired by the geometric information acquisition unit.

[0063] Embodiment 2 Next, a robot control system according to embodiment 2 will be described. The robot control system according to embodiment 2 differs from the robot control system 1 according to embodiment 1 in the configuration of a part of the command value calculation device, but other configurations are the same as those of the robot control system 1 according to embodiment 1. The same names or symbols are used for the same configurations as those of the robot control system 1 according to embodiment 1, and the description thereof will be omitted.

[0064] Fig. 14 is a block diagram showing a configuration of a robot control system according to embodiment 2. As shown in Fig. 14, the robot control system according to embodiment 2 includes a robot arm 100, a command value calculation device 400 as an information processing device, and a robot control device 300. The command value calculation device 400 includes an imaging unit 201, a device position acquisition unit 202, an imaging information acquisition unit 203, a marker direction acquisition unit 204, a marker position acquisition unit 205, a trajectory generation unit 206, an object information acquisition unit 211, a determination unit 212 as a limiting unit, a command value calculation unit 207, a storage unit 208, an input unit 209, and a display unit 210.

[0065] The object information acquisition unit 211 acquires geometric information of the workpiece. For example, the object information acquisition unit 211 acquires geometric information indicating standard or fundamental dimensions, position, shape, etc. of the workpiece by an external device (not shown) or an input operation by an operator to the input unit 209. The judgment unit 212 restricts the movement of a predetermined part of the robot arm along the trajectory generated by the trajectory generation unit 206 based on the geometric information of the workpiece acquired by the object information acquisition unit 211. For example, the judgment unit 212 compares the trajectory generated by the trajectory generation unit 206 with the geometric information of the workpiece acquired by the object information acquisition unit 211, and judges whether or not there is a problem with the trajectory generated by the trajectory generation unit 206. In other words, the judgment unit 212 compares the trajectory generated by the trajectory generation unit 206 with the geometric information of the workpiece acquired by the object information acquisition unit 211, and judges the validity of the trajectory generated by the trajectory generation unit 206.

[0066] For example, the judgment unit 212 compares the trajectory generated by the trajectory generation unit 206 with the geometric information of the work acquired by the object information acquisition unit 211, and judges whether the difference between the geometric information of the work identified based on the trajectory generated by the trajectory generation unit 206 and the geometric information of the work acquired by the object information acquisition unit 211 is equal to or greater than a preset threshold value. If the difference is equal to or greater than the threshold value, the judgment unit 212 restricts the output of the command value calculated by the command value calculation unit 207, thereby restricting the movement of a specified part of the robot arm along the trajectory generated by the trajectory generation unit 206.

[0067] FIG. 15 is a flowchart showing the process of the information processing method performed by the command value calculation device 400 according to the second embodiment. When the process of step ST6 is performed, the command value calculation device 400 acquires object information, which is geometric information of the workpiece (step ST9). When the process of step ST9 is performed, the command value calculation device 400 judges whether or not there is a problem with the trajectory generated in step ST6 based on the geometric information of the workpiece acquired in step ST9 (step ST10). When it is judged in the process of step ST10 that there is a problem with the trajectory (YES in step ST10), the command value calculation device 400 ends the process without outputting a command value for controlling the robot arm 100 to the robot control device 300. When it is judged in the process of step ST10 that there is no problem with the trajectory (NO in step ST10), the command value calculation device 400 performs the process of step ST7. In this way, the command value calculation device 400 according to the second embodiment can determine the validity of the generated trajectory based on the geometric information of the workpiece acquired by the target object information acquisition unit 211. This makes it possible to prevent the robot arm 100 from being controlled based on a trajectory with low validity, thereby improving the accuracy of the operation of the robot arm 100.

[0068] As described above, the robot control system according to embodiment 2 performs force control using a value obtained by removing from the torque measurement value other-axis error caused by contact force and other-axis error caused by the mass of the robot arm. Therefore, even in the case of a task in which the posture of the robot arm changes significantly during the task (for example, a task of chamfering a metal plate by following the edge of a cutter attached to the end of the robot arm), the robot arm can apply an accurate force to the work object, thereby improving the processing quality.

[0069] In addition, the present disclosure allows free combinations of the respective embodiments, or modifications of any of the components of the respective embodiments, or omission of any of the components of the respective embodiments. [Explanation of symbols]

[0070] 1: Robot control system 100: Robot arm 101: Torque sensor 102: Motor 103: Encoder 104: End effector 200: Command value calculation device (information processing device) 201: Imaging unit 202:Device position acquisition section 203: Imaging information acquisition unit 203a: Marker information acquisition unit 203b: Marker shape acquisition unit 204: marker direction acquisition unit (direction information acquisition unit) 205: Marker position acquisition unit (position information acquisition unit) 206: Trajectory generation unit (processing information generation unit) 207: Command value calculation unit 208: Storage section 209: Input section 210:Display section 211: Object information acquisition unit (geometric information acquisition unit) 212: Judgment section (restriction section) 300: Robot control device (industrial equipment control device) 301: Drive control unit 302: Storage section 400: Command value calculation device C1: Marker (first marker) C2: Marker (second marker) W1: Work

Claims

1. an imaging information acquisition unit that acquires imaging information of an image of a symbol displayed on a surface of an object based on a signal from an imaging unit that images the object handled by the industrial device; a position information acquisition unit that acquires position information indicating a position of the surface; a direction information acquisition unit that acquires direction information indicating a direction of the surface based on the imaging information acquired by the imaging information acquisition unit; a processing information generating unit that generates processing information required to execute a predetermined process on the object based on the position information acquired by the position information acquiring unit and the direction information acquired by the direction information acquiring unit.

23. An information processing apparatus comprising:

2. The symbol includes a two-dimensional code, The imaging information acquisition unit acquires imaging information including decoded information obtained by decoding the two-dimensional code and geometric information obtained from the appearance of the symbol.

2. The information processing apparatus according to claim 1,

3. a command value calculation unit that calculates a command value for the industrial equipment so that a predetermined portion of the industrial equipment moves in accordance with the processing information generated by the processing information generation unit; 2. The information processing apparatus according to claim 1,

4. the imaging information acquisition unit acquires, based on a signal from the imaging unit, first imaging information obtained by imaging a first symbol displayed on a first surface of the object, and second imaging information obtained by imaging a second symbol displayed on a second surface of the object disposed adjacent to the first surface; the position information acquisition unit acquires first position information indicating a position of the first surface and second position information indicating a position of the second surface; the direction information acquisition unit acquires first direction information indicating a direction of the first surface and second direction information indicating a direction of the second surface based on the first imaging information and the second imaging information acquired by the imaging information acquisition unit; The processing information generation unit generates processing information based on a ridge line formed by the first surface and the second surface, the processing information being obtained based on the first position information and the second position information acquired by the position information acquisition unit and the first direction information and the second direction information acquired by the direction information acquisition unit.

2. The information processing apparatus according to claim 1,

5. A geometric information acquisition unit that acquires geometric information of the object, The processing information generation unit generates the processing information based on the position information acquired by the position information acquisition unit, the direction information acquired by the direction information acquisition unit, and the geometric information of the object acquired by the geometric information acquisition unit.

2. The information processing apparatus according to claim 1,

6. A geometric information acquisition unit that acquires geometric information of the object; a limiting unit that limits movement of the predetermined portion of the industrial device according to the processing information based on the geometric information of the object.

4. The information processing apparatus according to claim 3.

7. An information processing device according to any one of claims 1 to 5; an industrial equipment control device that controls the industrial equipment based on the processing information generated by the processing information generation unit.

1. An industrial device control system comprising:

8. An information processing method performed by an apparatus including an imaging information acquisition unit, a position information acquisition unit, a direction information acquisition unit, and a processing information generation unit, The imaging information acquisition unit acquires imaging information of an image of a symbol displayed on a surface of an object handled by the industrial device based on a signal from an imaging unit that images the object; a step of acquiring position information indicating a position of the surface by the position information acquisition unit; a step of the direction information acquisition unit acquiring direction information indicating a direction of the surface based on the imaging information acquired by the imaging information acquisition unit; and generating processing information required for executing a predetermined process on the object based on the position information acquired by the position information acquisition unit and the direction information acquired by the direction information acquisition unit, by the processing information generation unit.

23. An information processing method comprising:

Images