Robot system and method for identifying restraint position of linear workpiece

Abstract

A robot system that holds a linear workpiece and performs work, and a method for identifying a constraint position of the linear workpiece, capable of automatically identifying the constraint position when the linear workpiece is constrained, are provided. [Solution] The robot system comprises a robot having a holder, a sensor that detects the magnitude and direction of the tension of the linear workpiece held by the holder, an imaging device that images a predetermined area including at least a portion of the linear workpiece, and a processing circuit.When the magnitude of the force detected by the sensor is equal to or greater than a reference value, the processing circuit acquires area data of the image of the predetermined area, performs a work recognition process that recognizes the linear workpiece in the area data, and identifies the bending point of the linear workpiece using the linear workpiece recognition area in the area data obtained by the work recognition process and the force direction data detected by the sensor in the area data, and identifies the bending point of the linear workpiece as the restraint position of the linear workpiece.

Description

[Technical Field]

[0001] The present disclosure relates to a robot system and a method for identifying a restraining position of a linear workpiece. [Background technology]

[0002] The following Patent Document 1 discloses a robot system that automates processes for handling linear workpieces at production sites, such as connecting cable connectors to predetermined positions. In such a robot system, the robot holds a predetermined holding position on the linear workpiece and moves the holding position, thereby moving the linear workpiece. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5672322 Summary of the Invention [Problem to be solved by the invention]

[0004] However, if a linear workpiece gets caught on an obstacle during work performed by such a robot system, the obstacle must be manually identified and the linear workpiece released from the constraint by the obstacle. It is inefficient to manually monitor and check whether a linear workpiece is caught in an actual production site. This issue is not addressed in the above-mentioned Patent Document 1. Therefore, the configuration of Patent Document 1 leaves room for improvement in terms of automatically identifying the constraint position of the linear workpiece when the linear workpiece gets caught.

[0005] The present disclosure has been made in consideration of the above-mentioned problems, and aims to provide a robot system that holds a linear workpiece and performs work while holding it, and a method for identifying the constraint position of a linear workpiece, which can automatically identify the constraint position when a constraint occurs on the linear workpiece. [Means for solving the problem]

[0006] A robot system according to one aspect of the present disclosure comprises a robot having a holder for holding a linear workpiece, a sensor for detecting the magnitude and direction of the tension of the linear workpiece held by the holder, an imaging device for imaging a predetermined area including at least a portion of the linear workpiece, and a processing circuit, wherein when the magnitude of the force detected by the sensor is equal to or greater than a reference value, the processing circuit acquires area data of the image of the predetermined area, performs a work recognition process to recognize the linear workpiece in the area data, identifies a bending point of the linear workpiece using the linear workpiece recognition area in the area data obtained by the work recognition process and the force direction data detected by the sensor in the area data, and identifies the bending point of the linear workpiece as a restraint position of the linear workpiece.

[0007] Another aspect of the present disclosure provides a method for identifying the constraint position of a linear workpiece, which detects the magnitude and direction of tension of a linear workpiece held by a holder of a robot, and when the magnitude of the tension of the linear workpiece is equal to or greater than a reference value, images a predetermined area including at least a portion of the linear workpiece, performs a work recognition process to recognize the linear workpiece in the area data obtained by the image of the predetermined area, identifies a bending point of the linear workpiece using the linear workpiece recognition area in the area data obtained by the work recognition process and the tension direction data of the linear workpiece in the area data, and identifies the bending point of the linear workpiece as the constraint position of the linear workpiece. [Effects of the Invention]

[0008] According to the present disclosure, in a robot system that performs work while holding a linear workpiece, when a constraint occurs on the linear workpiece, the constraint position can be automatically identified. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a schematic configuration of a robot system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a flowchart showing the flow of the restraint position identification process in this embodiment. [Figure 3] FIG. 3 is a top view of the work table shown in FIG. [Figure 4] FIG. 4 is a conceptual diagram of the first heat map image in this embodiment. [Figure 5] FIG. 5 is a conceptual diagram showing a method for generating a second heat map image in this embodiment. [Figure 6] FIG. 6 is a conceptual diagram of the third heat map image in this embodiment. [Figure 7] FIG. 7 is a diagram showing examples of heat map images according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment will be described in detail with reference to the drawings. In the following, the same or corresponding elements will be designated by the same reference numerals throughout the drawings, and redundant description thereof will be omitted.

[0011] 1 is a diagram showing a schematic configuration of a robot system according to an embodiment of the present disclosure. The robot system 1 is a system including a robot 2 that holds a linear workpiece W and performs a predetermined task. The robot system 1 includes the robot 2, a sensor 7, an imaging device 8, and a processing circuit 10. The robot 2 includes a holder 3 and a manipulator arm 4.

[0012] The manipulator arm 4 has a plurality of joints 5. The plurality of joints 5 form motion axes for changing the position and posture of the holder 3 held at the tip of the manipulator arm 4. In the example of FIG. 1, the manipulator arm 4 has six joints 5, but the number of joints 5 may be seven or more, or five or less. The manipulator arm 4 is fixed to a base 6. The base 6 may be fixed to a fixed object such as the ground, or may be configured to be movable. The holder 3 is fixed to the tip of the manipulator arm 4 via a sensor 7.

[0013] The holder 3 is configured, for example, by a robot hand that grasps the linear workpiece W and holds the linear workpiece W in a predetermined position. However, the configuration of the holder 3 is not limited to this, and may be, for example, a suction device that attracts the linear workpiece W to a predetermined position using suction or magnetic force. In the example of FIG. 1, the linear workpiece W is a cable with a connector plug at its tip. The holder 3 holds the connector plug. The sensor 7 detects the magnitude F and direction Df of the force applied to the holder 3. That is, the sensor 7 detects the magnitude F and direction Df of the tension of the linear workpiece W held by the holder 3 through the holder 3. In this embodiment, the sensor 7 is configured as a force sensor equipped with a six-axis strain gauge.

[0014] The imaging device 8 images a predetermined area Aw that includes at least a portion of the linear workpiece W. In this embodiment, the robot 2 moves the linear workpiece W in a crawling manner on the work table 9. Therefore, the imaging device 8 images an area that includes a portion of the surface of the work table 9 as the predetermined area Aw. The imaging device 8 generates area data that captures the image of the predetermined area Aw. In this embodiment, the imaging device 8 is configured as a camera that captures an optical image as the area data. In this embodiment, the optical image captured by the imaging device 8 is referred to as a workpiece image.

[0015] The multiple motion axes that make up the multiple joints 5 are driven by multiple robot drive motors. The robot 2 is equipped with a robot controller 11. The multiple robot drive motors are controlled by the robot controller 11. This allows the manipulator arm 4 to change the position and orientation of the tip in three dimensions. The robot controller 11 generates robot control command values ​​for driving the robot drive motors. The robot controller 11 is equipped with a robot control circuit that generates robot control command values ​​for controlling the robot drive motors. The robot control circuit has a computer such as a microcontroller or a PLC (Programmable Logic Controller).

[0016] The robot controller 11 is connected to a host controller 12. The host controller 12 acquires the operating state of the robot controller 11 and outputs control commands to the robot controller 11. The host controller 12 includes a processing circuit 10 that performs various signal processing. The processing circuit 10 generates control commands to control the manipulator arm 4.

[0017] The sensor 7 is also connected to a processing circuit 10. Force data regarding the magnitude F and direction Df of the force detected by the sensor 7 is sent to the processing circuit 10. The imaging device 8 is also connected to the processing circuit 10. A workpiece image captured by the imaging device 8 is sent to the processing circuit 10. The processing circuit 10 uses the force data acquired from the sensor 7 and the workpiece image acquired from the imaging device 8 to perform processing, which will be described later.

[0018] The processing circuit 10 includes a computer such as a microcontroller, a personal computer, or a PLC (Programmable Logic Controller). More specifically, the processing circuit 10 includes a processor, a memory, and peripheral circuits. The processor includes, for example, a CPU or an MPU. The memory includes, for example, a ROM, a RAM, a register, non-volatile storage, and the like. The peripheral circuits include an input / output interface, and the like. The upper controller 12 is connected to a monitor 13 that displays output. The upper controller 12 may also be connected to an output device other than the monitor 13, such as a speaker that outputs audio, or an input device for a user to input operations.

[0019] It should be noted that the functions of the elements disclosed herein can be performed using circuits or processing circuits, including general-purpose processors, special-purpose processors, integrated circuits, application-specific integrated circuits (ASICs), conventional circuits, and / or combinations thereof, configured or programmed to perform the disclosed functions. A processor is considered a processing circuit or circuit because it includes transistors and other circuitry. In this specification, a circuit, unit, means, or part is hardware that performs the recited functions or hardware that is programmed to perform the recited functions. The hardware may be hardware disclosed herein or other known hardware that is programmed or configured to perform the recited functions. Where hardware is a processor, which is considered a type of circuit, the circuit, unit, or means is a combination of hardware and software, and the software is used to configure the hardware and / or processor.

[0020] A control program for controlling the operation of the robot 2 is stored in the memory of the processing circuit 10. The processing circuit 10 controls the robot 2 based on the control program. The processing circuit 10 executes various arithmetic processing based on the control program.

[0021] The processing circuit 10 controls the operation of the manipulator arm 4 while the linear workpiece W is held by the holder 3 of the robot 2. For example, the processing circuit 10 controls the operation of the manipulator arm 4 so as to connect the connector plug at the end of the cable that is the linear workpiece W to a predetermined receptacle. When the linear workpiece W gets caught while controlling the operation of the robot 2 while holding the linear workpiece W, the processing circuit 10 executes the following restraint position identification process.

[0022] 2 is a flowchart showing the flow of the constraint position identification process in this embodiment. The processing circuit 10 acquires force sense data detected from the sensor 7 during operation of the manipulator arm 4, and monitors whether the magnitude F of the force detected from the sensor 7 exceeds a reference value Fo (step S1). If the magnitude F of the force detected from the sensor 7 exceeds the reference value Fo (Yes in step S1), the processing circuit 10 determines that the linear workpiece W has caught on some obstacle B, and executes a constraint position identification process to identify the position of the obstacle B, i.e., the constraint position Pa of the linear workpiece W.

[0023] In the restraint position identification process, the processing circuit 10 acquires a workpiece image capturing a predetermined area Aw (step S2). The processing circuit 10 executes a workpiece recognition process to recognize a linear workpiece W in the acquired workpiece image (step S3). In the workpiece recognition process, the processing circuit 10 divides each of the multiple pixels constituting the workpiece image into a linear workpiece recognition area where the linear workpiece W is located and the other area.

[0024] For example, the processing circuit 10 performs semantic segmentation on the workpiece image. In this case, a predetermined learning circuit acquires a plurality of labeled image data for each of the plurality of workpiece images, in which each pixel is labeled according to whether it represents a linear workpiece W portion. The learning circuit performs machine learning using the plurality of labeled image data to generate a segmentation model of the linear workpiece W. The learning circuit may be included in the processing circuit 10, or may be configured as a processing circuit separate from the processing circuit 10. For example, the learning circuit may be configured as a processing circuit of a predetermined server equipped with a memory device in which the plurality of labeled image data is stored, and may be communicatively connected to the processing circuit 10.

[0025] The generated segmentation model is stored in a memory device of the processing circuit 10. The processing circuit 10 recognizes the linear workpiece W in the acquired workpiece image using the segmentation model.

[0026] The processing circuit 10 identifies the bending point Pb of the linear workpiece W using the linear workpiece recognition area in the workpiece image obtained by the workpiece recognition process and the force direction data detected by the sensor 7 in the workpiece image. When the linear workpiece W is caught on an obstacle B, the portion of the linear workpiece W from the holding position by the holder 3 to the restraining position Pa is considered to be along the direction in which tension in the linear workpiece W is generated, and the portion of the linear workpiece W on the opposite side of the holding position from the restraining position Pa is considered to be not along the direction in which tension in the linear workpiece W is generated. In other words, when the linear workpiece W is caught on an obstacle B, the linear workpiece W is bent at the restraining position Pa.

[0027] For this reason, the processing circuit 10 identifies the restraint position Pa of the linear workpiece W by identifying the bending point Pb of the linear workpiece W. More specifically, the processing circuit 10 identifies the boundary A3 between the first area A1 along the force direction and the second area A2 deviating from the force direction Df in the linear workpiece recognition area in the workpiece image as the bending point Pb of the linear workpiece.

[0028] In order to clearly distinguish between the first area A1 and the second area A2, the linear workpiece W is moved in a predetermined direction, thereby identifying an area in the linear workpiece recognition area where the linear workpiece W moves and an area where the linear workpiece W does not move. That is, in the workpiece recognition process, the processing circuit 10 controls the robot 2 to move the holder 3 in a predetermined direction from the position of the holder 3 where the magnitude F of the force detected by the sensor 7 becomes equal to or greater than the reference value Fo (step S4). Hereinafter, the position of the holder 3 where the magnitude F of the force detected by the sensor 7 becomes equal to or greater than the reference value Fo will be referred to as the initial position Po.

[0029] Fig. 3 is a top view of the work table shown in Fig. 1. In Fig. 3, only the holder 3 of the robot 2 is shown. The predetermined direction in which the linear workpiece W is moved includes a direction intersecting with the direction Df of the force detected by the sensor 7. Furthermore, the predetermined direction in which the linear workpiece W is moved includes a direction intersecting with the optical axis of the imaging device 8.

[0030] In this embodiment, the processing circuit 10 moves the holder 3 in a plane perpendicular to the optical axis of the imaging device 8, i.e., a plane parallel to the work table 9. The processing circuit 10 controls the robot 2 to move the holder 3 in a circular orbit within the plane, such that the holder 3 moves on an arc having a predetermined radius from the initial position Po in a direction perpendicular to the force direction Df. That is, the processing circuit 10 moves the holder 3 from the initial position Po in the tangential directions Dt+ and Dt- of the circle having a predetermined radius. The movement may be in only one of the tangential directions Dt+ or Dt-, or may be reciprocated in both the tangential directions Dt+ and Dt-.

[0031] The processing circuit 10 acquires a plurality of workpiece images captured by the imaging device 8 at predetermined timings while moving the holder 3 (step S5). The predetermined timings may be set based on, for example, the frame rate of the imaging device 8. For example, the predetermined timing may be one frame at the frame rate of the imaging device 8.

[0032] The processing circuit 10 recognizes the linear workpiece W in each of the multiple workpiece images during the movement of the holder 3, and divides the workpiece into a linear workpiece recognition area and other areas. By calculating the difference between the multiple linear workpiece recognition areas recognized in each of the multiple workpiece images, the processing circuit 10 estimates, among the multiple linear workpiece recognition areas, a portion where the movement amount of the linear workpiece W is equal to or greater than a first threshold value as a first area A1, and estimates a portion where the movement amount of the linear workpiece W is less than the first threshold value as a second area A2. Note that the estimation criteria for the first area A1 and the second area A2 may use the movement frequency of the linear workpiece W in addition to or instead of the movement amount of the linear workpiece W.

[0033] The processing circuit 10 generates a first heat map image G1 according to the amount of movement of the linear workpiece W obtained from the difference between the multiple linear workpiece recognition areas (step S6). Fig. 4 is an image diagram of the first heat map image in this embodiment. In Fig. 4, the linear workpiece recognition area of ​​the linear workpiece W at the initial position Po of the holder 3 where the magnitude F of the force detected by the sensor 7 becomes equal to or greater than the reference value Fo is represented by Wv0, and the linear workpiece recognition area of ​​the linear workpiece W that has moved several frames from the initial position Po by moving the holder 3 is represented by Wv1.

[0034] The movement amount of the linear workpiece W is indicated by a gradation or multiple colors on the linear workpiece recognition area Wv0 of the first heat map image G1. For example, in the first heat map image G1, the linear workpiece recognition area Wv0 is displayed with bluer areas where the movement amount or movement frequency is greater and reddish areas where the movement amount or movement frequency is smaller. In Figure 4, the difference in gradation or color is indicated by different types of hatching.

[0035] In this way, the first heat map image G1 can distinguish between the first area A1, which is a part of the linear workpiece recognition area Wv0 where the amount of movement or the frequency of movement is large, and the second area A2, which is a part of the linear workpiece recognition area Wv0 where the amount of movement or the frequency of movement is small. This makes it possible to estimate that the bending point Pb of the linear workpiece W, i.e., the restraint position Pa, exists at the boundary A3 between the first area A1 and the second area A2.

[0036] However, when the holder 3 is moved, the restraining position Pa of the linear workpiece W due to the obstacle B is unknown, and therefore, when the linear workpiece W moves in conjunction with the movement of the holder 3, there is a possibility that the linear workpiece W may move away from the obstacle B depending on the direction of movement. For example, when the holder 3 is moved in the direction Dt+ in FIG. 3, there is a possibility that the position of the boundary portion A3 in the linear workpiece recognition area Wv0 may shift to a position farther from the holder 3 than the actual position where it is in contact with the obstacle B. In other words, depending on how the linear workpiece W is moved, the restraining position Pa may be erroneously detected.

[0037] Therefore, the processing circuit 10 generates a second heat map image G2 (step S7) using the force direction data detected by the sensor 7 in each of the multiple workpiece images while the holder 3 is moving. For this purpose, the processing circuit 10 draws a straight line L along the force direction Df on the workpiece image at each predetermined timing when the multiple workpiece images are generated.

[0038] FIG. 5 is an image diagram showing a method for generating a second heat map image in this embodiment. The workpiece image G20 shown in FIG. 5 shows the linear workpiece recognition area Wv0 of the linear workpiece W at the initial position Po of the holder 3, where the magnitude of the force F detected by the sensor 7 is equal to or greater than the reference value Fo. The direction of the force detected by the sensor 7 at this time is represented as Df0 (=Df). The processing circuit 10 identifies the force direction Df0 on the workpiece image G20 using data on the direction of the optical axis of the imaging device 8 and data on the force direction Df0 detected by the sensor 7. The processing circuit 10 draws a straight line L0 along the force direction Df0 on the workpiece image G20.

[0039] 5 shows a linear workpiece recognition area Wv1 of the linear workpiece W after the holder 3 has been moved several frames from the initial position Po. The direction of the force detected by the sensor 7 at this time is denoted as Df1. The processing circuit 10 draws a straight line L1 along the force direction Df1 in the workpiece image G21.

[0040] The processing circuit 10 integrates and superimposes the straight lines L0, L1, ... drawn at predetermined timings within a predetermined period. The processing circuit 10 generates a second heat map image G2 according to the degree of overlap of the multiple straight lines. In the example of Figure 5, the area Lx where the straight lines L0 and L1 overlap is displayed with a gradation or color different from the other parts of the straight lines L0 and L1.

[0041] 5 shows the overlap of two straight lines L0 and L1, but the second heat map image G2 can be generated by drawing more straight lines using more workpiece images and overlapping them. The degree of overlap of the straight lines L is indicated on the second heat map image G2 using a gradation or multiple colors. For example, the second heat map image G2 is displayed with bluer areas where the degree of overlap of the straight lines L is smaller and reddish areas where the degree of overlap is greater.

[0042] The processing circuit 10 identifies the bending point Pb of the linear workpiece W using the first heat map image G1 and the second heat map image G2 (step S8). More specifically, the processing circuit 10 identifies the bending point Pb of the linear workpiece W as the point where an area in the first heat map image G1 where the movement amount of the linear workpiece is less than a first threshold overlaps with an area in the second heat map image G2 where the degree of overlap of the straight line L is equal to or greater than a second threshold.

[0043] In this embodiment, the processing circuit 10 generates a third heat map image G3 according to the degree of overlap between the first heat map image G1 and the second heat map image G2. Fig. 6 is an image diagram of the third heat map image in this embodiment. In the third heat map image G3, a specific region Gx where a boundary A3 between a first region A1, which has a large movement amount or movement frequency when the holder 3 is moved, and a second region A2, which has a small movement amount or movement frequency, overlaps with the rotation center of the tension direction of the linear workpiece W when the holder 3 is moved, i.e., the region Lx, is indicated by a specific gradation or color.

[0044] The processing circuit 10 identifies the specific region Gx in the third heat map image G3 as a bending point Pb of the linear workpiece W. The processing circuit 10 identifies the identified bending point Pb of the linear workpiece W as a constraint position Pa of the linear workpiece W (step S9). In this embodiment, the processing circuit 10 displays the third heat map image G3 on the monitor 13 as a result of identifying the constraint position Pa of the linear workpiece W.

[0045] 7 is a diagram showing examples of heat map images in this embodiment. In FIG. 7, a first heat map image G1 and a second heat map image G2 are generated from several frames of workpiece images captured one frame at a time. In the third heat map image G3, the area indicated by the arrow Pa is shown in a different color from the other areas in the linear workpiece recognition area Wv0. Therefore, the area indicated by the arrow Pa can be identified as the restraint position Pa of the linear workpiece W.

[0046] According to this embodiment, in a robot system 1 that performs work while holding a linear workpiece W, if the linear workpiece W gets caught, the bending point Pb of the linear workpiece W in the workpiece image is identified based on the linear workpiece recognition area Wv0 in the workpiece image captured by the imaging device 8 and the magnitude F and direction Df of the force detected by the sensor 7. Therefore, if a constraint occurs on the linear workpiece W, the constraint position can be automatically identified. Furthermore, in the workpiece recognition process, it is not necessary to recognize or learn in advance about the environment other than the linear workpiece W, such as obstacles B and the workpiece table 9. Therefore, the time and financial costs required for learning can be reduced.

[0047] Furthermore, according to this embodiment, in the linear workpiece recognition area Wv0 in the workpiece image, the boundary A3 between the first area A1 along the force direction Df and the second area A2 deviating from the force direction Df is identified as the bending point of the linear workpiece W. Therefore, the bending point Pb of the linear workpiece W, which is identified as the restraint position Pa of the linear workpiece W, can be identified without erroneous detection.

[0048] Furthermore, in this embodiment, when the magnitude F of the force detected by the sensor 7 becomes equal to or greater than the reference value Fo, the robot 2 is controlled to move the holder 3 holding the linear workpiece W in a predetermined direction. By active sensing, in which workpiece images are taken while the linear workpiece W is being moved and detection is performed by the sensor 7, a portion of the linear workpiece recognition area where the movement amount or movement frequency of the linear workpiece W is equal to or greater than a first threshold is estimated to be a first area A1 along the direction of force Df, and a portion of the linear workpiece W where the movement amount or movement frequency is less than the first threshold is estimated to be a second area A2 away from the direction of force Df. Therefore, the first area A1 and the second area A2 can be easily and accurately distinguished.

[0049] Furthermore, by setting the specified direction in which the holder 3 is moved when performing active sensing to a direction that intersects with the direction Df of the force detected by the sensor 7, it is relatively easy to move the linear workpiece W while maintaining the tension on the linear workpiece W even when the restraint position Pa has not been identified.

[0050] Furthermore, according to this embodiment, a first heat map image G1 is generated that shows a change in the linear workpiece recognition area in accordance with the movement of the linear workpiece W by active sensing, and a second heat map image G2 is generated that shows a change in the direction of the force in accordance with the movement of the linear workpiece W. The area in the first heat map image G1 where the linear workpiece W is less than the first threshold value and the area in the second heat map image G2 where the degree of overlap of the straight line L in accordance with the direction of the force is equal to or greater than the second threshold value are identified as the bending area Pb of the linear workpiece W, so that the identification result of the linear workpiece W in the workpiece image and the detection result of the force direction Df by the sensor 7 can be easily and appropriately combined.

[0051] Furthermore, a third heat map image G3 is generated according to the degree of overlap between the first heat map image G1 and the second heat map image G2, and is displayed on the monitor 13 as a result of identifying the restraint position Pa of the linear workpiece W, thereby visually showing the user the position where the linear workpiece W is caught. This reduces the amount of work required to investigate the cause of the linear workpiece W getting caught.

[0052] Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments, and various improvements, changes, and modifications are possible within the scope of the spirit of the present disclosure.

[0053] [Other embodiments] For example, in the above embodiment, the imaging device 8 is installed at a predetermined fixed position. However, the imaging device 8 may be able to change the predetermined area Aw to be imaged. For example, the imaging device 8 may be installed on the manipulator arm 4 to which the holder 3 is connected. The imaging device 8 may also be configured to change its angle at its installation position. When the predetermined area Aw to be imaged by the imaging device 8 is changeable, for example, the processing circuit 10 may determine the approximate direction of the restraint position Pa of the linear workpiece W relative to the holder 3 from the movement direction of the holder 3 or the direction of the force detected by the sensor 7, and change the position or angle of the imaging device 8 so that the area along the determined direction becomes the predetermined area Aw to be imaged by the imaging device 8. Furthermore, when the robot 2 has multiple manipulator arms, the imaging device 8 may be installed on a manipulator arm other than the manipulator arm 4 to which the holder 3 is connected.

[0054] Furthermore, in the above embodiment, an example has been given in which the processing circuit 10 identifies the constraint position Pa of the linear workpiece W and generates a control command for the robot 2, but this is not limiting. That is, the processing circuit that identifies the constraint position Pa and the control circuit that generates a control command for the robot may be configured as separate circuits. For example, the processing circuit that identifies the constraint position Pa may be configured as a processing circuit of a server that can communicate with the upper controller 12. Furthermore, the processing circuit that identifies the constraint position Pa may include multiple processing circuits. For example, the processing circuit that generates the first heat map image G1 and the processing circuit that generates the second heat map image G2 may be configured as separate processing circuits.

[0055] In the above embodiment, during active sensing, the holder 3 is moved from the initial position Po along a circular orbit that moves on an arc having a predetermined radius in a direction perpendicular to the force direction Df. However, the movement mode is not limited to this. For example, the holder 3 may be moved linearly from the initial position Po in a direction perpendicular to the force direction Df. Alternatively, the holder 3 may be moved from the initial position Po in a direction that forms a predetermined angle with respect to the force direction Df that is less than or greater than 90 degrees. In this case, the holder 3 may be moved along a triangular orbit with the initial position Po as one of its vertices.

[0056] Furthermore, in the above embodiment, an example has been shown in which the restraint position Pa is located on the work table 9, but the identifiable restraint position Pa is not limited to this. In other words, the restraint position Pa does not have to be on the work table 9. For example, even if the restraint position Pa is a three-dimensional catch, such as when the linear workpiece W is caught on a pin or the like that protrudes above the work table 9 in a direction along the surface of the work table 9, it can be identified using the configuration of the present disclosure.

[0057] In addition, in the above embodiment, the constraint position Pa of the linear workpiece W is visually displayed using the third heat map image G3, but the output of the identified constraint position Pa is not limited to image output. For example, the processing circuit 10 may control the robot 2 to release the constraint of the linear workpiece W at the constraint position Pa based on the constraint position Pa of the linear workpiece W.

[0058] In this case, the processing circuit 10 identifies the three-dimensional coordinates of the constraint position Pa in real space from the planar coordinates of the constraint position Pa of the linear workpiece W in the third heat map image G3. For this purpose, the memory of the processing circuit 10 stores the correlation between the coordinate system of the workpiece image captured by the imaging device 8 and the coordinate system in real space. For example, the processing circuit 10 may identify the correlation from the position coordinates in real space of the installation position of the imaging device 8 and the direction of the optical axis of the imaging device 8, or may identify the correlation by capturing an image of a predetermined calibration target placed at a specific position with the imaging device 8.

[0059] The processing circuit 10 moves the holder 3 to the vicinity of the three-dimensional coordinates of the constraint position Pa, and causes the robot 2 to perform a predetermined constraint release operation. This enables the robot system 1 to automatically execute the process of detecting a snag on the linear workpiece W, identifying the constraint position Pa where the snag occurred, and releasing the snag using information on the identified constraint position Pa.

[0060] Furthermore, in the above embodiment, an example has been given in which the sensor 7 is a force sensor, but this is not limiting. For example, the sensor 7 may be a current detector that detects the value of a current flowing through a robot drive motor that drives each joint 5 of the manipulator arm 4. In this case, the processing circuit 10 can estimate the torque applied to each joint 5 from the current value of the robot drive motor, and determine the magnitude F and direction Df of the force applied to the holder 3 from the torque. Also, for example, the sensor 7 may be a torque sensor that detects the torque of the robot drive motor that drives each joint 5.

[0061] Furthermore, in the above embodiment, an example has been shown in which the imaging device 8 is a camera that captures optical images, but this is not limiting. For example, the imaging device 8 may be a distance sensor that acquires distance data. Furthermore, the area data generated by the imaging device 8 capturing an image of the predetermined area Aw does not have to be image data. For example, the imaging device 8 may generate point cloud data as the area data.

[0062] Furthermore, in the above embodiment, an example was shown in which the third heat map image G3 was generated according to the degree of overlap between the two-dimensional first heat map image G1 and the two-dimensional second heat map image G2. However, the two images superimposed to identify the bending point Pb of the linear workpiece W do not have to be two-dimensional images. For example, when the area data generated by the imaging device 8 is three-dimensional data, the processing circuit 10 may generate a three-dimensional first image according to the movement amount or movement frequency of the linear workpiece W and a three-dimensional second image according to the degree of overlap between the straight lines L0 and L1 along the force direction, and identify the bending point Pb of the linear workpiece W by superimposing the first and second images. In other words, the first image according to the movement amount or movement frequency of the linear workpiece W may be a two-dimensional image or a three-dimensional image, and the second image according to the degree of overlap between the straight lines L0 and L1 along the force direction may also be a two-dimensional image or a three-dimensional image.

[0063] Furthermore, when the robot 2 is controlled to release the constraint of the linear workpiece W at the constraint position Pa based on the identified constraint position Pa of the linear workpiece W, the first image and the second image may each be configured as point cloud data when they are not displayed on the monitor 13. That is, the processing circuit 10 may identify the bending point Pb of the linear workpiece W by comparing the respective point cloud data without actually drawing the first image and the second image.

[0064] Summary of this disclosure [Item 1] A robot system according to one aspect of the present disclosure comprises a robot having a holder for holding a linear workpiece, a sensor for detecting the magnitude and direction of the tension of the linear workpiece held by the holder, an imaging device for imaging a predetermined area including at least a portion of the linear workpiece, and a processing circuit, wherein when the magnitude of the force detected by the sensor is equal to or greater than a reference value, the processing circuit acquires area data of the image of the predetermined area, performs a work recognition process to recognize the linear workpiece in the area data, identifies a bending point of the linear workpiece using the linear workpiece recognition area in the area data obtained by the work recognition process and the force direction data detected by the sensor in the area data, and identifies the bending point of the linear workpiece as a restraint position of the linear workpiece.

[0065] According to the above configuration, in a robot system that performs work while holding a linear workpiece W, if the linear workpiece gets caught, the bending point of the linear workpiece in the area data is identified based on the linear workpiece recognition area in the area data obtained by photographing the linear workpiece with an imaging device and the magnitude and direction of the force detected by the sensor. Therefore, in a robot system that performs work while holding a linear workpiece, if the linear workpiece gets caught, the constraint position can be automatically identified.

[0066] [Item 2] In the robot system of item 1, the processing circuit may identify, within the linear workpiece recognition area in the area data, a boundary between a first area along the force direction and a second area deviating from the force direction as the bending point of the linear workpiece. According to this, within the linear workpiece recognition area in the area data, a boundary between the first area along the force direction and the second area deviating from the force direction is identified as the bending point of the linear workpiece. Therefore, the bending point of the linear workpiece can be identified without erroneous detection.

[0067] [Item 3] In the robot system of item 2, the processing circuit may, in the workpiece recognition process, control the robot to move the holder in a predetermined direction from the position of the holder where the magnitude of the force detected by the sensor becomes equal to or greater than the reference value, and calculate the difference between the multiple linear workpiece recognition areas recognized in each of the multiple area data captured at predetermined timings while moving the holder. By calculating the difference between the multiple linear workpiece recognition areas recognized in each of the multiple area data captured at predetermined timings while moving the holder, the portion of the multiple linear workpiece recognition areas where the linear workpiece movement amount or movement frequency is equal to or greater than a first threshold value is estimated as the first area, and the portion of the linear workpiece where the linear workpiece movement amount or movement frequency is less than the first threshold value is estimated as the second area. By capturing area data while moving the linear workpiece and performing detection by the sensor, the portion of the linear workpiece recognition area where the linear workpiece moves is estimated as the first area along the direction of the force, and the portion where the linear workpiece does not move is estimated as the second area away from the direction of the force. Therefore, the first area and the second area can be easily and accurately distinguished. This improves the accuracy of detecting the restraint position of the linear workpiece.

[0068] [Item 4] In the robot system of item 3, the predetermined direction may include a direction intersecting the direction of the force detected by the sensor. This makes it relatively easy to move the linear workpiece while maintaining the tension of the linear workpiece, even when the restraint position has not been specified.

[0069] [Item 5] In the robot system of item 3 or 4, the processing circuit may generate a first image according to the movement amount or movement frequency of the linear workpiece obtained from the difference between the multiple linear workpiece recognition areas, draw straight lines along the direction of the force on the workpiece image based on the area data at each predetermined timing, generate a second image according to the degree of overlap of the straight lines drawn at each predetermined timing within a predetermined period, and identify as the bend in the linear workpiece a location where an area in the first image where the movement amount or movement frequency of the linear workpiece is less than the first threshold and an area in the second image where the degree of overlap of the straight lines is equal to or greater than a second threshold overlap. This makes it possible to easily and appropriately combine the linear workpiece identification result in the area data and the force direction detection result by the sensor.

[0070] [Item 6] In the robot system of item 5, the processing circuit may generate a two-dimensional first heat map image as the first image, generate a two-dimensional second heat map image as the second image, generate a third heat map image according to the degree of overlap between the first heat map image and the second heat map image, and display the third heat map image on a predetermined monitor as a result of identifying the constraint position of the linear workpiece. By generating a third heat map image according to the degree of overlap between the first heat map image and the second heat map image and displaying this on the monitor as a result of identifying the constraint position of the linear workpiece, the user can visually see the position where the linear workpiece W is caught.

[0071] [Item 7] In the robot system of any one of items 1 to 6, the processing circuit may control the robot to release the constraint of the linear workpiece at the constraint position based on the constraint position of the linear workpiece. This allows the robot system to automatically execute a process of detecting a jam of the linear workpiece, identifying the constraint position where the jam occurred, and releasing the jam using information on the identified constraint position.

[0072] [Item 8] Another aspect of the present disclosure provides a method for identifying the constraint position of a linear workpiece, which detects the magnitude and direction of tension of a linear workpiece held by a holder of a robot, and when the magnitude of the tension of the linear workpiece is equal to or greater than a reference value, images a predetermined area including at least a portion of the linear workpiece, performs a work recognition process to recognize the linear workpiece in the area data obtained by the image of the predetermined area, identifies a bending point of the linear workpiece using the linear workpiece recognition area in the area data obtained by the work recognition process and the tension direction data of the linear workpiece in the area data, and identifies the bending point of the linear workpiece as the constraint position of the linear workpiece. [Explanation of symbols]

[0073] 1. Robot System 2. Robot 3 Cage 7 Sensors 8. Imaging device 10 Processing circuit 13 Monitor A1 1st area A2 2nd area A3 border Aw specified area G1 First heat map image G2 Second heatmap image G3 3rd heat map image Pa restraint position Pb bending point W Linear workpiece

Claims

1. a robot having a holder for holding a linear workpiece; a sensor for detecting the magnitude and direction of tension of the linear workpiece held by the holder; an imaging device that images a predetermined area including at least a portion of the linear workpiece; processing circuitry; The processing circuitry When the magnitude of the force detected by the sensor is equal to or greater than a reference value, Acquire area data obtained by imaging the predetermined area, and perform work recognition processing to recognize the linear workpiece in the area data; Identifying a bending point of the linear workpiece using a linear workpiece recognition area in the area data obtained by the workpiece recognition processing and force direction data detected by the sensor in the area data; A robot system that identifies the bending point of the linear workpiece as a constraint position of the linear workpiece.

2. The robot system described in claim 1, wherein the processing circuit identifies the boundary portion of the linear workpiece recognition area in the area data between a first area along the direction of the force and a second area deviating from the direction of the force as the bending point of the linear workpiece.

3. The processing circuitry In the workpiece recognition processing, controlling the robot so as to move the holder in a predetermined direction from a position of the holder where the magnitude of the force detected by the sensor becomes equal to or greater than the reference value; The robot system of claim 2, wherein the difference between multiple linear workpiece recognition areas recognized in each of multiple area data captured at predetermined timings while moving the holder is calculated, and the portion of the multiple linear workpiece recognition areas where the movement amount or movement frequency of the linear workpiece is greater than or equal to a first threshold value is estimated to be the first area, and the portion of the linear workpiece where the movement amount or movement frequency of the linear workpiece is less than the first threshold value is estimated to be the second area.

4. The robot system according to claim 3 , wherein the predetermined direction includes a direction intersecting the direction of the force detected by the sensor.

5. The processing circuitry generating a first image according to the movement amount or movement frequency of the linear workpiece obtained from the difference between the plurality of linear workpiece recognition areas; drawing a straight line along the direction of the force on a workpiece image based on the area data at each predetermined timing; generating a second image according to the degree of overlap of the straight lines drawn at each predetermined timing within a predetermined period of time; The robot system described in claim 3 or 4, wherein the bending point of the linear workpiece is identified as the area where an area in the first image where the movement amount or movement frequency of the linear workpiece is less than the first threshold value overlaps with an area in the second image where the overlap of the straight lines is equal to or greater than a second threshold value.

6. The processing circuitry generating a two-dimensional first heat map image as the first image, and generating a two-dimensional second heat map image as the second image; The robot system described in claim 5, wherein a third heat map image is generated according to the degree of overlap between the first heat map image and the second heat map image, and the third heat map image is displayed on a specified monitor as a result of identifying the constraint position of the linear workpiece.

7. The robot system according to claim 1 , wherein the processing circuit controls the robot to release the constraint on the linear workpiece at the constraint position based on the constraint position of the linear workpiece.

8. Detecting the magnitude and direction of tension of the linear workpiece held by the robot's holder; When the magnitude of the tension of the linear workpiece is equal to or greater than a reference value, taking an image of a predetermined area including at least a part of the linear workpiece; Execute a workpiece recognition process to recognize the linear workpiece in area data obtained by capturing an image of the predetermined area; Identifying a bending point of the linear workpiece using a linear workpiece recognition area in the area data obtained by the workpiece recognition processing and tension direction data of the linear workpiece in the area data; A method for identifying a constraint position of a linear workpiece, which identifies the bending point of the linear workpiece as a constraint position of the linear workpiece.

Images