Tool check device, tool check program, and tool check method for robot arm

Abstract

To provide a tool check device for robot arm, a tool check program and a tool check method which can accurately determine whether a tool satisfies a tool condition.SOLUTION: A tool check device for robot arm comprises: a tool movement control unit for controlling the robot arm such that a tool attached to the robot arm moves to a specified position; an imaging control unit for controlling an imaging device so as to image the tool that has moved to the specified position; and a determination unit for determining the presence / absence of satisfaction of a tool condition about a tool type or tool state that should be satisfied by the tool on the basis of the photographed image captured by the imaging device.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a tool checking device, a tool checking program, and a tool checking method for a robot arm. [Background technology]

[0002] Conventionally, there is known a robot arm to which a tool for performing processing or operation on an object is replaceably attached. For example, in the robot arm disclosed in Patent Document 1, any one of a plurality of types of tools is attached to the robot arm depending on the processing to be performed on the object. The robot arm can grasp an object by opening and closing the tool. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-158405 A Summary of the Invention [Problem to be solved by the invention]

[0004] In order for the robot arm to perform operations such as machining appropriately, it is necessary to satisfy tool conditions such as that the attached tool is of an appropriate type and that the attached tool is in an appropriate state (e.g., open or closed) according to the machining to be performed. In this regard, Patent Document 1 does not disclose a specific configuration for accurately determining whether a tool satisfies the tool conditions.

[0005] One embodiment of the present disclosure provides a tool checking device, a tool checking program, and a tool checking method for a robot arm that can accurately determine whether a tool satisfies a tool condition. [Means for solving the problem]

[0006] According to at least one embodiment of the present invention, a tool checking device for a robot arm includes: a tool movement control unit for controlling the robot arm so that a tool attached to the robot arm moves to a specified position; an imaging control unit for controlling the imaging device to image the tool that has moved to the specified position; a determination unit for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool is satisfied based on a captured image captured by the imaging device; Equipped with.

[0007] A tool check program for a robot arm according to at least one embodiment of the present invention includes: To your computer a tool movement control step for controlling the robot arm so that a tool attached to the robot arm moves to a fixed position; an imaging control step for controlling an imaging device to image the tool that has been moved to the fixed position; a determination step for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool is satisfied based on a photographed image photographed by the photographing device; Execute the command.

[0008] A method for checking a tool for a robot arm according to at least one embodiment of the present invention includes the steps of: a tool movement control step for controlling the robot arm so that a tool attached to the robot arm moves to a fixed position; an imaging control step for controlling an imaging device to image the tool that has been moved to the fixed position; a determination step for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool is satisfied based on a photographed image photographed by the photographing device; Equipped with. Effect of the Invention

[0009] According to some embodiments, it is possible to provide a tool checking device, a tool checking program, and a tool checking method for a robot arm that can accurately determine whether a tool satisfies a tool condition. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram illustrating a workpiece machining system according to an embodiment. [Diagram 2] 11A to 11C are diagrams illustrating a flow in which the tool 40 according to the embodiment moves to a specified position P2. [Diagram 3] FIG. 13 is a diagram showing a defined position P2 according to a tool type according to an embodiment. [Figure 4] 1 is a diagram showing reference image data that is data of a reference image for executing image processing according to an embodiment; [Figure 5A] 11A and 11B are diagrams illustrating a determination method performed by a determination unit according to an embodiment. [Figure 5B] 13 is another diagram illustrating a determination method by a determination unit according to an embodiment. FIG. [Figure 6] 2 is a block diagram showing an electrical configuration of the workpiece machining system according to one embodiment. FIG. [Figure 7] 1 is a flowchart of a processing control process according to an embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, some embodiments of the present invention will be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of components described as the embodiments or shown in the drawings are merely illustrative examples and are not intended to limit the scope of the present invention. For example, expressions expressing relative or absolute configuration, such as "in a certain direction," "along a certain direction," "parallel," "orthogonal," "center," "concentric," or "coaxial," not only strictly express such a configuration, but also express a state in which there is a relative displacement with a tolerance or an angle or distance to the extent that the same function is obtained. For example, expressions indicating that things are in an equal state, such as "identical," "equal," and "homogeneous," not only indicate a state of strict equality, but also indicate a state in which there is a tolerance or a difference to the extent that the same function is obtained. For example, expressions describing shapes such as a rectangular shape or a cylindrical shape do not only refer to rectangular shapes, cylindrical shapes, etc. in the strict geometric sense, but also refer to shapes that include uneven portions, chamfered portions, etc., to the extent that the same effect is obtained. On the other hand, the expressions "comprise," "include," "have," "includes," or "have" of one element are not exclusive expressions excluding the presence of other elements.

[0012] FIG. 1 is a diagram showing a workpiece processing system 1 according to one embodiment. The workpiece processing system 1 according to one embodiment is provided for processing a workpiece 5 using a tool 40. The workpiece 5 is an object to be processed by the tool 40. As an example, the workpiece 5 is food such as agricultural products, livestock products, or marine products. The food may be either fresh food or processed food. In the following, an embodiment in which the workpiece 5 is fresh meat will be described.

[0013] A workpiece processing system 1 according to one embodiment includes a transport device 7 for transporting a workpiece 5, a robot arm 30 for processing the workpiece 5, an imaging device 8 for photographing a tool 40, a lighting unit 4 for illuminating the imaging area of ​​the imaging device 8, and a tool checking device 50 for the robot arm. In one embodiment, the transport device 7 is a belt conveyor that transports the workpiece 5 in a horizontal direction. In one embodiment, the photographing device 8 is provided so as to photograph the tool 40 from above. In this embodiment, the photographed image 15 of the photographing device 8 is a planar image. FIG. 1 illustrates the photographed image 15 of a clamper 41 (described later), which is an example of the tool 40. In one embodiment, the robot arm 30 is configured to mount the tool 40. In one embodiment, a tool checking device 50 checks whether the tool 40 is correct based on the captured image 15. Details of the configurations of the robot arm 30, the tool 40, and the tool checking device 50 will be described later. In other embodiments, the transport device 7 may hold and transport the workpiece 5 in a suspended position. The photographing device 8 may be configured to photograph the tool 40 along the horizontal direction, or may be configured to photograph the tool 40 along a direction inclined relative to the horizontal direction. Furthermore, the workpiece machining system 1 may not be provided with the lighting unit 4. In this case, the function of the lighting unit 4 may be included in the photographing device 8.

[0014] The configuration of the robot arm 30 will be described. The robot arm 30 of one embodiment is an industrial robot. More specifically, as an example, the robot arm 30 is a multi-joint robot. The robot arm 30 may be a vertical multi-joint robot, a horizontal multi-joint robot, or a combination thereof.

[0015] The robot arm 30 in one embodiment includes robot arms 30a, 30b, and 30c. In one embodiment, the tool 40 attached to the robot arm 30 is made of a metal material. Moreover, the tool 40 in one embodiment has a surface that is more likely to reflect light than, for example, the workpiece 5. In one embodiment, the tool 40 includes a clamper 41 for gripping the workpiece 5 , a chuck 42 for chucking the workpiece 5 , and a knife 43 for cutting the workpiece 5 . In one embodiment, clamper 41 is mounted to robot arm 30a, and chuck 42 and knife 43 are mounted to robot arm 30b or robot arm 30c. In one embodiment, symmetrical tools 40 are prepared so that they can be used according to the type of workpiece 5. As a specific example, the chuck 42 includes chucks 42L and 42R, and the knife 43 includes knives 43L and 43R, and these tools 40 are selectively attached to the robot arms 30b and 30c. For example, when the workpiece 5 transported by the transport device 7 is either the left leg meat or the right leg meat of a livestock, the chuck 42L and the knife 43R are attached to each of the robot arms 30b and 30c. When the workpiece 5 is the other, the knife 43L and the chuck 42R are attached to each of the robot arms 30b and 30c. In one embodiment, these attachment operations are performed by an operator. In another embodiment, these attachment operations may be performed by another robot. In one embodiment, the clamper 41 and the chuck 42 perform opening and closing operations by receiving a driving force from a driving source. In one embodiment, an air cylinder (not shown) is used as the driving source. In this case, an inlet and an outlet provided on each of the clamper 41 and the chuck 42 are connected to the air cylinder via an air pipe. Note that a hydraulic cylinder or a motor may be used as the driving source.

[0016] In other embodiments, the workpiece machining system 1 may not include symmetrical tools 40. For example, the chuck 42 may include only one of the chucks 42L and 42R. Similarly, the knife 43 may include only one of the knives 43. In still another embodiment, each of the multiple robot arms 30 may be equipped with only a single tool 40. Furthermore, the workpiece machining system 1 is not limited to being provided with a plurality of robot arms 30. A single robot arm 30 may be selectively equipped with a plurality of types of tools 40, or may be equipped with only a single tool 40.

[0017] The configuration of a tool check device 50 for a robot arm (hereinafter, sometimes referred to as the tool check device 50) will be described. The tool check device 50 employs tool conditions related to the tool type or tool state that the tool 40 should satisfy as a check criterion. In one embodiment, the tool check is performed based on an evaluation value related to the captured image 15. The evaluation value in one embodiment is a brightness value related to the captured image 15, as described below.

[0018] The tool type is the type of the tool 40 to be attached to the robot arm 30. In one embodiment, the tool type is any one of the clamper 41, the chuck 42, or the knife 43. For example, if the knife 43L is attached to the robot arm 30b even though the chuck 42L should be attached, the tool condition related to the tool type is not satisfied. Such a case may occur, for example, when an operator improperly replaces the tool 40.

[0019] The tool state is a state that the tool 40 attached to the robot arm 30 should satisfy. In one embodiment, the tool state includes an open or closed state of the clamper 41 and the chuck 42, and a normal state of the knife 43. For example, if the clamper 41 or the chuck 42 should be in a closed state but the tool 40 is in an open state, the tool condition regarding the tool state is not satisfied. Such a case may occur, for example, when there is a defect in the work of connecting the clamper 41 or the chuck 42 to the air cylinder with an air pipe. Alternatively, if the knife 43 is missing but should be in a normal state, the tool condition regarding the tool state is not satisfied. Such a case may occur, for example, when the knife 43 is continuously used.

[0020] In one embodiment, the above tool type and tool state are managed in association with each other. Therefore, the tool checking device 50 can check whether the type and state of the tool 40 are appropriate in a single check. As a specific example, the tool checking device 50 can determine whether the tool conditions associated with the clamper 41 as the tool type and the open state as the tool state are satisfied in a single check. Alternatively, the tool checking device 50 can determine whether the tool conditions associated with the knife 43L as the tool type and the normal state as the tool state are satisfied in a single check.

[0021] In other embodiments, the tool type and the tool state may not be associated with each other. For example, the tool checking device 50 may determine whether the tool condition related to the tool type and the tool condition related to the tool state are satisfied individually. In other embodiments, the tool condition may be a condition related only to the tool type. That is, it may be determined whether or not the type of the tool 40 attached to the robot arm 30 is appropriate. Alternatively, the tool condition may be a condition related only to the tool state. For example, in an embodiment in which only a single tool 40 is attached to the robot arm 30, it is only necessary to determine whether or not the state of the tool 40 is appropriate.

[0022] The following describes the configuration of the tool check device 50. The tool check device 50 of one embodiment includes a condition acquisition unit 51, a tool movement control unit 52, an imaging control unit 53, an image processing unit 55, a brightness value acquisition unit 56, and a determination unit 59. The functions of these components are realized by a processor 91 (see FIG. 6), as will be described later.

[0023] The condition acquisition unit 51 in one embodiment is configured to acquire the tool condition according to a work schedule (operation schedule) of the robot arm 30 after it is determined that the tool condition is satisfied. For example, if work using the clamper 41 in an open state is scheduled as the work after it is determined that the tool condition is satisfied, the condition acquisition unit 51 acquires a tool condition in which the tool type is the clamper 41 and the tool state is an open state. In one embodiment, a plurality of tasks may be scheduled for each of a plurality of robot arms 30. In this case, the condition acquisition unit 51 may acquire tool conditions for the plurality of robot arms 30.

[0024] The tool movement control unit 52 in one embodiment is provided for controlling the robot arm 30 so that the tool 40 attached to the robot arm 30 moves to a specified position P2 (see FIG. 2). As a more specific example, the tool movement control unit 52 is provided for controlling the robot arm 30 so that the tool 40 moves from an inner position P1 (see FIG. 2) within the imaging range of the imaging device 8 to a specified position P2 at which the difference in evaluation value depending on whether or not the tool condition is satisfied is larger than that at the inner position P1. Moreover, the tool movement control unit 52 in one embodiment is configured to selectively control a robot arm 30 to be controlled among the multiple robot arms 30. In one embodiment, the robot arm 30 to be controlled is specified based on the tool conditions acquired by the condition acquisition unit 51.

[0025] The imaging control unit 53 in one embodiment is provided to control the imaging device 8 to capture an image of the tool 40 that has moved to the specified position P2.

[0026] The image processing unit 55 of one embodiment is configured to perform image processing associated with the tool condition on the captured image 15 and generate a processed image 18 (see FIGS. 5A and 5B ) in which a related region 17 associated with the tool condition is extracted. The generated processed image 18 is an image based on the captured image 15. The processed image 18 may be generated by performing image processing on the entire captured image 15, or may be generated by performing image processing on a partial image of the captured image 15. In one embodiment, the region of interest 17 is an area that is set so that the image will differ depending on whether the tool condition is satisfied or not. The processed image 18 from which the region of interest 17 is extracted is used by the determination unit 59 to determine whether the tool condition is satisfied. An example of the associated region 17 associated with a tool condition will be described. For example, the associated region 17 associated with a tool condition related to the tool state of the clamper 41 is set to be a region into or out of which at least a part of the movable part of the clamper 41 enters or exits depending on the state (open state or closed state) of the clamper 41. Note that the associated region 17 may be set at the time of judgment by the judgment unit 59, or may be set in advance before the judgment. In one embodiment, the region of interest 17 is the region along at least a portion of the contour of the tool 40 that satisfies the tool conditions and is the region to be trimmed. In one embodiment, the processed image 18 from which the relevant region 17 has been extracted is an image generated by mask processing. In this embodiment, the image processor 55 is configured to perform mask processing using a reference image 14 (see FIG. 4) associated with the tool condition to generate an image from which the relevant region 17 associated with the tool condition has been extracted as the processed image 18. The reference image 14 associated with the tool condition will be described later.

[0027] The luminance value acquisition unit 56 in one embodiment is configured to acquire the luminance value of a region included in the captured image 15 as an evaluation value based on the captured image 15. The region included in the captured image 15 is, for example, the entire region of the processed image 18. Note that the luminance value acquisition unit 56 may acquire only the luminance value in the associated region 17 as the region included in the captured image 15. In one embodiment, the brightness value acquisition unit 56 acquires the RGB brightness values ​​of an area included in the captured image 15 .

[0028] The determination unit 59 in one embodiment is configured to determine whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool 40 is satisfied, based on the captured image 15 captured by the imaging device 8. As a more specific example, the determination unit 59 is configured to determine whether or not a tool condition is satisfied, based on the captured image 15 of the tool 40 attached to the robot arm 30 that is to be controlled by the tool movement control unit 52. Note that in one embodiment in which there are multiple robot arms 30 to be controlled, the determination unit 59 may sequentially determine whether or not the tool condition is satisfied for each of the tools 40 attached to these robot arms 30. The determination unit 59 in one embodiment is configured to determine whether or not the tool condition is satisfied based on an evaluation value related to the captured image 15. The evaluation value is, for example, a luminance value of the processed image 18, and a more specific example is the sum of the luminance values ​​of the processed image 18. Note that the evaluation value may be a luminance value of the captured image 15 that has not been subjected to image processing.

[0029] In other embodiments, it is not necessary to provide the condition acquisition unit 51. For example, if the tool condition is uniquely determined, the image processing unit 55 may perform image processing associated with the tool condition to generate the processed image 18, and the determination unit 59 may determine whether the tool condition is satisfied based on the processed image 18.

[0030] 2 is a diagram showing a flow of movement of the tool 40 according to an embodiment to a specified position P2. In FIG. 2, a clamper 41 as an example of the tool 40 is illustrated as viewed along the conveying direction of the conveying device 7. In one embodiment, the clamper 41 moves from a movement start position P0 to a specified position P2 via an inner position P1 by the tool movement control unit 52 controlling the robot arm 30a. The movement start position P0 may be a position outside or within the imaging range of the imaging device 8. The inner position P1 and the specified position P2 are different positions within the imaging range. In one embodiment, the movement start position P0, the inner position P1, and the specified position P2 are concepts including a three-dimensional coordinate position and a rotational position (rotational attitude) with the horizontal direction as the axial direction. Therefore, in the process in which the clamper 41 moves from the inner position P1 to the specified position P2, the position of the clamper 41 in the three-dimensional coordinate system and the rotational position (rotational attitude) of the clamper 41 with the horizontal direction as the axial direction are adjusted. In one embodiment, the specified position P2 is a position where the difference in evaluation value for the captured image 15 between when the tool condition is satisfied and when it is not satisfied is larger than that at the inner position P1. As a more specific example, the clamper 41 at the specified position P2 reflects more light toward the image capture device 8 than when it is at the inner position P1. Therefore, when the clamper 41 is at the specified position P2, the luminance value of the captured image 15 when the tool condition is satisfied is larger. On the other hand, the luminance value of the captured image 15 when the tool condition is not satisfied does not change significantly regardless of whether the clamper 41 is located at the specified position P2 or the inner position P1. As a result, the difference in evaluation value for the captured image 15 between when the tool condition is satisfied and when it is not satisfied is larger at the specified position P2 than that at the inner position P1. The evaluation value for the captured image 15 at the inner position P1 may be obtained experimentally in advance (eg, by experiment or simulation) before the determination by the determination unit 59. In the above description, the process in which the clamper 41 moves to the specified position P2 has been described, but the other tools 40 (for example, the chuck 42 or the knife 43) also move in a similar manner by the tool movement control unit 52 controlling the robot arms 30b and 30c. A detailed description will be omitted.

[0031] In addition, in another embodiment, the movement start position P0 may coincide with the inner position P1. In other words, the tool movement control unit 52 may start the movement of the tool 40 from the inner position P1. In another embodiment, the rotational position (rotational attitude) of the tool 40 about the vertical direction as the axial direction may be adjusted in the process of the tool 40 moving to the specified position P2.

[0032] FIG. 3 is a diagram showing a defined position P2 of the tool 40 according to one embodiment. For convenience, in Figure 3, the center of gravity position of the connection point of the tool 40 with the robot arm 30 is illustrated as the specified position P2, but as described above, the specified position P2 may also include a rotational position (rotational posture) of the tool 40 with the horizontal or vertical direction as the axial direction. In one embodiment, different specified positions P2 are set according to the tool type to be satisfied. For example, different specified positions P2 are set in the conveying direction for each of the clamper 41, the chucks 42L and 42R, and the knives 43L and 43R. Note that these specified positions P2 may be at the same height position. In one embodiment, if the tool type to be satisfied is the same, the same specified position P2 is set regardless of the tool state to be satisfied. For example, if the tool type to be satisfied is the clamper 41, the same specified position P2 is set regardless of the tool state to be satisfied (open state or closed state). In one embodiment, the photographing device 8 has a wide photographing range such that any of the specified positions P2 can be photographed. In this case, a partial image extracted according to the specified position P2 from the image generated by the photographing device 8 becomes the photographed image 15. The image processing for extracting the partial image is, for example, a trimming process or a cropping process. In other embodiments, the shooting range of the captured image 15 may be narrow enough to include only one of the specified positions P2. In this case, the optical axis direction of the photographing device 8 may be adjusted according to the specified position P2, and the image generated by the photographing device 8 may be treated as the captured image 15 as it is.

[0033] FIG. 4 is a diagram showing reference image data 96 which is data of the reference image 14 for executing image processing according to one embodiment. The reference image 14 in one embodiment is associated with a tool condition. The image processing unit 55 of the embodiment performs mask processing on the captured image 15 using the reference image 14 associated with the tool condition. As a result, an image in which the related region 17 associated with the tool condition is extracted is generated as a processed image 18 (see FIGS. 5A and 5B). In one embodiment, as an example, a total of eight types of reference images 14a to 14h are prepared in association with the tool conditions.

[0034] 5A and 5B are diagrams for explaining a determination method by the determining units 59a and 59b (59) according to one embodiment. 5A and 5B, it is assumed that the tool condition to be satisfied is the open state of the clamper 41. "Check A" in the figures shows the process of checking the open state of the clamper 41 that satisfies the above tool condition. Also, "Check B" in the figures shows the process of checking the closed state of the clamper 41 that does not satisfy the above tool condition. In one embodiment, in both checks A and B, the workpiece 5 is placed below the tool 40 when photographed by the photographing device 8 (in Figs. 5A and 5B, the workpiece 5 is hatched to make the drawings easier to see). Note that the workpiece 5 does not have to be visible in the background of the tool 40 when photographed by the photographing device 8. In one embodiment, when checks A and B are performed, the image processing unit 55 performs mask processing on each captured image 15 using the reference image 14 associated with the above tool condition to generate each processed image 18. Then, the brightness value of the processed image 18 is acquired by the brightness value acquisition units 56a and 56b (56).

[0035] The luminance value acquisition unit 56a shown in FIG. 5A is configured to acquire a sum X1 of the luminance values ​​in the processed image 18. For example, if the number of pixels in the x direction (horizontal direction) of the processed image 18 is M, the number of pixels in the y direction (vertical direction) is N, and the luminance value of an arbitrary pixel is B, then the sum X1 of the luminance values ​​acquired by the luminance value acquisition unit 56a is defined by formula (1). Note that i is an arbitrary natural number equal to or less than the number of pixels in the horizontal direction of the processed image 18, and j is an arbitrary natural number equal to or less than the number of pixels in the vertical direction.

number

[0036] The determination unit 59a of the embodiment is configured to determine whether or not the tool condition is satisfied based on the acquired sum X1 of the luminance values. In one embodiment, the determination unit 59a determines whether the tool condition is satisfied in checks A and B based on the sum X1 of the luminance values ​​acquired by the luminance value acquisition unit 56a in each of checks A and B. For example, in check A, the clamper 41 is captured over almost the entire area of ​​the relevant region 17 of the processed image 18, and objects other than the clamper 41 (e.g., the workpiece 5) are hardly captured in the relevant region 17. In this case, the sum X1 of the luminance values ​​of the processed image 18 acquired by the luminance value acquisition unit 56a exceeds the threshold T1 serving as a determination criterion, and the determination unit 59a can determine that the tool condition is satisfied. On the other hand, in check B, the proportion of the clamper 41 in the relevant region 17 of the processed image 18 becomes smaller (most of the movable parts of the clamper 41 retract from the relevant region 17). As a result, the proportion of other objects (e.g., the workpiece 5) in the relevant region 17 becomes larger. As a result, the sum X1 of the luminance values ​​in the processed image 18 becomes equal to or smaller than the threshold T1, and the determination unit 59a can determine that the tool condition is not satisfied.

[0037] The luminance value acquisition unit 56b shown in FIG. 5B acquires the luminance value B ij and the brightness value Bs set for each pixel according to the tool conditions. ij The method is configured to obtain a sum X2 of the differences in the luminance values ​​specified by the following equation (2) using

number

[0038] The determination unit 59b in one embodiment determines whether or not the tool condition is satisfied based on the acquired sum X2 of the differences in the luminance values. In one embodiment, the determination unit 59b determines whether the tool condition is satisfied for each check based on the sum X2 of the differences in the luminance values ​​acquired by the luminance value acquisition unit 56b in each of the checks A and B. For example, in the check A, since the image difference between the processed image 18 and the normal image 12 is small, the sum X2 of the differences in the luminance values ​​falls below the threshold T2 serving as the determination criterion, and the determination unit 59b can determine that the tool condition is satisfied. On the other hand, in check B, the difference between the processed image 18 and the normal image 12 is large, so the sum X2 of the differences in the luminance values ​​is equal to or greater than the threshold T2. This allows the determining unit 59b to determine that the tool condition is not satisfied.

[0039] Although Figures 5A and 5B show an example in which the tool 40 that does not satisfy the tool conditions is a clamper 41 in a closed state, a similar judgment method can be applied and a similar judgment result can be obtained even if another tool 40 that does not satisfy the tool conditions is the target of judgment. In addition, in Figures 5A and 5B, the clamper 41 in an open state is given as an example of a tool condition that should be satisfied, but even if other tool conditions are to be judged, a similar judgment method can be applied and a similar judgment result can be obtained.

[0040] Fig. 6 is a block diagram showing an electrical configuration of a workpiece machining system 1 according to an embodiment. The components of the above-described tool check device 50 are realized by a machining control unit 90 shown in Fig. 6. A specific method of realizing the components will be described later with reference to Fig. 7.

[0041] The workpiece machining system 1 includes a machining control unit 90 including a processor 91 . The processor 91 is configured to read out a machining control program (tool check program) 95 stored in the ROM 92, load it into the RAM 93, and execute instructions included in the loaded machining control program 95. The processor 91 is a CPU, a GPU, an MPU, a DSP, various other arithmetic devices, or a combination of these. The processor 91 may be realized by an integrated circuit such as a PLD, an ASIC, an FPGA, and an MCU. The ROM 92 is an example of a storage device. The memory 94, which is a component of the machining control unit 90, is a non-volatile memory that stores reference image data 96 and specified position data 97 indicating the specified position P2. In one embodiment, the specified position data 97 may be a data table that also includes identification data indicating the robot arm 30 to be controlled. For example, the specified position data 97 may associate data indicating the specified position P2 of the clamper 41 with identification data of the robot arm 30a to which the clamper 41 is attached.

[0042] The processor 91 of the embodiment is connected to the reception button 6, the transport device 7, the robot arm 30, the imaging device 8, and the alarm device 9 via interfaces not shown.

[0043] The reception button 6 in one embodiment receives the tool conditions to be satisfied by the tool 40. The reception button 6 may be a button having a mechanical structure, or may be a touch panel button. In one embodiment, when an operator mounts a tool 40 on a robot arm 30, the operator inputs tool conditions into the reception button 6. The input tool conditions may be a plurality of conditions according to the number of robot arms 30, for example. The reception button 6 outputs the received tool conditions to the processor 91. When the operator inputs tool conditions into the reception button 6, the operator may also input the robot arm 30 corresponding to the tool conditions. The processor 91 acquires the data output from the reception button 6, thereby acquiring the tool conditions. In other embodiments, the reception button 6 may not be provided. In this case, the processor 91 may acquire the tool conditions indicated by the data included in the machining control program 95, for example.

[0044] In one embodiment, the conveying device 7, the robot arm 30, the imaging device 8, and the alarm device 9 each operate in response to a control signal received from the processor 91. In one embodiment, the robot arm 30 moves the tool 40 to a specified position P2 in response to the received control signal. Furthermore, in one embodiment, the robot arm 30 performs a processing operation on the workpiece 5 in response to the received control signal. In one embodiment, the photographing device 8 performs photographing in response to the received control signal, and outputs the generated photographed image 15 to the processor 91. In one embodiment, the processor 91 outputs the image acquired from the photographing device 8 to the RAM 93. The photographed image 15 may be stored in the memory 94 instead of the RAM 93. The alarm device 9 of one embodiment is a device for issuing an alarm in response to a received control signal when the processor 91 determines that the tool condition is not satisfied. The alarm device 9 of one embodiment may be an image display device, a speaker, a light emitting device, or a combination thereof.

[0045] 7 shows a flowchart of the processing control process according to one embodiment. In the processing control process, the processor 91 loads the processing control program 95 stored in the ROM 92 into the RAM 93, thereby executing the following steps. Information processed by the processor 91 in conjunction with the execution of the process is stored in the RAM 93 or memory 94 as appropriate. In the following description, "step" is abbreviated as "S".

[0046] The processor 91 controls the transport device 7 so that the workpiece 5 is transported to the machining area (S11). Next, the processor 91 acquires tool conditions to be satisfied by the tool 40 (S13). For example, the processor 91 acquires the tool conditions based on data output from the reception button 6. The processor 91 executing S11 functions as the condition acquisition unit 51. Note that in an embodiment in which a plurality of robot arms 30 are provided, the processor 91 may acquire tool conditions corresponding to each robot arm 30.

[0047] The processor 91 refers to the tool conditions acquired in S13 and the specified position data 97 stored in the memory 94, and selectively controls the robot arm 30 to be controlled (S15). As a result, the tool 40 attached to the robot arm 30 to be controlled moves to the specified position P2. For example, if the tool conditions acquired in S11 include the "clamper 41 in an open state," the clamper 41 attached to the robot arm 30a moves to the specified position P2, and the chucks 42 and knives 43 attached to the robot arms 30b and 30c retreat to other positions (for example, the movement start position P0) under the control of the processor 91. The processor 91 that executes S15 functions as the tool movement control unit 52. The processor 91 controls the photographing device 8 so as to photograph the tool 40 that has been moved to the specified position P2 by the execution of S15 (S17). The processor 91 stores the photographed image 15 based on the photographing by the photographing device 8 in, for example, the RAM 93. The processor 91 that executes S17 functions as the photographing control unit 53.

[0048] The processor 91 performs image processing on the captured image 15 generated in S17 (S19). In one embodiment, the processor 91 refers to the reference image data 96 stored in the memory 94 and acquires the reference image 14 according to the tool condition acquired in S13. Then, the processor 91 performs mask processing on the captured image 15 acquired in S17 using the acquired reference image 14. As a result, the processor 91 generates an image in which the relevant region 17 associated with the tool condition is extracted as the processed image 18. The processor 91 that executes S19 functions as the image processing unit 55.

[0049] The processor 91 obtains a luminance value of the processed image 18 based on the generated processed image 18 (S21). In one embodiment, the processor 91 obtains a sum X1 of the luminance values ​​or a sum X2 of the differences of the luminance values ​​based on, for example, the above-mentioned formula (1) or formula (2). When obtaining the sum X2 of the differences in the luminance values, the processor 91 refers to the normal image 12 stored in the memory 94 and calculates the luminance value Bs of each pixel. ij may be obtained. The processor 91 that executes S21 functions as the luminance value acquisition units 56a, 56b (56).

[0050] The processor 91 determines whether the tool condition acquired in S13 is satisfied based on the acquired brightness value (S23). For example, the processor 91 determines whether the tool condition is satisfied by comparing the above-mentioned sum X1 of the luminance values ​​or the sum X2 of the luminance value differences with the threshold value T1 or the threshold value T2. The processor 91 that executes S23 functions as the determination units 59a, 59b (59).

[0051] If it is determined that the tool condition is not satisfied (S23: NO), the processor 91 controls the alarm device 9 so as to issue an alarm (S25), and ends this control process. In one embodiment, when an alert is issued, the operator recognizes that the tool conditions are not met and can perform operations such as replacing the tool 40 on the robot arm 30 so that the tool conditions are met.

[0052] In one embodiment, when it is determined that the tool condition is satisfied (S23: YES), the processor 91 determines whether the tool check is completed (S27). For example, when there remains a tool condition among the multiple tool conditions acquired in S13 for which the satisfaction or non-satisfaction has not been determined (S27: NO), the processor 91 repeats S15 to S23. On the other hand, when the determination of all the tool conditions is completed (S27: YES), the processor 91 advances the process to S29.

[0053] The processor 91 controls the robot arm 30 so that the tool 40, which was at the specified position P2, retreats to another position (for example, the movement start position P0) (S29). After that, the processor 91 controls the photographing device 8 so that the workpiece 5 is photographed (S31), and analyzes the image generated by the photographing device 8 (S33). In one embodiment, the processor 91 executes image analysis for performing processing suitable for the photographed workpiece 5. As a specific example, when the workpiece 5 is a bone-in leg meat of a livestock, image analysis is executed for identifying the position of the bone contained in the workpiece 5. The analysis may be executed, for example, by inputting the image photographed in S31 to a learning model that has been machine-learned in advance. In this case, the processor 91 may be equipped with a GPU for executing arithmetic processing based on the machine-learned learning model. The processor 91 controls the robot arm 30 so that the processing operation of the workpiece 5 based on the result of the image analysis is performed (S35). After the processing of the workpiece 5 is completed, the processor 91 ends this control process.

[0054] In another embodiment, the execution timing of S11 may be after it is determined that the tool check is completed (S27: YES). In this case, the workpiece 5 is not captured in the captured image 15 in the shooting performed in S17. In other embodiments, for example, when the tool condition to be determined is uniquely determined, neither S13 nor S27 may be executed. Note that the uniquely determined tool condition does not only mean a single tool condition, but also means a plurality of tool conditions.

[0055] Hereinafter, a tool checking device 50 for a robot arm, a tool checking program 95 for a robot arm, and a tool checking method for a robot arm according to some embodiments will be described.

[0056] (1) A tool checking device 50 for a robot arm according to at least one embodiment of the present invention includes: a tool movement control unit 52 for controlling the robot arm 30 so that the tool 40 attached to the robot arm 30 moves to a specified position P2; an imaging control unit 53 for controlling the imaging device 8 to image the tool 40 that has moved to the specified position P2; a determination unit for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool is satisfied based on a photographed image taken by the photographing device; Equipped with.

[0057] According to the above configuration (1), before the photographing device 8 performs photographing, the tool movement control unit 52 controls the robot arm 30 so that the tool 40 moves to the specified position P2. This reduces the variation in the photographed image 15 of the tool 40 when the tool condition is satisfied. Therefore, the determination unit 59 can determine with high accuracy whether the tool condition is satisfied based on the photographed image 15. Therefore, it can be determined with high accuracy whether the tool 40 satisfies the tool condition.

[0058] (2) In some embodiments, in the configuration of (1) above, the determination unit 59 is configured to determine whether the tool condition is satisfied based on an evaluation value related to the captured image 15; The tool movement control unit 52 is configured to control the robot arm 30 so that the tool 40 moves from an inner position P1 within the shooting range of the shooting device 8 to the specified position P2 where the difference in the evaluation value depending on whether or not the tool conditions are satisfied is larger than that at the inner position P1.

[0059] According to the above configuration (2), the tool 40 moves to the specified position P2 under the control of the tool movement control unit 52, and the evaluation value for the captured image 15 changes significantly depending on whether the tool conditions are satisfied. Therefore, it is possible to accurately determine whether the tool conditions are satisfied. For example, the luminance value of the association region 17 when the tool condition is satisfied is greater when the clamper 41 is at the specified position P2 than when it is at the inner position P1. On the other hand, whether the clamper 41 is located at the inner position P1 or the specified position P2, if the tool condition is not satisfied, an object other than the tool 40, such as the workpiece 5, is reflected in the association region 17, and there is no significant difference in the luminance value of the association region 17 at both positions. As a result, the difference in luminance value depending on whether the tool condition is satisfied is greater when the clamper 41 is at the specified position P2. This enables the determination unit 59 to determine whether the tool condition is satisfied with high accuracy.

[0060] (3) In some embodiments, in the configuration of (1) or (2), The tool check device 50 includes a brightness value acquisition unit 56 for acquiring a brightness value of a region (related region 17) included in the captured image 15, The determination unit 59 is configured to determine whether the tool condition is satisfied or not based on the acquired luminance value.

[0061] According to the above configuration (3), the determination unit 59 can quantitatively determine whether the tool conditions are satisfied based on the luminance values ​​of areas included in the captured image 15. Therefore, it is possible to accurately determine whether the tool 40 satisfies the tool conditions.

[0062] (4) In some embodiments, in the configuration of (3) above, The luminance value acquisition unit 56a (56) is configured to acquire a sum X1 of the luminance values ​​of the region (related region 17) included in the captured image 15, The determination unit 59a (59) is configured to determine whether the tool condition is satisfied or not based on the acquired sum X1 of the luminance values.

[0063] According to the above configuration (4), the determination unit 59a (59) determines whether or not the tool conditions are satisfied based on the sum X1 of the luminance values ​​of the areas included in the captured image 15. Therefore, even when the photographing conditions of the tool 40 change, it is possible to accurately determine whether or not the tool conditions are satisfied. For example, when the tool 40 is used continuously, the photographing conditions may change due to the workpiece 5 adhering to the tool 40. In this case, even if the tool 40 to be judged satisfies the tool conditions, the luminance value in the photographed image 15 tends to decrease, and an erroneous judgment may occur as to whether the tool conditions are satisfied. In this regard, in an embodiment in which the luminance value acquisition unit 56a acquires the sum of luminance values ​​X1, even if the workpiece 5 adheres to the tool 40, the rate of decrease in the sum of luminance values ​​X1 is small. Therefore, even if the photographing conditions change, the accuracy of the judgment as to whether the tool conditions are satisfied is maintained.

[0064] (5) In some embodiments, in any one of the configurations (1) to (4) above, The tool movement control unit 52 is configured to selectively control the robot arm 30 to be controlled among the plurality of robot arms 30, The determination unit 59 is configured to determine whether or not the tool condition is satisfied for the tool 40 attached to the robot arm 30 to be controlled.

[0065] According to the above configuration (5), the tool movement control unit 52 selectively moves the robot arm 30 to which the tool 40 for which a determination regarding the tool condition is required is attached. Therefore, the tool check device 50 can efficiently determine whether the tool condition is satisfied.

[0066] (6) In some embodiments, in any one of the configurations (1) to (5) above, the tool check device 50 includes an image processing unit 55 for performing image processing associated with the tool condition on the captured image 15 and generating a processed image 18 in which a related region 17 associated with the tool condition is extracted; The determination unit 59 is configured to determine whether the tool condition is satisfied or not based on the processed image 18 .

[0067] According to the above configuration (6), a processed image 18 is generated in which the associated region 17 associated with the tool condition is extracted, and the determination unit 59 determines whether or not the tool condition is satisfied based on the generated processed image 18. Therefore, it is possible to accurately determine whether or not the tool 40 satisfies the tool condition based on the associated region 17 associated with the tool condition.

[0068] (7) In some embodiments, in the configuration of (6) above, The image processing unit 55 is configured to perform mask processing on the captured image 15 using a reference image 14 associated with the tool conditions, and generate an image in which the relevant region 17 associated with the tool conditions is extracted as the processed image 18.

[0069] According to the above configuration (7), the determining unit 59 can determine with high accuracy whether the tool 40 satisfies the tool condition based on the processed image 18 that has been subjected to the mask processing.

[0070] (8) A tool check program 95 for a robot arm according to at least one embodiment of the present invention includes: To your computer a tool 40 movement control step (S15) for controlling the robot arm 30 so that the tool 40 attached to the robot arm 30 moves to a specified position P2; an imaging control step (S17) for controlling the imaging device 8 to image the tool 40 that has moved to the specified position P2; a determination step (S23) for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool 40 is satisfied based on the photographed image 15 photographed by the photographing device 8; Execute the command.

[0071] According to the above configuration (8), for the reason described above in (1), it is possible to accurately determine whether the tool 40 satisfies the tool conditions.

[0072] (9) A method for checking a tool for a robot arm according to at least one embodiment of the present invention includes the steps of: a tool 40 movement control step (S15) for controlling the robot arm 30 so that the tool 40 attached to the robot arm 30 moves to a specified position P2; an imaging control step (S17) for controlling the imaging device 8 to image the tool 40 that has moved to the specified position P2; a determination step (S23) for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool 40 is satisfied based on the captured image 15 captured by the imaging device 8; Equipped with.

[0073] According to the configuration (9) above, for the reason (1) above, it is possible to accurately determine whether the tool 40 satisfies the tool conditions. [Explanation of symbols]

[0074] 8: Photographing device 14: Reference image 17: Related Areas 18: Processed image 30: Robot arm 40: Tools 50: Tool check device 52: Tool movement control section 53: Shooting control unit 55: Image processing section 56: Brightness value acquisition unit 59: Judgment section 95: Tool check program P1 :Inner position P2: Default position

Claims

1. A tool checking device for a robot arm in a workpiece processing system, comprising a robot arm equipped with a tool for processing a workpiece which is food, and an image capturing device for capturing an image of the workpiece, the system being configured to process the workpiece with the tool based on a result of image analysis of the captured image of the workpiece, a tool movement control unit for controlling the robot arm so that the tool attached to the robot arm moves to a specified position; an imaging control unit for controlling the imaging device to image the tool that has moved to the specified position; a determination unit for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool is satisfied based on a photographed image of the tool photographed by the photographing device; A tool checking device for a robot arm comprising:

2. A luminance value acquisition unit for acquiring a luminance value of an area included in the captured image, The determination unit is configured to determine whether the tool condition is satisfied based on the acquired luminance value, 2. A tool checking device for a robot arm as described in claim 1, wherein the tool movement control unit is configured to control the robot arm so that the tool moves from an inner position within the shooting range of the shooting device to the specified position where the difference in brightness value depending on whether the tool condition is satisfied is larger than that of the inner position before the shooting device photographs the tool under the control of the shooting control unit and the judgment unit judges whether the tool condition is satisfied.

3. a brightness value acquisition unit for acquiring a brightness value of an area included in the captured image, The tool check device for a robot arm according to claim 1 , wherein the determination unit is configured to determine whether or not the tool condition is satisfied based on the acquired luminance value.

4. the luminance value acquisition unit is configured to acquire a sum of luminance values ​​of the regions included in the captured image, The tool check device for a robot arm according to claim 3 , wherein the determination unit is configured to determine whether or not the tool condition is satisfied based on the acquired sum of the luminance values.

5. the tool movement control unit is configured to selectively control the robot arm to be a control target among the plurality of robot arms, 5. The tool checking device for a robot arm according to claim 1, wherein the determination unit is configured to determine whether or not the tool condition is satisfied for the tool attached to the robot arm to be controlled.

6. an image processing unit for performing image processing associated with the tool condition on the captured image and generating a processed image in which a related area associated with the tool condition is extracted, The tool check device for a robot arm according to claim 1 , wherein the determination unit is configured to determine whether or not the tool condition is satisfied based on the processed image.

7. 7. The tool checking device for a robot arm according to claim 6, wherein the image processing unit is configured to perform mask processing on the captured image using a reference image associated with the tool condition, and generate an image in which the relevant area associated with the tool condition is extracted as the processed image.

8. A program for checking a tool for a robot arm in a workpiece processing system comprising a robot arm equipped with a tool for processing a workpiece which is food, and an image capturing device for capturing an image of the workpiece, the system being configured to process the workpiece with the tool based on a result of image analysis of the captured image of the workpiece, comprising: To your computer a tool movement control step for controlling the robot arm so that the tool attached to the robot arm moves to a fixed position; an imaging control step for controlling the imaging device to image the tool that has been moved to the fixed position; a determination step for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool is satisfied based on a photographed image of the tool photographed by the photographing device; Tool checking program for the robot arm to execute.

9. A method for checking a tool for a robot arm in a workpiece processing system comprising a robot arm equipped with a tool for processing a workpiece which is a food product, and an image capturing device for capturing an image of the workpiece, the method being configured to process the workpiece with the tool based on a result of image analysis of the captured image of the workpiece, comprising: a tool movement control step for controlling the robot arm so that the tool attached to the robot arm moves to a fixed position; an imaging control step for controlling the imaging device to image the tool that has been moved to the fixed position; a determination step for determining whether or not a tool condition related to a tool type or a tool state that should be satisfied by the tool is satisfied based on a photographed image of the tool photographed by the photographing device; A tool checking method for a robot arm comprising:

Images