Tool inspection device for robot arm, storage device storing tool inspection program, and tool inspection method

Abstract

A tool inspection device for a robot arm includes an image processing section configured to perform image processing associated with a tool condition related to a tool type or a tool state that the tool should satisfy on a captured image of a tool attached to the robot arm, and generate a processed image in which a region of interest associated with the tool condition is extracted; and a determination section configured to determine whether the tool attached to the robot arm satisfies the tool condition based on the processed image.

Description

Technical Field

[0001] This disclosure relates to a tool inspection device for a robotic arm, a storage device for storing tool inspection programs, and a tool inspection method. Background Technology

[0002] Previously known robotic arms were equipped with interchangeable tools for processing or manipulating objects. For example, the robotic arm disclosed in Patent Document 1 can be equipped with any of a variety of tools depending on the processing performed on the object. This robotic arm can hold objects by opening and closing the tools.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2018-158405 Summary of the Invention

[0006] (a) Technical problems to be solved

[0007] In order to properly utilize the aforementioned robotic arm for processing and other operations, the following tool conditions need to be met depending on the processing being performed: the installed tool is of a suitable type, and the installed tool is in a suitable state (e.g., open or closed). Regarding this, Patent Document 1 does not disclose a specific structure for accurately determining whether the tool meets the tool conditions.

[0008] The purpose of this disclosure is to provide a tool inspection device for a robotic arm, a storage device for storing tool inspection programs, and a tool inspection method, which can determine with high precision whether a tool meets the tool conditions.

[0009] (II) Technical Solution

[0010] A tool inspection apparatus for a robotic arm according to at least one embodiment of the present invention includes: an image processing unit for performing image processing on an image of a tool mounted on the robotic arm, which is associated with tool conditions related to tool type or tool state that the tool should meet, and generating a processed image in which an associated region associated with the tool conditions is extracted; and a determination unit configured to determine, based on the processed image, whether the tool mounted on the robotic arm meets the tool conditions.

[0011] Regarding a storage device for storing a tool inspection program for a robotic arm according to at least one embodiment of the present invention, the program is used to inspect tools for a robotic arm, causing a computer to execute: an image processing step for performing image processing on an image of the tool mounted on the robotic arm, associating the image with tool conditions related to the tool type or tool state that the tool should meet, and generating a processed image with the associated region extracted from the tool conditions; and a determination step for determining, based on the processed image, whether the tool mounted on the robotic arm meets the tool conditions.

[0012] The tool inspection method for a robotic arm according to at least one embodiment of the present invention is used to inspect tools for a robotic arm, comprising: an image processing step for performing image processing on an image of the tool mounted on the robotic arm, which is associated with tool conditions related to tool type or tool state that the tool should meet, to generate a processed image in which an associated region related to the tool conditions has been extracted; and a determination step for determining, based on the processed image, whether the tool mounted on the robotic arm meets the tool conditions.

[0013] (III) Beneficial Effects

[0014] According to some embodiments, a tool inspection device for a robotic arm, a storage device for storing tool inspection programs, and a tool inspection method can be provided, which can determine with high precision whether a tool meets the tool conditions. Attached Figure Description

[0015] Figure 1 This is a diagram showing the workpiece processing system of the first embodiment.

[0016] Figure 2 This is a diagram representing reference image data, i.e., reference image data, used to perform image processing in one embodiment.

[0017] Figure 3A This is a diagram illustrating the determination method of the determination unit in the first embodiment.

[0018] Figure 3B This is another diagram illustrating the determination method of the determination unit in the first embodiment.

[0019] Figure 4 This is a block diagram illustrating the circuit structure of a workpiece processing system according to one embodiment.

[0020] Figure 5 This is a flowchart of the processing control procedure in one embodiment.

[0021] Figure 6 This is a diagram illustrating the workpiece processing system of the second embodiment.

[0022] Figure 7This is a diagram representing a learned model associated with tool conditions in one implementation method.

[0023] Figure 8 This is a diagram illustrating the generation process of a learned model in one implementation method.

[0024] Figure 9 This is a diagram illustrating the determination method of the determination unit in the second embodiment.

[0025] Figure 10 This is a diagram showing the workpiece processing system of the third embodiment.

[0026] Figure 11 This is a diagram showing the associated region data 21 of one embodiment.

[0027] Figure 12 This is a diagram illustrating the determination method of the determination unit in the third embodiment. Detailed Implementation

[0028] Hereinafter, some embodiments of the present invention will be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc., of the constituent components described as embodiments or shown in the drawings are not intended to limit the scope of the present invention, but are merely illustrative examples.

[0029] For example, expressions such as "in a certain direction", "along a certain direction", "parallel", "orthogonal", "center", "concentric" or "coaxial" that indicate relative or absolute configuration do not only indicate configuration in a strict sense, but also indicate a state of relative displacement with tolerance or at an angle or distance that can achieve the same level of functionality.

[0030] For example, expressions such as "same," "equal," and "homogeneous" that indicate that things are in an equal state do not only mean that they are strictly equal, but also that there is a difference in degree or that they can achieve the same function.

[0031] For example, expressions representing shapes such as quadrilaterals and cylinders do not only refer to quadrilaterals and cylinders in a strictly geometric sense, but also include shapes with concave and convex parts, chamfers, etc., within the range that can achieve the same effect.

[0032] On the other hand, expressions that "exist," "have," "possess," "include," or "have" a constituent element are not exclusive expressions that exclude the existence of other constituent elements.

[0033] A workpiece processing system 1 according to one embodiment will be described. The workpiece processing system 1 according to one embodiment is configured to process a workpiece 5 using a tool 40. The workpiece 5 is the object to be processed by the tool 40. As an example, the workpiece 5 is a food product such as agricultural products, livestock products, or aquatic products. The food product can be either fresh or processed. The first, second, and third embodiments where the workpiece 5 is fresh meat will be described below.

[0034] (First Implementation)

[0035] Figure 1 The workpiece processing system 1a(1) of the first embodiment shown includes: a conveying device 7 for conveying workpiece 5; a robotic arm 30 for processing workpiece 5; a camera device 8 for taking pictures of tool 40; an illumination unit 4 for illuminating the area captured by the camera device 8; and a tool inspection device 50a(50) for the robotic arm.

[0036] In one embodiment, the conveying device 7 is a belt conveyor that conveys the workpiece 5 in a horizontal direction.

[0037] In one embodiment, the camera device 8 is configured to capture images of the tool 40 from above. In this embodiment, the image 15 captured by the camera device 8 is a top-down view. Figure 1 The image 15 shows a clamp 41 (described later) as an example of a tool 40.

[0038] In one embodiment, the robotic arm 30 is configured to be equipped with a tool 40. In another embodiment, the tool inspection device 50 uses captured image 15 to inspect whether the tool 40 is suitable. The structure of the robotic arm 30, the tool 40, and the tool inspection device 50a will be described in detail below.

[0039] In another embodiment, the conveying device 7 may hold and convey the workpiece 5 in a suspended position. Furthermore, the camera device 8 may be configured to photograph the tool 40 in a horizontal direction or in a direction inclined relative to the horizontal direction. Additionally, the workpiece processing system 1a may not include the illumination unit 4. In this case, the camera device 8 may include the function of the illumination unit 4.

[0040] The structure of the robotic arm 30 will be described. In one embodiment, the robotic arm 30 is an industrial robot. More specifically, as an example, the robotic arm 30 is a multi-joint robot. The robotic arm 30 can be a vertical multi-joint robot, a horizontal multi-joint robot, or a combination thereof.

[0041] One embodiment of the robotic arm 30 includes robotic arms 30a, 30b, and 30c. In one embodiment, the tool 40 mounted on the robotic arm 30 is formed of a metallic material. Furthermore, in one embodiment, the tool 40 has a surface that, for example, more readily reflects light compared to the workpiece 5.

[0042] One embodiment of the tool 40 includes: a clamp 41 for gripping the workpiece 5, a clamping member 42 for clamping the workpiece 5, and a cutting tool 43 for cutting the workpiece 5.

[0043] In one embodiment, a clamp 41 is mounted on a robotic arm 30a, and a clamping member 42 and a cutting tool 43 are mounted on a robotic arm 30b or a robotic arm 30c.

[0044] In one embodiment, symmetrical tools 40 are prepared and used differently depending on the type of workpiece 5. As a specific example, clamping elements 42 include clamping elements 42L and 42R, and cutting tools 43 include cutting tools 43L and 43R. These tools 40 are selectively mounted on robotic arms 30b and 30c. For example, when the workpiece 5 being transported by the conveyor 7 is either the left or right leg meat of an animal, clamping elements 42L and cutting tools 43R are mounted on robotic arms 30b and 30c, respectively. Similarly, when the workpiece 5 is either the left or right leg meat of an animal, cutting tools 43L and clamping elements 42R are mounted on robotic arms 30b and 30c, respectively. In one embodiment, these mounting operations are performed by an operator. In another embodiment, these mounting operations can also be performed by another robot.

[0045] In one embodiment, the clamp 41 and clamping member 42 receive driving force from a drive source and perform opening and closing operations. In one embodiment, a cylinder (not shown) is used as the drive source. In this case, the inlet and outlet of each of the clamp 41 and clamping member 42 are connected to the cylinder via air pipes. Alternatively, a hydraulic cylinder or an electric motor may be used as the drive source.

[0046] In another embodiment, the workpiece processing system 1a may not have symmetrical tools 40. For example, the clamping element 42 may consist of only one of clamping elements 42L and 42R. Similarly, the cutting tool 43 may consist of only one of them. In yet another embodiment, multiple robotic arms 30 may each be equipped with only a single tool 40.

[0047] Furthermore, the workpiece processing system 1a is not limited to the case of setting up multiple robotic arms 30. A single robotic arm 30 may be selectively equipped with multiple tools 40, or it may be equipped with only a single tool 40.

[0048] The structure of the tool inspection device 50 (hereinafter sometimes referred to as the tool inspection device 50) for the robotic arm will be described. The tool inspection device 50 uses tool conditions related to the type or state of the tool that the tool 40 should meet as the judgment criteria for inspection. In one embodiment, the tool inspection is performed based on the processed image 18 (described later) after image processing of the captured image 15.

[0049] The tool type refers to the type of tool 40 that should be installed on the robotic arm 30. In one embodiment, the tool type is any one of clamp 41, clamping member 42, or cutting tool 43. For example, if a clamping member 42L should be installed but a cutting tool 43L is installed on the robotic arm 30b instead, the tool condition related to the tool type is not met. Such instances may occur, for example, when an error occurs during a tool 40 change operation performed by an operator.

[0050] The tool state refers to the state that the tool 40 mounted on the robotic arm 30 should meet. In one embodiment, the tool state includes: the open or closed state of the clamp 41 and clamping member 42, and the normal state of the cutting tool 43. For example, if the clamp 41 or clamping member 42 should be closed but the tool 40 is open, the tool conditions related to the tool state are not met. Such an instance may occur, for example, when an error occurs during operation where the clamp 41 or clamping member 42 is connected to a cylinder using an air hose. Alternatively, if the cutting tool 43 should be in a normal state but is damaged, the tool conditions related to the tool state are not met. Such an instance may occur, for example, during continuous use of the cutting tool 43.

[0051] In one embodiment, the tool type and tool status described above are associated and managed. Therefore, the tool inspection device 50 can check whether the type and status of the tool 40 are appropriate in a single inspection. As a specific example, the tool inspection device 50 can determine in a single inspection whether the tool condition associated with the tool type being a clamp 41 and the tool status being an open state is sufficient. Alternatively, it can determine in a single inspection whether the tool condition associated with the tool type being a cutting tool 43L and the tool status being a normal state is sufficient.

[0052] In another embodiment, tool type and tool state may not be associated. For example, the tool inspection device 50 may determine whether each of the tool conditions related to tool type and the tool conditions related to tool state is sufficient.

[0053] In another embodiment, the tool condition can be a condition that relates only to the type of tool. That is, it can be determined only whether the type of tool 40 installed on the robotic arm 30 is appropriate. Alternatively, the tool condition can also be a condition that relates only to the tool state. For example, in an embodiment where the robotic arm 30 is equipped with only a single tool 40, it is sufficient to determine only whether the state of the tool 40 is appropriate.

[0054] The structure of the tool inspection device 50a (50) will be described. One embodiment of the tool inspection device 50a includes: a condition acquisition unit 51, a tool movement control unit 52, a camera control unit 53, an image processing unit 55a (55), a brightness value acquisition unit 56, and determination units 59a and 59b (59). The functions of these components are controlled by a processor 91 (see below) as will be explained later. Figure 4 To achieve this.

[0055] In one embodiment, the condition acquisition unit 51 is configured to acquire tool conditions based on the planned operation (action plan) of the robotic arm 30 after determining that the tool conditions are sufficient. For example, as for the operation after determining that the tool conditions are sufficient, if the operation is planned to be performed by the gripper 41 in the open state, the condition acquisition unit 51 acquires the tool conditions that the tool type is the gripper 41 and the tool state is the open state.

[0056] In one embodiment, the tool movement control unit 52 is configured to control the robotic arm 30 to move the tool 40 to a predetermined position within the shooting area of ​​the camera device 8. In one embodiment, the predetermined position is a position that corresponds to and differs from each of the robotic arms 30a, 30b, and 30c. Alternatively, a predetermined position that is the same for all three robotic arms 30a, 30b, and 30c may be set. Or, the predetermined position may be set according to the tool's condition.

[0057] In one embodiment, the camera control unit 53 is configured to control the camera device 8 to take pictures of the tool 40 that has been moved to a predetermined position.

[0058] In one embodiment, the image processing unit 55a (55) is configured to perform image processing on the captured image 15 of the tool 40 mounted on the robotic arm 30, associating it with tool conditions that the tool 40 should meet. In one embodiment, the image processing unit 55a performs image processing on the captured image 15, associating it with tool conditions acquired by the condition acquisition unit 51 from a plurality of pre-prepared tool conditions.

[0059] One embodiment of the image processing unit 55a is configured to perform the above-described image processing to generate a processed image 18a(18) from which the associated region 17a(17) has been extracted (see reference). Figure 3A , Figure 3BIn one embodiment, the associated region 17a is defined such that it will appear differently in the image depending on whether the tool conditions are met or not. The processed image 18a, from which the associated region 17a is extracted, is used to determine whether the tool conditions are met.

[0060] In one embodiment, the associated region 17a is set in association with tool conditions. For example, the associated region 17a, which is associated with tool conditions related to the tool state of the clamp 41, is set to be a region in which at least a portion of the movable part of the clamp 41 can enter or exit, depending on the state of the clamp 41 (open state or closed state). Furthermore, the associated region 17a can be set when the determination unit 59a makes a determination, or it can be set in advance before the determination.

[0061] In one embodiment, the associated region 17a is a region along at least a portion of the contour of the tool 40 that satisfies the tool conditions, and is the region to be trimmed. In one embodiment, the image processing unit 55a performs a masking process to extract the associated region 17a.

[0062] In one embodiment, the brightness value acquisition unit 56 is configured to acquire the brightness value of the processed image 18a (18). In one embodiment, the RGB brightness value of each pixel of the processed image 18a is acquired.

[0063] In one embodiment, the determination units 59a and 59b (59) are configured to determine whether the tool 40 mounted on the robotic arm 30 meets the tool conditions based on the processed image 18a (18). In one embodiment, the determination units 59a and 59b determine whether the tool conditions obtained by the condition acquisition unit 51 are sufficient based on the brightness value obtained by the brightness value acquisition unit 56. The determination method for the determination units 59a and 59b will be described later.

[0064] In another embodiment, the condition acquisition unit 51 may be omitted. For example, if the tool condition is uniquely determined, the image processing unit 55a can generate the processed image 18a simply by performing image processing associated with the tool condition, and the determination units 59a and 59b can determine whether the tool condition is met based on the processed image 18a.

[0065] Alternatively, in another embodiment, the tool movement control unit 52 may not be provided. For example, when the tool inspection device 50a is located at a distance from the robotic arm 30, the tool inspection device 50a may not have the tool movement control unit 52.

[0066] Figure 2 This is a diagram showing reference image data 96, which is the data of a reference image 14 used for performing image processing according to one embodiment.

[0067] A reference image 14 of one embodiment is associated with tool conditions.

[0068] In one embodiment, the image processing unit 55a performs masking processing on the captured image 15 using a reference image 14 associated with tool conditions. As a result, an image is generated that extracts the associated region 17a associated with the tool conditions, which is then used as the processed image 18a (see reference). Figure 3A , Figure 3B ).

[0069] In one embodiment, as an example, a total of eight reference images 14a to 14h are prepared in association with tool conditions.

[0070] Furthermore, in one embodiment, the reference image 14 can be an image with the same size as the captured image 15 generated by the imaging device 8. Alternatively, it can be an image smaller than the captured image 15. In this case, after a portion of the captured image 15 is cut, the reference image 14 is used for masking. For example, if the specified position of the tool 40 during shooting differs depending on the tool condition, the area of ​​the captured image 15 that is the cutting object can also change according to the type of tool.

[0071] Figure 3A , Figure 3B This is a diagram illustrating a method for making a determination using determination units 59a, 59b (59) of one embodiment.

[0072] exist Figure 3A , Figure 3B In the determination shown, the tool condition that should be met is that the fixture 41 is in the open state. "Check A" in the figure represents the inspection process of the fixture 41 in the open state that meets the above tool condition. In addition, "Check B" in the figure represents the inspection process of the fixture 41 in the closed state that does not meet the above tool condition.

[0073] In one embodiment, during inspections A and B, the workpiece 5 is positioned below the tool 40 when the camera device 8 is used to take pictures (in...). Figure 3A , Figure 3B In the accompanying drawings, workpiece 5 is marked with shading lines for easier observation. Furthermore, when taking photos using the camera device 8, workpiece 5 can be kept out of the background of tool 40.

[0074] In one embodiment, when inspections A and B are performed, the image processing unit 55a uses the reference image 14a associated with the above-mentioned tool conditions to perform masking processing on each captured image 15 to generate each processed image 18a (18). Then, the brightness value acquisition units 56a and 56b (56) acquire the brightness value of the processed image 18a.

[0075] Figure 3AThe brightness value acquisition unit 56a shown is configured to acquire the sum of the brightness values ​​X2 of the processed image 18a.

[0076] When, for example, the number of pixels in the x-direction (horizontal) of the processed image 18a is set to M, the number of pixels in the y-direction (vertical) is set to N, and the brightness value of any pixel is set to B, the sum X2 of the brightness values ​​acquired by the brightness value acquisition unit 56a is defined by equation (2). Furthermore, i is any natural number less than or equal to the number of pixels in the horizontal direction of the processed image 18a, and j is any natural number less than or equal to the number of pixels in the vertical direction.

[0077] [Formula 2]

[0078]

[0079] Since the brightness value of the masked area in the processed image 18a is 0, the brightness value of the associated area 17a of the processed image 18a can be obtained using equation (2).

[0080] In another embodiment, it is also possible to perform a process that only obtains the brightness values ​​of the pixels in the associated region 17a. In this case, the same value as in equation (2) can also be obtained.

[0081] In one embodiment, the determination unit 59a is configured to determine whether the tool conditions are sufficient based on the sum of the acquired brightness values ​​x2.

[0082] In one embodiment, the determination unit 59a determines whether the tool conditions are sufficient in inspections A and B based on the sum X2 of the brightness values ​​acquired by the brightness value acquisition unit 56a in each inspection A and B. For example, in inspection A, the fixture 41 is reflected in approximately the entire associated region 17a of the processed image 18a, while objects other than the fixture 41 (e.g., workpiece 5) are not reflected in the associated region 17a. In this case, the sum X2 of the brightness values ​​of the processed image 18a acquired by the brightness value acquisition unit 56a exceeds the threshold T2 used as a determination criterion, and the determination unit 59a can determine that the tool conditions are met.

[0083] On the other hand, in inspection B, the proportion of the fixture 41 in the associated region 17a of the processed image 18a is relatively small (the movable part of the fixture 41 is basically withdrawn from the associated region 17a). As a result, other objects (e.g., workpiece 5) occupy a larger proportion in the associated region 17a. Consequently, the sum of the brightness values ​​X2 of the processed image 18a is below the threshold T2, and the determination unit 59a can determine that the tool condition is not met.

[0084] Figure 3BThe brightness value acquisition unit 56b shown is configured to acquire the sum X1 of the differences in brightness values ​​determined by the following formula (1), which uses the brightness value B of each pixel in the processed image 18a. ij And the brightness value Bs set for each pixel according to the tool conditions. ij .

[0085] [Formula 3]

[0086]

[0087] In one embodiment, the brightness value acquisition unit 56b acquires the brightness value B of each pixel in the processed image 18a during each inspection A and B. ij The brightness values ​​Bs of each pixel in the normal image 12 corresponding to each pixel in the processed image 18. ij The sum of the differences is X1.

[0088] In one embodiment, as a preparatory step, the image processing unit 55a performs masking processing on the captured image 15 of the tool 40 that is determined to meet the tool conditions. As a result, a processed image, i.e., a normal image 12, is pre-generated by extracting the associated region 17a related to the tool conditions. The brightness value acquisition unit 56b acquires the brightness value Bs by acquiring the normal image 12. ij .

[0089] Furthermore, in another embodiment, the image processing unit 55a may not generate a normal image 12. For example, it can simply use the brightness values ​​Bs that are set in association with each pixel of the processed image 18b. ij It can be pre-stored in a memory.

[0090] In another embodiment, B ij It can replace the brightness values ​​of each pixel in the processed image 18a with only the brightness values ​​of each pixel in the associated region 17a. In this case, Bs ij It only represents the brightness value of the pixel corresponding to the associated region 17a.

[0091] In one embodiment, the determination unit 59b determines whether the tool conditions are sufficient based on the sum of the differences X1 of the acquired brightness values.

[0092] In one embodiment, the determination unit 59b determines whether the tool conditions are sufficient in each inspection based on the sum X1 of the differences in brightness values ​​acquired by the brightness value acquisition unit 56b in each inspection A and B. For example, in inspection A, since the difference between the processed image 18a and the normal image 12 is small, the sum X1 of the differences in brightness values ​​is lower than the threshold T1 used as the determination criterion, and the determination unit 59b can determine that the tool conditions are met.

[0093] On the other hand, in inspection B, since the difference between the processed image 18a and the normal image 12 is large, the sum of the differences in brightness values ​​X1 is above the threshold T1. Therefore, the determination unit 59b can determine that the tool condition is not met.

[0094] In addition, Figure 3A , Figure 3B The example shown is a clamp 41 in a closed state where the tool 40 does not meet the tool condition. However, the same determination method is applied even when other tools 40 that do not meet the tool condition are used as the determination objects, and the same determination result can be obtained.

[0095] In addition, Figure 3A , Figure 3B In this example, the fixture 41 in the open state is given as an example of the tool conditions that should be met. However, even if other tool conditions are the objects of judgment, the same judgment method is applied and the same judgment result can be obtained.

[0096] Figure 4 This is a block diagram showing the circuit structure of a workpiece processing system 1a according to one embodiment. The constituent elements of the tool inspection device 50a (50) described above are formed by... Figure 4 The machining control unit 90 shown in the figure is implemented. It will be used later. Figure 5 Explain the specific implementation method.

[0097] The workpiece processing system 1 includes a processing control unit 90 containing a processor 91.

[0098] The processor 91 is configured to read the processing control program (tool inspection program) 95 stored in the ROM 92 and load it into the RAM 93, and execute the commands contained in the loaded processing control program 95. The processor 91 can be a CPU, GPU, MPU, DSP, or various other computing devices or combinations thereof. The processor 91 can be implemented using integrated circuits such as PLDs, ASICs, FPGAs, and MCUs. The ROM 92 is an example of a storage device.

[0099] The memory 94, which is a component of the machining control unit 90, is a non-volatile memory that stores reference image data 96.

[0100] In one embodiment, the processor 91 is connected to the receiving button 6, the conveying device 7, the robotic arm 30, the camera device 8, and the reporting device 9 via an interface not shown.

[0101] The receiving button 6 and receiving tool 40 of one embodiment should meet the tool conditions. The receiving button 6 can be a button with a mechanical structure or a touch panel button.

[0102] In one embodiment, when the operator attaches the tool 40 to the robotic arm 30, they input tool conditions to the receiving button 6. The input tool conditions may be, for example, multiple conditions corresponding to the number of robotic arms 30. The receiving button 6 outputs the received tool conditions to the processor 91. Furthermore, when the operator inputs tool conditions to the receiving button 6, they can simultaneously input the corresponding robotic arm 30.

[0103] The processor 91 obtains the tool conditions by acquiring the data output from the receive button 6.

[0104] Furthermore, in another embodiment, the receive button 6 may be omitted. In this case, the processor 91 may, for example, acquire the tool conditions represented by the data contained in the machining control program 95.

[0105] In one embodiment, the conveying device 7, robotic arm 30, camera device 8, and reporting device 9 operate according to control signals received from the processor 91. In another embodiment, the robotic arm 30 moves the tool 40 to a predetermined position based on the received control signals. In yet another embodiment, the robotic arm 30 also performs machining operations on the workpiece 5 based on the received control signals.

[0106] In one embodiment, the camera device 8 performs image capture according to the received control signal and outputs the generated captured image 15 to the processor 91. In another embodiment, the processor 91 outputs the image acquired from the camera device 8 to the RAM 93. Alternatively, the captured image 15 may be stored in the memory 94 instead of the RAM 93.

[0107] The reporting device 9 in one embodiment is a device for reporting based on a received control signal when the processor 91 determines that the tool conditions are not met. The reporting device 9 in one embodiment may be an image display device, a speaker, a light-emitting device, or a combination thereof.

[0108] Figure 5 This is a flowchart illustrating a machining control process according to one embodiment. In the machining control process, the processor 91 loads the machining control program 95 stored in the ROM 92 into the RAM 93, thereby executing the following steps. During the execution of the process, the information processed by the processor 91 is appropriately stored in the RAM 93 or the memory 94. In the following description, "step" will be abbreviated as "S".

[0109] The processor 91 controls the conveyor 7 to transport the workpiece 5 to the processing area (S11).

[0110] Next, the processor 91 acquires the tool conditions that the tool 40 should meet (S13). For example, the processor 91 acquires the tool conditions based on the data output from the receiving button 6. The processor 91, which executes S11, functions as a condition acquisition unit 51. Furthermore, in an embodiment where multiple robotic arms 30 are provided, the processor 91 can acquire the tool conditions corresponding to each robotic arm 30.

[0111] The processor 91 controls the robotic arm 30 to move the tool 40 to a predetermined position based on the tool conditions obtained in S13 (S15). For example, when the tool conditions obtained in S11 include "gripper 41 in an open state", the processor 91 controls the gripper 41 mounted on the robotic arm 30a to move to the predetermined position, and the clamping member 42 and the cutting tool 43 mounted on the robotic arms 30b and 30c to other positions. The processor 91, which executes S15, functions as the tool movement control unit 52.

[0112] The processor 91 controls the camera device 8 to capture an image of the tool 40, which has been moved to a predetermined position by executing S15 (S17). The processor 91 stores the captured image 15 generated by the camera device 8 in RAM 93, for example. The processor 91 executing S17 functions as a camera control unit 53.

[0113] Processor 91 performs image processing on the captured image 15 generated in S17 (S19). In one embodiment, processor 91 refers to reference image data 96 stored in memory 94 and acquires a reference image 14 corresponding to the tool conditions acquired in S13. Furthermore, the acquired reference image 14 is used to perform masking processing on the captured image 15 acquired in S17. As a result, processor 91 generates an image with the associated region 17a (17) associated with the tool conditions extracted as a processed image 18a (18). In addition, processor 91 executing S19 functions as an image processing unit 55a (55).

[0114] The processor 91 obtains the brightness value of the processed image 18a (17) based on the generated processed image 18a (S21). In one embodiment, the processor 91 obtains the sum of the differences in brightness values ​​X1 or the sum of brightness values ​​X2 based, for example, on the above-described formula (1) or formula (2).

[0115] Furthermore, when the processor 91 obtains the sum of the differences in brightness values ​​X1, it can refer to the normal image 12 stored in the memory 94 to obtain the brightness value Bs of each pixel. ij .

[0116] The processor 91 executing S21 functions as a brightness value acquisition unit 56a, 56b (56).

[0117] The processor 91 determines whether the tool conditions obtained in S13 (S23) are met based on the acquired brightness value.

[0118] For example, processor 91 determines whether the tool conditions are sufficient by comparing the sum of the differences in the brightness values ​​X1 or the sum of the brightness values ​​X2 with the threshold T1 or the threshold T2. Processor 91, which executes S23, functions as determination units 59a, 59b (59).

[0119] If the tool conditions are not met (S23: No), the processor 91 controls the reporting device 9 to execute a report (S25) and terminates the control process.

[0120] In one embodiment, the operator can identify situations where tool conditions are not met by executing a report, and perform operations such as replacing the tool 40 on the robotic arm 30 to meet the tool conditions.

[0121] In one embodiment, when it is determined that the tool conditions are met (S23: Yes), the processor 91 determines whether the tool check is complete (S27). For example, if there are still tool conditions among the multiple tool conditions obtained in S13 that have not been determined to be sufficient (S27: No), the processor 91 repeats S15 to S23. On the other hand, when all tool conditions have been determined (S27: Yes), the processor 91 causes the process to proceed to S29.

[0122] Processor 91 controls robotic arm 30 to retract tool 40 from a predetermined position to another position (S29). Then, processor 91 controls camera device 8 to capture images of workpiece 5 (S31), and analyzes the images generated by camera device 8 (S33). In one embodiment, processor 91 performs image analysis to perform processing suitable for the captured workpiece 5. As a specific example, if workpiece 5 is bone-in meat from livestock, image analysis is performed to determine the location of bones contained in workpiece 5. This analysis can be performed, for example, by inputting the images captured in S31 into a pre-processed machine learning model. In this case, processor 91 may have a GPU, which performs computational processing based on the machine learning model. Processor 91 controls robotic arm 30 to perform processing operations on workpiece 5 based on the results of the image analysis (S35). When processing of workpiece 5 is completed, processor 91 terminates this control process.

[0123] Furthermore, in another embodiment, S11 can be executed after it is determined that the tool inspection has been completed (S27: Yes). In this case, during the shooting performed in S17, the workpiece 5 is not reflected in the captured image 15.

[0124] In another implementation, for example, if the tool condition to be determined is uniquely determined, S13 and S27 may not be executed. Furthermore, the uniquely determined tool condition does not only refer to a single tool condition, but also to multiple tool conditions.

[0125] (Second Implementation)

[0126] Figure 6 This is a diagram showing the workpiece processing system 1b(1) of the second embodiment. The same reference numerals are used to label the same components as in the first embodiment, and detailed descriptions are omitted (the same applies to the third embodiment described later).

[0127] The workpiece processing system 1b replaces the tool inspection device 50a (50) and is equipped with the tool inspection device 50b (50).

[0128] The tool inspection device 50b replaces the brightness value acquisition unit 56 and the determination unit 59a (59) and includes: a storage unit 54, an evaluation data acquisition unit 58, and a determination unit 59c (59).

[0129] In one embodiment, the learned model 57 stored in the storage unit 54 is configured to output evaluation data related to whether the tool 40 meets the tool conditions when inputting data related to the processed image 18a(18) generated by the image processing unit 55a.

[0130] In one embodiment, the learned model 57 is a model that has undergone deep learning. As an example, the learned model 57 is a GAN (Generative Adversarial Network). More specifically, the GAN is EGBAD (Efficient GAN-Based Anomaly Detection). Furthermore, in another embodiment, the learned model 57 can be a CNN (Convolutional Neural Network) or an RNN (Recurrent Neural Network).

[0131] In one embodiment, the evaluation data acquisition unit 58 is configured to acquire evaluation data, which is the evaluation data output by inputting the processed image 18a generated by the image processing unit 55a into the learned model 57. Furthermore, as the processor unit implementing the function of the evaluation data acquisition unit 58, a dedicated GPU unit can be provided separately from the condition acquisition unit 51, the tool movement control unit 52, etc.

[0132] In one embodiment, the determination unit 59c(59) is configured to determine whether the tool conditions are sufficient based on the acquired evaluation data. The determination method will be described in detail below.

[0133] Figure 7 This is a diagram representing a learned model 57 associated with tool conditions in one implementation.

[0134] The learned model 57 of one embodiment is stored in the storage unit 54 in association with the tool conditions. For example... Figure 7 As shown, the learned models 57a to 57f (57) of one embodiment and the tool conditions related to tool type and tool state are stored in the storage unit 54.

[0135] Alternatively, in another embodiment, the learned model 57 and the tool conditions associated with either the tool type or the tool state may be stored in the storage unit 54. Alternatively, the storage unit 54 may store only a single learned model 57.

[0136] Figure 8 This is a diagram illustrating the generation process of the learned model 57a(57) in one implementation method.

[0137] For example, a pre-learning model 67a (67) is prepared to correspond to the tool conditions of the learned model 57a in the open state of the fixture 41. Furthermore, a normal image 12 is input to the pre-learning model 67a as teacher data 65. This normal image 12 is obtained by the image processing unit 55a performing image processing on multiple captured images 15 that determine the tool condition. Thus, the pre-learning model 67a can undergo machine learning to generate the learned model 57a.

[0138] The generation process for the other learned models 57b to 57h is the same, so detailed explanations are omitted.

[0139] In another embodiment, the image used as teacher data 65 may be the captured image 15 before image processing.

[0140] Figure 9 This is a diagram illustrating the determination method of the determination unit 59c(59) in one embodiment.

[0141] exist Figure 9 In this context, the tool conditions that should be met, as well as the tool 40 used as the judgment object in each inspection A and B, and the captured image 15 used as the image processing object, are all related to... Figure 3A , Figure 3B same.

[0142] exist Figure 9 In the determination shown, by checking A and B, the image processing unit 55a(55) generates a result that matches the original result. Figure 3A, 3B Image 18a(18) after the same processing.

[0143] The evaluation data acquisition unit 58 acquires the evaluation data of the processed image 18a using the learned model 57a corresponding to the tool state (the fixture 41 in the open state). Specifically, the evaluation data acquisition unit 58 inputs each processed image 18a into the learned model 57a and acquires each evaluation data value X3.

[0144] In one embodiment, the determination unit 59c determines whether the tool condition is sufficient by comparing the acquired value X3 with a threshold T3 used as a determination criterion. For example, in an embodiment where EGBAD is used as the learned model 57, the value X3 output from the Discriminator (not shown), a component of the learned model 57, is less than the threshold T3 in check A and greater than the threshold T3 in check B. Therefore, the determination unit 59c can determine that the tool condition is satisfied in check A and not satisfied in check B.

[0145] Furthermore, based on the specific model of the learned model 57, the value X3 of the evaluation data in check A can also exceed the threshold T3.

[0146] (Third Implementation)

[0147] Figure 10 This is a diagram showing the workpiece processing system 1c (1) of the third embodiment. The workpiece processing system 1c replaces the fixture 41 (tool 40) and tool inspection device 50a (50) of the workpiece processing system 1a and has a fixture 41a (tool 40) and a tool inspection device 50c (50).

[0148] The clamp 41a includes a support 44 and a pair of movable parts 47 rotatably supported by the support 44. In one embodiment, the pair of movable parts 47 are opened and closed using a driving force supplied by a cylinder (not shown). Furthermore, in another embodiment, one of the pair of movable parts 47 may be a fixed part fixed to the support 44.

[0149] The clamp 41a also includes an outer surface 46 with a marking 45. In one embodiment, the outer surface 46 is included in the surfaces of the support portion 44 and a pair of movable portions 47. In one embodiment, the marking 45 is formed on the support portion 44 and a movable portion 47 respectively. The marking 45 is text, graphics, symbols, or a combination thereof. In one embodiment, the marking 45 is text.

[0150] In one embodiment, the mark 45 is formed by electrolytic marking of the tool 40. In this embodiment, compared with the embodiment in which the mark 45 is affixed to the tool 40, the tool 40 can be kept clean, and compared with the embodiment in which the mark 45 is affixed by laser engraving, the increase in cost can be suppressed.

[0151] The tool inspection device 50c replaces the image processing unit 55a (55), the brightness value acquisition unit 56, and the determination unit 59a (59) and includes: an image processing unit 55b (55), a storage device 66, a determination processing unit 64, and a determination unit 59d (59).

[0152] In one embodiment, the image processing unit 55b is configured to perform image processing on the captured image 15 in association with tool conditions of the tool 40 having an outer surface 46 with markings 45. The image processing unit 55b can perform masking processing on the captured image 15 using a reference image (not shown), or it can perform cutting processing on the captured image 15. A processed image 18b (18) with the associated region 17b extracted is generated by image processing of the captured image 15.

[0153] In one embodiment, the storage device 66 stores association region data 21 that associates the association region 17b extracted by the image processing unit 55b with tool conditions. In one embodiment, the association region 17b is the inner region of the outline of the tool 40 that satisfies the tool conditions; more specifically, it is the region of the tool 40 that includes the mark 45. In one embodiment, the association region 17b may be a region along the outline of the tool 40, or it may not be a region along the outline.

[0154] In one embodiment, the determination processing unit 64 is configured to perform a determination of the mark 45 on the generated processed image 18b (18). For example, in an embodiment where the mark 45 is text, the determination processing unit 64 is configured to perform a determination of the text that is the mark 45 on the processed image 18b. As an example, the text determination process is optical character recognition processing.

[0155] In one embodiment, the determination unit 59d is configured to determine whether the tool conditions are sufficient based on the processing result of the determination processing unit 64. For example, in an embodiment where the determination processing unit 64 performs optical character recognition processing, the determination unit 59d can determine whether the tool conditions are sufficient based on whether the processed image 18b contains text, i.e., mark 45.

[0156] In another embodiment, the determination unit 59d can determine whether the tool conditions are sufficient based on the specific text identified by the determination processing unit 64.

[0157] Alternatively, in another embodiment, the mark 45 can replace text with a graphic such as a straight line or a circle. In this case, the determination unit 59d can also determine whether the tool conditions are sufficient by determining whether the mark 45 is present or absent through the determination processing unit 64.

[0158] Figure 11 The associated region data 21 of one embodiment is shown. In one embodiment, the associated region data 21 includes: associated region data 21a referenced when determining tool conditions related to tool type, and associated region data 21b referenced when determining tool conditions related to tool state.

[0159] In one embodiment, the data stored in the associated area data 21a is allocated according to the type of tool 40 (clamp 41, clamping members 42L, 42R, cutting tools 43L, 43R). For example, the associated area 17b represented by data A1 allocated to clamp 41 indicates an area containing mark 45 regardless of whether clamp 41 is in an open or closed state. More specifically, as an example, data A1 includes the associated area 17b containing mark 45 on the support 44.

[0160] In one embodiment, the data stored in the associated area data 21b is allocated according to the type of tool 40 (clamp 41, clamping member 42L, 42R). For example, the associated area 17b represented by data A2 allocated to clamp 41 represents the area where mark 45 is disposed inside only when clamp 41 is in the open state. Furthermore, the associated area 17b represented by data A2 may also be the area where mark 45 is disposed inside only when clamp 41 is in the closed state.

[0161] Furthermore, in another embodiment, the associated region data 21b can also be allocated according to the various states of the tool 40 (e.g., the open and closed states of the fixture 41). In this case, the associated region 17b represented by the data allocated according to the state of the tool 40 can all be the region where the mark 45 is disposed on the inside.

[0162] Figure 12 This is a diagram illustrating the determination method of the determination unit 59d(59) in one embodiment. Figure 12 In this context, the tool conditions that should be met, as well as the tool 40 used as the judgment object in each inspection A and B, and the captured image 15 used as the image processing object, are all related to... Figure 3A , Figure 3B same.

[0163] exist Figure 12 In the determination shown, the image processing unit 55b(55) generates the processed image 18b(18) through each check A and B.

[0164] In one embodiment of inspections A and B, the image processing unit 55b uses a reference image (not shown) corresponding to the tool conditions that should be met to perform masking processing on the captured image 15, obtaining a processed image 18b with the associated region 17b extracted. Figure 12 The image regions that are masked are omitted from the illustration. In addition, the determination processing unit 64 performs a process (e.g., optical character recognition processing) to determine the markers 45 in each of the associated regions 17b represented by the data A1 and A2 of the associated region data 21 in the processed image 18b (image of the captured image 15) acquired from the image processing unit 55b.

[0165] For example, in inspection A, it is confirmed that mark 45 (specifically the text "A") exists in the associated area 17b, which is related to tool type and tool status respectively. The determination unit 59d determines that the tool condition is met because mark 45 can be identified with respect to tool type and tool status respectively.

[0166] On the other hand, in inspection B, the determination processing unit 64 confirms that mark 45 exists in the associated area 17b related to tool type, but confirms that mark 45 does not exist in the associated area 17b related to tool state. In this case, the determination unit 59d determines that the tool conditions are insufficient because the tool conditions related to tool state are not sufficient. Alternatively, in inspection B, the determination unit 59 may determine that the tool conditions related to tool type are met, but determine that the tool conditions related to tool state are not met.

[0167] Furthermore, in another embodiment, the image processing unit 55b can generate a processed image 18b by performing a cutting process on the captured image 15, wherein the cutting process extracts each associated region 17b represented by data A1 and A2 of the associated region data 21. In this case, the associated regions 17b and the processed image 18b can be processed as follows: Figure 12 The image shown is the same. In this case, the determination unit 59d can also obtain the same determination result as described above by the determination processing unit 64 performing the processing of the determination mark 45.

[0168] The following describes some embodiments of the tool inspection device 50 for robotic arms, the tool inspection program (machining control program 95) for robotic arms, and the tool inspection method for robotic arms.

[0169] (1) A tool inspection device 50 for a robotic arm according to at least one embodiment of the present invention includes: an image processing unit 55 configured to perform image processing on an image 15 of a tool 40 mounted on a robotic arm 30, which is associated with tool conditions related to tool type or tool state that the tool 40 should meet, and generate a processed image 18 in which an associated region 17 associated with the tool conditions is extracted; and a determination unit 59 configured to determine, based on the processed image 18, whether the tool 40 mounted on the robotic arm 30 meets the tool conditions.

[0170] Based on the structure described in (1) above, the associated region 17 related to the tool conditions is extracted, and a processed image 18 is generated. The determination unit 59 determines whether the tool conditions are sufficient based on the generated processed image 18. Therefore, it is possible to determine with high accuracy whether the tool 40 meets the tool conditions.

[0171] Furthermore, the imaging device 8, which generates the captured image 15 as the basis for the processed image 18, also generates other images for image analysis and processing (S31). That is, the imaging device 8 has the functions of generating an image for determining whether the tool conditions are sufficient and generating an image for image analysis and processing. As a result, the structure of the tool inspection device 50 can be kept from becoming too complex, and the cost of the workpiece processing system 1 can be reduced.

[0172] Furthermore, according to the above structure, it is not necessary to install a dedicated sensor on the tool 40 for determining whether the tool conditions are sufficient. For example, in an embodiment where the workpiece 5 is fresh meat, there is a tendency for the space where the workpiece 5 is processed to become damp. In such an embodiment, according to the above structure, it is not necessary to take waterproof and anti-fouling measures for the tool 40 because of the electronic components, i.e., sensors, installed on the tool 40, and it is possible to easily determine whether the tool conditions are sufficient.

[0173] Furthermore, as a method for determining whether tool conditions are sufficient, a locking mechanism could be considered that allows only specific tools 40 to be mounted on the robotic arm 30. However, this method leads to increased mechanism complexity, and the types of tools 40 mounted on the robotic arm 30 are limited. Moreover, while the above method can determine tool conditions related to tool type, it is difficult to determine tool conditions related to tool state (e.g., it is difficult to properly determine the open / closed state of the clamp 41). Regarding this, according to an embodiment of the above structure, mechanism complexity and the reduction in the types of tools 40 can be avoided, and tool conditions related not only to tool type but also to tool state can be determined.

[0174] (2) In some embodiments, regarding the structure of (1) above, the tool inspection device 50a further includes brightness value acquisition units 56a and 56b (56), which are configured to acquire the brightness value of the processed image 18a (18), and the determination units 59a and 59b (59) are configured to determine whether the tool conditions are sufficient based on the acquired brightness value.

[0175] Based on the structure described in (2) above, the determination units 59a and 59b can quantitatively determine whether the tool conditions are sufficient based on the brightness value of the processed image 18a. Therefore, it is possible to determine with high precision whether the tool 40 meets the tool conditions.

[0176] (3) In some embodiments, regarding the structure of (2) above, the brightness value acquisition unit 56a is configured to acquire the sum of brightness values ​​X2 of the processed image 18a (18), and the determination unit 59a is configured to determine whether the tool conditions are sufficient based on the sum of the acquired brightness values ​​X2.

[0177] Based on the structure described in (3) above, the determination unit 59a determines whether the tool conditions are sufficient based on the sum of the brightness values ​​across the entire area of ​​the processed image 18a. Therefore, even when the shooting conditions of the tool 40 change, it is possible to determine whether the tool conditions are sufficient with high accuracy.

[0178] In addition, the shooting conditions of tool 40 include: the position of tool 40 during shooting, the degree of inclusion of objects other than tool 40 (such as workpiece 5), minor changes in the position of camera device 8, or a combination thereof.

[0179] (4) In some embodiments, regarding the structure of (2) above, the brightness value acquisition unit 56b is configured to acquire the sum of the differences X2 of the brightness values ​​determined using the following formula (1), where formula (1) uses: the brightness value of each pixel in the processed image 18a (18), i.e., B ij (i is any natural number less than the number of pixels in the horizontal direction of the processed image 18, j is any natural number less than the number of pixels in the vertical direction), and the brightness value Bs set for each pixel according to the tool conditions. ij The determination unit 59b is configured to determine whether the tool conditions are sufficient based on the sum of the differences X1 of the acquired brightness values.

[0180] [Formula 4]

[0181]

[0182] Based on the structure of (4) above, since the sum of the differences X1 determined by equation (1) varies depending on whether the tool conditions are sufficient, it is possible to determine with high precision whether the tool 40 meets the tool conditions.

[0183] (5) In some embodiments, regarding the structure of (1) above, the tool inspection device 50b further includes: a storage unit that stores a learned model 57 configured to output evaluation data related to whether the tool 40 meets the tool conditions when data related to the processed image 18 is input; and an evaluation data acquisition unit 58 configured to acquire the evaluation data output by inputting the processed image 18a (18) generated by the image processing unit 55a (55) to the learned model 57, and a determination unit 59c (59) configured to determine whether the tool conditions are sufficient based on the acquired evaluation data.

[0184] Based on the structure described in (5) above, the determination unit 59c determines whether the tool conditions are sufficient based on the evaluation data output from the learned model 57. Therefore, it is possible to determine with high accuracy whether the tool 40 meets the tool conditions.

[0185] (6) In some embodiments, regarding the structure of (1) above, the image processing unit 55 is configured to perform image processing on the captured image 15 in association with the tool conditions of the tool 40, the tool 40 having an outer surface 46 with a mark 45, the tool inspection device 50c (50) further includes a determination processing unit 64, which is configured to perform a process to determine the mark 45 on the generated processed image 18b (18), and the determination unit 59d is configured to determine whether the tool conditions are sufficient based on the processing result of the determination processing unit 64.

[0186] Based on the structure described above (6), as long as the determination result of the determination unit 64 is changed in a manner corresponding to whether the tool conditions are sufficient, the processed image 18b (18) associated with the tool conditions can be generated, and the tool 40 can be determined with high precision whether the tool conditions are met.

[0187] (7) In some embodiments, regarding the structure of (6) above, the determination processing unit 64 is configured to perform processing on the processed image 18b to determine the text as the mark 45.

[0188] Based on the structure described in (7) above, the determination unit 59d determines whether the tool conditions are sufficient based on the processing result of determining the text as mark 45 using the determination processing unit 64. Therefore, it is possible to determine with high accuracy whether the tool 40 meets the tool conditions.

[0189] (8) In some embodiments, regarding any of the structures in (1) to (7) above, regarding the tool inspection device 50, the condition acquisition unit 51 is configured to acquire the tool conditions based on the operation schedule of the robotic arm 30 after the determination unit 59a, 59b (59) determines that the tool conditions are sufficient, the image processing unit 55a (55) is configured to perform image processing on the captured image 15, associating the tool conditions acquired by the condition acquisition unit 51 with one of the multiple pre-prepared tool conditions, and the determination unit 59a, 59b (59) is configured to determine whether the tool conditions acquired by the condition acquisition unit 51 are sufficient.

[0190] Based on the structure described in (8) above, the determination units 59a and 59b determine whether the tool conditions corresponding to the predetermined operation of the robotic arm 30 are sufficient. Therefore, the determination units 59a and 59b can determine with high accuracy whether the robotic arm 30 should perform the predetermined operation.

[0191] (9) In some embodiments, with respect to any of the structures in (1) to (8) above, the image processing unit 55a (55) is configured to perform masking processing on the captured image 15 using a reference image 14 associated with the tool conditions, and generate an image that extracts the associated region 17 associated with the tool conditions as the processed image 18.

[0192] Based on the structure described above (9), the determination units 59a and 59b can determine with high precision whether the tool 40 meets the tool conditions based on the processed image 18a (18).

[0193] (10) The storage device (ROM92) of at least one embodiment of the present invention stores a tool inspection program (processing control program 95) for inspecting the tool 40 for the robotic arm. The program causes a computer to execute: an image processing step (S19) for performing image processing on an image 15 of the tool 40 mounted on the robotic arm 30, which is associated with tool conditions related to the tool type or tool state that the tool 40 should meet, to generate a processed image 18 with the associated region 17 extracted from the tool conditions; and a determination step (S23) for determining, based on the processed image 18, whether the tool 40 mounted on the robotic arm 30 meets the tool conditions.

[0194] Based on the structure of (10) above, for the same reason as (1) above, it is possible to determine with high precision whether tool 40 meets the tool conditions.

[0195] (11) A method for inspecting a tool 40 for a robotic arm according to at least one embodiment of the present invention comprises: an image processing step (S19) for performing image processing on an image 15 of the tool 40 mounted on the robotic arm 30, which is associated with tool conditions related to tool type or tool state that the tool 40 should meet, to generate a processed image 18 in which an associated region 17 associated with the tool conditions is extracted; and a determination step (S23) for determining, based on the processed image 18, whether the tool 40 mounted on the robotic arm 30 meets the tool conditions.

[0196] Based on the structure described in (11) above, for the same reason as in (1) above, it is possible to determine with high precision whether tool 40 meets the tool conditions.

[0197] The embodiments of this disclosure have been described above, but this disclosure is not limited to the above embodiments, but also includes: modifications to the above embodiments, and appropriate combinations of these embodiments.

[0198] For example, an implementation method consisting of at least two of the first, second, and third implementation methods described above can be used.

[0199] As a specific example, the sufficiency of the tool conditions can be determined based on the acquisition result of the brightness value acquisition unit 56 and the output result of the learned model 57. In this case, the determination unit 59 determines that the tool conditions are met only when both the acquisition result of the brightness value acquisition unit 56 and the output result of the learned model 57 indicate that the tool conditions are met. Thus, it is possible to suppress the situation where the actual tool 40 does not meet the tool conditions, but the determination unit 59 mistakenly determines that the tool conditions are met.

[0200] As another example, if either the result obtained by the brightness value acquisition unit 56 or the output result of the learned model 57 indicates that the tool condition is met, it can be determined that the tool condition is met. Thus, it is possible to suppress the situation where, although the actual tool 40 meets the tool condition, the determination unit 59 mistakenly determines that the tool condition is not met.

[0201] Explanation of reference numerals in the attached figures

[0202] 14-Reference image; 17-Associated region; 18-Processed image; 30-Robotic arm; 40-Tool; 41, 41a-Fixture; 42, 42L, 42R-Clamping parts; 43, 43L, 43R-Cutting tools; 45-Marking; 46-Outer surface; 50-Tool inspection device; 51-Condition acquisition unit; 54-Storage unit; 55-Image processing unit; 56-Brightness value acquisition unit; 57-Learned model; 57a~57h-Learned model; 58-Evaluation data acquisition unit; 59-Decision unit; 64-Determination processing unit; 95-Tool inspection procedure.

Claims

1. A tool inspection device for a robotic arm, which is a tool inspection device for a robotic arm in a workpiece processing system, the workpiece processing system comprising: a robotic arm equipped with a tool for processing a workpiece of food conveyed by a conveying device; and a camera device for acquiring a photographic image of the workpiece when the workpiece being conveyed by the conveying device is within a photographing area, the workpiece processing system being configured to: process the workpiece being conveyed by the conveying device using the tool based on the result of image analysis of the photographic image of the workpiece, the tool inspection device for the robotic arm comprising: A camera control unit is used to control the camera device to acquire images of the tool mounted on the robotic arm within the shooting area of ​​the camera device; An image processing unit is configured to perform image processing on the captured image of the tool mounted on the robotic arm, associating it with tool conditions related to the tool type or tool state that the tool should satisfy, and to generate a processed image from which the associated region associated with the tool conditions has been extracted; and The determination unit is configured to determine, based on the processed image, whether the tool mounted on the robotic arm meets the tool conditions.

2. The tool inspection device for a robotic arm according to claim 1, characterized in that, It also includes a brightness value acquisition unit, which is configured to acquire the brightness value of the processed image. The determination unit is configured to determine whether the tool conditions are sufficient based on the acquired brightness value.

3. The tool inspection device for a robotic arm according to claim 2, characterized in that, The brightness value acquisition unit is configured to acquire the sum of the brightness values ​​of the processed image. The determination unit is configured to determine whether the tool conditions are sufficient based on the sum of the acquired brightness values.

4. The tool inspection device for a robotic arm according to claim 2, characterized in that, The brightness value acquisition unit is configured to acquire the sum of differences X1 between brightness values ​​determined using the following formula (1), where formula (1) uses: the brightness value of each pixel in the processed image, i.e., B. ij And the brightness value Bs set for each pixel according to the tool conditions. ij , The determination unit is configured to determine whether the tool conditions are sufficient based on the sum of the differences X1 of the acquired brightness values. [Formula 1] i is any natural number less than the number of pixels in the horizontal direction of the processed image, and j is any natural number less than the number of pixels in the vertical direction.

5. The tool inspection device for a robotic arm according to claim 1, characterized in that, It also has: The storage unit stores the learned model, which is configured to output evaluation data related to whether the tool meets the tool conditions when inputting data related to the processed image. as well as The evaluation data acquisition unit is configured to acquire the evaluation data output by inputting the processed image generated by the image processing unit into the learned model. The determination unit is configured to determine whether the tool conditions are sufficient based on the acquired evaluation data.

6. The tool inspection device for a robotic arm according to claim 1, characterized in that, The image processing unit is configured to perform image processing on the captured image in association with the tool conditions of the tool, the tool having a marked outer surface. The tool inspection device for the robotic arm also includes a determination processing unit configured to perform a process to determine the mark on the generated processed image. The determination unit is configured to determine whether the tool conditions are sufficient based on the processing result of the determination processing unit.

7. The tool inspection device for a robotic arm according to claim 6, characterized in that, The determination processing unit is configured to perform a process on the processed image to determine the text that will be used as the marker.

8. The tool inspection device for a robotic arm according to any one of claims 1 to 7, characterized in that, The condition acquisition unit is configured to acquire the tool conditions based on the operation schedule of the robotic arm after the determination unit determines that the tool conditions are sufficient. The image processing unit is configured to perform image processing on the captured image, associating it with the tool condition acquired by the condition acquisition unit from among a plurality of pre-prepared tool conditions. The determination unit is configured to determine whether the tool conditions obtained by the condition acquisition unit are sufficient.

9. The tool inspection device for a robotic arm according to any one of claims 1 to 7, characterized in that, The image processing unit is configured to perform masking processing on the captured image using a reference image associated with the tool conditions, and generate an image from which the associated region associated with the tool conditions has been extracted as the processed image.

10. A storage device storing a tool inspection program for a robotic arm, the program being used to inspect tools for a robotic arm in a workpiece processing system, the workpiece processing system comprising: the robotic arm equipped with the tool for processing workpieces of food conveyed by a conveying device; and a camera device for acquiring photographic images of the workpiece when the workpiece being conveyed by the conveying device is within a photographing area, the workpiece processing system being configured to process the workpiece being conveyed by the conveying device using the tool based on image analysis results of the photographic images of the workpiece. The program causes the computer to execute: A camera control step is used to control the camera device to acquire images of the tool mounted on the robotic arm within the shooting area of ​​the camera device; The image processing step is used to perform image processing on the captured image of the tool mounted on the robotic arm, associating it with tool conditions that the tool should meet related to tool type or tool state, to generate a processed image from which the associated region associated with the tool conditions has been extracted; and The determination step is used to determine, based on the processed image, whether the tool mounted on the robotic arm meets the tool conditions.

11. A method for inspecting tools for a robotic arm, the method being used to inspect tools for a robotic arm in a workpiece processing system, the workpiece processing system comprising: the robotic arm equipped with the tool for processing workpieces of food conveyed by a conveying device; and a camera device for acquiring photographic images of the workpiece when the workpiece being conveyed by the conveying device is within a photographing area, the workpiece processing system being configured to: process the workpiece being conveyed by the conveying device using the tool based on the result of image analysis of the photographic images of the workpiece. The method comprises: A camera control process is used to control the camera device to acquire images of the tool mounted on the robotic arm within the shooting area of ​​the camera device; An image processing step is used to perform image processing on the captured image of the tool mounted on the robotic arm, associating it with tool conditions that the tool should meet, related to the tool type or tool state, to generate a processed image from which the associated region associated with the tool conditions has been extracted; and The determination process is used to determine, based on the processed image, whether the tool mounted on the robotic arm meets the tool conditions.

Images