Control device for meat processing apparatus, and meat processing apparatus

Abstract

This control device for a meat processing apparatus according to one embodiment comprises: a position information acquisition unit that acquires position information on meat in time-series imaging data including meat during meat processing; a positional deviation detection unit that detects the presence or absence of a positional deviation of the meat during the meat processing on the basis of a difference between position information on meat at a reference time point and position information of meat at the latest time point; and a correction command output unit that outputs, to a positional deviation correction device for correcting the positional deviation of meat, a control command for correcting the position of meat when the positional deviation detection unit detects the positional deviation.

Description

Control device for meat processing device, and meat processing device

[0001] The present disclosure relates to a control device for a meat processing device and a meat processing device.

[0002] Meat processing machines that automatically transport and process meat are known. For example, a meat processing machine described in Patent Document 1 includes multiple conveyors for transporting meat. In the meat processing machine described in Patent Document 1, the meat transported by the conveyors is processed by a multi-axis robot equipped with meat processing tools.

[0003] Japanese Patent Application Laid-Open No. 2022-170894

[0004] In a device for processing meat on a conveyor, such as the meat processing device described in Patent Document 1, the meat on the conveyor is photographed with a camera, for example, and image data is obtained, and the desired processing, such as cutting, is performed based on the image data. If the position of the meat shifts during processing, there is a risk that the desired processing will not be possible.

[0005] In view of the above circumstances, at least one embodiment of the present disclosure aims to provide a control device for a meat processing device and a meat processing device that can correct the position of meat even if the position of the meat shifts during processing.

[0006] (1) A control device for a meat processing device according to at least one embodiment of the present disclosure includes: a position information acquisition unit that acquires position information of the meat in time-series image data including the meat during meat processing; a position deviation detection unit that detects whether or not the meat is misaligned during meat processing based on the difference between the position information of the meat at a reference time point and the position information of the meat at an immediately preceding time point; and a correction command output unit that, when the position deviation detection unit detects the position deviation, outputs a control command to a position deviation correction device that corrects the position deviation of the meat to correct the position of the meat.

[0007] (2) A meat processing apparatus according to at least one embodiment of the present disclosure comprises a control device having the configuration described above in (1), a conveyor device for transporting the meat, an imaging device for acquiring the imaging data, and the positional deviation correction device.

[0008] According to at least one embodiment of the present disclosure, even if the position of the meat shifts during meat processing, the position of the meat can be corrected.

[0009] 1 is a diagram showing a meat processing device according to an embodiment; FIG. 2 is a diagram showing a configuration related to detecting the presence or absence of workpiece positional deviation and correcting the workpiece positional deviation, among the functional configurations of a control device according to an embodiment; FIG. 3 is a diagram showing the functional configuration of a position estimation device according to an embodiment; FIG. 4 is a diagram showing an example of a captured image, which is image data obtained by capturing an image with a color camera; FIG. 5 is a diagram for explaining workpiece position information estimated using a trained model; FIG. 6 is a diagram showing an example of a captured image during workpiece processing; FIG. 7 is a diagram showing an example of a case where the position of the workpiece at the most recent time point has deviated downstream in the direction of conveyance of the workpiece by the conveying device, relative to the position of the workpiece at the reference time point; FIG. 8 is a diagram showing an example of a case where the position of the workpiece at the most recent time point has deviated upstream in the direction of conveyance of the workpiece by the conveying device, relative to the position of the workpiece at the reference time point; FIG. 9 is a flowchart showing the procedure of a process related to detecting workpiece positional deviation and correcting the positional deviation, which is performed by a control device according to an embodiment; FIG. 10 is a flowchart showing the procedure of a process for acquiring workpiece position information, which is performed by a position estimation device according to an embodiment;

[0010] Several embodiments of the present disclosure will be described below with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of components described as embodiments or shown in the drawings are not intended to limit the scope of the present disclosure and are merely illustrative examples. For example, expressions expressing relative or absolute arrangements, such as "in a certain direction," "along a certain direction," "parallel," "orthogonal," "center," "concentric," or "coaxial," not only strictly express such arrangements, but also express relative displacements with a tolerance or angle or distance to the extent that the same function is achieved. For example, expressions expressing the equality of things, such as "same," "equal," and "homogeneous," not only express strict equality, but also express tolerance or differences to the extent that the same function is achieved. For example, expressions expressing shapes such as a square or cylindrical shape not only express shapes such as a square or cylindrical shape in the strict geometric sense, but also express shapes including concave and convex portions, chamfered portions, etc., to the extent that the same effect is achieved. On the other hand, the expressions "comprise," "include," "have," "includes," or "have" of one element are not exclusive expressions that exclude the presence of other elements.

[0011] 1 is a diagram showing a meat processing apparatus 1 according to one embodiment. The meat processing apparatus 1 according to one embodiment is an apparatus for processing meat (workpiece 5) using a tool 40.

[0012] A meat processing apparatus 1 according to one embodiment includes a conveying device 7 for conveying a workpiece 5, a robot arm 30 for processing the workpiece 5, a color camera 8 and a 3D camera 9 for photographing the workpiece 5, a control device 50 for controlling the robot arm 30 and the conveying device 7, and a position estimation device 60 for estimating the position of the workpiece 5. Note that the control device 50 and the position estimation device 60 may be different devices or may be the same device.

[0013] In one embodiment, the transport device 7 is a belt conveyor that transports the workpiece 5. In one embodiment, the color camera 8 is an imaging device used to estimate the position of the workpiece 5 during machining, as will be described later, and is installed to capture an image of the workpiece 5 from above. In this embodiment, the captured image 15 (see FIG. 3 , which will be described later) obtained by imaging with the color camera 8 is a planar image. In one embodiment, the 3D camera 9 is an imaging device for acquiring the three-dimensional shape of the workpiece 5, which is used to identify the machining position of the workpiece 5, and is installed to capture an image of the workpiece 5 from above.

[0014] The configuration of the robot arm 30 will be described. In one embodiment, the robot arm 30 is an industrial robot. More specifically, as an example, the robot arm 30 is an articulated robot. The robot arm 30 may be a vertical articulated robot, a horizontal articulated robot, or a combination thereof. In one embodiment, the robot arm 30 is configured to be equipped with a tool 40 for machining a workpiece 5.

[0015] In one embodiment, the robot arm 30 includes robot arms 30a, 30b, and 30c. For example, in one embodiment, the tool 40 includes a clamper 41 for gripping and fixing the workpiece 5, a chuck 42 for chucking the workpiece 5, and a knife 43 for cutting the workpiece 5. For example, in one embodiment, the clamper 41 is attached to the robot arm 30a, and the chuck 42 and the knife 43 are attached to the robot arm 30b or the robot arm 30c. In one embodiment, the clamper 41 and the chuck 42 are opened and closed by a drive mechanism. In one embodiment, an air cylinder (not shown) is used as the drive mechanism. In this case, an inlet and an outlet provided in each of the clamper 41 and the chuck 42 are connected to the air cylinder via an air pipe. Note that a hydraulic cylinder or a motor may be used as the drive mechanism.

[0016] The meat processing apparatus 1 is not limited to being provided with a plurality of robot arms 30. A single robot arm 30 may be selectively equipped with a plurality of types of tools 40, or may be equipped with only a single tool 40.

[0017] The control device 50 of one embodiment includes a processor 51 that executes various arithmetic processes and a memory 52 that non-temporarily or temporarily stores various data processed by the processor 51. The processor 51 is realized by a CPU, GPU, MPU, DSP, NPU (Neural Processing Unit), various other arithmetic devices, or a combination of these. The memory 52 is realized by a ROM, RAM, flash memory, or a combination of these. In the following explanation, the control contents of the control device 50 will be mainly explained with regard to detecting the presence or absence of positional deviation of the workpiece 5 during machining and correcting the positional deviation of the workpiece 5.

[0018] 2A is a diagram showing the functional configuration of the control device 50 of one embodiment, which is related to detecting the presence or absence of positional deviation of the workpiece 5 and correcting the positional deviation of the workpiece 5. The control device 50 of one embodiment includes, as its functional configuration, a position information acquisition unit 55, a positional deviation detection unit 56, and a correction command output unit 57.

[0019] The position information acquisition unit 55 is configured to acquire position information of the workpiece 5 in the time-series image data including the workpiece 5 being machined, that is, the workpiece 5 being machined.

[0020] The positional deviation detection unit 56 is configured to detect whether or not the workpiece 5 is misaligned during processing based on the difference between the positional information of the workpiece 5 at a reference point in time (described later) and the positional information of the workpiece 5 at the most recent point in time.

[0021] The correction command output unit 57 is configured to output a control command to a position misalignment correction device for correcting the position of the workpiece 5 when the position misalignment detection unit 56 detects a position misalignment of the workpiece 5. In one embodiment of the meat processing apparatus 1, the position misalignment correction device is the robot arm 30, but it may also be a device dedicated to correcting the position of the workpiece 5. The control content of the control device 50 will be described in detail later.

[0022] A position estimation device 60 according to one embodiment includes a processor 61 that executes various types of arithmetic processing and a memory 62 that non-temporarily or temporarily stores various types of data processed by the processor 61. The processor 61 is realized by a CPU, an MPU, a DSP, or any other type of arithmetic device, or a combination thereof, together with a GPU or an NPU. The memory 62 is realized by a ROM, a RAM, a flash memory, or a combination thereof.

[0023] 2B is a diagram illustrating a functional configuration of the position estimation device 60 according to an embodiment. The position estimation device 60 according to an embodiment includes, as functional components, an image acquisition unit 65, a storage unit 66, a coordinate conversion unit 68, and a position information output unit 69.

[0024] The image acquisition unit 65 is a component for acquiring imaging data (images) captured by the color camera 8 .

[0025] The storage unit 66 is configured to store various data necessary for the calculation processing of the position estimation device 60, and stores, for example, a trained model 67 used to estimate position information described below. The trained model 67 is a prediction model for inputting the imaging data acquired by the image acquisition unit 65 and estimating the position information of the workpiece 5 based on the imaging data, and is capable of estimating the position of the workpiece 5 in the imaging data.

[0026] The coordinate conversion unit 68 is configured to convert the coordinate values ​​in the image data, which are the position information of the workpiece 5 estimated by the trained model 67, into coordinate values ​​in a coordinate system related to the control of the robot arm 30.

[0027] The position information output unit 69 is configured to output the position information of the workpiece 5 after being converted by the coordinate conversion unit 68 to the position information acquisition unit 55 of the control device 50. The control content of the position estimation device 60 will be described in detail later.

[0028] In the meat processing apparatus 1 according to one embodiment configured as described above, the control device 50 drives the conveying device 7 to convey the workpiece 5 to a predetermined position, and then stops the conveying device 7. The control device 50 then causes the 3D camera 9 to capture an image of the workpiece 5 on the conveying device 7, and based on the obtained image data, controls the robot arm 30 to process the workpiece 5. More specifically, the control device 50 identifies a processing position for the workpiece 5 based on the image captured by the 3D camera 9, and controls the robot arm 30 based on the identification result.

[0029] The workpiece 5 on the transport device 7 is processed, for example, by the robot arm 30c having the knife 43 attached thereto, cutting the workpiece 5 with the knife 43, or by the robot arm 30b having the chuck 42 attached thereto, gripping and removing bone portions with the chuck 42. When processing the workpiece 5, the workpiece 5 may be gripped by the clamper 41 attached to the robot arm 30a, and fixed on the transport device 7.

[0030] When the processing of the workpiece 5 is completed, the control device 50 drives the transport device 7 to transport the workpiece 5 to the next process.

[0031] In the meat processing apparatus 1 according to one embodiment, if the workpiece 5 moves undesirably and becomes misaligned while being processed, the trajectory of the tool 40 moved by the robot arm 30 will be misaligned relative to the workpiece 5, which may result in processing not being performed as desired. Therefore, the meat processing apparatus 1 according to one embodiment is configured to detect whether or not the workpiece 5 has become misaligned while being processed, and if a misalignment of the workpiece 5 is detected, correct the misalignment of the workpiece 5. Detection of a misalignment of the workpiece 5 will be described in detail below.

[0032] In one embodiment, the control device 50 repeatedly transmits an imaging command to the position estimation device 60 to repeatedly capture images of the workpiece 5 while the workpiece 5 is being processed. The control device 50 transmits an imaging command to the position estimation device 60, for example, every 300 milliseconds while the workpiece 5 is being processed. The position estimation device 60 transmits an imaging command to the color camera 8 every time an imaging command is input. Note that the control device 50 may be configured to transmit an imaging command directly to the color camera 8.

[0033] In one embodiment, the color camera 8 captures an image of the workpiece 5 each time an imaging command is input. This allows for obtaining imaging data that includes the workpiece 5, i.e., time series imaging data of the workpiece 5. The color camera 8 transmits the obtained imaging data to the position estimation device 60 each time an image of the workpiece 5 is captured.

[0034] 3 is a diagram showing an example of a captured image 15, which is image data obtained by capturing an image with the color camera 8. The captured image 15 includes all or part of the workpiece 5 on the conveying device 7. The angle of view of the color camera 8 is preferably set so that both end portions 7a of the belt conveyor, which is the conveying device 7, in the conveyor width direction, which is a direction perpendicular to the conveying direction, are included in the captured image 15.

[0035] The position estimation device 60 of one embodiment acquires position information of the workpiece 5 based on the imaging data transmitted from the color camera 8. Specifically, the processor 61 of the position estimation device 60 of one embodiment inputs the imaging data acquired by the image acquisition unit 65 to the trained model 67 stored in the storage unit 66 every time imaging data from the color camera 8 is input to the image acquisition unit 65, and obtains a prediction result of the position of the workpiece 5 in the imaging data. Note that the processor 61 may be configured to input text data "meat" representing the workpiece 5 (meat) whose position is to be predicted, together with the imaging data acquired by the image acquisition unit 65, to the trained model 67 stored in the storage unit 66.

[0036] In one embodiment, the processor 61 uses the trained model 67 to estimate the position of the workpiece 5 in the input imaging data, identifies the area in the input imaging data where the presence of the workpiece 5 is estimated as a rectangular area surrounded by a rectangular frame, and outputs information on the position and size of the identified rectangular area as position information of the workpiece 5 to the coordinate conversion unit 68 together with the imaging data. FIG. 4 is a diagram for explaining the position information of the workpiece 5 estimated using the trained model 67. In FIG. 4, the rectangular frame 16 surrounding the workpiece 5 indicates a rectangular area 16a representing the area in the captured image 15 where the workpiece 5 is estimated to be present. Note that FIG. 4 also shows the outline points P1 and P2 of the workpiece 5, the ankle point P3, the bone point P4, and the line segment L1 connecting the ankle point P3 and the bone point P4, which are referenced in correcting the positional deviation of the workpiece 5, as described below. These points P1-P4 and the line segment L1 will be described later.

[0037] FIG. 5 is a diagram showing an example of a captured image 15 during machining of a workpiece 5. As shown in FIG. 5, during machining of the workpiece 5, the robot arm 30 and the tool 40 overlap with the workpiece 5 in a planar view. In such a case, when attempting to estimate the position of the workpiece 5 using conventional image processing, the robot arm 30 and the tool 40 in the captured image 15 become an obstacle, making it difficult to correctly estimate the position of the workpiece 5. In one embodiment, the trained model 67 can correctly estimate the position of the workpiece 5 even if the robot arm 30 and the tool 40 cover part of the workpiece 5 in the captured image 15, as shown in FIG. 5.

[0038] In one embodiment, the coordinate conversion unit 68 of the position estimation device 60 converts the position information of the workpiece 5 estimated using the trained model 67, i.e., the coordinate values ​​representing the position and size information of the rectangular area 16a, into coordinate values ​​in a coordinate system related to the control of the robot arm 30.

[0039] The position information output unit 69 of the position estimation device 60 in one embodiment outputs the position information of the workpiece 5 after being converted by the coordinate conversion unit 68 to the position information acquisition unit 55 of the control device 50. The position information output unit 69 in one embodiment outputs the position information of the workpiece 5 after being converted by the coordinate conversion unit 68 to the position information acquisition unit 55 of the control device 50, for example, every 300 ms.

[0040] In one embodiment, the position information acquisition unit 55 of the control device 50 acquires the position information of the workpiece 5 output from the position information output unit 69 of the position estimation device 60 .

[0041] After starting machining of the workpiece 5, the positional deviation detection unit 56 of the control device 50 in one embodiment temporarily stores the positional information of the workpiece 5 first acquired by the positional information acquisition unit 55 in the memory 52 as positional information of the workpiece 5 at a reference time. The positional deviation detection unit 56 then compares the positional information of the workpiece 5 sequentially acquired by the positional information acquisition unit 55, i.e., the positional information of the workpiece 5 at the most recent time, with the positional information of the workpiece 5 at the reference time, to detect whether the position of the workpiece 5 at the most recent time is deviated from the position of the workpiece 5 at the reference time, i.e., whether or not there is a positional deviation of the workpiece 5. Details of the detection of a positional deviation of the workpiece 5 by the positional deviation detection unit 56 will be described later.

[0042] In one embodiment, the correction command output unit 57 of the control device 50 outputs a control command to the robot arm 30 to correct the position of the workpiece 5 when the positional deviation detection unit 56 determines that a positional deviation of the workpiece 5 has occurred. For example, the correction command output unit 57 generates a control command to correct the position of the workpiece 5 based on the difference between the positional information of the workpiece 5 at a reference time point temporarily stored in the memory 52 and the positional information of the workpiece 5 at the most recent time point, and outputs the control command to the robot arm 30. In one embodiment, the processor 51 of the control device 50 temporarily stops machining of the workpiece 5 when the positional deviation detection unit 56 determines that a positional deviation of the workpiece 5 has occurred. Thereafter, in order to determine the exact position of the misaligned workpiece 5, the processor 51 controls the robot arm 30 to completely retract from the imaging angle of the 3D camera 9, and then causes the 3D camera 9 to capture an image of the workpiece 5 on the conveyance device 7. The processor 51 identifies the position of the misaligned workpiece 5 based on the image captured by the 3D camera 9. The processor 51 also detects, for example, the outline points P1 and P2 of the workpiece 5 shown in FIG. 4 based on the image captured by the 3D camera 9. The outline point P1 is, for example, the portion of the outer edge of the workpiece 5 closest to one of a pair of sides parallel to the transport direction of the rectangular region 16a (corresponding to the pair of sides 16c in FIG. 6C described later). The outline point P2 is, for example, the portion of the outer edge of the workpiece 5 closest to the other of the pair of sides parallel to the transport direction of the rectangular region 16a. The ankle point P3 is a portion detected as the ankle when the workpiece 5 is a thigh. The bone point P4 is, for example, a portion detected as a characteristic point among the portions detected as the bone 6 exposed on the top surface of the workpiece 5. The line segment L1 is, for example, a line segment connecting the ankle point P3 and the bone point P4.

[0043] The correction command output unit 57 generates a control command to correct the position of the workpiece 5 based on the identified position, each of the points P1-P4, and the line segment L1, and outputs the control command to the robot arm 30. The control command generated by the correction command output unit 57 will be described later. In response to this, the robot arm 30 corrects the position of the workpiece 5, for example, by pressing the workpiece 5 with the tool 40. Thereafter, the processor 51 again causes the 3D camera 9 to capture an image of the workpiece 5 on the transport device 7, and controls the robot arm 30 to resume machining of the workpiece 5 based on the obtained image data.

[0044] If the positional deviation detection unit 56 determines that the positional deviation of the workpiece 5 has occurred after the machining of the workpiece 5 is resumed, the above-described correction of the position of the workpiece 5 is performed again. However, if the processor 51 determines that the number of occurrences of the above-described positional deviation for one workpiece 5 exceeds a specified number, the machining of this workpiece 5 is terminated.

[0045] (Regarding detection of positional deviation of the workpiece 5 by the positional deviation detection unit 56) The following describes detection of positional deviation of the workpiece 5 by the positional deviation detection unit 56. It is known that if the robot arm 30 or the tool 40 covers part of the workpiece 5 in the captured image 15, the size and position of the rectangular frame 16 surrounding the workpiece 5 shown in FIG. 4 , which is the positional information of the workpiece 5 estimated by the trained model 67, will fluctuate slightly. Therefore, in one embodiment of the control device 50, the positional deviation detection unit 56 determines that a positional deviation of the workpiece 5 has occurred when the difference between the positional information of the workpiece 5 sequentially acquired by the positional information acquisition unit 55, i.e., the positional information of the workpiece 5 at the most recent point in time, and the positional information of the workpiece 5 at the reference point in time, is greater than or equal to a certain magnitude.

[0046] FIG. 6A is a diagram showing an example of a case where the position of the workpiece 5 at the most recent time point has shifted downstream in the direction of transport of the workpiece 5 by the transport device 7, compared to the position of the workpiece 5 at the reference time point. FIG. 6B is a diagram showing an example of a case where the position of the workpiece 5 at the most recent time point has shifted upstream in the direction of transport of the workpiece 5 by the transport device 7, compared to the position of the workpiece 5 at the reference time point. FIG. 6C is a diagram showing an example of a case where the position of the workpiece 5 at the most recent time point has shifted in the conveyor width direction, compared to the position of the workpiece 5 at the reference time point. Note that FIG. 6C shows an example of a case where the position of the workpiece 5 at the most recent time point has shifted upward in the conveyor width direction, compared to the position of the workpiece 5 at the reference time point. In FIGS. 6A, 6B, and 6C, the dashed-line frame 17 indicates a rectangular area 17a of the position information of the workpiece 5 at the reference time point.

[0047] In one embodiment, the positional deviation detection unit 56 detects a positional deviation when it determines that, compared to the positions of a pair of opposing sides 17b in the transport direction of the workpiece 5 in a rectangular area 17a in the positional information of the workpiece 5 at a reference time, a pair of opposing sides 16b in the transport direction of the workpiece 5 in a rectangular area 16a in the positional information of the workpiece 5 at the most recent time deviates downstream in the transport direction by more than a threshold value Th1 included in the judgment criteria described below for a specified number of times N1 or more consecutively, as shown in Figure 6A, for example.

[0048] In one embodiment, the positional deviation detection unit 56 detects a positional deviation when it determines that, compared to the positions of a pair of opposing sides 17b in the transport direction of the workpiece 5 in a rectangular area 17a in the positional information of the workpiece 5 at a reference time, a pair of opposing sides 16b in the transport direction of the workpiece 5 in a rectangular area 16a in the positional information of the workpiece 5 at the most recent time deviates upstream in the transport direction by more than a threshold value Th2 included in the judgment criteria described below for a specified number of times N2 or more consecutively, as shown in Figure 6B, for example.

[0049] In one embodiment, the positional deviation detection unit 56 detects a positional deviation when it determines that the positions of a pair of opposing sides 16c in the conveyor width direction in a rectangular area 16a in the positional information of the workpiece 5 at the most recent time point deviate in the same direction by more than a threshold value Th3 included in the judgment criteria described below for a specified number of times N3 or more, as shown in Figure 6C, for example.

[0050] The above-mentioned judgment criteria include the above-mentioned threshold values ​​Th (Th1, Th2, Th3) and the specified number of times N (N1, N2, N3), which are stored in advance in the memory 52 of the control device 50. The above-mentioned threshold values ​​Th and the specified number of times N are preset values, respectively, depending on the type of workpiece 5 (i.e., the part of meat, such as thigh, rib, shoulder, or arm), the processing type, such as scoring or cutting, and the progress stage of processing. Hereinafter, these threshold values ​​Th and the specified number of times N are also referred to as positional deviation detection parameters. For example, if the workpiece 5 is a thigh, examples of the above-mentioned processing type include deboning only the hipbone, deboning the hipbone and femur, and deboning the entire workpiece, including the tibia. For example, if the workpiece 5 is loose, examples of the above-mentioned processing type include separating the ribs into sheets or deboning each rib individually. As the above processing mode, for example, if the work 5 is a shoulder, the processing may be such that the bone and the shoulder loin are separated as a unit, or such that only the bone is removed.

[0051] In this way, the positional deviation detection unit 56 of one embodiment detects a positional deviation when it determines that the difference between the positional information of the workpiece 5 at a reference point in time and the positional information of the workpiece 5 at the most recent point in time is greater than or equal to the threshold value Th included in the judgment criteria stored in the memory 52 of the control device 50 for a specified number of times N or more in succession.

[0052] (Flowchart) Figure 7 is a flowchart showing the procedure for detecting and correcting positional deviation of the workpiece 5 in a meat processing process, which is performed by the control device 50 in one embodiment. The program for executing the process shown in the flowchart of Figure 7 is read from the memory 52 and executed by the processor 51. When processing of the workpiece 5 is started, in step S1, the processor 51 acquires positional deviation detection parameters from the memory 52 and proceeds to step S3 of the main loop. Note that the processor 51 reads from the memory 52 and executes a program for processing the workpiece 5, which is separate from the program for executing the processes described below. That is, the processor 51 causes the 3D camera 9 to capture an image of the workpiece 5 transported to a predetermined position by the transport device 7, and controls the robot arm 30 to process the workpiece 5 based on the obtained image data.

[0053] In step S3, the processor 51 determines whether or not the machining of the workpiece 5 is completed, i.e., whether or not all the processes for machining the workpiece 5 have been performed. If it is determined that the machining of the workpiece 5 is completed, the processor 51 ends the process shown in the flowchart of FIG.

[0054] If it is determined that the processing of the workpiece 5 has not been completed, then in step S5, the processor 51 transmits an image capturing command to the position estimation device 60. Thereafter, in step S7, the position information acquisition unit 55 of the control device 50 acquires the position information of the workpiece 5 from the position information output unit 69 of the position estimation device 60.

[0055] Next, in step S9, the positional deviation detection unit 56 of the control device 50 performs a deviation determination process shown in FIG. 9, which will be described later.

[0056] Next, in step S11, the positional deviation detection unit 56 determines whether or not it has been determined that the workpiece 5 is misaligned in the deviation determination process shown in Fig. 9. If it is determined that the workpiece 5 is not misaligned, that is, if a negative determination is made in step S11, the processor 51 returns to the beginning of the main loop, that is, step S3.

[0057] If it is determined that a positional deviation of the workpiece 5 has occurred, that is, if step S11 is judged as positive, the process proceeds to step S13, where the processor 51 judges whether or not the number of times that it has been determined that a positional deviation has occurred for one workpiece 5 has exceeded a specified number. If step S13 is judged as positive, that is, if it is judged that the number of times that it has been determined that a positional deviation has occurred for one workpiece 5 has exceeded the specified number, the processor 51 ends the processing shown in the flowchart of FIG.

[0058] If step S13 is negatively determined, i.e., if it is determined that the number of times that misalignment has occurred for one workpiece 5 does not exceed a predetermined number, the process proceeds to step S15, where the correction command output unit 57 of the control device 50 causes the robot arm 30 to perform a misalignment correction operation to correct the position of the workpiece 5. Specifically, if step S13 is negatively determined, the process proceeds to step S15, where the processor 51 temporarily stops the processing of the workpiece 5 being performed by a program other than this program. Thereafter, to determine the exact position of the misaligned workpiece 5, as described above, the processor 51 controls the robot arm 30 to completely retract from the imaging angle of the 3D camera 9, and then causes the 3D camera 9 to capture an image of the workpiece 5 on the conveyance device 7. The processor 51 identifies the position of the misaligned workpiece 5 based on the image captured by the 3D camera 9. Furthermore, the processor 51 detects, for example, points P1-P4 shown in FIG. 4 and calculates the line segment L1 based on the image captured by the 3D camera 9. The correction command output unit 57 outputs a control command to the robot arm 30 to correct the position of the workpiece 5 based on the identified position, each of the points P1-P4, and the line segment L1.

[0059] 6C , when the correction command output unit 57 detects that the outline point P1 is located beyond the upper side 17c of the pair of sides 17c in the rectangular region 17a and outside in the conveyor width direction (upper side in the figure), the correction command output unit 57 generates a control command to correct the position of the workpiece 5 so as to return the outline point P1 to the inside (lower side in the figure) of the upper side 17c of the pair of sides 17c in the conveyor width direction, and outputs the control command to the robot arm 30. Although not shown, when the correction command output unit 57 detects that the outline point P2 is located beyond the lower side 17c of the pair of sides 17c in the rectangular region 17a and outside in the conveyor width direction (lower side in the figure), the correction command output unit 57 generates a control command to correct the position of the workpiece 5 so as to return the outline point P2 to the inside (upper side in the figure) of the lower side 17c of the pair of sides 17c in the conveyor width direction, and outputs the control command to the robot arm 30.

[0060] For example, when the correction command output unit 57 detects that the position of the ankle point P3 has moved away from a position near the center of the conveyor width direction in the rectangular area 17a, it generates a control command to correct the position of the workpiece 5 so that the position of the ankle point P3 moves closer to the center of the conveyor width direction in the rectangular area 17a, and outputs the control command to the robot arm 30.

[0061] For example, when the correction command output unit 57 detects that the calculated inclination of the line segment L1 deviates from a predetermined inclination, it combines the contents of the correction of the positional deviation based on the positions of the outline points P1 and P2 described above with the contents of the correction of the positional deviation based on the position of the ankle point P3, and generates a control command to correct the position of the workpiece 5 so that the inclination of the line segment L1 approaches the predetermined inclination, and outputs the control command to the robot arm 30.

[0062] Based on the control command, the robot arm 30 corrects the position of the workpiece 5, for example, by pressing the workpiece 5 with the tool 40. Thereafter, as described above, the processor 51 causes the 3D camera 9 to again capture an image of the workpiece 5 on the transport device 7, and based on the obtained image data, resumes the processing of the workpiece 5 that was being performed by a program separate from this program.

[0063] After step S15 is executed, the processor 51 returns to the top of the main loop, that is, step S3.

[0064] 8 is a flowchart showing the procedure of a process for acquiring position information of a workpiece 5, which is performed by the position estimation device 60 of one embodiment. The program for executing the process shown in the flowchart of FIG. 8 is read from the memory 62 and executed by the processor 61. When machining of the workpiece 5 is started, in step S21 the processor 61 determines whether or not an end command for notifying that machining of the workpiece 5 has been completed has been received from the control device 50. Note that when the processor 51 of the control device 50 ends the process shown in the flowchart of FIG. 7, it outputs an end command to the position estimation device 60. When a positive determination is made in step S21, i.e., when it is determined that an end command has been received, the processor 61 ends the process shown in the flowchart of FIG. 8.

[0065] If a negative decision is made in step S21, that is, if it is determined that an end command has not been received, the process proceeds to step S23, where the processor 61 determines whether or not a photographing command has been received from the control device 50. If a negative decision is made in step S23, that is, if it is determined that a photographing command has not been received, the processor 61 returns to the top of the main loop, that is, to step S21.

[0066] If the result of step S23 is affirmative, that is, if it is determined that an image capturing command has been received, the process proceeds to step S25, where the processor 61 transmits an image capturing command to the color camera 8, causing the color camera 8 to capture an image. As a result, the image capturing data obtained by the color camera 8 is transmitted to the image acquisition unit 65 of the position estimation device 60.

[0067] Next, in step S27, the processor 61 inputs the imaging data acquired by the image acquisition unit 65 into the trained model 67 stored in the memory unit 66, and obtains a prediction result for the position of the workpiece 5 in the imaging data.

[0068] Next, in step S29, the coordinate conversion unit 68 of the position estimation device 60 converts the position information of the workpiece 5 estimated using the learned model 67, i.e., the coordinate values ​​representing the information on the position and size of the above-mentioned rectangular area 16a, into coordinate values ​​in a coordinate system related to the control of the robot arm 30.

[0069] Next, in step S31, the position information output unit 69 of the position estimation device 60 outputs the position information of the workpiece 5 after being converted by the coordinate conversion unit 68 to the position information acquisition unit 55 of the control device 50. When step S31 is performed, the processor 61 returns to the beginning of the main loop, i.e., step S21.

[0070] 9 is a flowchart showing the steps of a deviation determination process performed by the control device 50 in one embodiment. The processor 51 reads and executes a program for executing the process shown in the flowchart of FIG. 9 from the memory 52. ​​When the deviation determination process is started, the positional deviation detection unit 56 determines in step S41 whether or not this is the first call since the deviation determination process started. If a positive determination is made in step S41, that is, if it is determined that this is the first call since the deviation determination process started, the process proceeds to step S43, where the positional deviation detection unit 56 temporarily stores the positional information of the workpiece 5 acquired by the positional information acquisition unit 55 in step S7 of FIG. 7 in the memory 52 as the positional information of the workpiece 5 at the reference time point.

[0071] After step S43 is performed, the process proceeds to step S53, where the positional deviation detection unit 56 determines that the workpiece 5 is not misaligned.

[0072] If step S41 is judged to be negative, that is, if it is judged that this is not the first call after the start of the deviation judgment process, the process proceeds to step S45, and the position deviation detection unit 56 judges whether the position of a pair of opposing sides 16b in the transport direction of the workpiece 5 in rectangular area 16a in the position information of the workpiece 5 at the most recent time point is shifted downstream in the transport direction by more than threshold value Th1, relative to the position of a pair of opposing sides 17b in rectangular area 17a in the position information of the workpiece 5 at the reference time point, as shown in Figure 6A, for example.

[0073] If the result of step S45 is affirmative, the process proceeds to step S51, where the positional deviation detection unit 56 determines whether or not the affirmative determination in step S45 has occurred consecutively a specified number of times N1 or more.

[0074] If the result of step S51 is affirmative, the process proceeds to step S55, where the positional deviation detection unit 56 determines that the workpiece 5 is misaligned.

[0075] If a negative determination is made in step S51, the process proceeds to step S53, where the positional deviation detection unit 56 determines that no positional deviation of the workpiece 5 has occurred.

[0076] If step S45 is judged negative, the process proceeds to step S47, and the position deviation detection unit 56 determines whether the position of a pair of opposing sides 16b in the transport direction of the workpiece 5 in the rectangular area 16a in the position information of the workpiece 5 at the most recent time point is shifted upstream in the transport direction by more than the threshold value Th2, for example, as shown in Figure 6B, relative to the position of a pair of opposing sides 17b in the transport direction of the workpiece 5 in the rectangular area 17a in the position information of the workpiece 5 at the reference time point.

[0077] If the result of step S47 is affirmative, the process proceeds to step S51, where the positional deviation detection unit 56 determines whether or not the affirmative determination in step S47 has occurred consecutively a specified number of times N2 or more.

[0078] If the result of step S51 is affirmative, the process proceeds to step S55, where the positional deviation detection unit 56 determines that the workpiece 5 is misaligned.

[0079] If a negative determination is made in step S51, the process proceeds to step S53, where the positional deviation detection unit 56 determines that no positional deviation of the workpiece 5 has occurred.

[0080] If step S47 is judged negative, the process proceeds to step S49, and the position deviation detection unit 56 judges whether the positions of a pair of opposing sides 16c in the conveyor width direction in rectangular area 16a in the position information of the workpiece 5 at the most recent time point are shifted in the same direction by more than threshold value Th3, for example, as shown in Figure 6C, relative to the positions of a pair of opposing sides 17c in the conveyor width direction in rectangular area 17a in the position information of the workpiece 5 at the reference time point.

[0081] If the determination in step S49 is affirmative, the process proceeds to step S51, where the positional deviation detection unit 56 determines whether or not the affirmative determination in step S49 has occurred consecutively a specified number of times N3 or more.

[0082] If the result of step S51 is affirmative, the process proceeds to step S55, where the positional deviation detection unit 56 determines that the workpiece 5 is misaligned.

[0083] If a negative determination is made in step S51, the process proceeds to step S53, where the positional deviation detection unit 56 determines that no positional deviation of the workpiece 5 has occurred.

[0084] If a negative determination is made in step S49, the process proceeds to step S53, where the positional deviation detection unit 56 determines that no positional deviation of the workpiece 5 has occurred.

[0085] The present disclosure is not limited to the above-described embodiments, but also includes modifications to the above-described embodiments and appropriate combinations of these modifications.

[0086] The contents described in each of the above embodiments can be understood, for example, as follows: (1) The control device 50 of the meat processing device 1 according to at least one embodiment of the present disclosure includes a position information acquisition unit 55 that acquires position information of the meat (work 5) in time-series image data (captured images 15) including the meat (work 5) during meat processing, a position deviation detection unit 56 that detects whether or not the meat (work 5) is misaligned during meat processing based on the difference between the position information of the meat (work 5) at a reference time point and the position information of the meat (work 5) at the most recent time point, and a correction command output unit 57 that, when the position deviation detection unit 56 detects a position deviation, outputs a control command to a position deviation correction device (robot arm 30) that corrects the position of the meat (work 5).

[0087] According to the configuration (1) above, even if the position of the meat (workpiece 5) shifts during processing, the position of the meat (workpiece 5) can be corrected by the position shift correction device (robot arm 30).

[0088] (2) In some embodiments, the configuration of (1) above may include a storage unit (memory 52) that stores criteria for determining whether or not there is misalignment depending on the part of the meat (workpiece 5), the processing form, and the progress of processing. The misalignment detection unit 56 may detect whether or not there is misalignment based on the criteria stored in the storage unit (memory 52) and the difference between the position information of the meat (workpiece 5) at a reference point in time and the position information of the meat (workpiece 5) at the most recent point in time.

[0089] According to the above configuration (2), it is possible to improve the accuracy of detecting the presence or absence of positional deviation.

[0090] (3) In some embodiments, in the configuration of (2) above, the positional deviation detection unit 56 may detect a positional deviation when it determines that the difference between the positional information of the meat (work 5) at a reference time point and the positional information of the meat (work 5) at the most recent time point is equal to or greater than a threshold value Th included in the judgment criteria stored in the storage unit (memory 52) for a specified number of times N or more in succession.

[0091] According to the configuration (3) above, it is possible to determine that the meat (work 5) is misaligned when there is a high possibility that the meat (work 5) is misaligned, thereby improving the accuracy of detecting whether or not there is a misalignment.

[0092] (4) In some embodiments, in any of the configurations (1) to (3) above, the position information may be information representing the position and size of rectangular areas 16a, 17a representing the area where meat (workpiece 5) is present.

[0093] According to the configuration (4) above, the position and size of the area where the meat (workpiece 5) is present can be represented by a relatively simple shape, thereby reducing the computational load for detecting whether or not there is a positional deviation and shortening the processing time for detecting whether or not there is a positional deviation.

[0094] (5) In some embodiments, in the configuration of (3) above, the position information may be information representing the position and size of rectangular regions 16 a, 17 a representing the region where the meat (workpiece 5) is present. The positional deviation detection unit 56 may detect a positional deviation when it determines that the pair of opposing sides 16 b in the rectangular region 16 a in the positional information of the meat (workpiece 5) at the most recent time point deviates downstream in the conveying direction by more than a threshold value Th1 included in the judgment criteria, relative to the positions of the pair of opposing sides 17 b in the rectangular region 17 a in the positional information of the meat (workpiece 5) at a reference time point, and this has occurred a specified number of times or more consecutively.

[0095] According to the configuration (5) above, it is possible to determine that the meat (work 5) is misaligned when there is a high possibility that the meat (work 5) is misaligned, thereby improving the accuracy of detecting whether or not there is a misalignment.

[0096] (6) In some embodiments, in the configuration of (3) above, the position information may be information representing the position and size of rectangular regions 16 a, 17 a representing the region where the meat (workpiece 5) is present. The positional deviation detection unit 56 may detect a positional deviation when it determines that the pair of opposing sides 16 b in the rectangular region 16 a in the positional information of the meat (workpiece 5) at the most recent time point deviates upstream in the conveying direction by more than a threshold value Th2 included in the judgment criteria, relative to the positions of the pair of opposing sides 17 b in the rectangular region 17 a in the positional information of the meat (workpiece 5) at a reference time point, and this has occurred a specified number of times or more consecutively.

[0097] According to the configuration (6) above, it is possible to determine that the meat (work 5) is misaligned when there is a high possibility that the meat (work 5) is misaligned, thereby improving the accuracy of detecting whether or not there is a misalignment.

[0098] (7) In some embodiments, in the configuration of (3) above, the position information may be information representing the position and size of rectangular regions 16 a, 17 a representing the region where the meat (workpiece 5) is present. The positional deviation detection unit 56 may detect a positional deviation when it determines that the positions of a pair of opposing sides 16 c in the rectangular region 16 a in the direction perpendicular to the conveyance direction (conveyor width direction) of the meat (workpiece 5) in the positional information of the meat (workpiece 5) at the most recent time point deviate in the same direction by more than a threshold value Th3 included in the judgment criteria, with respect to the positions of a pair of opposing sides 17 c in the rectangular region 17 a in the positional information of the meat (workpiece 5) at a reference time point (conveyor width direction).

[0099] According to the configuration (7) above, it is possible to determine that the meat (work 5) is misaligned when there is a high possibility that the meat (work 5) is misaligned, thereby improving the accuracy of detecting whether or not there is a misalignment.

[0100] (8) In some embodiments, in any of the configurations (1) to (7) above, the location information acquisition unit 55 may acquire location information from a trained model 67 for estimating location information from imaging data (captured image 15).

[0101] According to the configuration (8) above, relatively accurate position information of the meat (workpiece 5) can be obtained from the imaging data (captured image 15) including the meat (workpiece 5) during meat processing.

[0102] (9) In some embodiments, in any of the configurations (1) to (8) above, the imaging data (image 15) may be imaging data (image 15) of meat (workpiece 5) being processed from above.

[0103] According to the configuration (9) above, position information of the meat (work 5) can be obtained based on image data (image 15) obtained by imaging the meat (work 5) from above during meat processing, which is relatively easy to image.

[0104] (10) A meat processing apparatus 1 according to at least one embodiment of the present disclosure includes a control device 50 having any of the configurations (1) to (9) above, a conveyor device (transport device 7) for transporting meat (workpiece 5), an imaging device (color camera 8) for acquiring imaging data (captured image 15), and a positional deviation correction device (robot arm 30).

[0105] According to the configuration (10) above, even if the position of the meat (workpiece 5) shifts during processing, the position of the meat (workpiece 5) can be corrected by the position shift correction device (robot arm 30).

[0106] REFERENCE SIGNS LIST 1 meat processing device 5 workpiece 7 conveying device 8 color camera 15 captured image 30 robot arm 40 tool 50 control device 60 position estimation device

Claims

1. A control device for a meat processing device, comprising: a position information acquisition unit that acquires position information of the meat in time-series image data including the meat during meat processing; a position deviation detection unit that detects whether or not the meat is misaligned during meat processing based on the difference between the position information of the meat at a reference time point and the position information of the meat at the most recent time point; and a correction command output unit that, when the position deviation detection unit detects a position deviation, outputs a control command to a position deviation correction device that corrects the position of the meat.

2. A control device for a meat processing device as described in claim 1, which is provided with a memory unit that stores judgment criteria for judging whether or not there is a positional deviation depending on the part of the meat, the processing form, and the progress of processing, and the positional deviation detection unit detects whether or not there is a positional deviation based on the judgment criteria stored in the memory unit and the difference between the positional information of the meat at the reference time point and the positional information of the meat at the most recent time point.

3. A control device for a meat processing device as described in claim 2, wherein the position deviation detection unit detects the position deviation when it determines that the difference between the position information of the meat at the reference time point and the position information of the meat at the most recent time point is equal to or greater than a threshold included in the judgment criteria stored in the memory unit for a specified number of consecutive times.

4. A control device for a meat processing device according to any one of claims 1 to 3, wherein the position information is information representing the position and size of a rectangular area representing the area in which the meat is present.

5. A control device for a meat processing device according to claim 3, wherein the position information is information representing the position and size of a rectangular area representing the area where the meat is present, and the position deviation detection unit detects the position deviation when it determines that a pair of opposing sides in the conveying direction of the meat in the rectangular area in the position information of the meat at the most recent time point deviates upstream in the conveying direction by more than a threshold included in the judgment criterion from the positions of a pair of opposing sides in the conveying direction of the meat in the rectangular area in the position information of the meat at the reference time point, and this has occurred consecutively for more than the specified number of times.

6. A control device for a meat processing device according to claim 3, wherein the position information is information representing the position and size of a rectangular area representing the area where the meat is present, and the position deviation detection unit detects the position deviation when it determines that a pair of opposing sides in the conveying direction of the meat in the rectangular area in the position information of the meat at the most recent time point deviates downstream in the conveying direction by more than a threshold included in the judgment criterion from the positions of a pair of opposing sides in the conveying direction of the meat in the rectangular area in the position information of the meat at the reference time point, and this has occurred consecutively for more than the specified number of times.

7. A control device for a meat processing device according to claim 3, wherein the position information is information representing the position and size of a rectangular area representing the area where the meat is present, and the position deviation detection unit detects the position deviation when it determines that a pair of opposing sides in a direction perpendicular to the conveying direction of the meat in the rectangular area in the position information of the meat at the most recent time deviate in the same direction by more than a threshold included in the judgment criterion from the positions of a pair of opposing sides in a direction perpendicular to the conveying direction of the meat in the rectangular area in the position information of the meat at the reference time, and this has occurred consecutively for more than the specified number of times.

8. A control device for a meat processing device according to any one of claims 1 to 3, wherein the location information acquisition unit acquires the location information from a trained model for estimating the location information from the imaging data.

9. A control device for a meat processing device according to any one of claims 1 to 3, wherein the imaging data is imaging data obtained by imaging the meat from above during processing.

10. A meat processing device comprising: a control device according to any one of claims 1 to 3; a conveyor device for transporting the meat; an imaging device for acquiring the imaging data; and the positional deviation correction device.

Images