Meat action point calculation system, meat processing system, and meat action point calculation method

Abstract

To provide an edible meat work point calculation system, an edible meat processing system, and an edible meat work point calculation method capable of identifying a work point with high accuracy while reducing the burden of teacher data preparation.SOLUTION: An edible meat work point calculation system 10 includes an image data acquisition unit 11 and a work point acquisition unit 12. The image data acquisition unit 11 is configured to acquire image data representing a photographed image of edible meat. The work point acquisition unit 12 is configured to input the image data acquired by the image data acquisition unit 11 to a learning model 70 that has undergone machine learning using the image data and the correct data indicating key points of the edible meat 5 contained in the photographed image as teacher data, and to acquire work point data for identifying at least one work point that a robot 4 gives to work in the edible meat.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a system for calculating a point of action for meat, a meat processing system, and a method for calculating a point of action for meat. [Background technology]

[0002] Conventionally, there is known a system for calculating the point of action of meat by calculation. The point of action defines, for example, the cutting position of the meat. In Patent Document 1, the contour of a bone part is detected based on a first image obtained by photographing the cut surface of the meat part irradiated with blue visible light to cause excitation and emission, and the outer contour of the cut surface is detected based on a second image obtained by photographing the cut surface of the meat part irradiated with white visible light. Then, the position of the bone part is calculated according to a predetermined rule from the relative position of the detected outer contour of the cut surface and the contour of the bone part, and the cutting position of the meat part is determined based on the calculated position of the bone part. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-7982 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above-mentioned patent document, in which the position of a bone portion is determined according to a predetermined rule, if the amount of bone exposure in the photographed meat portion varies, the position of the bone portion may not be determined accurately. Furthermore, when determining the position of a bone portion using a learning model instead of the above-mentioned method, it is preferable to reduce the burden of preparing training data, but the above-mentioned patent document does not disclose a configuration for achieving this.

[0005] An object of the present disclosure is to provide a meat action point calculation system, a meat processing system, and a meat action point calculation method that can identify the action point with high accuracy with less burden in preparing training data. [Means for solving the problem]

[0006] At least one embodiment of the present disclosure relates to a system for calculating a point of action for meat, an image data acquisition unit configured to acquire image data representing a captured image of the meat; an action point acquisition unit configured to input the image data acquired by the image data acquisition unit into a learning model that has been machine-trained using the image data and correct answer data that indicates key points of the meat contained in the photographed image as training data, and to acquire action point data for identifying at least one action point on the meat at which the robot imparts an action; Equipped with.

[0007] A meat processing system according to at least one embodiment of the present disclosure comprises: The meat action point calculation system; the robot configured to impart an action to the action point acquired by the action point acquisition unit; Equipped with.

[0008] At least one embodiment of the present disclosure provides a method for calculating an action point for meat, comprising: an image data acquisition step of acquiring image data representing a photographed image of meat; The method further comprises an action point acquisition step of inputting the image data acquired by the image data acquisition step into a learning model that has been machine-trained using the image data and correct answer data that indicates key points of the meat contained in the photographed image as training data, and acquiring action point data for identifying at least one action point on the meat that is the target of an action to be applied by the robot. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to provide a meat action point calculation system, a meat processing system, and a meat action point calculation method that reduce the burden of preparing training data and can identify the action point with high accuracy. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a conceptual diagram illustrating the overall configuration of a meat processing system according to an embodiment of the present disclosure. [Figure 2] FIG. 1 is a conceptual diagram illustrating meat according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a conceptual diagram of a captured image according to an embodiment of the present disclosure. [Figure 4] 10 is a graph conceptually showing a comparison result between the estimation accuracy of the point of action using a learning model and the estimation accuracy of the point of action using a conventional method. [Figure 5] FIG. 1 is an enlarged view conceptually illustrating a defined region in a left carcass according to one embodiment of the present disclosure. [Figure 6] FIG. 1 is an explanatory diagram conceptually illustrating teacher data according to an embodiment of the present disclosure. [Figure 7] FIG. 1 is a conceptual diagram illustrating the configuration of a learning model according to an embodiment of the present disclosure. [Figure 8] 1 is an explanatory diagram conceptually illustrating a first pressing device and a second pressing device according to an embodiment of the present disclosure. [Figure 9] 1 is a flowchart illustrating a method for calculating a point of action of meat according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, several embodiments of the present disclosure will be described with reference to the accompanying drawings. However, the dimensions, materials, shapes, relative arrangements, etc. of components described as embodiments or shown in the drawings are merely illustrative examples and are not intended to limit the scope of the present disclosure. For example, expressions expressing relative or absolute arrangement such as "in a certain direction," "along a certain direction," "parallel," "orthogonal," "center," "concentric," or "coaxial" not only express such an arrangement exactly, but also express a state in which there is a relative displacement with a tolerance or an angle or distance to the extent that the same function is obtained. For example, expressions such as "identical," "equal," and "homogeneous" that indicate that something is in an equal state not only indicate a state of strict equality, but also indicate a state in which there is a tolerance or a difference to the extent that the same function is obtained. For example, expressions representing shapes such as a square shape or a cylindrical shape not only represent shapes such as a square shape or a cylindrical shape in the strict geometric sense, but also represent shapes including uneven portions, chamfered portions, etc., to the extent that the same effect can be obtained. On the other hand, the expressions "comprise," "include," or "have" one element are not exclusive expressions that exclude the presence of other elements. Note that the same components will be denoted by the same reference numerals and the description thereof will be omitted.

[0012] <1. Overview of Meat Processing System 1> An overview of a meat processing system 1 will be illustrated with reference to Figures 1 to 3. Figure 1 is a conceptual diagram showing the overall configuration of a meat processing system 1 according to an embodiment of the present disclosure. Figure 2 is a conceptual diagram showing meat 5 according to an embodiment of the present disclosure. Figure 3 is a conceptual diagram of a captured image 20 showing meat 5 according to an embodiment of the present disclosure.

[0013] The meat processing system 1 identifies a point of action P of the meat 5 based on the output results from the learning model 70 to which image data 28 showing a photographed image 20 of the meat 5 has been input, and executes a processing operation. The point of action P is the point at which the robot 4 applies an action to the meat 5. The action applied to the meat 5 is a concept that includes cutting, grasping, pressing, pounding, deboning, irradiating the meat 5 with light, discharging a liquid, or a combination of these, and is almost synonymous with processing. The following describes an example in which the action applied to the meat 5 is cutting the meat 5. Moreover, the meat 5 is, as an example, the carcass of a livestock carcass. More specifically, the meat 5 includes a left carcass 5L of the livestock and a right carcass 5R of the livestock. Each of the left carcass 5L and the right carcass 5R includes a backbone 6 and multiple ribs 9, at least some of which are exposed and visible. Note that in the conceptual diagrams of Figures 2 and 3, only the lower ribs 9 are shown. Hereinafter, when there is no particular distinction between the left carcass 5L and the right carcass 5R, they may be referred to as "meat 5." Note that the ribs 9 protrude to the left from the backbone 6 in the left carcass 5L shown in Figure 2, and protrude to the right from the backbone 6 in the right carcass 5R shown in the same figure.

[0014] In this embodiment, imaginary cutting lines K and M in the meat 5 are defined by at least one action point P (see FIG. 2) identified based on the output results from the learning model 70. A forebody 41 is cut off from the meat 5 by cutting along the imaginary cutting line K, and the remaining meat 5 is divided into a middle body 42 and a hind body 43 by cutting along the imaginary cutting line M. In this embodiment, the imaginary cutting line K includes an imaginary cutting line KL defined by left application points P1 and P2 in the left carcass 5L and an imaginary cutting line KR defined by right application points P3 and P4 in the right carcass 5R. Meanwhile, the imaginary cutting line M includes an imaginary cutting line ML defined by left application point P5 in the left carcass 5L and the first set angle, and an imaginary cutting line MR defined by right application point P6 in the right carcass 5R and the second set angle. As will be described in detail later, the left application point P1 and the right application point P4 are located between any two of the multiple ribs 9. Both imaginary cutting lines ML and MR are set so as to pass through only the lumbar vertebrae of the bones that make up the meat 5, and the left application point P5 and the right application point P6 are set so as to overlap the lumbar vertebrae of the meat 5. In other embodiments, the imaginary cut-off line KR may be defined by the left application points P1 and P2 and the right application point P3. Specifically, the imaginary cut-off line KR may be defined as a virtual line that is symmetrical to the imaginary cut-off line KL and passes through the right application point P3. The imaginary cut-off line ML may be defined by the left application point P5 and another left application point. The same applies to the imaginary cut-off line MR. In the following description, the left action points P1, P2, P5 and the right action points P3, P4, P6 may be referred to as "action points P" when no particular distinction is made between them.

[0015] <2. Example of overall configuration of meat processing system 1> 1 and 2, an example of the overall configuration of a meat processing system 1 according to one embodiment of the present disclosure is shown. The meat processing system 1 includes a horizontally extending rail 45 and a conveying device 46 configured to grip meat 5 and convey it along the rail 45. The conveying device 46 in this example is configured to convey a suspended left carcass 5L and a right carcass 5R at a time. In other embodiments, the conveying device 46 may be configured to convey only either the left carcass 5L or the right carcass 5R in one conveyance, in which case the left carcasses 5L and the right carcasses 5R are conveyed alternately.

[0016] The meat processing system 1 further includes a first processing station 51 and a second processing station 52 arranged in order from the upstream side along the conveying line of the conveying device 46, and action point calculation systems for meat (hereinafter sometimes simply referred to as "action point calculation systems") 10A and 10B arranged corresponding to the first processing station 51 and the second processing station 52, respectively. The first processing station 51 is configured to cut the meat 5 along an imaginary cutting line K. The second processing station 52 is configured to cut the meat 5 along an imaginary cutting line M. The first processing station 51 is controlled by the action point calculation system 10A, and the second processing station 52 is controlled by the action point calculation system 10B. The first processing station 51 and the second processing station 52 have the same configuration, and the load point calculation system 10A and the load point calculation system 10B also have the same configuration. In the following, the configurations of the first processing station 51 and the load point calculation system 10A will be mainly described, and when there is no particular distinction between the load point calculation system 10A and the load point calculation system 10B, they may be referred to as the "load point calculation system 10."

[0017] 3. Example of the Configuration of the First Machining Station 51 and the Point-of-Work Calculation System 10A 1 and 3 , the configuration of a first processing station 51 and a point-of-load calculation system 10A according to one embodiment of the present disclosure is illustrated. The first processing station 51 includes a white light source 7, an imaging device 8 configured to capture an image of meat 5 illuminated by the white light source 7 and generate a captured image 20, a first pressing device 31 and a second pressing device 32 for pressing the meat 5 while the imaging device 8 is capturing the image, and a robot 4 equipped with a cutter 3 for cutting the meat 5. The first pressing device 31 and the second pressing device 32 in this example press the meat 5 in a horizontal direction while sandwiching it between them. Details of the first pressing device 31 and the second pressing device 32 will be described later.

[0018] As an example, the above-described photographing device 8 simultaneously photographs both the left carcass 5L and the right carcass 5R transported by the transporting device 46. Therefore, the photographed image 20 generated by the photographing device 8 includes a first area 21 in which the left carcass 5L of the livestock is captured, and a second area 22 in which the right carcass 5R of the livestock is captured. In this embodiment, each of the first area 21 and the second area 22 includes a defined area 23 in which a plurality of ribs 9 are captured.

[0019] The load calculation system 10A includes at least one processor and at least one memory. The processor may be a CPU, a GPU, an MPU, a DSP, or a combination thereof. The processor may be implemented by an integrated circuit such as a PLD, an ASIC, an FPGA, or an MCU. The memory is configured to temporarily or non-temporarily store various data and may be implemented by, for example, a RAM, a ROM, a flash memory, or a combination thereof. The load calculation system 10A may be implemented by a combination of a control device such as a controller and a personal computer equipped with a GPU.

[0020] As shown in Figure 1, the above-mentioned point of action calculation system 10A comprises an image data acquisition unit 11 configured to acquire image data 28 (see Figure 3) showing a photographed image 20 of meat 5, a memory unit 13 that stores a learning model 70, a point of action acquisition unit 12 configured to input the image data 28 acquired by the image data acquisition unit 11 into the learning model 70 to acquire point of action data for identifying the point of action P on the meat 5, and a processing command unit 14 configured to send a processing command to the robot 4 based on the point of action data acquired by the point of action acquisition unit 12.

[0021] As will be described in detail later, in the learning model 70 of this example, image data 28 representing the captured image 20 and correct answer data 27 representing key points (feature points) of the meat 5 contained in the captured image 20 are prepared as training data 25 (see FIG. 5) for machine learning (deep learning). These key points correspond to the action points P. In other words, the action point data for identifying the action points P, which is the output result from the learning model 70, is data related to the coordinates of the action points P on the meat 5, such as distribution data representing heat maps centered on each of the action points P. The action point acquisition unit 12 (see FIG. 1) acquires this action point data and sends it to the processing command unit 14. The processing command unit 14 considers the coordinates with the highest likelihood in each of the four heat maps to be the action points P and identifies the imaginary cutting lines K. The processing command unit 14 then sends a command to the robot 4 to perform a cutting operation along the imaginary cutting lines K. In other words, the robot 4 is configured to apply an action to the action points P acquired by the action point acquisition unit 12. In other embodiments, the action point data output from the learning model 70 may be only the specific coordinates of the action point P.

[0022] <4. Example of comparison results between learning model 70 and conventional methods> 4 is a graph conceptually showing the results of a comparison between the estimation accuracy of the point of action P using the learning model 70 of the point of action calculation system 10A and the estimation accuracy of the point of action P using the conventional method disclosed in Patent Document 1. In this comparison, meat 5 was prepared for which the coordinates of the point of action P, which should be the correct solution, were known in advance, and the accuracy of the estimated coordinates of the point of action P obtained by the learning model 70 and the conventional method was verified. In the graph in the figure, the correct coordinates are plotted on the horizontal axis, and the estimated coordinates are plotted on the vertical axis. Ideally, the estimated coordinates obtained should match the correct coordinates, and the closer the estimated coordinates are to the two-dot chain line S, the higher the estimation accuracy. In Figure 4, the allowable range of deviation from the ideal is indicated by dashed lines H1 and H2.

[0023] The "◯" in the graph indicates the coordinate of the point of action P identified based on the point of action data acquired by the point of action acquisition unit 12, and the "×" indicates the coordinate of the point of action P acquired by the conventional method. As can be seen from Fig. 4, it was confirmed that the estimation accuracy of the point of action P using the learning model 70 of the point of action calculation system 10A is significantly higher than the estimation accuracy of the point of action P using the conventional method disclosed in Patent Document 1.

[0024] According to the above configuration, since the learning model 70 configured to output point of action data is a model that outputs key points, the correct answer data 27 prepared as the training data 25 need only be coordinate data associated with the image. In other words, the correct answer data 27 need only be coordinate data indicating the screen coordinates that indicate the point of action P. This eliminates the need for image processing such as edge processing and masking on the image data 28 prepared as the training data 25, thereby simplifying the training data 25. This reduces the burden of preparing the training data 25 and realizes a point of action calculation system 10 for meat that can identify the point of action P with high accuracy.

[0025] In another embodiment, the first processing station 51 may be configured to grip the meat 5. In this case, the action point P acquired by the action point acquisition unit 12 indicates the gripping position. In this case, the processing command unit 14 does not specify the imaginary cutting lines K, M, and the robot 4 attaches a clamp instead of the cutter 3.

[0026] In addition, in this embodiment, the first processing station 51 of the meat processing system 1 is equipped with a white light source 7 and an imaging device 8 configured to photograph the meat 5 illuminated by the white light source 7 and generate a photographed image 20, and does not have a blue light source. According to the above configuration, the photographing device 8 photographs the meat 5 illuminated by the white light source 7, which is more versatile than special light sources such as blue light sources, and therefore the equipment can be simplified.

[0027] <5. Example of action point data output from learning model 70 of action point calculation system 10A> 3 and 5, details of the point of action data output from the learning model 70 of the point of action calculation system 10A are illustrated. Fig. 5 is an enlarged view conceptually showing the specified area 23 in the left carcass 5L according to one embodiment of the present disclosure. The action point data output from the learning model 70 can be expressed as a heat map centered on the action point P. This heat map shows the distribution of the probability (likelihood) that two or more pixels constituting the captured image 20 correspond to the action point P as a probability density distribution. In FIG. 5, the heat maps for the left action points P1 and P2 are conceptually illustrated, and the heat maps for the right action points P3 and P4 are omitted. The action point data output from the learning model 70 is sent to the action point acquisition unit 12.

[0028] The machining command unit 14, which acquires the action point data from the action point acquisition unit 12, regards the pixels with the highest likelihood in the heat maps corresponding to the left action points P1 and P2 as the left action points P1 and P2. This allows the machining command unit 14 to identify the imaginary cutting line KL. Although detailed illustration is omitted, by using a similar principle, the machining command unit 14 can identify the imaginary cutting line KR (see FIG. 2) based on the heat maps of the right action points P3 and P4. In this example, the left action point P1 and the right action point P4 are located between any two of the ribs 9, and the left action point P2 and the right action point P3 are located so as to overlap the spine 6. In other words, the imaginary cutting line K is defined at a position that avoids the ribs 9. This makes it possible to prevent damage to the ribs 9 when the robot 4 (see FIG. 1) uses the cutter 3 to cut the meat 5 along the imaginary cutting line K. In order for the action point P to satisfy the above-mentioned positional conditions, the training data 25 used in the learning stage of the learning model 70 should be prepared accordingly. The learning process of the learning model 70 will be described below.

[0029] <6. Example of the learning process of the learning model 70 of the action point calculation system 10A> Referring to Figure 6, an example of the learning process of the learning model 70 of the point-of-action calculation system 10A is shown. Figure 6 is an explanatory diagram conceptually showing training data 25 according to an embodiment of the present disclosure. The training data 25 is made up of multiple sets of data, with each set consisting of image data 28 representing a captured image 20 prepared for learning and supervised data 27 associated with the image data 28. The supervised data 27 indicates a probability density distribution of key points (feature points) of meat 5 included in the captured image 20 represented by the corresponding image data 28.

[0030] The correct answer data 27 is prepared, for example, as follows. The operator identifies the coordinates of the action points P (left action points P1, P2 and right action points P3, P4) that should be considered correct for the captured image 20 represented by the image data 28. The coordinates of the identified action points P are then input to the heat map generation unit 35, thereby generating correct answer data 27. Here, the heat map generation unit 35 is constructed using a specified algorithm and generates a Gaussian distribution centered on the coordinates of the left action points P1, P2 and the right action points P3, P4 in the captured image 20. This distribution indicates the probability (likelihood) that each of two or more pixels constituting the captured image 20 corresponds to the action point P. In the example of FIG. 6, the correct heat maps corresponding to the left action points P1, P2 and the right action points P3, P4, respectively, are indicated by symbols Q1 to Q4. The image resolution of the correct answer data 27 representing the heat maps (Q1 to Q4) is the same as that of the captured image 20.

[0031] A certain amount of training data 25, which associates image data 28 with the above-mentioned supervised data 27, is input to the learning model 70. Various parameters such as weighting coefficients in the learning model 70 are adjusted and learning is performed so that the mean square error between the heat map output from the learning model 70 and the heat maps (Q1 to Q4) included in the supervised data 27 converges to a specified value such as 0. Such a learning model 70 is realized by, for example, HRNet, ResNet, or DenseNet.

[0032] The learning model 70, which has been machine-learned using the correct answer data 27 indicating the probability density distribution of the key points as the training data 25, is configured to output the probability density distribution of the action point P as action point data. The size and appearance of meat 5 tend to vary. Variation in the size of meat 5 is due to, for example, the growth of livestock. When the size of meat 5 varies, the position of the point of application P of meat 5 also varies, which can hinder learning by the learning model 70. Furthermore, variation in the appearance of meat 5 is due to, for example, variations in the process of obtaining meat 5 from livestock. For example, when the number of visible ribs 9 in meat 5 varies, the relative positional relationship between the visible ribs 9 and the position of the point of application P changes, which can hinder learning by the learning model 70. In this regard, according to the above-described configuration, the learning model 70 incorporates a probability density distribution of key points that should be correct as correct answer data 27 during the learning process. This probability density distribution comprehensively reflects the various variations in meat 5 described above. Therefore, even if the meat 5 shown in the captured images 20 of the image data 28 input to the learning model 70 that has completed learning varies, the probability density distribution indicated by the output action point P data is likely to include the correct position of the action point P. Therefore, the action point P of the meat 5 can be identified with high accuracy.

[0033] As described above, the captured image 20 includes a first area 21 in which the left carcass 5L is shown and a second area 22 in which the right carcass 5R is shown, and the action point P has left action points P1 and P2 at which the robot 4 applies an action on the left carcass 5L, and right action points P3 and P4 at which the robot 4 applies an action on the right carcass 5R. According to the above configuration, by inputting image data 28 into a single learning model 70, the action point acquisition unit 12 can acquire action point data for identifying the left action points P1, P2 and the right action points P3, P4, respectively. This allows the number of learning models 70 stored in the memory unit 13 to be reduced. Therefore, both the computational load of causing the learning model 70 to perform machine learning and the computational load of inputting image data 28 into the learning model 70 that has completed learning to obtain action point data are reduced. Therefore, the computational load for identifying the action point P can be reduced.

[0034] As described above, the captured image 20 includes a specified area 23 that shows multiple ribs 9 of the meat 5. The point of action data output by the learning model 70 includes point of action data for identifying the left point of action P1 and the right point of action P4 that are located so as to avoid the multiple ribs 9. This is achieved by specifying the correct coordinates of the left point of action P1 and the right point of action P4, which are the basis of the correct answer data 27, so as to avoid the multiple ribs 9 during the learning stage of the learning model 70. According to the above configuration, it is possible to apply an action to the meat 5 without damaging the ribs 9 of the meat 5. For example, if the meat 5 is cut so as to avoid the ribs 9, the protrusion of the broken ribs 9 and the generation of bone chips from the ribs 9 are suppressed. This makes it possible to maintain a good appearance of the meat 5. Furthermore, when an operator carries the processed meat 5, the safety of the operator can be ensured.

[0035] <7. Example of specific configuration of learning model 70> Referring to FIG. 7, a specific configuration of the learning model 70 in the action point calculation system 10A is illustrated. FIG. 7 is a conceptual diagram showing the configuration of the learning model 70 according to an embodiment of the present disclosure. The learning model 70 is a neural network including an input layer 78, an intermediate layer 77, and an output layer 79, and HRNet is used in this embodiment. In HRNet, a first network configured so that image data with the same resolution as the image data 28 is transmitted from a lower layer to a higher layer, and a second network configured so that image data with a lower resolution than the image data 28 is transmitted from the middle of the first network to a higher layer are constructed in parallel. The output layer 79 is configured to output image data with the same resolution as the image data 28. These parallel networks also perform learning in parallel.

[0036] The intermediate layer 77 of such a learning model 70 includes a first intermediate layer 71 and a second intermediate layer 72. The first intermediate layer 71 includes a first neuron 81 to which first data, which is image data with the same resolution as the image data 28 input to the input layer 78, is input. The second intermediate layer 72 includes a second neuron 82 to which second data, which is image data with the same resolution as the image data 28, is input from the first neuron 81, and a third neuron 83 to which third data with a lower resolution is input from the first neuron 81. According to the above configuration, a network that transmits image data with the same resolution as image data 28 and a network that transmits image data with a lower resolution than image data 28 are constructed in parallel, so that the point of action P can be identified with higher accuracy.

[0037] 8. Examples of the first pressing device 31 and the second pressing device 32 of the first processing station 51 3 and 8, configurations of the first pressing device 31 and the second pressing device 32 of the first processing station 51 are illustrated. Fig. 8 is an explanatory diagram conceptually showing the first pressing device 31 and the second pressing device 32 according to an embodiment of the present disclosure.

[0038] The first pressing device 31 of this embodiment is configured to horizontally press the photographing target part 5T of the meat 5 (left carcass 5L and right carcass 5R) being transported by the transport device 46 to a position a specified distance away from the photographing device 8. In this embodiment, the first pressing device 31 performs a similar operation even during the learning stage of the learning model 70, and photographing is performed using the photographed image 20.

[0039] If the shooting distance, which is the distance between the camera device 8 and the meat 5 when photographing, fluctuates, the appearance of the captured images 20 will vary greatly even when similar pieces of meat 5 are photographed. As a result, this can be an obstacle to learning by the learning model 70. In this regard, with the above configuration, the shooting distance between the meat 5 and the camera device 8 when photographing is stabilized, which reduces variation in the captured images 20 and allows the learning model 70 to learn well. As a result, the point of action P can be identified with high accuracy.

[0040] The first pressing device 31 of this embodiment includes a pair of first pressing members 31A configured to enter the photographing range of the photographing device 8 and press the meat 5 at a position that avoids the point of application P. The pair of first pressing members 31A appear in the photographed image 20 (see FIG. 3). In this example, a pair of first pressing members 31A are provided, one above the other. Each first pressing member 31A is a bar extending along the conveying line of the conveying device 46. The lower first pressing member 31A of the pair of first pressing members 31A may be configured so that its vertical position can be adjusted in response to an instruction from an operator. In such an embodiment, even during the learning stage of the learning model 70, the pair of first pressing members 31A perform the same operation to capture the captured image 20. In another embodiment, a pair of first pressing members 31A may be provided on the left and right sides, and in this case, each first pressing member 31A may be a bar extending in the vertical direction.

[0041] According to the above configuration, the pair of first pressing members 31A are stably captured at specific positions in the photographed image 20, which limits the area in the photographed image 20 where differences in the meat 5 appear, thereby facilitating effective learning by the learning model 70. As a result, the point of application P can be identified with high accuracy.

[0042] The second pressing device 32 (see FIG. 8 ) is configured to press the meat 5 horizontally from the opposite side to the first pressing device 31 with a weaker pressing force than the first pressing device 31. The second pressing device 32 may press the meat 5 with, for example, a bar, or may press the meat 5 with a pillar-shaped member such as a block. In this embodiment, after the pair of first pressing members 31A press the part 5T to be photographed to a specified position, the second pressing device 32 presses against the meat 5. In other words, the meat 5 is sandwiched between the first pressing device 31 and the second pressing device 32 and photographed by the photographing device 8. The pressing force (F2 in FIG. 8) by the second pressing device 32 is lower than the pressing force (F1 shown in FIG. 8) by the first pressing member 31A. Therefore, even when the second pressing device 32 presses the meat 5, the meat 5 is not pushed back towards the imaging device 8, but the meat 5 is sandwiched between the first pressing device 31 and the second pressing device 32, so that the imaging target site 5T is flattened.

[0043] According to the above configuration, the meat 5 to be photographed is sandwiched between the first pressing device 31 and the second pressing device 32, so the part 5T of the meat 5 to be photographed is flattened and is more likely to appear in the photographed image 20. Furthermore, because the pressing force of the second pressing device 32 is weaker than that of the first pressing device 31, fluctuations in the photographing distance can be suppressed. This further suppresses variations that hinder learning of the meat 5 shown in the photographed image 20, allowing for good learning by the learning model 70. As a result, the point of action P can be identified with high accuracy.

[0044] 9. Example of the Configuration of the Second Machining Station 52 and the Point of Action Calculation System 10B Returning to FIG. 1, the configuration of the second processing station 52 and the point of application calculation system 10B will be illustrated. As described above, the second processing station 52 has the same configuration as the first processing station 51, and the load point calculation system 10B has the same configuration as the load point calculation system 10A. That is, the second processing station 52 includes a white light source 7, an imaging device 8, a first pressing device 31, a second pressing device 32, and a robot 4. These components are the same as those in the first processing station 51. The point of action calculation system 10B also includes an image data acquisition unit 11, a point of action acquisition unit 12, a memory unit 13 that stores a learning model 70, and a processing command unit 14. These components are the same as those in the point of action calculation system 10A.

[0045] The captured image 20 acquired by the image data acquisition unit 11 of the point of action calculation system 10B shows the meat 5 with the forebody 41 cut off (not shown). The learning model 70 in the point of action calculation system 10B is machine-learned using correct answer data 27 indicating key points of the meat 5 corresponding to the left point of action P5 and the right point of action P6 of the meat 5 as training data 25. The left point of action P5 and the right point of action P6 prepared as the correct answer data 27 overlap only with the lumbar vertebrae of the bones of the meat 5. In other embodiments, a learning model 70 corresponding to each of the left carcass 5L and the right carcass 5R may be prepared, or a single learning model 70 corresponding to both the left carcass 5L and the right carcass 5R may be prepared, as in the case of the point of action calculation system 10A.

[0046] In addition, error detection may be performed by comparing the positional relationship of the load point P acquired by the load point acquisition unit 12 of the load point calculation system 10B with the positional relationship of the load point P acquired by the load point acquisition unit 12 of the load point calculation system 10A.

[0047] Specifically, the vertical positional relationship between the left carcass 5L and the right carcass 5R (hereinafter referred to as the "first positional relationship") is calculated based on the vertical positional relationship between the left points of action P1, P2 and the right points of action P3, P4 acquired by the point of action acquisition unit 12 of the point of action calculation system 10A. For example, the first positional relationship is the amount of vertical deviation of the centers of gravity of the left carcass 5L and the right carcass 5R. Next, the vertical positional relationship between the left action point P5 and the right action point P6 acquired by the action point acquisition unit 12 of the action point calculation system 10B is used to determine the vertical positional relationship between the left carcass 5L and the right carcass 5R (hereinafter referred to as the "second positional relationship"). If the meat 5 is processed properly in the first processing station 51 and the second processing station 52 in that order, there will be almost no change between the first positional relationship and the second positional relationship. Conversely, if some kind of trouble occurs during the processing of the meat 5, the first positional relationship and the second positional relationship will change significantly. Therefore, if it is determined that the difference between the first positional relationship and the second positional relationship exceeds the allowable value, the point of application calculation system 10B may issue a notification to that effect before the processing by the robot 4. This allows the operator to perform appropriate processing of the meat 5 (for example, manually cutting the meat 5).

[0048] <10. Example of how to obtain the point of action> 9 is a flowchart showing a method for calculating the point of action P of meat 5 according to one embodiment of the present disclosure. Below, a method for calculating the point of action P by the point of action calculation system 10A is illustrated, and steps may be abbreviated as "S." This flowchart is executed, for example, by the point of action calculation system 10A.

[0049] First, image data 28 showing the photographed image 20 of the meat 5 transported to the first processing station 51 is acquired by the image data acquisition unit 11 (S11). Next, the image data 28 acquired in S11 is input into the learning model 70, and the action point acquisition unit 12 acquires action point data for identifying the action points P (left action points P1, P2 and right action points P3, P4) applied by the robot 4 to the meat 5 (S13). Next, the processing command unit 14 sends a processing command to the robot 4 to perform a cutting operation along the imaginary cutting line K identified based on the action point P acquired by the action point acquisition unit 12 (S15). As a result, the meat 5 is cut along the imaginary cutting line K.

[0050] <11. Summary> The contents of the above-described embodiments can be understood, for example, as follows.

[0051] 1) At least one embodiment of the present disclosure relates to a system (10) for calculating a point of action for meat. an image data acquisition unit (11) configured to acquire image data (28) representing a captured image (20) of meat (5); an action point acquisition unit (12) configured to input the image data (28) acquired by the image data acquisition unit (11) into a learning model (70) that has been machine-trained using the image data (28) and correct answer data (27) indicating key points of the meat (5) contained in the photographed image (20) as training data (25), and to acquire action point data for identifying at least one action point (P) on the meat (5) to which the robot (4) applies an action; Equipped with.

[0052] According to the configuration 1), the learning model (70) configured to output point of action data is a model that outputs key points, so the correct answer data (27) as the prepared teacher data (25) can be coordinate data associated with the captured image (20). This simplifies the teacher data (25). This reduces the burden of preparing the teacher data (25), and realizes a point of action calculation system (10) for meat that can identify the point of action (P) with high accuracy.

[0053] 2) In some embodiments, the meat application point calculation system (10) described in 1) above, The learning model (70) machine-learned using the correct answer data (27) indicating the probability density distribution of the key points as the training data (25) is configured to output the probability density distribution of the action point (P) as the action point data.

[0054] The size or appearance of meat (5) tends to vary. Variations in the size of meat (5) are due to, for example, the growth of livestock. When the size of meat (5) varies, the position of the point of action (P) of the meat (5) also varies, which can hinder the learning of the learning model (70). Furthermore, variations in the appearance of meat (5) are due to, for example, variations in the process of obtaining meat (5) from livestock. For example, when the number of visible ribs (9) in the meat (5) varies, the relative positional relationship between the visible ribs (9) and the position of the point of action (P) becomes inconsistent, which can hinder the learning of the learning model (70). In this regard, according to the configuration of 2) above, the learning model (70) incorporates a probability density distribution of key points that should be considered as correct answers as correct answer data (27) during the learning process. This probability density distribution comprehensively reflects the various variations in meat (5) described above. Therefore, even if the meat (5) shown in the captured image (20) of the image data (28) input to the learned learning model (70) varies, the probability density distribution indicated by the output action point data is likely to include the correct position of the action point (P), making it possible to identify the action point (P) of the meat (5) with high accuracy.

[0055] 3) In some embodiments, the meat application point calculation system (10) described in 1) or 2) above, The photographed image (20) includes a first area (21) in which a left carcass (5L) of the livestock is captured, and a second area (22) in which a right carcass (5R) of the livestock is captured, The at least one action point (P) is At least one left action point (P1, P2, P5) at which the robot (4) acts on the left carcass (5L); and at least one right action point (P3, P4, P6) at which the robot (4) applies an action to the right carcass (5R).

[0056] According to the configuration 3) above, the image data (28) is input to the learning model (70), and the action point acquisition unit (12) can acquire action point data for identifying the left action point (P1, P2, P5) and the right action point (P3, P4, P6). This allows the number of learning models (70) to be reduced, thereby reducing the calculation load for identifying the action point (P).

[0057] 4) In some embodiments, the meat application point calculation system (10) according to any one of 1) to 3) above, The photographed image (20) includes a specified area (23) in which a plurality of ribs (9) of the meat (5) are captured, The learning model (70) is configured to output the action point data for identifying the action point (P) that avoids the plurality of ribs (9).

[0058] According to the above configuration 4), the meat (5) can be treated without damaging the ribs (9) of the meat (5), and the appearance of the meat (5) can be maintained in good condition.

[0059] 5) In some embodiments, the meat application point calculation system (10) according to any one of 1) to 4) above, The learning model (70) a first hidden layer (71) including a first neuron (81) to which first data having the same resolution as the input image data (28) is input; a second hidden layer (72) including a second neuron (82) to which second data having the same resolution as the image data (28) is input from the first neuron (81), and a third neuron (83) to which third data having a lower resolution is input from the first neuron (81); It is a neural network including

[0060] According to the configuration 5) above, a network that transmits image data with the same resolution as the image data (28) and a network that transmits image data with a lower resolution than the image data (28) are constructed in parallel, so that the point of action (P) can be identified with higher accuracy.

[0061] 6) The meat processing system (1) according to at least one embodiment of the present disclosure The meat action point calculation system (10) according to any one of 1) to 5) above, the robot (4) configured to impart an action to the action point (P) acquired by the action point acquisition unit (12); Equipped with.

[0062] According to the configuration 6) above, for the same reason as in 1), the burden of preparing the training data 25 is reduced, and a meat processing system 1 that can identify the point of action P with high accuracy is realized.

[0063] 7) In some embodiments, the meat processing system (1) described in 6) above, The robot (4) The apparatus includes a cutter (3) for cutting the meat (5) along an imaginary cutting line (K, M) determined by the action point data acquired by the action point acquisition unit (12).

[0064] According to the above configuration 7), the meat (5) can be cut well in accordance with the point of action data acquired by the point of action acquisition unit (12).

[0065] 8) In some embodiments, the meat processing system (1) described in 6) or 7) above, a white light source (7); The apparatus further includes an image capturing device (8) configured to capture an image of the meat (5) illuminated by the white light source (7) to generate the captured image (20).

[0066] According to the above-mentioned configuration 8), the photographing device (8) photographs the meat (5) illuminated by the white light source (7), which is more versatile than special light sources such as blue light sources, and therefore the equipment can be simplified.

[0067] 9) In some embodiments, the meat processing system (1) according to any one of 6) to 8) above, an imaging device (8) configured to photograph the meat (5) and generate the photographed image (20); a first pressing device (31) configured to press the meat (5) to a position a specified distance away from the photographing device (8); Further provided are:

[0068] If the shooting distance, which is the distance between the camera device (8) and the meat (5) during shooting, fluctuates, the appearance of the captured images (20) will vary significantly even when similar pieces of meat (5) are photographed. As a result, this can be an obstacle to learning by the learning model (70). In this regard, according to the configuration of 9) above, the shooting distance between the meat (5) and the camera device (8) during shooting is stabilized, thereby suppressing variation in the captured images (20) and enabling the learning model (70) to learn effectively. As a result, the point of action (P) can be identified with high accuracy.

[0069] 10) In some embodiments, the meat processing system (1) described in 9) above, The first pressing device (31) includes a pair of first pressing members (31A) configured to enter the photographing range of the photographing device (8) and press the meat (5) at positions avoiding the point of application (P).

[0070] According to the configuration 10), the pair of first pressing members (31A) are stably captured at specific positions in the photographed image (20), thereby limiting the area in the photographed image (20) where differences in the meat (5) appear, and thus facilitating learning by the learning model (70). As a result, the point of action (P) can be identified with high accuracy.

[0071] 11) In some embodiments, the meat processing system (1) described in 9) or 10) above, The food processor further includes a second pressing device (32) configured to press the meat (5) from the opposite side to the first pressing device (31) with a pressing force weaker than that of the first pressing device (31).

[0072] According to the configuration of 11), the meat (5) to be photographed is sandwiched between the first pressing device (31) and the second pressing device (32), so that the portion (5T) of the meat (5) to be photographed is flattened and easily captured in the photographed image (20). In addition, because the pressing force of the second pressing device (32) is weaker than that of the first pressing device (31), fluctuations in the photographing distance can be suppressed. Therefore, variations that hinder learning of the meat (5) captured in the photographed image (20) can be further suppressed, and learning by the learning model (70) can be performed well. As a result, the point of action (P) can be identified with high accuracy.

[0073] 12) At least one embodiment of the present disclosure provides a method for calculating a point of action for meat, an image data acquisition step (S11) of acquiring image data (28) representing a photographed image (20) of meat (5); an action point acquisition step (S13) of inputting the image data (28) acquired in the image data acquisition step (S11) into a learning model (70) that has been machine-learned using the image data (28) and correct answer data (27) indicating key points of the meat (5) contained in the photographed image (20) as training data (25), and acquiring action point data for identifying at least one action point (P) of the meat (5) to which the robot (4) applies an action; Equipped with.

[0074] According to the configuration 12) above, for the same reason as in 1), the burden of preparing the training data (25) is reduced, and a method for calculating the point of action for meat that can identify the point of action (P) with high accuracy is realized. [Explanation of symbols]

[0075] 1: Meat processing system 3: Cutter 4: Robot 5: Meat 5L: Left carcass 5R: Right carcass 7: White light source 8: Imaging device 9:Ribs 10: Application point calculation system 11: Image data acquisition unit 12: Point of action acquisition part 20: Photographed image 21:First area 22:Second area 23: Specified area 25: Training data 27: Correct data 28: Image data 31: First pressing device 31A: First pressing member 32: Second pressing device 70: Learning model 71: First middle class 72: Second middle class 77: Middle class 81: First neuron 82: Second neuron 83: Third neuron K, M: Virtual cutting line P: Point of action

Claims

1. an image data acquisition unit configured to acquire image data representing a captured image of the meat; an action point acquisition unit configured to input the image data acquired by the image data acquisition unit into a learning model that has been machine-learned using the image data and ground truth data that indicates a probability density distribution of key points of the meat included in the photographed image as training data, and to acquire action point data for identifying at least one action point on the meat at which the robot imparts an action; Equipped with The learning model is configured to output a probability density distribution of the action point as the action point data, which is data related to the coordinates of the action point on the meat. Point of action calculation system for meat.

2. The learning model is configured to output a heat map centered on the action point, which shows the probability density distribution. The system for calculating the action point for meat according to claim 1.

3. The photographed image includes a first area in which a left carcass of the livestock is captured and a second area in which a right carcass of the livestock is captured, The at least one action point is At least one left action point on the left carcass at which the robot acts; and at least one right action point at which the robot applies an action on the right carcass. The system for calculating the action point for meat according to claim 1 or 2.

4. the captured image includes a specified area in which a plurality of ribs of the meat are captured, The learning model is configured to output the action point data for identifying the action point avoiding the plurality of ribs. The system for calculating the point of action for meat according to any one of claims 1 to 3.

5. The learning model is a first hidden layer including a first neuron to which first data having the same resolution as the input image data is input; a second hidden layer including a second neuron to which second data having the same resolution as the image data is input from the first neuron, and a third neuron to which third data having a lower resolution is input from the first neuron; is a neural network that includes 5. The system for calculating the point of action for meat according to claim 1.

6. The meat application point calculation system according to any one of claims 1 to 5, the robot configured to impart an action to the action point acquired by the action point acquisition unit; A meat processing system comprising:

7. The robot a cutter for cutting the meat along a virtual cutting line determined by the action point data acquired by the action point acquisition unit.

7. The meat processing system of claim 6.

8. A white light source; and an imaging device configured to capture an image of the meat illuminated by the white light source to generate the captured image.

8. The meat processing system according to claim 6 or 7.

9. an imaging device configured to photograph the meat and generate the photographed image; a first pressing device configured to press the meat to a position a specified distance away from the photographing device; 9. The meat processing system of claim 6, further comprising:

10. The first pressing device includes a pair of first pressing members configured to enter a photographing range of the photographing device and press the meat at positions avoiding the point of application.

10. The meat processing system of claim 9.

11. An action point calculation system for calculating action point data for identifying at least one action point on meat to which the robot applies an action; the robot configured to impart an action to the action point specified by the action point data calculated by the action point calculation system; A meat processing system comprising: The application point calculation system includes: an image data acquisition unit configured to acquire image data representing a captured image of the meat; an action point acquisition unit configured to input the image data acquired by the image data acquisition unit into a learning model that has been machine-trained using the image data and correct answer data that indicates key points of the meat contained in the photographed image as training data, and to acquire action point data for identifying at least one action point on the meat at which the robot imparts an action; Including, an imaging device configured to photograph the meat and generate the photographed image; a first pressing device configured to press the meat to a position a specified distance away from the photographing device; The meat processor further includes a second pressing device configured to press the meat from the opposite side to the first pressing device with a pressing force weaker than that of the first pressing device. Meat processing system.

12. an image data acquisition step of acquiring image data representing a photographed image of meat; an action point acquisition step of inputting the image data acquired in the image data acquisition step into a learning model that has been machine-learned using the image data and ground truth data that indicates a probability density distribution of key points of the meat included in the photographed image as training data, and acquiring action point data for identifying at least one action point on the meat that is a target for the robot to apply an action; Equipped with In the action point acquisition step, the learning model is caused to output a probability density distribution of the action points as the action point data, which is data relating to the coordinates of the action points on the meat. Method of calculating the point of action for meat.

Images