Method and apparatus for determining a physical position of an engagement part of a leaf lard

Abstract

This disclosure relates to a computer-implemented method for determining a physical position of an engagement part of a leaf lard. The leaf lard is located on an inner surface of a left or right part of a pig carcass. The engagement part of the leaf lard is suitable for engagement by a leaf lard removal tool for at least partially separating the leaf lard from the pig carcass. The computer- implemented method comprises receiving, from an image system, data representing an image, preferably a 3D image, of at least part of the left or, respectively, right part of the pig carcass. The image does not comprise an image region that represents the engagement part. The method also comprises determining a reference image region in the image. The reference image region represents an anatomic characteristic feature of the pig carcass. Further, the method comprises determining, based on the determined reference image region, the physical position of the engagement part of the leaf lard.

Description

[0001] Method and apparatus for determining a physical position of an engagement part of a leaf lard

[0002] FIELD OF THE INVENTION

[0003] This disclosure relates to a computer-implemented method for determining a physical position of an engagement part of a leaf lard, in particular to such method wherein the physical position of the engagement part is determined on the basis of an image that does not comprise an image region representing the engagement part. This disclosure further relates to a computer, computer program and computer-readable storage medium for performing such method.

[0004] BACKGROUND

[0005] WO22106383 A1 discloses a tool for removing leaf lard from an animal carcass. Typically, for a leaf lard removal tool to effectively perform its function, it needs to engage the to-be-removed leaf lard at a certain part of the leaf lard. This part may be referred to herein as the engagement part of the leaf lard. Of course, which part of a leaf lard can be considered the engagement part, depends on the tool that is used for removing the leaf lard. For the leaf lard removal tool as disclosed in WO22106383 A1 , for example, the engagement part is a lower part of the leaf lard when the pig carcass is suspended with its head down, as also illustrated by figure 8 of WO22106383 A1 .

[0006] A complication is that every pig carcass has a different size and shape, which means that the position of the leaf lard and thus the position of the engagement part relative to the leaf lard removal tool differs for each pig carcass.

[0007] A leaf lard removal tool preferably operates automatically, so that it can repeatedly, without human intervention, remove leaf lard from pig carcasses that pass by the leaf lard removal tool while being suspended from a transport chain. For effective and automatic operation of a leaf lard removal tool, technology is desired that automatically and accurately determines the physical position of the engagement part of the leaf lard of each pig carcass part that passes by the leaf lard removal tool.

[0008] However, a further complicating factor is that the leaf lard sits on an interior surface of a pig carcass and that the leaf lard is not readily visible, even though internal organs will typically have been at least partially removed already. Part of the pig’s belly often blocks the view on the leaf lard. This is especially true if the left part and the right part of the pig carcass hang close to each other on a transport chain. Then, the left part of the pig carcass often obstructs the view on the leaf lard that sits in the right part of the pig carcass, and vice versa. In some countries, regulations prescribe that the left part and the right part of a pig’s carcass remain connected to each other until the carcass has been approved for human consumption. If that is the case, then the left and right part of the pig carcasses hang relatively close to each other on a transport chain.

[0009] In light of the above, using conventional vision systems to detect the leaf lard would require a complex camera setup. For example, several cameras may need to be installed very close to the transport chain and at an angle that allows them to capture an image of the leaf lard. In another example, one or more movable cameras may need to be used that at least partially move into and out of the pig carcass for capturing an image of the leaf lard. Hence, there is a need in the art for technology that allows to automatically and accurately detect an engagement part of a leaf lard using a relatively simple set-up.

[0010] SUMMARY

[0011] Therefore, one aspect of this disclosure relates to a computer-implemented method for determining a physical position of an engagement part of a leaf lard. The leaf lard is located on an inner surface of a left or right part of a pig carcass. The engagement part of the leaf lard is suitable for engagement by a leaf lard removal tool for at least partially separating the leaf lard from the pig carcass. The computer-implemented method comprises receiving, from an image system, data representing an image, preferably a 3D image, of at least part of the left or, respectively, right part of the pig carcass. The image does not comprise an image region that represents the engagement part. The method also comprises determining a reference image region in the image. The reference image region represents an anatomic characteristic feature of the pig carcass. Further, the method comprises determining, based on the determined reference image region, the physical position of the engagement part of the leaf lard.

[0012] This computer-implemented method is highly advantageous in that the physical position of the engagement part is determined based on an image that does not even show the engagement part itself. The image system does not need to capture an image of the engagement part itself and therefore complex camera systems are not required. Within a left part (or right part) of a pig carcass, each characteristic feature of the pig carcass has a relative position relative to the engagement part of the leaf lard that sits in that left part (or right part). Typically, for relevant characteristic features, this relative position is approximately the same for every pig carcass. Hence, once a characteristic feature with a known approximate position relative to the engagement part of the leaf lard is identified in the image, it is possible to determine the position of the engagement part itself.

[0013] The step of determining the physical position of the engagement part may be performed based on a known spatial relationship between the characteristic feature and the engagement part of the leaf lard.

[0014] The left part (or the right part) of the pig carcass may still be connected to its right (or left) counterpart, for example in that a piece of skin of the pig carcass is still intact near the head of the pig carcass, which piece of skin holds the left and right part together. Alternatively, the left part (or right part) of the pig carcass is separated completely from its right (or left) counterpart.

[0015] The engagement part is for example a lower part of the leaf lard when the pig carcass is suspended with its head down. The leaf lard removal tool is for example any of the leaf lard removal tools as disclosed in WO22106383 A1 .

[0016] The physical position of the engagement part is determined with such accuracy that the leaf lard removal tool properly engages the leaf lard when it is moved to the determined physical position and performs an engagement action at that physical position, such as suctioning and / or clamping the leaf lard. As referred to herein, a physical position may be understood as a position in the earth’s frame of reference. As referred to herein, an image region in an image may be understood to have a virtual position in the image’s frame of reference, which is preferably a three-dimensional frame of reference.

[0017] An image region typically comprises a plurality of pixels. However, an image region may also comprise one and only one pixel. If the image is a 3D image, then each pixel in the image would typically be associated with a virtual position (x’, y’, z’) within the image’s frame of reference and with one or more pixel colour values. Herein x’, y’, and z’ are coordinates indication virtual position on three respective virtual, preferably orthogonal, axes.

[0018] Typically, the step of determining the physical position of the engagement part of the leaf lard based on the determined reference image region is performed based on calibration data that link virtual positions that are relative to the reference image region to physical positions. The calibration data may be said to provide a mapping between the image’s frame of reference and the earth’s frame of reference.

[0019] The image not comprising an image region that represents the engagement part may be understood as that the physical position of the engagement part is linked to a corresponding virtual position in the image’s frame of reference, at which virtual position no pixels of the image are present.

[0020] In an embodiment, the anatomic characteristic feature of the pig carcass is an area of a rib cage of the pig carcass, or a rib of the pig carcass, or a groin of the pig carcass, or an atlas of the pig carcass, or a spine of the pig carcass, or an armpit of the pig carcass, or a front leg of the pig carcass, or a head of the pig carcass, or a nose of the pig carcass. Preferably, the anatomic characteristic feature of the pig carcass is an edge area of the rib cage, more preferably a lower edge area of the rib cage when the pig carcass is suspended with its head down.

[0021] These characteristic features are advantageous in that they can be identified effectively and reliably in the image and have a sufficiently strong spatial correlation with the engagement part of the leaf lard. However, other characteristic features may also be used without departing from the scope of the methods disclosed herein.

[0022] In an embodiment, the method comprises determining one or more further reference image regions that respectively represent one or more further anatomic characteristic features of the pig carcass. For example, the method may comprise determining a first reference image region that represents a rib of the pig carcass and a second reference image region that represents an atlas of the pig carcass. The physical position of the engagement part of the leaf lard may be determined based on all determined reference image regions. This step may then involve determining, for each determined reference image region, a particular physical position based on the reference image region in question, and then determining the average physical position of all determined particular physical positions as the physical position of the engagement part of the leaf lard.

[0023] In an embodiment, the step of determining the physical position of the engagement part comprises a step of determining a virtual position of the engagement part of the leaf lard based on and relative to the determined reference image region. In this embodiment, the method also comprises determining, based on calibration data that link virtual positions that are relative to the reference image region to physical positions, and based on the determined virtual position of the engagement part, the physical position of the engagement part of the leaf lard.

[0024] Additionally or alternatively, the method comprises determining, based on the calibration data, and based on the reference image region, a physical position of the characteristic feature and determining, based on the determined physical position of the characteristic feature, the physical position of the engagement part of the leaf lard.

[0025] This embodiment provides for a convenient method for determining the physical position of the engagement part. The virtual position relative to the determined reference image region may be understood as a position in the image’s frame of reference. It should be appreciated, however, since the image does not comprise an image region representing the engagement part, that the image does not have pixels at that virtual position.

[0026] In an embodiment, the step of determining the physical position of the engagement part comprises a step of determining, based on calibration data referred to above, and based on the reference image region, a physical position of the characteristic feature and determining, based on the determined physical position of the characteristic feature, the physical position of the engagement part of the leaf lard.

[0027] The step of determining the virtual position of the engagement part based on the reference image region and / or the step of determining the physical position of the engagement part based on the physical position of the characteristic feature may be performed based on a known spatial relationship between the characteristic feature and the engagement part of the leaf lard.

[0028] This spatial relationship may have been determined by measuring in a plurality of pig carcasses, the relative position of the leaf lard’s engagement part relative to the characteristic feature concerned. One way of performing this measurement would be to temporarily install an additional image system, in addition to the image system which captures the image of the left (or right) part of the pig carcasses, in which image the reference image region is determined. The image system and the additional image system may then each capture an image of a left or right part of a pig carcass from different angles. The image captured by the additional image system, which is preferably a 3D image, may then comprise an image region representing the engagement part of the leaf lard in the pig carcass part. Based on common features that are present in both images, the two images can be merged which results in a merged image, preferably a 3D image, that comprises an image region representing the engagement part of the leaf lard and the reference image region representing the characteristic feature in question. A user and / or a computer may then label in this merged image both the reference image region representing the characteristic feature and the image region representing the engagement part. Typically, a computer would label the reference image region, because the computer would typically have been trained already to identify the reference image region, while a user would manually label the image region representing the engagement part of the leaf lard. In any case, the labelling allows to determine the relative position of the engagement part relative to the characteristic feature. By performing such measurement for a plurality of pig carcass parts, a set of relative positions of the engagement part relative to the characteristic feature is obtained. The spatial relationship to be used for determining the virtual position of the engagement part based on the reference image region, may for example be determined as the average relative position of all relative positions in the set of relative positions referred to above.

[0029] Another way to measure the relative position of the characteristic feature relative to the engagement part of the leaf lard, although quite laborious, would be to physically examine a plurality of pig carcasses and physically measure this relative position. This would typically involve an experienced butcher cutting up the pig carcass such that an accurate measurement can be performed.

[0030] The calibration data link virtual positions that are relative to the reference image region, such as the virtual position of the engagement part of the leaf lard, to physical positions. This may be understood as that the calibration data provide a mapping between the image’s frame of reference and the earth’s frame of reference. Such calibration data may have been obtained by performing a method involving accurately determining the physical position of the image system that is used to capture the image of the left or right part of the pig carcass. If the image system itself is properly calibrated, then each pixel in the image as obtained by the image system is associated with a relative physical position relative to the image system. Then, since the physical position in the earth’s frame of reference of the image system is known, for each pixel in the image as obtained by the image system, an associated physical position in the earth’s frame of reference is known. Herewith, a mapping between the image’s frame of reference and the earth’s frame of reference is provided, which mapping may be included in the calibration data.

[0031] In an embodiment, the computer-implemented method comprises receiving displacement data indicating a displacement of the pig carcass between a first time at which the data representing the image was acquired by the image system and a second time at which the leaf lard removal tool is to remove the leaf lard from the pig carcass. In this embodiment, the method also comprises determining, based on the virtual position of the engagement part of the leaf lard and / or based on the determined physical position of the characteristic feature, and based on the calibration data, and based on the displacement data, the physical position of the engagement part of the leaf lard.

[0032] This embodiment is advantageous in that it allows to accurately determine the physical position of the engagement part even if the pig carcass part moves in the time period between the moment at which the image system acquires the data representing the image and the moment at which the leaf lard removal tool is to engage the leaf lard at its engagement part. This embodiment, for example, enables to accurately determine the engagement part of the leaf lard of a left or right part of a pig carcass that moves continuously and passes by the image system and leaf lard removal tool at a substantially constant velocity. This embodiment also enables to use a static scanner, e.g. a line scanner, that scans the left or right part of the pig carcass as it passes by the scanner. This embodiment also enables to install the image system and the leaf lard removal tool at a distance from each other, which is of course convenient. If the leaf lard removal tool and the image system would need to be positioned at roughly the same position, then the image system and leaf lard removal tool may easily hinder each other.

[0033] The left or right part of the pig carcass is typically suspended from a transport chain that is driven by a drive gear. The displacement data can for example be determined by detecting the angular displacement of the drive gear in the time period between the time at which the data representing image is acquired by the image system and the time at which the leaf lard removal tool is to engage the leaf lard. Based on the detected angular displacement, it is possible to determine the distance over which the pig carcass has moved in that time period.

[0034] In an embodiment, the step of determining the reference image region in the image is performed using a mathematical model. The mathematical model may have been constructed by receiving training data and performing a machine learning algorithm based on the training data for constructing the mathematical model. The training data comprise a plurality of training images of respective reference left or right parts of pig carcasses. The training data indicate, for each training image out of the plurality of training images, an image region in the training image in question, which image region represents the characteristic feature of the reference left or right part of the pig carcass in question.

[0035] This embodiment allows to automatically determine the reference image region in the image in a convenient manner. The training data may simply be obtained by a user labelling the characteristic feature in a plurality of images of left or right parts of pig carcasses.

[0036] Performing the machine learning algorithm, for example, comprises training a neural network.

[0037] In an embodiment, the image system comprises a line scanner and / or a 3D snapshot camera system and / or a light detection and ranging system, i.e. lidar system, and / or a stereo camera system.

[0038] These types of image systems are advantageous in that they are configured to capture data representing a 3D image.

[0039] In an embodiment, the image system is situated outside of the pig carcass. This embodiment further simplifies the setup of the image system.

[0040] In an embodiment, the method comprises controlling the image system to acquire the data representing the image.

[0041] In an embodiment, the method comprises receiving a signal from a sensor, which signal indicates that the left or right part of the pig carcass has a predefined position relative to the image system. The signal for example indicates that the left or right part of the pig carcass is approaching the image system. In this embodiment, the step of controlling the image system to acquire the data representing the image is performed based on the signal received from the sensor.

[0042] The sensor is for example a lidar.

[0043] In an embodiment, the method comprises controlling the leaf lard removal tool to move to the determined physical position and to engage the leaf lard at its engagement part for at least partially separating the leaf lard from the pig carcass.

[0044] The leaf lard removal tool may be installed on a robot that is configured to control the position and / or orientation of the leaf lard removal tool. Phrased differently, the robot is configured to adopt different states, wherein each state causes the leaf lard removal tool to have a certain position and / or orientation. Controlling the leaf lard removal tool to move to the determined physical position of the engagement part may comprise controlling this robot such that the leaf lard removal tool moves to the determined physical position of the engagement part or, phrased differently, may comprise controlling the robot to adopt a state in which it causes the leaf lard removal tool to be at the determined physical position of the engagement part. Preferably, the robot has been calibrated such that it can be accurately controlled to move the leaf lard removal tool to the determined physical position of the engagement part. Such calibration may have been performed by placing a calibration element at a known physical position (in the earth’s frame of reference) that is within reach of the robot and controlling the robot to adapt a state in which it causes the leaf lard removal tool, in particular a part of the calibration tool that is configured to engage the leaf lard, to be at the calibration element. This state, which is for example characterized by a certain posture of the robot, may then be stored in association with the physical position of the calibration element. By performing this calibration several times while the calibration element is at different physical positions, different physical positions will thus be stored in association with respective states of the robot. These physical position-robot state pairs may constitute calibration data in that they may enable to determine, for any physical position, which state the robot must adopt to cause the leaf lard removal tool to be at that physical position.

[0045] Preferably, the left or right part of the pig carcass keeps moving while the leaf lard is at least partially separated from the pig carcass. Hence, preferably, the leaf lard removal tool is controlled to move along with the pig carcass part at the same velocity, for example by controlling the robot referred to above.

[0046] One aspect of this disclosure relates to a computer comprising means for performing the computer-implemented method according to any of the preceding claims.

[0047] One aspect of this disclosure relates to a computer comprising a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform any of the computer- implemented method disclosed herein.

[0048] One aspect of this disclosure relates to a leaf lard positioning system comprising any of the image systems disclosed herein and comprising any of the computers disclosed herein having means for performing any of the methods disclosed herein that involves controlling the image system to acquire the data representing the image.

[0049] The leaf lard positioning system optionally comprises the sensor referred to above that is configured to output a signal indicative of a position of the left or right part of the pig carcass relative to the image system.

[0050] One aspect of this disclosure relates to a leaf lard removal system comprising any of the leaf lard removal tools disclosed herein and any of the computers disclosed herein having means for performing any of the methods disclosed herein that involves controlling the leaf lard removal tool to move to the determined physical position and to engage the leaf lard at its engagement part for at least partially separating the leaf lard from the pig carcass.

[0051] In an embodiment, the leaf lard removal system comprises any of the leaf lard positioning systems disclosed herein.

[0052] One aspect of this disclosure relates to a computer program or suite of computer programs comprising at least one software code portion or a computer program product storing at least one software code portion, the software code portion, when run on a computer system, being configured for executing any of the computer-implemented methods disclosed herein.

[0053] One aspect of this disclosure relates to a non-transitory computer-readable storage medium storing at least one software code portion, the software code portion, when executed or processed by a computer, is configured to perform any of the methods disclosed herein.

[0054] One aspect of this disclosure relates to a computer program comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the computer- implemented methods described herein.

[0055] One aspect of this disclosure relates to a computer-readable data carrier having stored thereon any of the computer programs described herein.

[0056] The computer-readable data carrier may be hard disk, for example, or a signal.

[0057] One aspect of this disclosure relates to a computer readable storage medium having computer readable program code embodied therewith, and a processor, preferably a microprocessor, coupled to the computer readable storage medium, wherein responsive to executing the computer readable program code, the processor is configured to perform any of the computer-implemented methods described herein.

[0058] One aspect of this disclosure relates to a data carrier signal carrying any of the computer programs described herein.

[0059] Elements and aspects discussed for or in relation with a particular embodiment may be suitably combined with elements and aspects of other embodiments, unless explicitly stated otherwise. Embodiments of the present invention will be further illustrated with reference to the attached drawings, which schematically will show embodiments according to the invention. It will be understood that the present invention is not in any way restricted to these specific embodiments.

[0060] BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Aspects of the invention will be explained in greater detail by reference to exemplary embodiments shown in the drawings, in which:

[0062] FIG. 1 schematically illustrates a leaf lard removal system according to an embodiment;

[0063] FIG. 2 illustrates a leaf lard removal tool engaging a leaf lard at its engagement part;

[0064] FIG. 3 is an image as obtained by an image system according to an embodiment;

[0065] FIG. 4 schematically illustrates a setup for determining a spatial relationship between an anatomic characteristic feature and an engagement part according to an embodiment;

[0066] FIG. 5 is a flow chart illustrating a method for determining a spatial relationship between an anatomic characteristic feature and an engagement part according to an embodiment;

[0067] FIG. 6A is an image as obtained by an image system according to an embodiment;

[0068] FIG. 6B is an image as obtained by an additional image system according to an embodiment;

[0069] FIG. 6C is a merged image obtained by merging the image of FIG. 6A with the image of FIG. 6B;

[0070] FIG. 7 is a flow chart illustrating a method for determining the physical position of the engagement part according to an embodiment; FIG. 8A is a flow chart illustrating a method according to an embodiment for determining the physical position of the engagement part which method involves determining a virtual position of the engagement part;

[0071] FIG. 8B is a flow chart illustrating a method according to an embodiment for determining the physical position of the engagement part which method involves determining the physical position of the characteristic feature;

[0072] FIG. 9 is a flow chart illustrating a method for constructing a mathematical model that may be used for determining the reference image region;

[0073] FIG. 10 schematically illustrates a computer according to an embodiment.

[0074] DETAILED DESCRIPTION OF THE DRAWINGS

[0075] In the figures, identical reference numbers indicate identical or similar elements.

[0076] Figure 1 schematically illustrates a leaf lard removal system 2 according to an embodiment. In the depicted set up, pig carcasses 8 (individually denoted as 8a, b and c) are suspended with their heads down from a transport chain 10. The pig carcasses 8 are suspended from hooks 14 (individually denoted as 14a, b and c) which are fixed to the transport chain. The pig carcasses 8 move along with the transport chain 10. Preferably, the transport chain 10 moves at a constant velocity relative to the earth’s frame of reference. In the depicted embodiment, the transport chain 10 is driven by a rotating element 12, which may be a driving gear 12 that engages the transport chain 10. Also visible in figure 1 are support belts 11 a and 11 b which may be implemented to stabilize the pig carcasses 8 to some extent.

[0077] The leaf lard removal system 2 may comprise a sensorthat is configured to measure an angular displacement of the rotating element 12 in order to monitor displacement of the transport chain 10 and thus of the pig carcasses 8 that are suspended from the transport chain 10.

[0078] Each pig carcass 8a, 8b, 8c comprises a left part 16 and a right part 18 that are still connected somewhere near the head of the pig carcass. Because of this connection, the left part 16 and right part 18 are positioned relatively close to each other, which complicates the detection of the engagement part of the leaf lard.

[0079] In figure 1 , the leaf lard removal system 2 comprises a computer 100 according to an embodiment, image system 4 and leaf lard removal tool 6. The computer 100 may be configured to perform any of the computer-implemented methods disclosed herein and is configured, in the depicted embodiment, to control the image system 4 to acquire data representing an image of at least part of the left 16 and / or right part 18 of the pig carcass 8, wherein the image does not comprise an image region that represents the engagement part of the leaf lard. The image system 4 may have a static position relative to the earth’s frame of reference and would typically be configured to capture an image of the left 16 and / or right 18 part of every suspended pig carcass 8 that passes by the image system 4. In a preferred embodiment, as shown, the image system 4 is situated outside of the pig carcasses 8. The computer 100 may control the image system 4 by sending appropriate control signals to the image system 4. Such control signals may be sent wirelessly and / or via a wired connection. In an example, the system 2 comprises a sensor 5 that is configured to detect that a pig carcass 8 approaches the image system 5 and to send a sensor signal to the computer 100. In response, the computer 100 may control the image system 4 to start operating and acquire image data.

[0080] The image system 4 may be configured to capture a 3D image of at least part of the left 16 and / or right 18 part of each pig carcass 8. The image system may comprise a line scanner and / or 3D snapshot camera system and / or a light detection and ranging system, i.e. lidar system and / or a stereo camera system. In the embodiment of figure 1 , the image system 4 is a line scanner, preferably a 3D line scanner. If a 3D line scanner is used, then the image as captured by the image system may be a so-called point cloud. Such a point cloud may be understood as a 3D image referred to herein. A 3D line scanner typically comprises a laser that is configured to project a line of laser light onto the pig carcass 8, and at least two cameras that are configured to continuously record the changing distance and shape of the laser line in three dimensions as the pig carcass 8 moves relative to the 3D line scanner.

[0081] The leaf lard removal tool 6 may be any leaf lard removal tool known in the art, for example the leaf lard removal tool as disclosed in WO22106383 A1 . In figure 1 , the leaf lard removal tool 6 is positioned on a robot 7 (see figure 1). The robot 7 is configured to position and orientate the leaf lard removal tool in a desired position and orientation. By controlling robot 7, the position and orientation of the leaf lard removal tool 6 can be controlled to move to the determined physical position of an engagement part of a leaf lard. Once the leaf lard removal tool 6 arrives at the determined physical position, it can engage the leaf lard at its engagement part and at least partially separate the leaf lard from the pig carcass, for example as described in WO22106383 A1 .

[0082] Figure 2 illustrates a leaf lard removal tool 6 as it engages a leaf lard 20 of a pig carcass at an engagement part 22 of the leaf lard 20. Figure 2 shows a right part of a pig carcass including an abdominal wall 24 with spinal column 26, ribs 28 where part of some are located behind the leaf lard 20, the cut open abdominal wall 26 and the leaf 20 are located in the abdominal wall 24. This illustration may simulate a part of a half pig carcass hanging from its hind leg i.e. with head down. A leaf lard removal tool 6 is directed towards the leaf lard 20 and with the front end of the tool 6 pointing partly to the ground. An opening 30 of a suction cavity is indicated with a dotted line to indicate that it is located in the tool 6 on the side directed towards the leaf lard 20. The opening 30 can engage the leaf lard 20 at the engagement part 22 of the leaf lard. In this example, the engagement part is the pointed end of the leaf lard 20, which is the lower part of the leaf lard 20 for carcass parts hanging from the hind legs. The tool 6 comprises a rotatable mandrel for winding the leaf lard 20 around the mandrel. As the leaf lard winds around the mandrel, the tool 6 moves upwards until a significant part of the leaf lard is wound around the mandrel. Herewith, at least part of the leaf lard is separated from the pig carcass.

[0083] Figure 3 shows a 3D image of at least part of a right part of a pig carcass. This image has been obtained by a 3D line scanner. Hence, every pixel in the image or, in other words, every point of the point cloud has a virtual position (x’, y’, z’) in the image’s frame of reference. The apostrophe denotes that these are coordinates of a virtual position, which may be understood as a position in the image’s frame of reference. To illustrate, point 32 has a virtual position in the image’s frame of reference of (x1 y1 z1 ’), point 34 has virtual position (x2’, y2’, z2’), point 36 has virtual position (x3’, y3’, z3’). Position 38 at (x4’, y4’, z4’) does not indicate a point of the point cloud, because position 38 indicates the virtual position of the engagement part of the leaf lard, which engagement part is not visible in the image. In this embodiment, it can be said that the point cloud as obtained by the 3D line scanner does not comprise points that represent the engagement part of the leaf lard.

[0084] As referred to herein, an image region may comprise one or more pixels. If the image is a point cloud, then the image region may comprise one or more points. To illustrate, point 34 may be a reference image region referred to herein as it represents an anatomic characteristic feature which is a rib of the pig carcass, in particular an end point of the fifth rib. Point 36 may also be a reference image region referred to herein as it represents an anatomic characteristic feature, namely an armpit of the pig carcass. Image region 33 comprises a plurality of pixels, in this case a plurality of points, and also represents an anatomic characteristic feature of the pig carcass, namely an edge area of the rib cage, in particular the lower edge area of the rib cage. When the virtual position 38 is determined based on and relative to a reference image region that comprises several pixels, such as reference image region 33, then this may be performed by determining the virtual position 38 based on and relative to a particular virtual position within the reference image region. For example, position 32 at (x1 ’, y1 ’, z1 ’) is the middle point within reference image region 33 and virtual position 38 may be determined based on and relative to this middle point 32.

[0085] The reference image regions that represent respective anatomic characteristic features of the pig carcass may be automatically detected by a mathematical model, for example an image segmentation model know in the art, that has been obtained using a machine learning algorithm. Typically used mathematical models for detecting features within an image are developed for 2D images. Hence, determining one or more reference image regions in a 3D image may involve converting the 3D image to a 2D image, which may be performed by simply discarding the z-values as measured by the 3D image system, and inputting the 2D image into the mathematical model. Once the one or more reference image regions have been detected in the 2D image, the 2D image can be converted back to a 3D image, which may be performed by adding the previously discarded z-value again to each pixel.

[0086] The virtual position 38 of the engagement part may be determined based on and relative to an image region by using a previously determined displacement vector. Figure 3 illustrates three displacement vectors v1 , v2, v3. Each of these displacement vectors allows to determine the virtual position of the engagement part. Displacement vector v1 = (x4’ - xT , y4’ - yT , z4’ - zT), displacement vector v2 = (x4’ - x2’ , y4’ - y2’ , z4’ - z2’), displacement vector v3 = (x4’ - x3’ , y4’ - y3’ , z4’ - z3’).

[0087] Figure 4 illustrates a setup for determining the spatial relationship between a characteristic feature of a left part 16 of pig carcass 8 and the engagement part of the left part’s leaf lard. In addition to image system 4 that is configured to capture an image of the pig carcass 8, which image does not show the engagement part itself, an additional image system 40 is installed at a different position, having a different angle relative to the transport chain. Additional image system 40 is configured to capture an image that does show the engagement part of the left part’s 16 leaf lard. Thus, the image as captured by image system 40 does comprise an image region representing the engagement part of the leaf lard.

[0088] In this example, both image system 4 and additional image system 40 are 3D line scanners. Further, both the image system 4 and additional image system 40 may be controlled by computer 100 and / or may provide image data representing their respective captured images to computer 100.

[0089] Figure 5 is a flow chart illustrating a computer-implemented method for determining the spatial relationship between a characteristic feature of the pig carcass and engagement part of the leaf lard. It should be appreciated that such spatial relationship may be indicated by a displacement vector referred to above.

[0090] Herein, step 42 comprises receiving image data from image system 4, wherein the image data represents a first image of at least part of the left (or right) part a pig carcass. This first image does not comprise an image region representing the engagement part of the leaf lard. In parallel, step 44 is performed which comprises receiving image data from image system 40, wherein the image data represents a second image of at least part of the left (or right) part of the pig carcass. The second image does comprise an image region representing the engagement part.

[0091] Figure 6A is an example of a first image obtained by image system 4. The first image comprises a reference image region 34 representing an end of the sixth rib. Figure 6B is an example a second image obtained by image system 40. This second image comprises an image region 38 representing the engagement part of the leaf lard.

[0092] Step 46 (see figure 5) comprises merging the first and second image to obtain a merged image. Figure 6C shows an example of such merged image. In this example, the merged image is a 3D image, in particular a point cloud that comprises image region 38 representing the engagement part and image region 34 representing the characteristic feature. If these image regions have been correctly identified and labelled, step 52 (figure 5) may be performed which comprises determining the spatial relationship between characteristic feature and engagement part. Note that the labeling of the image regions 34 and 38 may be performed prior to merging the two images. Step 52 may be performed by determining a displacement vector, such as displacement vector v2 indicated in figure 6c, where v2 = (dx’, dy’, dz’).

[0093] Of course, the method of figure 5 may be performed for many pig carcasses, e.g. a 100 pig carcasses, which allows to determine an average spatial relationship, for example by determining an average displacement vector.

[0094] Figure 7 is a flow chart illustrating a computer-implemented method for determining the physical position of an engagement part of a leaf lard. Herein, steps 53 and 59 are optional steps as indicated by the dashed outline.

[0095] Step 53 comprises controlling an image system to acquire image data representing an image of at least part of the left or, respectively, right part of a pig carcass. This image does not comprise an image region that represents the engagement part.

[0096] Step 54 comprises receiving the image data from the image system.

[0097] Step 56 comprises determining a reference image region in the image, wherein the reference image region represents an anatomic characteristic feature of the pig carcass. Step 58 comprises determining, based on the reference image region determined in step 56, the physical position of the engagement part of the leaf lard.

[0098] Step 59 comprises comprising controlling the leaf lard removal tool to move to the determined physical position and to engage the leaf lard at its engagement part for at least partially separating the leaf lard from the pig carcass. Controlling the leaf lard removal tool may be performed by sending appropriate control signals to the leaf lard removal tool itself and / or to a robot on which the leaf lard removal tool is mounted. After the leaf lard removal tool has engaged the leaf lard at its engagement part, it may be controlled to indeed at least partially separate the leaf lard from the pig carcass. In an embodiment, the leaf lard is substantially completely separated from the pig carcass, however, it may also be that some part of the leaf lard remains connected to the pig carcass.

[0099] Figures 8A and 8B show respective flow charts illustrating embodiment in which calibration data is used to determine the physical position of the engagement part.

[0100] Figure 8A is a flow chart illustrating a computer-implemented method according to an embodiment. This embodiment comprises a step 60 that comprises determining a virtual position of the engagement part of the leaf lard based on and relative to the determined reference image region. As explained above, input in this step may be the spatial relationship between the characteristic feature of the pig carcass as represented by the reference image region and the engagement part. This spatial relationship may have been determined using the method of figure 5.

[0101] This embodiment also comprises a step 62 that comprises determining, based on calibration data that link virtual positions that are relative to the reference image region to physical positions, and based on the determined virtual position of the engagement part, the physical position of the engagement part of the leaf lard. The calibration data may for example have been obtained by denoting the physical position (e.g. in the earth’s frame of reference) of the laser scan line that is projected by the image system 4 if the image system 4 is a line scanner. The calibration data then provide a link between virtual positions in the frame of reference of images as captured by the image system and physical positions (in the earth’s frame of reference).

[0102] Figure 8B is a flow chart illustrating a computer implemented method according to an embodiment. This embodiment comprises a step 63 that comprises determining, based on calibration data, and based on the reference image region, a physical position of the characteristic feature and determining, based on the determined physical position of the characteristic feature, the physical position of the engagement part of the leaf lard.

[0103] This embodiment also comprises step 64 that comprises determining, based on the physical position of the characteristic feature as determined in step 63, the physical position of the engagement part of the leaf lard. This step may also be performed based on the known spatial relationship between characteristic feature and engagement part and this spatial relationship may be obtained by performing the method of figure 5.

[0104] As illustrated by both figure 8A and 8B, displacement data may also play a role in the determination of the physical position. In the embodiment of figure 8A, the displacement data may be input for either step 60 or step 62. In the embodiment of figure 8B, the displacement data may be input for either step 63 or step 64. The displacement data indicate a displacement of the pig carcass between a first time at which the data representing the image was acquired by the image system and a second time at which the leaf lard removal tool is to remove the leaf lard from the pig carcass. As such, the displacement data enable to account for a movement in the time period between the time at which the image was acquired and the time at which the leaf lard removal tool engages the leaf lard.

[0105] Figure 9 is a flow chart illustrating a method according to an embodiment in which a mathematical is used for determining, in step 56, the reference image region in the image.

[0106] Figure 9 explains how this mathematical model may be constructed. In a step 65, an unlabelled image of a pig carcass part (left or right part) is received. Then, in step 66, an image region is indicated in the image, which image region represents some anatomic characteristic feature of the pig carcass part. This labeling may be performed manually, for example by a user indicating the image region in the image. The labelled image is included, in step 68, in training data. In step 70 it is checked whether there are more images to be labelled. If this is the case (“yes”), then steps 65, 66 and 68 are performed again for a next image. Steps 65, 66 and 68 are performed as long as there are still images that are to be labelled. If it is determined in step 70 that no more images are to labelled (“no”), then the accumulated training data are input into a machine learning algorithm as indicated in step 72. The result of step 72 is a mathematical model that can be used to automatically determine the reference image region representing the anatomic characteristic feature in question.

[0107] Examples of machine learning algorithms that may be used are neural network algorithms, such as convolutional neural networks, deep learning algorithms, regression models.

[0108] Figure 10 schematically illustrates a data processing 100, also referred to as a computer, according to an embodiment. The data processing system 100 may for example represent a controller as described herein.

[0109] In data processing system 100, a system bus 102 connects the different components of the data processing system 100. In particular, the system bus 102 depicted in figure 10 connects the Central Processing Unit (CPU) 104, memory elements 106, input devices 108, output devices 110 and communication devices 112 with each other so that they can exchange information. The system bus 102 may be understood to serve both as data bus, address bus and control bus known in the art.

[0110] The CPU 104 is configured to perform steps as per the instructions comprised in a computer program. To illustrate, based on such instructions, the CPU 104 may perform any of the computer- implemented methods described herein. Typically, the CPU 104 is embodied as a microprocessor, which can be implemented on a single metal-oxide-semiconductor integrated circuit chip. The CPU 104 comprises a control unit 114, an arithmetic logical unit (ALU) 116 and a plurality of registers 118.

[0111] The control unit 114 is configured to retrieve instructions from a main memory 120. Typically, the control unit 114 comprises a binary decoder to convert the retrieved instructions into timing and control signals that direct the operation of for example the ALU 116. ALU 116 is configured to perform logical operations, such as additions, subtraction, multiplication, division, and Boolean operations, that are required for carrying out the instructions. The registers 118 are small memory elements that can be read and written at relatively high speed. A register may for example store an instruction, a storage address, or any other kind of data. In addition, the CPU 104 may contain hardware caches known in the art (not shown). Preferably, the CPU has different levels of caches. These hardware caches may be understood as an intermediate state between the faster registers 119 and the slower main memory

[0112] 120.

[0113] Memory elements 106 comprise a main memory 120. The main memory 120, also referred to as primary storage in the art, has stored data that is directly accessible to the CPU 104. The CPU 104 may continuously read instructions, i.e. read computer programs, stored in the main memory 120 and execute these instructions. The main memory 120 is typically a random access memory (RAM).

[0114] Memory elements 106 further comprise so-called secondary storage 122, which may be embodied as one or more hard disk drives and / or as one or more solid state drives. Typically, this secondary storage is non-volatile. Further, the memory elements may comprise other storage devices 124, such as removable storage devices, e.g. CD, DVD, USB flash drives, floppy disks, et cetera.

[0115] Input devices 108 may be understood as devices that are used to provide information to the computer 100, in particular to the CPU 104. Non-limiting examples of input devices are a keyboard, a microphone, a joystick, a mouse, a touch sensitive screen, a drive gear referred to herein, image system 4, image system 40, sensor 5, et cetera. Output devices 110 may be understood as devices that output information out of the computer and / or as devices that are controlled by the computer. Non-limiting examples of output devices 110 are a display, a printer, a headphone, a loudspeaker, any of the image systems referred to herein, the robot referred to herein, the leaf lard removal tool referred to herein, et cetera.

[0116] Communication devices 112 may be understood as devices that allow the computer system to communicate with other computers, such as with a server computer, client computer, or any other type of remote device. Non-limiting examples of communication devices 112 include modems, cable modems, ethernet cards, Bluetooth modules, et cetera.

Claims

CLAIMS1. A computer-implemented method for determining a physical position of an engagement part of a leaf lard, the leaf lard being located on an inner surface of a left or right part of a pig carcass, the engagement part of the leaf lard being suitable for engagement by a leaf lard removal tool for at least partially separating the leaf lard from the pig carcass, the computer-implemented method comprising:- receiving, from an image system, data representing an image, preferably a 3D image, of at least part of the left or, respectively, right part of the pig carcass, the image not comprising an image region that represents the engagement part;- determining a reference image region in the image, wherein the reference image region represents an anatomic characteristic feature of the pig carcass;- determining, based on the determined reference image region, the physical position of the engagement part of the leaf lard.

2. The computer-implemented method according to claim 1 , wherein the anatomic characteristic feature of the pig carcass is an area of a rib cage of the pig carcass, preferably an edge area of the rib cage, or a rib of the pig carcass, or a groin of the pig carcass, or an atlas of the pig carcass, or a spine of the pig carcass, or an armpit of the pig carcass, or a front leg of the pig carcass, or a head of the pig carcass, or a nose of the pig carcass.

3. The computer-implemented method according to claim 1 or 2, wherein the step of determining the physical position of the engagement part comprises- determining a virtual position of the engagement part of the leaf lard based on and relative to the determined reference image region, and comprises determining, based on calibration data that link virtual positions that are relative to the reference image region to physical positions, and based on the determined virtual position of the engagement part, the physical position of the engagement part of the leaf lard; and / or- determining, based on the calibration data, and based on the reference image region, a physical position of the characteristic feature and determining, based on the determined physical position of the characteristic feature, the physical position of the engagement part of the leaf lard.

4. The computer-implemented method according to claim 3, further comprising receiving displacement data indicating a displacement of the pig carcass between a first time at which the data representing the image was acquired by the image system and a second time at which the leaf lard removal tool is to remove the leaf lard from the pig carcass, and based on the virtual position of the engagement part of the leaf lard and / or based on the determined physical position of the characteristic feature, and based on the calibration data, and based on the displacement data, determining the physical position of the engagement part of the leaf lard.

5. The computer-implemented method according to any of the preceding claims, wherein the step of determining the reference image region in the image is performed using a mathematical model that has been constructed by- receiving training data, the training data comprising a plurality of training images of respective reference left or right parts of pig carcasses, wherein the training data indicates, for each training image out of the plurality of training images, an image region in the training image in question, which image region represents the characteristics feature of the reference left or right part of the pig carcass in question, and- performing a machine learning algorithm based on the training data for constructing the mathematical model.

6. The computer-implemented method according to any of the preceding claims, wherein the image system comprises- a line scanner, and / or- a 3D snapshot camera system, and / or- a light detection and ranging system, i.e. lidar system, and / or- a stereo camera system.

7. The computer-implemented method according to any of the preceding claims, wherein the image system is situated outside of the pig carcass.

8. The computer-implemented method according to any of the preceding claims, further comprising controlling the image system to acquire the data representing the image.

9. The computer-implemented method according to any of the preceding claims, further comprising controlling the leaf lard removal tool to move to the determined physical position and to engage the leaf lard at its engagement part for at least partially separating the leaf lard from the pig carcass.

10. A computer comprising means for performing the computer-implemented method according to any of the preceding claims.11 . A leaf lard positioning system comprising the computer according to claim 10, the computer comprising means for performing the computer-implemented method according to claim 8, and the image system as defined in claim 1 .

12. A leaf lard removal system comprising the computer according to claim 10, the computer comprising means for performing the computer-implemented method according to claim 9, and the leaf lard removal tool as defined in claim 1 .

13. The leaf lard removal system according to claim 12 further comprising the leaf lard positioning system according to claim 11 .

14. A computer program or suite of computer programs comprising at least one software code portion or a computer program product storing at least one software code portion, the software code portion, when run on a computer system, being configured for executing the method according to any of claims 1 - 9.

15. A non-transitory computer-readable storage medium storing at least one software code portion, the software code portion, when executed or processed by a computer, is configured to perform the method according to any of claims 1 - 9.

Images