Milking robot and method for attaching teat cups to dairy cows by milking robot in automated milking process

Abstract

The invention relates to a method for attaching teat cups (15) in an automated milking process wherein the automated milking process is performed by a milking robot (10) arranged for sequentially attaching teat cups (15) to teats (3) of a dairy cow (1), the method comprising the following steps: e) determining an identifier of the dairy cow (1) to be milked; f) following an attachment sequence in which the four teat cups (15) are to be attached, sequentially attempting to attach the four teat cups (15) to the identified dairy cow (1); g) measuring at least one parameter characterizing the step of attaching at least one of said teat cups (15) to the respective teat (3); and h) deriving, by machine learning, at least one optimized sequence for attaching the four teat cups (15) to the identified dairy cow (1) on the basis of the at least one measured parameter. The invention also relates to a milking robot (10) and to a computer program product designed to carry out the method.

Description

TECHNICAL FIELD

[0001] The invention relates to a method for attaching teat cups in an automated milking process, wherein the automated milking process is performed by a milking robot which is arranged for attaching teat cups to the teats of a dairy animal one after the other. The invention further relates to a milking robot which is arranged for attaching teat cups to the teats of a dairy animal one after the other, comprising sensors for evaluating the attachment process, and to a computer program product for performing the method. BACKGROUND

[0002] In automated milking, robots are used for attaching teat cups of a milking apparatus to the teats of a dairy animal, in particular a cow. For this purpose, information about the spatial position of the teats is required. Typically, three-dimensional (3D) information about the position of the teats in space is determined, for example by using a stereo imaging method with two cameras or by imaging with only one camera in combination with a "time-of-flight" (TOF) measurement.

[0003] Based on the gathered information about the position of the teats, a device for placing the teat cups onto the teats can be actuated. In practice, typically a robot arm is used for this purpose, for example, the robot arm can grasp the teat cups one after the other and place them onto the teats. Alternatively, a milking robot is used, wherein a complete milking set with four teat cups is mounted on the robot arm, the teat cups can be selectively lifted in order to be attached one after the other. Typically, a predetermined order is used for the attachment process, sometimes slightly adapted to the animal due to the geometry of the udder, since two different predetermined orders are used. According to the document DE 102015 112 308 A1, the visibility of the teats in the imaging process can also be taken into account in the order.

[0004] Once all teat cups have been successfully attached, the main milking process is started, typically preceded by a cleaning step for cleaning the teats. Then, the milking is started by applying the commonly used vacuum pulsatile milking method.

[0005] A reliable teat cup attachment process is crucial for a successful milking. Delays in the attachment operation due to unnecessary multiple attachment attempts cause additional stress and irritation for the dairy animal. SUMMARY

[0006] It is therefore an object of the invention to provide a method for an automated attachment process, wherein the attachment of the teat cups can be performed with a low overall failure rate and as few attachment attempts as possible. It is a further object to describe a milking robot and a computer program product for performing such an attachment method.

[0007] This object is achieved by a method and a milking robot having the respective features of the independent claims. Advantageous embodiments and further developments are specified in the dependent claims.

[0008] The method according to the invention is characterized by the following steps:

[0009] The attachment sequence, which follows the order in which the four teat cups are to be attached, successively attempts to attach the four teat cups to the identified cow. During the attachment, at least one parameter is measured, which characterizes the attachment step of at least one of the teat cups to the respective teat. Then, based on the at least one measured parameter, at least one optimized order of attaching the four teat cups to the identified cow is derived by machine learning.

[0010] The invention relies on the finding that the optimal attachment order differs for a specific cow or, in other words, that each cow has its own optimal attachment order, which can also change over time with the physiological changes of the cow. Machine learning can be used as a computer-implemented tool to individually derive this optimal attachment order based on data measured during previous attachment processes.

[0011] In an advantageous embodiment of the method, the at least one measured parameter is a value related to the success of the attachment step. The machine learning is thus trained to derive the most successful order. If the teat cup is successfully attached to the teat, the value related to the success of the attachment step can for example be set to a first predetermined value, for example the value "1"; and if the teat cup is not successfully attached to the teat, the value can be set to a second value different from the first value, for example the value "0". Preferably, if the teat cup is not successfully attached to the teat and causes at least one other teat cup to detach, the value related to the success of the attachment step is set to a third predetermined value different from the first and second values.

[0012] In another advantageous embodiment of the method, the at least one measured parameter is related to the duration of the attachment step. In this way, the attachment order can be further optimized, in particular to speed up the milking process. Further parameters considered can be related to the lactation situation of the cow.

[0013] In another advantageous embodiment of the method, the measured parameter is determined from camera images taken during the attaching step. In order to reliably determine the success of the attaching step, optical recognition techniques are used. Since the attaching process itself uses a camera image-based control, a camera is usually available. Alternatively or additionally, the measured parameter is determined from measured pressure data of a vacuum line connected to the teat cup. During the attaching, the teat cup is connected to the vacuum line. The vacuum is responsible for fixing the teat cup in place and using it for milking in a pulsatile mode. If the pressure in the teat cup remains at a level close to atmospheric pressure during the attaching process, the attaching process is not successful. Thus, the pressure in the vacuum line also provides a reliable measure for the success of the attaching step.

[0014] In another advantageous embodiment of the method, the steps of identifying the dairy cow, attaching the teat cup and measuring at least one parameter characterizing the attaching step are repeated using different random attaching orders in a training phase, also referred to as exploration phase.

[0015] In another advantageous embodiment of the method, the steps of identifying the dairy cow, attaching the teat cup, measuring at least one parameter characterizing the attaching step and deriving at least one optimized attaching order are repeated using one of the at least one optimized orders previously determined for attaching teat cups. Accordingly, even with an already optimized order, data collected during the application of this order can be used to further train the method. Thereby, the method will advantageously continuously adapt to physiological changes of the dairy cow.

[0016] In another advantageous embodiment of the method, the machine learning is based on a reinforcement learning model. In the attaching process, the measured parameter is related to a first state describing which teat cups have been attached before the attaching step and a second state describing which teat cups are intended to be attached after the attaching step, and wherein the machine learning is based on the measured parameter and the first and second state. Thus, the attaching process shows the characteristics of a Markov Decision Process (MDP). Markov Decision Processes can be handled in machine learning by Q-learning algorithms or Deep Q-Network algorithms. Thus, these techniques are well suited for the task of optimizing the attaching order.

[0017] Preferably, for each different combination of the first and second state, the measured parameters of a plurality of attaching steps are combined and expressed as a Q-value. The optimized attaching order is selected based on the Q-values. More preferably, information about the at least one measured parameter and the first and second state is transmitted from the milking robot to a superior computing unit which determines the Q-values. In particular, if there is more than one milking robot and it is used for milking the dairy cow, training data can be gathered by each milking robot and jointly analyzed thereon to determine the Q-values.

[0018] Further, the Q-value is preferably transferred from the superior computing unit to the milking robots. Thus, the Q-value can be used by each milking robot to determine at least one optimized order to use in a subsequent attachment process.

[0019] In addition to the Q-value, a strategy for selecting an order to use in a subsequent attachment process is also transferred from the superior computing unit to the milking robots. The strategy is a kind of guideline for the milking robots how to derive an optimized order from the Q-value. It reflects the quality of the Q-value and defines e.g. a random contribution factor in the process of determining the order. The random contribution factor can help to keep the model dynamic.

[0020] The milking robot according to the invention is characterized in that it is adapted to perform an attachment process of attaching four teat cups to teats of a cow in different attachment orders, and in that it is adapted to measure at least one parameter characterizing an attachment step in which at least one teat cup is attached to a respective teat, wherein based on the at least one measured parameter, at least one optimized order for attaching the four teat cups to the identified cow is derived by machine learning. Thereby the same advantages are obtained as described in connection with the inventive method.

[0021] In an advantageous embodiment, the milking robot is further adapted to transfer the at least one measured parameter and a first state describing which teat cups have been attached before the attachment step and a second state describing which teat cups are intended to be attached after the attachment step to the superior computing unit, and to receive from the superior computing unit a Q-value combining the measured parameters of multiple attachment steps. In this way, a kind of distributed system is created with multiple milking robots that optionally contribute measured data to train the model and benefit from the jointly trained model.

[0022] The computer program product according to the invention comprises a computer program executable by at least one processor of at least one computer, which computer program product causes the at least one processor to perform the method as described above when the computer program is executed on the at least one processor. Thereby the same advantages are obtained as described in connection with the inventive method. For example, the computer program can be executed on at least one processor of a local control unit of a milking robot and / or on at least one processor of a superior computing unit. BRIEF DESCRIPTION OF DRAWINGS

[0023] The invention is explained in more detail below based on embodiments with reference to the attached drawings. The drawings show:

[0024] Figure 1 is a perspective view of a cow udder when attaching teat cups in an automated milking process;

[0025] Figure 2 is a perspective view of one embodiment of a milking robot;

[0026] Figure 3 is a more detailed illustration of the mechanical arm of the milking robot of Figure 2 ;

[0027] Figure 4 is a schematic representation of an automated milking system for performing the method according to the invention;

[0028] Figure 5 is a state diagram representing an exemplary attachment procedure;

[0029] Figure 6 is a flow chart of one part of a method for performing an attachment procedure;

[0030] Figure 7 is a flow chart of another part of a method for performing an attachment procedure;

[0031] Figure 8 is an example of a first data table used in an embodiment of the attachment procedure; and

[0032] Figure 9 is an example of a second data table used in an embodiment of the attachment procedure.

[0033] Identical or similar components in the figures are denoted by the same reference numerals. For clarity and legibility, not all elements are denoted with reference numerals in all figures. DETAILED DESCRIPTION

[0034] Directional designations such as, for example, "upper", "lower", "left", "right" refer to the representation of the respective drawing. In the case of vertical directions such as "upper" and "lower", the depicted directions generally correspond to the actual directions.

[0035] Figure 1 A udder 2 of a cow 1 is shown, on which there are four teats 3a-d. For milking the cow 1, a milking robot is used, which first attaches four teat cups 15a-d to the respective teats 3a-d, cleans the teats 3a-d in a flushing step, and then milks the teats 3a-d by applying a vacuum-pulsatile milking procedure. The teat cups 15a-d are attached in succession. Figure 1 A snapshot of the attachment procedure is depicted, in which the teat cup 15b has been attached, while the teat cups 15a, 15c and 15d have not yet been attached. In the following, if all teats 3a-3d are referred to, they are referred to as "teats 3". In the same way, if all teat cups 15a-d are referred to, they are referred to as "teat cups 15". Similarly, if a non-specific teat or teat cup is referred to, it is referred to as "teat 3" or "teat cup 15", respectively.

[0036] Figure 2A perspective view of an exemplary milking robot 10 is shown, which performs the attachment task shown in Figure 1 and the subsequent cleaning and milking processes.

[0037] For example, the milking robot 10 can be installed within a milking parlor. For milking, the cows enter the milking parlor and stand with their side facing the milking robot 10, i.e. one of the longitudinal sides of the milking parlor is constituted by the milking robot 10. In an alternative arrangement, multiple such milking robots 10 can be positioned in a circular rotary milking hall arrangement, each robot being aligned along a radial direction. The milking robot 10 then also functions as a partition to form independent milking halls.

[0038] The milking robot 10 comprises a robot arm 12, which can be displaced in a vertical direction by an arm drive unit. The arm drive unit is visible in an access opening in the housing 11 of the robot 10. This access opening is normally closed by a cover during operation. It is depicted open here for illustrative purposes only.

[0039] The robot arm 12 is movable up and down and pivotable about a first pivot axis located close to the housing 11. At the free end of the robot arm 12, a holder 13 is arranged, which can be pivoted about a second pivot axis by a drive unit (not shown here). Mounted on the holder 13 is a milking cluster carrier 14, which carries a milking cluster providing four teat cups 15.

[0040] The four teat cups 15 are connected by milk and vacuum lines 17 to a milking device (not shown here either) located in the housing of the milking robot 10.

[0041] In the exemplary embodiment shown, each teat cup 15 is mounted on the milking cluster carrier 14 by a segmented element 16. The segmented element 16 comprises a plurality of annular hollow segments, inside which a pulling member (e.g. a rope or chain) extends, which can be tensioned via an actuator. In Figure 2 all segmented elements 16 are tensioned, so that all four teat cups 16 are in the raised position shown. The pulling member of the segmented element 16 can be selectively actuated via its actuator, so that each individual teat cup 15 can be in the raised or lowered position individually. The robot arm 12 as well as the actuators for raising and / or lowering the teat cups 15 are controlled by a local control unit 19 located within the robot 10. This control unit also controls the valves of the milk and vacuum lines 17 as well as other actors and sensors used in the attachment, cleaning and milking processes.

[0042] Furthermore, an imaging sensor 18 is arranged on the holder 13. This imaging sensor can for example be a TOF camera, which is able to create a two-dimensional (2D) image with a specified number of image points (pixels), wherein in each case for an image point additionally distance information is provided by the imaging sensor 18, which is imaged by the respective pixel, of the distance between the imaging sensor 18 and the object. In this sense, the imaging sensor 18 can also be regarded as a 3D sensor, since it provides lateral and depth information. In alternative embodiments, other three-dimensional imaging techniques can be used, for example two conventional cameras arranged stereoscopically for performing triangulation. The 3D information provided by the imaging sensor 18 is preferably also analyzed locally by the control unit 19.

[0043] The arrangement and orientation of the imaging sensor 18 is chosen such that, in the appropriate orientation of the holder 13, in principle all teats of the animal to be milked can be located in the field of view of the 3D sensor 18. Thus, in addition to possible teat occlusions, the 3D sensor 18 also simultaneously provides information from which the position and orientation of all teats of the milking animal can be determined. The field of view corresponds to the field of view shown in Figure 1

[0044] Figure 4 The resulting overall system is shown in a schematic representation. According to the system shown in Figure 4 The control unit 19 of the milking robot 10 is connected to a superior computing unit 20, which is schematically represented by the cloud, according to the system shown in

[0045] This makes it possible for the milking robot 10 according to the invention to perform the attachment process in an arbitrary order, i.e. there is no fixed order in which the individual teat cups 15a-15d are attached to the respective teat 3a-3d. It also makes it possible for the milking robot 10 to find an optimized order for each individual cow by means of machine learning, as will be explained below in connection with Figures 5 to 9

[0046] Figure 5 The progress of an exemplary attachment process is first shown in a state diagram. It serves to describe the data representation of the process and the various states which the system goes through in the progress of the process.

[0047] ​​The attachment process starts from an initial state 30 in which none of the teat cups 15 is attached to the corresponding teat 3.

[0048] Each state of the process is described by a state representation 33, which is essentially a four-digit binary number. Each digit represents one of the corresponding teat cups or teats, e.g. in the order "a, b, c, d", so that the first digit represents the attachment state of the teat cup 15a or the teat 3a, the second digit represents the attachment state of the teat cup 15b or the teat 3b, the third digit represents the attachment state of the teat cup 15c or the teat 3c, and the fourth digit represents the attachment state of the teat cup 15d or the teat 3d.

[0049] The attachment state itself is binary, i.e. either the teat cup 15 is attached (represented by the binary number "1") or not (represented by the binary number "0"). Accordingly, the state representation 33 of the initial state 30 is "0000".

[0050] Starting from this initial state 30, the milking robot 10 in this example attempts to attach the teat cup 15b to the teat 3b. The attempt to attach a particular teat cup 15 is a sub-step in the attachment process and is also referred to as "action" in the following. There are generally four possible actions, each represented by the index (capital letter) of the teat cup 15a-d that the robot 10 attempts to attach. Accordingly, actions A-D can be performed, corresponding to the attempt to attach the teat cups 15a-d.

[0051] Assuming that the first action B of the attachment process is successful, then an intermediate state 31, more specifically a first intermediate state 311, is taken, which is represented by the binary number "0100" as the new state representation 33.

[0052] Each action is intended to a transition 34 in the state graph. The transition 34 either leads to the system entering a different state or causes the system to remain in the same state.

[0053] In a certain particular attachment process, a transition 34 can or can not occur. However, if the same sub-step is performed several times for the same udder, then a probability P of the transition can be derived for each sub-step. The probability P depends on the starting state (e.g. the initial state 30), the state to be reached (e.g. the first intermediate state 311), and the action related to this transition (here the action B of attaching the teat cup 15b in this example). Thus, the probability P can be expressed as P (starting state, action, state reached), or, for this particular sub-step, as P (0000, B, 0100). The probability is a number between 0 (never successful) and 1 (always successful). There is an inverse probability (1-P) that gives the probability that the process remains in the same state or finally enters another state after a certain action has been attempted. Figure 5All transitions 34 shown are connected to such probabilities that can be derived through a larger number of trials.

[0054] Since the probability P of transition 34 depends only on the state involved, the attachment process satisfies the characteristics of a Markov decision process (MDP).

[0055] In the example shown, an attempt is then made to attach another nipple cup 15, namely nipple cup 15c. Successful attachment results in another intermediate state 31, denoted here as intermediate state 312, with state 33 being "0110". If this second attachment process fails, the system can remain in the same state or enter another intermediate state 31, such as intermediate state 313, with state 33 being "0010". In this case, the associated transition 34 is partially successful because nipple cup 15c has indeed been attached, but it causes the previously attached nipple cup 15b to detach. Accordingly, the system then attempts another transition 34 to reattach nipple cup 15b, which, if successful, also results in intermediate state 312.

[0056] exist Figure 5 In the example shown, the system undergoes several transitions 34 through intermediate states 311-317 before finally ending in final state 32. Final state 32 is represented by the binary number "1111". In this final state 32, all nipple cups 15a-15d have been successfully attached to nipples 3a-3d, meaning the attachment process was successful and milking can continue. The nipples are cleaned first, and then the actual milking process begins.

[0057] According to the present invention, the order for attaching four teat cups 15a-d to each identified cow 1 is optimized using machine learning, wherein the machine learning is based on at least one value measured during the attachment process. Figure 4 In the embodiment shown, some parts of the procedure are executed by the milking robot 10, and some parts of the procedure are executed by the upper-level computing unit 20.

[0058] Figure 6 and Figure 7 A flowchart illustrating an embodiment of this innovative method is shown, wherein... Figure 6 A portion of the method performed by the milking robot 10 is shown and Figure 7 A portion of the method executed by the higher-level computing unit 20 is shown.

[0059] Figure 6 The flowchart illustrates a milking process for a single cow. The process begins in step S11, where the cow (e.g., ...)... Figure 4 The cow shown in the picture 1) enters the milking parlor.

[0060] After the cow 1 enters the milking parlor in step Sll, it is identified (e.g. by reading an RFID tag) that the cow 1 is chipped. The read information is sent to the control unit 19 of the milking robot 10 and eventually also to the farm management system, which in turn provides additional information about the cow 1.

[0061] It is assumed that the cow 1 has not used the automated milking system before or has used it with low frequency. Therefore, in this example, the milking system is first operated in training mode for this individual cow 1. It is to be noted that the distinction between training mode (also referred to as exploration mode) and production mode (also referred to as exploitation mode) is not mandatory. As will be shown below, in other embodiments, training and production can be performed simultaneously.

[0062] In a next step S12 of this training or exploitation mode, a predetermined and / or arbitrary first action of the attachment sequence is selected. By way of example, the attachment of the teat cup 15b (i.e. action B) is selected as the next step of the attachment process.

[0063] In a next step S13, the milking robot 10 is controlled to attach the teat cup 15b to the teat 3b of the cow 1, i.e. to perform action B. Then, the milking robot 10 determines whether this step was successful. There are several ways to determine whether the attachment process was successful. For example, the cameras 18 used to determine the position of the teats 3a-d during the attachment process can also be used to observe whether the attachment process was successful. Another alternative or additional way is to measure the pressure build-up in the milking system during the attachment process. During attachment, the teat cup 15b to be attached is connected to the vacuum line. The vacuum is responsible for fixing the teat cup 15b in place and is used for milking in a pulsatile mode. If the pressure in the teat cup 15b and accordingly in the associated vacuum line 17 remains around atmospheric pressure during the attachment process, the attachment process was not successful. Typically, a certain time limit is set until it is finally decided that the sub-step result of the attachment process was not successful.

[0064] The data related to the attachment action are collected by the control unit 19 of the milking robot 10 and, as an example, stored in the database 20. Figure 8The table 21 in the example in Fig. 21 shows the attachment procedure for the cow 1 identified as animal number "123". For example, the first attachment sub-step in step S13 is performed for the cow 1 identified as animal number "123". The attachment procedure starts from state "0000", which means that at the beginning of this step S13, none of the teat cups 15a-d is attached. The action to be performed is to try to attach the teat cup 15b, which is indicated with the corresponding index "B" in the "action" column. The last column in table 21 shows the result of the attachment procedure. In the shown embodiment, there are two possible options: if the procedure is successful, then the reward is "1"; and if the procedure is not successful, then the reward is "0". As an example, the reward can be calculated as "number of teat cups attached after the action" minus "number of teat cups attached before the action". Thus, if the attachment procedure is not successful and a previously attached teat cup is detached during the action, the reward can even become negative.

[0065] In alternative embodiments, an extended reward scheme can be implemented. For example, the duration of an attachment sub-step can be used as an additional reward means. Furthermore, the reward can also be calculated using the milked data.

[0066] In the next step S14, it is checked whether all four teat cups 15 are successfully attached. If not, the method continues with step S15, in which the next action to be performed is selected. This can again be done according to a predetermined scheme or can simply be chosen randomly.

[0067] The method then returns to step S13 to perform the next sub-sequence. As is apparent from the example in table 21 in Fig. 21, the next actions to be performed are "A", "C" and "D". All four sub-steps are successful, so the reward "1" is obtained. Figure 8

[0068] When all four teat cups are successfully attached, the method continues with step S16, in which the data collected according to table 21 is sent from the local control unit 19 to the superior computing unit 20. Then, after the cow 1 has left the milking parlor, the method can again start with step S11 for another cow (e.g. cow number 555 in the example of table 21 in Fig. 21 ). Figure 8

[0069] The data sent in step S16 is received by the superior computing unit 20 in step S21 (Fig. 22), and stored in a database per cow-individual in the next step S22. Figure 7

[0070] ​​​Since the attachment process is similar to a Markov decision process, the machine learning can for example be based on an algorithm called Q-learning. Based on the received measured rewards, a Q-value is calculated for each possible transition defined by a starting state and an action. The Q-table is per cow-individual and maintained by the superior computing unit 20. All data received in step S21 is used to calculate or refine the Q-value table in a subsequent step 23.

[0071] Furthermore, an epsilon value is also determined, for example such that it can be similar to the reliability of the Q-values. The epsilon value is related to a so-called epsilon policy. According to the epsilon policy, a random component is mixed in when selecting the next attachment action. The more reliable the Q-values are, the lower the epsilon value can be chosen and the less the random component will be.

[0072] Figure 9 An example of a part of the Q-table 22 is shown in table 21. Once the exploration phase is completed, the milking robot 10 can use the Q-value table 22 to determine the most successful attachment order for the recognized cows. This happens in the so-called exploitation phase. As already stated before, in embodiments the exploration phase and the exploitation phase are not necessarily separate.

[0073] By means of Figure 7 The exploitation phase is further explained by means of the flowchart in

[0074] Steps S12 and S15 are different in the exploitation phase: in step S12, the Q-table 22 and the epsilon value for the cows recognized in step S11 are retrieved from the superior computing unit 20. Then, in steps S12 and S15, the next action to be performed is not determined based on a predetermined order or purely randomly, but based on the received Q-values while following the received epsilon policy. Thus, the milking robot 10 uses the most efficient attachment order using the learned data.

[0075] As Figure 8 As shown in table 21, data about the actual success of the performed sub-steps can also be sent to the superior computing unit 20 in the exploitation phase. This data can be used to improve the Q-values and possibly to continuously adapt the epsilon value. It can alternatively be provided that no reward information is taken into account in the exploitation phase. In both cases, it can be intended to repeat the exploration phase after a certain time to take into account changes in the cow physiology. During retraining, previous data can be discarded and new data can be acquired to create a new Q-table.

[0076] The first evaluation of the method according to the invention showed that the attachment performance was improved by about 8%, which means that the failed sub-steps were reduced by 8%.

[0077] In alternative embodiments, Q-learning can be used as the machine learning algorithm and combined with a policy called "Upper Confidence Bound" (UCB) instead of the ε policy.

[0078] One difference with the ε policy algorithm is that there is no global value (like the ε value) that is related to the reliability of the Q values and that influences whether the next attachment step is chosen randomly or not. In contrast, according to the UCB algorithm, the confidence level is defined and determined individually for each action. The UCB algorithm does not perform exploration by simply choosing an arbitrary action (the probability of selection of this action is the same for all actions and can eventually even remain constant over time), but quickly changes its exploration-exploitation balance as it gathers more knowledge about the individual success rates of the actions.

[0079] The results show that the application of the UCB algorithm in the present invention leads to less exploration and a very steep learning curve. The milking robot only needs very few attachment steps to find a good working attachment sequence.

[0080] In further alternative embodiments, machine learning algorithms other than basic Q-learning can be used instead, such as "State-Action-Reward-State-Action" (SARSA) or Deep Q Networks (DQN).

[0081] List of reference signs

[0082] 1 cow

[0083] 2 udder

[0084] 3 teat

[0085] 3a, b, c, d specific teat

[0086] 10 milking robot

[0087] 11 housing

[0088] 12 carrying arm

[0089] 13 arm segment

[0090] 14 milking kit

[0091] 15 teat cup

[0092] 15a, b, c, d specific teat cup

[0093] 16 segment element

[0094] 17 milk and vacuum tube

[0095] 18 imaging sensor (camera)

[0096] 19 control unit

[0097] 20 superior computing unit

[0098] 21 table

[0099] 22 table of Q values

[0100] 30 initial state

[0101] 31 intermediate state

[0102] 311-317 specific intermediate states

[0103] 32 final state

[0104] 33 state representation

[0105] 34 transition

[0106] S11-S16 method steps (milking robot)

[0107] S21-23 method steps (superior computing unit)

Claims

1. A method for attaching teat cups (15) in an automated milking process, wherein the automated milking process is performed by a milking robot (10) arranged for attaching teat cups (15) in sequence to teats (3) of a dairy cow (1), the method comprising the steps of: a) determining an identifier of a dairy cow (1) to be milked; b) following an attachment sequence of four teat cups (15) to be attached, sequentially attempting to attach the four teat cups (15) to the identified dairy cow (1); c) measuring at least one parameter characterizing the attachment step of at least one of the teat cups (15) to a respective teat (3); and d) based on the at least one measured parameter, deriving at least one optimized sequence for attaching the four teat cups (15) to the identified dairy cow (1) by machine learning.

2. The method according to claim 1, wherein the at least one measured parameter is a value related to success of the attachment step.

3. The method according to claim 2, wherein the value related to success of the attachment step is set to a first predetermined value if the attachment of a teat cup to a teat is successful, and to a second value different from the first value if the attachment of a teat cup to a teat is not successful.

4. The method according to claim 3, wherein the value related to success of the attachment step is set to a third predetermined value different from the first and second values if the attachment of a teat cup (15) to a teat is not successful and results in detachment of at least one other teat cup (15).

5. The method according to one of claims 1 to 4, wherein the at least one measured parameter is related to duration of the attachment step.

6. The method according to one of claims 2 to 5, wherein the measured parameter is determined from camera images taken during the attachment step.

7. The method according to one of claims 2 to 6, wherein the measured parameter is determined from measured pressure data of a vacuum line (16) connected to the teat cup (15).

8. The method according to one of claims 1 to 7, wherein steps a) to c) are repeated using different random attachment sequences in a training phase.

9. The method according to one of claims 1 to 8, wherein steps a) to d) are repeated using one of the at least one optimized sequences determined in a previously performed step d) as the attachment sequence in step b).

10. The method according to one of claims 1 to 9, wherein the machine learning is based on a reinforcement learning model.

11. The method according to one of claims 1 to 10, wherein the measured parameter is related to a first state describing which teat cups have been attached before the attachment step and a second state describing which teat cups are intended to be attached after the attachment step, and wherein the machine learning is based on the measured parameter and the first and second states.

12. The method according to claim 11, wherein for each different combination of the first and second states, the measured parameters of a plurality of attachment steps are combined and expressed as a Q-value.

13. The method according to claim 12, wherein the machine learning in step d) is based on the Q-values.

14. The method according to claim 13, wherein the machine learning in step d) uses a Q-learning algorithm or a deep Q-network algorithm.

15. The method according to one of claims 11 to 14, wherein information about the at least one measured parameter and a first state and a second state is transmitted from the milking robot to a superior computing unit (20) determining the Q-values.

16. The method according to claim 15, wherein the Q-values are transmitted from the superior computing unit (20) to the milking robot (10), and wherein the milking robot (10) determines the at least one optimized sequence for use in a subsequent attachment process.

17. The method according to claim 16, wherein the strategy for selecting the sequence for use in a subsequent attachment process is also transmitted from the superior computing unit (20) to the milking robot (10).

18. A milking robot (10) arranged for attaching teat cups (15) in succession to teats (3) of a dairy cow (1), comprising sensors for assessing the attachment process, characterized in that The milking robot - is adapted to perform the attachment process in different attachment sequences, wherein four teat cups (15) are to be attached to teats (3) of a cow (1); and - is adapted to measure at least one parameter characterizing an attachment step of at least one of the teat cups (15) to a respective teat (3), wherein based on the at least one measured parameter, at least one optimized sequence for attaching the four teat cups (15) to the recognized cow (1) is derived by machine learning.

19. The milking robot (10) according to claim 18, further adapted to transmit the at least one measured parameter and a first state describing which teat cups (15) have been attached before the attachment step and a second state describing which teat cups (15) are intended to be attached after the attachment step to the superior computing unit (20), in turn the milking robot (10) receives from the superior computing unit (20) a Q-value combining the measured parameters of multiple attachment steps.

20. A computer program product comprising a computer program executable by at least one processor of at least one computer, which computer program product, when the computer program is executed on the at least one processor, enables the at least one processor to carry out the method according to any one of claims 1 to 17.

Images