Method and multi-body system for supervising a joint

Abstract

A method (500) and a multi-body system (100) comprising two body segments (101, 102, 103, 104) assembled via joints (105, 106, 107), wherein one of the body segments (104) comprises a camera (110) movable between a first position (P1) and a second position (P2). The method (500) comprises: detecting (501) a reference object (200a, 200b, 200c, 200d) with the camera (110) positioned at the first position (P1); determining (502) a reference value of the reference object (200a, 200b, 200c, 200d); requesting (503) movement of the camera (110) from the first position (P1) to the second position (P2); initiating (504) movement of the camera (110) from the first position (P1) to the second position (P2) while iteratively sampling images of the reference object (200a, 200b, 200c, 200d); determining a current object value of the reference object (200a, 200b, 200c, 200d) based on the sampled images; matching the reference value with the current object value; and determining (505) success of the camera movement based on the matching.

Description

TECHNICAL FIELD

[0001] The present document discloses a method and a multi-body system. More specifically, a method and a multi-body system for supervising functionality of a joint in a multi-body system are described. BACKGROUND

[0002] Some machines, (autonomous) vehicles and / or robots having moving parts can operate without human supervision, or at least without continuous human supervision. This can be the case especially for machines working in harsh or dusty environments, such as mines, subterranean, traffic, agricultural environments, etc.

[0003] These machines having moving parts / body segments, such as robots, can be referred to as multi-body systems. A multi-body system comprises at least two body segments assembled via a joint, wherein the two body segments are movable relative to each other via the joint.

[0004] One problem with such machines operating without continuous supervision is to know whether the machine is working properly or has failed due to some unexpected event. Another problem is to predict when to perform maintenance.

[0005] For machines operating in agricultural environments, such as milking robots, these problems can be particularly severe, as milking robots operate between living creatures, which can cause unexpected problems as the animals can hit or otherwise affect the machine.

[0006] It is desirable to develop a solution to address these problems related to multi-body systems having moving segments. SUMMARY

[0007] It is therefore an object of embodiments disclosed herein to supervise movement of a multi-body system and to detect a failure of the multi-body system.

[0008] According to a first aspect, the object is achieved by a method of a multi-body system, the multi-body system comprising two body segments assembled via a joint, wherein one of the body segments comprises a camera movable between a first position and a second position. The method comprises detecting a reference object with the camera in the first position. In addition, the method comprises determining a reference value of the reference object. The method further comprises requesting a movement of the camera from the first position to the second position. The method also comprises initiating the movement of the camera from the first position to the second position. During the movement, successive iterations of actions are performed, such as sampling an image of the reference object, determining a current object value of the reference object based on the sampled image, and matching the determined reference value with the current object value. The method further comprises determining a success of the requested movement of the camera from the first position to the second position based on the matching.

[0009] Accordingly, the functionality of the mobile section of the multi-body system is monitored and it can be determined whether the mobile section is functioning as intended or requires maintenance / human inspection. Some indication that human inspection / maintenance is required can be when the comparison result is that no movement is detected at all, or when the camera never reaches the second position due to e.g. dirt or intervening obstacles.

[0010] In a first possible implementation of the method according to the first aspect, the method additionally comprises determining a status of the multi-body system based on the determined success of the camera movement. The method can further comprise outputting the determined status.

[0011] For example in some of any embodiments, the status can be output to a user, e.g. a color code (green / yellow / red), a scale, e.g. between 1-10, an avatar of a happy / sad / terrified / angry. Thereby, the user can immediately understand the status of the multi-body system and can take appropriate measures.

[0012] The status output can also, or alternatively in embodiments where the movement time has been measured and compared to a reference value, comprise an estimated time to a near maintenance moment. Thereby, when a delay over time is detected, repair and maintenance measures can be performed.

[0013] In a second possible implementation of the method according to the first aspect, or the first implementation thereof, the method can comprise measuring a time of the camera movement from the first position to the second position. The method can further comprise comparing the measured time to a reference time limit. Furthermore, the method can also comprise triggering a maintenance measure when the measured time exceeds the reference time by a threshold limit.

[0014] By measuring the time of completing the movement to the second position, and comparing it to a reference value, a delay over time is detected, and based thereon, wear over time of (the bearing and other similar mobile components / sections) can be suspected, and repair measures, such as lubrication and / or replacement at an optimal time, can be triggered. Thereby, safety is enhanced since mechanical failure of the multi-body system due to wear can be avoided. Emergency stops and delays are avoided or at least reduced.

[0015] In a third possible implementation of the method according to the first aspect, or any implementation thereof, the method can further comprise determining a success rate of the camera movement based on the stored determined success of the camera movement. The method can further comprise comparing the determined success rate to a reference value. Furthermore, the method can also comprise triggering a maintenance measure when the determined success rate is lower than the reference value.

[0016] Thanks to the disclosed solution, the farmer / operator can diagnose and identify mechanical sections of the multi-body system that have a high operational resistance, enabling them to be replaced or subjected to repair and maintenance measures.

[0017] In a fourth possible implementation of the method according to the first aspect or any implementation thereof, the determined reference value of the reference object can comprise a reference vector between a reference point of the multi-body system and a position of the reference object, wherein the camera is located at the first position. Also, the determined current object value of the reference object can be based on the sampled image comprising an object vector between the reference point of the multi-body system and the reference object in the sampled image. The matching can be performed by a length comparison between the reference vector and the object vector.

[0018] By calculating the reference vector and the object vector, respectively, and using them for the matching, an easy to implement method is provided.

[0019] In a fifth possible implementation of the method according to the fourth implementation of the first aspect, the method can further comprise estimating an angle between the reference vector and the object vector. Also, the movement of the camera from the first position to the second position can be determined to be successful when the estimated angle between the determined reference vector and the determined object vector is approximately zero; or unsuccessful when the estimated angle between the determined reference vector and the determined object vector is different from zero except for a reference limit; or unconfirmed, otherwise.

[0020] By calculating the reference vector and the object vector, respectively, and calculating an angle between them during the movement of the subject segment and the camera, the progress of the movement can be tracked and plotted in a graph, for example, whereby the movement can be analyzed.

[0021] In a sixth possible implementation of the method according to the first aspect or any of the first, second or third implementation thereof, the determined reference value of the reference object and the current object value of the reference object comprise coordinates, thereby defining a position of the reference object.

[0022] By determining the reference object coordinates of the reference value and comparing them with the current object coordinates during the movement of the camera, a reliable method for determining whether the movement has successfully terminated is provided. The coordinates can be absolute or relative, for example, with respect to the camera position.

[0023] In a seventh possible implementation of the method according to the first aspect or any implementation thereof, a plurality of reference objects can be detected with the camera located at the first position. In addition, the plurality of reference objects can be captured in the sampled image. The matching can further comprise mapping the reference value to the current object value.

[0024] In a scenario where the multi-body system is operated in the context of milking of an animal, the animal typically comprises four teats. Automatic milking of the animal is typically facilitated by being able to detect as many of the four teats as possible, and it is an advantage to use the teats as reference objects for the camera, since fast detection of the teats will facilitate the milking.

[0025] In a ninth possible implementation of the method according to the first aspect or any previous implementation thereof, the reference object can comprise a movable body part of an animal, such as a teat of an animal.

[0026] By measuring the time consumption from the request to the termination of the movement, the deterioration of the multi-body system capabilities over time (due to wear, lack of cleaning, lubrication and / or other service measures) can easily be measured, and when the start reference time limit is reached, repair measures can be triggered, thereby enabling reliable functionality of the multi-body system.

[0027] In a ninth possible implementation of the method according to the first aspect or any implementation thereof, the reference object can comprise a movable body part of an animal, such as a teat of an animal.

[0028] According to a second aspect of the present invention, the object is achieved by a multi-body system. The multi-body system comprises two body segments assembled via a joint, wherein one of the body segments comprises a camera movable between a first position and a second position. The multi-body system comprises a control unit configured to detect a reference object via the camera positioned in the first position. The control unit is further configured to determine a reference value of the reference object. Furthermore, the control unit is additionally configured to request a movement of the camera from the first position to the second position. The control unit is configured to initiate the movement of the camera from the first position to the second position while iteratively sampling images of the reference object; determine a current object value of the reference object based on the sampled images; match the determined reference value with the current object value. Additionally, the control unit is configured to determine a success of the movement of the camera from the first position to the second position based on the matching.

[0029] Accordingly, the functionality of the body segments of the multi-body system is monitored, and it can be determined whether the body segments are functioning as intended or require maintenance / human inspection.

[0030] In a first possible implementation of the multi-body system according to the second aspect, the control unit can be further configured to determine a status of the multi-body system based on the determined success of the movement of the camera. The control unit can additionally be configured to output the determined status.

[0031] For example, in some any embodiment, the status can be output to a user, e.g. a color code (green / yellow / red), e.g. a scale between 1-10, an avatar of happy / sad. Accordingly, the user can immediately understand the status of the multi-body system, and can take appropriate measures.

[0032] The status output can further, or alternatively in embodiments where the movement time has been measured and compared to the reference value, comprise an estimated time to the moment of approaching maintenance. Accordingly, when a delay over time is detected, repair and maintenance measures can be performed.

[0033] In a third possible implementation of the multi-body system according to the second aspect or any implementation thereof, the control unit can be configured to determine a success rate of the camera movement based on the stored success of the determined camera movement. In addition, the control unit can be configured to compare the determined success rate to a reference value. The control unit can further be configured to trigger a maintenance measure when the determined success rate is below the reference value.

[0034] By measuring the time to complete the movement to the second position and comparing it to a reference value, a delay over time is detected and based thereon, a suspicion of wear over time (of the bearing and other similar moving parts / segments) can be made and a maintenance measure, such as lubrication and / or replacement at an optimal time, can be triggered. Hereby, safety is enhanced as mechanical failure of the multi-body system due to wear can be avoided. Emergency stops and delays are avoided or at least reduced.

[0035] In a third possible implementation of the multi-body system according to the second aspect or any implementation thereof, the control unit can be configured to determine a success rate of the camera movement based on the stored success of the determined camera movement. In addition, the control unit can be configured to compare the determined success rate to a reference value. The control unit can further be configured to trigger a maintenance measure when the determined success rate is below the reference value.

[0036] Thanks to the disclosed solution, the farmer / operator can diagnose and identify mechanical segments of the multi-body system having a high operational resistance, enabling replacement or maintenance measures thereof.

[0037] In a fourth possible implementation of the multi-body system according to the second aspect or any implementation thereof, the control unit can be configured to determine a reference vector between a reference point of the multi-body system and a position of a reference object, wherein the camera is located in the first position. In addition, the control unit can be configured to determine an object vector between the reference point of the multi-body system and the reference object in the sampled image. Additionally, the control unit can be configured to match the determined reference value to a current object value by a length comparison between the reference vector and the object vector.

[0038] By calculating the reference vector and the object vector, respectively, and using them for matching, an easy to implement method is provided.

[0039] In a fifth possible implementation of the multi-body system according to the second aspect or any implementation thereof, the control unit can be configured to estimate an angle between the reference vector and the object vector. In addition, the control unit can be further configured to determine that the movement of the camera from the first position to the second position is successful when the estimated angle between the determined reference vector and the determined object vector is approximately zero. The control unit can be further configured to determine that the movement of the camera from the first position to the second position is unsuccessful when the estimated angle between the determined reference vector and the determined object vector is different from zero except for a reference limit. The control unit can be further configured to determine that the movement of the camera from the first position to the second position is unconfirmed, otherwise.

[0040] By calculating the reference vector and the object vector, respectively, and calculating the angle between them during the movement of the subject segment and the camera, the progress of the movement can be tracked and, for example, plotted in a graph, whereby the movement can be analyzed.

[0041] In a sixth possible implementation of the multi-body system according to the second aspect or at least some implementations thereof, the control unit can be configured to determine a reference value in the form of coordinates, thereby defining the position of the reference object. Moreover, the control unit can be configured to determine a current object value in the form of coordinates, thereby defining the position of the reference object in the sampled image. In addition, the control unit can be additionally configured to match the determined reference value with the current object value by comparison between the coordinates of the reference object.

[0042] By determining the reference object coordinates of the reference value and comparing them with the current object coordinates during the movement of the camera, a reliable method for determining whether the movement has successfully terminated is provided. The coordinates can be absolute or relative, for example, relative to the camera position.

[0043] In a seventh possible implementation of the multi-body system according to the second aspect or any implementation thereof, the control unit can be configured to detect a plurality of reference objects with the camera in the first position. In addition, the control unit can be further configured to capture the plurality of reference objects in the sampled image. Moreover, the control unit can be configured to perform the matching by mapping the reference value with the current object value.

[0044] In a scenario where the multi-body system is operated in the context of milking of an animal, the animal typically comprises four teats. Automatic milking of an animal is typically facilitated by being able to detect as many of the four teats as possible, and why using the teats as reference objects for the camera is an advantage, since fast detection of the teats will facilitate the milking.

[0045] In an eighth possible implementation of the multibody system according to the second aspect, or any implementation thereof, the control unit may be configured to trigger maintenance measures when the measurement time period between the requested movement and the start of camera movement from the first position exceeds a threshold limit for the start-up reference time.

[0046] By measuring the time elapsed from request to termination of movement, the deterioration of multibody system capabilities over time (due to wear, lack of cleaning, lubrication, and / or other service measures) can be easily measured. When the start-up reference time limit is reached, maintenance measures can be triggered, thereby ensuring the reliable functionality of the multibody system.

[0047] In a ninth possible implementation of the multibody system according to the second aspect, or any implementation thereof, the control unit may also be configured to determine a reference value for a reference object. Furthermore, the control unit may be configured to sample an image of the reference object. The control unit may also be configured to determine a current object value for the reference object based on the sampled image. When the reference object includes a movable body part of an animal, the control unit may also be configured to match the determined reference value with the current object value.

[0048] According to a third aspect of the invention, this objective is achieved by a computer program. The computer program includes instructions that, when executed by a control unit in a multibody system according to the second aspect or any embodiment thereof, cause the control unit to perform the method according to the first aspect or any embodiment thereof.

[0049] Due to the described aspects, the success of the movement can be determined by establishing reference values ​​for the reference object before the main body segment and camera move from the first position to the second position, then continuously sampling the image of the reference object, and matching the current object value with the reference value. When the current object value corresponds to the determined reference value, it can be concluded that the movement to the second position has been successfully completed. If the current object value never approaches or corresponds to the determined reference value (at least not within the time limit), it is concluded that the movement failed and the multibody system / main body segment / camera and / or joints are malfunctioning and require maintenance.

[0050] Accordingly, the expected movement of the main body segment / camera between the first and second positions is reliably monitored and supervised, and problems or malfunctions in the multibody system are detected, even when the main body segment / camera is part of a robot or vehicle that is also in motion.

[0051] Other advantages and additional novel features will become apparent from the detailed description that follows. Attached Figure Description

[0052] Embodiments of the invention will now be described in further detail with reference to the accompanying drawings, in which:

[0053] Figure 1 FIGURE illustrates an example of a portion of a multibody system according to embodiments of the application;

[0054] Figure 2A FIGURE illustrates an example of a multibody system according to embodiments of the application;

[0055] Figure 2B FIGURE illustrates an example of a multibody system according to embodiments of the application;

[0056] Figure 2C FIGURE illustrates an example of a multibody system according to embodiments of the application;

[0057] Figures 3A-3F FIGURE illustrates an example of a segment / camera moving between two positions according to embodiments of the application;

[0058] Figure 4A FIGURE illustrates an example of a chart according to embodiments of the application;

[0059] Figure 4B FIGURE illustrates an example of a chart according to embodiments of the application;

[0060] Figure 4C FIGURE illustrates an example of a chart according to embodiments of the application;

[0061] Figures 5A-5B FIGURE illustrates a flowchart illustrating an embodiment of a method;

[0062] Figure 6 is a diagram depicting a multibody system according to embodiments of the application. DETAILED DESCRIPTION

[0063] Embodiments of the application described herein are defined as a method, a multibody system, and a computer program, which can be put into practice in embodiments described below. These embodiments can be exemplified and implemented in many different forms and are not limited to the examples set out herein; instead, the illustrative examples of the embodiments are provided so that the disclosure will be thorough and complete.

[0064] Still other objectives and features can become apparent from the following detailed description considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are designed solely for purposes of illustration and not as a definition of the limits of the embodiments disclosed herein, for which reference is to be made to the appended claims. Further, the drawings are not necessarily drawn to scale and that, unless otherwise indicated, the drawings are merely intended to conceptually illustrate the structures and procedures described herein. As such, the drawings provided are illustrative in nature only and, thus, should not be considered to be limiting of the embodiments disclosed herein.

[0065] Figure 1At least a part of a multi-body system 100 is illustrated. The multi-body system 100 comprises at least two body segments 103, 104 assembled via a joint 107. One of the body segments 104 comprises a camera 110, wherein the body segment 104 and the camera 110 are movable between a first position PI and a second position P2. The camera 110 can be, for example, a three-dimensional (3D) camera, such as a stereo camera or a range camera.

[0066] The at least two body segments 103, 104 of the multi-body system 100 can be located on other movable segments or structures of, for example, a milking robot or similar device. However, these movements are known and a compensation for these movements can be calculated when performing the method described herein.

[0067] The camera 110 can be fixedly mounted on the body segment 104 such that the body segment 104 and the camera 110 are jointly movable between the first position PI and the second position P2.

[0068] The movement between the first position PI and the second position P2 can be achieved by a mechanical hand 125, such as a pneumatic or hydraulic cylinder, an electric motor, a steam engine or other corresponding energy transfer structure, acting on the body segment 104 on which the camera 110 is mounted, thereby moving the body segment 104 / camera 110 between the first position PI and the second position P2. The respective first position PI and second position P2 of the body segment 104 / camera 110 thereby correspond to respective end positions of the mechanical hand 125.

[0069] In some embodiments, the mechanical hand 125 can be provided in a plurality of fixed positions, thereby guiding the body segment 104 to a corresponding plurality of more than two positions PI, P2.

[0070] In the illustrated embodiment, the body segment 104 and the composed camera 110 are illustrated in the first position PI and the second position P2, respectively.

[0071] The multi-body system 100 can comprise an agricultural structure with a moving segment, such as a milking robot. The camera 110 can be used to capture images of one or more teats of an animal, thereby guiding the positioning of a milking cup on the teats, thereby enabling the milking robot to perform automatic milking of the animal.

[0072] The animal can be, for example, a cow or similar female milk producing mammal. Since the animal is alive and can move freely in unpredictable ways, various problems can arise that can affect the operation of the multi-body system 100.

[0073] For example, the animals can accidentally kick, lean or step on the body segments 103, 104, thereby affecting their functionality. Also, dirt or hair of the animals can stick to the body segments 103, 104, the joints 107 or the robot arm 125, thereby affecting the movement of the body segment 104 and the camera 110. For example, the body segment 104 and the camera 110 can not be able to move at all, i.e. even when the robot arm 125 is triggered to move from the first position P1 to the second position P2. Another problem is to determine when the movement is ready, i.e. when the body segment 104 and the camera 110 have reached the second position P2. Yet another problem can be to detect wear of the joints 107 over time without human intervention and / or using specific dedicated sensors for these purposes.

[0074] When located in the first position P1, the camera 110 can not be able to detect the teats of the animal that is currently subjected to the milking equipment installation. The reason can be that another teat or other body part of the animal can block a part of the view of the camera 110, or that the animal can be strangely located within the milking position. It can then be necessary to move the body segment 104 and the camera 110 to the second position P2 to enable the camera 110 to capture images of the relevant teat or even possibly all teats.

[0075] In order to be able to make correct calculations of the position of the teats seen by the camera 110, the camera position must be known. If the body segment 104, the joints 107 or the robot arm 125 malfunctions, the calculations will be wrong, resulting in malfunctions during automatic milking.

[0076] The method provided comprises detecting one or more of the reference objects 200a, 200b, 200c, 200d when the body segment 104 and the camera 110 are located in the first position P1. The reference objects 200a, 200b, 200c, 200d can be static and comprise e.g. a part of the stable or the milking equipment, or dynamic, i.e. movable, and comprise a body part of the animal, such as a teat, as Figure 2A is schematically shown in Fig. 2B.

[0077] Additionally, a reference value is determined for the reference object 200a, 200b, 200c, 200d. Then a movement of the subject section 104 and the camera 110 from the first position P1 to the second position P2 is determined. Then the movement of the subject section 104 and the camera 110 is initiated to move from the first position P1 to the second position P2 while iteratively sampling images of the reference object 200a, 200b, 200c, 200d; determining a current object value for the reference object 200a, 200b, 200c, 200d based on the sampled images; and matching the determined reference value with the current object value. Based on the matching, a success of the movement of the subject section 104 and the camera 110 from the first position P1 to the second position P2 is determined.

[0078] The reference value can for example comprise an estimated value related to the reference object as would be perceived by the camera 110 when the subject section 104 / camera 110 has successfully moved to the second position P2.

[0079] In other embodiments, the reference value can comprise a difference between an estimated value of how the reference object as perceived by the camera 110 when the subject section 104 / camera 110 is still in the first position P1 and the reference object as would be perceived by the camera 110 when the subject section 104 / camera 110 is in the second position P2.

[0080] According to some embodiments of the disclosed solution, the nipple position(s) captured by the camera 110 before the subject section 104 / camera 110 is moved from the first position P1 to the second position P2 is used as a reference for real-time monitoring of actual changes. This is discussed and schematically illustrated in Figures 3A-3F

[0081] The robot / multibody system 100 can normally move during the position change, so an algorithm can be designed to handle the robot movement. The animal can also move during the change, and there can be other sources of uncertainty, so the algorithm can be designed to give the following results: confirm the success of the movement from the first position P1 to the second position P2, determine that the movement was not successful, and determine that the success of the movement cannot be confirmed at all. Additionally, statistical analysis can be applied in some embodiments to further reduce false positive confirmations, thereby increasing the reliability of the disclosed method.

[0082] ​Further, in some embodiments, the provided solution can measure the time from the requested movement of the first position P1 to the second position P2 until the actual movement starts and the time when the actual movement is successful. The frame rate of the camera 110 can be, for example, 50 milliseconds, or about 50 milliseconds, between each sample. However, a denser frame rate between each sample, for example 10 milliseconds, can result in an increased measurement accuracy, while a sparser frame rate between each sample, for example 200 milliseconds, can result in a reduced hardware cost, as a camera with a smaller capacity can be used. Measurements can also be taken during the position change, to detect, for example, if the body segment 104 with the camera 110 is stuck in the middle between the positions P1, P2.

[0083] Figure 2A A general overview of examples of environments to which the presented solutions can be applied is presented, Figure 2B Details of an example of a milking robot 101 are presented, while Figure 2C An example of a milking robot 101 is disclosed, wherein milking equipment is attached to the body segment of the milking robot 101. The functionality of the invention is disclosed in Figures 3A-3F and described in the corresponding parts of the description.

[0084] Figure 2A A scenario in an agricultural environment, such as a barn, a milking parlor, a stable, etc., is illustrated. The agricultural environment comprises a multi-body system 100. The multi-body system 100 in the illustrated embodiment comprises an agricultural structure with a moving segment, like a milking robot 101. The milking robot 101 comprises at least two body segments assembled via joints, which are movable relative to each other via the joints, for example by operation of a manipulator 125 on one of the body segments, thereby placing the body segments in at least two defined positions relative to each other, i.e. a first position P1 and a second position P2.

[0085] The purpose of the disclosed solution is to monitor the success of the intended movement of the two body segments from the first position P1 to the second position P2 via the joints.

[0086] The milking robot 101 is only seen as an example of an agricultural structure / multi-body system 100. The agricultural structure with a moving segment can be, for example, a mixer truck with a rotating auger, a milking parlor with a rotating platform, or a fence with a gate, for example. In different embodiments, the agricultural structure can be stationary or mobile.

[0087] In the depicted scenario, the milking robot 101 is operating on an animal 105.

[0088] The multi-body system 100 further includes a control unit 140 for controlling the operation of the milking robot 101. The control unit 140 may be included within the milking robot 101 or outside it but connected via a wired or wireless communication interface. In the illustrated embodiment, the milking robot 101 and the control unit 140 communicate wirelessly via respective transceivers 120, 130.

[0089] The transmitted radio signals of the transceivers 120, 130 may be based on various different technologies in different embodiments, such as ultra-wideband (UWB), Bluetooth, WiFi, etc.

[0090] The multi-body system 100 may also include a database 150 connected to or associated with the control unit 140. Various data associated with the movement of the milking robot 101 and / or the body section may be stored in the database 150, possibly associated with identity references and / or timestamps. Moreover, one or more threshold limits may be stored in the database 150, which will be discussed further later in this article.

[0091] When the movement of the body section is unsuccessful, an alert may be sent to a predefined electronic device 160 of an administrator of the milking robot 101, such as a farmer service provider, or a similar person working in or temporarily visiting the agricultural environment.

[0092] The electronic device 160 may include, for example, a cellular mobile phone, a fixed or portable computing device, a computer tablet, a pair of smart glasses, smart contact lenses, an augmented reality device, a smart watch, or a similar device having a user interface and wireless communication capabilities.

[0093] The alert may be output as audio information, visual information, and / or tactile information on the electronic device 160 to notify the farmer about the status of the multi-body system 100.

[0094] The moving body section may be the last movable link of the milking robot 101, sometimes referred to as the end effector. The moving body section may include sensors such as a camera 110 for obtaining, for example, a clear view of the nipples of the animal 105 during the milking operation. For example, when the field of view of the camera 110 is restricted when the moving body section 104 / camera 110 is in the first position P1, the body section 104 / camera 110 may be instructed to move to the second position P2 to obtain a new camera view of the nipples from the second defined position P2. Accordingly, the camera 110 can clearly see the rear nipples of the animal 105. However, various problems may occur, resulting in the body section 104 / camera 110 getting stuck and unable to reach the second position P2, joint deterioration, etc. The body section 104 may be affected, for example, by the animal 105, such as being kicked or otherwise damaged.

[0095] Figure 2BA non-limiting example of a milking robot 101 is presented, comprising three body segments 102, 103, 104 assembled via respective joints 105, 106, 107. The last movable body segment 104 comprises a camera 110.

[0096] Some or all of these different body segments 102, 103, 104 are movable relative to each other. Also, the milking robot 101 as a whole is movable relative to the stable. It can not be necessary to mention that the animal 105 or parts of its body, like the tail, are movable relative to the stable. When performing the calculations for determining the teat positions, the known movements of the milking robot 101 and / or its body segments 102, 103, 104 can be compensated for. The movements of the animal 105 during the movement of the camera between the two positions P1 and P2 are unknown and can be compensated for by applying an appropriate (relatively large) error margin, e.g. about 5-10% of the measured values.

[0097] Figure 2C An alternative embodiment of a multi-body system 100 is illustrated, comprising a milking robot 101 having a plurality of body segments 103, 104 assembled via respective joints 107. The multi-body system 100 can be similar to the previously discussed multi-body system 100, with the difference that a milking equipment 170 is mounted on the milking robot, e.g. on the body segment 104, while being movable relative to the camera 110 which is fixedly mounted on the body segment 103. The teat cup of the milking equipment 170 has a known position on the body segment 104. Hence, the teat cup / plurality of teat cups can be used as a reference object 170.

[0098] It is noted that the position of the camera is known.

[0099] Figures 3A-3F The relative views of the reference objects 200a, 200b, 200c, 200d as perceivable by the camera 110 when the body segment 104 and the camera 110 mounted thereon are successfully moved from a first position P1 to a second position P2 are schematically illustrated in one embodiment.

[0100] When it is determined that the movement is started, the body segment 104 and the camera 110 can be located in the first position P1, as Figure 3A illustrated. An image of the reference object 200a can be captured.

[0101] In addition, the future position of the reference object 200a as it will be perceived by the camera 110 when located in the second position P2 can be calculated. Also, as Figure 3BA reference vector 310 between the reference point 300 of the body segment 104, e.g. the joint 107, and the calculated future position of the reference object 200a is illustrated. In some embodiments, the reference vector 310 can be stored in memory and held as a reference until the body segment 104 and the camera 110 are moved to the second position P2 is successfully terminated.

[0102] Images can then be captured continuously during the movement of the body segment 104 and the camera 110 to the second position P2. Figure 3C The calculation of an object vector 320 between the reference point 300 of the body segment 104 and the reference object 200a as captured by the camera 110 is illustrated. Also, in some embodiments, the object vector 320 can be compared to the reference vector 310 and a difference a between them can be calculated. In the illustrated example, the difference a is initially about 18 degrees. Figure 3C In the illustrated example, the body segment 104 and the camera 110 are still at the first position P1, while Figure 3D The illustrated example shows how the difference a between the captured position of the reference object 200a and the expected position of the reference object 200a as perceived by the camera 110 decreases as the camera 110 approaches the second position P2.

[0103] In the illustrated example, the body segment 104 and the camera 110 have approached P2, resulting in a small difference a between the captured position of the reference object 200a and the expected position of the reference object 200a. Figure 3E

[0104] Figure 3F The illustrated example shows when the body segment 104 and the camera 110 have successfully reached the second position P2. In this successful example, the difference a between the captured position of the reference object 200a and the expected position of the reference object 200a does not exist, and it can be concluded that the movement to the second position P2 has been successful and that the body segment 103, 104 and the joint 107 are working properly.

[0105] Figure 4A An example of a graph resulting from the described method being performed in an embodiment is illustrated. The vertical axis indicates the size of the difference a between the captured position of the reference object 200a and the expected position of the reference object 200a. The horizontal axis indicates the time elapsed since determining that a change of position from the first position P1 to the second position P2 is to be initiated.

[0106] In the illustrated example, the difference a, here measured as the angle between the reference vector 310 and the object vector 320, increases to about 18 degrees when the body segment 104 and the camera 110 are at the first position P1, and then decreases approximately linearly to zero when the body segment 104 and the camera 110 reach the second position P2. The vertical black line illustrates when a conclusion can be drawn that the movement into the second position P2 is successfully completed.​

[0107] Figure 4B Graphically illustrates a scenario in which the body segment 104 and the camera 110 are stuck in the first position P1 and do not move significantly. The difference a or expected angle error is detected, but never returns to about 0. The reason is that the body segment 104 and the camera 110 do not move significantly from the first position P1 and never reach the second position P2, at least not within a predetermined or configurable time limit, illustrated by the vertical black line after about 750 milliseconds in the graph of Figure 4B .

[0108] Figure 4C Graphically illustrates a scenario in which the body segment 104 and the camera 110 have started to move away from the first position P1 but seem to get stuck halfway. The break in the line in the graph is due to one or several images captured by the camera 110 failing to be used. Again, the vertical black line after about 750 milliseconds can illustrate a predetermined or configurable time limit.

[0109] Figure 5A and 5B illustrates an example of a method 500 according to an embodiment. Figure 5A and 5B The flowchart in and illustrates a method 500 in relation to a multi-body system 100. The multi-body system 100 comprises at least two body segments 101, 102, 103, 104 assembled via joints 105, 106, 107, wherein one 104 of the body segments comprises a camera 110 movable between a first position P1 and a second position P2.

[0110] The movement can be performed by a robot arm 125 acting on the body segment 104 comprising the camera 110 and moving from a first end position to a second end position, such that the camera 110 in an ideal case moves between the first position P1 and the second position P2 without hindrance.

[0111] The multi-body system 100 can be any kind of mechanical structure comprising body segments 101, 102, 103, 104 movable in relation to each other. In some embodiments, the multi-body system 100 can be specifically arranged to operate in an agricultural environment, and for example represented by any one of: a robot with a moving arm, such as a milking robot, a mixer truck with a rotating auger, a milking parlour with a rotating platform, a gate with a gate, or similar.

[0112] However, the examples of the multi-body system 100 in an agricultural environment provided herein are to be seen as illustrative and not limiting examples. In other embodiments, the multi-body system 100 can operate in for example an industrial environment, a construction environment, or a mining environment.

[0113] To correctly trigger the incident avoidance measure, the method 500 can comprise a number of steps 501-513. However, some of these steps 501-513 can only be performed in some alternative embodiments, such as steps 503 and / or 506-513. In addition, the described steps 501-513 can be performed in a slightly different chronological order than suggested by the numbering in some embodiments. The method 500 can comprise the following steps:

[0114] The step 501 comprises detecting, with the camera 110, a reference object 200a, 200b, 200c, 200d when the camera 110 and the body section 104 in which the camera 110 is fixedly mounted is in the first position P1.

[0115] In some embodiments, the reference object 200a, 200b, 200c, 200d can be detected 501 with the camera 110 when the plurality of reference objects are in the first position P1.

[0116] In some embodiments, the reference object can comprise a movable body part of an animal, such as one or several teats of an animal.

[0117] The reference object 200a, 200b, 200c, 200d can in some embodiments be a static object, such as a milking equipment 170, or be part of an automatic milking station (AMS).

[0118] The step 502 comprises determining a reference value of the reference object 200a, 200b, 200c, 200d.

[0119] The determined 502 reference value of the reference object 200a, 200b, 200c, 200d can comprise a reference vector 310 between a reference point 300 of the multi-body system 100 and a position of the reference object 200a, 200b, 200c, 200d when the camera 110 / body section 104 is in the first position P1; or when the camera 110 / body section 104 is in the second position P2.

[0120] The current object value of the reference object 200a, 200b, 200c, 200d determined based on the sampled image can comprise an object vector 320 between the reference point 300 of the multi-body system 100 and the reference object 200a, 200b, 200c, 200d in the sampled image.

[0121] In some embodiments, where the reference value comprises the reference vector 310 and the current object value comprises the object vector 320, the step of determining the reference value can comprise estimating a difference or an angle a between the reference vector 310 and the object vector 320.

[0122] The reference values determined 502 of the reference objects 200a, 200b, 200c, 200d and the current object values of the reference objects 200a, 200b, 200c, 200d can comprise respective coordinates, defining the position of the reference objects 200a, 200b, 200c, 200d, e.g. absolute or relative coordinates. In different embodiments, the coordinates or positions of the reference objects 200a, 200b, 200c, 200d can be determined by a positioning algorithm based on trilateration, multilateration or triangulation.

[0123] The step 503 comprises requesting a movement of the camera 110 and the body section 104 on which the camera 110 is fixedly mounted from the first position P1 to the second position P2.

[0124] The step 504 comprises initiating a movement of the camera 110 and the body section 104 on which the camera 110 is fixedly mounted from the first position P1 to the second position P2 while iteratively sampling images of the reference objects 200a, 200b, 200c, 200d; determining current object values of the reference objects 200a, 200b, 200c, 200d based on the sampled images; and matching the reference values determined 502 with the current object values.

[0125] The movement of the body section 104 / camera 110 can be performed between the two extreme positions by initiation of the mechanical hand 125.

[0126] In some embodiments, a plurality of reference objects 200a, 200b, 200c, 200d can be captured in the sampled images, and the matching can comprise mapping the reference values with the current object values of the plurality of reference objects 200a, 200b, 200c, 200d.

[0127] In some embodiments, where the reference values comprise reference vectors 310 and the current object values comprise object vectors 320, the matching of the reference values determined 502 with the current object values can then be performed by a length comparison between the reference vectors 310 and the object vectors 320 in some embodiments.

[0128] In other embodiments, where an estimate of the angle a between the reference vectors 310 and the object vectors 320 has been estimated, the matching can be performed by comparing the angle a with zero, possibly except for a threshold limit, e.g. a difference of 5%, a difference of 10%, etc. The threshold limit can be set to 0%, perhaps especially in embodiments where the reference objects 200a, 200b, 200c, 200d are static.

[0129] The advantage of applying threshold limits when performing the matching is that when the reference object 200a, 200b, 200c, 200d comprises a body part of the animal 105, such as a teat, a simultaneous movement of the animal 105 can occur during the movement of the camera 110 from the first position Pi to the second position P2. Thus, even if the movement to the second position P2 has been successfully performed, the matching can be slightly off.

[0130] It can be noted that the body segment 104 / camera 110 never reaches the second position P2; or can even not leave the first position Pi.

[0131] The step 505 comprises determining the success of the movement of the body segment 104 / camera 110 from the first position Pi to the second position P2 based on the matching of the determined 502 reference value and the current object value.

[0132] In some embodiments, in which the reference value comprises a reference vector 310 and the current object value comprises an object vector 320, the movement of the body segment 104 / camera 110 from the first position Pi to the second position P2 can be determined 505 to be successful when the estimated angle a between the reference vector 310 of the determination 502 and the determined object vector 320 is approximately zero; or unsuccessful when the estimated angle a between the reference vector 310 of the determination 502 and the determined object vector 320 is different from zero except for a reference limit; or unconfirmed, otherwise.

[0133] In this alternative embodiment, the reference vector 310 has been determined 502 when the body segment 104 / camera 110 is located in the first position Pi to be the position of the reference object 200a as it would be perceived when the body segment 104 / camera 110 has successfully moved to the second position P2.

[0134] However, in other embodiments, the situation can be the opposite. The success of the movement of the body segment 104 / camera 110 from the first position Pi to the second position P2 can be determined 505 to be successful when the estimated angle a between the reference vector 310 of the determination 502 and the determined object vector 320 is approximately a predetermined value; or unsuccessful when the estimated angle a between the reference vector 310 of the determination 502 and the determined object vector 320 is different from the predetermined value except for a reference limit; or unconfirmed, otherwise.

[0135] In this alternative embodiment, the reference vector 310 has been determined 502 as the position of the reference object 200a as perceived when the body segment 104 / camera 110 is in the first position Pl, when the body segment 104 / camera 110 is in the first position Pl. An estimate of the angle a between the reference vector 310 determined 502 and the future position of the reference object 200a has then been calculated, as will be perceived when the body segment 104 / camera 110 has successfully moved to the second position P2, here referred to as a predetermined value, which is then used for comparison.

[0136] Accordingly, the functionality of the moving segments 103, 104 of the multi-body system 100 is hereby monitored, and it can be determined whether the moving segments are operating as intended or require maintenance / human inspection.

[0137] Step 506, which can only be performed in some embodiments, comprises measuring the time of the camera movement of the body segment 104 / camera 110 from the first position Pl to the second position P2.

[0138] Step 507, which can only be performed in some embodiments in which step 506 has been performed, comprises comparing the measured 506 time with a reference time limit.

[0139] Step 508, which can only be performed in some embodiments in which step 507 has been performed, comprises triggering a maintenance measure when the time measured 506 exceeds the reference time limit by a threshold limit, according to the comparison 507.

[0140] In some embodiments, a maintenance measure can be triggered 508 when the measured time period between the requested 503 movement and the movement 504 of the body segment 104 / camera 110 from the first position Pl exceeds a start reference time limit by a threshold limit.

[0141] Step 509, which can only be performed in some embodiments, comprises determining a success rate of the camera movement based on the stored success of the camera movement determined 505.

[0142] Step 510, which can only be performed in some embodiments in which step 509 has been performed, comprises comparing the success rate determined 509 with a reference value.

[0143] Step 511, which can only be performed in some embodiments in which step 510 has been performed, comprises triggering a maintenance measure when the success rate determined 509 is lower than the reference value, according to the comparison 510.

[0144] Step 512, which can only be performed in some embodiments, comprises determining a status of the multi-body system 100 based on the success of the camera movement determined 505.

[0145] The status can be outputted and presented to a user, e.g. in the form of a rating, e.g. 0-10 or 1-100 (non-limiting any examples), a color such as green (status: OK), red (status: malfunctioning) and / or yellow (status: some abnormal situation occurred during execution of the method 500 and the status cannot be determined). Other types of status outputting can be outputting avatars of happy / sad / panicking, e.g. letters such as AF.

[0146] The status outputting can also, or alternatively in embodiments where the movement time has been measured and compared to a reference value, comprise an estimated time to a moment of approaching maintenance. Hereby, when a delay over time is detected, repair and maintenance measures can be performed.

[0147] Step 513, which can only be performed in embodiments where step 512 has been performed, comprises outputting the status determined 512.

[0148] In some embodiments where the status determined 512 is below a threshold limit, an alarm can be issued to the electronic device 160 of the farmer. Also, or alternatively, the movement of the body segment 104, and thereby the camera 110; or the movement of the entire milking robot, can be deactivated and / or prohibited. Hereby, severe accidents, e.g. injuries of animals 105 close to the multi-body system 100, can be avoided, thereby reducing animal suffering and promoting animal happiness on the farm. Also, a safer working environment is created and consequences of accidents can be avoided or at least reduced.

[0149] Figure 6 An embodiment of a multi-body system 100 is illustrated. The multi-body system 100 comprises two body segments 101, 102, 103, 104 assembled via joints 105, 106, 107, where one of the body segments 104 comprises a camera 110 mounted thereon. Hereby, the camera 110 is fixedly mounted on the body segment 104 such that they move together. The body segment 104 / camera 110 is movable between a first position PI and a second position P2.

[0150] The multi-body system 100 can perform at least some of the previously described steps 501-513 according to the above described and Figure 5A and 5B The method 500 illustrated in Fig. 5 can be used to perform at least some of the previously described steps 501-513 to determine the success of the movement of the body segment 104 / camera 110 from the first position PI to the second position P2.

[0151] The multi-body system 100 comprises a control unit 140 for controlling movement of the body section 104 comprising the camera 110. The control unit 140 is configured to detect, via the camera 110, the reference object 200a, 200b, 200c, 200d when the body section 104 / camera 110 is in the first position PI. In addition, the control unit 140 is configured to determine a reference value of the reference object 200a, 200b, 200c, 200d. The control unit 140 is further configured to request movement of the body section 104 / camera 110 from the first position PI to the second position P2. The control unit 140 is furthermore configured to initiate movement of the body section 104 / camera 110 from the first position PI to the second position P2 while iteratively sampling images of the reference object 200a, 200b, 200c, 200d; determining a current object value of the reference object 200a, 200b, 200c, 200d based on the sampled images; and matching the determined reference value with the current object value. In addition, the control unit 140 is configured to determine success of the movement of the body section 104 / camera 110 from the first position PI to the second position P2 based on the matching.

[0152] In some embodiments, the control unit 140 can be configured to determine a status of the multi-body system 100 based on the determined success of the camera movement. The control unit 140 can be further configured to output the determined status of the multi-body system 100, e.g. via the electronic device 160 of the associated farmer.

[0153] In yet some embodiments, the control unit 140 can be configured to measure a time of the camera movement from the first position PI to the second position P2. Also, the control unit 140 can be configured to compare the measured time with a reference time limit; and trigger a maintenance measure when the measured time exceeds the reference time limit by a threshold limit.

[0154] Also, or additionally, the control unit 140 can be configured to determine a success rate of the camera movement based on the stored determined success of the camera movement; compare the determined success rate with a reference value; and in some embodiments, trigger a maintenance measure when the determined success rate is below the reference value.

[0155] The control unit 140 can additionally be configured to determine a reference vector 310 between the reference point 300 of the multi-body system 100 and the position of the reference object 200a, 200b, 200c, 200d of the camera 110 when the body segment 104 / camera 110 is in the first position P1. Moreover, the control unit 140 can be configured to determine an object vector 320 between the reference point 300 of the multi-body system 100 and the reference object 200a, 200b, 200c, 200d in the sampled image. The control unit 140 can further be configured to match the determined reference value with the current object value by a length comparison between the reference vector 310 and the object vector 320.

[0156] In some embodiments, the control unit 140 can be configured to estimate an angle a between the reference vector 310 and the object vector 320. Moreover, the control unit 140 can be configured to determine that the movement of the body segment 104 / camera 110 from the first position P1 to the second position P2 is successful when the estimated angle a between the determined reference vector 310 and the determined object vector 320 is approximately zero. Additionally, the control unit 140 can be configured to determine that the movement is unsuccessful when the estimated angle a between the determined reference vector 310 and the determined object vector 320 is different from zero except for a reference limit. The control unit 140 can further be configured to determine that the movement is not confirmed, otherwise, for example, in case of insufficient data or images that can capture the reference object 200a, 200b, 200c, 200d.

[0157] In some alternative embodiments, the control unit 140 can be configured to determine a reference value in the form of coordinates, thereby defining the position of the reference object 200a, 200b, 200c, 200d. Moreover, the control unit 140 can be configured to determine a current object value in the form of coordinates, thereby defining the position of the reference object 200a, 200b, 200c, 200d in the sampled image; and match the determined reference value with the current object value by comparing the coordinates of the reference object 200a, 200b, 200c, 200d.

[0158] The control unit 140 can further be configured to detect a plurality of reference objects 200a, 200b, 200c, 200d with the camera 110 in the first position P1. Moreover, the control unit 140 can be configured to capture the plurality of reference objects 200a, 200b, 200c, 200d in the sampled image. Additionally, the control unit 140 can be configured to map the reference value with the current object value.

[0159] In some embodiments, the control unit 140 can be configured to trigger a maintenance measure when a measured time period between the requested movement and the movement of the body segment 104 / camera 110 from the first position P1 exceeds a threshold limit of an initiation reference time limit.

[0160] The control unit 140 of the multi-body system 100 can be configured to determine a reference value of the reference object 200a, 200b, 200c, 200d. Also, the control unit 140 can be configured to sample an image of the reference object 200a, 200b, 200c, 200d. The control unit 140 can additionally be configured to determine a current object value of the reference object 200a, 200b, 200c, 200d based on the sampled image. Furthermore, the control unit 140 can be further configured to match the determined reference value with the current object value when the reference object 200a, 200b, 200c, 200d comprises a movable section of the animal 105.

[0161] The control unit 140 can comprise a receiver 610 configured for receiving information from the camera 110 via a wireless signal receiver and / or transceiver.

[0162] The control unit 140 comprises processing circuitry 620 configured for performing various calculations and operations to perform the method 500 according to at least some of the aforementioned steps 501-513, to determine the success of the movement of the subject section 104 / camera 110 from the first position PI to the second position P2.

[0163] Such processing circuitry 620 can comprise one or more instances of processing circuitry, i.e. a Central Processing Unit (CPU), a processing unit, a processing circuit, a processor, a dedicated integrated circuit (ASIC), a microprocessor, or other processing logic that can interpret and execute instructions. Thus, as used herein the expression “processing circuitry” can comprise multiple processing circuits, e.g. any, some or all of the processing circuitry listed above.

[0164] Further, in some embodiments, the control unit 140 can comprise a memory 625. The optional memory 625 can comprise a physical device for storing data or a program, i.e. a sequence of instructions, on a temporary or permanent basis. According to some embodiments, the memory 625 can comprise an integrated circuit comprising silicon-based transistors. The memory 625 can comprise, for example, a memory card, a flash memory, a USB memory, a hard disk, or another similar volatile or non-volatile storage unit for storing data, such as a ROM (Read-Only Memory), a PROM (Programmable-Read-Only Memory), an EPROM (Erasable PROM), an EEPROM (Electrically Erasable PROM), etc. in different embodiments.

[0165] Further, the control unit 140 can comprise a signal transmitter 630. The signal transmitter 630 can be configured for transmitting signals to the camera 110, the multi-body system 100, the electronic device 160, and / or the database 150 via a wired or wireless communication interface.

[0166] In some alternative embodiments, the multi-body system 100 can further comprise an alarm device configured to output an alarm when the body section 104 comprising the camera 110 is determined to be dysfunctional. The alarm device can alert both the animal 105 and the farmer to stay away from the body section 104 as it can lead to serious accidents even if the emergency stop of the milking robot is triggered.

[0167] The multi-body system 100 can additionally comprise a database 150 configured to store reference values for the reference objects 200a, 200b, 200c, 200d; and / or a measured time value regarding the time taken to move the body section 104 / camera 110 from the first position PI to the second position P2.

[0168] However, in some alternative embodiments, the multi-body system 100 can further comprise additional units for performing the method 500 according to steps 501-513.

[0169] The above described steps 501-513 to be performed in the multi-body system 100 can be implemented by one or more processing circuitries 620 within the control unit 140, together with a computer program for performing at least some of the functions of steps 501-513. The computer program thus comprises instructions which, when the computer program is executed by the control unit 140 in the multi-body system 100, cause the control unit 140 to perform the method 500 according to at least some of the steps 501-513.

[0170] The above described computer program can for example be provided in the form of a computer readable medium, i.e. a data carrier which carries the computer program code for performing at least some of the steps 501-513 according to some embodiments when loaded into one or more processing circuitries 620 of the control unit 140. The data carrier can be for example a hard disk, a CD ROM disc, a memory stick, an optical storage device, a magnetic storage device or any other appropriate medium such as a diskette or tape which can store machine readable data in non-transitory form. The computer program can additionally be provided on a server and downloaded remotely to the control unit 140, for example over an internet or intranet connection.

[0171] The terms used in the description of embodiments as shown in the attached drawings are not intended to limit the described method 500; multi-body system 100; control unit 140; computer program and / or computer readable medium. Various changes, substitutions and / or alterations can be made without departing from embodiments of the invention defined by the appended claims.

[0172] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items. The term "or" as used herein is to be interpreted as the mathematical OR, i.e., inclusive OR; and not the mathematical XOR, unless otherwise specifically indicated. Furthermore, the singular forms "a", "an" and "the" are to be interpreted as "at least one", thus also possibly including multiple entities of the same kind, unless otherwise specifically stated. It will be further understood that the terms "includes", "comprises", "including" and / or "comprising", specifies the presence of stated features, actions, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, actions, integers, steps, operations, elements, components, and / or groups thereof. A single unit such as a processor can fulfill the functions of several items recited in the claims. The mere fact that certain measures or features are recited in mutually different dependent claims, in different combinations of claims, or with reference to different embodiments does not indicate that a combination of these measures and features cannot be used to advantage. A computer program can be stored / distributed on a suitable medium, such as an optical storage medium or a solid-state storage medium supplied together with, or as a part of, other hardware, but can also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

Claims

1. A method (500) for a multibody system (100) including a milking robot, the milking robot comprising two main body segments (101, 102, 103, 104) assembled via joints (105, 106, 107), wherein one of the main body segments (104) includes a camera (110) movable between a first position (P1) and a second position (P2), wherein the method (500) comprises the following steps: The camera (110) located at the first position (P1) detects (501) reference objects (200a, 200b, 200c, 200d), wherein the reference objects (200a, 200b, 200c, 200d) include body parts of an animal (105); Calculate the future position of the reference objects (200a, 200b, 200c, 200d) when they are in the second position (P2), as perceived by camera 110; Determine (502) the reference values ​​of the reference objects (200a, 200b, 200c, 200d), wherein the determined reference values ​​(502) include the reference point (300) of the multibody system (100) and the reference vector (310) between the calculated future positions of the reference objects (200a, 200b, 200c, 200d); Request (503) the movement of the camera (110) from the first position (P1) to the second position (P2); Initiate (504) the movement of the camera (110) from the first position (P1) to the second position (P2), while iteratively: Images of the reference objects (200a, 200b, 200c, 200d) are sampled; The current object value of the reference objects (200a, 200b, 200c, 200d) is determined based on the sampled image, wherein the determined current object value includes the object vector (320) between the reference point (300) of the multibody system (100) and the reference objects (200a, 200b, 200c, 200d) in the sampled image. The determined reference value (502) is matched with the current object value, wherein the matching includes a length comparison between the reference vector (310) and the object vector (320); and Based on the matching, the success of the movement of the camera (110) from the first position (P1) to the second position (P2) is determined (505).

2. The method (500) according to claim 1, further comprising the following steps: Based on the success of the camera movement determined (505), the state of the multibody system (100) is determined (512); and Output (513) to determine the state of (512).

3. The method (500) according to claim 1, further comprising the following steps: Measure (506) the time it takes for the camera to move from the first position (P1) to the second position (P2); Compare the measurement (506) time with the reference time limit (507); and When the measurement (506) time exceeds the reference time and reaches the threshold limit, maintenance measures (508) are triggered.

4. The method (500) according to claim 3, further comprising the following steps: Based on the success of the camera movement determined (505) in the stored determination, the success rate of the camera movement is determined (509). The success rate of the determined (509) was compared with the reference value (510); and When the success rate of the determination (509) is lower than the reference value, maintenance measures (511) are triggered.

5. The method (500) according to claim 1, further comprising: Estimate the angle (α) between the reference vector (310) and the object vector (320); and wherein the movement of the camera (110) from the first position (P1) to the second position (P2) is determined as Success occurs when the estimated angle (α) between the determined reference vector (310) and the determined object vector (320) is approximately zero; or The estimated angle (α) between the determined (502) reference vector (310) and the determined object vector (320) is not successful when it is different from zero in addition to the reference constraint.

6. The method (500) according to claim 1, wherein The camera (110) located at the first position (P1) detects (501) multiple reference objects (200a, 200b, 200c, 200d). Multiple reference objects (200a, 200b, 200c, 200d) were captured in the sampled image; and The matching includes mapping the reference value to the current object value.

7. The method (500) according to claim 3, wherein When the measurement time interval between the movement of the request (503) and the start of the movement (504) of the camera (110) from the first position (P1) exceeds the threshold limit of the start reference time, the maintenance measure (508) is triggered.

8. A multibody system (100) including a milking robot, the milking robot comprising two main body sections (101, 102, 103, 104) assembled via joints (105, 106, 107), wherein one of the main body sections (104) includes a camera (110) movable between a first position (P1) and a second position (P2), wherein the multibody system (100) includes Control unit (140), which is configured to Reference objects (200a, 200b, 200c, 200d) are detected via the camera (110) located at the first position (P1), wherein the reference objects (200a, 200b, 200c, 200d) include body parts of an animal (105); Calculate the future position of the reference objects (200a, 200b, 200c, 200d) when they are in the second position (P2) as perceived by the camera 110; determine the reference value of the reference objects (200a, 200b, 200c, 200d), wherein the reference value includes a reference vector (310) between the reference point (300) of the multibody system (100) and the calculated future position of the reference objects (200a, 200b, 200c, 200d). The request is made to move the camera (110) from the first position (P1) to the second position (P2); Initiate the movement of the camera (110) from the first position (P1) to the second position (P2), while iteratively: Images of the reference objects (200a, 200b, 200c, 200d) are sampled; The current object value of the reference objects (200a, 200b, 200c, 200d) is determined based on the sampled images, wherein, The current object value includes the object vector (320) between the reference point (300) of the multibody system (100) and the reference objects (200a, 200b, 200c, 200d) in the sampled image. The determined reference value is matched with the current object value, wherein the matching includes a length comparison between the reference vector (310) and the object vector (320); and The success of the movement of the camera (110) from the first position (P1) to the second position (P2) is determined based on the matching.

9. The multibody system (100) according to claim 8, wherein the control unit (140) is configured to Based on the success of the determined camera movement, the state of the multibody system (100) is determined; and Output the defined state.

10. The multibody system (100) according to any one of claims 8 or 9, wherein the control unit (140) is configured to Measure the time it takes for the camera to move from the first position (P1) to the second position (P2); Compare the measurement time with the reference time limit; and Maintenance measures are triggered when the measurement time exceeds the reference time threshold; and / or The maintenance measure is triggered when the measurement time between the requested movement and the start of the movement of the camera (110) from the first position (P1) exceeds the start reference time limit and reaches the threshold limit.

11. The multibody system (100) according to claim 8, wherein the control unit (140) is configured to The success rate of the camera movement is determined based on the stored and determined success rate of the camera movement. Compare the determined success rate with the reference value; and Maintenance measures are triggered when the determined success rate is lower than the reference value.

12. The multibody system (100) according to claim 8, wherein the control unit (140) is configured to Estimate the angle (α) between the reference vector (310) and the object vector (320); and wherein the movement of the camera (110) from the first position (P1) to the second position (P2) is determined as Success occurs when the estimated angle (α) between the determined reference vector (310) and the determined object vector (320) is approximately zero; or It is unsuccessful when the estimated angle (α) between the determined reference vector (310) and the determined object vector (320) is different from zero, except for the reference constraint.

13. The multibody system (100) according to claim 8, wherein the control unit (140) is configured to Multiple reference objects (200a, 200b, 200c, 200d) are detected by the camera (110) located at the first position (P1). Multiple reference objects (200a, 200b, 200c, 200d) were captured in the sampled image; and The matching includes mapping the reference value to the current object value.

Images