Monitoring livestock in agricultural pens

Abstract

A method comprising: receiving an image of a scene including one or more animals; defining region boundaries for each of the one or more animals; evaluating the suitability of each of the region boundaries for further processing based at least in part on a predetermined set of parameters; and determining at least one of: (i) a physical state of at least one of the one or more animals based at least in part on the further processing, and (ii) an identity of at least one of the one or more animals.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 689,251 filed on June 25, 2018 and U.S. Provisional Patent Application No. 62 / 827,203 filed on April 1, 2019, both of which are entitled “MONITORING LIVESTOCK IN AN AGRICULTURAL PEN,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to computer imaging. Background Art

[0004] Modern farms keep livestock of a single species with the goal of keeping the animals in near-optimal physical condition for maximum growth and production. Such physical condition can include the physical state of the animals, such as weight and health, as well as the physical condition of the enclosure, such as ventilation and enclosure equipment, proper functioning of feeding equipment, watering equipment, etc. Therefore, farms continuously monitor the feeding, environmental conditions, health, weight, growth, and reproduction of the livestock.

[0005] Handling of livestock is mainly carried out by humans, who are responsible for feeding, weighing, controlling the physical state of the environment (e.g. ventilation), identifying the health of the animals, detecting disease and death, etc.

[0006] Weight is an important indicator of the health and value of the animals in the pen. Farmers need to keep a close eye on the weight of their animals to be profitable. Keeping animals above a certain weight can significantly reduce their market value, causing farmers to lose significant income. Current procedures for weighing animals are inefficient as they are time-consuming and labor-intensive. Furthermore, weighing animals using a scale can stress the animals and reduce their growth rate. As a result, farmers tend to make decisions related to the weight of their animals without actually weighing them and without knowing their actual weight, which can lead to lost profits.

[0007] Delayed detection of physical conditions (e.g., pregnancy, parturition, and / or disease) in animals in pens can result in abuse of newborn animals, high mortality rates, and epidemics.

[0008] The foregoing examples of the related art and limitations associated therewith are illustrative and not exclusive. Other limitations of the related art will become apparent to those skilled in the art by reading the specification and studying the drawings. Summary of the invention

[0009] The following embodiments and aspects thereof are described and illustrated in conjunction with systems, tools, and methods, which are exemplary and illustrative, not limiting in scope.

[0010] In an embodiment, a method is provided, the method comprising: receiving an image of a scene including one or more animals; defining area boundaries for each of the one or more animals; evaluating the suitability of each of the area boundaries for further processing based at least in part on a predetermined set of parameters; and determining at least one of: (i) a physical state of at least one of the one or more animals based at least in part on the further processing, and (ii) an identity of at least one of the one or more animals.

[0011] In an embodiment, a system is also provided, comprising: at least one hardware processor; and a non-transitory computer-readable storage medium having program instructions stored thereon, the program instructions being executable by the at least one hardware processor to: receive an image of a scene comprising one or more animals, define area boundaries for each of the one or more animals, evaluate the suitability of each of the area boundaries for further processing based at least in part on a predetermined set of parameters, and determine at least one of the following: (i) determining a physical state of at least one of the one or more animals based at least in part on the further processing, and (ii) an identity of at least one of the one or more animals.

[0012] In an embodiment, a computer program product is also provided, which includes a non-transitory computer-readable storage medium having program instructions embodied therein, the program instructions being executable by at least one hardware processor to: receive an image of a scene including one or more animals; define area boundaries for each of the one or more animals; evaluate the suitability of each of the area boundaries for further processing based at least in part on a predetermined set of parameters; and determine at least one of the following: (i) a physical state of at least one of the one or more animals based at least in part on the further processing, and (ii) an identity of at least one of the one or more animals.

[0013] In some embodiments, the scene includes a livestock living environment.

[0014] In some embodiments, the images are captured from an overhead angle relative to the one or more animals.

[0015] In some embodiments, relative to each of the identified area boundaries, the parameter group includes at least one of: a head angle of the area boundary in the image, a position of the area boundary in the image relative to an acquisition point of the image, visibility of the animal in the scene, the position of the animal in the scene, occlusion of the animal in the scene, and a body posture of the animal in the scene.

[0016] In some embodiments, the evaluating comprises assigning a suitability score to each of the region boundaries, and wherein the further processing is performed only if the suitability score assigned to a region boundary exceeds a specified threshold.

[0017] In some embodiments, the definition includes applying a machine learning detection algorithm, and wherein the machine learning detection algorithm is trained using a training set comprising: (i) multiple images of scenes, each of the scenes comprising one or more animals; and (ii) a label associated with each of the one or more animals in the images.

[0018] In some embodiments, the further processing includes, for each of the animals, identifying a segment boundary associated with a body trunk of the animal.

[0019] In some embodiments, the segment boundaries associated with the body trunk of the animal exclude at least the head of the animal, the tail / tail of the animal, and one or more limbs of the animal.

[0020] In some embodiments, the identification of the segment boundaries associated with the body trunk of the animal comprises applying a trained machine learning segmentation algorithm, and wherein the machine learning segmentation algorithm is trained using a training set comprising: (i) a plurality of images of the animal, wherein the images are captured from an overhead perspective; and (ii) labels associated with segment boundaries of at least some of the body trunk, head, tail, and one or more limbs of each of the animal in the plurality of images.

[0021] In some embodiments, the determination of the physical state is based at least in part on a comparison between the segment boundaries associated with the body trunk of each of the animals and reference data.

[0022] In some embodiments, the determination of the physical state comprises applying a trained machine learning algorithm, and wherein the machine learning algorithm is trained using a training set comprising: (i) a plurality of images of the animal's torso, wherein the images are captured from an overhead angle; and (ii) a label associated with the physical state.

[0023] In some embodiments, for each of said animals, said physical state is at least one of developmental stage, size, weight, surface parameters, color, condition, health, maturity, and likeness of said animal.

[0024] In some embodiments, the determination of the identity of at least some of the one or more animals is based at least in part on: (i) detecting an identification area associated with each of the area boundaries, and (ii) matching the detected identification areas to known identities.

[0025] In some embodiments, the identification area is located in a specific body area of ​​the animal.

[0026] In some embodiments, the identification area includes one or more of: one or more of: one or more of a colored label, a grayscale label, a tattoo, a barcode, a label with text, a label with numbers, and a Quick Response (QR) code.

[0027] In some embodiments, the one or more animals are a group of animals of the same type and breed.

[0028] In some embodiments, the animal is selected from the group of livestock animals comprising cattle, cows, pigs, sheep, and goats.

[0029] In some embodiments, the method further comprises and the program instructions are further executable to (i) receive a sequence of images of the scene, and (ii) track at least one of the one or more animals in the sequence of images.

[0030] In some embodiments, the sequence of images is a video stream.

[0031] In some embodiments, the tracking is based on at least one of: (i) defined area boundaries, (ii) segment boundaries, or (iii) an identified area associated with the at least one of the one or more animals.

[0032] In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the drawings and study of the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The present disclosure will now be described in conjunction with certain examples and embodiments with reference to the following illustrative drawings so that the present disclosure may be more fully understood. In the drawings:

[0034] Figure 1A is a block diagram of an exemplary system for automatically identifying individual animals in a livestock herd environment and determining one or more physical, developmental and / or health status parameters according to an embodiment of the present invention;

[0035] Figure 1B It is a top view of the animal;

[0036] Figure 2A , 2B , 2C and 2D are schematic diagrams of a process for identifying an individual animal in a livestock herd environment and determining one or more physical, developmental and / or health status parameters according to an embodiment of the present invention;

[0037] Figure 3A and 3B is a flow chart of the functional steps of a process for identifying an individual animal in a livestock herd environment and determining one or more physical, developmental and / or health status parameters according to an embodiment of the present invention;

[0038] Figure 4A-4B illustrates a process executed by a detection and identification module according to an embodiment of the present invention;

[0039] Figure 5 is an example of a result image created by an evaluation module according to an embodiment of the present invention; and

[0040] Figure 6 The process executed by the monitoring module according to the embodiment of the present invention is illustrated.

[0041] It should be understood that for simplicity and clarity of illustration, the elements shown in the figures are not necessarily drawn to scale. For example, for clarity, the size of some elements may be enlarged relative to other elements. In addition, reference numerals may be repeated in the drawings to indicate the same or similar elements where deemed appropriate. DETAILED DESCRIPTION

[0042] In the following description, various aspects of the present invention will be described. For the purpose of explanation, specific configurations and details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to those skilled in the art that the present invention may be implemented with all the specific details not provided herein. In addition, well-known features may be omitted or simplified to avoid obscuring the present invention.

[0043] Methods and systems according to embodiments of the present invention are directed to providing an automated computerized system for (i) identifying individual animals within a livestock herd environment, and (ii) determining one or more physical, developmental and / or health status parameters regarding the identified animals.

[0044] In some embodiments, livestock groups include, for example, cattle, cows, cows, bulls, calves, pigs, sows, boars, piglets, horses, sheep, goats, and / or deer. In some embodiments, animal groups have the same type, breed, and / or race.

[0045] In some embodiments, the present disclosure provides for the calculation and / or estimation of multiple parameters associated with the physical, developmental and / or health status of livestock (e.g., individual identity, size, volume, weight, surface parameters, color, state, health, maturity, similarity, behavioral state, mood, visual attention, and other relevant parameters).

[0046] In some embodiments, livestock are housed in a group housing environment, such as a pen, feeding area, and / or similar enclosure for housing multiple animals. In some embodiments, the present disclosure provides a video stream of one or more images and / or video images of the housing environment captured by an imaging device. In some embodiments, the images are captured from an overhead angle so that the images substantially capture the back region of each animal.

[0047] In some embodiments, the present disclosure uses one or more object detection methods to detect one or more animals in an image. In some embodiments, the present disclosure is configured to define the boundaries of each animal in the image, for example, by a bounding box. As used herein, the term "bounding box" refers to an area in an image that surrounds content. For example, one or more bounding boxes can be, for example, a square or rectangular frame around an object being identified in an image.

[0048] In some embodiments, the present disclosure may then be configured to evaluate the suitability of each detected animal in the image for further processing, for example by assessing at least one of the completeness of the animal within the bounding box (e.g., when the animal is only partially represented in the image), animal body pose (e.g., standing, lying down), visibility, occlusion (e.g., when animals are standing close to each other or when one animal is stepping over another), and / or position relative to the imaging device (which determines the perspective of the animal in the image).

[0049] In some embodiments, the present disclosure may be configured to assign a suitability score to each animal detected in an image, wherein further processing of the image may be provided when the suitability score exceeds, for example, a specified threshold.

[0050] In some embodiments, further processing of the image may include, for example, segmenting each bounding box associated with an animal in the image into segments associated with, for example, the animal's body trunk, the animal's head, limbs, and tail.

[0051] In some embodiments, the present disclosure further provides for determining the physical, developmental, and / or health status of an animal based at least in part on a comparison between a segment associated with the animal's body trunk and reference data. In some embodiments, the physical, developmental, and / or health status of the animal may include, for example, size, volume, weight, surface parameters, color, state, hygiene, maturity, similarity, behavioral state, mood, visual attention, and other relevant parameters.

[0052] In some embodiments, the present disclosure may be further configured to identify one or more animals detected in an image based on, for example, known animal identities. In some embodiments, animal identification is based at least in part on detecting an identification region associated with a bounding box of the animal and matching the detected identification region to a known identity. In some embodiments, the identification region may be located in a specific body region of the animal. In some embodiments, the identification region includes at least one of: one or more colored tags, tattoos, barcodes, tags with text and / or numbers, and / or quick response (QR) codes.

[0053] Figure 1A 1 is a block diagram of an exemplary system 100 for automatically identifying individual animals in a livestock herd environment and determining one or more physical, developmental and / or health status parameters about the identified animals. The system 100 described herein is merely an exemplary embodiment of the present invention and may actually have more or fewer components than shown, may combine two or more components, or may have different component configurations or arrangements. The various components of the system 100 may be implemented in hardware, software, or a combination of hardware and software. In various embodiments, the system 100 may include dedicated hardware devices, such as mobile devices, cellular phones, digital cameras, etc.

[0054] In some embodiments, the system 100 may include a hardware processor 110, a communication module 112, a memory storage device 114, a user interface 116, and an imaging device 118, as well as a light source 120. The system 100 may store, in its non-volatile memory, such as the storage device 114, software instructions or components configured to operate a processing unit (also referred to as a "hardware processor," "CPU," or simply "processor"), such as the hardware processor 110. In some embodiments, the software components may include an operating system, including various software components and / or drivers for controlling and managing general system tasks (e.g., memory management, storage device control, power management, etc.) and facilitating communication between various hardware components and software components.

[0055] In some embodiments, a non-transitory computer-readable storage device 114 (which may include one or more computer-readable storage media) is used to store, retrieve, compare, and / or annotate captured image frames. Image frames may be stored on the storage device 114 based on one or more attributes or tags, such as a timestamp, a user-entered tag, or the result of an applied image processing method indicating the association of the frame, etc.

[0056] The software instructions and / or components that operate the hardware processor 110 may include instructions for receiving and analyzing a plurality of image frames captured by the imaging device 118. For example, the hardware processor 110 may include an image processing module 110a that receives one or more images and / or image streams from the imaging device 118 and applies one or more image processing algorithms thereto. In some embodiments, the image processing module 110a and / or the machine learning module 110b include one or more algorithms configured to perform object detection, segmentation, discrimination, recognition, and / or classification in the images captured by the imaging device 118 using any suitable image processing techniques. The image streams received by the image processing module 110a may vary in resolution, frame rate (e.g., between 15 and 35 frames per second), format, and protocol depending on the characteristics and purpose of their respective source devices. Depending on the embodiment, the image processing module 110a may route the image stream through various processing functions, or route the image stream to an output circuit or another logical destination that sends the processed image stream for presentation, for example, on the display 116a, over a network to a recording system. In the image processing module 110a, the image stream processing algorithm can improve visibility and reduce or eliminate distortion, glare or other undesirable effects in the image stream provided by the imaging device. The image stream processing algorithm can reduce or remove fog, smoke, contaminants or other blurs present in the image stream. The image stream processing module 110a can apply the image stream processing algorithms alone or in combination.

[0057] In some embodiments, system 100 includes a communication module (or a set of instructions), a contact / motion module (or a set of instructions), a graphics module (or a set of instructions), a text input module (or a set of instructions), a global positioning system (GPS) module (or a set of instructions), a speech recognition and / or speech replication module (or a set of instructions), and one or more applications (or a set of instructions).

[0058] For example, communication module 112 can connect system 100 to a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. Communication module 112 facilitates communication with other devices through one or more external ports and also includes various software components for processing data received by system 100.

[0059] In some embodiments, the user interface 116 of the system 100 includes a display monitor 116a for displaying images, a control panel 116b for controlling the system 100, and a speaker 116c for providing audio feedback. In some variations, the display 116a can be used as a viewfinder and / or a real-time display for still and / or video images acquired by the imaging device 118. The image stream presented by the display 116a can be an image stream originating from the imaging device 118.

[0060] Imaging device 118 is broadly defined as any device that captures images and represents them as data. Imaging device 118 may include a 2D or 3D camera and / or a camera configured to capture images in which animals may be detected and / or identified even in low-light physical conditions (e.g., at night). One or more imaging devices 118 may be mounted in a livestock environment so that multiple areas of interest (e.g., feeding areas and watering areas) may be captured, typically at known angles. Imaging device 118 may be mounted, for example, on the ceiling of an enclosure so that it may capture an overhead image of the entire enclosure or one or more areas of interest, such as Figure 1B shown.

[0061] The imaging device 118 can be optically based, but can also include depth sensors, radio frequency imaging, ultrasound imaging, infrared imaging, etc. In some embodiments, the imaging device 118 can be configured to detect RGB (red-green-blue) spectral data. In other embodiments, the imaging device 118 can be configured to detect at least one of monochrome, ultraviolet (UV), near infrared (NIR), and short wave infrared (SWIR) spectral data. In some embodiments, the imaging device 118 includes a digital imaging sensor selected from a group including silicon-based detectors, complementary metal oxide semiconductors (CMOS), charge coupled devices (CCDs), indium gallium arsenide (InGaAs), and polarization sensitive sensor elements. The imaging device 118 may also include, for example, zoom, magnification, and / or focusing capabilities. The imaging device 118 may also include functions such as color filtering, polarization, and / or glare removal to achieve optimal visualization. The imaging device 118 may also include an image stream recording system that is configured to receive and store records of image streams received, processed, and / or presented by the system 100.

[0062] In some embodiments, system 100 includes one or more user input control devices, such as a physical or virtual joystick, mouse, and / or click wheel. In other variations, system 100 includes one or more peripheral interfaces, RF circuits, audio circuits, microphones, input / output (I / O) subsystems, other input or control devices, optical or other sensors, and external ports. System 100 may also include one or more sensors, such as proximity sensors and / or accelerometers. Each of the above modules and applications corresponds to a set of instructions for performing one or more of the above functions. These modules (i.e., instruction sets) do not need to be implemented as separate software programs, processes, or modules, so various subsets of these modules can be combined or otherwise rearranged in various embodiments.

[0063] The system 100 described herein is only an exemplary embodiment of the present system, and may have more or fewer components than shown, may combine two or more components, or may have different component configurations or arrangements. The various components of the system 100 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits. In various embodiments, the system 100 may include a dedicated hardware device, or may form an addition or extension to an existing medical device (e.g., a colposcope). In addition, various aspects of the present system that may be implemented by computer program instructions may be executed on a general-purpose computer, a special-purpose computer, or other programmable data processing device.

[0064] In some embodiments, the system 100 may be configured to control one or more devices, appliances, and / or systems associated with the operation of livestock environments and enclosures, such as ventilation systems; feeding systems; painting equipment for marking specific animals (with paint); heating systems; etc. In some embodiments, additional devices that may be controlled by the system 100 may include, for example, devices that maintain a database to which data may be written; or devices that run applications, which may include, for example, messaging applications.

[0065] In some embodiments, system 100 may further include a sensor module, which may include sensors that provide information about environmental conditions (e.g., temperature, humidity) and / or animal behavior, movement, position, and / or speed.

[0066] In some embodiments, the user application can be installed on any user device such as a computer or mobile device, and can communicate with the system 100, for example, through the communication module 112. The user application can provide information about each individual animal in the fence and the entire fence, the entire building and / or the entire farm to the user at a given time and / or over time. In addition, the user application can provide a prediction of the physical state (e.g., weight, health, etc.) of the animal, for example, based on the data stored in the database. In addition, the user application can provide alarms and notifications about the physical state of the warning animal or fence. In one example, the user application may include or may be connected to a user interface 116 for displaying, for example, images, instructions and / or notifications (e.g., via text or other content displayed on a monitor). The user interface 116 can also be designed to receive input from an external user. For example, a user interface device may include a monitor and a keyboard and / or a mouse and / or a touch screen to enable an external user to interact with the system. In some embodiments, the user interface may include a light that can light up or change color or another indicator such as an audio player to emit a sound. Optionally, a user interface can be provided on a multi-purpose device such as a smart phone, a tablet computer or a personal computer.

[0067] Reference now Figure 2A , 2B , 2C and 2D, which are schematic diagrams of a monitoring process 200 constructed and operated in accordance with an embodiment of the present invention.

[0068] In one embodiment ( Figure 2A ), the monitoring process 200 can receive a sequence of images 201 (e.g., 2D images), process each of the images 201, and create an output including at least one of the physical state of at least some of the animals in the images and / or the identity of at least some of the animals.

[0069] In one embodiment, monitoring process 200 includes operations such as Figure 1A The image processing module 110a in the embodiment can be used to apply: a detection module 210 for detecting individual animals from an image; an identification module 220 for assigning identities to individual animals; an assessment module 230 for assessing the physical state of the animal and / or enclosure; and an output module 240 for generating appropriate output based on the assessed identification and physical state.

[0070] In one embodiment, Figure 2BAs shown, monitoring process 200 can implement process 20. In step 22, monitoring process 200 can receive an image of an area of ​​a fence. In step 24, monitoring process 200 can identify a specific animal based on image pixels associated with the specific animal via recognition module 220. In step 26, monitoring process 200 can evaluate a physical condition associated with the identified animal from the image via evaluation module 230.

[0071] In one embodiment, Figure 2C and 2D As shown, image 201 may be a top-down image of an area in a fence where detection module 210 and recognition module 220 may operate simultaneously. In step 211, detection module 210 may detect and segment different animals present in image 201, and in step 221, recognition module 220 may identify each specific animal in image 201. After detecting, segmenting, and identifying each animal, in step 231, assessment module 230 may assess the physical state of each animal or fence, and in step 241, output generation module 240 may create and send an output.

[0072] The order of the steps of detecting (and segmenting) and identifying animals in image 201 may vary in different embodiments. In one embodiment, detection module 210 may first operate in step 221 to detect different animals present in image 201, and then recognition module 220 may operate in step 211 to identify each specific animal ( Figure 2C In another embodiment, the monitoring process 200 may first activate the recognition module 220 to first identify each specific animal, and then activate the detection module 210 to detect and segment the different animals.

[0073] Now refer to Figure 3A Describe the functional steps that are the process used to detect animals in an image.

[0074] At step 300, the system such as Figure 1A The system 100 in may receive a sequence of images depicting a top view of an area in a livestock enclosure.

[0075] In some embodiments, at step 302, the image processing module 110a can be configured to detect animals (e.g., pigs) in the image by, for example, first defining a bounding box associated with each animal in the image. In some embodiments, the image processing module 110a can use multiple segmentation processes, such as a trained multi-region convolutional neural network (MR-CNN) or any other suitable shape detection and segmentation algorithm. For example, the MR-CNN can be trained, for example, to (i) classify images into images containing animals and images not containing animals, (ii) in images containing animals, separate between animals by creating a bounding shape around each animal, and (iii) for each pixel in the image, determine whether it belongs to an animal, the background, or another animal. Other multiple segmentation processes can be used by the image processing module 110a to detect animals in the image, including other machine learning algorithms that can be trained to detect animals.

[0076] The assessment module 230 may assess the physical state of the animal from the image (or group of images) 201, typically based on the detection and segmentation processes performed by the image processing module 110a and / or by applying image analysis algorithms (e.g., shape detection and motion detection algorithms) to the image (or multiple images). In some embodiments, the assessment module 230 may receive as input a vector that includes information extracted from the image 201, such as pixels, their distance from the middle of the image, their x and y coordinates, the center of gravity of the detected animal (e.g., based on the bounding shape of the animal), etc., and may use a machine learning algorithm to train on such vectors and / or otherwise use such vectors (e.g., as described below) to assess the physical state of the animal from the image.

[0077] In some embodiments, the algorithm can be trained on a training set that includes multiple images of overhead views of areas in livestock pens containing multiple animals. In some embodiments, the training set images can be labeled with appropriate labels for the presence of animals in the images and / or areas associated with "animals" and "background."

[0078] In some embodiments, animal detection may be performed on images obtained from the 3D camera and / or the IR camera using known techniques.

[0079] In some embodiments, at step 304, after the bounding box associated with each of the animals on the image is correctly defined, the present disclosure may be configured to apply selection criteria to the detected animals and select only those animals that meet one or more criteria for further processing. The criteria may be associated with the detected shape, (which may indicate the physical state of the identified animal). For example, the criteria related to the shape may include criteria related to the bounding shape and / or other shape parameters of the animal. The criteria related to the bounding shape may include, for example, "having a complete bounding shape in the image"; "the bounding shape is within a specific angle or distance from the vertical direction of the camera" (e.g., directly below the camera); "the bounding shape includes the entire animal"; "the shape of the bounding shape is a specific shape" (e.g., when processing animals, the shape of the bounding shape should be a rectangle or ellipse, not a square or circle), etc. The criteria related to the shape of the detected animal may include, for example, the outline of the animal, its rotation (i.e., the angle relative to the camera axis), its posture (i.e., whether it is standing, sitting or lying, etc.), etc. In some embodiments, the selection criteria can be based on multiple parameters, including, but not limited to, the completeness of the animal within the bounding box (e.g., when the animal is only partially represented in the image), animal body posture (e.g., standing, lying down), visibility, occlusion (e.g., when the animals are standing close to each other or when one animal is stepping over another), and / or position relative to the imaging device, which determines the perspective of the animal in the image.

[0080] For example, in some embodiments, a machine learning classifier can be trained to classify animal shapes to determine suitability for further processing. The purpose of the suitability classification stage is to discard animals in the image that are partial, incomplete, occluded, too far from the image acquisition point, and / or too close to other animals, because such animals may not be properly segmented in subsequent stages. In some embodiments, the machine learning classifier is trained on a training set that includes multiple images of an overhead view of an area in a livestock enclosure containing multiple animals, where each animal can be labeled based on, for example, the shape of the animal's body (e.g., an animal lying on the ground can present a more rounded body shape than a standing animal); whether it appears in the image as a whole or partial animal; whether it is in contact with another animal; whether it is occluded by another animal and / or another object; and / or based on its position relative to the point where the image was acquired (which affects the perspective of the animal in the image).

[0081] The trained classifier can then be applied to one or more images acquired as in step 300 to predict the suitability of each animal in the image for further processing. In some embodiments, the trained classifier can assign a suitability score, wherein further processing of the image can be provided when the suitability score exceeds, for example, a specified threshold.

[0082] In some embodiments, at step 306, the present disclosure may be configured to perform image segmentation to segment, for example, the body trunk of each animal in the image from, for example, the head, limbs, and tail of the animal.

[0083] In some embodiments, the segmentation step may include a process of semantic segmentation, the purpose of which is to recover image regions that directly correspond to objects in the image by labeling each pixel in the image as a semantic category. In some cases, a convolutional neural network (CNN) such as MR-CNN may be used for image semantic segmentation.

[0084] In some embodiments, a machine learning segmentation algorithm may be trained on a training set of multiple images of a top view (i.e., dorsal region) of an animal, wherein regions in the images corresponding to the semantic categories "body trunk," "head," "limbs," and / or "tail / tail" are labeled in the images. The trained segmentation algorithm may then be applied to one or more images as acquired in step 300 to predict the corresponding category for each segment in the image.

[0085] In some embodiments, the image including the segmented body trunk region of the animal may be processed to determine a physical state of the animal at step 308. In some embodiments, such physical state may be at least one of size, volume, weight, condition, hygiene, health, and / or maturity. In some embodiments, the determination is based at least in part on a comparison with reference data.

[0086] In some embodiments, a machine learning algorithm may be trained to predict the physical state of one or more animals on a training set, the training set comprising a plurality of images of body trunk segments of a plurality of animals, wherein each segment is labeled according to, for example, one or more physical state parameters associated with such animals. The trained machine learning algorithm may then be applied to one or more images such as those acquired in step 300 to predict one or more physical state parameters associated with the target animal.

[0087] In one embodiment, the evaluation module 230 can predict the weight of the pig by assigning pixel weights to pixels belonging to animal body parts and calculating the weight of the animal using linear regression. Additional parameters (e.g., the number of pixels of the animal's body parts, the distance of each pixel from the center of the image, etc.) can be used to evaluate or predict the weight of the animal, and a linear regression method can be used to find the best linear correlation of each parameter with the pixel weights. For example, in an image of an animal at the periphery of the camera FOV, the body of the animal is larger and may contain more pixels than the body of the same animal in an image of an animal at the center of the camera FOV. Using a linear regression method, the weight of the animal can be adjusted to the position of the animal in the image. In some embodiments, additional parameters (e.g., as described above) can be used to train a neural network to evaluate the weight of the animal based on the body parts of the animal in the image.

[0088] Machine learning methods can be trained to take into account the perspective of the photography (e.g., ignore some pixels when the pig is at the edge of the image, i.e., ignore those pixels that belong to the height projection and therefore should not be considered in the weight calculation), the shape and / or position of the animal in the image, etc.

[0089] Because the animal moves in the enclosure, the image 201 may capture only a portion of the animal in the image, or may capture the animal in different postures, some of which do not provide enough information required for the assessment. In these cases, the assessment results may vary between consecutive images, for example, when the animal is lying down, the image may include only a subset of pixels required to calculate its weight, and the resulting assessment may be lower than the actual value.

[0090] When provided with a sequence of images, the assessment module 230 may take into account the weight assessed in previous images of the tracked animal and improve the accuracy of the assessment of the animal's physical condition. For example, the assessment module 230 may calculate an average between weights calculated for the tracked animal in different images in the sequence, calculate median values, ignore outliers, consider weights assessed in most images, and / or use any other statistical or non-statistical considerations when assessing the weight of an animal based on multiple images of the animal. The assessment of weight may be further assisted by using information from previous images (e.g., images of the animal in the same enclosure and / or other animals captured within the past few days) stored in the database 110 to smooth the weight assessment.

[0091] The functional step is the process used to identify individuals / single animals in the image and will now be referred to as Figure 3B In some embodiments, the present disclosure is configured to identify individual animals and specific animals in an image based on pixels of the image, which pixels are associated with the specific animal. In one embodiment, visible markers associated with the specific animal are detected in the image and used to identify the specific animal.

[0092] At step 310, a system such as Figure 1A The system 100 in may receive a sequence of images depicting a top view of an area in a livestock enclosure.

[0093] In some embodiments, at step 312, the image processing module 110a can be configured to detect animals (e.g., pigs) in the image by, for example, first defining a bounding box associated with each animal in the image. In some embodiments, the image processing module 110a can use multiple segmentation processes, such as a trained multi-region convolutional neural network (MR-CNN) or any other suitable shape detection and segmentation algorithm. The MR-CNN can be trained, for example, to (i) classify images into images containing animals and images not containing animals, (ii) in images containing animals, separate between animals by creating a bounding shape around each animal, and (iii) determine for each pixel in the image whether it belongs to an animal, background, or another animal. Other multiple segmentation processes can be employed by the image processing module 110a to detect animals in the image, including other machine learning algorithms that can be trained to detect animals.

[0094] In one embodiment, at step 314, the recognition module 220 of the image processing module 110a identifies a specific animal based on visible markings and based on the position of the visible markings relative to the specific animal in the image. For example, the visible markings may be within the bounding shape of the animal, or within a predetermined distance from the bounding shape. In another example, the visible markings may be on the head of the animal. In one embodiment, the visible markings include color tags attached to specific locations on the head of each animal. One or more color tags (possibly of different shapes) may be attached to specific locations relative to the animal, for example, in the ears of the animal and / or at specific locations relative to each other, thereby generating multiple unique combinations, each uniquely identifying a specific animal. In some embodiments, the tags may have different gray levels, and the image analysis of the camera and / or and / or the image of the fence may use grayscale image technology to avoid problems related to lighting in the fence. In some embodiments, each color (or grayscale) tag may include one or more colors or gray levels. The different possible combinations of color (or grayscale) tags, their different shapes, and their possible relationships to each other are referred to herein as "color codes."

[0095] Identification module 220 can detect the color / shape of the tag and provide an identification of each animal based on the color / shape combination. Identification module 220 can detect one or more visible color codes at predetermined locations relative to a particular animal in an image and can assign a unique identity to the animal based on the detection.

[0096] At step 316, in some embodiments, the visible marker may include a pattern that is unique to a particular animal (e.g., a unique texture of the animal's skin at a particular location on the animal's body, a unique pattern of dirt or other natural elements on the animal's skin, etc.). Once a particular animal is detected and a transient or other visible marker is associated with the particular animal, the animal may be tracked throughout the image of the enclosure and identified at any point based on the visible marker. In the case of a transient visible marker, tracking and identification of the animal may last for a limited time, such as a few minutes to a few hours.

[0097] By applying known tracking algorithms on the image, for example by using optical flow, the animal can be tracked throughout the image of the enclosure.

[0098] In some embodiments, the tracked animal can be identified in at least one image based on a color code. Once the animal is identified based on the color code, other potentially more visible markings on the identified animal (such as the transient patterns mentioned above) can be associated with the identified animal, and the animal can then be identified in future images based on the more visible markings.

[0099] In some embodiments, animals can be identified by using a facial recognition algorithm. For example, a camera is not located on the ceiling of the enclosure but is located in a position to capture the animal's face, providing a facial image of the animal in a specific position. The same animal can be simultaneously photographed by a camera located on the ceiling of the enclosure. The two images can be correlated to be able to assign an identity to the animal in the top view image based on the animal's facial image.

[0100] Reference now Figure 4A, which is a schematic diagram of a process 400 that may be implemented by the recognition module 220 to identify a particular animal using a visible mark associated with the particular animal, such as a color mark. The color mark recognition process 400 can identify an animal that is marked with two colors in a 2D image encoded with a red, green, and blue (RGB) color model. The input of the process 400 can be an RGB image 201. In step 410, the recognition module 220 can convert the RGB image into a hue, saturation, and value (HSV) model. In step 420, the recognition module 220 can use, for example, morphological filtering to identify the colors in the HSV model. In step 430, the recognition module 220 can check whether the two colors are detected within a predetermined distance range between each other, such as whether the distance between the two identified colors is less than a predetermined value. If the two colors are detected within the predetermined range, then in step 440, the recognition module 220 can assign the identification to the particular animal according to a predefined color code configuration, for example, pink and blue meet the color code of pig number X. If two colors are detected at a distance outside the predetermined range, or if more than two colors are detected within the predetermined range, the recognition module 220 may declare that the recognition is unsuccessful in step 450 .

[0101] Reference now Figure 4B , which includes a sub-image 201B, which includes a pig from the pigs captured in image 201. It can be noted that the recognition module 220 can process the original image 201 received as input or the image 201A after being processed by the image processing module 110a. The recognition module 220 can detect the color tags 410 and 420 in the sub-image 201B and can determine whether the tags are within a predetermined distance range from each other. The recognition module 220 can then determine the color of each tag (for example, the color of tag 410 is identified as pink and the color of tag 420 is identified as blue). Once the colors on the color tags are determined, the pig can be identified.

[0102] In some embodiments, the color code depends on the position of the first color (or grayscale) label relative to the second color (or grayscale) label.Alternatively or additionally, the color code may depend on the shape of the different color (or grayscale) labels.

[0103] An alternative color tag recognition process may be implemented using a trained neural network or other machine learning technique to recognize color labels in an image and identify the pig based on the color code and possibly the pig's shape.

[0104] The recognition module 220 may use the sequence of images to track animals to improve the accuracy of their identification. Since each animal may move in the pen, the images 201 may not capture the entire color mark of the pig in each image, and therefore the recognition module 220 may not be able to identify the animal in some images. When the sequence of images is provided to the recognition module 220, the probability of capturing the entire color mark in at least one of the provided images increases, and the identity of the animal obtained in one image of the sequence can be used for other images of the sequence in which the animal is tracked and the identification fails.

[0105] The identification module 220 may use any other suitable technology to identify the animal. Alternative image processing identification mechanisms used by the identification module 220 may be based on tattoos, bar codes, quick response (QR) codes, or any other uniquely identifiable visible markings located on the upper body of the animal that are visible to the imaging device 118. Identification of individual animals may also be aided by sensors other than image sensors, such as including one or more of RFID sensors (for identifying electronic collars and / or electronic ear tags (e.g., RFID tags), or sensors for identifying non-electronic collars, non-electronic ear tags, etc.

[0106] Reference now Figure 5 , which is formed by the image 201B ( Figure 4B ) is an example of a result image 201C created by the evaluation module 230 operating in accordance with the embodiment of the present invention. It can be noted that the same process can be implemented for all animals in the bounding box selected by the image processing module 110a. Figure 5 As can be seen in , evaluation module 230 can divide or separate the shape in image 201C into three parts representing head 510, body 520 and legs 530.

[0107] In one embodiment, the identification module 220 can work after the assessment module 230 separates the specific animal into head and body parts. In this case, the identification module 220 can identify the specific animal based on the head (e.g., based on the color tag attached to the animal's ear as described above), and the assessment module 230 can assess the physical condition of the specific animal based on the body parts.

[0108] The assessment module 230 may assess the animal's physical health based on the animal's behavior and position over time as captured by the images. For example, if the animal does not change its posture and / or position in a predetermined number of consecutive images, the assessment module 230 may assess that the animal is sick or dead. If the animal is not detected near a feeding or drinking device in the enclosure in a predetermined number of consecutive images, the assessment module 230 may assess that the animal is in poor health. If the average speed of the animal calculated based on the animal's position in a predetermined number of consecutive images is higher or lower than the average speed of other animals in the enclosure or its own speed in the past (e.g., known from information stored in the database 110), the assessment module 230 may assess that the animal is sick.

[0109] In one embodiment, the assessment module 230 can utilize machine learning techniques for video content analysis to assess the physical state of the animal, for example, whether the animal is giving birth or the animal is attacking, etc. In some embodiments, if it is determined that the animal is giving birth for longer than a predetermined time and / or if it is determined that the time between the births of the pups is longer than a predetermined time, a signal can be sent to alert a veterinarian of a difficult delivery.

[0110] The assessment module 230 can assess the physical state of the fence, such as when a number of animals appear to be gathered near a certain device in the fence in a predetermined number of consecutive images, the device in the fence is malfunctioning. In one embodiment, the location of the device may be known (predetermined). In another embodiment, the device is automatically detected by the processor 13 using object recognition technology, for example.

[0111] The assessment module 230 can identify physical conditions associated with individual animals by comparing the animal's position (and possible shape) in consecutive images, such as the animal is pregnant or giving birth; the animal is sneezing and / or coughing and / or vomiting and / or losing weight, etc.

[0112] When multiple sensors are installed in the enclosure, the assessment module 230 can consider the information collected by the sensors during the assessment process. The assessment module 230 can determine the location of a particular animal based on the image of the enclosure, and can provide an assessment of the physical condition of the animal based on the animal's location in the enclosure and the information received from one or more sensors.

[0113] The additional information collected by the sensors may be, for example, temperature, noise, humidity, etc. The combination of the input received from the sensors and the images captured by the imaging device 118 may improve the quality of the assessment. For example, a directional microphone that indicates that an animal is coughing and the location of the coughing animal, a temperature sensor that indicates high temperatures at a specific location in the enclosure, and the detection and / or identification of an animal based on the image at the same location where high temperatures and coughing noises are detected may increase the likelihood that the animal (and possibly the identified animal) is sick.

[0114] In some embodiments, a control module operably connected to the system 100 can react to a specific physical state evaluated by the evaluation module 230. For example, the control module can trigger a ventilation device in a fence to change the ventilation in the fence based on the estimated total weight of the pig; or trigger a feeding device to change the food mixture / amount to match the weight of a specific pig approaching a feeder, etc. In one embodiment, the control module can trigger a spraying device to mark a pig that weighs approximately a predetermined weight, or a pig that is suspected of being sick. For example, a pig detected in an image and determined to be in a physical state that needs to be marked with paint can be tracked to different locations in the fence throughout the image of the fence. Once the pig is determined to be in close proximity to the spraying device, the control module can send a signal to trigger the spraying device. The signal can include information such as the pig's location, the pig's posture, the pig's speed, etc., to help accurately paint the pig.

[0115] Reference now Figure 6 , which is an example of a process 600 implemented by the monitoring process 200 constructed and operated according to an embodiment of the present invention. In step 610, the monitoring process 200 can receive a new image (e.g., from the camera 15). In step 620, the image processing module 110a can detect different animals in the image. In step 630, the image processing module 110a can select only a part of the animal (which meets the specific criteria as described in detail above). In step 640, the recognition module 220 can identify different animals, for example, based on color coding, as described above. In step 660, the evaluation module 230 can divide the animal into its parts, and in step 670, the evaluation module 230 can evaluate the weight of the animal's body parts. In step 680, the control module 240 can store the evaluated weight on the database 110 locally or in the cloud, and in step 690, the evaluation module 230 can check whether there are other animals in the image to be processed, and if so, it can continue to process the next animal (step 650). If there are no more animals to process in the image, the process may return to step 610 and begin processing another image.

[0116] In one embodiment, when the color markings are on the pig's ears and the physical condition to be assessed is the pig's weight, the identity of a particular animal may be based on the animal's head and the physical condition may be assessed based on the animal's body part.

[0117] Those skilled in the art will appreciate that the steps shown in the various processes are not limiting and each process may be implemented in various variations.

[0118] As described above, the assessment module 230 can use information calculated and stored in the database 110 in the cloud. This information may include images, image-related vectors, weight, posture, location, identification, and other parameters related to an animal or multiple animals or an entire enclosure. This information can be extracted from previously acquired short-term (e.g., minutes or hours) and / or long-term (e.g., days) images.

[0119] As described above, the system 100 can be used to monitor multiple physical states and send multiple control signals to multiple devices to perform actions. For example, the system 100 can track the physical state (e.g., weight) of individual animals on a farm, and when the animal reaches a predetermined physical state (e.g., a predetermined weight), a predetermined action can be triggered. The physical state can be a configured intermediate weight, and the action for the physical state can be to update the food formula, or the physical state can be a configured final weight, and the action can be to spray the animal so that it can be easily identified.

[0120] The physical conditions monitored by system 100 may be irregular activity in the enclosure, such as animals gathering near a water source, animals giving birth, animals coughing, etc., and the action for this may be creating an appropriate alarm and sending it to a predetermined destination, such as a farmer and veterinarian, via a messaging system (such as email, SMS, etc.).

[0121] Physical conditions may also be related to facilities in the enclosure, and assessment of the physical conditions may trigger actions such as start / increase / stop / decrease ventilation, heating, etc., for example based on indications received from sensors (e.g., high temperature, etc.).

[0122] In one embodiment, the imaging device 118 may be a camera that provides a series of continuous images, enabling the system 100 to track a particular animal over time in order to improve identification of the animal or assessment of the physical condition of the animal or enclosure. As described above, color, grayscale, or other visible markings may not be clearly visible in each image, weight assessments may vary between images, health assessments may change, etc., such that the use of multiple images may improve identification of the animal or assessment of the physical condition of the animal or enclosure. In addition, the physical condition of some animals or enclosures may be better assessed when multiple images are considered. For example, when assessing the health of an animal, the number of visits to a feeding location during the day and night and the activity of the animal may provide relevant indications of a healthy physical condition. Additional examples of physical conditions that may be identified when using multiple images may be persistent aggressive behavior by a particular animal; birthing, for example, which may be identified through video content analysis, etc.

[0123] The system 100 can provide information to farmers located remotely and can trigger actions to maintain an optimal environment for livestock to achieve optimal growth. As described above, the system 100 can assess the physical state of the livestock in the pen and provide signals to activate some actions. For example, provide instructions that the food formula should be changed according to the actual size of the livestock; provide early detection of diseases, births and deaths; provide early detection of faulty facilities in the pen (such as feeders and waterers); provide information needed to select the best shipping date based on the actual weight of the animals (thereby minimizing costs and maximizing revenue from the pen). The system 100 provides these instructions while minimizing human interaction to minimize livestock exposure to humans, thereby reducing livestock diseases and stress events.

[0124] Additionally, the system 100 may provide energy savings by automatically adjusting ventilation based on, for example, the actual average / total weight of the livestock.

[0125] Unless otherwise noted, it will be apparent from the following discussion that throughout the specification, discussions using terms such as "assessing," "analyzing," "processing," "computing," "calculating," "determining," "detecting," "identifying," and the like refer to the actions and / or processes of a computer or computing system or similar electronic computing device that manipulates and / or transforms data represented as physical quantities (e.g., electronic quantities) in registers and / or memories of the computing system into other data similarly represented as physical quantities in the computing system's memories, registers, or other such information storage, transmission, or display devices.

[0126] Embodiments of the present invention may include a device for performing the operations herein. The device may be specially constructed for the desired purpose, or it may include a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. When indicated by software, the resulting device may transform a general-purpose computer into an inventive element discussed herein. Instructions may define inventive devices that operate with a desired computer platform. Such a computer program may be stored in a computer-readable storage medium, suitable for storing electronic instructions and capable of being coupled to a computer system bus.

[0127] The present invention may be a system, method and / or computer program product. The computer program product may include a computer-readable storage medium (or multiple media) having computer-readable program instructions thereon for causing a processor to perform various aspects of the present invention.

[0128] A computer-readable storage medium may be a tangible device capable of saving and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disc (DVD), a memory stick, a floppy disk, a mechanical encoding device having instructions recorded thereon, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium itself should not be interpreted as a transient signal, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire. In contrast, a computer-readable storage medium is a non-transient (i.e., non-volatile) medium.

[0129] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include copper transmission cables, optical transmission fibers, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions to be stored in a computer-readable storage medium in the corresponding computing / processing device.

[0130] The computer-readable program instructions for performing the operation of the present invention may be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Java, Smalltalk, C++, etc., and conventional process programming languages, such as "C" programming language or similar programming languages. The computer-readable program instructions may be executed completely on the user's computer, partly on the user's computer, as an independent software package, partly on the user's computer and partly on a remote computer or completely on a remote computer or server. In the latter case, the remote computer may be connected to the user's computer via any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (e.g., by using the Internet of an Internet service provider). In certain embodiments, an electronic circuit including, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA) may execute computer-readable program instructions to personalize the electronic circuit by utilizing the state information of the computer-readable program instructions, so as to perform various aspects of the present invention.

[0131] Various aspects of the present invention are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to embodiments of the present invention. It should be understood that each block in the flowchart and / or block diagram and the combination of blocks in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0132] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device create a device for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, which can instruct a computer, a programmable data processing device, and / or other device to operate in a specific manner, so that the computer-readable storage medium having the instructions stored therein includes an article of manufacture, which includes instructions for implementing various aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0133] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing device, or other device to cause a series of operational steps to be performed on the computer, other programmable device, or other device, thereby producing a computer-implemented process, so that the instructions executed on the computer, other programmable device, or other device implement the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0134] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention.In this regard, each block in the flow chart or block diagram can represent a module, segment or part of an instruction, which includes one or more executable instructions for realizing (one or more) specified logical functions.In some alternative embodiments, the function mentioned in the block may not occur in the order mentioned in the figure.For example, the two blocks shown in succession can actually be performed substantially simultaneously, or these blocks can sometimes be performed in reverse order, depending on the functions involved.It will also be noted that each block of the block diagram and / or flow chart description and the combination of the blocks in the block diagram and / or flow chart description can be realized by a system based on special-purpose hardware, which performs a specified function or action or performs a combination of special-purpose hardware and computer instructions.

[0135] The description of various embodiments of the present invention is given for the purpose of illustration, but is not intended to be exhaustive or limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terms used herein are selected to best explain the principles of the embodiments, for practical applications or technical improvements of technologies found in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for estimating the weight of one or more animals, the method comprising: receiving an image depicting a scene including one or more animals; processing the image to define a bounding box for each of the one or more depicted animals, the bounding box containing a corresponding one of the one or more depicted animals; evaluating the suitability of each of the depicted one or more animals for further processing performed to determine a weight of the depicted one or more animals by determining whether the bounding box of each of the depicted one or more animals has a particular shape; as well as further processing the image to determine the weight of those of the depicted one or more animals assessed as suitable for further processing; The further processing includes, for each of the animals depicted as suitable for the further processing: segmenting the corresponding bounding box to determine the boundaries of a segment associated with the body trunk of the corresponding depicted animal; and determining the weight of the corresponding depicted animal based on the boundaries of the segment.

2. The method according to claim 1, wherein the scene comprises a livestock herd living environment.

3. The method of any one of claims 1-2, wherein the images are captured from an overhead angle relative to the one or more animals.

4. The method of claim 1 , wherein evaluating the suitability is further performed based on a predetermined parameter set regarding each of the one or more depicted animals, the parameter set comprising at least one of: a head angle of a bounding box in the image, a position of the bounding box in the image relative to an acquisition point of the image, visibility of the one or more depicted animals in the scene, a position of the one or more depicted animals in the scene, occlusion of the one or more depicted animals in the scene, and a body pose of the one or more depicted animals in the scene.

5. The method of claim 1 , wherein evaluating the suitability comprises assigning a suitability score to the bounding box of each of the one or more depicted animals, and wherein the one or more depicted animals are evaluated as suitable for further processing when the suitability score assigned to the bounding box exceeds a specified threshold.

6. The method of claim 1, wherein the defining comprises applying a machine learning detection algorithm, the machine learning detection algorithm being trained to: (i) classifying images into images depicting animals and images not depicting animals; and (ii) in said images depicting animals, separating between the depicted animals by creating a bounding box around each depicted animal in said image.

7. A method according to claim 1, wherein the boundary of the segment associated with the body trunk of the corresponding depicted animal excludes at least the head of the corresponding depicted animal, the tail of the corresponding depicted animal, and one or more limbs of the corresponding depicted animal.

8. The method of claim 7, wherein the determining of the boundaries of the segment associated with the body trunk of the animal comprises applying a trained machine learning segmentation algorithm, and wherein the machine learning segmentation algorithm is trained using a training set comprising: (i) a plurality of images of an animal, wherein the images are captured from an overhead perspective; as well as (ii) labels associated with boundaries of segments of at least some of the body trunk, head, tail, and one or more limbs of each of the animals in the plurality of images.

9. The method of any one of claims 7-8, wherein said determining the weight of said respective depicted animal comprises: extracting image information based on the corresponding bounding box, the image information including coordinates of pixels in the image, distances of pixels of the segment from the middle of the image, the number of pixels in the segment, and a center of gravity of the corresponding depicted animal in the image; The vector comprising the image information is used as input to a trained machine learning algorithm to determine the weight of the corresponding depicted animal.

10. The method according to claim 9, further comprising: detecting a visible marker associated with a particular animal among the depicted one or more animals; and The particular animal is identified based on (a) the visible marker and (b) the position of the visible marker relative to the corresponding bounding box.

11. The method of claim 10, wherein the visible markings include one or more tags attached to the ears of the particular animal, and wherein identifying the particular animal includes: When the particular animal is assessed as suitable for further processing, further segmenting the corresponding bounding box to determine boundaries of a segment associated with the head of the particular animal, and The particular animal is identified based on the tag and the head.

12. The method of claim 11, wherein the one or more tags for the particular animal comprises a plurality of tags having more than one color, shade of gray, and / or shape providing a code that uniquely identifies the particular animal.

13. The method of claim 12, wherein identifying the specific animal comprises checking whether two of the more than one colors are detected within a predetermined distance from each other to: assigning an identification to the particular animal according to a predefined color code configuration when the distance between the two colors is within the predetermined distance range; and When the distance between the two colors exceeds the predetermined distance range or when more than two of the more than one colors are detected within the predetermined distance range, it is declared that recognition is unsuccessful.

14. The method according to any one of claims 11-13, further comprising (i) receiving an image sequence of the scene, and (ii) tracking at least one of the one or more depicted animals in the image sequence, wherein the tracking is based on at least one of: (i) a defined bounding box, (ii) the boundary of the segment associated with the body trunk, or (iii) the boundary of the segment associated with the head.

15. A system for estimating the weight of one or more animals, the system comprising: at least one hardware processor; as well as a non-transitory computer-readable storage medium having program instructions stored thereon, the program instructions being executable by the at least one hardware processor to: receiving an image depicting a scene including one or more animals, defining a bounding box for each of the one or more depicted animals, the bounding box containing a corresponding depicted animal of the one or more depicted animals, evaluating the suitability of each of the depicted one or more animals for further processing performed to determine a weight of the depicted one or more animals by determining whether the bounding box of each of the depicted one or more animals has a particular shape; as well as further processing the image to determine the weight of those of the depicted one or more animals assessed as suitable for further processing; Wherein the at least one hardware processor is configured to perform the further processing for each of the depicted animals suitable for the further processing by the following steps: segmenting the corresponding bounding box to determine the boundaries of a segment associated with the body trunk of the corresponding depicted animal; and determining the weight of the corresponding depicted animal based on the boundaries of the segment.

16. The system of claim 15, wherein the scene comprises a livestock living environment.

17. The system of any one of claims 15-16, wherein the images are captured from an overhead angle relative to the one or more animals.

18. The system of claim 17, wherein the at least one hardware processor is configured to evaluate the suitability further based on a predetermined parameter set relative to each of the one or more depicted animals, the parameter set comprising at least one of: a head angle of the bounding box in the image, a position of the bounding box in the image relative to an acquisition point of the image, visibility of the one or more depicted animals in the scene, a position of the one or more depicted animals in the scene, occlusion of the one or more depicted animals in the scene, and a body pose of the one or more depicted animals in the scene.

19. The system of claim 18, wherein the at least one hardware processor is configured to evaluate the suitability by assigning a suitability score to the bounding box of each of the one or more depicted animals, and to evaluate the one or more depicted animals as suitable for further processing when the suitability score assigned to the bounding box exceeds a specified threshold.

20. The system of claim 19, wherein the at least one hardware processor is configured to define the bounding box by applying a machine learning detection algorithm, the machine learning detection algorithm being trained to: (i) classifying images into images depicting animals and images not depicting animals; and (ii) in said images depicting animals, separating between the depicted animals by creating a bounding box around each depicted animal in said image.

21. A system according to claim 15, wherein the boundaries of the segment associated with the body trunk of the corresponding depicted animal exclude at least the head of the corresponding depicted animal, the tail of the corresponding depicted animal, and one or more limbs of the corresponding depicted animal.

22. The system of any one of claims 15-16, wherein the at least one hardware processor is further configured to determine the boundary of the segment associated with the body trunk of the animal by applying a trained machine learning segmentation algorithm, and wherein the machine learning segmentation algorithm is trained using a training set, the training set comprising: (i) a plurality of images of an animal, wherein the images are captured from an overhead perspective; as well as (ii) labels associated with boundaries of segments of at least some of the body trunk, head, tail, and one or more limbs of each of the animals in the plurality of images.

23. The system of any one of claims 15-16, wherein the at least one hardware processor is further configured to determine the weight of the corresponding depicted animal by: (i) extracting image information based on the corresponding bounding box, the image information including coordinates of pixels in the image, distances of pixels of the segment from the middle of the image, the number of pixels in the segment, and the center of gravity of the corresponding depicted animal in the image; and (ii) using the vector comprising the image information as input to a trained machine learning algorithm to determine the weight of the corresponding depicted animal.

24. The system of claim 23, wherein the at least one hardware processor is further configured to: detecting a visible marker associated with a particular animal among the depicted one or more animals; and The particular animal is identified based on (a) the visible marker and (b) the position of the visible marker relative to the corresponding bounding box.

25. The system of claim 24, wherein the visible indicia comprises one or more tags attached to the ears of the particular animal, and wherein the at least one hardware processor is configured to further identify the particular animal by: When the particular animal is assessed as suitable for further processing, segmenting the corresponding bounding box to determine boundaries of a segment associated with the head of the particular animal, and The particular animal is identified based on the tag and the head.

26. The system of claim 25, wherein the one or more tags for the particular animal comprises a plurality of tags having more than one color, shade of gray, and / or shape providing a code that uniquely identifies the particular animal.

27. The system of claim 26, wherein the at least one hardware processor is further configured to identify the specific animal by checking whether two of the more than one colors are detected within a predetermined distance from each other so as to: assigning an identification to the particular animal according to a predefined color code configuration when the distance between the two colors is within the predetermined distance range; and When the distance between the two colors exceeds the predetermined distance range or when more than two of the more than one colors are detected within the predetermined distance range, it is declared that recognition is unsuccessful.

28. A system according to any one of claims 25-27, wherein the instructions are further executable to (i) receive a sequence of images of the scene, and (ii) track at least one of the one or more animals depicted in the sequence of images based on at least one of (i) a defined bounding box, (ii) the boundaries of the segment associated with the body trunk, or (iii) the boundaries of the segment associated with the head.

Images