Autonomous livestock monitoring

Abstract

The invention provides a method and a system for identifying one or more animals of a plurality of animals, wherein each animal is associated with a unique identifier. The system includes at least one animal recording module, at least one animal identification module, a memory for storing a database, and at least one processor, the modules configured to perform a method of identifying one or more animals of a plurality of animals.

Description

Technical Field

[0001] This invention relates to a method and system for autonomous livestock monitoring. More specifically, this invention provides a method and system for identifying individual animals within an animal population. Background Technology

[0002] Video analytics technology has made rapid progress in recent years. Complex machine learning methods and algorithms are now becoming readily available. Furthermore, video capture technology has become widespread, with both fixed cameras and camera-equipped drones improving the technology while reducing costs.

[0003] The availability of this technology creates opportunities for the agricultural industry to leverage the advantages of video analytics to improve animal monitoring and achieve higher animal welfare standards. Higher animal welfare standards can improve farm efficiency, prevent animal suffering, and address consumer confidence in the high welfare standards of the animals providing the protein.

[0004] A crucial part of animal surveillance is identifying individual animals. Some existing animal surveillance systems use sensor technologies that rely on human intervention, such as adding RFID tags, to identify animals. This technology is inherently unscalable and prone to mismatches and inaccuracies. For example, sensors such as RFID tags can detach from animals and / or become broken, preventing the RFID system from identifying them. Even when RFID sensors function as intended, the success rate of RFID-based identification is not 100%.

[0005] Other animal monitoring solutions rely on images obtained from dedicated 3D cameras. These dedicated cameras are difficult to configure, maintain, and upgrade—often requiring complex configuration and calibration procedures to ensure they function properly.

[0006] Other animal systems that use imaging sensors and sensors such as radio frequency identification (RFID) have timing issues because these systems may become out of sync with each other due to daylight saving time or differences in computing device time.

[0007] Therefore, known animal monitoring solutions often have several problems, such as difficulty in scaling or updating; inaccuracy or unreliability; limited operational range; time-related issues; and operational challenges in outdoor, harsh or adverse conditions or in poorly lit spaces.

[0008] Therefore, there is a need to provide a cost-effective and efficient solution for animal monitoring to overcome the shortcomings of existing solutions. Summary of the Invention

[0009] The present invention aims to provide a method and system for identifying individual animals from a plurality of animals, in order to overcome the problems of known solutions.

[0010] This invention provides a method and system that can dynamically and autonomously identify animals using multiple measurement methods and continuously learn from the acquired operational data.

[0011] The purpose of this invention is to provide a method and system for identifying individual animals in an animal population (e.g., cattle, pigs, etc.), which can be integrated into and used in a farm environment, which is challenging for many known identification techniques.

[0012] This system can be used to enhance existing systems. For example, existing RFID systems are not always perfect, and the new system can supplement them when existing RFID systems fail. Therefore, when an existing RFID system malfunctions, this system can identify animals and provide the identification information to downstream systems, such as milking parlor management systems, to prevent delays. The system can also alert users to animals that the RFID system has failed to identify. Thus, users can quickly resolve any potential problems, such as replacing or repairing RFID equipment (such as tags). The system can also dynamically react to delays between the scanned animal and the animal appearing under the animal recording module. This eliminates the need to place the RFID reader directly under or inside the animal recording module.

[0013] Furthermore, because the system can determine the identity of each animal by autonomously comparing or associating it with animals identified through the animal identification module during the first time window, it is dynamic and self-scalable. For example, even if no setup data is provided when an animal is detected in the population, the same animal will be scanned again as appropriate. The system can also identify animals using its visual layer, such as using identification vectors with previously formed identification vectors for the same animal. All animals can be automatically registered and identified when appropriate. Moreover, once a match is successful, the animal data can be used to further fine-tune the algorithm to improve future performance. Therefore, the system can adapt to the constantly changing environmental conditions of the farm. The system can then monitor the population to see if there are changes in animal numbers.

[0014] According to a first aspect of the present invention, a method is provided for identifying one or more animals among a plurality of animals, each of the plurality of animals being associated with a unique identifier, the method comprising:

[0015] a) Obtain, via at least one animal recording module and within a first time window, timestamped image data associated with a subset of multiple animals, the timestamped image data including data for identifying the subset of multiple animals in the timestamped image data;

[0016] b) Obtaining timestamped identification data, including one or more unique identifiers, via at least one animal identification module within a second time window, wherein the second time window is offset from the first time window;

[0017] c) Store in a database the association between data identifying a subset of multiple animals obtained in a first time window via at least one animal record module and one or more unique identifiers obtained in a second time window via at least one animal identification module;

[0018] d) Repeat steps (a) through (c) for multiple first time windows and multiple second time windows to store multiple associations in the database;

[0019] e) Analyze multiple relationships stored in the database to generate a mapping between unique identifiers and data that identifies multiple animals.

[0020] According to a further aspect of the invention, a system is provided for identifying one or more of a plurality of animals, each of which is associated with a unique identifier. The system includes: at least one animal recording module; at least one animal identification module; a memory for storing a database; and at least one processor configured to implement the following method:

[0021] a) Obtain, via at least one animal recording module and within a first time window, timestamped image data associated with a subset of multiple animals, the timestamped image data including data for identifying the subset of multiple animals in the timestamped image data;

[0022] b) Obtaining timestamped identification data, including one or more unique identifiers, via at least one animal identification module within a second time window, wherein the second time window is offset from the first time window;

[0023] c) Store in a database the association between data identifying a subset of multiple animals obtained in a first time window via at least one animal record module and one or more unique identifiers obtained in a second time window via at least one animal identification module;

[0024] d) Repeat steps (a) through (c) for multiple first time windows and multiple second time windows to store multiple associations in the database;

[0025] e) Analyze multiple relationships stored in the database to generate a mapping between unique identifiers and data that identifies multiple animals.

[0026] Some animal identification systems rely on a pre-determined assignment of single RFID reads to animal detection. In the real world, this is impossible in many scenarios, such as when the RFID receiver is not within the camera's field of view. So how are they matched? This is a common scenario on farms, where it's impossible to pinpoint the exact animal number / identity of a passing object / cow to each individual animal, but it can be narrowed down to a set of possibilities. The advantage of this disclosure is that it provides an arrangement for collecting visually recognizable features of animals as they pass a camera and assigning a batch of RFID signals. Over time, the system is able to compare the visual detection results with common RFID signals in the list and ultimately infer the correct animal RFID signal and match it to the visually detected animal. Animal instances can be detected and then sorted according to the order in which they appear in a video clip, synchronized with the actual animal ID (e.g., ID from the RFID tag). For example, RFID readers can be placed along a path to capture the ID information from the RFID tag attached to each animal. The order in which animals appear in a video clip may be correlated with the order in which the RFID reader captures each animal ID. A time window is used to synchronize IDs from the animal recording module and the animal identification module. These synchronized IDs, along with images extracted from each animal in the video recording, can be used as training labels to train the ID classification network. For example, once an animal is identified / registered, the system can use data associated with those animals to retrain the neural network to distinguish animals and adapt to the farm's constantly changing conditions. If too many reads occur within a second time window, the system can be readjusted to reduce the time window and improve relevance. In this way, the window is narrowed, reducing the number of potential candidates, making association easier when comparing or relating information from or derived from the animal recording and animal identification modules.

[0027] According to embodiments of this disclosure, prior to adjustment, the system is configured to repeat steps a) through c) until the system identifies each animal identified by at least one animal recording module in a first time window and each animal identified by at least one animal identification module in a second time window at least four times. In this way, a history of read results and vectors can be obtained from the animal recording module and the animal identification module before the final animal identification is assigned. The history of read results and vectors may require a specific animal to appear at least four times before the system registers the identified animal. For example, the system may obtain four animals, such as four cows. The reader (such as the animal identification module) may obtain information such as cows numbered 1, 2, 3, and 4, while the animal recording module (such as a camera) may obtain information such as black cows, white cows, black and white cows, and brown cows. During the next observation period, the reader may obtain cows numbered 2, 3, 4, and 5, so cow number 1 is not detected this time, while the camera captures all cows except the black cow. Therefore, two days later, the system knows that cow #1 is a black cow. In the next observation, the system will obtain the same information; that is, the reader will obtain information such as cows numbered 1, 2, 3, and 4, while the animal recording module (such as a camera) may obtain information such as black cows, white cows, black and white cows, and brown cows. The system can determine that the black cow appeared later, and cow #1 also appeared later, and then determine that cow #1 is a black cow.

[0028] According to embodiments of this disclosure, the identified animals are registered in the system after at least four identifications of each animal identified in a first time window and an animal identified in a second time window. During the registration phase, video recordings of each animal can be captured using a 2D camera. During the registration phase, detected animal instances can be assigned arbitrary unique identifiers (IDs). Each registered animal's arbitrary unique ID is associated with the animal's actual ID obtained from a separate source (e.g., an RFID device, such as a tag, marker, etc.). Therefore, each instance detected in the video recording of a registered animal is associated with that animal's actual ID. Each video recording can be segmented into a set of separate frames, and animal instances can be detected in each frame. A segmentation mask can be applied to each detected animal instance in each frame, thereby detecting reference points and their corresponding positions in each frame. Furthermore, before registering the animal in the system, the system matches or associates the animal with an animal recording module (at the feature vector level) and an animal identification module (at the identifier level, such as an RFID ID). Once an animal is registered but does not appear in the animal identification module, the system records this identification as a malfunction of the animal identification module. However, the registered animal can still be identified through processing and analysis performed at the animal record module. In other words, if the identification module does not receive animal information, the system can still identify the animal via the animal record module once it has been registered. For example, once an animal is registered or recorded in the system, after four instances of association—that is, if the animal identification module stops operating due to reasons such as system interruption—the system is configured to still identify the animal based on the visual vector information obtained from those four instances, even if the animal identification module has not received or acquired animal identification information. In this way, the system can learn which animal is which. Each new animal introduced into the system, or each animal whose identification (such as an RFID device in the form of a tag) may be changed, will be processed until registration is complete, meaning the system has processed the animal at least four times. It is understandable that if the animal is unique (e.g., has unique identifying characteristics), then the animal may be identified faster than four instances.

[0029] According to embodiments of this disclosure, each animal identified by at least one animal recording module is associated with an animal identified by at least one animal identification module within a first time window and a second time window. In this way, by using time windows, the number of potential candidates can be narrowed down, making correlation analysis a much easier task when comparing or associating information from or obtained from the animal recording module and the animal identification module. This improves the system's processing power and efficiency.

[0030] According to embodiments of this disclosure, the animal identification module has a detection range, and the two-dimensional imaging sensor has a field of view, such that the detection range and the field of view do not overlap. The animal recording module and the animal detection module are two independent systems that are combined to identify individual animals. The individual systems may be inconsistent; for example, the clock times of each system may be different or out of sync. The system is configured to compensate for the differences between the two systems and combine information from the two individual systems or modules to provide animal identification.

[0031] According to embodiments of this disclosure, the field of view coverage distance is up to 30 meters, preferably up to 10 meters. The required field of view for the animal recording module can be based on the environment of the system employed. Therefore, the animal recording module has an extended field of view depending on its installation or deployment. For example, an animal recording module can be installed on the ceiling of a barn, and a suitable animal recording module can be used to observe all animals in the area below the barn, thus utilizing an animal recording module with a suitable range large enough to detect animals within its field of view.

[0032] According to some embodiments of this disclosure, the first position and the second position are separated by a distance D, which is up to 50 meters or greater, preferably up to 10 meters. The spacing distance can be determined based on the installation of the system at the installation location. A smaller distance results in a smaller time window and a smaller number of candidate objects (or animals to be associated). Therefore, by adjusting the spacing distance, the efficiency and processing power of the system can be improved.

[0033] According to some embodiments of this disclosure, the detection range of the animal identification module is up to 5 meters, preferably up to 3 meters. The detection range can be based on system settings and desired range. This range ensures that when one or more animals pass by the animal detection module, the unique identifier of the animal can be acquired or detected.

[0034] According to some embodiments of this disclosure, for example, when the system times of the animal recording module and the animal identification module are aligned, the width or offset of the second time window relative to the first time window can be up to 20 minutes, preferably up to 5 minutes. According to other embodiments of this disclosure, for example, when the system times of the animal recording module and the animal identification module are not aligned (e.g., because only one module is using daylight saving time), the width or offset of the second time window relative to the first time window can be up to 1 hour and 20 minutes, preferably up to 1 hour and 5 minutes. The width and / or offset of the second time window is adjustable. The larger the scale of the time window, the more candidate objects or animals can pass through (one or more) cameras and (one or more) RFID readers. Therefore, the system can be configured to avoid excessive differences in time offsets, resulting in an overly wide RFID time window and too many candidate objects. In this way, the time window can be adjusted to improve relevance and ensure processing efficiency while minimizing the amount of data / information the system needs to process.

[0035] According to embodiments of this disclosure, a first time window and a second time window are provided with initial time window values, wherein these values ​​are based on the speed or speed range of at least one or more animals moving between the animal identification module and the animal recording module. In this way, the system is configured to adjust the time windows based on information obtained via the animal recording module and the animal identification module. Therefore, the time windows are based on both systems and the information obtained by these two systems. By providing the ability to adapt or modify the time windows, the system can be configured to improve processing and efficiency. That is, the time windows can be modified to change the amount of candidate object information or animal information detected or obtained within that time window.

[0036] According to embodiments of this disclosure, the initial time window value is based on user input. In this way, the initial start-up value can be input by a user, such as a farmer who has deployed or installed the system on a farm. This value can be based on the locations of the animal recording module and the animal identification module, and the distance between them. For example, if it takes 5 seconds for an animal to walk from one module to another, the window can be set to + / - 20 seconds.

[0037] According to embodiments of this disclosure, the two-dimensional imaging sensor is a color camera and can be time-aware, enabling the allocation of capture time for images or videos. In this way, the present invention can utilize readily available cameras to identify individual animals from a plurality of animals. For example, a 2D camera operating at 20 frames per second can be used. Therefore, this solution is cost-effective as it eliminates the need for dedicated equipment as in existing solutions. Furthermore, by licensing the use of 2D cameras such as those of the present invention (such as readily available 2D cameras), the need for custom hardware cameras and configurations (such as, for example, 3D imaging systems) is avoided. The 2D camera can be mounted in a fixed overhead position or on a movable object (e.g., a drone). For example, both the animal recording module and the animal recognition module can be time-aware, meaning both are capable of allocating capture time for data such as image, video, or recognition data.

[0038] According to embodiments of this disclosure, the animal identification module includes a radio frequency identification (RFID) reader. In this way, when an animal passes within the detection range of the identification module, the system can obtain or detect the animal's unique identifier. The RFID reader can be time-aware, enabling the capture time to be assigned to the unique identifier.

[0039] According to embodiments of this disclosure, data associated with an identifier for each of a subset of animals is received from an RFID device attached to each of the subsets of animals at an RFID reader. For example, the RFID device may be in the form of an RFID tag configured to send identification information when read by an RFID reader. The identification information provides a unique identifier for the animal to which the RFID device is attached.

[0040] According to embodiments of this disclosure, the two-dimensional imaging sensor has a field of view angle between 30 and 150 degrees, preferably between 20 and 160 degrees. The choice of the two-dimensional imaging sensor can depend on the environment of the system setup and the position of the imaging sensor. The field of view angle allows for the acquisition of real-time footage of animals passing through the imaging sensor's field of view, thereby enabling the analysis and processing of real-time video footage for animal identification.

[0041] According to a second aspect of this disclosure, a system for identifying animals from a plurality of animals is provided, the system comprising:

[0042] Animal detection systems, including:

[0043] A device for obtaining video recordings of animals moving through space, the video recordings being captured from a top-down angle by a two-dimensional imaging device;

[0044] A device for obtaining an identifier of an animal moving through space, the identifier being detected by an animal identification module spaced apart from a two-dimensional imaging device;

[0045] The user equipment is communicatively coupled to the animal detection system and is configured to receive information associated with the animal's identity.

[0046] The animal detection system includes:

[0047] At least one processor is configured to perform the method according to the first aspect of the invention.

[0048] This approach provides a device for identifying animals in a continuous and autonomous manner, requiring little or no human intervention. The system can compare visual detection results with a list of common RFID signals and ultimately infer the correct animal RFID signal, matching it with the visually detected animal(s). Furthermore, it can issue alerts to inform the user that identification devices (such as RFID tags) are lost or detached from animals, or identify which animal is a specific animal so the user can assign health insights to that animal. Therefore, a registration phase can be provided where the system learns which animal is a specific animal, i.e., visually identifies that animal, and after the registration phase, insights into the animal's health and welfare status can be provided whenever that animal is observed. Thus, the system can save costs and time involved in performing such tasks. Additionally, alerts can be sent to the user's device to inform the user that an animal identification (e.g., a tag) is not associated with an animal identified in the animal recording module but not by the animal identification module. The system can also inform the user which animal is identified, i.e., the animal is identified, so the user can assign or provide health insights to that identified animal. The system can also alert the user to animals that the RFID system has failed to identify. Therefore, users can quickly resolve any problems that may arise, such as replacing or repairing RFID devices (such as tags).

[0049] According to a third aspect of this disclosure, sorting gates, revolving halls, robotic halls, or herringbone halls include systems according to a second aspect of the invention. The system of the invention can be installed in any location to facilitate the movement of animals from one place to another. For example, the system can be installed or deployed in sorting gates, revolving halls, robotic halls, or herringbone halls. Such arrangements can have aisles and / or passageways for animals to move from one area to another; for example, aisles allow animals to be grouped while ensuring uninterrupted passage, or aisle arrangements guide animals to specific locations for animal milking. The dimensions of the passageways can be designed to allow several animals to move in a predetermined direction and / or reside in specific locations. For example, passageways can be configured to accommodate animals moving one after another or side-by-side in a single or multiple row manner. Passageways can be further divided into multiple sections, each configured to accommodate several animals moving one after another or side-by-side in a single or multiple row manner, or can have multiple exit points and entry points.

[0050] According to a fourth aspect of the invention, an animal farm is provided that includes the system according to the second aspect. The system of the invention can be installed anywhere on the farm. For example, the system can be installed on walkways and / or passageways for animals to move to other areas of the farm, such as a walkway between a feeding station and a milking station. The dimensions of the passageway can be designed to allow a number of animals to move in a predetermined direction. For example, the passageway can be configured to accommodate animals passing one after another or side-by-side in a single or multiple row manner. The passageway can be further divided into multiple sections, each section being configured to accommodate a number of animals moving one after another or side-by-side in a single or multiple row manner. Attached Figure Description

[0051] This application will now be described with reference to the accompanying drawings, in which:

[0052] Figure 1 A system for identifying animals is shown;

[0053] Figure 2a The operation is shown Figure 1 The system's processing;

[0054] Figure 2b It shows Figure 2a Processing with optional animal flow detection;

[0055] Figure 3 The operation is shown Figure 1 The system's processing;

[0056] Figure 4 It shows Figure 1 The installation status of a part of the system;

[0057] Figure 5aThe image shows a segmented image and bounding box of the animal in a video recording frame;

[0058] Figure 5b It shows Figure 5a A segmented image with reference points associated with animal body parts;

[0059] Figure 5c Showing a series of segmented images with reference points;

[0060] Figure 6a The image shows a segmented image and bounding box of an animal with reference points associated with its body parts;

[0061] Figure 6b Showing a series of segmented images with reference points;

[0062] Figure 7 The key regions of interest are shown when extracting features related to animal physical condition scores;

[0063] Figure 8a The process for adding generated identification vectors associated with unknown animals to a database is illustrated;

[0064] Figure 8b The process for uploading one or more unknown identification vectors to a database is illustrated;

[0065] Figure 9 The process for adding identification vectors to the database based on data obtained during the registration phase is illustrated;

[0066] Figure 10 The processing for training a machine learning network to generate recognition vectors and the recognition network is shown;

[0067] Figure 11 The process for identifying animals using a trained recognition vector generation and recognition network is shown;

[0068] Figure 12 The generated identification vectors and RFID tags for the animal are shown, along with selected known identification vector information and associated match scores; and

[0069] Figure 13 It shows a portion of a specific identification vector, as well as a database of known or unknown identification vectors;

[0070] Figure 14 It shows Figure 2a Processing with additional refined time window feedback layout;

[0071] Figure 15 The arrangement for associating data from the animal recording module and the animal identification module is shown;

[0072] Figure 16a and Figure 16b It shows Figure 15 An example arrangement of the system installed on the sorting gate;

[0073] Figure 17a , Figure 17b , Figure 17c and Figure 17d It shows Figure 15 Example arrangement of the system when installed in the revolving hall;

[0074] Figure 18a , Figure 18b and Figure 18c It shows Figure 15 Example layout of the system installed in a gable-shaped hall;

[0075] Figure 19 The operation is shown Figure 15 The system's processing;

[0076] Figure 20 The process of refining the time window is shown;

[0077] Figure 21 A process for identifying one or more animals among a plurality of animals is illustrated according to one aspect of the present invention;

[0078] Figure 22 A system for identifying one or more animals among a plurality of animals is shown according to one aspect of the invention;

[0079] Figure 23 An example layout of a system for identifying animals is shown;

[0080] Figure 24 An example layout of a system for identifying animals is shown;

[0081] Figure 25 An example setup of a system for identifying animals is shown. Detailed Implementation

[0082] The present invention will be described using exemplary embodiments shown in the figures. While the invention has been shown and described with reference to certain illustrated embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of the invention as covered by the appended embodiments.

[0083] Figure 1An example of a system 100 for identifying animals is shown. System 100 includes an animal recording module 110. Animal recording module 110 includes at least one two-dimensional imaging sensor. Animal recording module 110 provides video from the two-dimensional imaging sensor, such as a camera. In some embodiments, the two-dimensional imaging sensor is a color (e.g., RGB) camera. Video can be defined as a series of two-dimensional images or frames.

[0084] Two-dimensional imaging sensors are used to acquire images of animals. The frame rate can be adjusted as needed and may vary in different embodiments as required.

[0085] In some embodiments, processing can occur on a video data stream that has not yet been divided into frames. However, regardless of how the video data is divided, it is essentially equivalent, at least in terms of result, to processing that occurs within data frames. Therefore, for the sake of brevity, only processing of data frames will be discussed.

[0086] The animal recording module 110 may also include sensors for identifying animals. Any sensing technology capable of sharing information about the animals can be used. For example, a camera can identify the number on the tag attached to each animal. In some embodiments, the sensor includes at least one electronic receiver for receiving wireless signals. The at least one electronic receiver for receiving wireless signals may be a radio frequency identification (RFID) reader for receiving wireless signals, such as RFID information from an RFID tag attached to the animal.

[0087] A two-dimensional imaging sensor can be configured to track one or more animals, or to capture images of any animal moving within its field of view. In some embodiments, the two-dimensional imaging sensor can be positioned above a channel to image one or more animals as they pass beneath it. The two-dimensional imaging sensor can be time-aware, enabling the allocation of capture time for images or videos.

[0088] In some embodiments, at least one additional two-dimensional imaging sensor may also be located at a different location than the first two-dimensional imaging sensor. Using multiple two-dimensional imaging sensors helps ensure that any passing animals are recorded. In some cases, an animal is recorded when it crosses a first area of ​​the space, while a second two-dimensional imaging sensor may record it when it crosses a second area of ​​the space, which is different from the first area. The first area may be an area near the entrance of the space, and the second area may be an area near the exit of the space. In an exemplary arrangement, at least one two-dimensional imaging sensor may be located inside a farm building, such as, but not limited to, a barn, and is configured to record it as it moves toward an area within the barn. The two-dimensional imaging sensor may be configured to track one or more animals, or it may be configured to capture images of any animal moving within its field of view. In some embodiments, the two-dimensional imaging sensor may be placed above a passageway in the barn to image one or more animals as they pass beneath it. This arrangement may operate in conjunction with a sensor arrangement above a passageway as described above to enable monitoring of animals at individual locations within the farm environment. The advantage of this arrangement is that animals can be easily confined to the field of view, and the direction of the animal's movement and the ground it crosses during imaging can be known. This helps limit changes in animal movement caused by environmental factors.

[0089] Animal recording module 110 is communicatively coupled to communication link 120. The animal recording module can be communicatively coupled to communication link 120 via wired or wireless connection.

[0090] Animal videos acquired from two-dimensional imaging sensors can be stored in a database on a local or remote computer server. The database can take any suitable form, such as a relational database (e.g., an SQL database) or a fixed-content storage system for storing video recordings.

[0091] The animal recording module 110 can also be configured to transmit real-time video clips of the observed animals to the processing platform 130 via a wired or wireless communication network for further processing and analysis. For example, video clips can be retrieved directly from the animal recording module 110 via the processing platform 130 and processed and analyzed via the communication link 120. Alternatively, video clips can be retrieved from a database for processing and analysis.

[0092] This software application platform can run on a server, such as a remote or local computer application server, and can be configured to determine animal identity based on the processing and / or analysis of video clips. The platform can also be configured to determine other insights about the animal, such as its health status and behavior. An example of a health status insight is lameness, as lameness provides a good reflection of an animal's health, much like a physical condition score. An example of a behavioral insight is an animal's activity level, as assessing activity provides a good reflection of an animal's behavior. Other examples of behavioral aspects can be animal activity levels, and other examples of health status can be animal physical condition.

[0093] When combined with robust animal identification, the inferred health and / or behavioral insights can provide users with important information. For example, this means that a sick animal can be quickly identified even in a large population. Therefore, the system can accurately assess the number of sick animals even if they are present in a large population. Analysis results can be transmitted via wired or wireless communication to user electronic devices 140a–140c, which run software applications that allow end users to view and manage data obtained from the livestock monitoring software platform 130. For example, end users can share the Internet Protocol (IP) addresses of existing camera imaging equipment used in the farm environment for security and / or animal observation purposes, so that the livestock monitoring software platform can analyze and process live video footage. The system provides the ability to autonomously identify one or more animals and extract insights into their status, including activity scores, physical condition scores, and other insights.

[0094] For example, this invention can be used to identify animals among multiple animals in a herd as part of livestock monitoring. By identifying an animal, the behavior and / or health status of that identified animal can also be monitored and associated accordingly. For example, a farmer may be interested in the overall health status of his / her herd. This invention can also, or alternatively, identify animals in the herd and associate welfare indicators of the identified animals with each identified animal. In this way, the farmer can receive notifications about the status of one or more monitored animals, each identified in the herd, along with associated behavioral / health status insights. A computer software application running on devices 140a–140c can also initiate actions to address potential health problems.

[0095] This invention identifies animals based on their visual characteristics and one or more other properties. For example, visual characteristics can be patterned markings on the animal. For instance, individual Friesian cows each have a unique pattern. Numerical or patterned markings can also be added to the animal to artificially provide visual cues for identification. Numerical or patterned markings can be added to the animal by any manual or automated method known in the art.

[0096] The present invention also determines one or more characteristics of an animal based on changes in the animal during video footage. For example, determining an animal's gait can aid in animal identification. One or more characteristics may also include activity determination and / or health status determination.

[0097] These determinations can be delivered to end users, for example, in the form of notifications and optionally in an operational format, such as 140a–140c. For instance, a list of identified animals can be printed or emailed to a veterinarian, or sent to the farmer via text message as an emergency alert. For example, the farmer may receive notifications about the status of the monitored animals on a computer software application running on his / her electronic device and can accordingly initiate certain actions, such as contacting a veterinarian to address one or more of the identified animals, for example, activity problems or weight problems (such as being underweight).

[0098] Figure 2a and Figure 2b Method 200 for identifying animals is shown.

[0099] In step 210, several animals can be observed in a graphical manner. For example, in step 220, when an animal crosses a walkway, data associated with the animal can be obtained through the animal recording module. This data includes video from a two-dimensional imaging sensor. Therefore, the obtained video data may be a dataset associated with the animal's movement through the space or walkway. This dataset is then processed to extract features of the observed animal (230). These features include visual features and one or more characteristics of the observed animal. One or more characteristics may include the animal's movement patterns and / or health score. The system then extracts parameters (230) from the obtained dataset associated with the animal.

[0100] During extraction step 230, the video is segmented into a set of frames. Visual features for each animal are then determined from at least one frame. In some cases, visual features may be extracted from multiple or all frames. Examples of visual features include, but are not limited to, animal pattern markings.

[0101] In addition to determining visual features, during extraction 230, the system identifies instances representing animal shapes in each frame to form detected animal instances. A set of reference points is then identified for each detected instance. The reference points in each set represent the specific location of the animal in each frame.

[0102] Then, by processing at least some of the identified reference point sets in a first module, which includes a trained neural network, one or more characteristics of the animal can be determined. The trained neural network identifies the characteristics of the animal, thereby deriving the determined set of identified reference points.

[0103] Once the animal's visual features and one or more characteristics have been determined, they are combined to generate a parameter set. Therefore, this parameter set includes the visual features and one or more characteristics of the identified animal. Then, for each animal, a recognition vector 240 is generated based on the parameter set. The following will refer to... Figure 12 Let's discuss more details about the generated recognition vectors.

[0104] In step 250, known identification vectors are selected from a first known identification vector database. Each known identification vector corresponds to a unique registered animal. The selection of known identification vectors may include a selection from the entire first database. In other embodiments, the selection will be a subset of the first database.

[0105] In some cases, the selection can be a single known identification vector. In this case, the identification process is more like confirming a selected identity associated with the selected known identification vector.

[0106] Selection can also be based on other signals. For example, acquired RFID information can be used to limit the number of candidate animals. In this case, selection can include only known identification vectors with acquired RFID values. Any signal or method used to select known identification vectors will result in less processing time because the number of candidate known identification vectors to be processed is reduced. An initial range can be added to ensure that the selection of known identification vectors covers all possible candidates. For example, the selection of known identification vectors can include all known identification vectors with RFID values ​​sensed within 5 seconds of the video data acquisition time. This range can then be further refined as matching progresses successfully, as described below. Figures 15 to 20 As stated above.

[0107] Then, a list of 255 matching scores is determined. This is done by comparing the generated identification vector with selected known identification vectors. Any vector comparison method capable of providing matching scores can be used. For example, comparisons can be made based on whether one or more parameters of the generated and known identification vectors are within a threshold or range. Increasing the number of parameters to be compared between vectors increases the likelihood of a match. In other examples, the cosine similarity between the generated and known identification vectors can be used for comparison. In some embodiments, each matching score is determined by evaluating the number of common values ​​between the generated and compared known identification vectors. Common values ​​are not necessarily identical values, but rather values ​​that fall within a range of values ​​being compared. The actual range depends on the nature of the values ​​being compared. Due to the large variation in the underlying measurements, some values ​​may have a large associated range. Essentially, this is a test of vector similarity, and several techniques are known to be available for performing this test.

[0108] Then test 260 is performed to see if at least one of the selected known recognition vectors has a matching score that exceeds the threshold.

[0109] If at least one of the selected known recognition vectors exceeds the threshold, a selection is made from the selected known recognition vectors that exceed the threshold. In some cases, only one selected known recognition vector will exceed the threshold; in such cases, the single selected known recognition vector exceeding the threshold can be used to identify the animal. However, if multiple selected known recognition vectors exceed the threshold, further selection is required. This further selection can be based on any criterion, such as which of the selected known recognition vectors exceeding the threshold has the highest matching score. The result is the determination of a single selected known recognition vector.

[0110] The animals are then associated with a single selected known identification vector.275 Each known identification vector is associated with a unique registered animal. Therefore, the animals in the video can be identified280 as unique registered animals with selected known identification vectors. See below for further details. Figure 14 Further, after a successful match, the system can then refine the RFID scan time window under consideration using the offset between the RFID scan time and the animal appearing under the animal detection sensor. This will result in a reduction in the processing of potential candidates and an increase in the likelihood of a successful match.

[0111] If test 260 shows that no selected known identification vector has a matching score exceeding the threshold, then 290 the animal's identity is determined to be unknown.

[0112] Optionally, before extracting the parameter set from the acquired dataset associated with the animal in step 230, the continuous movement of the animal across the specified space can be verified, such as... Figure 2b As shown. For example, flow detection processing can be provided to detect whether the movement of an animal across a space is continuous or obstructed during video recording. Generally, the natural movement of an animal across a space may be interfered with by obstacles such as other animals, which may lead to biased results. To improve the accuracy of animal identification, analysis can be performed only on intervals showing the animal moving freely across the passage. In other arrangements, it may not be necessary to monitor the free flow of animal movement, such as when only a single animal is monitored within the field of view of a two-dimensional imaging sensor. Preferably, video intervals showing an animal approaching another animal (which may obstruct its movement) or showing an animal stopping completely due to an animal or other obstacle in front of it can be ignored from the identification analysis. Analysis of the animal's activity characteristics is performed only when the animal is moving freely to improve the analysis results. The animal can still be identified by analyzing visual features and physical condition. A determination of continuous animal movement can be obtained by analyzing the bounding boxes generated by the segmentation algorithm. If sufficient free movement intervals cannot be extracted, the animal is not analyzed. The use of a flow detection system has the advantage that animal identification can be selectively performed on videos showing the animal moving freely across a passage. If the flow detection system determines that the movement of animal 222 is interrupted, the interval of the detected animal is ignored from the analysis, and in step 224 the process moves to the next animal.

[0113] Figure 3 A method 300 for identifying animals and calculating at least one movement characteristic of an animal is illustrated. Several animals 305 can be detected from data obtained from at least one two-dimensional imaging sensor, such as those described above, in an illustrated manner. Animals can be detected in each frame. Animals can be detected using, for example, object detection algorithms. For each detected animal 310, data associated with that detected animal is processed to calculate 315 visual features and 320 reference points. Optionally, the data is processed to calculate 325 body condition attributes of the detected animal, such as the amount of fat coverage at the animal's body location. Reference points can be used to assist in the assessment, helping to select appropriate parts of the animal's body for evaluation.

[0114] Optionally, the system can also be configured to receive, 330 reads, the radio frequency identification (RFID) signal of the detected animal. Optionally, a mask, such as a segmentation mask, can be used to distinguish animals from multiple animals that may be close to each other. Steps 315, 320, and 325 can be implemented for the animal associated with the segmentation mask. In this way, the system is able to clearly distinguish each corresponding animal from multiple animals, thereby ensuring that any calculation of animal attributes is assigned to the correct animal. In the case of monitoring only one animal, such a segmentation mask is not needed in such instances. The system can then use one or more calculated visual features 315, calculated animal reference points 320, calculated animal body condition scores / attributes 325, or RFID reads and assign these to the tracked animal 340 being monitored by the imaging sensor. The system can be configured to simultaneously track animals 355 using, for example, an animal tracking algorithm and assign information based on the detected animals 310 to the tracked animals. Once the system assigns information to the tracked animals, it can be further configured to: calculate 345 movement characteristics for each tracked animal 340, and then determine its identity 350. In this way, the system not only determines the identity of the tracked animal, but also supplements that animal's identity with information related to its health and behavior in the form of the calculated movement and / or physical condition characteristics 345.

[0115] Figure 4A possible arrangement 400 for recording the capture of animals 430a to 430n (such as cattle, pigs, horses, etc.) is shown. One or more animals may pass through the walkway in an orderly manner. For example, animals labeled #C100, #C101, #C102, #C109, #C108, #C110, #C112. In other instances, the animals may end up passing through the walkway in a chaotic manner and may pass each other. Therefore, assessing the order of the animals by recording and / or readings when the animals pass through the lower half of the walkway may differ from assessing the order by recording and / or readings that can be obtained when the animals pass through the upper half of the walkway in the manner illustrated in the example. The animals are recorded using video from a two-dimensional imaging sensor 460a. Recordings of the animals at other locations along the entire walkway are captured using another two-dimensional imaging sensor 460b. The two-dimensional imaging sensors 460a and 460b may be collectively referred to as 460. Additional recordings of the animal are also received via at least one electronic receiver 470 of the animal recording module for receiving wireless signals. The electronic receiver may be, for example, an RFID reader, and the wireless signal may be, for example, RFID information transmitted from an RFID tag. In some applications, a two-dimensional imaging sensor 460 is mounted in a top position, and a building 405 in the farm environment (having an entrance 410 and an exit 420) can be used to capture video recordings. The imaging sensor may be a color camera. The imaging sensor may also be an existing camera used for other purposes, such as security and / or animal observation in a farm environment. Once video footage is acquired, an image processing device can extract a set of reference points associated with specific locations on the animal's body from the animal recording module. Other visual features attached to the animal (such as identification numbers) and / or other visual features of the animal (such as body patterns) may also be recorded. One or more characteristics of the animal are determined by processing at least some of the reference point sets using a trained neural network. In this way, a set of parameters including visual features and one or more characteristics of the determined animal (one or more animals) can be generated.

[0116] In some embodiments, the electronic receiver 470 may be located near or adjacent to the position of the two-dimensional imaging sensor 460, at a height suitable for receiving wireless signals for animal identification. When an animal passes by the receiver 470, a wireless signal for identifying the animal can be transmitted from the wireless sensor 440 attached to the animal.

[0117] For example, the wireless signal used to identify the animal may be an RFID tag attached to at least one location on the animal's body, the RFID tag being configured to emit a wireless signal containing information associated with the animal to which the tag is attached. This wireless signal is received by an electronic receiver 470, which in this example may be an RFID reader, configured to receive the wireless signal and obtain information about the identified animal from the RFID tag output. Therefore, the RFID tag is communicatively coupled to the RFID reader so that when the animal passes the RFID reader and moves across walkway 415, the associated RFID information of the animal is sent to the RFID reader. The associated RFID information of the animal may include a unique identifier for the corresponding animal.

[0118] Once video footage is acquired, the processing device extracts a set of parameters from the acquired data associated with the animal moving through space. This data includes video from a two-dimensional imaging sensor.

[0119] Extraction may include the above references Figure 2a The steps explained are as follows: In each set of reference points shown in each detected instance, each reference point has x-coordinates and y-coordinates. The x and y coordinate values ​​of each reference point can be extracted to determine the position of each reference point, thereby determining the movement of each corresponding body part in the set of frames. Based on the extracted x-coordinate values, y-coordinate values, or x and y coordinate values ​​corresponding to each frame's set of reference points, the processing platform can determine a characteristic score that indicates characteristics of the animal, such as movement patterns.

[0120] The relative changes in the positions of reference points within frames and / or between frames can allow for the determination of the following insights. Changes in the positions of reference points between frames can be used to analyze animal activity, providing behavioral insights to aid in animal identification. Changes in the positions of reference points within frames can be used to assess the degree of lameness in animals, providing health insights to aid in animal identification.

[0121] Changes in reference points can help locate regions in image data within a single frame to assess excessive fat coverage for body condition scores, providing health insights to aid in animal identification. Therefore, changes in the coordinates of reference points can indicate problems within an animal, such as activity levels, which can be used to identify specific animals.

[0122] Figure 5aFurther illustration details the processing of animal-related data obtained from the animal recording module. The figure shows a segmented image 500 obtained from the animal recording module 110 during extraction step 230. An instance of an animal body 510 is detected in the segmented image 500. The animal instance 510 is detected by an object detection algorithm, which creates a bounding box 511 around the detected instance 510. Then... Figure 5b Example 510 illustrates the identification of reference point sets 510_1 to 510_n by processing detected animal segmentation images 400. Figure 5b In the example shown, twelve reference points (shown as circles) were identified along the animal's body, from head to tail. Then... Figure 5c The image set 500_1 to 500_n of animal 510 is shown, one frame per image, with identified reference points. Therefore, this is the reference point set.

[0123] EP application EP20183152.6, filed on June 30, 2020, describes an exemplary process that can be used to determine a reference point on the location of an animal instance.

[0124] In summary, this exemplary process performs the following steps:

[0125] i) Provide data input, such as several videos of animals from different farms. For example, a dataset of three hundred video recordings of animals moving across a predetermined space could be provided to train the system's standard ID classification network to identify reference points on the animals. The ID classification network could be a neural network, such as ResNet50, ResNet101, or other types of convolutional neural network (CNN) architectures. During the training phase, each reference point can be manually labeled. Furthermore, the classification network can be trained on historical video recordings from the same animal to handle animals exhibiting naturally anomalous movements, avoiding false positives.

[0126] ii) Detect reference points, which can be done using the ID classification network trained in step i). The standard classification layer in the convolutional neural network can be replaced by deconvolutional layers to scale the animal's feature map into a heatmap representing the probable region for each reference point, which is obtained by training on the data from step i).

[0127] iii) Extract the x and y coordinates of each reference point in each frame. The x and y coordinates of the reference points can be considered as the regions with the largest amplitudes in the heatmap generated in step ii). These reference points can then be associated with a specific instance of each animal using a pre-computed segmentation mask.

[0128] Using at least twelve reference points ensures processing efficiency while minimizing the amount of data / information that the processing equipment needs to process, thereby improving the ability to identify problems with animal movement patterns (such as lameness) or physical conditions (such as underweight animals).

[0129] In an alternative setup, reference points are extracted via an algorithm trained to identify the shape of an animal within the field of view of a 2D imaging sensor. These reference points can then be combined with at least one or more of the following: an algorithm for determining the activity level of one or more monitored animals, an algorithm for determining the Body Condition Score (BCS) of one or more monitored animals, or as an auxiliary feature set for identifying the monitored animal. For example, the identification algorithm could not use reference points but instead feed data of a specific animal from a 2D imaging sensor into a ResNet50 neural network. The outputs from one or more of the reference point algorithm, activity algorithm, and body condition scoring algorithm are then used to support the identification algorithm and provide the overall feature vector output for identifying the corresponding animal. This algorithm could be, for example, an algorithm based on the "You Only Look Once-Pose" (YOLO pose).

[0130] One or more characteristics of an animal can also be determined using convolutional neural networks. For ease of processing, the set of reference points can be formed into a multidimensional array. To form the array, the x and y coordinate values ​​of each reference point are extracted, generating a first array and a second array for each individual frame. The first array contains the x-coordinate values ​​of the reference points in each frame, and the second array contains the y-coordinate values ​​of the reference points in each individual frame. The values ​​from the first and second arrays are then combined to create a multidimensional array of x and y coordinate values, the size of which is defined by the number of individual frames and the number of reference points.

[0131] A convolutional neural network is trained to determine one or more characteristics of an animal based on movement obtained from corresponding reference points. In one example neural network, the steps for determining an animal's activity score include: performing successive convolution and pooling iterations on the x and y coordinate values ​​of an extracted set of reference points corresponding to each individual frame of the input data. The initial values ​​of the kernels and filters are randomly chosen and progressively improved by training the network on the provided data. After each convolution, batch normalization and rectified linear unit (ReLU) activation can be performed. After successive convolution and pooling iterations, average pooling can be performed on the output. Linear activation can be performed on the output of the average pooling operation. After linear activation, a score indicating the animal's activity score is generated. In this manner, initial filtering and max pooling, performed through successive convolution and pooling iterations, are passed along the time axis, which can indicate movement across the input matrix from one reference point to another, such as from head to ear to tail, etc. Final filtering is then performed along the feature axis, which indicates movement from the first frame to the second frame to the third frame, etc., before being passed to a final fully connected layer with linear activation in the image processing platform. The final result of the analysis is an activity score, such as a floating-point number between 0.0 and 100.0, indicating the activity level. For example, a score of 0.0 might indicate a high level of activity in the animal, while a score of 100.0 might indicate a low level of activity. These values ​​can be converted to other metrics using thresholds. For example, 0.0 can be converted to a ROMS score of 0, which might indicate no lameness or a low level of lameness, while 100.0 can be converted to a ROMS score of 3, meaning a high level of lameness in the animal. In this way, insights based on scores determined by CNNs can be delivered to the end users of the system, for example, in the form of notifications and optionally in an actionable format, such as printing out animal lists or sending them to veterinarians via email, or sending text messages to farmers to issue emergency alerts.

[0132] In some cases, the aforementioned method of evaluating activity scores without using a neural network can be used to train the network. Alternatively, activity scores can be labeled by experts to provide training data. The activity score information can then be used as an auxiliary dataset to aid in animal identification, while the primary dataset might be data from a 2D imaging sensor of a specific animal fed into a ResNet50 neural network to identify the animal. In this way, one or more features, such as activity scores, supplement the animal identity determined via, for example, a ResNet50 neural network.

[0133] Figure 6a and Figure 6b An example of an animal body 610 is illustrated, which can be constructed using methods similar to... Figure 4 b and Figure 4Method c was detected in segmented image 600. In the example arrangement, 16 reference points are shown in circles, which are identified along the animal's body from head to tail. Figure 6b The image set 600_1 to 600_n, segmented from animal 610 for each frame, is shown, with 16 reference points. Increasing the number of reference points is advantageous because it provides a method for generating regions of interest, which health and / or behavior algorithms can then use to provide insights.

[0134] Figure 7 The illustration shows an instance of an animal body 710 that can be detected in the segmented image 700 and used for body condition scoring analysis. The animal instance 710 can be detected by an object detection algorithm, which uses a similar approach to a reference image. Figure 4 a to Figure 4 The method described in c creates a bounding box around the detected instance. Figure 7 The example 710_1 to 710_n, identified by processing an instance 710 of a detected animal segmentation image 700, is shown. Figure 7 In the example shown, 16 reference points (shown as circles) were identified along the animal's body from head to tail. An object detection algorithm analyzes the entire animal body and can be configured to utilize these reference points to generate regions of interest 720a to 720d. This algorithm can then provide health and / or behavioral insights of the animal within one or more corresponding regions of interest. In the example setup, an algorithm such as the "You Only Look Once" (YOLO) algorithm can be used to generate the aforementioned bounding boxes for each animal. These bounding boxes can then be passed to a reference point algorithm so that it can identify reference points from the corresponding animals detected within the bounding boxes. These reference points are then used to determine insights into the animal, such as, but not limited to, health and / or behavioral insights. More specifically, Figure 7 The diagram illustrates the key regions of interest that the body condition scoring neural network will focus on to determine the body condition score. These are the coccyx and tail region 720a, the hamate and posterior vertebral region 720b, the ribs and mid-vertebral region 720c, and the anterior vertebral region 720d. Analysis of these regions will yield an estimate of the animal's overall fat coverage.

[0135] Figure 8a An example method 800 for managing animals identified as unknown is shown. The identification vectors associated with the unknown animals can be stored in a database for future cross-referencing. After generating the identification vectors of the observed animals, according to... Figure 2aIn step 260, the animal is identified 290 as an unknown animal. The generated identification vector for the unknown animal is added 292 to a second database containing unknown identification vectors, i.e., unknown or unregistered animals. In this way, the second database can store records of unknown identification vectors. In other words, the generated identification vectors of animals whose visual features and one or more characteristics are not sufficiently similar to known identification vectors are stored for future use. This will then allow for the future registration of currently unknown animals.

[0136] like Figure 8b As shown, the system performs subsequent processing 810 on the unknown identification vector database, comparing more than 297 identification vectors to identify similar identification vectors in the database and to identify matching identifications obtained from another data source (such as RFID tags associated with animals). The system then associates these previously unknown identification vectors with specific animals. The unknown identification vectors identified along with the animals are then added to the first database 298. In this way, the system can automatically increase the number of known identification vectors in the system. In other words, the system can automatically expand to include new animals. This means that the system can be integrated into existing systems, such as those that use RFID and RFID tags attached to animals to identify them. Existing systems may not be able to provide identification for every single animal due to problems such as RFID tags detaching from animals; therefore, when an animal passes by an RFID reader, it cannot be identified due to the lack of an RFID tag. Accordingly, this system complements such existing systems because it is able to adapt to identify new animals, and thus the system can automatically expand to include animals that may otherwise be unidentified, for example, due to the lack of ear tags for RFID identification, or because they were only recently incorporated into the animal population (e.g., in a farm setting) and have not been tagged. Furthermore, the identified animals can be used to further fine-tune the neural network 299 to improve its ability to distinguish different animals in the coming days, as farm conditions (such as weather and light) will gradually change over time.

[0137] Figure 9An example method 900 for identifying one or more animals is shown. The method includes: registration 910 for registering one or more animals into a system; generating identification vectors in step 920 for generating identification vectors from a set of parameters generated for each animal; and adding the identification vectors 930 to a first database. As shown in the arrangement, the registration step can be adapted to allow animals to be registered into the system by preparing a labeled dataset of one or more animals, thereby ensuring that the identification vector of each animal is assigned to the correct animal, thus avoiding misidentification. In this way, a database of known identification vectors can be provided, such that each known identification vector corresponds to a unique animal, so that it can be compared with observed animals in the future to identify whether an observed animal is a known animal.

[0138] Figure 10 Treatment 1000 for registered animals is shown. For example, refer to the discussion above. Figure 4 During the registration step, one or more animals 1010, 210 under observation may move along walkway 415. Records of one or more animals 1020 are then obtained from animal recording module 110. The identification of one or more animals is also obtained from additional source 470. For example, the actual identification of an animal can be determined by an RFID reader configured to receive a radio frequency identification (RFID) tag 440 attached to the animal. Alternatively, identification can be obtained by visually reading a numbered tag 450 on the animal. Information is recorded in the order in which one or more animals pass by two-dimensional imaging sensor 460. Arrangements can be implemented to ensure that the order of one or more animals does not change during registration processing; for example, the walkway size can be designed so that one or more animals pass through the walkway in a single file, thus ensuring that one or more animals cannot overtake each other while crossing the walkway, thereby enabling tracking of one or more animals.

[0139] The registration data can then be stored in a computer-readable format (e.g., JavaScript Object Notation (JSON), Extensible Markup Language (XML), etc.) and can include an animal identification list ordered by the animal passing through animal record modules 110, 460, and 470. The video clips associated with the registration can then be passed to the video processing algorithm of processing device 130. The video processing algorithm can then be adapted to perform at least the following tasks:

[0140] i) Each video frame is fed into a previously trained object detection network 1030, which creates a bounding box for each animal instance / image detected in each video frame and optionally creates a segmentation mask for each animal instance / image detected in each video frame. The segmentation mask enables the object detection algorithm to focus on the corresponding animal within the created bounding box, for example, when another animal or overlapping portions of multiple animals appear within the created bounding box of the corresponding animal.

[0141] ii) The bounding boxes generated for each animal are then passed to the target tracking algorithm 1030, which groups multiple bounding boxes across frames, generates identification vectors, and assigns identifications obtained from animal identification devices (such as RFID tags) corresponding to the individual animals in the segment. References will be made below. Figures 14 to 20 Further description provides a further exemplary arrangement in which, once a group of animals has been processed within a time window, the system is configured to collect all tracked bounding boxes and all ID reads from any ID device (such as an animal recording module, which could be a camera, and an animal identification module, which could be an RFID reader). By comparing the collected reads, the identification vectors, and the number of times the identification module has identified or detected an ID nearby whenever an animal with those vectors is located below the camera, the system can extract the most common animal identifications.

[0142] iii) For each animal detected in the video recording, at least ten bounding boxes can be selected from the corresponding frame number extracted as an image.

[0143] The detected animals(s) can then be sorted according to their order of appearance in the fragment and synchronized with the actual animal identification IDs 1050, 440 obtained earlier from supplementary source 470. The synchronized identification of the registered animals, together with the images extracted in step iii), can be used as training data for determining the matching score in step 1060.

[0144] The trained vector generation and recognition network 1070 processes new video segments according to each of steps (i) to (iii) described above. For each animal detected in the video recording, each image generated in step (iii) can be passed through the trained vector generation and recognition network 1070 to generate an array of recognition vectors for each registered animal. These recognition vectors can then be averaged from several images (e.g., five, ten, or more) extracted from the video frame at step (iii) above to generate a recognition vector for the detected animal. Once all segments have been processed, the recognition vector for each animal is post-processed to assign the most probable recognition to the animal based on the recognition vectors of all other animals. By post-processing the recognition vector generated for each animal, the assignment of the same ID to multiple animals can be prevented. Furthermore, post-processing allows for the inspection of animals that may not be registered in the system or whose registration data may be problematic. In this way, by assigning recognitions through animal identification means, the system provides a method for easily detecting individual animals among multiple animals, assigning a unique identification code to each animal, thereby avoiding potential problems such as misidentification and thus preventing the assignment of recognitions to the wrong animals. Therefore, based on the registration data, the system can identify each animal without human intervention, such as reading RFID tags.

[0145] Figure 11 The diagram illustrates a process 1100 for identifying animals based on a previously trained vector generation and recognition network 1070. For example, it can be performed according to a reference... Figure 4 The arrangement discussed herein provides an arrangement for capturing images of animal 1110. This arrangement can be configured to capture multiple animals or individual animal records 1120, 460, 110. These records can then be analyzed by a processing platform 130. The processing platform can then utilize object detection and tracking algorithms 1130 (such as those discussed previously) to detect and track one or more animals in the records. A trained vector generation and recognition network 1070 can then process the data of the one or more animals detected and tracked by the image processing platform. In this way, new fragments acquired by the animal recording device can be processed by the illustrated arrangement to evaluate the identity of one or more animals in the new fragment using the previously trained vector generation and recognition network 1070 in step 1140. This provides a way to check animal identifications that may not yet be registered in the system. Advantageously, the system continuously updates animal identification information, thereby training and learning, as the system is able to autonomously analyze the information of the individual animals acquired.

[0146] Figure 12The arrangement of individual animals 1210 can be identified using a trained vector generation and recognition network. Several bounding boxes exist for the same animal from different consecutive frames 1210_1 to 1210_n, similar to... Figure 5c , Figure 6b The bounding boxes shown are generated by a trained vector generation and recognition network 1070 (as referenced above). Figure 10 and Figure 11 The data is transmitted to extract animal features (e.g., reference points, patterns, etc.) via, for example, but not limited to, deep learning models (such as ResNet50, etc.), and the reference points are extracted to generate a vector 1220 including identification information. The identification vectors are then averaged to form a generated identification vector 1230. In addition to feature extraction, the RFID reader 470 obtains RFID information from the animal's RFID tag 1212. Alternatively or additionally, other sensor technologies for providing animal identification may be used. Wireless signal information is extracted from the wireless transmitter 1212 to form a wireless sensor readout vector 1240. Each animal detection, detection 1, detection 2, etc., has an associated identification vector and a wireless signal readout vector. In other words, the system can perform several detections on the animal and obtain a dataset associated with the animal's movement through a space or walkway via the animal recording module. Using a processing device platform, a set of parameters associated with the obtained dataset is extracted from the animal recording module. The system can be configured to generate an identification vector based on the set of parameters for each detection, as shown in 1250. The system can be configured to select known identification vectors from a first database of known identification vectors, where each known identification vector corresponds to a unique animal. The system can determine a list of matching scores by comparing the generated identification vector with the selected known identification vectors, as shown in 1260. Any vector comparison method capable of providing matching scores can be used. For example, comparisons can be made based on whether one or more parameters of the generated and known identification vectors are within a threshold or range. Increasing the number of parameters to be compared between vectors increases the likelihood of a match. In other examples, the cosine similarity between the generated and known identification vectors can be used for comparison. In yet another example, each matching score is determined by evaluating the number of common values ​​between the generated and compared known identification vectors. Common values ​​are not necessarily identical values, but rather values ​​that fall within a range of the values ​​being compared. The actual range depends on the nature of the values ​​being compared. Due to the large variation in the underlying measurements, some values ​​may have a large associated range.

[0147] In response to determining that at least one known identification vector has a matching score exceeding a threshold, at least one known identification vector is selected, and the selected known identification vector is used to identify the animal, as shown in the figure, where wireless signal information is associated with the animal number (a cow in the illustrated example of 1260), as well as the matching score and identification vector information.

[0148] As shown in the figure, a series of signals, such as wireless signals from RFID tags, can be detected. Each signal includes a time and is associated with a unique animal 1210. Selecting a known identification vector from a first database of known identification vectors may further include determining the time when the animal recording modules 110, 460, and 470 acquire data 1230 and 1240, and the time range within the determined time range. The series of signals can be filtered to select only those signals within the time range. In the illustrated example, the series of signals are 1561, 1734, 2631, and 1934 within the time range t1 to t4. The signals are filtered to select those signals 1265 within the time range t2 to t3 from the series of signals. From this time range, the known identification vector of the unique animal with the selected signal having the highest matching score is selected, thereby determining the identity of the observed animal.

[0149] Figure 13Table 1300 illustrates a set of parameters including visual features, health and behavioral characteristics, and wireless signal identification information. This table contains information associated with the visual features of an animal of length n1, information associated with the reference point of an animal of length n2, information associated with the movement pattern of an animal of length n3, information associated with the health score (such as a physical condition score) of an animal of length n4, and information associated with the wireless signal of an animal of length n5. The lengths n1 to n5 can vary in different embodiments and do not necessarily need to be the same size. The reference point, movement pattern, and health score information are examples of one or more characteristics of the animal discussed earlier. The activity and physical condition score information are further examples of one or more characteristics of the animal discussed earlier. Therefore, the table may include visual features, activity characteristics, physical condition characteristics, and wireless signal ID. The identification vector information of each frame in the animal's record is averaged and stored in a database. Based on the matching score information discussed above, the identification vector information is either stored in the known animal database 1310, thus being regarded as a known identification vector corresponding to the unique animal 1320, or stored in the unknown identification vector database 1330, thus being regarded as an identification vector not corresponding to the unique animal. Then, in response to the animal identification determination process discussed above, the known animal database and the unknown identification vector database are continuously referenced and updated. Accordingly, information about each known and unknown animal is retained in the respective database. More specifically, during continuous animal monitoring, for unknown animals whose identities have not yet been determined, after processing the animal and determining its identity as a known animal, if the confidence level of its identity exceeds a predetermined threshold, it is promoted to the known animal database. Conversely, when the confidence level of the identity of a previously known animal falls below the predetermined threshold, the system reconsiders or identifies the animal as an unknown animal and removes it from the known animal database to the unknown animal database. The system automatically and continuously updates the database based on the analysis and determination of whether the monitored animal is a known or unknown animal. In other words, the system trains and learns based on the monitored animals, but never forgets them, and therefore continuously updates itself based on acquired and previously acquired information, whether the animal is newly introduced, part of a new herd, or part of an existing herd. Due to the autonomous nature of the system, it reaches a certain level of confidence in the monitored animal and assigns it to the known database, and when the confidence level decreases, the system determines that the animal is unknown and assigns it to the unknown database.

[0150] Figure 14 A method 200a for identifying individuals is illustrated. Method 200a is shown in the diagram. Figure 2aThe method shown in 200 is similar, except that it includes step 295. Each of steps 210, 220, 230, 240, 250, 255, 260, 270, 275, 280, and 290 is similar to the one described above. Figure 2a The description is the same as in the previous section. In the arrangement shown, if a match is successful, the system can refine the RFID scan time window 295 under consideration using the offset between the RFID scan time and the time the animal appears under the animal detection sensor or video / animal recording module 110. This arrangement not only captures video of animal identification information from a device such as RFID during a specific time window, but also captures identification information or RFID information along with video over a sustained period. When the system processes the data, a time window is generated; that is, a time window is used to process a portion of the captured data. This time window can be adjusted and refined to improve the processing and determination of animal identity. This will reduce the processing of potential candidates and increase the likelihood of a successful match.

[0151] In a similar way, Figure 2b The arrangement can be further modified, incorporated into step 295 after step 280, and fed back to step 250, so as to incorporate a refined or adjusted time window for receiving animal identification information at the animal identification module, thereby improving the processing of potential candidates and increasing the likelihood of a successful match.

[0152] Figure 15 An exemplary system arrangement 1400 for determining animal identity is shown. In this embodiment, one or more animals 1405_1 to 1405_6 can be observed by at least one animal recording module 1410 (e.g., a camera) when the animals(s) are within the field of view 1415 of the animal recording module 1410. In this way, visual information of the animals can be obtained as they pass through the field of view of the animal recording module 1410, as previously described. The data includes video from a two-dimensional imaging sensor. The two-dimensional imaging sensor may be a color camera. The animal recording module 1410 is located at a first position 1420L. image The two-dimensional imaging sensor can have a field of view between 30 and 150 degrees, preferably between 20 and 160 degrees. The animal recording module can be configured to identify each animal within a first time window and associate each animal 1405_1 to 1405_6 with the timestamp corresponding to the detection point of that animal.

[0153] At least one animal identification module 1440 detects one or more animals or subsets of animals 1405_1 to 1405_6. Animals may not follow the necessary path, or animal identifiers (e.g., RFID tags) may detach from the animals, and therefore not all animals observed by the animal recording module 1410 can be detected or identified at the animal identification module 1440. The animal identification module is configured to acquire data associated with the identifier of each animal in the subset of animals 1405_1 to 1405_6 at a second time within a second time window. The identifier may be (e.g., but not limited to) RFID information from an RFID device (e.g., an RFID tag) attached to (one or more) animals, and the animal identification module may be (e.g., but not limited to) an RFID reader. At least one animal identification module has a detection range 1445 for detecting information associated with (one or more) animals 1405_1 to 1405_6. The detection range may be up to 5 meters (5m), preferably up to 3 meters. Understandably, depending on the type and location of the RFID reader deployed, the detection range may be greater than or less than 5 meters. For example, other technologies such as active tags can also be used, with ranges up to 100 meters (100m). At least one animal identification module 1440 is located at the second position 1430 L. identifier The first position 1420 and the second position 1450 are spaced apart. The first position 1420 and the second position 1450 may be separated by a distance D, which can be up to 50 meters, preferably up to 10 meters. It is understood that the distance D may be greater or less than these distances and can be determined based on the system's installation. A smaller distance results in a smaller time window, thus reducing the number of candidate objects or animals.

[0154] The system can be configured to operate when the animal recording module and the animal recognition module are physically close to or in the same location. In some arrangements, the detection range and field of view do not overlap. In some arrangements, the detection range and field of view may partially overlap, for example, by as much as 10% to 20%, or they may completely overlap, or they may not overlap at all. It is understood that the detection range may or may not be within the field of view, depending on the installation arrangement. At least one animal recognition module 1440 can be configured to recognize individual animals within a second time window and associate each animal 1405_1 to 1405_6 with the timestamp corresponding to the detection point of that animal.

[0155] In the illustrated embodiment, the animal recording module 1410 and the animal identification module 1440 continuously capture / acquire data. The system can be configured to process the acquired data and artificially overlay or place time windows on the acquired data to refine or adjust the processing or analysis of this information as part of determining the identity of one or more animals 1405_1 to 1405_6. For example, once the animal recording module observes or detects an animal, the time window for the animal identification module could be camera time + / - 300 seconds, i.e., a 10-minute time window; or the time window could be camera time + 600 seconds and camera time - 300 seconds. The + / - values ​​of the time window can be adjusted or refined to different values ​​over time based on how well the system adapts to the acquired data. The width of the time window can be either reduced or increased, or, for the respective modules, the time windows can be moved relative to each other.

[0156] In the illustrated embodiment, a correlation analysis can be performed on each animal identified by at least one animal recording module 1440 within a first time window and an animal identified by at least one animal identification module 1440 within a second time window. The time windows are configured to be adjusted to maximize the correlation between the two independent systems 1410 and 1440, thereby increasing the probability of a successful animal match(s). Thus, there are two independent systems: one system knows that a known identity appears at a known location at a known time, and the other system knows that an unknown identity appears at the location of the animal recording module at another time.

[0157] Over time, the system can compare the visual detection results from the recording module with a common list of RFID signals, or with RFID reads from RFID devices (e.g., RFID tags) obtained from the identification module, and ultimately infer the correct animal RFID signal and match it with the visually detected animal. Thus, the combined use of two independent systems identifies one or more animals. The system can be configured such that, during the registration phase or processing, the system requires at least four occurrences or at least four instances of observing or seeing the animal before it can identify or register or enroll the animal in, for example, a database of known animals as described above. Advantageously, these two independent systems overcome any time-related problems arising from the use of two systems such as RFID readers and cameras. Some systems may become out of sync due to daylight saving time or differences in computing device time, resulting in a desynchronization between the animal recording module and the animal identification module. These systems and methods address such synchronization problems due to the arrangement of animal image retrieval and identification.

[0158] In one example, the system and method obtain a list of read results and then refine it over an adjusted time period. This allows the system to determine, based on the cow's visual behavior and / or health status, that it has previously seen a specific animal (e.g., cow #1). However, based on what's seen in the animal record module, there might be another 20 candidates. The system can determine that 7 of these 20 candidates are similar, thus narrowing the number of candidate animals down to 7. The system knows it has seen cow #1 twice and that it is one of these seven candidates. This candidate list continues to be refined until the system finally reaches a stage where the candidate cow can only be cow #1. Depending on the farm's setup and the system's settings for the farm, the system may need up to four days to lock in or be able to register or enroll the animal's identity. Alternatively, an exclusion process can be considered. For example, if the system initially determines that the animal could be cow #1 or cow #2, but the system later assigns cow #2, then the system knows that the animal currently being identified cannot be cow #2, therefore it must be cow #1, and thus it will identify the animal as cow #1. Therefore, the system continuously iterates through continuous evaluation to determine that the animal can no longer be the cow, but must be any of the remaining candidates.

[0159] Furthermore, through the entry process, or when any new animal is introduced to the farm, or when the identifier (such as an RFID device) on a particular animal is replaced, the system can identify that animal / the animals in the future based on data obtained from the animal recording module and the animal identification module. This arrangement is not limited to the animal recording module 1410 detecting one or more animals, and then the animal identification module 1440 detecting one or more animals after the animal recording module 1410; the arrangement can also enable the detection of one or more animals at the animal identification module 1440, and then at the animal recording module 1410.

[0160] Figure 16a and Figure 16b The diagram shows... Figure 15 The example arrangement described herein is when the system is installed or deployed, for example, at sorting gate 1510. Figure 16a In arrangement 1500a, a sorting gate 1510 is shown to have one inlet path and two outlet paths. The number of inlet or outlet paths of the sorting gate may be more or fewer than shown in the figure. It is understood that other inlet and outlet path arrangements may be considered. Animals 1405 are shown on each inlet and outlet path of the sorting gate. In the arrangement shown, an animal recording module 1410 may be located along one or more outlet paths of the sorting gate 1510, while an animal identification module 1440 may be located at the sorting gate 1510. Figure 16bIn the arrangement 1500b shown, the animal recording module 1410 may be located along the entrance path leading to the sorting gate 1710, while the animal identification module 1440 may be located at the sorting gate.

[0161] Figure 17a , Figure 17b , Figure 17c and Figure 17d The diagram shows... Figure 15 The system described is installed or deployed in an example arrangement, such as in revolving hall 1610. It is understood that other entrance and exit path arrangements can be considered. The entrance or exit paths in revolving hall 1610 may be more or fewer than those shown in the diagram. Figure 17a In arrangement 1600a, a rotating hall 1610 is shown to have an entrance path and an exit path. One or more animals 1405 are shown in each of the entrance and exit paths of the rotating hall. In the illustrated arrangement, an animal recording module 1410 may be located along the entrance path of the rotating hall 1610, while an animal identification module 1440 may be located on the entrance path side of the rotating hall 1610 or near the rotating hall 1610. Figure 17b In the arrangement 1600b shown, the animal recording module 1410 can be located along the exit path of the rotating hall 1610, while the animal identification module 1440 can be located on the entrance path side of the rotating hall 1610 or near the rotating hall 1610. Figure 17c In the arrangement 1600c shown, the animal recording module 1410 can be located along the entrance path of the rotating hall 1610, while the animal identification module 1440 can be located on the exit path side of the rotating hall 1610 or near the rotating hall 1610. Figure 17d In the arrangement 1600d shown, the animal recording module 1410 may be located along the exit path of the rotating hall 1610, while the animal identification module 1440 may be located on the exit path side of the rotating hall 1610 or near the rotating hall 1610.

[0162] Figure 18a , Figure 18b and Figure 18c The diagram shows... Figure 15 The example arrangement described herein is for when the system is installed or deployed, for example, in a herringbone hall 1710. It is understood that other entrance and exit path arrangements may be considered. The entrance or exit paths in herringbone hall 1710 may be more or fewer than those shown in the diagram. Figure 18aIn arrangement 1700a, a herringbone hall 1710 is shown having two entrance paths and one exit path. The herringbone hall 1710 may include multiple animal identification modules 1440. One or more animals 1405 are shown in each of the entrance and exit paths of the herringbone hall, and within the herringbone hall 1710. In the illustrated arrangement, the herringbone hall 1710 includes two entrance paths and a single entrance path. Animal recording modules 1410 may be located along the exit path of the herringbone hall 1710, while at least one or more animal identification modules 1440 may be located within the herringbone hall 1710. Figure 18b In the illustrated arrangement 1700b, the herringbone hall 1710 includes two entrance paths and two exit paths, with the exit paths positioned relative to each other. The herringbone hall 1710 includes at least two animal recording modules 1410, with at least one located along one of the exit paths and the other along the other exit path. At least one or more animal identification modules 1440 may be located at the herringbone hall 1710. Figure 18c In the illustrated arrangement 1700c, the herringbone hall 1710 includes two entrance paths and two exit paths, with the exit paths positioned relative to each other. The herringbone hall 1710 includes at least two animal recording modules 1410, with at least one located at or near one of the entrance paths within or near the herringbone hall 1710, and at least another located at or near the other entrance path within or near the herringbone hall 1710. At least one or more animal identification modules 1440 may be located within the herringbone hall 1710.

[0163] Figures 14 to 1 The method for identifying an animal from multiple animals, as shown in Figure 8, is performed by a system in 1800. Figure 19 As shown in the image.

[0164] In this embodiment, the method includes: in step 1810, within a first time window, acquiring data associated with multiple animals 1405_1 to 1405_6 via at least one animal recording module 1410, the data including video from a two-dimensional imaging sensor. For example, at 1 p.m., an animal may be detected at the animal recording module (such as a camera). The time window can initially be set to a camera time + / - range, for example, from -300 seconds to +300 seconds from 1 p.m. The acquired data associated with the multiple animals can be stored in a database or server, etc., for further processing and / or analysis.

[0165] In step 1830, the method includes acquiring, within a second time window, data associated with the identifier of each animal in a subset of multiple animals 1405_1 to 1405_6 via at least one animal identification module 1440. For example, at 1:05 PM, the animal identification module (such as an RFID reader) may detect one or more animals. The time window may initially be set to a camera time + / - range, for example, from -300 seconds to +300 seconds from 1:00 PM. The acquired data associated with the identifier of each of the multiple animals in the subset may be stored in a database or server, etc., for further processing and / or analysis.

[0166] At least one animal recording module 1410 may be located in a first position, and at least one animal identification module 1440 may be located in a second position, wherein the first position and the second position are spaced apart.

[0167] At least one animal identification module 1440 has a detection range 1445, and the field of view 1415 of the two-dimensional imaging sensor has a field of view, such that the detection range and the field of view do not overlap.

[0168] At least one animal recording module 1410 is configured to identify each animal within a first time window and associate each animal with the timestamp corresponding to the detection point of that animal.

[0169] At least one animal recognition module 1440 is configured to recognize each animal within a second time window and associate each animal with the timestamp corresponding to the detection point of that animal.

[0170] In step 1850, the method includes: associating each animal identified by at least one animal recording module in a first time window with an animal identified by at least one animal identification module in a second time window.

[0171] In step 1860, the method can be configured to repeat steps 1810, 1830, and 1850 until the system identifies each animal identified by at least one animal recording module in the first time window and each animal identified by at least one animal identification module in the second time window at least four times. For example, before final identity assignment, the system may need to read the history of results and vectors (as described above for identification vectors). Reading the history of results and vectors may require a specific animal to appear at least four times before the system registers the identified animal.

[0172] In step 1870, the method includes adjusting the width of the second time window relative to the first time window, or adjusting the offset of the second time window relative to the first time window. For example, this adjustment can be based on a comparison of the number of matched animals within the current time window with the number of matched animals within a time window larger than the current time window. In this way, after the registration phase, the system operates such that, after registration, insights about an animal can be provided whenever an animal is seen or identified. This step can be performed after at least four instances or at least four animal registrations.

[0173] In step 1890, steps 1810, 1830, and 1850 are repeated, taking into account the adjusted width and / or offset of the time window. Steps 1810, 1830, and 1850 may be executed consecutively.

[0174] Figure 20 The flowchart shown describes adjustments or refinements. Figure 19The method 1900 for the time window is shown. In this embodiment, in step 1910, an initial time window is set. The initial time window can be based on individualized considerations for the relevant farm setup. This can be based on the farm manager's / owner's estimate of the time required for an animal to travel from an animal identification module (such as an RFID reader) to an animal recording module (such as a camera) or from an animal recording module (such as a camera) to an animal identification module (such as an RFID reader). The initial estimate can then be set to differ from that time by a few minutes or seconds. If no estimate is available, the initial time window can be set more broadly, for example, up to 20 minutes. In a further example, the system can be configured to have a time window of up to 60 or 70 minutes. In this way, the system can address clock variations caused by daylight saving time, such as daylight saving time causing clocks to be out of sync for up to one hour. It is recommended that the upper or lower limit of the time interval used by the system should not exceed 20 minutes. For example, the larger the time window, the more candidate objects or animals can pass through (one or more) cameras and (one or more) RFID readers. The system can be configured to avoid excessively large time offset differences, which could lead to an overly wide RFID time window and too many candidate objects. In step 1920, after collecting a sufficient number of successful ID matches from one or more animals, the time offsets 1920 are statistically measured in step 1930, and in step 1940, a refined time window is set from the 5th percentile to the 95th percentile of the sorted time offsets. For example, a sufficient number of time offsets might represent a milking cycle. Some farms might milk 1000 cows in 10 hours, while others might milk 300 cows in 5 hours. The system can be configured to operate during a milking cycle and adjust the time window after the cycle. Subsequent milking cycles can then be used to refine the time window.

[0175] In a further example of the above arrangement, the following steps can be performed to narrow down the time window: 1) During the farm setup phase, the system sets an initial window. This initial window can be set manually.

[0176] 2) For each scan (such as an RFID scan) performed by an animal identification module (e.g., an RFID reader), the following steps are performed:

[0177] The system detects, tracks, and extracts key features of each animal that passes beneath the camera of the animal recording module during an initial time window. Detection, tracking, and extraction of key features can be performed as described previously.

[0178] If specific RFID information exists in a previously acquired matching database, the system attempts to identify the animal from the candidate set or animal set that passed through during the initial time window.

[0179] For example, the system can uniquely identify an animal. That is, similar values ​​or information obtained for that animal may have a cosine similarity higher than the 0.5 threshold. In this case, the system can calculate the time offset of this match, for example, the difference in seconds between the RFID scan timestamp and the timestamp of the tracked object.

[0180] The system records this offset, as well as any other offsets that can be matched, for example, during the milking of an animal.

[0181] 3) Once the system attempts to find a unique match for each RFID scan within the initial time window, the system will perform the following steps:

[0182] The system is configured to determine a new time offset window for farm settings.

[0183] Using the time offset from step 2), the system is configured to calculate the 5th percentile and the 95th percentile to provide a lower and upper limit for the new time window.

[0184] If the time window changes, repeat steps 2) and 3).

[0185] This step ensures that the system continuously refines the time window that associates the timestamp offset of the RFID scan with the timestamp of the tracked object.

[0186] By shortening the time window, the number of candidates that can be considered can be reduced, thereby improving the registration of new animals / candidates and helping to identify unique matches.

[0187] The more matches the system obtains or identifies, the more offset time examples the system receives, allowing for a better time window for system evaluation.

[0188] This invention can also be used to build historical records. For example, before the system can confidently assign an identity to an animal (which can then be used to adjust time windows), the system may need to see a particular animal at least four times. This could be, for example, four times in one day, twice a day in a two-day period, or once a day in a four-day period. When the system is able to align the most frequent occurrences of RFID reads over four days, this becomes the system providing a set of baseline offsets and vectors, such as the previously described vectors for each animal. For some animals, this may take longer than four days. For example, if animals look very similar to another animal, and they always pass close to each other through animal identification module 1440 and animal recording module 1410, then the system may need to take even longer to identify each of these animals.

[0189] For example, Figure 15 The system shown can be configured to establish a historical record based on information collected from the animal record module 1410 and the animal identification module 1440. This is considered a registration. For example, during the first four days of operation, the system can:

[0190] This begins with setting an initial time window during the installation setup phase, for example, but not limited to, on a farm. The initial time window can be set manually by the user.

[0191] For each scan (such as an RFID scan) at animal identification module 1440, the system performs the following operations:

[0192] During the initial time window, key features of each animal passing below the animal recording module (such as a camera) are detected, tracked, and extracted. The detection, tracking, and extraction of key features can be performed as described previously.

[0193] If the animal's RFID tag is seen at animal identification module 1440 at least four times within these four days:

[0194] Register the RFID tag:

[0195] The system retrieves candidate objects (animals) for the day and compares them with previous batches of candidate objects. In other words, a set of candidate objects is generated each time RFID information is seen or detected.

[0196] Using features extracted by the system, the system identifies whether a particular animal appears more frequently than any other animal, for example, at least four times. If the system has identified that a particular animal appears more frequently than any other animal, it adds the animal to the matching database, including information about the number of times the animal has been seen.

[0197] If the system cannot identify a unique animal that appears frequently, the system is configured to skip or ignore the animal's RFID information and attempt to register the animal in a future registration phase.

[0198] Referring to the above Figure 10 It describes the treatment of 1000 registered animals. (Reference) Figures 14 to 20 This invention processes a group of animals. Once this group of animals is processed within a time window, the system collects all tracked bounding boxes and all identification (ID) reads from any identification device (i.e., at least one animal identification module 1440 and at least one animal recording module 1410 within the system installation area). Then, by comparing the collected reads, identification vectors, and the number of times an animal with these identification vectors is identified as being near the identification module whenever it is below the animal recording module (i.e., the camera), the system refines this information into the most common identifications, enabling the system to identify the animal. The system is dynamic and self-expanding. For example, even if no setup data is provided when an animal is in the monitored population, the same animal may be scanned again as appropriate. The system then identifies the animal based on a match between its identification vector and the identification vector previously formed for the same animal. When appropriate, all animals can be automatically registered and identified in the manner described above.

[0199] The following is for reference. Figure 21 A method 2100 is described for identifying one or more animals from a plurality of animals. The plurality of animals, and subsets of these animals, may include cattle, pigs, and / or sheep, and may include, for example, at least two animals, at least ten animals, or at least one hundred animals. The subset of animals may include one or more such animals. Method 2100 may be implemented by one or more processors, such as... Figure 22 The processor (one or more) shown is 2210.

[0200] Method 2100 includes: obtaining (step 2102) timestamped image data associated with a subset of multiple animals; obtaining (step 2104) timestamped identification data including one or more unique identifiers; storing (step 2106) the associations between the data identifying the subset of multiple animals and the one or more unique identifiers in a database; repeating (step 2108) steps 2102 to 2106 to store multiple associations in the database; and analyzing (step 2110) the multiple associations stored in the database to generate a mapping between unique identifiers and data identifying multiple animals. Optionally, the method further includes using the mapping to identify (step 2112) one or more animals.

[0201] In step 2102, method 2100 includes acquiring time-stamped image data associated with a subset of a plurality of animals via at least one animal recording module within a first time window. The at least one animal recording module may include one or more image capture devices, such as a camera or multiple cameras. Figure 22 A suitable animal recording module 2202 for method 2100 is shown and described below. This animal recording module, or each animal recording module, can apply timestamps to image data. This animal recording module, or each animal recording module, can track timestamps.

[0202] The width of the first time window can be up to 20 minutes, 10 minutes, 5 minutes, 1 minute, or 30 seconds. For example, the initial width of the first time window can be 10 minutes or 5 minutes. Timestamped image data will include images and / or videos of a subset of multiple animals captured within the 10-minute or 5-minute time window. The width of the first time window can be set based on user input. Alternatively, the width of the first time window can be set based on a default value, or based on the speed or speed range at which at least one or more animals move through or between the animal identification module and the animal recording module.

[0203] The time-stamped image data obtained within the first time window via at least one animal recording module includes image or video data. Specifically, the time-stamped image data may include images of one or more animals, similar to those described above regarding Figures 5 to 6. Figure 7 The image described in the image. The timestamped image data can be captured from a top-down view.

[0204] In a preferred embodiment, the timestamped image data includes data identifying a subset of multiple animals within the timestamped image data. Specifically, the data identifying the subset of multiple animals includes one or more visual features, reference points, and / or feature vectors. Optionally, step 2102 includes generating one or more visual features, reference points, and / or feature vectors for each animal in the subset of multiple animals (i.e., each animal captured in the image or video data). The timestamped image data may include the visual appearance of one or more animals in the video, as well as derived behavioral and health data.

[0205] One or more visual features, reference points, and / or feature vectors can be generated in the manner outlined above with respect to the preceding embodiments. Specifically, generating feature vectors for each animal in a subset of multiple animals involves using a trained neural network. For example, the neural network can generate an overall feature vector using the output of one or more of a reference point algorithm, an activity algorithm, and a physical condition scoring algorithm applied to one or more frames of, for example, video data (see above). Figures 5a to 7 (Description). Visual features, reference points, and / or feature vectors can be generated locally by at least one animal recording module, or by one or more processors (such as...). Figure 22 The processor 2210 shown is remotely generated.

[0206] The generated visual features, reference points, and / or feature vectors can be stored in a database and used for automatic retraining according to a set schedule. This schedule can be any time period, but is typically limited because training the model is costly. Multiple additional cameras can be used to create a full-body representation of each animal. These additional vectors are advantageous because the additional cameras will see the animal from multiple angles, and therefore, over time, a more complete representation of the animal's entire body as seen from one or more of the additional cameras can be constructed.

[0207] In step 2104, method 2100 includes obtaining timestamped identification data, including one or more unique identifiers, via at least one animal identification module within a second time window. In a preferred embodiment, the unique identifier includes a serial number, such as a serial number corresponding to an RFID tag applied to the ear of an animal (such as a cow). Figure 22 A suitable animal identification module 2204 for method 2100 is shown below. The animal identification module can apply timestamps to the identification data. The animal identification module can be configured to track timestamps.

[0208] The width of the second time window can be up to 20 minutes, up to 10 minutes, up to 5 minutes, up to 1 minute, or up to 30 seconds. For example, the initial width of the second time window can be 10 minutes or 5 minutes. The timestamped identification data will include one or more unique identifiers captured by the animal identification module within this 10-minute or 5-minute time window. The width of the second time window can be set based on user input. Alternatively, the width of the second time window can be set based on a default value, based on the speed or speed range at which at least one or more animals move through or between the animal identification module and the animal recording module.

[0209] In the example, the second time window is offset from the first time window. This means that the animal recording module can collect image data during the first time period (i.e., the first time window), while the animal recognition module can collect recognition data during the second time period (i.e., the second time window), and the first and second time periods are at least partially separated in time. In some non-limiting examples, the offset can be as long as 20 minutes, 10 minutes, 5 minutes, 1 minute, or 30 seconds. For example, the initial width of each of the first and second time windows may be 5 minutes, and the initial width of the offset may also be 5 minutes. The width of the offset can be set based on user input. Alternatively, the width of the offset can be set based on a default value, based on the speed or speed range at which at least one or more animals move through or between the animal recognition module and the animal recording module. The offset can be positive or negative: the first time window or the second time window may occur first in time. The width of the offset can also be based on the asynchrony between the animal recording module and the animal identification module, that is, the system time of the animal recording module and the animal identification module may not be aligned. For example, if only one module is using daylight saving time, the width or offset can be adjusted by ±1 hour.

[0210] The offset refers to the time difference between the first and second time windows. For example, the first and second time windows can be temporally separate so that they do not overlap. The offset can correspond to the time difference between the start of the first time window and the start of the second time window. Alternatively, the offset can correspond to the time difference between the start of the first time window and the start of the second time window. In another alternative, the offset can correspond to the time difference between the midpoint of the first time window and the midpoint of the second time window. In yet another alternative, the offset can correspond to the time difference between the end of the first time window and the end of the second time window. In the example, the initial width of each of the first and second time windows might be 5 minutes, and the initial width of the offset is also 5 minutes, so that there is a 5-minute interval between the end of the first time window and the second time window, and so on.

[0211] The animal recording module has a field of view, and the animal recognition module has a detection range. The following will discuss... Figure 22 To explain further, in the preferred embodiment, the detection range and field of view do not overlap, and the distance between the animal recording module and the animal recognition module is greater than the detection range of the animal recognition module. Separate time windows, separated by offsets, are provided for collecting image and recognition data, allowing the animal recording module and the animal recognition module to detect specific animals as they move past and / or between these modules.

[0212] In step 2106, method 2100 includes: generating an association between data identifying a subset of multiple animals obtained via an animal recording module within a first time window and one or more unique identifiers obtained via the at least one animal identification module within a second time window, and storing the association in a database. This association is stored in the database for future use in identifying specific animals, for example, identifying a specific animal based on partial identification data.

[0213] In step 2108, method 2100 includes repeating steps 2102 through 2106 for multiple first time windows and multiple second time windows to store multiple associations in a database. For example, steps 2102 through 2106 may be repeated at least 4 times, at least 10 times, or at least 100 times. Each association includes one or more visual features generated from timestamped image data obtained from the first time windows. For each repetition of steps 2102 through 2106, the widths of the first and second time windows, as well as their offsets, can each remain constant / unchanged. Alternatively, method 1900, for example, can be used to adjust the widths of the first and second time windows, as well as the offsets, between repetitions.

[0214] In step 2110, method 2100 includes analyzing multiple associations stored in a database to generate a mapping between unique identifiers and data identifying multiple animals. The mapping may be a table, for example, similar to... Figure 12 Table 1260 is shown below. In a preferred embodiment, the mapping includes one or more unique identifiers and one or more corresponding visual features, reference points, and / or feature vectors.

[0215] Each repetition of steps 2102 and 2104 yields different subsets of multiple animals and different sets of unique identifiers. This means that each repetition of steps 2102 to 2106 may generate a different set of associations in step 2016. Ultimately, the database will be populated with a large number of associations that can be used to match each unique identifier with the corresponding visual feature reference points and / or feature vectors of a specific animal among multiple animals. Each unique identifier obtained by the animal identification module can be matched with the corresponding animal's visual feature reference points and / or feature vectors, for example, through an exclusion process.

[0216] Step 2110 may further include refining the mapping between unique identifiers and data identifying multiple animals by matching one or more generated visual features, reference points, and / or feature vectors with each unique identifier. Once a particular animal has been identified at least four times, the method may include registering the animal into the system.

[0217] Method 2100 may include adjusting the width of one or more of the first time window, the second time window, and the offset. For example, method 2100 may include adjusting the width of one or more of the first time window, the second time window, and the offset based on user input. Alternatively, method 2100 may include adjusting the width of one or more of the first time window, the second time window, and the offset based on statistical analysis, for example, according to the above description regarding... Figure 20 Methods of explanation.

[0218] In optional step 2112, method 2100 includes using mapping to identify (i.e., re-identify) one or more animals. For example, using mapping to identify one or more animals may include: receiving image data associated with one or more of a plurality of animals from a device (such as a smartphone or surveillance camera); and using a processor (such as...) Figure 22 The processor(s) shown 2110 generate visual features, reference points, and / or feature vectors of one or more animals from the received image data; and identify (one or more) unique identifiers associated with one or more animals from the received image data from the mapping. By matching the visual features, reference points, and / or feature vectors generated by the processor(s) with the visual features, reference points, and / or feature vectors in the mapping, (one or more) unique identifiers associated with one or more animals from the received image data can be identified.

[0219] Generally, once the system is confident in identifying an individual from multiple animals using image data and mapping, it no longer needs to rely on an animal identification module (for example, if the animal identification module cannot identify an individual animal, it can still be identified via an animal recording module). In other words, once an animal is identified and registered in the system, it can also be identified via a third image recording device without the need for an animal identification (RFID) module.

[0220] Figure 21The disclosed method has two phases: a registration phase (steps 2102 to 2110) and a re-identification phase (step 2112). The registration phase can be used for the first few days after setup (i.e., after deployment at the site or farm), when no animals in the herd are known or mapped, and the system iteratively learns to match the animals' visual appearance (visually identified via time-stamped image or video data) with time-stamped RFID reads (i.e., time-stamped identification data). In the re-identification phase, the visual representation of a registered animal can be used to identify that animal, even without RFID read support. The re-identification phase also supports the presence of additional cameras without accompanying RFID scanners. Animals registered to the system using a first camera and RFID scanner can be re-identified at the additional location using only their visual representation. For new animals added to the herd later, each animal begins its registration phase on its first day, and enters the re-identification phase when its visual representation and its scan ID can be matched using the learned association between visual appearance and identification data (i.e., using mapping). Not all animals need to be registered for the system to enter the re-identification phase: registration is done on an animal-by-animal basis after deployment. RFID scanners or other animal identification modules are not mandatory for the re-identification phase. RFID scanners can assist in the re-identification phase by verifying identification results, but they are not essential, as the system can select the animal whose visual representation is closest.

[0221] Now will target Figure 22 A system 2200 is described for identifying one or more animals among a plurality of animals.

[0222] System 2200 can be provided as part of a sorting gate, revolving hall, or herringbone hall, such as Figure 4 , Figure 15 , Figures 16a to 16b , Figures 17a to 17d and Figures 18a to 18c The layout is shown. This system can be installed in an animal farm, for example, as part of the farm's sorting gate, revolving hall, or herringbone hall.

[0223] System 2200 includes: at least one animal recording module 2202; at least one animal identification module 2204; a memory 2208 for storing a database; and one or more processors 2210. Optionally, system 2200 may also include user equipment 2220.

[0224] In embodiments, one or more processors 2210 are configured to implement any of the example methods disclosed herein, such as Figure 21Method 2100. System 2200 is configured to identify one or more of a plurality of animals, each of which is associated with a unique identifier.

[0225] Animal record module 2202 can be used with Figure 1 , Figure 4 , Figure 12 and Figures 15 to 18c The animal recording module shown in the earlier embodiment is identical. In this example, animal recording module 2202 includes a two-dimensional image sensor, which may be a color camera. Animal recording module 2202 is configured to generate image and / or video data. The animal recording module has a field of view (e.g., a top-down field of view, which may include the upper side of the animal, such as a cow—see Figures 5 through 8 above). In a preferred embodiment, the animal recording module has a horizontal field of view of up to 360 degrees (e.g., using a fisheye device that can capture a 360-degree view and combine the images to represent a planar view), up to 180 degrees, up to 120 degrees, or up to 60 degrees.

[0226] Animal recognition module 2204 can be with Figure 4 , Figure 12 and Figures 15 to 18c The animal identification module shown in the earlier embodiment is exactly the same. In this example, animal identification module 2204 includes a radio frequency identification (RFID) reader. The RFID reader is configured to receive information from an RFID device (such as an RFID tag) attached to each of the multiple animals (e.g., attached to the ear of each animal). Animal identification module 2204 is an RFID scanner and has a detection range. In embodiments, the detection range is less than 50 meters, less than 10 meters, less than 5 meters, less than 2 meters, or less than 1 meter.

[0227] In a preferred embodiment, the field of view of the animal recording module 2202 and the detection range of the recognition module 2204 do not overlap. In other words, the animal recording module 2202 is located at a first position, and the animal recognition module 2204 is located at a second position, and the first and second positions do not overlap: the first and second positions are separated by a distance D. In the example, the distance D is at least 50 meters, at least 10 meters, at least 5 meters, at least 2 meters, or at least 1 meter, and the distance D is greater than the detection range of the animal recognition module. In the illustrative example, the distance D is at least 5 meters, the detection range of the animal recognition module 2204 is less than 5 meters, and so on. Furthermore, in a preferred embodiment, the animal recognition module is located outside the field of view of the animal recording module, and the animal recording module is located outside the detection range of the animal recognition module (e.g., with...). Figure 15(Disclosed in the manner described). In a preferred example, the first and second locations are on the same site or farm (e.g., in a single hall or yard) and are only a few minutes apart, meaning that an animal like a cow would take at most a few minutes, such as 5 minutes, to walk between the two locations.

[0228] like Figure 22 As shown, memory 2208 and processor(s)2210 are provided as part of processing platform 2206. Processing platform 2206 includes interface 2212 that allows processing platform 2206 to connect to network 2230. Processing platform 2206 can communicate with at least one animal recording module 2202, at least one animal identification module 2204, and user equipment 2220 via one or more wired or wireless links, and via interface 2212 and network 2230. In the example, memory 2208 includes computer instructions to cause processor(s)2100 to perform steps 2102 to 2110 of method 2100.

[0229] User equipment 2200 is communicatively connected, for example, via network 2230 or via a wired communication link to at least one animal recording module 2202, at least one animal identification module 2204, memory, and / or processor. In an exemplary embodiment, system 2200 and / or user equipment 2220 include additional animal recording modules (such as two-dimensional image sensors) and / or additional animal identification modules. User equipment 2220 may be a mobile imaging device, such as a digital camera or a smartphone. User equipment 2220 is configured to send data to and / or receive data from at least one processor via network 2230.

[0230] In the example use case, user device 2220 is a smartphone. A user (such as a livestock owner) can use user device 2220 to take a photo or video of a specific animal that has been registered in system 2200, but the animal's RFID tag has been lost (e.g., the RFID tag has fallen off the animal's ear). Processor 2210 can use the photo or video of the specific animal, along with a mapping stored in memory 2208, to identify a unique identifier (corresponding to the RFID tag) corresponding to that animal.

[0231] In another example use case, the user device is a surveillance camera located in a fixed location (e.g., on a farm). The surveillance camera may be located in an area of ​​the farm where there are no RFID scanners. When a user wants to identify a specific animal being monitored by the surveillance camera, the processor 2210 can use the images or videos collected by the surveillance camera, along with a mapping stored in memory 2208, to identify a unique identifier (corresponding to an RFID tag) corresponding to that animal.

[0232] exist Figure 23 In another example use case shown, user device 2302 (a third-party device such as a smartphone) is used to identify animal 2304. User device 2302 is equipped with an animal recording module (e.g., a camera) having a field of view 2302a, and is able to use the animal recording module to capture image data of animal 2304 within its field of view 2302a. In this example, animal 2304 has been registered with the system. User device 2302 can access visual feature database 2306, which includes mapping 2208 (as described above). By comparing the visual features of animal 2304 captured by user device 2302 with mapping 2208 stored in visual feature database 2306 (step 2112), the animal's identity (i.e., a unique identifier corresponding to the registered animal 2304) can be determined.

[0233] exist Figure 24 In another example use case shown, surveillance camera 2402 is used to identify animal 2404 from a group of animals. Surveillance camera 2402 is equipped with an animal recording module (e.g., a camera) having a field of view 2402a, and is capable of capturing image data of animal 2404 within its field of view 2402a using this animal recording module. In this example, animal 2404 is one of many animals registered in the system. Surveillance camera 2402 can access visual feature database 2406, which includes mapping 2208 (as described above). By comparing the visual features of animal 2404 captured by surveillance camera 2402 with the mapping 2208 stored in visual feature database 2406 (step 2112), the animal's identity (i.e., a unique identifier corresponding to the registered animal 2404) can be determined.

[0234] exist Figure 25 In another example use case shown, surveillance camera 2502 is used to identify animal 2504 within fence 2508. Surveillance camera 2502 is equipped with an animal recording module (e.g., a camera) having a field of view 2502a, and is capable of capturing image data of animal 2504 within its field of view 2402a using this module. In this example, animal 2504 is one of many animals registered in the system. Surveillance camera 2502 has access to visual feature database 2506, which includes mapping 2208 (as described above). By comparing the visual features of animal 2504 captured by surveillance camera 2502 with the mapping 2208 stored in visual feature database 2506 (step 2112), the animal's identity (i.e., a unique identifier corresponding to the registered animal 2504) can be determined.

[0235] Animals can be registered using a first animal recording module and an animal identification module (e.g., in a hall), and then re-identified using a second animal recording module (e.g., a surveillance camera on a fence). This example can be used to distinguish which animals are in which fence. This example can be further extended to determine more data or characteristics of the animals, such as the time spent lying down or standing. This example can be further extended to determine which animals are in the wrong fences via system integration with data that stores the expected fence for each animal.

[0236] In another example, animals can be registered using a first animal recording module and an animal identification module (e.g., in a hall), and then re-identified using an additional animal recording module (e.g., a camera (fixed or mobile) pointing at the trailer) to determine which animals have been loaded onto the trailer.

[0237] Once an animal is registered, if a registered animal cannot be identified by means of an RFID scanner in the future, but can be successfully identified using data from an animal recording module (e.g., a camera), the identification data or ID corresponding to that animal can be reported back to the processing platform 2206 or the central server for correction. If the processing platform 2206 or the central server does not support correction, the identification data or ID corresponding to the animal can also be exported to a separate system for recording corrections. At the end of a predetermined time period, a report can be generated to highlight all available corrections. An interface can be provided that allows users to query the system for assistance by providing an identifier, location, and / or timestamp if they are unable to confirm the animal's identity.

[0238] In some alternative embodiments, the functions and / or operations specified in the flowcharts, sequence diagrams, and / or block diagrams may be reordered, processed sequentially, and / or in parallel without departing from the scope of the invention. Furthermore, any flowchart, sequence diagram, and / or block diagram may include more or fewer modules than those shown, consistent with embodiments of the invention.

[0239] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the embodiments of the invention. As used herein, the singular forms “a,” “an,” and “the” are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the terms “comprising” and / or “including” as used in this specification refer to the presence of the specified features, elements, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, elements, steps, operations, elements, components, and / or combinations thereof. Furthermore, to a certain extent, when the terms “comprising,” “having,” “having,” “with,” “consisting of,” or variations thereof are used in the detailed description or claims, these terms are intended to be inclusive in a manner similar to the term “comprising.”

[0240] This invention may be further defined according to the following provisions:

[0241] 1. A method for identifying an animal, the method comprising:

[0242] Data associated with animal movement through space is obtained via an animal recording module (220), including video from a two-dimensional imaging sensor;

[0243] Extract the parameter set (230) from the data by performing the following steps:

[0244] Determine the animal's visual characteristics based on the data;

[0245] Identify instances in the data that represent animal shapes to form the detected instances;

[0246] Identify the set of reference points in each detected instance;

[0247] One or more characteristics of an animal are determined by processing a set of reference points at at least some identification sites in a first module, which includes a trained neural network; and

[0248] Generate a parameter set that includes visual features and one or more identified properties;

[0249] Generate (240) recognition vectors from the generated parameter set;

[0250] Select (250) known identification vectors from the first database of known identification vectors, each of which corresponds to a unique registered animal;

[0251] The (260) matching score list is determined by comparing the generated identification vector with the selected known identification vector;

[0252] In response to determining that at least one selected known identification vector has a matching score exceeding a threshold;

[0253] Based on at least one criterion, (270) at least one of the known identification vectors is selected;

[0254] The animals are associated using selected known identification vectors (275); and

[0255] The animal is identified (280) as the only registered animal with a selected known identification vector; and

[0256] In response to determining that no selected known identification vector has a matching score exceeding a threshold, the animal is identified (290) as an unknown animal.

[0257] 2. According to the method of Clause 1, wherein:

[0258] By performing further steps, including segmenting the video into a set of frames, a parameter set is extracted from the data;

[0259] Determining an animal's visual characteristics based on data includes: determining the animal's visual characteristics based on at least one frame; and

[0260] Determining instances representing animal shapes in the data to form detected instances includes: determining instances representing animal shapes in each frame to form detected instances.

[0261] 3. The method according to Clause 2, wherein determining instances representing animal shapes in each frame to form detected instances includes:

[0262] Generate bounding boxes in each frame where an instance of an animal body is identified;

[0263] Based on the generated bounding boxes, the animal's movement across frames is tracked to determine whether the animal's movement across space is continuous or intermittent;

[0264] Detected instances are formed from the bounding boxes corresponding to the continuous movements of the animal.

[0265] 4. According to the method of Clause 2 or Clause 3, the identification of the reference point set includes:

[0266] Extract the x and y coordinates of each reference point;

[0267] For each individual frame, at least a first array and a second array are generated, the first array including the x-coordinate values ​​of the reference point in each individual frame, and the second array including the y-coordinate values ​​of the reference point in each individual frame;

[0268] The values ​​from the first and second arrays are combined to create a multidimensional array of x and y coordinate values, the size of which is defined by the number of individual frames and the number of reference points.

[0269] 5. The method according to any of the preceding clauses, wherein identifying the set of reference points in each detected instance includes applying a pre-computed segmentation mask to the detected instances of animal bodies.

[0270] 6. The method according to any of the preceding clauses, wherein identifying an animal as an unknown animal includes adding the generated identification vector to a second database containing unknown identification vectors.

[0271] 7. According to any of the methods in the preceding clauses, wherein:

[0272] The animal recording module also includes an electronic receiver for receiving wireless signals;

[0273] The data also includes wireless signals that identify the animals.

[0274] 8. The method according to any of the preceding clauses, wherein the two-dimensional imaging sensor is a color camera.

[0275] 9. According to any of the methods in the preceding clauses, determining one or more characteristics includes:

[0276] In a second module that includes a trained neural network, the animal’s health score is determined by processing at least some of the identified reference point sets.

[0277] 10. According to any of the methods in the preceding clauses, determining one or more characteristics includes:

[0278] In the second module, which includes a trained neural network, the animal’s behavioral score is determined by processing at least some of the identified reference point sets and identifying changes in at least some of the coordinate values ​​of the identified reference point sets.

[0279] 11. According to any of the methods in the preceding clauses, wherein:

[0280] The method also includes detecting a series of signals, each of which includes time and is associated with a unique animal;

[0281] Selecting known identification vectors from the first database of known identification vectors also includes:

[0282] Determine the time when the animal recording module acquires data, and the time range within that determined time period;

[0283] Filter the series of signals to select only the signals within the time range from the series of signals;

[0284] Select the known identification vector of the unique animal for the chosen signal.

[0285] 12. The method according to any of the preceding clauses, wherein the number of common values ​​between the generated identification vector and the known identification vector being compared is evaluated by determining the cosine similarity between the generated identification vector and the known identification vector.

[0286] 13. A system for identifying animals, the system comprising:

[0287] Animal recording module, used to obtain data associated with animals moving through space;

[0288] The identification system is configured to determine the identity of an animal;

[0289] The user equipment is communicatively coupled to the identification system and is configured to receive information associated with the determination of the animal's identity;

[0290] The identification system includes at least one processor configured to perform the method of any one of Clauses 1 to 12.

[0291] 14. Animal farms, including systems pursuant to Clause 13.

[0292] 15. The methods of Clauses 1 to 12 or the systems of Clause 13 are used for the purpose of identifying animals.

[0293] The present invention can be further defined according to the following examples:

[0294] 1. A method (1400, 1800) for a system for identifying an individual animal (1405) from a plurality of animals (1405_1 to 1405_6), the method comprising the steps of:

[0295] a) Within a first time window, data associated with multiple animals (1405_1 to 1405_6) is obtained (1810) via at least one animal recording module (1410) in the first time, the data including video from a two-dimensional imaging sensor;

[0296] b) During the second time window, data associated with the identifier of each animal in a subset of multiple animals (1405_1 to 1405_6) is obtained (1830) via the animal identification module (1440) at the second time.

[0297] The animal recording module is located in the first position, the animal recognition module is located in the second position, and the first position and the second position are separated.

[0298] In this embodiment, at least one animal recording module is configured to identify each animal within a first time window and associate each animal with the timestamp corresponding to the detection point of that animal;

[0299] In this embodiment, at least one animal recognition module is configured to recognize each animal within a second time window and associate each animal with the timestamp corresponding to the detection point of that animal;

[0300] c) Associate each animal identified in the first time window of at least one animal record module with the animal identified in the second time window of at least one animal identification module (1850).

[0301] d) Adjust the width of the second time window relative to the first time window and / or the offset of the second time window relative to the first time window (1870);

[0302] Repeat steps (1890) a) to c) by combining the adjusted width and / or offset of the time window.

[0303] 2. According to the method of Example 1, wherein, prior to adjustment, the system is configured to repeat steps (1860) a) to c) until the system identifies each animal identified by at least one animal recording module in the first time window and each animal identified by at least one animal identification module in the second time window at least four times.

[0304] 3. According to the method of Example 2, the identified animals are registered in the system after at least four identifications of each animal identified in the first time window and the animals identified in the second time window.

[0305] 4. The method according to any of the foregoing examples, wherein each animal identified by at least one animal record module is associated with an animal identified by at least one animal identification module during a first time window and a second time window (1850).

[0306] 5. The method according to any of the foregoing examples, wherein the animal identification module has a detection range (1445) and the two-dimensional imaging sensor has a field of view (1415) such that the detection range and the field of view do not overlap;

[0307] 6. The method according to any of the foregoing examples, wherein the first position and the second position are separated by a distance D, which is up to 50 meters, preferably up to 10 meters.

[0308] 7. The method according to any of the foregoing examples, wherein the width or offset of the second time window relative to the first time window is up to 20 minutes, preferably up to 5 minutes.

[0309] 8. The method according to any of the foregoing examples, wherein the first time window and the second time window are set with an initial time window value, wherein the value is based on the speed or speed range of at least one or more animals (1405_1 to 1405_6) moving between the animal identification module (1440) and the animal recording module (1410).

[0310] 9. According to any of the methods in the preceding examples, wherein the initial time window value is based on user input.

[0311] 10. The method according to any of the foregoing examples, wherein the two-dimensional imaging sensor is a color camera.

[0312] 11. The method according to any of the foregoing examples, wherein the animal identification module includes a radio frequency identification (RFID) reader.

[0313] 12. According to the method of Example 11, the data associated with the identifier of each animal in a subset of multiple animals is information received at an RFID reader from an RFID device attached to each animal in the subset of multiple animals.

[0314] 13. A system for identifying an animal from a plurality of animals, the system comprising:

[0315] Animal detection systems, including:

[0316] A device for obtaining (1410) video recordings of an animal (1405) moving through space, the video recordings being captured from a top-down angle by a two-dimensional imaging device (1410);

[0317] A means for obtaining (1440) an identifier of an animal (1405) moving through space, the identifier being detected by an animal identification module (1440) spaced apart from a two-dimensional imaging device (1410);

[0318] User equipment (140a, 140b, 140c) is communicatively coupled to the animal detection system and is configured to receive information related to the identity of the animal (1405);

[0319] The animal detection system includes:

[0320] At least one processor is configured to perform a method according to any one of Examples 1 to 12.

[0321] 14. Sorting doors, revolving halls, or herringbone halls, including the system described in Example 13.

[0322] 15. Animal farms, including systems based on Example 13.

Claims

1. A method for identifying one or more animals among a plurality of animals, each of the plurality of animals being associated with a unique identifier, the method comprising: a) Obtain, via at least one animal recording module and within a first time window, timestamped image data associated with a subset of the plurality of animals, the timestamped image data including data for identifying the subset of the plurality of animals in the timestamped image data; b) Obtaining timestamped identification data, including one or more unique identifiers, via at least one animal identification module within a second time window, wherein the second time window is offset from the first time window; c) Store in a database the association between data identifying a subset of the plurality of animals obtained via the at least one animal recording module within the first time window and the one or more unique identifiers obtained via the at least one animal identification module within the second time window; d) Repeat steps (a) to (c) for multiple first time windows and multiple second time windows to store multiple associations in the database; e) Analyze the multiple relationships stored in the database to generate a mapping between the unique identifier and the data that identifies the multiple animals.

2. The method according to claim 1, wherein, The animal recording module has a field of view, and the animal recognition module has a detection range, wherein the detection range and the field of view do not overlap.

3. The method according to claim 1 or 2, wherein, The distance between the animal recording module and the animal recognition module is greater than the detection range of the animal recognition module.

4. The method according to any one of the preceding claims, wherein, The data for identifying a subset of the multiple animals includes one or more visual features, reference points, and / or feature vectors.

5. The method according to any one of the preceding claims, wherein, Step (b) includes generating one or more visual features, reference points, and / or feature vectors for each animal in a subset of the plurality of animals.

6. The method according to claim 5, wherein, Generating visual features, reference points, and / or feature vectors for each animal in a subset of the plurality of animals includes using a trained neural network.

7. The method according to any one of the preceding claims, wherein, The timestamped image data includes video data.

8. The method according to any one of the preceding claims, wherein, The timestamped image data was captured from a top-down view.

9. The method according to any one of the preceding claims, wherein, The width of the first time window and / or the second time window is greater than 1 hour, up to 20 minutes, up to 10 minutes, up to 5 minutes, up to 1 minute, or up to 30 seconds.

10. The method according to any one of the preceding claims, wherein, The offset between the first time window and the second time window can be up to 20 minutes, up to 10 minutes, up to 5 minutes, up to 1 minute, or up to 30 seconds. Optionally, the offset between the first time window and the second time window may additionally include a 1-hour offset, such as ±1 hour offset corresponding to daylight saving time.

11. The method according to any one of the preceding claims, wherein, The first time window, the second time window, and / or the offset have initial values, or each of the first time window, the second time window, and / or the offset has an initial value.

12. The method according to claim 11, wherein, The initial values ​​of the first time window, the second time window, and / or the offset are based on user input.

13. The method according to claim 11, wherein, The initial values ​​of the first time window, the second time window, and / or the offset are based on the speed or speed range at which at least one or more animals move between the animal identification module and the animal recording module.

14. The method according to any one of the preceding claims, wherein, The method includes adjusting one or more of the first time window, the second time window, and the offset.

15. The method according to claim 14, wherein, The method includes adjusting one or more of the first time window, the second time window, and the offset based on user input.

16. The method of claim 14, wherein, The method includes adjusting one or more of the first time window, the second time window, and the offset based on statistical analysis.

17. The method according to any of the preceding claims, wherein, Step (d) includes repeating steps (a) through (c) at least four times, at least ten times, or at least one hundred times.

18. The method according to any one of the preceding claims, wherein, The mapping includes one or more unique identifiers and one or more visual features, reference points, and / or feature vectors.

19. The method according to claim 18, wherein, Step (e) includes refining the mapping between the unique identifiers and the data identifying the plurality of animals by matching the generated visual features, reference points and / or feature vectors with each unique identifier.

20. The method according to any one of the preceding claims, wherein, Each association includes one or more visual features, reference points, and / or feature vectors generated from timestamped image data obtained in the first time window.

21. The method according to any of the preceding claims, wherein, Each association includes one or more unique identifiers in the timestamped identification data obtained within the second time window.

22. The method according to any one of the preceding claims, wherein, The method includes registering animals into the system.

23. The method according to any one of the preceding claims, wherein, The method also includes using the mapping to identify one or more animals.

24. The method according to claim 23, wherein, Identifying one or more animals using the mapping includes: receiving image data associated with one or more of the plurality of animals, optionally, the image data being received from a user device or other imaging device, such as a user device or other imaging device located in a third position.

25. The method according to claim 23 or 24, wherein, Identifying one or more animals using the mapping includes generating visual features, reference points, and / or feature vectors for one or more of the plurality of animals.

26. The method according to claim 25 or 25, wherein, Identifying one or more animals using the mapping includes: identifying from the mapping a unique identifier associated with one or more of the plurality of animals in the received image data.

27. The method according to any of the preceding claims, wherein, The unique identifier includes the serial number.

28. The method according to any one of the preceding claims, wherein, The animals include cattle, pigs, and / or sheep.

29. The method according to any one of the preceding claims, wherein, The plurality of animals includes at least two animals, at least 10 animals, or at least 100 animals.

30. A system for identifying one or more animals among a plurality of animals, wherein, Each of the plurality of animals is associated with a unique identifier, the system comprising: At least one animal record module; At least one animal identification module; Memory used to store the database; At least one processor, the at least one processor being configured to implement the method of any of the preceding claims.

31. The system according to claim 30, wherein, The animal recording module includes a two-dimensional image sensor.

32. The system according to claim 31, wherein, The two-dimensional imaging sensor is a color camera.

33. The system according to any one of claims 30 to 32, wherein, The animal recording module is configured to generate image data and / or video data.

34. The system according to any one of claims 30 to 33, wherein, The animal recording module has a field of view.

35. The system according to any one of claims 30 to 34, wherein, The animal recording module has a top-down field of view.

36. The system according to any one of claims 30 to 35, wherein, The animal recording module has a horizontal field of view of up to 360 degrees, up to 180 degrees, up to 120 degrees, or up to 60 degrees.

37. The system according to any one of claims 30 to 36, wherein, The animal identification module includes a radio frequency identification (RFID) reader.

38. The system according to claim 37, wherein, The RFID reader is configured to receive information from the RFID device attached to each of the plurality of animals.

39. The system according to any one of claims 30 to 38, wherein, The animal identification module is an RFID scanner.

40. The system according to any one of claims 30 to 39, wherein, The animal identification module has a detection range.

41. The system according to claim 40, wherein, The detection range is less than 50 meters, less than 10 meters, less than 5 meters, less than 2 meters, or less than 1 meter.

42. The system according to any one of claims 30 to 41, wherein, The animal recording module has a field of view, and the animal recognition module has a detection range, wherein the detection range does not coincide with the field of view.

43. The system according to any one of claims 30 to 42, wherein, The animal recording module is located in the first position.

44. The system according to claim 43, wherein, The animal identification module is located in the second position.

45. The system according to claim 44, wherein, The first position and the second position do not overlap.

46. ​​The system according to claim 44 or 45, wherein, The first position and the second position are separated by a distance D.

47. The system according to claim 46, wherein, The distance D is at least 50 meters, at least 10 meters, at least 5 meters, at least 2 meters, or at least 1 meter.

48. The system according to claim 46 or 47, wherein, The distance D is greater than the detection range of the animal recognition module.

49. The system according to any one of claims 30 to 48, wherein, The animal identification module is located outside the field of view of the animal recording module.

50. The system according to any one of claims 30 to 49, wherein, The animal recording module is located outside the detection range of the animal identification module.

51. The system according to any one of claims 30 to 50, wherein, The system also includes user equipment or other imaging devices, for example, located in a third position.

52. The system according to claim 51, wherein, The user equipment or other imaging device is communicatively coupled to the at least one animal recording module, the at least one animal identification module, the memory, and / or the processor.

53. The system according to claim 51 or 52, wherein, The user equipment or other imaging equipment includes an additional animal recording module and / or an additional animal identification module.

54. The system according to any one of claims 51 to 53, wherein, The user equipment or other imaging device includes a two-dimensional image sensor.

55. The system according to any one of claims 51 to 54, wherein, The user equipment or other imaging device is a mobile imaging device or a smartphone.

56. The system according to any one of claims 51 to 55, wherein, The user equipment or other imaging device is configured to send data to the at least one processor and / or receive data from the at least one processor.

57. A sorting gate, revolving hall, robotic hall, or herringbone hall, comprising the system according to any one of claims 30 to 56.

58. An animal farm comprising a sorting gate, a revolving hall, or a herringbone hall as described in claim 57.

59. An animal farm comprising the system according to any one of claims 30 to 56.

Images