Systems and methods for behavioral analysis and AI-based digital biomarker development

Abstract

This disclosure describes a digitally enabled platform that includes a camera-integrated animal (e.g., rodent) home cage, a cloud-based infrastructure, machine learning algorithms for generating digital biomarkers, and a user interface for collecting and visualizing continuous metrics of rat and mouse digital biomarkers. Metrics include movement, locomotion, wheel, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. A suite of sensors continuously monitors experimental conditions, minimizing the need for human intervention with study subjects.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 446,002, filed February 15, 2023, U.S. Provisional Patent Application No. 63 / 445,939, filed February 15, 2023, U.S. Provisional Patent Application No. 63 / 445,941, filed February 15, 2023, and U.S. Provisional Patent Application No. 63 / 599,310, filed November 15, 2023, and incorporates by reference the contents thereof. [Background technology]

[0002]

[0002] Traditional methods for monitoring the behavior of research animals include direct observation and automated behavioral monitoring. Direct observation involves an observer observing animals and recording their behavior. This can be done in real time or by reviewing video or recordings of the animals' behavior. Automated behavioral monitoring involves using sensors or cameras to track animal behavior without the need for a human observer. This is a more objective and efficient method of monitoring behavior, but it can also be more costly and less flexible than direct observation. In some automated applications, researchers use wearable sensors to track animals' heart rate, body temperature, and other physiological measurements. This information can be used to identify changes in animals' stress levels and assess their overall health. Summary of the Invention [Problem to be solved by the invention]

[0003]

[0003] While significant advances have been made in quantifying animal behavior, the utilization of emerging technologies that enable rich visual metrics and real-time analysis, enabling the development of digital biomarkers (DBs), has yet to be fully realized. Furthermore, standard methods for behavioral phenotyping often lack scalability for complex studies and require expert intervention. Therefore, there is a need for systems and methods that provide more efficient and accurate automated behavioral monitoring of laboratory animals. [Means for solving the problem]

[0004] This disclosure describes a digitally enabled platform that includes a camera-integrated animal (e.g., rodent) home cage, edge computing components, a cloud-based infrastructure, machine learning (ML) algorithms for generating digital biomarkers (DBs), and a user interface for collecting and visualizing continuous metrics of DBs for animals such as rats and mice. The metrics include movement, ambulation, wheel activity, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching behavior. A series of sensors are provided to continuously monitor experimental conditions, minimizing the need for human intervention with research subjects. High-resolution cameras live-stream animal activity to the cloud 24 / 7 at 30 fps in Ultra High Definition (UHD) 4K resolution. These data are accessible to veterinary professionals and animal behavior scientists through an easy-to-use interface, enabling intuitive live monitoring and visualization of DBs to identify significant events, including behavioral and physiological changes. It includes an integrated data science workbench, which provides a platform for computational scientists and biostatisticians to streamline and automate DB and ML workflows. The deep learning module integrated into the system can perform multi-animal detection, segmentation, and pose estimation. Annotation and shared learning functions built into the user interface facilitate collaboration among team members. The system is scalable to multi-cage stacks for complex research, DB development, and industrial deployment. This technology enables improved translatability, accelerated throughput, increased usability, and enhanced reproducibility.

[0005]

[0005] Computer vision-based behavioral monitoring of laboratory animals is an underutilized technology in drug development. The emergence of end-to-end machine learning (ML) offers opportunities to enhance preclinical research with dynamic and comprehensive approaches to rodent phenotyping.

[0006]

[0006] Digital biomarkers (DBs) are measurable characteristics or features collected from digital health technologies and used as indicators of normal biological processes, pathogenic processes, or responses to exposure or intervention.

[0007] In the clinical field, digital biomarkers are collected from digital health technologies (DHTs), systems that use computing platforms, connectivity, software, and sensors for healthcare and related applications. In preclinical research, digital biomarkers are collected using digital technologies such as radio frequency identification, capacitance-based electrode arrays, and computer vision.

[0008] In a first aspect, a system is provided. The system includes a cage housing including a cage bottom and a cage top. The system also includes a cage data unit including a top-down camera configured to capture images of a top-down field of view including one or more animals housed within the cage housing. The cage data unit also includes a lighting module configured to illuminate the top-down field of view. The system also includes a housing having an opening configured to receive the cage housing. The system further includes an air conditioning system coupled to the cage housing and a controller operable to perform an operation. The operation includes causing the top-down camera to capture images of the field of view.

[0009]

[0009] In a second aspect, a rack-mounted cage system is provided. The rack-mounted cage system includes a base frame and a plurality of vertical members coupled to the base frame. The rack-mounted cage system also includes a plurality of bays formed by spaces between the plurality of vertical members. At least a portion of the plurality of bays include a cage data unit. The system also includes a cage housing having a cage top and a cage bottom. The cage data unit also includes a top-down camera configured to capture images of a top-down field of view including one or more animals housed within the cage bottom. The cage data unit also includes a lighting module configured to illuminate the top-down field of view. The cage data unit also includes an air conditioning system coupled to the cage housing. The plurality of bays also include a controller operable to perform an operation. The operation includes causing the top-down camera to capture images of the field of view.

[0010] In a third aspect, a method is provided. The method includes receiving, via a central server, images of a top-down field of view. The images include live or past images of one or more animals contained within the cage housing. The method also includes displaying a user interface via the display. The user interface includes a video stream viewer. The video stream viewer is configured to display a user-navigable stream of live or past images. The user interface also includes at least one data graph. The at least one data graph includes information indicative of at least one of average locomotion speed, wheel occupancy, or waterer occupancy. The user interface further includes an annotation feed. The annotation feed includes information regarding animal behavior or information regarding digital biomarkers associated with the one or more animals.

[0011]

[0011] In a fourth aspect, a method is provided. The method includes receiving an image of a top-down field of view. The image includes a live or past image of one or more animals contained within a cage housing. The method includes determining a location of a particular animal within the image using a trained machine learning model. Determining the location of the particular animal includes applying an image segmentation mask. The method further includes assigning an identifier to the particular animal using the trained machine learning model. The method further includes assigning at least one bounding box corresponding to the location of the particular animal within the image.

[0012] In a fifth aspect, a method for training a machine learning model is provided. The method includes receiving, as training data, a plurality of images of a top-down field of view including one or more animals contained in a cage housing. The method also includes training the machine learning model using an unsupervised learning method based on the training data to form a trained machine learning model. The unsupervised learning method includes at least one of k-means clustering, hierarchical clustering, or density-based clustering.

[0013]

[0013] These and other aspects, advantages, and alternatives will become apparent to those skilled in the art upon reading the following detailed description, with reference, as appropriate, to the accompanying drawings. Moreover, it should be understood that the description provided in this summary section and elsewhere herein is intended to illustrate, by way of example, and not to limit, the claimed subject matter. [Brief explanation of the drawings]

[0014] [Figure 1] FIG. 1 illustrates a system according to an exemplary embodiment. [Figure 2]

[0015] FIG. 1 illustrates a rack mount cage system in accordance with an exemplary embodiment. [Figure 3]

[0016] FIG. 1 illustrates a method according to an exemplary embodiment. [Figure 4]

[0017] FIG. 1 illustrates a method according to an exemplary embodiment. [Figure 5]

[0018] FIG. 1 illustrates a method according to an exemplary embodiment. [Figure 6]

[0019] FIG. 1 is a schematic block diagram according to an exemplary embodiment. [Figure 7]

[0020] FIG. 1 illustrates a system in accordance with an exemplary embodiment. [Figure 8A]

[0021] FIG. 1 illustrates a rack mount cage system in accordance with an exemplary embodiment. [Figure 8B]

[0022] FIG. 1 illustrates a rack mount cage system in accordance with an exemplary embodiment. [Figure 9]

[0023] FIG. 10 illustrates a user interface in accordance with an exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015]

[0024] Exemplary methods and systems are described herein. It should be understood that, as used herein, the words "exemplary," "example," and "illustrative" mean "serving as an example, instance, or illustration." Any embodiment or feature described herein as "exemplary," "example," or "illustrative" is not necessarily to be construed as preferred or advantageous over other embodiments or features. Moreover, the exemplary embodiments described herein are not limiting. It will be readily understood that certain aspects of the disclosed systems and methods can be arranged and combined in a wide variety of different configurations.

[0016]

[0025] It should be understood that the following embodiments, as well as other embodiments described herein, are offered for illustrative purposes and are not intended to be limiting.

[0017] I. Overview

[0026] With only a small percentage of drug candidates making it to clinical trials and even fewer reaching the market, pharmaceutical companies are under immense competitive pressure to innovate in drug discovery and development. One challenge is the limitations of in vivo research, which, due to various factors, only adopt local animal facility rules and limit data sharing and standardization. This creates inevitable silos across companies, hindering the free flow of scientific information and intellectual insights throughout the pharmaceutical industry and between industry and academia. Furthermore, designing, executing, and analyzing data from in vivo preclinical studies is time-consuming and labor-intensive. Furthermore, these studies rely on expert opinion and evaluation, introducing subjectivity into research results. Despite the large staff required to operate animal facilities, animal observations and biometric measurements are still recorded sporadically and in a manner that disrupts the animals' natural activities. Aside from animal welfare concerns, collecting endpoint data from stressed animals can affect data quality and reliability. One solution is to use automation in animal handling and care, which significantly reduces variability and limits or completely eliminates human disturbance to animal habitats. While such techniques have been evaluated, technological limitations have prevented their widespread adoption. Recently, computer vision and image processing, powered by artificial intelligence (AI), have been effectively utilized to collect, quantify, and model animal activity. Developments in machine learning and large-scale cloud data transfer capabilities have enabled the practical and economical acquisition, storage, integration, and analysis of video analytics to gain novel insights from that data. To develop clinically relevant biomarkers from in vivo studies, a robust pipeline of data acquisition, processing, analysis, visualization, and scientific collaboration is required. Therefore, an integrated, scalable rodent cage system has been built to enable the development of AI-powered digital biomarkers through continuous computer vision-based behavioral analysis that runs seamlessly on the cloud, with instant access to acquired video and animal metrics.The system will enable an efficient and collaborative platform that can enable effective database development, validation, adoption, and regulatory approval.

[0018] II. Exemplary Systems

[0027] 1 illustrates a system 100 according to an exemplary embodiment. Various elements of system 100 can be based on disposable mouse / rat bottom cages with a raised design that meets EU animal housing standards and facilitates housing of multiple animals.

[0019]

[0028] System 100 includes a cage top 131 attachable to a cage bottom 130 to form a cage housing 133. Cage housing 133 can be configured to house one or more animals 132. In some examples, one or more animals 132 can include one or more mice or rats. Other animals observable within a physical housing or predefined space are possible and contemplated.

[0020]

[0029] System 100 also includes a cage data unit 120. Cage data unit 120 includes a top-down camera 122 configured to capture images 126 of a top-down field of view 124 through a cage top 131 toward a cage bottom 130.

[0021]

[0030] In some examples, the cage housing 133 can include a food hopper 123. The food hopper 123 can be sized appropriately for two 500-gram rats with a large feeding area to minimize aggression. The cage housing 133 also includes a water bottle holder 125. In such a scenario, the water bottle holder 125 can hold up to 600 mL of water with two drinking stations. The cage is designed to allow food, water, and running wheels to be positioned within the cage to minimize subject obstruction and allow for proper camera focus. In other words, at least one of the food hopper 123 or the water bottle holder 125 can be shaped to reduce obstruction of the top-down view 124. Side observation slots are sized and positioned to minimize climbing and allow food and water levels to be adjusted by an animal care technician. It will be understood that the food hopper 123, water bottle holder 125, and other aspects of the cage housing 133 may vary based on the number and / or type of animals housed.

[0022]

[0031] In some embodiments, the cage housing 133 may further include static vents 128 along the front surface of the cage top 131. In such a scenario, the static vents 128 are air permeable and serve to provide ventilation for the cage housing 133 in the event of a power outage or other unforeseen event.

[0023]

[0032] In various examples, the cage bottom 130 can be configured to be attachably coupled to the cage top 131. In such a scenario, the cage bottom 130 and the cage top 131 can be sized in accordance with European Directive 2010 / 63 / EU. Additionally or alternatively, the cage bottom 130 and the cage top 131 can comprise a transparent plastic material. As an example, the transparent plastic material can comprise polyethylene terephthalate glycol (PETG). In such a scenario, the top-down camera 122 can be configured to capture images through the transparent cage top 131 material. Additionally, the cage bottom 130 can be molded to include a floor area of ​​at least 80 square inches (516.13 square centimeters). Also, in some embodiments, the sidewalls of the cage bottom 130 and the cage top 131 can be at least 5 inches high. In some examples, the cage bottom 130 can be covered with bedding 134. In such an embodiment, the litter 134 may include at least one of paper litter, wood shavings, corn cobs, or cellulosic paper (eg, Alpha Dry).

[0024]

[0033] In an exemplary embodiment, cage data unit 120 may include two cameras positioned at different locations (e.g., top and top-side) and with different fields of view (e.g., top-down and oblique). In various examples, the cameras may record at 30 fps at a resolution of 4032 x 3040 pixels. For example, top-down camera 122 may be configured to capture images at 30 frames per second at ultra-high definition (UHD) 4K resolution. It will be understood that other camera resolutions and frame capture rates are possible and are contemplated herein.

[0025]

[0034] In some embodiments, cage data unit 120 may also include an oblique camera 127. In such a scenario, oblique camera 127 is configured to capture images of an oblique field of view 129. For example, oblique camera 127 may be configured to capture images at 30 frames per second at 2K (e.g., 2560 x 1440 pixels) resolution.

[0026]

[0035] The cage data unit 120 also includes an illumination module 140 configured to illuminate the top-down field of view 124. In some embodiments, the illumination module 140 can include multiple infrared light sources 142. For example, the multiple infrared light sources 142 can include one or more 940 nm near-infrared (NIR) light-emitting diodes (LEDs). It will be understood that infrared light sources that emit light at other infrared wavelengths (e.g., 900 nm or greater) are also possible and contemplated. In some embodiments, the emission wavelength of the infrared light source 142 can be selected based on wavelengths of light observable by an animal. By way of example, the emission wavelength of the infrared light source 142 can be outside the animal's visible range so as to be undetectable by the animal. The illumination module 140 can also include multiple visible light sources 144. In some examples, the multiple visible light sources 144 can include one or more 5000K white LEDs capable of providing 100 lux of illumination (at the cage floor). In some cases, a visible light source 144 may be utilized during daytime operation and an infrared light source 142 may be utilized during nighttime operation. In various embodiments, the illumination module 140 may be configured to illuminate the cage housing 133 through the transparent material of the cage top 131. In other words, the cage data unit 120 is a collection of electronics, a camera, and a housing that is mostly located above the cage housing 133 when inserted into the opening 111. In such a scenario, the cage housing 133 can be easily removed from the cage data unit 120 and the opening 111 without a lengthy or complicated cutting process.

[0027]

[0036] In an example, the multiple infrared light sources 142 and the multiple visible light sources 144 can be arranged in an interleaved arrangement 148. In such a scenario, the interleaved arrangement 148 can be selected to uniformly illuminate the cage housing 133 with visible and / or infrared light. In various examples, the lighting module 140 can include a light diffuser 146 that can be positioned along the downward-facing surface of the opening 111. The light diffuser 146 can effectively redirect / refract the light in a diffuse direction, distributing the light more evenly along the floor.

[0028]

[0037] The lighting module 140 is a fully controllable cage lighting system that provides 100 lux (at the cage floor) of 5000K white LEDs during the day and 940 nm near-infrared (NIR) LEDs at night with minimal impact on circadian rhythms. The system has LED persistence and robustness to ensure lighting does not change during soft reboots. Additionally, the cage data unit 120 can include a lighting sensor (e.g., light sensor 112) to determine ambient lighting. In such a scenario, the lighting module 140 can be controlled based on the amount of ambient light.

[0029]

[0038] Cage data unit 120 further includes housing 110 having openings 111 configured to accommodate cage top 131 and cage bottom 130, which together form cage housing 133. In some embodiments, housing 110 need not include openings 111. In such scenarios, cage data unit 120 and cage housing 133 can be configured to be optically coupled to one another. That is, cage data unit 120 can be configured to capture images through cage top 131 of cage housing 133. The cage is designed to allow food, water, and running wheels to be placed within the cage to minimize subject occlusion and allow proper camera focusing. Side observation slots are sized and positioned to minimize climbing and to allow food and water levels to be adjusted by an animal care technician.

[0030]

[0039] The system 100 is equipped with fully controllable cage airflow. In such embodiments, the cage data unit 120 also includes an air conditioning system 113 coupled to the cage housing 133. In various examples, the air conditioning system 113 can include at least one fan 114. By way of example, the air conditioning system 113 can provide dedicated supply and exhaust fans with a flow rate of up to 20 liters per minute (LPM). The air conditioning system 113 can further include a HEPA filter 115. For example, the HEPA filter 115 can include one or more high-quality 99.999% HEPA filters capable of generating positive or negative cage pressure. The air conditioning system 113 can include one or more air flow connectors 116 and a vibration isolation mechanism 117. In such a scenario, the vibration isolation mechanism 117 can effectively reduce vibration and / or noise levels that may adversely affect the animals 132. Additionally, the air conditioning system 113 can include one or more medical-grade flow sensors and can be configured to continuously measure airflow into and / or out of the cage housing 133. In the event of a power loss, a filter on top of the cage (e.g., static vent 128) can provide sufficient airflow.

[0031]

[0040] In various examples, the vibration isolation mechanism 117 may include a plurality of spring elements 119 coupling the at least one fan 114 to the housing 110. In such embodiments, the plurality of spring elements 119 may be configured to dampen vibrations caused by operation of the at least one fan 114. Damping the vibrations of the at least one fan 114 may effectively minimize the potential impact of sound and / or vibrations on housed animals.

[0032]

[0041] In some scenarios, the air conditioning system 113 may further include a PID controller 118, where "PID" stands for proportional, integral, and derivative. These are the three terms used to calculate the controller's output. The proportional term is calculated as the difference between the setpoint and the actual process value. The larger the difference, the larger the controller's output. The integral term is used to remove steady-state error. It is calculated as the sum of the error over time. The integral term increases the controller's output as long as there is an error. The derivative term is used to prevent overshoot. The derivative term is calculated as the rate of change of the error. The derivative term decreases the controller's output as the error increases.

[0033]

[0042] The PID controller 118 can be configured to control the at least one fan 114 to provide a constant pressure or constant flow rate of air to the cage housing 133 when within the opening 111 of the housing 110. The PID controller 118 acts as a control loop mechanism that uses feedback to continuously regulate and control the flow of air into the cage housing 133.

[0034]

[0043] In some embodiments, the cage housing 133 and / or the air conditioning system 113 may include a plurality of air flow connectors 116 configured to connect a rear of the cage housing 133 to the air conditioning system 113. In such a scenario, the air flow connectors 116 are configured to attachably couple to the air conditioning system 113 when the cage housing 133 is installed within the opening 111.

[0035]

[0044] Cage data unit 120 further includes a controller 150 operable to perform operations, including causing top-down camera 122 to capture an image 126 of top-down field of view 124. It will be understood that the operation of controller 150 relates to controlling many functions of system 100, as described herein.

[0036]

[0045] In some exemplary embodiments, the operation of the controller 150 may also include identifying one or more animals from at least some of the captured images. Such identification may be performed using a trained machine learning model.

[0037]

[0046] In some examples, these operations also include assigning a bounding box based on each identified animal and determining the centroid of the bounding box.

[0047] Additionally, the operations may include determining a segmentation mask within the bounding box.

[0038]

[0048] In some examples, the actions may include using the trained machine learning model 160 to classify at least some of the captured images 126 as associated with at least one animal behavior type among a plurality of animal behavior types 162. The plurality of animal behavior types 162 include movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizure, gait, rearing, and scratching. It will be understood that other animal behavior types 162 are possible and contemplated.

[0039]

[0049] The operations may further include using the trained machine learning model 160 to assign at least one digital biomarker of a plurality of digital biomarkers 164 to at least a portion of the captured image 126. In such a scenario, the plurality of digital biomarkers 164 may include heart rate, respiration rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), and skin temperature. Additionally or alternatively, the digital biomarkers 164 may include assessments of fur condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, the animal's overall health, disease status, scratching, and various activities including marble burying or nesting. It will be understood that other digital biomarkers 164 are possible and contemplated.

[0040]

[0050] In various examples, the controller 150 can include multiple processors 152, such as graphics processor units (GPUs) and central processing units (CPUs), and memory 154. In various embodiments, the GPUs, CPUs, and memory 154 can be coupled to a shared substrate 156. Additionally, or alternatively, the controller can be specifically configured to perform machine learning tasks. As an example, the controller 150 can be an NVIDIA Jetson Nano series module. Other computing modules are possible and contemplated.

[0041]

[0051] In some examples, the cage data unit 120 may further include a communications interface 190. In such a scenario, the communications interface 190 may be configured to communicatively couple the controller 150 to at least one of the gateway 10 or the central server 12.

[0042]

[0052] In some exemplary embodiments, gateway 10 may include a Supermicro box with an Intel i7 CPU and a TPM 2.0 module responsible for uploading video and metrics to the cloud. In some embodiments, gateway 10 may have 1-2 TB of disk space for buffering video in the event of a loss of internet connectivity, sufficient for buffering at least 24 hours of video. Gateway 10 connects to a hardware switch, allowing units (e.g., system 100) to be logically located behind gateway 10 and hidden from the site network where they are installed. In some embodiments, a single gateway 10 can support up to 10 cage units, and appropriate communication with the units uses AWS IoT Greengrass, an open-source edge runtime and cloud service that helps build, deploy, and manage device software. Greengrass enables the extension of AWS IoT capabilities to devices, enabling them to collect and analyze data locally, react to local events, and securely communicate with each other. AWS Greengrass also orchestrates Docker containers to automate the deployment of data in a consistent, usable, and scalable manner.

[0043]

[0053] To handle background processing, Redis manages task queues and uploads to the Cloud Video API and Metrics API. Redis is an open-source in-memory data structure store that can be used as a database, cache, message broker, and streaming engine. The raw 4K video is uploaded to an S3 bucket dedicated to AWS customers, and a Kafka bus is used to handle the high-volume, high-throughput, and low-latency video and metrics streams.

[0044]

[0054] In an exemplary embodiment, cage data unit 120 may include a user interface 180. In such a scenario, user interface 180 may include a touchscreen 182 disposed along the front of the housing. Further, in such an example, operation may include adjusting at least one operational aspect of system 100 in response to receiving a user command via touchscreen 182.

[0045]

[0055] In some embodiments, the cage housing 133 can include at least one furniture object 164 positioned along the cage bottom 130. In such examples, the furniture object 164 can include at least one of a running wheel, a rolling wheel, a rolling bar, or a ladder. As an example, a running wheel, along with other objects (chew blocks, hidden structures) can provide insight into the animal's activity and provide an enriched environment. Other types of furniture objects relevant to the study of animal behavior are possible and contemplated.

[0046]

[0056] In various examples, the cage data unit 120 can also include a light sensor 112. In such a scenario, the light sensor 112 can be configured to provide information indicative of the ambient light level to the controller 150. In one embodiment, the light sensor 112 can be configured to sense the brightness of the ambient light in visible and / or infrared wavelengths. The information provided by the light sensor 112 can be used to control the lighting module 140. For example, operation can include causing at least one of the infrared light source 142 or the visible light source 144 to illuminate the cage bottom 130 based on the ambient light level and / or the time of day.

[0047]

[0057] The light sensor 112 can be utilized to control other elements of the system 100. For example, the operation of the controller 150 can also include adjusting the exposure settings of the top-down camera 122 based on the ambient light level and / or adjusting the capture mode of the top-down camera 122 based on the ambient light level.

[0048]

[0058] 6 illustrates a schematic block diagram 600 according to an exemplary embodiment. Diagram 600 illustrates how a digital cage (e.g., system 100) communicatively interacts with a gateway (e.g., gateway 10), a cloud service (e.g., central server 12), and / or the Internet. As illustrated in diagram 600, video information can be provided to the gateway via hypertext transfer protocol (HTTP) requests and responses. Various versions of HTTP (e.g., HTTP / 2 or HTTP / 3) are possible and contemplated.

[0049]

[0059] As shown in diagram 600, Message Queuing Telemetry Transport (MQTT) can be used to provide various information or metrics from the system 100 to the server 10. MQTT is a lightweight publish-subscribe messaging protocol that can be used to connect remote devices with a small code footprint and minimal network bandwidth.

[0050]

[0060] 7 illustrates system 700, which may be similar to or identical to system 100, according to an exemplary embodiment. As shown in FIG. 7, system 700 may include a cage data unit (e.g., cage data unit 120) and a cage housing (e.g., cage housing 133) that may be inserted into an opening in the cage data unit's housing (e.g., housing 110). In some embodiments, the cage housing may include a cage top (e.g., cage top 131) and a cage bottom (e.g., cage bottom 130). In various examples, the cage top may include a static vent (e.g., static vent 128). II. Exemplary Rackmount Cage System

[0061] Traditional animal husbandry racks are designed to accommodate many cages in a small footprint, sometimes providing individual ventilation for each cage position. More recently, animal husbandry racks have integrated electronics into the rack to support continuous monitoring of the animals and provide feedback to operators managing the animal care. Unfortunately, these integrated electronics generate heat that must be managed to protect both the electronics and the animals.

[0051]

[0062] Provided herein is an improved animal housing rack design that utilizes (or "leverages") the continuous airflow and exhaust of individually ventilated cage racks to capture and exhaust heat from associated electronic equipment without the need for additional fans and without the risk of introducing unfiltered, "dirty" air into a room containing one or more cages housing laboratory animals.

[0052]

[0063] In these designs, electronic equipment is housed in one or more sealed compartments, each with air intakes and exhausts connected to the rack's blower and exhaust structures in the same way as an individually ventilated cage, thereby utilizing these systems to absorb and dissipate heat associated with the electronic equipment.

[0053]

[0064] Further improvements to the animal housing rack include the ability to individually attach each cage position to a horizontal plenum on the rack via a clip mechanism that holds the cage rails while ensuring an airtight connection for the cage's air intake and exhaust. By individually clipping the rails to each cage position, different cage types can be placed in each cage position, allowing the rack to be assembled modularly as needed. For example, to enable digital biomarker capture, some cage positions on the rack can be equipped with simple cage rails, while others can be equipped with video capture tops and rails. As video capture demand increases, non-video cage positions can be converted to video cage positions.

[0054]

[0065] A further improvement to animal rearing racks is provided herein: a home cage design that allows for continuous video recording and real-time processing of the video into digital biomarkers. When capturing home cage video, each cage must be illuminated with both visible and infrared (IR) LEDs to observe animals during both light and dark cycles, ensuring the camera has sufficient sensitivity in both ranges of the spectrum. Because IR focuses on a different plane than visible light, focusing the camera lens can be challenging. This separation of focal planes is exacerbated by the short focal length required in rearing cages. In the design provided herein, each cage position is equipped with one or more cameras that view the cage from a top-down or oblique perspective, as well as one or more GPU systems for real-time processing of the video into digital biomarkers. To provide a complete view of the entire cage while also capturing detailed, high-spatial-resolution video of individual animals, the system includes multiple video cameras. A single wide-angle camera captures the entire cage footprint, while one or more cameras with zoom lenses capture higher-resolution video and are equipped with both visible and infrared LEDs to illuminate each cage. To ensure an unobstructed view of the entire cage area, these designs also feature food hoppers, running wheels, and water bottle holders around the outer edges of the cage.

[0055]

[0066] A further improvement to the animal rearing rack is provided herein, which has a design with a touchscreen at each cage location. The touchscreen can function as a digital cage card connected to a central system managing the animals and can automatically detect and alert the technician managing the animals in the cage to special requirements for a particular cage or issues with the cage location that the technician needs to address. The touchscreen allows the technician to enter information into the central system or use the screen to request veterinarian or other support. For example, the system can alert the technician to a potential problem requiring attention and provide details about the nature of the problem on the screen. The technician can address the issue and enter information via the touchscreen to report the status. If the issue needs to be escalated to a veterinarian or other personnel with more or different experience, the technician can escalate via the touchscreen, thereby sending an alert to the veterinarian or other personnel. The veterinarian or other personnel can then log into a web-based system, view the cage via a live video feed, and provide instructions to the technician on how to resolve the issue. The instructions are returned to the cage-level screen, informing the technician on how to resolve the issue. Once completed, the technician can view the status via the touchscreen.

[0056]

[0067] 2 illustrates a rack-mounted cage system 200 according to an exemplary embodiment. In such an embodiment, individual cage elements (e.g., system 100) can be arranged in dedicated racks that can include air fans, filters, lighting, cameras, sensors, and a controller (e.g., a Jetson Nano single-board computer) for component control, data acquisition, optional local processing, or data cloud transfer. Each cage in rack-mounted cage system 200 can be specifically identified and controlled using a touchscreen for digital cage care and basic data entry and readout.

[0057]

[0068] Rack mount cage system 200 may include a base frame 204. Rack mount cage system 200 may also include a plurality of vertical members 206 coupled to base frame 204. In an exemplary embodiment, rack mount cage system 200 includes a plurality of bays 208 formed by spaces between the plurality of vertical members 206. In such a scenario, at least a portion of the plurality of bays may be configured to house a cage system that may be similar or identical to system 100 shown and described with reference to FIG. 1 . For example, each of the plurality of bays 208 may house a cage data unit (e.g., cage data unit 120) having a housing (e.g., housing 110). The housing may include an opening (e.g., opening 111) configured to house a cage top (e.g., cage top 131) and a cage bottom (e.g., cage bottom 130). In such a scenario, the cage top may be attached to the cage bottom to form a cage housing (e.g., cage housing 133). In other embodiments, housing 110 need not include an opening. In such an example, the cage housing may be configured to be optically coupled with the cage data unit. For example, the cage housing can be coupled to a rack mount system so that it is mounted near the cage data unit.

[0058]

[0069] As described elsewhere herein, the cage data unit may include a top-down camera (e.g., top-down camera 122) configured to capture an image (e.g., image 126) of a top-down field of view (e.g., top-down field of view 124) including one or more animals (e.g., animal 132) contained within the cage housing. The cage data unit may also include a lighting module (e.g., lighting module 140) configured to illuminate the top-down field of view.

[0059]

[0070] Each cage data unit can also include an air conditioning system (e.g., air conditioning system 113) coupled to a cage housing (e.g., cage housing 133) and a controller (e.g., controller 150) operable to perform an action. The action can include causing the top-down camera to capture an image of a top-down field of view.

[0060]

[0071] The operations may also include identifying one or more animals from at least a portion of the captured image. The identifying may be performed using a trained machine learning model. Additionally or alternatively, the operations may include assigning a bounding box based on each identified animal and determining a centroid of the bounding box. Further, the operations may include determining a segmentation mask within the bounding box.

[0061]

[0072] In some embodiments, the operations may also include using a trained machine learning model (e.g., trained machine learning model 160) to classify at least a portion of the captured images as associated with at least one animal behavior type among a plurality of behavior types (e.g., behavior type 162). The plurality of animal behavior types may include movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. In some examples, other types of animal behaviors are possible.

[0062]

[0073] Additionally or alternatively, the operations may also include assigning at least one digital biomarker (e.g., digital biomarker 164) of a plurality of digital biomarkers to at least a portion of the captured image using a trained machine learning model. In various embodiments, the plurality of digital biomarkers may include heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), and skin temperature. Additionally or alternatively, the digital biomarkers may include assessments of coat condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, the animal's overall health, disease status, scratching, and various activities including marble burying or nesting.

[0063]

[0074] In various embodiments, the controller may include a graphics processor unit (GPU), a central processing unit (CPU), and memory. In such a scenario, the GPU, CPU, and memory are coupled to a shared substrate (e.g., shared substrate 156). In some embodiments, the controller may be configured to perform machine learning tasks. Additionally or alternatively, the controller may be an NVIDIA Jetson Nano series module. In some embodiments, the controller may include an NVIDIA Maxwell GPU, which can provide up to 1.4 TFLOPS of performance. The controller may include 4 GB of LPDDR4 RAM, allowing multiple neural networks to run in parallel. The controller may also include 16 GB of eMMC storage. The controller may include a MicroSD card slot, a USB 3.0 port, one or more HDMI ports, and / or one or more MicroUSB power ports.

[0064]

[0075] In some examples, rack mount cage system 200 can include various systems that can be distributed and / or shared among multiple cages within each bay. As one example, each bay's respective air conditioning system is coupled to a shared tower blower unit or house air 210. Additionally or alternatively, rack mount cage system 200 can include power supply 212. In such a scenario, at least some of the bays 208 and corresponding cages can be configured to receive power via power supply 212.

[0065]

[0076] In various embodiments, the rack mount cage system 200 can include at least some of the bays 208 configured to house drawers, shelves 214, or environmental monitoring units 216. In some embodiments, the environmental monitoring units 216 can include measuring and recording various environmental metrics, including sound / acoustics, vibration, airflow, air quality, CO2 and ammonia levels, temperature, and humidity.

[0066]

[0077] In some embodiments, the rack-mounted cage system 200 can be used to obtain visual cage surveys. For example, a cage identifier 218 can be positioned within the camera's field of view. The cage identifier 218 can include a barcode, a QR code, an encoded pattern, or another type of visible or infrared identifier or symbol. The cage identifier 218 can be etched into plastic, attached with a sticker, or printed on the back of a cage card. The cage data unit 120 can be configured to recognize the identity of a cage based on the cage identifier 218 present in images captured by one or more of the cameras. The cage data unit 120 can use this information to associate data acquired by a particular camera with a cage and the animals residing therein. This functionality advantageously enables continuous data recording as cages are moved from slot to slot within the rack. Such visual cage surveys can also provide the number of cages in the rack, the number of associated animals, and other metrics related to the census storage of animals within the animal housing facility.

[0067]

[0078] In some embodiments, the rack mount cage system 200 may include a visual identification module configured to recognize a unique cage identifier 218 associated with the cage housing 133. In some embodiments, the unique cage identifier 218 may include at least one of a barcode, a QR code, or another graphical identifier. The unique cage identifier 218 may be located within a top-down field of view of the top-down camera. In some embodiments, the visual identification module is operable to recognize the unique identifier from an image captured by the top-down camera. In examples, the unique cage identifier 218 may be integrated into the cage housing 133 by at least one of etching it into a surface of the cage, attaching a label as a sticker, or printing it on a cage card. In some such examples, the operations of the controller may also include associating data captured by the top-down camera with a particular cage housing based on the recognized unique cage identifier 218. In various examples, the operations may include maintaining a continuous record of the cage housing 133 and one or more animals therein as the cage housing is rearranged within the rack mount cage system 200.

[0068]

[0079] In some examples, the operations may also include using the recognized unique cage identifier 218 to track the number of cage housings within the rack mount cage system 200.

[0069] These operations may also include compiling data related to the number of animals in each cage house. Additionally, the operations may include conducting a census of animals within the animal housing facility based on the tracked cage housing and aggregated animal data.

[0070]

[0080] In some examples, the visual identification module 220 may include an image processing algorithm configured to decode the unique cage identifier 218 from the captured image. Additionally or alternatively, the visual identification module 220 may include a database for storing associations between the decoded unique identifiers and the respective cage housings.

[0071]

[0081] In various embodiments, the operations may include updating the database in real time when a unique cage identifier is recognized or when a cage housing is added to or removed from the rack mount cage system 200.

[0072]

[0082] In some examples, the user interface (e.g., user interface 180 or a remote user interface) can be configured to display information about the animal census, including the total number of cage housings, the total number of animals, and the distribution of animals among different cage housings. In such a scenario, the user interface is further configured to allow manual updating of information related to particular cage housings or animals based on visual inspection or additional data entry.

[0073]

[0083] In some embodiments, the unique cage identifier is configured to be durable and resistant to environmental conditions within an animal facility, including humidity, temperature fluctuations, and cleaning processes.

[0074]

[0084] FIG. 8A illustrates a rack-mounted cage system 800 according to an exemplary embodiment. The rack-mounted cage system 800 may be similar to or identical to the rack-mounted cage system 200. In various embodiments, the rack-mounted cage system 800 may include 15 bays (e.g., bay 208) arranged in a 5x3 array. The bays may accommodate 15 cage data units (e.g., cage data units 120) and 15 cage housings (e.g., cage housing 133). Rack-mounted cage systems with a greater or fewer number of bays and / or cage data units and / or cage housings are possible and contemplated. As described above, the rack-mounted cage system 800 may include various shared resources, such as a power source (e.g., power source 212), an environmental monitoring unit (e.g., environmental monitoring unit 216), and / or a shared air conditioning unit (e.g., a shared tower blower or house air 210).

[0075]

[0085] FIG. 8B illustrates a rack mount cage system 820 according to an exemplary embodiment.

[0076] III. Exemplary Methods

[0086] The digital biomarker development process is a complex, multifaceted undertaking that requires cross-functional team collaboration. To facilitate this collaboration, this disclosure describes a set of tools developed to enable research sharing both within and outside of an organization. These tools also provide access to video and metrics through export features within the applications. A Kubernetes-based data science workspace provides a secure, scalable environment for algorithm development and access to key research data. This set of tools is designed to streamline the digital biomarker development process and enable teams to collaborate more effectively.

[0077]

[0087] The Data Science Workspace is an end-to-end system designed to facilitate the creation, management, and collaboration of in vivo studies for digital biomarker development and visualization. The system allows users to create and manage studies, define groups (including multiple groups per study), define cages and animals (numerous cages per group and up to three animals per cage), and import groups, cages, and animals from Excel files. Users can start and stop recording, manage animals (including marking animals as dead or euthanized, adding animals to cages, or removing animals from cages), and access interactive plots and videos. The system currently supports these metrics: movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. Users can visualize metrics across the entire study, limit the timeline to the current day, 24 hours, or 7 days by clicking a button, or zoom in by selecting a plot. Users can navigate through video recorded from both the top and side cameras, select points on the plot to snap a video at that time, and click the video to expand it to full screen for more detailed visualizations. Users can download one-minute video clips and use them in presentations or reports. Three video options are available: standard, high contrast, and overlay. The overlay shows segmentation, bounding boxes, and key points derived from the machine learning pipeline. Users can export group- or cage-level research metrics as .csv files. Annotations are also supported, allowing users to create comments or notes (highlighting importance with @person or #tag), pin videos to specific dates and times, reply inline to facilitate communication, and share research with colleagues inside and outside the organization. Guest users are allowed but can be limited to read and annotation access only.

[0078]

[0088] 3 illustrates a method 300 according to an exemplary embodiment. It will be understood that the method 300 may include fewer or more steps or blocks than those explicitly illustrated or disclosed herein. Furthermore, each step or block of the method 300 may be performed in any other order, and each step or block may be performed one or more times. In some embodiments, some or all of the blocks or steps of the method 300 may be performed by the controller 150, the gateway 10, the central server 12, and / or other elements of the system 100 and / or the rack mount cage system 200, as illustrated and described in connection with FIGS. 1 and 2.

[0079]

[0089] 9 illustrates a user interface 900 according to an exemplary embodiment. The user interface 900 may include various windows or regions. As an example, the user interface 900 may include an average movement graph 902, a wheel occupancy graph 904, and a water dispenser occupancy graph 906. Additionally or alternatively, the user interface 900 may include a live or historical video stream viewer 908. Additionally, the user interface 900 may include a cage feed 910.

[0080]

[0090] Block 302 includes receiving, via a central server (e.g., central server 12), an image (e.g., captured image 126) of a top-down field of view (e.g., top-down field of view 124). In some examples, the image includes a live or past image of one or more animals (e.g., animals 132) housed within a cage housing (e.g., cage housing 133).

[0081]

[0091] Block 304 includes displaying a user interface (e.g., user interface 180) via a display. In some examples, the user interface may include a video stream viewer (e.g., video stream viewer 908). In such a scenario, the video stream viewer may be configured to display a user-navigable stream of live or past images.

[0082]

[0092] The user interface may also include at least one data graph that may include information indicative of at least one of average travel speed (e.g., average travel graph 902), wheel occupancy (e.g., wheel occupancy graph 904), or water dispenser occupancy (e.g., water dispenser occupancy graph 906).

[0083]

[0093] In various examples, the user interface can further include an annotation feed (e.g., cage feed 908). In such a scenario, the annotation feed includes information about animal behavior types or digital biomarkers associated with one or more animals.

[0084]

[0094] In some embodiments, displaying the user interface may include displaying live or historical information via at least one of a video stream viewer, at least one data graph, or an annotation feed.

[0085]

[0095] In some examples, the method 300 may also include capturing the image using a top-down camera (e.g., top-down camera 122) disposed within a cage data unit (e.g., cage data unit 120) that is attachable to and optically coupleable to the cage housing (e.g., cage housing 133).

[0086]

[0096] In an exemplary embodiment, method 300 may also, or alternatively, include using a local computing device (e.g., controller 150) and a trained machine learning model (e.g., trained machine learning model 160) to classify at least a portion of the images as associated with at least one animal behavior type among a plurality of behavior types (e.g., animal behavior types 162). In some examples, the plurality of animal behavior types may include at least one of movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching.

[0087]

[0097] Additionally, method 300 may also include, in response to classifying at least a portion of the image as associated with at least one animal behavior type, adding a new annotation to the annotation feed indicating the at least one animal behavior type along with a link to a corresponding video clip.

[0088]

[0098] In some embodiments, the new annotation may include at least one of a free-form comment, a hashtag taxonomy group, and / or an "at" username reference to a user (e.g., @username).

[0089]

[0099] Additionally, method 300 may also include using a local computing device and a trained machine learning model to assign, to at least a portion of the image, at least one digital biomarker of a plurality of digital biomarkers (e.g., digital biomarker 164). In some examples, the plurality of digital biomarkers may include heart rate, respiration rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), and skin temperature.

[0090]

[0100] In some examples, the method 300 can include determining a location of a particular animal within the image using a local computing device and a trained machine learning model. In such a scenario, determining the location of the particular animal can include applying an image segmentation mask to the image to enable recognition of the particular animal within the image.

[0091]

[0101] In some embodiments, determining the location of a particular animal may include applying an object detection method, which in such a scenario includes at least one of a thresholding method, an edge detection method, or a clustering method.

[0092]

[0102] In various examples, displaying the user interface may also include displaying at least one bounding box corresponding to the location of the particular animal via the video stream viewer.

[0093]

[0103] Method 300 may also, or alternatively, include assigning an identifier to the particular animal using a local computing device and a trained machine learning model, in such a scenario, the identifier may be based on at least one of an ear tag identifier or a tail tattoo identifier.

[0094]

[0104] The method 300 may also include dynamically tracking the location of a particular animal using a local computing device and a trained machine learning model.

[0095]

[0105] Further, method 300 may include determining a cage-in or cage-out state based on the image. In such a scenario, a cage-in state may include the cage housing being in a desired position (e.g., properly positioned within opening 111). A cage-out state may include the cage housing not being in a desired position. Method 300 may also include displaying, via a user interface, the cage-in or cage-out state.

[0096]

[0106] The currently disclosed model accepts frames in a video stream and makes predictions about animal (e.g., mouse or rat) detection, instance segmentation, key points, and the location of other objects such as running wheels, water tubes, and feeding areas. The model can process one minute of 760x1008 video at 30 fps and inference in 30.2 seconds at 60 fps, making it useful for collecting fundamental metrics for multiple animals in real time. Furthermore, the model uses a shared backbone for all of its various tasks, making it useful for running several of the models separately on-device or at the edge. Video inference treats all incoming frames independently, so inference can be distributed across batches across different time steps or sources.

[0097]

[0107] Hydra-style models use a shared backbone as a feature extractor that outputs a pyramid of feature maps, which are then sent to various heads responsible for different prediction tasks. A hydra-style model is a type of machine learning (ML) model composed of multiple smaller models, each responsible for learning different aspects of the problem. The smaller models are connected to share information and perform tasks collaboratively. The overall model has 5.52 million parameters, making it very lightweight. For example, the trained model can be efficient in terms of parameter space and / or overall computational bandwidth. During training, each head outputs a scalar loss, and the total loss is the weighted sum of the losses of all individual tasks. The final loss weight is considered an adjustable hyperparameter.

[0098]

[0108] The Hydra model is very lightweight, requiring only 5.52 million parameters to provide instance segmentation, detection, and critical point prediction. Therefore, training the model is relatively fast, requiring only 10–12 hours of operation on a single A100 GPU. This model is running in production and can process one minute of video at 30 fps in approximately 22 seconds. Each cage in the system outputs 2,592,000 frames every 24 hours, equivalent to 26 terapixels per cage per day (10 megapixels × 2.6 million captures). Because there is no shortage of training data to choose from, the embodiments described herein provide a continuous learning pipeline that selectively samples from production data based on several criteria. Because labeling critical points and polygons can be time-consuming and computationally expensive, this becomes an optimization problem in which the desired goal is to minimize the number of samples while maximizing the information gain and distribution expansion of the training data input to the model.

[0099]

[0109] To enable the extraction of features necessary to reliably and thoroughly identify digital biomarkers from video-generated data in real time, continuously, and at reasonable cost, the systems and methods described herein can easily capture high frame rate and high spatial resolution video from the complete home cage environment, along with other potential sensor data streams, and process said video or other data streams through a collection of machine learning models to extract many different digital biomarkers in real time using local ("edge") computing capabilities. The system uses (or "leverages") one or more high-resolution digital cameras at the level of the cage or collection of cages, nearby an embedded computing system and a collection of custom machine vision models organized in a Hydra framework on said computer.

[0100]

[0110] Beneficial features of this system include the local proximity of cameras or other sensors to the embedded computing platform that runs this collection of machine vision models organized in the Hydra framework. This arrangement provides cost-effective and efficient local data collection and processing, thereby minimizing the time and cost of transferring biomarker-related information to cloud storage.

[0101]

[0111] In some embodiments, loss masking can be used to utilize all data even if some tasks are not labeled for a batch of data. For example, if some data contains only polygons, you can flag the key point data as missing and mask the key point loss for those instances, and still use such information during key point training.

[0102]

[0112] The development of a digitally enabled platform for computer vision-based monitoring of laboratory animal behavior represents a major advancement in the field of drug development and other related applications. This platform offers several key features and advantages that address the limitations of traditional approaches to behavioral phenotyping and provide a path to enhanced research capabilities. Platform scalability is one of these. Traditional approaches to behavioral phenotyping often struggle to accommodate complex studies using large numbers of animals. However, this platform offers the ability to scale to multi-cage stacks, enabling researchers to conduct broader and more detailed investigations. The integrated data science workbench further enhances scalability by providing computational scientists and biostatisticians with a powerful platform to streamline and automate workflows related to database and machine learning analysis. This not only improves efficiency but also facilitates collaboration among team members through annotation and shared learning capabilities built into the user interface.

[0103]

[0113] This platform addresses challenges associated with data sharing and standardization in the pharmaceutical industry. Its cloud-based infrastructure allows easy access to data collected by veterinary specialists and behavioral scientists. An intuitive user interface provides live monitoring and visualization of digital biomarkers, enabling the identification of significant events and behavioral or physiological changes. This promotes collaboration and knowledge sharing among various research teams, and even between academia and the pharmaceutical industry. The platform can acquire, store, integrate, and analyze large-scale video analytics, yielding novel insights and facilitating the development of clinically relevant biomarkers from in vivo studies. The detection head is based on YOLOX, an anchor-free single-stage object detection algorithm. YOLOX uses separate heads for classification and regression tasks, eliminating the anchor box and employing a novel training strategy called SimOTA, which improves training speed and accuracy. In some examples, the detection head is configured to detect animals such as mice, running wheels, food containers, and water containers.

[0104]

[0114] The segmentation head outputs a feature map the same size as the original-resolution image, with each pixel labeled as mouse or non-mouse. The decoder then runs as a top-down path from the highest level of the pyramid, merging and upsampling the feature maps at each step up to level 3 of the pyramid. The final prediction is obtained by upsampling the original frame-resolution output. The challenge then arises when segmenting multiple animals, where each pixel labeled as mouse must be individually distinguished. Each pixel in the segmentation map can be assigned to its nearest centroid (the centroid comes from the keypoint head). Because mice are not convex objects, traditional systems often fail to segment this type of instance. For example, one mouse's tail may be located very close to the centroid of another mouse. Furthermore, the borders where mice meet are linear. To overcome this, an offset factor and / or another type of adjustment can be added to the model, which performs the task of calculating which centroid a pixel is closest to by voting for each pixel and acts as a bias term when calculating the pixel's nearest centroid.

[0105]

[0115] Keypoints Head predicts five important keypoints: left ear, right ear, nose tip, tail base, and tail tip. Each keypoint is represented as a 2D isotropic Gaussian distribution whose mean is the keypoint coordinate and whose variance is an adjustable hyperparameter. The decoding step is similar to how a segmentation head merges and upsamples feature maps. There can be more or fewer keypoints, It is expected.

[0106]

[0116] Once the model is trained, the inference step performs several subtasks, including linking keypoints to individual animals. The first step is to use the YOLOX detection head to output a collection of bounding box proposals for the animal of interest. Simultaneously, the segmentation head outputs a proposal for the presence or absence of the animal for each pixel. To obtain instance segmentations (different instances of the animal), the model uses centroids and offsets (each pixel essentially votes which centroid it is closest to using an offset vector). Finally, for each detected animal, all keypoints are associated with the instance by conditioning on the already computed instance segmentations and boxes.

[0107]

[0117] While the model is making real-time predictions, the system creates a basin of images where, for example, the number of detected animals does not match the expected number of animals in a cage, or a polygon does not meet certain requirements, or some important points are missed or placed in the wrong place. Given the number of frames, this basin can fill up quickly, making selective sampling from the basin for retraining crucial. To do this, the retraining system uses image embeddings to cluster similar images close to each other and stratified sampling to attempt to sample a variety of "miss" cases in a representative manner.

[0108]

[0118] To accelerate inference, NVIDIA's TensorRT framework can be utilized. In such scenarios, such capabilities give the inference engine access to dynamic batching and concurrent model execution through NVIDIA's Triton.

[0109]

[0119] 4 illustrates a method 400 according to an exemplary embodiment. It will be understood that the method 400 may include fewer or more steps or blocks than those explicitly illustrated or disclosed herein. Furthermore, each step or block of the method 400 may be performed in any other order, and each step or block may be performed one or more times. In some embodiments, some or all of the blocks or steps of the method 400 may be performed by the controller 150, the gateway 10, the central server 12, and / or other elements of the system 100 and / or the rack mount cage system 200, as illustrated and described in connection with FIGS. 1 and 2.

[0110]

[0120] Block 402 includes receiving an image (e.g., captured image 126) of a top-down field of view (e.g., top-down field of view 124). In such a scenario, the image includes a live or past image of one or more animals (e.g., animals 132) housed within a cage housing (e.g., cage housing 133).

[0111]

[0121] Block 404 includes using a trained machine learning model (e.g., trained machine learning model 160) to determine the location of the particular animal in the image. In such a scenario, determining the location of the particular animal includes applying an image segmentation mask. In some examples, determining the location of the particular animal may include applying an object detection method. In such a scenario, the object detection method includes at least one of a thresholding method, an edge detection method, or a clustering method.

[0112]

[0122] Block 406 includes using the trained machine learning model to assign an identifier to the particular animal. In some examples, the identifier can be based on at least one of an ear tag identifier or a tail tattoo identifier.

[0113]

[0123] Block 408 involves assigning at least one bounding box that corresponds to the location of a particular animal within the image.

[0124] In some examples, method 400 further includes using the trained machine learning model to classify at least a portion of the images as associated with at least one animal behavior type (e.g., animal behavior type 162) of a plurality of animal behavior types. In some examples, the plurality of animal behavior types include movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching.

[0114]

[0125] Additionally or alternatively, method 400 may include using a trained machine learning model to assign at least one digital biomarker (e.g., digital biomarker 164) of a plurality of digital biomarkers to at least a portion of the image. In such a scenario, the plurality of digital biomarkers may include heart rate, respiration rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), and skin temperature. Additionally or alternatively, the digital biomarkers may include assessments of coat condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, the animal's overall health, disease status, scratching, and activities including marble burying or nesting.

[0115]

[0126] In various embodiments, the method 400 may include using a trained machine learning model to dynamically track the location of a particular animal.

[0127] In various examples, the method 400 can include determining the pose of the animal. In such a scenario, multiple key points on the target animal can be identified and tracked. In doing so, traditional object position information can be augmented with object orientation vector information based on prior vector information to provide a probability of how the object may be expected to move.

[0116]

[0128] In a particular embodiment, using a laboratory mouse as an example, such pose estimation reveals the direction the mouse is facing. Because a mouse facing a particular direction has limited degrees of freedom to turn, tracking algorithms can use (or "exploit") the pose estimation to improve information about the animal's movement and provide greater accuracy.

[0117]

[0129] Similar or identical techniques that leverage pose estimation to improve the accuracy of tracking algorithms can also be applied to other tasks beyond object tracking, such as measuring how an object interacts with other objects in its environment. For example, biological studies utilizing animal models can quantify the amount of time an animal spends performing a particular task or activity (such as the time spent at a running wheel, a feeder, a waterer, or some other element in the environment). Standard approaches to this problem that use or leverage object position as input (in the form of a center of mass, bounding box, etc.) can produce false positives because the tracked object may be in close proximity to environmental objects without being directly involved.

[0118]

[0130] For example, if the tracked mouse is sitting under a running wheel, traditional object tracking methods that only utilize the object's position may produce false positives.

[0119]

[0131] In contrast, in the first step, using the means and methods provided herein, a pose estimate of the tracked object is extracted and used as input to an algorithm that measures the impact of environmental objects in a conventional object tracking algorithm. Because the set of possible poses in which the tracked object interacts with the environmental object is limited, false positives are reduced and the performance of the tracking algorithm is improved. In a specific example, a mouse sitting under a running wheel but with its body axis oriented perpendicular to the plane of rotation of the running wheel would trigger a false positive in a conventional tracking approach, but using the pose estimation-based approach provided herein, the mouse would correctly be interpreted as not running on the wheel.

[0120]

[0132] These improvements are generally applicable to any moving object that has properties related to its position or orientation. In certain embodiments, the means and methods provided herein are applicable to any experimental situation in which the direction or orientation of a laboratory animal is a relevant factor in whether the animal is interacting with, rather than simply being in proximity to, elements in its environment, including other animals. These means and methods are also applicable to situations in which humans are confined to a specific, defined space whose activity is monitored, such as individuals incarcerated in a prison, inanimate objects such as automobiles, and especially autonomously driven objects (e.g., by computer means rather than human action).

[0121]

[0133] In some examples, the method 400 can include determining a cage-in or cage-out state based on the image. In such a scenario, the cage-in state can include the cage housing being in a desired position. Conversely, the cage-out state can include the cage housing not being in a desired position.

[0122]

[0134] 5 illustrates a method 500 for training a machine learning model, according to an example embodiment. It will be understood that method 500 may include fewer or more steps or blocks than those explicitly illustrated or disclosed herein. Furthermore, each step or block of method 500 may be performed in any other order, and each step or block may be performed one or more times. In some embodiments, some or all of the blocks or steps of method 500 may be performed by controller 150, gateway 10, central server 12, and / or other elements of system 100 and / or rack mount cage system 200, as illustrated and described in connection with FIGS. 1 and 2 .

[0123]

[0135] Block 502 includes receiving as training data a plurality of images (e.g., captured images 126) of a top-down field of view (e.g., top-down field of view 124) including one or more animals (e.g., animals 132) housed in a cage housing (e.g., cage housing 133).

[0124]

[0136] Block 504 also includes training a machine learning model using an unsupervised learning method based on the training data to form a trained machine learning model (e.g., trained machine learning model 160). In such a scenario, the unsupervised learning method includes at least one of k-means clustering, hierarchical clustering, or density-based clustering.

[0125]

[0137] In various embodiments, the method 500 can include identifying at least one particular animal based on the image.

[0138] Additionally or alternatively, the method 500 may include identifying one or more significant points associated with the body of a particular animal.

[0126]

[0139] The method 500 can include performing a trajectory analysis for a particular animal based on the time-dependent positions of one or more points of interest.

[0140] Additionally, the method 500 may include determining a probable future location of a particular animal based on trajectory analysis.

[0127]

[0141] Finally, the method 500 may include providing the trajectory analysis as training data for training a machine learning model.

[0142] In some demonstrative embodiments, method 500 may include providing the trained machine learning model to at least one controller (e.g., controller 150). In these scenarios, the controller may be configured, at runtime, to use the trained machine learning model to classify at least a portion of images captured by a top-down camera (e.g., top-down camera 124) as being associated with at least one animal behavior type (e.g., animal behavior type 162) of a plurality of animal behavior types. In such scenarios, the plurality of animal behavior types include at least one of movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching.

[0128]

[0143] In various examples, the controller can be configured, at runtime, to use the trained machine learning model to assign at least one digital biomarker of a plurality of digital biomarkers (e.g., digital biomarker 164) to at least a portion of the images captured by the top-down camera. In such a scenario, the plurality of digital biomarkers can include heart rate, respiration rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), and skin temperature. The digital biomarkers can also, or instead, include assessments of coat condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, the animal's overall health, disease status, scratching, and various activities including marble burying or nesting.

[0129] IV. Enumerated Exemplary Embodiments (EEE)

[0144] In this section, various exemplary embodiments of the present invention are presented. These embodiments are provided to demonstrate the versatility and adaptability of the present invention in various situations and scenarios. It is important to note that these examples are not intended to limit the scope of the present invention, but rather are provided to enable a clearer understanding of the present invention and its potential applications. While each embodiment described herein includes specific details and configurations, it will be understood that these embodiments can be modified or adjusted without departing from the basic principles and novel features of the present invention.

[0130]

[0145] EEE1 is The top of the case and Cage bottom and a cage housing comprising: a top-down camera configured to capture images of a top-down field of view including one or more animals contained within the cage housing; an illumination module configured to illuminate a top-down field of view; an air conditioning system coupled to the cage housing; a cage data unit comprising: a controller operable to perform an action, the action comprising causing a top-down camera to capture an image of a top-down field of view; The system comprises:

[0131]

[0146] EEE2 is an EEE1 system, and its operation is as follows: Further comprising using the trained machine learning model to identify one or more animals from at least some of the captured images.

[0132]

[0147] EEE3 is the system of EEE2, wherein the operations further comprise assigning a bounding box based on each identified animal.

[0148] EEE4 is the system of EEE3, wherein the operations further comprise determining the centroid of the bounding box.

[0133]

[0149] EEE5 is the system of EEE4, wherein the operations further comprise determining a segmentation mask within the bounding box.

[0150] EEE6 is an EEE5 system, and its operation is as follows: Further comprising determining a plurality of key points on each identified animal.

[0134]

[0151] EEE7 is an EEE1 system, and its operation is as follows: The method further comprises using the trained machine learning model to classify at least some of the captured images as associated with at least one animal behavior type among a plurality of animal behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching.

[0135]

[0152] EEE8 is an EEE1 system, and its operation is as follows: Further comprising using the trained machine learning model to assign at least one digital biomarker of a plurality of digital biomarkers to at least a portion of the captured images, the plurality of digital biomarkers comprising assessment of heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, fur condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and activity including marble burying or nesting.

[0136]

[0153] EEE9 is the system of EEE1, and the controller is a graphics processor unit (GPU); a central processor unit (CPU); Memory and wherein the GPU, CPU, and memory are coupled to a shared substrate, and the controller is configured to perform machine learning tasks.

[0137]

[0154] The EEE10 is a system based on the EEE9, and the controller is an NVIDIA Jetson Nano series module.

[0155] EEE11 is a system of EEE1, and the lighting module is a plurality of infrared light sources; a plurality of visible light sources; a light diffuser disposed along a downward-facing surface of the housing; Equipped with.

[0138]

[0156] EEE12 is the system of EEE11, in which the multiple infrared light sources and multiple visible light sources are arranged in an interleaved arrangement, the interleaved arrangement being selected to uniformly illuminate the cage bottom with visible or infrared light.

[0139]

[0157] EEE13 is the system of EEE11, wherein the plurality of infrared light sources are configured to emit infrared light having a wavelength of at least 900 nm.

[0158] The EEE14 is a system based on the EEE1, with the top-down camera configured to capture images at 30 frames per second in ultra-high-definition (UHD) 4K (4032 x 3040 pixels) resolution.

[0140]

[0159] The EEE15 is a system based on the EEE1, and the cage housing is A food hopper and Water bottle holder and Further provided are:

[0141]

[0160] EEE16 is the system of EEE15, wherein at least one of the food hopper or water bottle holder is shaped to reduce obstruction of the top-down view.

[0142]

[0161] EEE17 is the EEE1 system, and the top of the cage is The cage has a static vent along the front of the top, which is air permeable.

[0162] EEE18 is an EEE1 system, and the air conditioning system is With at least one fan, HEPA filter and one or more air flow connectors; Vibration isolation mechanism Equipped with.

[0143]

[0163] EEE19 is the system of EEE18, wherein the air conditioning system further comprises a PID controller, the PID controller configured to control the at least one fan when installed to provide a constant pressure or a constant flow rate to the cage housing.

[0144]

[0164] EEE20 is the system of EEE18, wherein the vibration isolation mechanism comprises a plurality of spring elements coupling the at least one fan to the housing, the plurality of spring elements being configured to damp vibrations caused by operation of the at least one fan.

[0145]

[0165] EEE21 is the system of EEE18, wherein the cage housing further comprises a plurality of airflow connectors disposed along a rear surface of the cage top, the airflow connectors configured to attachably couple to an air conditioning system when the cage housing is installed within the housing.

[0146]

[0166] EEE22 is a system of EEE1, further comprising a communication interface, which communicatively couples the controller to at least one of a gateway or a central server.

[0147]

[0167] EEE23 is the system of EEE1, wherein the cage data unit further comprises an oblique angle camera, the oblique angle camera configured to capture images of an oblique field of view.

[0168] EEE24 is a system of EEE23, where the oblique camera is configured to capture images at 30 frames per second at 2K (2560 × 1440 pixels) resolution.

[0148]

[0169] EEE25 is the system of EEE1, further comprising a user interface, the user interface comprising a touchscreen disposed along the front of the housing.

[0149]

[0170] EEE26 is a system of EEE25, and its operation is as follows:

[0171] Further comprising adjusting at least one operational aspect of the system in response to receiving a user command via the touchscreen.

[0150]

[0172] EEE27 is the system of EEE1, further comprising at least one furniture object within the cage bottom, the furniture object comprising at least one of a running wheel, a rotating wheel, a rotating bar, or a ladder.

[0151]

[0173] EEE28 is the system of EEE1, wherein the housing further comprises a light sensor, the light sensor configured to provide information indicative of an ambient light level to the controller, and operation of the controller comprises: Adjusting the exposure settings of a top-down camera based on ambient light levels, or Further comprising adjusting an acquisition mode of the top-down camera based on an ambient light level.

[0152]

[0174] The EEE29 is a system of the EEE1, where the cage bottom is attachably coupled to the cage top, and the cage bottom is sized in accordance with European Directive 2010 / 63 / EU.

[0175] EEE30 is a system of EEE1, except that the cage bottom comprises a transparent plastic material, the transparent plastic material comprises polyethylene terephthalate glycol (PETG), the cage bottom comprises a floor area of ​​at least 516.13 square centimeters (80 square inches), and the cage bottom sidewalls are at least 12.7 centimeters (5 inches) high.

[0153]

[0176] EEE31 is the system of EEE1, wherein the cage bottom comprises bedding, the bedding comprising at least one of paper bedding, wood shavings, corncobs, or cellulosic paper.

[0154]

[0177] EEE32 is a system of EEE1, in which the one or more animals comprise one or more mice or rats.

[0178] The EEE33 is a rackmount cage system A base frame; a plurality of vertical members coupled to the base frame; a plurality of bays formed by spaces between the plurality of vertical members, at least a portion of the plurality of bays comprising: 1. A cage housing comprising: The top of the case and Cage bottom and a cage housing comprising: A cage data unit, comprising: a top-down camera configured to capture images of a top-down field of view including one or more animals contained within the cage housing; an illumination module configured to illuminate a top-down field of view; an air conditioning system coupled to the cage housing; a cage data unit comprising: a plurality of bays, each comprising: A controller operable to perform an operation, the operation comprising: a controller causing a top-down camera to capture an image of a top-down field of view; Equipped with.

[0155]

[0179] The EEE34 is a rackmount cage system of the EEE33. Further comprising using the trained machine learning model to identify one or more animals from at least some of the captured images.

[0156]

[0180] The EEE35 is a rackmount cage system for the EEE34. Further comprising assigning a bounding box based on each identified animal.

[0181] The EEE36 is a rackmount cage system for the EEE35. The method further comprises determining a centroid of the bounding box.

[0157]

[0182] The EEE37 is a rackmount cage system of the EEE36. The method further comprises determining a segmentation mask within the bounding box.

[0183] The EEE38 is a rackmount cage system of the EEE37.

[0184] Further comprising determining a plurality of key points on each identified animal.

[0158]

[0185] The EEE39 is a rackmount cage system for the EEE33. The method further comprises using the trained machine learning model to classify at least some of the captured images as associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching.

[0159]

[0186] The EEE40 is a rackmount cage system for the EEE33. Further comprising using the trained machine learning model to assign at least one digital biomarker of a plurality of digital biomarkers to at least a portion of the captured images, the plurality of digital biomarkers comprising assessment of heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, fur condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and activity including marble burying or nesting.

[0160]

[0187] The EEE41 is a rackmount cage system for the EEE33, and the controller is a graphics processor unit (GPU); a central processor unit (CPU); Memory and The GPU, CPU, and memory are coupled to a shared substrate.

[0161]

[0188] The EEE42 is a rack-mounted cage system of the EEE41, with each bay's respective air conditioning system configured to cool the controller.

[0189] The EEE43 is a rack-mounted cage system of the EEE33, with each bay's individual air conditioning system tied to a shared tower blower unit or house air.

[0162]

[0190] EEE44 is the rack mount cage system of EEE33, further comprising a power supply, and at least some of the bays are configured to receive power via the power supply.

[0191] EEE45 is a rackmount cage system of EEE33, in which at least some of the bays are configured to house drawers, shelves, or environmental monitoring units.

[0163]

[0192] The EEE46 is a rackmount cage system for the EEE33. Further comprising a cage identifier, the cage identifier comprising at least one of a bar code, a QR code, an encoded pattern, or another type of visible or infrared identifier or symbology, the cage identifier providing information corresponding to a given cage housing.

[0164]

[0193] EEE47 is the rack mount cage system of EEE46, wherein the cage identifier is provided by at least one of etching into plastic, affixing by sticker, or printing on the back of the cage card.

[0165]

[0194] The EEE48 is a rackmount cage system of the EEE46. Further comprising a visual identification module configured to determine a cage identifier based on one or more images captured by at least one of the cameras.

[0166]

[0195] EEE49 is receiving, via a central server, images of a top-down field of view, the images comprising live or past images of one or more animals contained within the cage housing; Displaying a user interface via the display, the user interface comprising: a video stream viewer configured to display a user-navigable stream of live or historical images; At least one data graph and a displaying step; The method includes:

[0167]

[0196] EEE50 includes the method of EEE49, wherein the at least one data graph comprises information indicative of at least one of average travel speed, wheel occupancy, or water heater occupancy.

[0168]

[0197] EEE51 includes the method of EEE49, wherein the user interface further comprises an annotation feed, the annotation feed comprising information about animal behavior types or digital biomarkers associated with one or more animals.

[0169]

[0198] EEE52 is a method of EEE49, The method further comprises capturing the image using a top-down camera located in a cage data unit mountable to the cage housing.

[0170]

[0199] EEE53 is the method of EEE49, wherein the step of displaying the user interface comprises displaying live or historical information via at least one of a video stream viewer, at least one data graph, or an annotation feed.

[0171]

[0200] EEE54 is a method of EEE49, The method further includes using the local computing device and the trained machine learning model to classify at least a portion of the images as being associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching.

[0172]

[0201] EEE55 is a method of EEE54, The method further includes adding a new annotation to the annotation feed indicating the at least one animal behavior type together with a link to a corresponding video clip in response to classifying at least a portion of the image as being associated with the at least one animal behavior type.

[0173]

[0202] EEE56 is the method of EEE55, wherein the new annotation comprises at least one of a free-form comment, a hashtag taxonomy group, or an "at" username reference to the user.

[0174]

[0203] EEE57 is a method of EEE49, The method further includes using a local computing device and the trained machine learning model to assign at least one digital biomarker of a plurality of digital biomarkers to at least a portion of the image, wherein the plurality of digital biomarkers comprises heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), and skin temperature.

[0175]

[0204] EEE58 is a method of EEE49, The method further comprises determining the location of a particular animal within the image using a local computing device and the trained machine learning model.

[0176]

[0205] EEE59 is the method of EEE58, wherein determining the location of the particular animal comprises applying an image segmentation mask to the image so as to recognize the particular animal in the image.

[0177]

[0206] EEE60 is the method of EEE58, wherein determining the location of the particular animal comprises applying an object detection method, the object detection method comprising at least one of a thresholding method, an edge detection method, or a clustering method.

[0178]

[0207] EEE61 is the method of EEE58, wherein the step of displaying the user interface also comprises the step of displaying, via the video stream viewer, at least one bounding box corresponding to a location of the particular animal.

[0179]

[0208] EEE62 is a method of EEE58, The method further comprises assigning an identifier to the particular animal using a local computing device and the trained machine learning model.

[0180]

[0209] EEE63 is the method of EEE62, wherein the identifier is based on at least one of an ear tag identifier or a tail tattoo identifier.

[0210] EEE64 is a method of EEE58, The method further comprises dynamically tracking the location of a particular animal using a local computing device and the trained machine learning model.

[0181]

[0211] EEE65 is a method of EEE49, determining a cage-in state or a cage-out state based on the image, wherein the cage-in state comprises the cage housing being in a desired position and the cage-out state comprises the cage housing not being in a desired position; indicating a cage-in or cage-out status via a user interface; Further provided are:

[0182]

[0212] EEE66 is receiving a top-down view image, the image comprising a live or past image of one or more animals contained within the cage housing; determining a location of a particular animal in the image using the trained machine learning model, wherein determining the location of the particular animal comprises applying an image segmentation mask; assigning an identifier to a particular animal using the trained machine learning model; assigning at least one bounding box corresponding to the location of a particular animal within the image; The method comprises:

[0183]

[0213] EEE67 is a method of EEE66, The method further comprises assigning at least one bounding box corresponding to the location of a particular animal within the image.

[0184]

[0214] EEE68 is a method of EEE66, The method further includes using the trained machine learning model to classify at least a portion of the images as being associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching.

[0185]

[0215] EEE69 is a method of EEE66, The method further comprises using the trained machine learning model to assign at least one digital biomarker from a plurality of digital biomarkers to at least a portion of the image, the plurality of digital biomarkers comprising assessments of heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, coat condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and activities including marble burying or nesting.

[0186]

[0216] EEE70 is the method of EEE66, wherein determining the location of the particular animal comprises applying an object detection method, the object detection method comprising at least one of a thresholding method, an edge detection method, or a clustering method.

[0187]

[0217] EEE71 is the method of EEE66, wherein the identifier is based on at least one of an ear tag identifier or a tail tattoo identifier.

[0218] EEE72 is a method of EEE66, The method further comprises using the trained machine learning model to dynamically track the location of the particular animal.

[0188]

[0219] EEE73 is a method of EEE66, The method further comprises the step of estimating a pose of the particular animal, wherein estimating the pose comprises determining a direction in which the particular animal is facing.

[0189]

[0220] EEE74 is a method of EEE66, determining a facing direction of the particular animal relative to at least one other object within the cage housing; determining that the particular animal is interacting with at least one other object based on the facing direction; Further provided are:

[0190]

[0221] EEE75 is a method of EEE66, The method further includes determining a cage-in state or a cage-out state based on the image, where the cage-in state comprises the cage housing being in a desired position and the cage-out state comprises the cage housing not being in a desired position.

[0191]

[0222] EEE76 is a method for training a machine learning model, and the method is receiving a plurality of images of a top-down field of view as training data, the images including one or more animals contained within a cage housing; training the machine learning model using an unsupervised learning method based on the training data to form a trained machine learning model, the unsupervised learning method comprising at least one of k-means clustering, hierarchical clustering, or density-based clustering; Equipped with.

[0192]

[0223] EEE77 is a method of EEE76, Identifying at least one specific animal based on the image; identifying one or more significant points associated with the body of a particular animal; performing a trajectory analysis for the particular animal based on the time-dependent positions of one or more points of interest; determining an estimated future location of a particular animal based on the trajectory analysis; providing the trajectory analysis as training data for training a machine learning model; Further provided are:

[0193]

[0224] EEE78 is a method of EEE76, The trained machine learning model The method further includes providing at least one controller configured, upon execution, to classify, using the trained machine learning model, at least a portion of the images captured by the top-down camera as being associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, walking, rearing, and scratching.

[0194]

[0225] EEE79 is a method of EEE76, The trained machine learning model The method further includes providing to at least one controller configured, upon execution, to use the trained machine learning model to assign at least one digital biomarker of a plurality of digital biomarkers to at least a portion of the image captured by the top-down camera, the plurality of digital biomarkers comprising assessments of heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, fur condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and activities including marble burying or nesting.

[0195] V. Conclusion

[0226] The foregoing detailed description has described various features and functions of the disclosed systems, devices, and methods with reference to the accompanying drawings. In the drawings, like symbols generally refer to like components unless otherwise indicated by context. The illustrative embodiments described in the detailed description, drawings, and claims are not intended to be limiting. Other embodiments may be utilized, and other changes may be made, without departing from the scope of the subject matter presented herein. It will be readily understood that the aspects of the present disclosure, as generally described herein and illustrated in the drawings, can be arranged, substituted, combined, separated, and designed in a wide variety of configurations, all of which are expressly contemplated herein.

[0196]

[0227] With respect to any or all of the message flow diagrams, scenarios, and flowcharts in the figures and as described herein, each step, block, and / or communication may represent the processing of information and / or the transmission of information according to the exemplary embodiments. Alternative embodiments are included within the scope of these exemplary embodiments. In these alternative embodiments, for example, functions described as steps, blocks, transmissions, communications, requests, responses, and / or messages may be performed in an order other than that shown or described, including substantially simultaneously or in reverse order, depending on the functionality involved. Furthermore, more or fewer steps, blocks, and / or functions may be used in any of the message flow diagrams, scenarios, and flowcharts described herein, and these message flow diagrams, scenarios, and flowcharts may be combined with each other, either partially or in whole.

[0197]

[0228] Steps or blocks representing the processing of information may correspond to circuitry that can be configured to perform specific logical functions of the methods or techniques described herein. Alternatively, or in addition, steps or blocks representing the processing of information may correspond to modules, segments, or portions of program code (including associated data). The program code may include one or more instructions executable by a processor to implement specific logical functions or actions in the method or technique. The program code and / or associated data may be stored in any type of computer-readable medium, such as a storage device, including a disk drive, hard drive, or other storage medium.

[0198]

[0229] Computer-readable media may also include non-transitory computer-readable media, such as computer-readable media that store short-term data, such as register memory, processor cache, and / or random access memory (RAM). Computer-readable media may also include non-transitory computer-readable media that store program code and / or data for longer periods, such as secondary or permanent long-term storage, such as read-only memory (ROM), optical or magnetic disks, and / or compact disc read-only memory (CD-ROM). Computer-readable media may also be any other volatile or non-volatile storage system. Computer-readable media may be considered, for example, as a computer-readable storage medium or a tangible storage device.

[0199]

[0230] Additionally, steps or blocks representing one or more information transmissions may correspond to information transmissions between software and / or hardware modules within the same physical device, although other information transmissions may occur between software and / or hardware modules in different physical devices.

[0200]

[0231] While various aspects and embodiments are disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are illustrative and not intended to be limiting, with the true scope being indicated by the following claims.

Claims

1. The top of the case and Cage bottom and a cage housing comprising: a top-down camera configured to capture images of a top-down field of view including one or more animals contained within the cage housing; an illumination module configured to illuminate the top-down field of view; a cage data unit comprising: an air conditioning system coupled to the cage housing; A controller operable to perform an operation, the operation comprising: causing the top-down camera to capture an image of the top-down field of view; a controller comprising: A system comprising:

2. 2. The system of claim 1, wherein the operation comprises: using a trained machine learning model to identify the one or more animals from at least some of the captured images. The system further comprises:

3. 3. The system of claim 2, wherein the operation comprises: Assigning a bounding box based on each identified animal The system further comprises:

4. 4. The system of claim 3, wherein the operation comprises: determining the centroid of the bounding box; The system further comprises:

5. 5. The system of claim 4, wherein the operation comprises: determining a segmentation mask within the bounding box; The system further comprises:

6. 6. The system of claim 5, wherein the operation comprises: Determining multiple key points on each identified animal The system further comprises:

7. 2. The system of claim 1, wherein the operation comprises: using a trained machine learning model to classify at least some of the captured images as associated with at least one animal behavior type among a plurality of animal behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. The system further comprises:

8. 2. The system of claim 1, wherein the operation comprises: using a trained machine learning model to assign at least one digital biomarker from a plurality of digital biomarkers to at least a portion of the captured image, the plurality of digital biomarkers comprising heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, coat condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and assessment of activities including marble burying or nesting. The system further comprises:

9. 2. The system of claim 1, wherein the controller: a graphics processor unit (GPU); a central processor unit (CPU); Memory and wherein the GPU, the CPU, and the memory are coupled to a shared substrate, and the controller is configured to perform machine learning tasks.

10. 10. The system of claim 9, wherein the controller comprises an NVIDIA Jetson Nano series module.

11. 10. The system of claim 1, wherein the lighting module comprises: a plurality of infrared light sources; a plurality of visible light sources; a light diffuser disposed along a downward-facing surface of the housing; A system comprising:

12. 12. The system of claim 11, wherein the plurality of infrared light sources and the plurality of visible light sources are arranged in an interleaved arrangement, the interleaved arrangement being selected to uniformly illuminate the cage bottom with visible or infrared light.

13. 12. The system of claim 11, wherein the plurality of infrared light sources are configured to emit infrared light having a wavelength of at least 900 nm.

14. 10. The system of claim 1, wherein the top-down camera is configured to capture images at 30 frames per second in ultra-high definition (UHD) 4K (4032x3040 pixels) resolution.

15. 10. The system of claim 1, wherein the cage housing comprises: A food hopper and Water bottle holder and The system further comprises:

16. 16. The system of claim 15, wherein at least one of the food hopper and the water bottle holder is shaped to reduce obstruction of the top-down view.

17. 10. The system of claim 1, wherein the cage top comprises: Static vents along the front of the cage top wherein the static vent is air permeable.

18. 2. The system of claim 1, wherein the air conditioning system comprises: With at least one fan A HEPA filter; one or more air flow connectors; Vibration isolation mechanism A system comprising:

19. 20. The system of claim 18, wherein the air conditioning system further comprises a PID controller, the PID controller configured to control the at least one fan to provide a constant pressure or a constant flow rate to the cage housing when installed.

20. 20. The system of claim 18, wherein the vibration isolation mechanism comprises a plurality of spring elements coupling the at least one fan to the housing, the plurality of spring elements configured to damp vibrations caused by operation of the at least one fan.

21. 20. The system of claim 18, wherein the cage housing further comprises a plurality of airflow connectors disposed along a rear surface of the cage top, the airflow connectors configured to attachably couple to the air conditioning system when the cage housing is installed within the housing.

22. The system of claim 1 , further comprising a communications interface, the communications interface communicatively coupling the controller to at least one of a gateway and a central server.

23. 10. The system of claim 1, wherein the cage data unit further comprises an oblique angle camera, the oblique angle camera configured to capture images of an oblique field of view.

24. 24. The system of claim 23, wherein the oblique camera is configured to capture images at 30 frames per second at 2K (2560 x 1440 pixels) resolution.

25. The system of claim 1 , further comprising a user interface, the user interface comprising a touchscreen disposed along a front surface of the housing.

26. 26. The system of claim 25, wherein the action comprises: adjusting at least one operational aspect of the system in response to receiving a user command via the touchscreen; The system further includes:

27. 10. The system of claim 1, further comprising at least one furniture object within the cage bottom, the furniture object comprising at least one of a running wheel, a rolling wheel, a rolling bar, and a ladder.

28. 10. The system of claim 1, wherein the housing further comprises a light sensor, the light sensor configured to provide information indicative of an ambient light level to the controller, and wherein the operation of the controller comprises: adjusting the exposure setting of the top-down camera based on the ambient light level; or adjusting an acquisition mode of the top-down camera based on the ambient light level; A system comprising:

29. 10. The system of claim 1, wherein the cage bottom is attachably coupled to the cage top, and the cage bottom is sized in accordance with European Directive 2010 / 63 / EU.

30. 10. The system of claim 1, wherein the cage bottom comprises a transparent plastic material, the transparent plastic material comprising polyethylene terephthalate glycol (PETG), the cage bottom comprises a floor area of ​​at least 80 square inches, and the cage bottom sidewalls are at least 5 inches high.

31. 10. The system of claim 1, wherein the cage bottom comprises bedding, the bedding comprising at least one of paper bedding, wood shavings, corn cobs, and cellulosic paper.

32. 10. The system of claim 1, wherein the one or more animals comprise one or more mice or rats.

33. A base frame; a plurality of vertical members coupled to the base frame; a plurality of bays formed by spaces between said plurality of vertical members; 1. A rack mount cage system comprising:

1. A cage housing comprising: The top of the case and Cage bottom and a cage housing comprising: A cage data unit, comprising: a top-down camera configured to capture images of a top-down field of view including one or more animals contained within the cage housing; an illumination module configured to illuminate the top-down field of view; a cage data unit comprising: an air conditioning system coupled to the cage housing; A controller operable to perform an operation, the operation comprising: causing the top-down camera to capture an image of the top-down field of view; a controller comprising: A rack mount cage system comprising:

34. 34. The rack mount cage system of claim 33, wherein the operation comprises: using a trained machine learning model to identify the one or more animals from at least some of the captured images. A rackmount cage system that also includes:

35. 35. The rack mount cage system of claim 34, wherein the operation comprises: Assigning a bounding box based on each identified animal A rackmount cage system that also includes:

36. 36. The rack mount cage system of claim 35, wherein the operation comprises: determining the centroid of the bounding box; A rackmount cage system that also includes:

37. 37. The rack mount cage system of claim 36, wherein the operation comprises: determining a segmentation mask within the bounding box; A rackmount cage system that also includes:

38. 38. The rack mount cage system of claim 37, wherein the operation comprises: Determining multiple key points on each identified animal A rackmount cage system that also includes:

39. 34. The rack mount cage system of claim 33, wherein the operation comprises: using a trained machine learning model to classify at least some of the captured images as associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. A rackmount cage system that also includes:

40. 34. The rack mount cage system of claim 33, wherein the operation comprises: using a trained machine learning model to assign at least one digital biomarker from a plurality of digital biomarkers to at least a portion of the captured image, the plurality of digital biomarkers comprising heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, coat condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and assessment of activities including marble burying or nesting. A rackmount cage system that also includes:

41. 34. The rack mount cage system of claim 33, wherein the controller: a graphics processor unit (GPU); a central processor unit (CPU); Memory and wherein the GPU, the CPU, and the memory are coupled to a shared substrate.

42. 42. The rack mount cage system of claim 41, wherein the air conditioning system for each bay is configured to cool the controller.

43. 34. The rack mount cage system of claim 33, wherein the air conditioning system for each bay is coupled to a shared tower blower unit or house air.

44. 34. The rack mount cage system of claim 33, further comprising a power supply, wherein at least some of the bays are configured to receive power via the power supply.

45. 34. The rack mount cage system of claim 33, wherein at least some of the bays are configured to house drawers, shelves, or environmental monitoring units.

46. 34. The rack mount cage system of claim 33, Cage Identifier wherein the cage identifier comprises at least one of a bar code, a QR code, an encoded pattern, and another type of visible or infrared identifier or symbology, the cage identifier providing information corresponding to a given cage housing.

47. 47. The rack mount cage system of claim 46, wherein the cage identifier is provided by at least one of etching into the plastic, attaching with a sticker, and printing on the back of a cage card.

48. 47. The rack mount cage system of claim 46, a visual identification module configured to determine the cage identifier based on one or more images captured by at least one of the cameras; A rackmount cage system is also provided.

49. receiving, via a central server, images of a top-down view, said images comprising live or past images of one or more animals contained within a cage housing; displaying a user interface via a display, said user interface comprising: a video stream viewer configured to display a user-navigable stream of the live or historical images; and At least one data graph and a displaying step; A method for providing

50. 50. The method of claim 49, wherein the at least one data graph comprises information indicative of at least one of average travel speed, wheel occupancy, and water heater occupancy.

51. 50. The method of claim 49, wherein the user interface further comprises an annotation feed, the annotation feed comprising information about animal behavior types or information about digital biomarkers associated with the one or more animals.

52. 50. The method of claim 49, capturing the image using a top-down camera located in a cage data unit mountable to the cage housing. A method further comprising:

53. 50. The method of claim 49, wherein displaying the user interface comprises displaying live or historical information via at least one of the video stream viewer, the at least one data graph, and the annotation feed.

54. 50. The method of claim 49, using a local computing device and a trained machine learning model to classify at least some of the images as associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. A method further comprising:

55. 55. The method of claim 54, In response to classifying at least a portion of the image as being associated with at least one animal behavior type, adding a new annotation to the annotation feed indicating the at least one animal behavior type along with a link to a corresponding video clip. A method further comprising:

56. 56. The method of claim 55, wherein the new annotation comprises at least one of a free-form comment, a hashtag taxonomy group, and an "at" username reference to a user.

57. 47. The method of claim 46, assigning, using a local computing device and a trained machine learning model, at least one digital biomarker of a plurality of digital biomarkers to at least a portion of the image, the plurality of digital biomarkers comprising heart rate, respiration rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), and skin temperature. The method further comprises:

58. 47. The method of claim 46, determining the location of a particular animal within said image using a local computing device and a trained machine learning model; A method further comprising:

59. 59. The method of claim 58, wherein determining the location of the particular animal comprises applying an image segmentation mask to the image so as to recognize the particular animal within the image.

60. 59. The method of claim 58, wherein determining the location of the particular animal comprises applying an object detection method, the object detection method comprising at least one of a thresholding method, an edge detection method, and a clustering method.

61. 59. The method of claim 58, wherein the step of displaying the user interface also comprises the step of displaying at least one bounding box corresponding to the location of the particular animal via the video stream viewer.

62. 59. The method of claim 58, assigning an identifier to the particular animal using the local computing device and the trained machine learning model. A method further comprising:

63. 63. The method of claim 62, wherein the identifier is based on at least one of an ear tag identifier and a tail tattoo identifier.

64. 59. The method of claim 58, dynamically tracking the location of the particular animal using the local computing device and the trained machine learning model. A method further comprising:

65. 47. The method of claim 46, determining a cage-in state or a cage-out state based on the image, the cage-in state comprising the cage housing being in a desired position and the cage-out state comprising the cage housing not being in the desired position; displaying the cage-in state or the cage-out state via the user interface; A method further comprising:

66. receiving a top-down view image, said image comprising a live or past image of one or more animals contained within a cage housing; determining, using a trained machine learning model, a location of a particular animal in the image, wherein determining the location of the particular animal comprises applying an image segmentation mask; assigning an identifier to the particular animal using the trained machine learning model; A method for providing

67. 67. The method of claim 66, assigning at least one bounding box corresponding to the location of the particular animal within the image; A method further comprising:

68. 67. The method of claim 66, using the trained machine learning model to classify at least some of the images as associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. A method further comprising:

69. 67. The method of claim 66, using the trained machine learning model to assign at least one digital biomarker from a plurality of digital biomarkers to at least a portion of the image, the plurality of digital biomarkers comprising assessments of heart rate, respiratory rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, coat condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and activity including marble burying or nesting. A method further comprising:

70. 67. The method of claim 66, wherein determining the location of the particular animal comprises applying an object detection method, the object detection method comprising at least one of a thresholding method, an edge detection method, and a clustering method.

71. 67. The method of claim 66, wherein the identifier is based on at least one of an ear tag identifier and a tail tattoo identifier.

72. 67. The method of claim 66, using the trained machine learning model to dynamically track the location of the specific animal. A method further comprising:

73. 67. The method of claim 66, Estimating the posture of the particular animal. and wherein estimating the pose comprises determining a direction the particular animal is facing.

74. 67. The method of claim 66, determining a facing direction of the particular animal relative to at least one other object within the cage housing; determining, based on the facing direction, that the particular animal is interacting with the at least one other object; A method further comprising:

75. 67. The method of claim 66, determining a cage-in state or a cage-out state based on the image, the cage-in state comprising the cage housing being in a desired position and the cage-out state comprising the cage housing not being in the desired position. A method further comprising:

76. 1. A method for training a machine learning model, comprising: receiving a plurality of images of a top-down field of view as training data, the images including one or more animals contained within a cage housing; training a machine learning model using an unsupervised learning method based on the training data to form a trained machine learning model, the unsupervised learning method comprising at least one of k-means clustering, hierarchical clustering, and density-based clustering; A method for providing

77. 77. The method of claim 76, identifying at least one specific animal based on the image; identifying one or more significant points associated with the body of said particular animal; performing a trajectory analysis for the particular animal based on the time-dependent positions of the one or more points of interest; determining an estimated future location of the particular animal based on the trajectory analysis; providing the trajectory analysis as training data for training the machine learning model; A method further comprising:

78. 77. The method of claim 76, providing the trained machine learning model to at least one controller, the at least one controller configured, upon execution, to use the trained machine learning model to classify at least a portion of images captured by a top-down camera as being associated with at least one animal behavior type among a plurality of behavior types, the plurality of animal behavior types comprising movement, locomotion, wheeling, food and water possession, loss of righting reflex, seizures, gait, rearing, and scratching. A method further comprising:

79. 77. The method of claim 76, providing the trained machine learning model to at least one controller, wherein the at least one controller is configured, upon execution, to use the trained machine learning model to assign at least one digital biomarker of a plurality of digital biomarkers to at least a portion of images captured by the top-down camera, the plurality of digital biomarkers comprising assessments of heart rate, respiration rate, size, weight, sleep patterns, physical activity, electrodermal activity (EDA), skin temperature, fur condition, eye clarity, presence and condition of lesions, posture, loss of righting reflex, seizure status, seizure type, overall health of the animal, disease status, scratching, and activity including marble burying or nesting. A method further comprising:

Images