Method for tracking and recording three-dimensional activity trajectory of animals based on depth camera

Abstract

The application discloses a three-dimensional activity track tracking and measuring and recording method of animals based on a depth camera, and mainly comprises the following steps: S1, installing the depth camera at the front side of a transparent cage at a distance of 1.5-2 m, adjusting the distance according to the size of the cage, and making the field of view cover the front of the cage and the picture as large as possible; S2, accurately positioning eight vertexes in the cage; S3, acquiring and removing the background noise points in real time; S4, simultaneously measuring and recording the three-dimensional track data in real time, calling the infrared camera of Kinect2.0, synchronously recording the four-part picture, and facilitating the experimenter to review the data; and the method is suitable for measuring the activity amount, movement, muscle tension and other related behavior indexes of experimental animals having three-dimensional activities in the cage, such as accurate activity amount, bounce frequency, position preference, movement track, whether there is an activity mode, track abnormality and the like, and the measurement of the indexes is helpful to the modeling of various senile degenerative diseases, the effect evaluation of drug effect and the effect evaluation of rehabilitation therapy.

Description

Technical Field

[0001] This invention belongs to the field of moving object detection and tracking technology, specifically relating to a method for accurately tracking, measuring and recording the three-dimensional movement trajectory of a single experimental animal based on a depth camera. Background Technology

[0002] Against the backdrop of an aging society, an increasing number of biomedical experiments require behavioral data such as changes in activity levels in experimental animals used to model various degenerative diseases, as well as precise three-dimensional activity trajectories, to serve as evaluation indicators for disease modeling and rehabilitation treatment experiments.

[0003] Current methods for automatically analyzing the 3D trajectory of laboratory animals mainly rely on 2D images from ordinary cameras. These images are then analyzed using depth models to roughly estimate the animal's spatial position within its cage. However, these methods suffer from significant error fluctuations and fail to achieve the tracking and measurement precision required for quantitative analysis. An optimized approach involves using multiple ordinary cameras to simultaneously capture images of the object from different angles. The principle is the same as with a single camera: depth model recognition is used to estimate the target object's position in 3D space.

[0004] Currently, there are no existing methods for tracking the 3D trajectory of laboratory animals in cages using depth cameras. Microsoft Kinect 2.0 SDK 2.0 provides the nearest 3D point cloud information within the field of view and effective depth range captured by the depth camera. However, applying this data to the measurement and recording of the 3D trajectory of laboratory animals in cages presents several key challenges: 1. How to accurately locate and reconstruct the positions of the eight vertices inside the cage in the recorded 3D image to accurately define the recording space; 2. How to remove the image of inherent objects within the measurement space from the tracking data, ensuring data accuracy within ±1cm; 3. How to improve acquisition and processing efficiency, ensuring accurate real-time tracking while simultaneously recording tracking data and image data, and maintaining a frame rate of over 30 frames per second. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides a method for accurately tracking, measuring, and recording the three-dimensional activity trajectory of a single experimental animal based on a depth camera. This method is applicable to measuring behavioral indicators such as activity level, movement, and muscle tone in experimental animals that engage in three-dimensional activity within a cage. Examples include accurate activity levels, jumping frequency, positional preferences, movement trajectories, and the presence of activity patterns or trajectory abnormalities. Measuring these indicators is helpful for modeling various age-related degenerative diseases, evaluating drug efficacy, and assessing the effectiveness of rehabilitation therapies.

[0006] To achieve the above-mentioned technical objectives, the present invention is implemented through the following technical solution:

[0007] A method for accurately tracking, measuring, and recording the three-dimensional movement trajectory of a single experimental animal based on a depth camera includes the following steps:

[0008] S1: Install the depth camera 1.5 to 2 meters in front of the transparent cage. Adjust the distance according to the size of the cage to make the image as large as possible while covering the cage.

[0009] S2: Precise positioning of the eight vertices of the cage's internal space;

[0010] S3: Acquisition and real-time elimination of background noise;

[0011] S4: While measuring and recording 3D trajectory data in real time, the Kinect 2.0 infrared camera is used to simultaneously record four-frame images, making it easy for the experimenter to fully review the data;

[0012] Preferably, the vertex localization method of S2 is as follows:

[0013] S2.1: Display the real-time 3D point cloud data from the camera on a 3D screen to ensure that the operator receives intuitive visual feedback when adjusting the positioning of the eight vertices of the cage.

[0014] S2.2: Based on the spatial position of the eight vertices, calculate in real time and use different colors to indicate the point cloud pixels inside and outside the space;

[0015] S2.3: Use reflective cloth to assist in the positioning of far-end vertices and other vertices with impaired visibility: Since the accurate position of the three-dimensional point cloud of each vertex in the cage space is often not very clear, a reflective cloth can be used. It is bright in the infrared image but not in the point cloud data. It can be placed in the corner of each vertex and rotated in three dimensions with the mouse to assist in the calibration and positioning.

[0016] S2.4: The calibrated vertex physical positions and the corresponding physical distances between vertices in the 3D image are fed back to the operator in real time on the screen. By comparing with the actual size of the cage, the accurate position of the cage vertex can be further adjusted. This allows for accurate framing of the recording space and facilitates alignment of subsequent 3D tracking and recording data.

[0017] Preferably, the method for calibrating point cloud pixels inside and outside the space is as follows:

[0018] S2.2.1: First, calculate the minimum / maximum depth value (Z) of any point on the front / back face based on the X / Y (horizontal / vertical) position of the point and the spatial linear mapping relationship;

[0019] S2.2.2: Next, based on the depth value Z of the point and its proportion of the minimum / maximum depth value, calculate the minimum / maximum horizontal axis position (X) on the left / right faces of the point and the minimum / maximum height position (Y) on the upper / lower faces of the point according to the spatial linear mapping relationship.

[0020] S2.2.3: Finally, determine whether the X / Y / Z (horizontal / vertical / depth) three-dimensional position of the point is within its corresponding minimum / maximum range. If it is within the range, mark it in yellow in real time; otherwise, mark it in blue.

[0021] Preferably, the specific method for acquiring and real-time eliminating background noise in S3 is as follows:

[0022] S3.1: Based on the recording space defined by the aforementioned fixed-point positioning method steps, generate a three-dimensional data array of just the right size to record and save noise data generated by inherent objects within the recording space;

[0023] S3.2: During the tracking and recording process, for each point cloud data in the recording space, traverse 27 positions in a 3*3*3 grid around the data point in the 3D noise data frame. If no noise is found in the recording, then the data point is judged as one of the current valid data values.

[0024] S3.3: Sampling rate: Related to host performance. An 8th generation CPU i7 and a P2200 graphics card can achieve a sampling rate of around 50Hz. Methods to optimize running efficiency include limiting the range + interval + rotating sampling calculation. For example, the Kinect 2.0 depth camera has 512*424 3D data pixels. First, the effective range is defined by the XYZ range values ​​of eight specified vertices. Second, sampling is performed at intervals of n points each time, which reduces the amount of calculation per frame by n*n times. Then, an auxiliary rotating parameter is added, which changes one by one between 0 and n in each frame. This ensures that all pixels are calculated once every n frames. This can ensure the accuracy of the data while taking into account the running efficiency.

[0025] Preferably, the four-panel screens are: 1) infrared video recording data; 2) display of cage position and tracking position on the three-dimensional canvas; 3) display of infrared video recording data and cage position and tracking position on the three-dimensional canvas; 4) experiment number, date, record of real-time elapsed time (milliseconds), and real-time three-dimensional trajectory value (cm).

[0026] The beneficial effects of this invention are:

[0027] The 3D point cloud data from the Kinect depth camera is based on real-time feedback values ​​from real space, unaffected by perspective distortion. Combined with our acquisition method—precisely locating the cage space, removing background noise, optimizing computational efficiency, and comprehensively recording data and image acquisition—it can efficiently and in real-time analyze the accurate spatial coordinates of a single animal within the cage. The quantization precision and accuracy of this data are higher than data values ​​estimated from 2D planar images. Furthermore, the hardware architecture using this invention is simpler to install compared to multiple ordinary cameras. This provides a more feasible and reliable method for measuring animal behavior trajectories and activity levels in experiments. Attached Figure Description

[0028] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 This is a schematic diagram of the software operation interface for adjusting the display position value when placing a detection object in Embodiment 1 of the present invention;

[0030] Figure 2 This is a schematic diagram of the operation interface of the spatial point setting software in Embodiment 1 of the present invention;

[0031] Figure 3 This is a schematic diagram of a real-world embodiment of the present invention;

[0032] Figure 4 This is a screenshot of a four-frame video recording from Embodiment 1 of the present invention. Detailed Implementation

[0033] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1

[0035] A method for accurately tracking, measuring, and recording the three-dimensional movement trajectory of a single experimental animal based on a depth camera includes the following steps:

[0036] S1: Install the depth camera 1.5 to 2 meters in front of the transparent cage. Adjust the distance according to the size of the cage to make the image as large as possible while covering the cage.

[0037] S2: Precise positioning of the eight vertices of the cage's internal space;

[0038] S3: Acquisition and real-time elimination of background noise;

[0039] S4: While measuring and recording 3D trajectory data in real time, the Kinect 2.0 infrared camera is used to simultaneously record four-frame images, making it easy for the experimenter to fully review the data;

[0040] Preferably, the vertex localization method of S2 is as follows:

[0041] S2.1: Display the real-time 3D point cloud data from the camera on a 3D screen to ensure that the operator receives intuitive visual feedback when adjusting the positioning of the eight vertices of the cage.

[0042] S2.2: Based on the spatial position of the eight vertices, calculate in real time and use different colors to indicate the point cloud pixels inside and outside the space;

[0043] S2.3: Use reflective cloth to assist in the positioning of far-end vertices and other vertices with impaired visibility: Since the accurate position of the three-dimensional point cloud of each vertex in the cage space is often not very clear, a reflective cloth can be used. It is bright in the infrared image but not in the point cloud data. It can be placed in the corner of each vertex and rotated in three dimensions with the mouse to assist in the calibration and positioning.

[0044] S2.4: The calibrated vertex physical positions and the corresponding physical distances between vertices in the 3D image are fed back to the operator in real time on the screen. By comparing with the actual size of the cage, the accurate position of the cage vertex can be further adjusted. This allows for accurate framing of the recording space and facilitates alignment of subsequent 3D tracking and recording data.

[0045] Preferably, the method for calibrating point cloud pixels inside and outside the space is as follows:

[0046] S2.2.1: First, calculate the minimum / maximum depth value (Z) of any point on the front / back face based on the X / Y (horizontal / vertical) position of the point and the spatial linear mapping relationship;

[0047] S2.2.2: Next, based on the depth value Z of the point and its proportion of the minimum / maximum depth value, calculate the minimum / maximum horizontal axis position (X) on the left / right faces of the point and the minimum / maximum height position (Y) on the upper / lower faces of the point according to the spatial linear mapping relationship.

[0048] S2.2.3: Finally, determine whether the X / Y / Z (horizontal / vertical / depth) three-dimensional position of the point is within its corresponding minimum / maximum range. If it is within the range, mark it in yellow in real time; otherwise, mark it in blue.

[0049] Preferably, the specific method for acquiring and real-time eliminating background noise in S3 is as follows:

[0050] S3.1: Based on the recording space defined by the aforementioned fixed-point positioning method steps, generate a three-dimensional data array of just the right size to record and save noise data generated by inherent objects within the recording space;

[0051] S3.2: During the tracking and recording process, for each point cloud data in the recording space, traverse 27 positions in a 3*3*3 grid around the data point in the 3D noise data frame. If no noise is found in the recording, then the data point is judged as one of the current valid data values.

[0052] S3.3: Sampling rate: Related to host performance. An 8th generation CPU i7 and a P2200 graphics card can achieve a sampling rate of around 50Hz. Methods to optimize running efficiency include limiting the range + interval + rotating sampling calculation. For example, the Kinect 2.0 depth camera has 512*424 3D data pixels. First, the effective range is defined by the XYZ range values ​​of eight specified vertices. Second, sampling is performed at intervals of n points each time, which reduces the amount of calculation per frame by n*n times. Then, an auxiliary rotating parameter is added, which changes one by one between 0 and n in each frame. This ensures that all pixels are calculated once every n frames. This can ensure the accuracy of the data while taking into account the running efficiency.

[0053] Preferably, the four-panel screens are: 1) infrared video recording data; 2) display of cage position and tracking position on the three-dimensional canvas; 3) display of infrared video recording data and cage position and tracking position on the three-dimensional canvas; 4) experiment number, date, record of real-time elapsed time (milliseconds), and real-time three-dimensional trajectory value (cm).

[0054] In the description of this specification, references to terms such as "an embodiment," "example," "specific example," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0055] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. A method for accurately tracking, measuring, and recording the three-dimensional movement trajectory of a single experimental animal based on a depth camera, characterized in that, Includes the following steps: S1: Install the depth camera 1.5 to 2 meters in front of the side of the transparent cage. Adjust the distance according to the size of the cage to make the image as large as possible while covering the cage. S2: Precise positioning of the eight vertices of the cage's internal space; S3: Acquisition and real-time elimination of background noise; S4: While measuring and recording 3D trajectory data in real time, the Kinect 2.0 infrared camera is used to simultaneously record four-frame images, making it easy for the experimenter to fully review the data; The vertex localization method of S2 is as follows: S2.1: Display the real-time 3D point cloud data from the camera on a 3D screen to ensure that the operator receives intuitive visual feedback when adjusting the positioning of the eight vertices of the cage. S2.2: Based on the spatial position of the eight vertices, calculate in real time and use different colors to indicate the point cloud pixels inside and outside the space; S2.3: Use reflective cloth to assist in the positioning of far-end vertices and other vertices with impaired visibility: Since the accurate position of the three-dimensional point cloud of each vertex in the cage space is often not very clear, a reflective cloth can be used. It is bright in the infrared image but not in the point cloud data. It can be placed in the corner of each vertex and rotated in three dimensions with the mouse to assist in the calibration and positioning. S2.4: The calibrated vertex physical positions and the corresponding physical distances between vertices in the 3D image are fed back to the operator in real time on the screen. By comparing with the actual size of the cage, the accurate position of the cage vertex can be further adjusted. This allows for accurate framing of the recording space and facilitates alignment of subsequent 3D tracking and recording data. The method for calibrating point cloud pixels inside and outside the space is as follows: S2.2.1: First, calculate the minimum / maximum depth values ​​corresponding to the front / back faces based on the X / Y position of any point according to the spatial linear mapping relationship; S2.2.2: Next, based on the depth value Z of the point and its proportion of the minimum / maximum depth value, calculate the minimum / maximum horizontal axis position on the left / right faces of the point, as well as the minimum / maximum height position on the upper / lower faces of the point, according to the spatial linear mapping relationship. S2.2.3: Finally, determine whether the X / Y / Z three-dimensional position of the point is within its corresponding minimum / maximum range. If it is within the range, mark it in yellow; otherwise, mark it in blue. The specific method for obtaining and real-time eliminating background noise in S3 is as follows: S3.1: Based on the recording space defined by the aforementioned fixed-point positioning method steps, generate a three-dimensional data array of just the right size to record and save noise data generated by inherent objects within the recording space; S3.2: During the tracking and recording process, for each point cloud data in the recording space, traverse 27 positions in a 3*3*3 grid around the data point in the 3D noise data frame. If no noise is found in the recording, then the data point is judged as one of the current valid data values. S3.3: Sampling rate: The method to optimize the running efficiency includes limiting the range + interval + rotating sampling calculation. Specifically, firstly, the effective range is defined by the XYZ range values ​​of the specified eight vertices. Secondly, sampling is performed at intervals of n points each time. Then, an auxiliary rotating parameter is added, which changes one by one between 0 and n in each frame to ensure that all pixels are calculated once every n frames.

2. The method for accurately tracking, measuring, and recording the three-dimensional movement trajectory of a single experimental animal based on a depth camera according to claim 1, characterized in that, The four-panel screens are: 1) infrared video recording data; 2) display of cage positions and tracking points on a 3D canvas; 3) display of infrared video recording data and cage positions and tracking points on a 3D canvas; 4) experiment number, date, record of real-time elapsed time, and real-time 3D trajectory value.

Images