Behavior tracking recording method in animal experiment process

Abstract

The invention relates to the technical field of animal behavior monitoring, in particular to a behavior tracking and recording method in an animal experiment process, and the method comprises the steps: obtaining an experiment video in the animal experiment process; dividing the experimental video based on the behavior characteristics of the animal, predicting the position of the animal in the segmented video, and determining the initiative of the animal according to the difference between the predicted position and the actual position; and based on the initiative of the animal in all the segmented videos of the experimental video and the change curve of the gravity center of the animal in all the segmented videos, tracking and recording the behavior of the animal. According to the invention, animal behaviors are tracked and monitored through a high-precision motion capture technology, and the initiative and motion state track of the animal behaviors in the experiment process are fully identified.

Description

Technical Field

[0001] This application relates to the technical field of animal behavior monitoring, and particularly relates to a method for tracking and recording animal behavior during animal experiments. Background Art

[0002] In the research of various human diseases, animals are usually used as research models to conduct ethological experiments, including open field experiments to study spontaneous activities and exploratory behaviors of animals, novel object recognition experiments to judge animal memory ability, water maze experiments to study animal spatial learning ability and memory, etc. And these experiments often require tracking and quantifying the movement trajectories of animals in order to conduct further analysis.

[0003] Currently, the method for tracking and recording animal behavior is to monitor animals in the experimental video through computer vision, taking animals as points, and plotting the changes of the points where animals are located in consecutive frames to complete the tracking and recording of animal behavior. This method fails to capture the actions of animals. When the activity of animals is relatively strong during the experiment, a large number of movement traces will be generated in the experimental area. When tracking and recording animal behavior by the changes of the points where animals are located, not only the movement trajectory characteristics of animals under different behaviors are ignored, but also the initiative of animal behavior is ignored. Summary of the Invention

[0004] In order to solve the above technical problems, this application provides a method for tracking and recording animal behavior during animal experiments to solve the existing problems.

[0005] The method for tracking and recording animal behavior during animal experiments in this application adopts the following technical solutions:

[0006] An embodiment of this application provides a method for tracking and recording animal behavior during animal experiments, and this method includes the following steps:

[0007] Step 1: Obtain the experimental video during the animal experiment;

[0008] Step 2: Divide the experimental video based on the behavioral characteristics of the animal, predict the position of the animal in the segmented video, and determine the initiative of the animal according to the difference between the predicted position and the actual position; specifically:

[0009] S1, Obtain the representative pixel points of the animal area in each frame of the experimental video, and use them to analyze the behavior of the animal in the animal area to segment the experimental video, and obtain several segmented videos;

[0010] S2. For any segmented video, analyze the change characteristics representing the animal behavior between all adjacent frame images, determine the predicted position of any representative pixel point in each frame image in the next frame image; and determine the initiative of the animal in any segmented video by combining the differences between the actual positions of the representative pixel points.

[0011] Step 3: Based on the initiative of the animal in all segmented videos of the experimental video and the change curve of the animal's center of gravity in all segmented videos, track and record the animal's behavior.

[0012] Preferably, in step S1, the method for segmenting the experimental video includes:

[0013] S11. Obtain the representative pixel points of the head, tail, center of gravity, and four limbs of the animal area in each frame image.

[0014] S12. Denote the straight-line distance from the head to the tail as the body length l of the animal, and respectively obtain the angles α and β between the lines connecting the two limbs on both sides of the body and the center of gravity.

[0015] S13. Combine the animal's body length l, angles α and β, and the distance between the heads in adjacent frame images to judge the behavior of the animal in each frame image.

[0016] S14. Merge the consecutive frame images with the same behavior and denote them as a segmented video.

[0017] Preferably, in step S1, the animal area is obtained by using the Otsu method for each frame image.

[0018] Preferably, in step S13, the method for judging the behavior of the animal in each frame image is:

[0019] Denote the index of the i-th behavior in each frame image as L i , where L1 represents the walking index, L2 represents the scanning index, and L3 represents the standing index.

[0020]

[0021] where norm represents the normalization function, and d head,1 , d head,2 respectively represent the distances between the representative pixel points of the head of the animal area in the current frame and the previous frame, and the next frame image.

[0022] Calculate the mean value of the indexes of all behaviors in each frame image, and take the behavior with the largest difference between the index in each frame image and the mean value of the indexes as the behavior of the animal in each frame image.

[0023] Preferably, in step S2, the method for determining the initiative of the animal in any segmented video includes:

[0024] S21. When any segmented video is a segmented video of an animal's walking or standing behavior, analyze the change distribution characteristics of the representative pixel points of the animal's head between any frame image and its previous two frame images, and construct a state transition model for this segmented video;

[0025] Wherein, the state transition model consists of a number of representative vectors and their corresponding transition probabilities;

[0026] S22. When any segmented video is a segmented video of an animal's scanning behavior, analyze the change distribution characteristics of the angle between the representative pixel points of the two limbs of the animal and the line connecting the centers of gravity between any frame image and its previous two frame images, and obtain the state transition model for this segmented video;

[0027] S23. For any frame image in any segmented video, use half of the body length of the animal in any frame image as the radius, draw a circle with the representative pixel point of the animal's head as the center, and evenly place a preset number of particles in the circle; perform curve fitting on the representative pixel points of the animal's head within the time window of any frame image to obtain the distance between any particle and the fitting point of the corresponding image in the fitting curve; and combine the curve distances between the corresponding fitting points on the fitting curve between all adjacent frame images within the time window of any frame image to determine the degree to which any particle in any frame image conforms to the animal's movement trend;

[0028] S24. Starting from any representative pixel point in any frame image, draw all the representative vectors in the state transition model of the segmented video where it is located to obtain the positions of the endpoints of the representative vectors; obtain the minimum value of the distances between any particle and the positions of the endpoints of all the representative vectors, use the endpoint position corresponding to the minimum value as the predicted position of the particle, and record the minimum value as the distance between the particle and the predicted position; based on the degree to which any particle in any frame image conforms to the animal's movement trend, the distance between any particle and its predicted position, the transition probability of the predicted position of any particle, and the coordinates of any particle, determine the predicted position of any representative pixel point in any frame image in the next frame image;

[0029] S25. Take the cumulative result of the distances between the predicted positions and the actual positions of all the representative pixel points in all the frame images of any segmented video as the initiative of the animal in any segmented video.

[0030] Preferably, the construction method of the state transition model in step S21 includes:

[0031] S211. Take the direction of the straight line connecting the representative pixel points of the animal's head in the previous frame image of any frame image and the frame image before the previous frame image as the polar axis of the polar coordinate;

[0032] S212. Use the vector in polar coordinates of the straight line connecting the representative pixel points of the head of the animal in the previous frame image and the current frame image of any frame as the motion vector of the current frame image;

[0033] S213. Use the region growing algorithm to cluster all the motion vectors of the segmented video to obtain several clusters;

[0034] S214. Take the mean of all the motion vectors of each cluster as the representative vector of the cluster, and obtain the ratio of the number of motion vectors of the cluster to the number of all motion vectors in the segmented video as the transition probability of the representative vector of the cluster;

[0035] S215. Construct the state transition model of the segmented video with all the representative vectors and their corresponding transition probabilities where a ′ 1, a ′ t are the first and the t-th representative vectors respectively, and Pa1, Pa t are the transition probabilities of the first and the t-th representative vectors respectively.

[0036] Preferably, the method for constructing the state transition model in step S22 includes:

[0037] S221. For any frame image and its previous frame image, obtain the ratio of the angles α and β between the two limbs on both sides of the body and the line connecting the center of gravity respectively. Take the arctangent of the difference between the ratios obtained from the current frame image and its previous frame image as the direction of the scanning feature vector of the current frame image, and take half of the body length of the animal in this frame image as the modulus length of the scanning feature vector of the current frame image;

[0038] S222. Use the region growing algorithm to cluster all the scanning feature vectors of the segmented video to obtain several clusters;

[0039] S223. Denote the mean of the scanning feature vectors of each cluster as the representative vector of the cluster, and obtain the ratio of the number of scanning feature vectors of the cluster to the number of all scanning feature vectors in the segmented video as the transition probability of the representative vector of the cluster;

[0040] S224. Construct the state transition model of the segmented video with all the representative vectors and their transition probabilities in the segmented video where h ′ 1, h ′ t are the first and the t-th representative vectors respectively, and Ph1, Ph t are the transition probabilities of the first and the t-th representative vectors respectively.

[0041] Preferably, the method for determining the degree to which any particle in any frame image conforms to the animal movement trend is as follows:

[0042] Denote the degree to which the z-th particle in the j-th frame image conforms to the animal movement trend as Q z ,

[0043]

[0044] where Q z represents the degree to which the z-th particle conforms to the animal movement trend; D z represents the distance between the z-th particle and the fitting point corresponding to the image in the fitting curve; exp represents the exponential function with the natural constant e as the base; M represents the size of the time window; g m , g m-1 respectively represent the curve distances between the m-th and (m - 1)-th frames in the time window of the j-th frame image and their corresponding fitting points on the fitting curve with respect to the previous frame image.

[0045] Preferably, the method for determining the predicted position of any representative pixel point in any frame image in the next frame image includes:

[0046] Denote the coordinates of the predicted position of the b-th representative pixel point in the j-th frame image in the (j + 1)-th frame image as (x b , y b ), where Z represents the number of particles in the (j + 1)-th frame image, and (x z , y z ) are the coordinates of the z-th particle;

[0047] where W z represents the weight of the z-th particle, and W z = Q z × P z × exp(-E z ); Q z represents the degree to which the z-th particle conforms to the animal movement trend; P z represents the transition probability of the predicted position of the z-th particle; exp represents the exponential function with the natural constant e as the base; E z represents the distance between the z-th particle and its predicted position.

[0048] Preferably, the method for tracking and recording the behavior of the animal includes:

[0049] Obtain the initiative of all segmented videos in the experimental video, count the maximum and minimum values among all the initiatives, and perform an equal-proportion mapping on the preset temperature range based on the maximum and minimum values, where the maximum initiative value corresponds to the highest temperature and the minimum initiative value corresponds to the lowest temperature, to generate a thermal image;

[0050] Meanwhile, for each segmented video of different behaviors, trajectory curves are respectively plotted, where the trajectory curves are obtained by curve fitting after connecting the representative pixel points of the center of gravity of the animal in all frame images of each segmented video;

[0051] Using the obtained thermal images and all the trajectory curves, the behaviors of the animals are tracked and recorded.

[0052] The present application has at least the following beneficial effects:

[0053] The present application takes into account that the purpose of animal behavior experiments is to analyze the behaviors of animals, and the behaviors of animals come from the decision-making results of animals, that is, the initiative of animals. The present application tracks and monitors animal behaviors through high-precision motion capture technology, analyzes the continuous animal motion trajectory characteristics under different behaviors, and simultaneously identifies the initiative in animal behavior characteristics, tracks and records the behaviors of animals during the animal experiment process, fully identifies the initiative of animal behaviors and the motion state trajectories during the experiment process, and is beneficial to improving the analysis efficiency and effect of the experimenter on the experiment process. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required to be used in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0055] Figure 1 It is a flowchart of a method for tracking and recording behaviors during an animal experiment provided by the present application;

[0056] Figure 2 It is a schematic diagram of the process of an animal experiment;

[0057] Figure 3 It is a flowchart of the determination process of the initiative of animals;

[0058] Figure 4 It is a schematic diagram of walking, scanning and standing in animal behaviors;

[0059] Figure 5 It is a flowchart of a method for segmenting an experimental video;

[0060] Figure 6 It is a frame image in an experimental video of an open field experiment;

[0061] Figure 7 It is a schematic diagram of the representative pixel points of the head, tail, center of gravity and four limbs of the animal area;

[0062] Figure 8 Flow chart of the method for determining the initiative of animals in any segmented video

[0063] Figure 9 Flow chart of the method for constructing the state transition model of the segmented video of the walking or standing behavior of animals

[0064] Figure 10 Motion vector a of the j-th frame image j Schematic diagram

[0065] Figure 11 Flow chart of the method for constructing the state transition model of the segmented video of the scanning behavior of animals Specific implementation mode

[0066] In order to further elaborate on the technical means and effects adopted by the present application to achieve the predetermined invention purpose, the following combines the drawings and preferred embodiments to detail the specific implementation mode, structure, features and effects of a method for behavior tracking and recording in an animal experiment proposed according to the present application. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, the specific features, structures or characteristics in one or more embodiments can be combined in any suitable form

[0067] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the technical field to which the present application belongs

[0068] The following specifically describes the specific solution of a method for behavior tracking and recording in an animal experiment provided by the present application with reference to the drawings

[0069] A method for behavior tracking and recording in an animal experiment provided by an embodiment of the present application

[0070] Specifically, the following is provided a method for behavior tracking and recording in an animal experiment. Please refer to Figure 1 , and the method includes the following steps

[0071] Step 1: Obtain the experimental video during the animal experiment

[0072] Set up the experimental scene, place the camera and light source directly above the experimental area, and be able to fully monitor the experimental scene, such as Figure 2As shown. Place the animal in the experimental scenario, start the camera to record the experimental process, end the experiment after a preset duration, turn off the camera, and transfer the experimental video during the animal experiment to the computer. Among them, in this embodiment, the setting of the experimental scenario and the duration of the experiment are both set by the implementer according to the specific purpose of the experiment. In one implementation manner of this embodiment, the duration of the experiment is set to half an hour, and the experimental scenario is set as a cylindrical frame with a circular bottom diameter of 100 cm and a box height of 30 cm.

[0073] Step 2: Divide the experimental video based on the behavioral characteristics of the animal, predict the position of the animal in the segmented video, and determine the initiative of the animal according to the difference between the predicted position and the actual position.

[0074] The behavior of the animal is a continuous decision-making process. The predicted position is based on the action inertia and the posterior probability of statistics, and the actual position is based on the action mutation of the decision. The active behavior of the animal can be obtained through the difference between the predicted position and the actual position. The active behavior is reflected in the posture change and the movement state change of the animal. Therefore, the experimental video can be segmented based on the posture and movement state of the animal, and then the area and initiative characteristics of each segment can be determined according to the difference between the predicted position and the actual position.

[0075] In this application, the flowchart for determining the initiative of the animal is as shown in the appendix Figure 3 as follows:

[0076] S1, Obtain the representative pixel points of the animal area in each frame of the experimental video for analyzing the behavior of the animal in the animal area to segment the experimental video and obtain several segmented videos.

[0077] The behaviors of the animal mainly include walking, scanning, and standing, as Figure 4 shown, where walking is an action of stretching the body length and moving the body quickly; scanning is an action of only turning the head; standing is standing still. This application aims to divide the video into several segments, each segment being a kind of behavior, and the characteristics for distinguishing these behaviors lie in the posture and movement state of the animal.

[0078] Label the animal area in each frame of the image. The posture index of the animal can be obtained through the body length and turning angle of the animal, and the movement state of the animal can be obtained through the change of the animal position between frames, so that the behavior of the animal can be segmented in the video.

[0079] In this application, the flowchart for segmenting the experimental video is as shown in the appendix Figure 5 as follows:

[0080] First, obtain the animal area in each frame of the experimental video. As Figure 6It is a frame image in the experimental video of an open field experiment. Since there is often a large gray-scale difference between the animal and the experimental environment, threshold segmentation can be performed by the Otsu method to obtain the animal region. Here, the Otsu method is a well-known technique and will not be elaborated further.

[0081] S11, obtain the representative pixel points of the head, tail, center of gravity, and four limbs of the animal region in each frame image;

[0082] As Figure 7 shown, first, manually label the representative pixel points of the head, tail, center of gravity, and four limbs of the animal region in each frame image, and then train the animal regions in all frame images through a neural network to obtain the representative pixel points of the head, tail, center of gravity, and four limbs of the animal region in each frame image.

[0083] S12, record the straight-line distance from the head to the tail as the body length l of the animal, and respectively obtain the angles α and β between the two limbs on both sides of the body and the line connecting to the center of gravity;

[0084] S13, combine the animal body length l, angles α and β, and the distance between the heads in adjacent frame images to judge the behavior of the animal in each frame image;

[0085] Preferably, in this embodiment, the method for judging the behavior of the animal in each frame image is:

[0086] Record the index of the i-th behavior in each frame image as L i , where L1 represents the walking index, L2 represents the scanning index, and L3 represents the standing index;

[0087]

[0088] where norm represents the normalization function, and d head,1 and d head,2 respectively represent the distances between the representative pixel points of the head of the animal region in the current frame and the previous frame and the next frame images.

[0089] Furthermore, calculate the mean value of the indices of all behaviors in each frame image, and take the behavior with the largest difference between the index in each frame image and the mean value of the indices as the behavior of the animal in each frame image.

[0090] S14, merge the consecutive frame images with the same behavior and record them as a segmented video.

[0091] So far, several segmented videos can be obtained through the method in step S1.

[0092] S2. For any segmented video, analyze the change characteristics representing the animal behavior of the pixel points between all adjacent frame images, and determine the predicted position of any representative pixel point in each frame image in the next frame image; and determine the initiative of the animal in any segmented video in combination with the difference between the actual positions of the representative pixel points.

[0093] Particle filtering refers to a method of approximating the probability density distribution by finding a set of random samples (particles) propagating in the state space and replacing the integral operation with the sample mean. By statistically analyzing the changes in the animal state of the segmented video of each behavior, the state transition probability of the animal can be obtained.

[0094] For different behaviors, there are different state transition methods, so the state transition probabilities are also different and need to be analyzed separately. Further, the movement of the animal is continuous. Therefore, when performing particle filtering, adjustments need to be made to the arrangement of the particles and the reference degree of different particles in the mean calculation.

[0095] In this application, the flow chart of the method for determining the initiative of the animal in any segmented video is as shown in the appendix Figure 8 as follows:

[0096] S21. When the segmented video is a segmented video of the animal's walking or standing behavior, analyze the change distribution characteristics of the representative pixel points of the animal's head between any frame image and its previous two frame images, and construct the state transition model of this segmented video.

[0097] In this application, the flow chart of the method for constructing the state transition model of the segmented video of the animal's walking or standing behavior is as shown in the appendix Figure 9 as follows:

[0098] S211. Take the direction of the straight line connecting the representative pixel points of the animal's head in the previous frame image of any frame image and the previous frame image of its previous frame image as the polar axis of the polar coordinate;

[0099] S212. Take the vector of the straight line connecting the representative pixel points of the animal's head in the previous frame image of any frame image and the said any frame image in the polar coordinate as the motion vector of the said any frame image;

[0100] In an embodiment of this application, the schematic diagram of the motion vector a j of the j-th frame image is as shown in the appendix Figure 10 as follows. Among them, for the j-th frame image, record the direction of the straight line connecting the representative pixel points of the animal's head in the (j - 2)-th frame image and the (j - 1)-th frame image as the polar axis of the polar coordinate in the j-th frame image. On the basis of the polar coordinate, obtain the vector formed by the straight line connecting the representative pixel points of the animal's head in the (j - 1)-th frame image and the j-th frame image as the motion vector a jThe motion vectors reflect the changes in the animal's behavioral state. By statistically analyzing them, the state transition probability representing the behavior of the segmented video can be obtained.

[0101] It should be noted that the motion vectors of the first and second frames of images in any segmented video are not calculated.

[0102] S213. Use the region growing algorithm to cluster all the motion vectors of the segmented video to obtain several clusters.

[0103] It should be noted that when using the region growing algorithm for clustering, the first motion vector is used as the growing seed point; in the order of the acquisition time of all frame images, the cosine similarity between motion vectors greater than 0.8 is used as the growing condition; after not meeting the growing condition, any un-grown motion vector is used as the growing seed point to continue growing until all motion vectors are grown. Among them, the region growing algorithm is a well-known technology and will not be elaborated.

[0104] S214. Take the mean of all the motion vectors of each cluster as the representative vector of the cluster, and obtain the ratio of the number of motion vectors of the cluster to the number of all motion vectors in the segmented video as the transition probability of the representative vector of the cluster.

[0105] S215. Construct the state transition model of the segmented video with all the representative vectors and their corresponding transition probabilities. where, a ′ 1, a ′ t are the first and the t-th representative vectors respectively, and Pa1, Pa t are the transition probabilities of the first and the t-th representative vectors respectively.

[0106] S22. When any segmented video is a segmented video of the animal's scanning behavior, analyze the change distribution characteristics of the angle between the representative pixel points of the two limbs of the animal and the line connecting the centers of gravity in any frame image and its previous two frame images, and obtain the state transition model of the segmented video.

[0107] In this application, the flowchart of the method for constructing the state transition model of the segmented video of the animal's scanning behavior is as shown in the appendix Figure 11 as follows:

[0108] S221. For any frame image and its previous frame image, respectively obtain the ratio of the angles α and β between the two limbs on both sides of the body and the line connecting the centers of gravity. Take the arctangent of the difference between the ratios obtained from any frame image and its previous frame image as the direction of the scanning feature vector of the frame image, and take half of the body length of the animal in the frame image as the modulus length of the scanning feature vector of the frame image.

[0109] In a specific embodiment, taking the j-th frame image as an example, obtain the ratio of the angles α and β between the two limbs on both sides of the animal region in the j-th frame image and the line connecting the centers of gravity and that in the (j - 1)-th frame image The difference therebetween is denoted as the scanning change feature of the j-th frame image: Scanning change feature h j reflects the change in the state of the animal.

[0110] Meanwhile, since the body length of the mouse remains unchanged and the length between the head and the center of gravity of the body is basically unchanged, therefore, in the state transition model, only the change in its scanning angle is concerned, regardless of the body length. Accordingly, take half of the animal body length as the modulus length of the scanning feature vector of the j-th frame image, and take the arctangent value of the scanning change feature of the j-th frame image as the direction of the scanning feature vector of the j-th frame image.

[0111] S222, Use the region growing algorithm to cluster all the scanning feature vectors of this segmented video to obtain several cluster classes;

[0112] Among them, the clustering method is the same as that of all the above-mentioned motion vector clustering methods.

[0113] S223, Denote the mean value of the scanning feature vectors of each cluster class as the representative vector of this cluster class, and obtain the ratio of the number of the scanning feature vectors of this cluster class to the number of all the scanning feature vectors in this segmented video as the transition probability of the representative vector of this cluster class;

[0114] S224, Use all the representative vectors and their transition probabilities in this segmented video to form the state transition model of this segmented video where h ′ 1, h ′ t are the 1st and t-th representative vectors respectively, and Ph1, Ph t are the transition probabilities of the 1st and t-th representative vectors respectively.

[0115] S23, For any frame image in any segmented video, take half of the animal body length in any frame image as the radius, draw a circle with the representative pixel point of the animal's head as the center, and evenly place a preset number of particles in the circle; perform curve fitting on the representative pixel point of the animal's head within the time window of any frame image to obtain the distance between any particle and the fitting point corresponding to the image to which it belongs in the fitting curve; and combine the curve distances between the corresponding fitting points on the fitting curve between all adjacent frame images within the time window of any frame image to determine the degree to which any particle in any frame image conforms to the animal's motion trend.

[0116] According to the state transition model of all animal behaviors, the state of animals in any frame image of any segmented video can be predicted. Since the state transition model is based on statistics and uses the frequency of event occurrence as the transition probability, the movement of animals in consecutive frames is not only related to the previous frame but also should refer to the movement trend of the animals.

[0117] In a specific implementation, taking the j-th frame image in any segmented video as an example, first obtain the body length of the animal in the j-th frame image. Use half of the animal's body length as the radius, and take the representative pixel point of the animal's head in the j-th frame image as the center of the circle to draw a circle. Use this circle as the distribution space of the particles and evenly place a preset number of particles. In this embodiment, the preset number Z is taken as 100.

[0118] Furthermore, set the time window of the j-th frame image. In this embodiment, the size M of the time window is set to 9, that is, the 4 frames before and the 4 frames after the j-th frame image form the time window of the j-th frame image. If the number of frames before or after is insufficient, make up the difference on the other side.

[0119] For any particle, the more it conforms to the movement law of the animal's head in the time window, the more it conforms to the movement trend of the animal. Thus, calculate the degree to which any particle conforms to the movement trend of the animal.

[0120] Use the least squares method to perform polynomial fitting on the representative pixel points of the animal's head in the time window of the j-th frame image to fit them into a curve. Obtain the distance D between any particle and the fitting point of the corresponding image in the fitting curve, and calculate the degree to which each particle conforms to the movement trend of the animal. Taking the z-th particle in the j-th frame image as an example, record the degree to which the z-th particle in the j-th frame image conforms to the movement trend of the animal as Q z ,

[0121] where, Q z represents the degree to which the z-th particle conforms to the movement trend of the animal; D z represents the distance between the z-th particle and the fitting point of the corresponding image in the fitting curve; exp represents the exponential function with the natural constant e as the base; M represents the size of the time window; g m 、g m-1 respectively represent the curve distances between the m-th and (m - 1)-th frames in the time window of the j-th frame image and the corresponding fitting points on the fitting curve of their previous frame images. Among them, it is necessary to satisfy m - 1 > 1, so it can only start from 3). represents the difference in the movement of the animal on consecutive frames. The larger this value is, the less the particle conforms to the movement law of the animal. Among them, the more the particle conforms to the movement trend of the animal and the more its position conforms to the state in the state transition model, the greater the particle weight.

[0122] S24. Starting from any representative pixel point in any frame of the image, draw all the representative vectors in the state transition model of the segmented video to which it belongs, and obtain the positions of the endpoints of the representative vectors. Obtain the minimum value of the distances between any particle and the positions of the endpoints of all the representative vectors, take the endpoint position corresponding to the minimum value as the predicted position of the particle, and record the minimum value as the distance between the particle and the predicted position. Based on the degree to which any particle in any frame of the image conforms to the animal movement trend, the distance between any particle and its predicted position, the transition probability of the predicted position of any particle, and the coordinates of any particle, determine the predicted position of any representative pixel point in the next frame of the image.

[0123] Taking any segmented video as an example, for the j-th frame of the image, predict the position of the animal in the (j + 1)-th frame of the image:

[0124] First, obtain all the representative pixel points of the animal in the j-th frame of the image. Taking any representative pixel point as an example, starting from the representative pixel point, draw a representative vector in the image and mark the position of the endpoint of the representative vector. Obtain the distance between any particle and the position of the endpoint of any representative vector, and take the minimum value. Take the endpoint position corresponding to the minimum value as the predicted position of the particle, and record the minimum value as the distance E between the particle and the predicted position. z 。

[0125] It should be understood that the closer the distance between the particle and the predicted position is, the greater the weight of the particle is. In addition, the greater the transition probability corresponding to the predicted position is, and the more the particle conforms to the animal movement law, the greater the weight of the particle is. Thus, calculate the weight of the particle. Taking the weight W of the z-th particle as an example, the expression is: z For example, the expression is: W z = Q z × P z × exp(-E z );

[0126] Among them, W z is the weight of the z-th particle; Q z represents the degree to which the z-th particle conforms to the animal movement trend; P z represents the transition probability of the predicted position of the z-th particle; exp represents the exponential function with the natural constant e as the base; E z represents the distance between the z-th particle and its predicted position.

[0127] Furthermore, weight all the weights of the particles to obtain the predicted position of any representative pixel point. Taking the coordinates (x b , y b ) of the predicted position of the b-th representative pixel point in the (j + 1)-th frame of the image in the j-th frame of the image as an example, the expression is: Among them, (x b , yb ) is the predicted position coordinates of the b-th representative pixel point in the (j + 1)-th frame image, Z represents the number of particles in the (j + 1)-th frame image, and W z is the weight of the z-th particle; (x z , y z ) are the coordinates of the z-th particle.

[0128] S25. Take the accumulated result of the distances between the predicted positions and the actual positions of all representative pixel points in all frame images of any segmented video as the initiative of the animal in the segmented video.

[0129] For any segmented video, the greater the difference between the predicted position and the actual position of the animal representative pixel points in each frame image, the smaller the degree to which the animal acts according to habits and inertial behaviors, and the stronger the initiative. The calculation expression for calculating the initiative of the animal in any segmented video is as follows:

[0130] where R represents the initiative of the animal in any segmented video; J represents the number of frame images in the segmented video; B represents the number of representative pixel points in each frame image; n j,b represents the distance between the b-th representative pixel point in the j-th frame image and its predicted position.

[0131] Step 3: Based on the initiative of the animal in all segmented videos of the experimental video and the change curve of the animal's center of gravity in all segmented videos, track and record the animal's behavior.

[0132] According to the above Step 2, obtain the initiative of all segmented videos in the experimental video, count the maximum value and the minimum value among all the initiatives, and perform an equal-proportion mapping on the preset temperature range based on the maximum value and the minimum value, so that the maximum initiative corresponds to the highest temperature and the minimum initiative corresponds to the lowest temperature, and generate a thermal image. In this embodiment, the preset temperature range takes the value of [-100, 100], which can be specifically set by the implementer.

[0133] It should be understood that the higher the temperature in the thermal image, the stronger the initiative of the animal, and the lower the temperature, the weaker the initiative of the animal.

[0134] At the same time, for each segmented video with different behaviors, draw a trajectory curve respectively, where the trajectory curve is obtained by curve fitting after connecting the center-of-gravity representative pixel points of all frame images in each segmented video. In this embodiment, the least squares method is used for curve fitting.

[0135] Use the obtained thermal image and all the trajectory curves to track and record the animal's behavior.

[0136] It should be noted that the higher the temperature in the thermal image, the more active the behavior of the animal at this time, indicating a stronger sense of initiative. Therefore, when analyzing the trajectory curves of different segmented videos, appropriate consideration should be given to predicting the future behavior trajectory of the animal when it has a sense of initiative. The lower the temperature, the more likely the behavior of the animal at this time is an inertial action. Therefore, when analyzing the trajectory curves of different segmented videos, appropriate consideration should be given to predicting the future behavior trajectory of the animal in an inertial state.

[0137] It should be understood that the thermal image and the trajectory curve can be used to reflect the key points of analysis in the animal behavior tracking process. At the same time, during the repeated experiment process, by recording the changes in the parameters, the experimental environment and the animal state can be analyzed, which can ensure the efficient tracking and analysis of the animal behavior by the experimenter.

[0138] Each embodiment in this application is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key point of each embodiment is to illustrate the differences from other embodiments.

[0139] It should be noted that unless otherwise specified and limited, terms such as "including", "comprising" or any other variant thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or further includes elements inherent to such article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the article or device including the said element. In addition, the term "and / or" used herein includes any and all combinations of one or more of the related listed items.

[0140] Those skilled in the art will readily conceive of other embodiments of the present application after considering the specification and practicing the invention herein. This application is intended to cover any variations, uses, or adaptations of the present application, which follow the general principles of the present application and include common general knowledge or conventional technical means in the technical field not invented by the present application.

[0141] It should be understood that the present application is not limited to the exact structure already described and shown in the drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A method for tracking and recording behavior during an animal experiment, characterized in that: The method comprises the following steps: Step 1: Obtain the experimental video during the animal experiment; Step 2: Segment the experimental video based on the behavioral characteristics of the animal, predict the position of the animal in the segmented video, and determine the initiative of the animal based on the difference between the predicted position and the actual position; specifically: S1, obtaining representative pixel points of the animal region of each frame of the experimental video, which are used to analyze the behavior of the animals in the animal region to segment the experimental video and obtain a number of segmented videos; S2, for any segmented video, analyzing the changing characteristics of the representative pixels between all adjacent frame images reflecting the animal behavior, determining the predicted position of any representative pixel in each frame image in the next frame image; and determining the initiative of the animal in any segmented video based on the difference between the actual positions of the representative pixels; Step 3: Based on the initiative of the animal in all segmented videos of the experimental video and the changing curve of the center of gravity of the animal in all segmented videos, the behavior of the animal is tracked and recorded.

2. The method for tracking and recording behavior during an animal experiment as claimed in claim 1, characterized in that: In step S1, the method for segmenting the experimental video includes: S11, obtaining representative pixel points of the head, tail, center of gravity and limbs of the animal region in each frame of the image; S12, record the straight-line distance from the head to the tail as the animal's body length l, and obtain the angles α and β between the two limbs on both sides of the body and the line connecting the center of gravity; S13, judging the behavior of the animal in each frame of the image based on the animal's body length l, the angles α and β, and the distance between the heads in adjacent frames of the image; S14, merging consecutive frame images with the same behavior and recording them as a segmented video.

3. The method for tracking and recording behavior during an animal experiment as claimed in claim 1, characterized in that: In step S1, the animal region is obtained by applying the Otsu method to each frame of the image.

4. The method for tracking and recording behavior during an animal experiment as claimed in claim 2, characterized in that: In step S13, the method for determining the behavior of the animal in each frame of image is as follows: The index of the i-th behavior in each frame is marked as L i , where L1 represents the walking index, L2 represents the scanning index, and L3 represents the standing index; Among them, norm represents the normalization function, d head,1 ,d head,2 Respectively represent the distance between the head representative pixel points of the animal area in the current frame, the previous frame, and the next frame; The mean index of all behaviors in each frame of image is calculated, and the behavior with the largest difference between the index in each frame of image and the mean index is taken as the behavior of the animal in each frame of image.

5. The method for tracking and recording behavior during an animal experiment as claimed in claim 4, characterized in that: In step S2, the method for determining the initiative of the animal in any segmented video includes: S21, when any segmented video is a segmented video of an animal's walking or standing behavior, analyzing the change distribution characteristics of the pixel points representing the animal's head between any frame image and the two frames before it, and constructing a state transition model of the segmented video; Wherein, the state transition model is composed of a number of representative vectors and their corresponding transition probabilities; S22, when any segmented video is a segmented video of the scanning behavior of an animal, analyzing the change distribution characteristics of the angle between the pixel points representing the two limbs of the animal and the line connecting the center of gravity between any frame image and the previous two frames image, and obtaining a state transition model of the segmented video; S23, for any frame image in any segmented video, take half of the body length of the animal in any frame image as the radius, draw a circle with the representative pixel point of the animal's head as the center, and evenly place a preset number of particles in the circle; perform curve fitting on the representative pixel point of the animal's head in the time window of any frame image, and obtain the distance between any particle and the fitting point of the corresponding image in the fitting curve; and combine the curve distance between the corresponding fitting points on the fitting curve between all adjacent frame images in the time window of any frame image to determine the degree to which any particle in any frame image conforms to the animal's movement trend; S24, taking any representative pixel point in any frame image as the starting point, drawing all representative vectors in the state transition model of the segmented video where it is located, and obtaining the position of the end point of the representative vector; obtaining the minimum value of the distance between any particle and the position of the end point of all representative vectors, taking the end point position corresponding to the minimum value as the predicted position of the particle, and recording the minimum value as the distance between the particle and the predicted position; based on the degree to which any particle in any frame image conforms to the animal movement trend, the distance between any particle and its predicted position, the transition probability of the predicted position of any particle, and the coordinates of any particle, determining the predicted position of any representative pixel point in any frame image in the next frame image; S25, taking the accumulated result of the distances between the predicted positions and the actual positions of all representative pixel points in all frame images in any segmented video as the initiative of the animal in any segmented video.

6. A method for tracking and recording behavior during an animal experiment as claimed in claim 5, characterized in that: The method for constructing the state transition model in step S21 includes: S211, taking the direction of the straight line connecting the pixel points representing the head of the animal in the previous frame image of any frame image and the previous frame image of the previous frame image as the polar axis of the polar coordinates; S212, taking a vector in polar coordinates of a straight line connecting a previous frame image of any frame image and representative pixel points of the animal's head in any frame image as a motion vector of any frame image; S213, clustering all motion vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters; S214, taking the mean of all motion vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of motion vectors of the cluster to the number of all motion vectors in the segmented video as the transfer probability of the representative vector of the cluster; S215: All representative vectors and their corresponding transition probabilities constitute a state transition model for the segmented video. Among them, a ′ 1.a ′ t are the 1st and tth representative vectors, Pa1 and Pa t are the transition probabilities of the 1st and tth representative vectors respectively.

7. The method for tracking and recording behavior during an animal experiment as claimed in claim 5, characterized in that: The method for constructing the state transition model in step S22 includes: S221, for any frame image and its previous frame image, respectively obtain the ratios of the angles α and β between the two limbs on both sides of the body and the line connecting the center of gravity, use the arc tangent value of the difference between the ratios obtained for any frame image and its previous frame image as the direction of the scanning feature vector of the any frame image, and use half of the body length of the animal in the frame image as the modulus length of the scanning feature vector of the frame image; S222, clustering all scanned feature vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters; S223, recording the mean of the scanning feature vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of scanning feature vectors of the cluster to the number of all scanning feature vectors in the segmented video as the transfer probability of the representative vector of the cluster; S224, all representative vectors and their transition probabilities in the segmented video constitute a state transition model of the segmented video Among them, h ′ 1.h ′ t are the 1st and tth representative vectors, Ph1 and Ph t are the transition probabilities of the 1st and tth representative vectors respectively.

8. The method for tracking and recording behavior during an animal experiment as claimed in claim 5, characterized in that: The method for determining the degree to which any particle in any frame image conforms to the animal movement trend is as follows: The degree to which the zth particle in the jth frame image conforms to the animal's movement trend is recorded as Q z , Among them, Q z Indicates the degree to which the zth particle conforms to the animal's movement trend; D z represents the distance between the zth particle and the fitting point of the corresponding image in the fitting curve; exp represents the exponential function with the natural constant e as the base; M represents the size of the time window; g m , g m-1 They respectively represent the curve distances between the corresponding fitting points on the fitting curve of the m-th frame and the m-1-th frame image in the time window of the j-th frame image and their previous frame image.

9. The method for tracking and recording behavior during an animal experiment as claimed in claim 5, characterized in that: The method for determining the predicted position of any representative pixel point in any frame image in the next frame image includes: The coordinates of the predicted position of the bth representative pixel in the jth frame image in the j+1th frame image are marked as (x b ,y b ), Where Z represents the number of particles in the j+1th frame image, (x z ,y z ) is the coordinate of the zth particle; Among them, W z represents the weight of the zth particle, W z =Q z ×P z ×exp(-E z );Q z Indicates the degree to which the zth particle conforms to the animal's movement trend; P z represents the transition probability of the predicted position of the zth particle; exp represents the exponential function with the natural constant e as the base; E z represents the distance of the zth particle from its predicted position.

10. The method for tracking and recording behavior during an animal experiment as claimed in claim 2, characterized in that: The method for tracking and recording the behavior of the animal comprises: Obtain the activity of all segmented videos in the experimental video, count the maximum and minimum values ​​of all the activities, and perform proportional mapping on the preset temperature range based on the maximum and minimum values, wherein the maximum activity corresponds to the highest temperature and the minimum activity corresponds to the lowest temperature, and generate a thermal image; At the same time, for each segmented video of different behaviors, a trajectory curve is drawn respectively, wherein the trajectory curve is obtained by connecting the pixel points representing the center of gravity of the animal in all frame images in each segmented video and performing curve fitting; The obtained thermal images and all trajectory curves are used to track and record the behavior of the animals.

Images