A behavior tracking and recording method during animal experiments

Abstract

The present application relates to the field of animal behavior monitoring technology, and specifically to a method for tracking and recording animal behavior during an experiment. The method comprises: obtaining an experimental video of the animal during the experiment; segmenting the experimental video based on the animal's behavioral characteristics, predicting the animal's position in the segmented video, and determining the animal's initiative based on the difference between the predicted position and the actual position; and tracking and recording the animal's behavior based on the animal's initiative in all segmented videos of the experimental video and the curve of the animal's center of gravity in all segmented videos. The present application aims to track and monitor animal behavior using high-precision motion capture technology, fully identifying the animal's initiative and motion state trajectory during the experiment.

Description

Technical Field

[0001] The present application relates to the technical field of animal behavior monitoring, and in particular to a method for tracking and recording behavior during an animal experiment. Background Art

[0002] In the study of various human diseases, animals are usually used as research models to conduct animal behavioral experiments, including open field experiments to study animals' spontaneous activities and exploratory behaviors, novel object recognition experiments to judge animals' memory abilities, and water maze experiments to study animals' spatial learning abilities and memory. These experiments often require tracking and quantifying the animals' movement trajectories for further analysis.

[0003] The current method for tracking and recording animal behavior is to use computer vision to monitor animals in experimental videos. Using the animals as points, the method maps the changes in the animal's location across consecutive frames to track and record the animal's behavior. This method fails to capture the animal's movements. When animals are highly active during an experiment, they produce numerous movement traces within the experimental area. Tracking animal behavior based on changes in the animal's location not only ignores the characteristics of the animal's movement trajectory under different behaviors, but also overlooks the initiative of the animal's behavior. Summary of the Invention

[0004] In order to solve the above technical problems, the present application provides a behavior tracking and recording method during an animal experiment to solve the existing problems.

[0005] The present invention relates to a method for tracking and recording behavior during an animal experiment using the following technical solutions:

[0006] One embodiment of the present application provides a method for tracking and recording behavior during an animal experiment, the method comprising the following steps:

[0007] Step 1: Obtain experimental videos during animal experiments;

[0008] Step 2: Segment the experimental video based on the animal's behavioral characteristics, predict the animal's position in the segmented video, and determine the animal's initiative based on the difference between the predicted position and the actual position; specifically:

[0009] S1, obtaining representative pixel points of the animal region of each frame of the experimental video, for analyzing the behavior of the animals in the animal region to segment the experimental video, and obtaining a plurality of segmented videos;

[0010] S2, for any segmented video, analyzes the changing characteristics of the representative pixels between all adjacent frames reflecting the animal's behavior, determines the predicted position of any representative pixel in each frame in the next frame; and determines the initiative of the animal in any segmented video based on the difference between the actual positions of the representative pixels;

[0011] Step 3: Based on the animal's initiative in all segments of the experimental video and the change curve of the animal's center of gravity in all segments, the animal's behavior is tracked and recorded.

[0012] Preferably, in step S1, the method for segmenting the experimental video includes:

[0013] S11, obtaining representative pixel points of the head, tail, center of gravity and limbs of the animal region in each frame of the image;

[0014] S12, record the straight-line distance from the head to the tail as the animal's body length l, and obtain the angles α and β between the two limbs on both sides of the body and the line connecting the center of gravity;

[0015] S13, judging the behavior of the animal in each frame of the image based on the animal's body length l, the angles α and β, and the distance between the heads in adjacent frames of the image;

[0016] S14, merging consecutive frame images with the same behavior and recording them as a segmented video.

[0017] Preferably, in step S1, the animal region is obtained by applying the Otsu method to each frame of the image.

[0018] Preferably, in step S13, the method for determining the behavior of the animal in each frame of image is:

[0019] The index of the i-th behavior in each frame is marked as L i , where L1 represents the walking index, L2 represents the scanning index, and L3 represents the standing index;

[0020]

[0021] Among them, norm represents the normalization function, d head,1 d head,2 Respectively represent the distance between the head representative pixel points of the animal area in the current frame, the previous frame, and the next frame;

[0022] The mean index of all behaviors in each frame of image was calculated, and the behavior with the largest difference between the index in each frame of image and the mean index was taken as the behavior of the animal in each frame of image.

[0023] Preferably, in step S2, the method for determining the initiative of the animal in any segmented video includes:

[0024] S21, when any segmented video is a segmented video of an animal walking or standing, analyzing the change distribution characteristics of the pixel points representing the animal's head between any frame image and the two frames before it, and constructing a state transition model for the segmented video;

[0025] Wherein, the state transition model is composed of several representative vectors and their corresponding transition probabilities;

[0026] S22, when any segmented video is a segmented video of an animal's scanning behavior, analyzing the distribution characteristics of changes in the angles between the pixel points representing the animal's two limbs and the line connecting the center of gravity between any frame image and the two frames before it, to obtain a state transition model for the segmented video;

[0027] S23, for any frame image in any segmented video, using half the body length of the animal in any frame image as the radius, drawing a circle with the pixel representing the animal's head as the center, and evenly placing a preset number of particles in the circle; performing curve fitting on the pixel representing the animal's head within the time window of any frame image, obtaining the distance between any particle and the fitting point of the corresponding image on the fitting curve; and combining the curve distances between corresponding fitting points on the fitting curve between all adjacent frame images within the time window of any frame image, determining the degree to which any particle in any frame image conforms to the animal's movement trend;

[0028] S24, taking any representative pixel point in any frame image as a starting point, plotting all representative vectors in the state transition model of the segmented video in which it is located, and obtaining the positions of the end points of the representative vectors; obtaining the minimum value of the distance between any particle and the positions of the end points of all representative vectors, using the end point position corresponding to the minimum value as the predicted position of the particle, and recording the minimum value as the distance between the particle and the predicted position; determining the predicted position of any representative pixel point in any frame image in the next frame image based on the degree to which any particle in any frame image conforms to the animal movement trend, the distance between any particle and its predicted position, the transition probability of any particle's predicted position, and the coordinates of any particle;

[0029] S25, taking the accumulated distances between the predicted positions and the actual positions of all representative pixels in all frame images in any segmented video as the initiative of the animal in any segmented video.

[0030] Preferably, the method for constructing the state transition model in step S21 includes:

[0031] S211, using the direction of the straight line connecting the pixel points representing the animal's head in the previous frame of any image and the previous frame of the previous frame as the polar axis of the polar coordinate;

[0032] S212, taking a vector in polar coordinates of a straight line connecting a previous frame image of any frame image and a pixel point representing the animal's head in any frame image as a motion vector of the any frame image;

[0033] S213, clustering all motion vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters;

[0034] S214, taking the mean of all motion vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of motion vectors of the cluster to the number of all motion vectors in the segmented video, and using this ratio as the transition probability of the representative vector of the cluster;

[0035] S215: All representative vectors and their corresponding transition probabilities constitute a state transition model for the segmented video. Among them, a ′ 1.a ′ t are the 1st and tth representative vectors, Pa1 and Pa t are the transition probabilities of the 1st and tth representative vectors respectively.

[0036] Preferably, the method for constructing the state transition model in step S22 includes:

[0037] S221, for any frame image and its previous frame image, respectively obtain the ratios of the angles α and β between the two limbs on both sides of the body and the line connecting the center of gravity, use the arc tangent of the difference between the ratios obtained for any frame image and its previous frame image as the direction of the scanning feature vector of the any frame image, and use half the body length of the animal in the frame image as the modulus of the scanning feature vector of the frame image;

[0038] S222, clustering all scanned feature vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters;

[0039] S223, recording the mean of the scan feature vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of scan feature vectors of the cluster to the number of all scan feature vectors in the segmented video as the transition probability of the representative vector of the cluster;

[0040] S224: All representative vectors and their transition probabilities in the segmented video constitute a state transition model of the segmented video. Among them, h ′ 1.h ′ t are the 1st and tth representative vectors, Ph1 and Ph t are the transition probabilities of the 1st and tth representative vectors respectively.

[0041] Preferably, the method for determining the degree to which any particle in any frame image conforms to the animal movement trend is:

[0042] The degree to which the zth particle in the jth frame image conforms to the animal's movement trend is recorded as Q z ,

[0043]

[0044] Among them, Q z Indicates the degree to which the zth particle conforms to the animal's movement trend; D z represents the distance between the zth particle and the fitting point of the corresponding image in the fitting curve; exp represents the exponential function with the natural constant e as the base; M represents the size of the time window; g m 、g m-1 They respectively represent the curve distances between the corresponding fitting points on the fitting curve of the m-th frame and the m-1-th frame image in the time window of the j-th frame image and their previous frame image.

[0045] Preferably, the method for determining the predicted position of any representative pixel point in any frame image in the next frame image includes:

[0046] The coordinates of the predicted position of the bth representative pixel in the jth frame image in the j+1th frame image are marked as (x b ,y b ), Among them, Z represents the number of particles in the j+1 frame image, (x z ,y z ) is the coordinate of the zth particle;

[0047] Among them, W z represents the weight of the zth particle, W z =Q z ×P z ×exp(-E z );Q z Indicates the degree to which the zth particle conforms to the animal's movement trend; P z represents the transition probability of the predicted position of the zth particle; exp represents the exponential function with the natural constant e as the base; E z represents the distance between the zth particle and its predicted position.

[0048] Preferably, the method for tracking and recording the behavior of an animal comprises:

[0049] Obtain the activity of all segmented videos in the experimental video, count the maximum and minimum values of all the activity, and perform proportional mapping on the preset temperature range based on the maximum and minimum values, where the maximum activity corresponds to the highest temperature and the minimum activity corresponds to the lowest temperature, to generate a thermal image;

[0050] At the same time, trajectory curves are drawn for each segmented video of different behaviors, wherein the trajectory curves are obtained by connecting the pixel points representing the center of gravity of the animal in all frame images in each segmented video and performing curve fitting;

[0051] The obtained thermal images and all trajectory curves are used to track and record the behavior of the animals.

[0052] This application has at least the following beneficial effects:

[0053] This application takes into account that the purpose of animal behavior experiments is to analyze animal behavior, and animal behavior is the result of animal decision-making, that is, the animal's initiative. This application uses high-precision motion capture technology to track and monitor animal behavior, analyze the continuous animal movement trajectory characteristics under different behaviors, and at the same time identify the initiative in animal behavior characteristics. It tracks and records animal behavior during animal experiments, fully identifying the initiative of animal behavior and movement state trajectory during the experiment, which is conducive to improving the efficiency and effectiveness of the experimenter's analysis of the experimental process. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the technical solutions and advantages of the embodiments of the present application or the prior art, the following is a brief introduction to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0055] Figure 1 A flowchart of a method for tracking and recording behavior during an animal experiment provided in this application;

[0056] Figure 2 Schematic diagram of the animal experiment process;

[0057] Figure 3 Flowchart of the process for determining animal initiative;

[0058] Figure 4 Schematic diagram of walking, scanning, and standing in animal behavior;

[0059] Figure 5 Flowchart of the method for segmenting experimental videos;

[0060] Figure 6 This is a frame of image from an experimental video of an open field experiment;

[0061] Figure 7 Schematic diagram of representative pixels of the head, tail, center of gravity, and limbs of the animal area;

[0062] Figure 8 A flow chart of a method for determining the initiative of an animal in any segmented video;

[0063] Figure 9 Flowchart of a method for constructing a state transition model for segmented videos of animals walking or standing;

[0064] Figure 10 is the motion vector a of the j-th frame image j Schematic diagram of;

[0065] Figure 11 Flowchart of the method for building a state transition model for segmented videos of animal scanning behavior. DETAILED DESCRIPTION

[0066] To further illustrate the technical means and effects adopted by this application to achieve the intended invention objectives, the following, in conjunction with the accompanying drawings and preferred embodiments, describes in detail a method for tracking and recording behavior during animal experiments proposed in this application, including its specific implementation, structure, features, and effects. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. In addition, specific features, structures, or characteristics of one or more embodiments may be combined in any suitable form.

[0067] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0068] The following describes in detail a method for tracking and recording behavior during an animal experiment provided by the present application with reference to the accompanying drawings.

[0069] An embodiment of the present application provides a method for tracking and recording behavior during an animal experiment.

[0070] Specifically, the following method for tracking and recording behavior during animal experiments is provided. Figure 1 , the method comprises the following steps:

[0071] Step 1: Obtain experimental videos during animal experiments.

[0072] Set up the experimental scene and place the camera and light source directly above the experimental area so that the experimental scene can be fully monitored. Figure 2As shown. The animal is placed in the experimental scene, and the camera begins to record the experimental process. After the preset time, the experiment ends, the camera is turned off, and the experimental video of the animal experiment is transferred to the computer. In this embodiment, the setting of the experimental scene and the duration of the experiment are set by the implementer in combination with the purpose of the specific experiment. In one implementation of this embodiment, the duration of the experiment is set to half an hour, and the experimental scene is set as a cylindrical frame with a bottom diameter of 100 cm and a box height of 30 cm.

[0073] Step 2: Segment the experimental video based on the animal's behavioral characteristics, predict the animal's position in the segmented video, and determine the animal's initiative based on the difference between the predicted position and the actual position.

[0074] Animal behavior is a continuous decision-making process. The predicted position is based on behavioral inertia and statistical posterior probabilities, while the actual position is based on the sudden change in the decision. The difference between the predicted and actual positions can be used to determine the animal's active behavior. Active behavior is reflected in changes in the animal's posture and movement state. Therefore, experimental videos can be segmented based on the animal's posture and movement state. The difference between the predicted and actual positions can then be used to determine the region and active characteristics of each segment.

[0075] In this application, the process flow chart of determining the initiative of animals is shown in the attached Figure 3 As shown, specifically:

[0076] S1, obtaining representative pixel points of the animal region of each frame image in the experimental video, for analyzing the behavior of the animals in the animal region to segment the experimental video, and obtaining a plurality of segmented videos.

[0077] The animal's behavior mainly includes walking, scanning and standing, e.g. Figure 4 As shown, walking is the movement of stretching the body and moving quickly; scanning is the movement of only the head; and standing is the movement of standing still. This application aims to divide the video into several segments, each segment represents a behavior, and the characteristics that distinguish these behaviors are the posture and movement state of the animal.

[0078] The animal area in each frame of the image is marked. The animal's posture index can be obtained through the animal's body length and turning angle. The animal's movement state can be obtained through the change in the animal's position between frames, so that the animal's behavior can be segmented in the video.

[0079] In this application, the flow chart of the method for segmenting the experimental video is shown in the attached Figure 5 The specific steps are as follows:

[0080] First, obtain the animal region of each frame in the experimental video. Figure 6This is a frame of image from an experimental video of an open field experiment. Since there is often a large grayscale difference between the animal and the experimental environment, threshold segmentation can be performed using the Otsu method to obtain the animal area. The Otsu method is a well-known technology and will not be described in detail.

[0081] S11, obtaining representative pixel points of the head, tail, center of gravity and limbs of the animal region in each frame of the image;

[0082] like Figure 7 As shown, first, the representative pixel points of the head, tail, center of gravity and limbs of the animal area in each frame image are manually labeled, and then the animal areas on all frame images are trained through a neural network to obtain the representative pixel points of the head, tail, center of gravity and limbs of the animal area in each frame image.

[0083] S12, record the straight-line distance from the head to the tail as the animal's body length l, and obtain the angles α and β between the two limbs on both sides of the body and the line connecting the center of gravity;

[0084] S13, judging the behavior of the animal in each frame of the image based on the animal's body length l, the angles α and β, and the distance between the heads in adjacent frames of the image;

[0085] Preferably, in this embodiment, the method for determining the behavior of the animal in each frame of image is:

[0086] The index of the i-th behavior in each frame is marked as L i , where L1 represents the walking index, L2 represents the scanning index, and L3 represents the standing index;

[0087]

[0088] Among them, norm represents the normalization function, d head,1 d head,2 Respectively represent the distance between the head representative pixels of the animal area in the current frame, the previous frame, and the next frame.

[0089] Furthermore, the mean index value of all behaviors in each frame image is calculated, and the behavior with the largest difference between the index in each frame image and the mean index value is taken as the behavior of the animal in each frame image.

[0090] S14, merging consecutive frame images with the same behavior and recording them as a segmented video.

[0091] At this point, several segmented videos can be obtained through the method in step S1.

[0092] S2, for any segmented video, analyze the changing characteristics of the representative pixels between all adjacent frame images reflecting the animal's behavior, determine the predicted position of any representative pixel in each frame image in the next frame image; and determine the initiative of the animal in any segmented video based on the difference between the actual positions of the representative pixels.

[0093] Particle filtering approximates the probability density distribution by finding a set of random samples (particles) that propagate through the state space, replacing integral operations with the sample mean. By statistically analyzing the changes in the animal's state in segmented videos of each behavior, the state transition probability of the animal can be obtained.

[0094] Different behaviors have different state transitions, resulting in different state transition probabilities, which require separate analysis. Furthermore, animal movements are continuous, so when performing particle filtering, the particle arrangement and the reference level of different particles in mean calculations need to be adjusted.

[0095] In this application, the flow chart of the method for determining the initiative of an animal in any segmented video is shown in the attached Figure 8 The specific steps are as follows:

[0096] S21, when any segmented video is a segmented video of an animal walking or standing, analyze the change distribution characteristics of the pixel points representing the animal's head between any frame image and the two frames before it, and construct a state transition model for the segmented video.

[0097] In this application, the flow chart of the state transition model construction method of the segmented video of the animal walking or standing behavior is shown in the attached figure. Figure 9 As shown, specifically:

[0098] S211, using the direction of the straight line connecting the pixel points representing the animal's head in the previous frame of any image and the previous frame of the previous frame as the polar axis of the polar coordinate;

[0099] S212, taking a vector in polar coordinates of a straight line connecting a previous frame image of any frame image and a pixel point representing the animal's head in any frame image as a motion vector of the any frame image;

[0100] In one embodiment of the present application, the motion vector a of the j-th frame image j The schematic diagram is as attached Figure 10 As shown. For the j-th frame image, the direction of the straight line connecting the representative pixel points of the animal's head in the j-2-th frame image and the j-1-th frame image is recorded as the polar axis of the polar coordinates in the j-th frame image. Based on the polar coordinates, the vector formed by the straight line connecting the representative pixel points of the animal's head in the j-1-th frame image and the j-th frame image is obtained as the motion vector a of the j-th frame image. jThe motion vector reflects the change of the animal's behavioral state, and statistics on it can be used to obtain the state transition probability representing the behavior of the segmented video.

[0101] It should be noted that the motion vectors of the first and second frames of any segmented video are not calculated.

[0102] S213, clustering all motion vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters;

[0103] It should be noted that when using the region growing algorithm for clustering, the first motion vector is used as the growth seed point; according to the acquisition time sequence of all frame images, the cosine similarity between motion vectors is greater than 0.8 as the growth condition; if the growth condition is not met, any motion vector that has not been grown is used as the growth seed point until all motion vectors are grown. The region growing algorithm is a well-known technology and will not be described in detail.

[0104] S214, taking the mean of all motion vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of motion vectors of the cluster to the number of all motion vectors in the segmented video, and using this ratio as the transition probability of the representative vector of the cluster;

[0105] S215: All representative vectors and their corresponding transition probabilities constitute a state transition model for the segmented video. Among them, a ′ 1.a ′ t are the 1st and tth representative vectors, Pa1 and Pa t are the transition probabilities of the 1st and tth representative vectors respectively.

[0106] S22, when any segmented video is a segmented video of an animal's scanning behavior, analyze the change distribution characteristics of the angle between the pixel points representing the animal's two limbs and the line connecting the center of gravity between any frame image and the two frames before it, and obtain a state transition model of the segmented video.

[0107] In this application, the flow chart of the state transition model construction method of the segmented video of the animal scanning behavior is shown in the attached figure. Figure 11 As shown, specifically:

[0108] S221, for any frame image and its previous frame image, respectively obtain the ratios of the angles α and β between the two limbs on both sides of the body and the line connecting the center of gravity, use the arc tangent of the difference between the ratios obtained for any frame image and its previous frame image as the direction of the scanning feature vector of the any frame image, and use half the body length of the animal in the frame image as the modulus of the scanning feature vector of the frame image;

[0109] In a specific embodiment, taking the jth frame image as an example, the ratio of the angles α and β between the two limbs on both sides of the body of the animal region and the center of gravity in the jth frame image is obtained. and in the j-1 frame image The difference between them is recorded as the scanning change feature of the j-th frame image: Scan change characteristics h j It reflects the changes in the animal's state.

[0110] At the same time, since the body length of the mouse remains unchanged and the length between the head and the center of gravity of the body remains basically unchanged, the state transition model only concerns the change of its scanning angle, but has nothing to do with the body length. Accordingly, half of the animal's body length is used as the modulus of the scanning feature vector of the j-th frame image, and the arc tangent value of the scanning change feature of the j-th frame image is used as the direction of the scanning feature vector of the j-th frame image.

[0111] S222, clustering all scanned feature vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters;

[0112] The clustering method is the same as all the motion vector clustering methods mentioned above.

[0113] S223, recording the mean of the scan feature vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of scan feature vectors of the cluster to the number of all scan feature vectors in the segmented video as the transition probability of the representative vector of the cluster;

[0114] S224: All representative vectors and their transition probabilities in the segmented video constitute a state transition model of the segmented video. Among them, h ′ 1.h ′ t are the 1st and tth representative vectors, Ph1 and Ph t are the transition probabilities of the 1st and tth representative vectors respectively.

[0115] S23, for any frame image in any segmented video, use half the body length of the animal in any frame image as the radius, draw a circle with the representative pixel point of the animal's head as the center, and evenly place a preset number of particles in the circle; perform curve fitting on the representative pixel point of the animal's head in the time window of any frame image, and obtain the distance between any particle and the fitting point of the corresponding image in the fitting curve; and combine the curve distance between the corresponding fitting points on the fitting curve between all adjacent frame images in the time window of any frame image to determine the degree to which any particle in the said any frame image conforms to the animal's movement trend.

[0116] Based on the state transition model of all animal behaviors, the animal state in any frame image of any segmented video can be predicted. Since the state transition model is based on statistics, the frequency of event occurrence is used as the transition probability. However, the movement of the animal in consecutive frames is not only related to the previous frame, but also the movement trend of the animal should be considered.

[0117] In a specific embodiment, taking the j-th frame image in any segmented video as an example, first obtain the body length of the animal in the j-th frame image, take half of the animal's body length as the radius, and draw a circle with the pixel representing the head of the animal in the j-th frame image as the center of the circle. The circle is used as the distribution space of the particles, and a preset number of particles are evenly placed. In this embodiment, the preset number Z is set to 100.

[0118] Furthermore, a time window for the j-th frame image is set. In this embodiment, the size M of the time window is set to 9, that is, the 4 frames before and 4 frames after the j-th frame image constitute the time window of the j-th frame image. If the number of frames before or after is insufficient, it is supplemented on the other side.

[0119] For any particle, the more it conforms to the movement pattern of the animal's head in the time window, the more the particle conforms to the animal's movement trend. From this, the degree to which any particle conforms to the animal's movement trend is calculated.

[0120] The representative pixel points of the animal's head in the time window of the j-th frame image are fitted with a polynomial using the least squares method to fit a curve. The distance D between any particle and the fitting point of the corresponding image in the fitting curve is obtained, and the degree to which each particle conforms to the animal's movement trend is calculated. Taking the z-th particle in the j-th frame image as an example, the degree to which the z-th particle in the j-th frame image conforms to the animal's movement trend is recorded as Q z ,

[0121] Among them, Q z Indicates the degree to which the zth particle conforms to the animal's movement trend; D z represents the distance between the zth particle and the fitting point of the corresponding image in the fitting curve; exp represents the exponential function with the natural constant e as the base; M represents the size of the time window; g m 、g m-1 They represent the curve distances between the corresponding fitting points of the m-th and m-1-th frames in the time window of the j-th frame and their previous frames on the fitting curve, respectively. Among them, m-1>1 needs to be satisfied, so we can only start from 3). Represents the difference in animal motion across consecutive frames. The larger the value, the less the particle conforms to the laws of animal motion. The more a particle conforms to the animal's motion trends and the more its position conforms to the state in the state transition model, the greater its weight.

[0122] S24, taking any representative pixel point in any frame image as the starting point, draw all representative vectors in the state transition model of the segmented video in which it is located, and obtain the position of the end point of the representative vector; obtain the minimum value of the distance between any particle and the position of the end point of all representative vectors, use the end point position corresponding to the minimum value as the predicted position of the particle, and record the minimum value as the distance between the particle and the predicted position; based on the degree to which any particle in any frame image conforms to the animal movement trend, the distance between any particle and its predicted position, the transition probability of the predicted position of any particle, and the coordinates of any particle, determine the predicted position of any representative pixel point in any frame image in the next frame image.

[0123] Taking any segmented video as an example, for the j-th frame image, predict the position of the animal in the j+1-th frame image:

[0124] First, obtain all representative pixel points of the animal in the j-th frame image; take any representative pixel point as an example, draw a representative vector in the image with the representative pixel point as the starting point, and mark the position of the end point of the representative vector; obtain the distance between any particle and the position of the end point of any representative vector, and take the minimum value, and use the end point position corresponding to the minimum value as the predicted position of the particle, and record the minimum value as the distance E between the particle and the predicted position z .

[0125] It should be understood that the closer the particle is to the predicted position, the greater the weight of the particle; in addition, the greater the transfer probability corresponding to the predicted position and the more the particle conforms to the law of animal movement, the greater the weight of the particle. The weight of the particle is calculated in this way, with the weight W of the zth particle z For example, the expression is: W z =Q z ×P z ×exp(-E z );

[0126] Among them, W z is the weight of the zth particle; Q z Indicates the degree to which the zth particle conforms to the animal's movement trend; P z represents the transition probability of the predicted position of the zth particle; exp represents the exponential function with the natural constant e as the base; E z represents the distance between the zth particle and its predicted position.

[0127] Furthermore, the weights of all particles are weighted to obtain the predicted position of any representative pixel point, so as to convert the coordinates (x b ,y b ) as an example, the expression is: Among them, (x b ,yb ) is the predicted position coordinate of the bth representative pixel in the j+1th frame image, Z represents the number of particles in the j+1th frame image, W z is the weight of the zth particle; (x z ,y z ) is the coordinate of the zth particle.

[0128] S25, taking the accumulated distances between the predicted positions and the actual positions of all representative pixels in all frame images in any segmented video as the initiative of the animal in any segmented video.

[0129] For any segmented video, the greater the difference between the predicted position of the animal's representative pixel in each frame and the actual position, the less the animal behaves according to habit and inertia, and the more proactive it is. The expression for calculating the proactiveness of the animal in any segmented video is:

[0130] Where R represents the activity of the animal in any segmented video; J represents the number of image frames in the segmented video; B represents the number of representative pixels in each frame; n j,b Represents the distance between the bth representative pixel in the jth frame image and its predicted position.

[0131] Step 3: Based on the animal's initiative in all segments of the experimental video and the change curve of the animal's center of gravity in all segments, the animal's behavior is tracked and recorded.

[0132] According to step 2 above, the activity of all segments in the experimental video is obtained, and the maximum and minimum values of all such activity are counted. Based on the maximum and minimum values, a proportional mapping is performed on the preset temperature range, so that the maximum activity corresponds to the highest temperature and the minimum activity corresponds to the lowest temperature, thereby generating a thermal image. In this embodiment, the preset temperature range is [-100, 100], but the specific value can be set by the implementer.

[0133] It should be understood that, in a thermal image, locations with higher temperatures represent greater animal initiative, and locations with lower temperatures represent weaker animal initiative.

[0134] At the same time, trajectory curves are drawn for each segmented video of different behaviors, wherein the trajectory curves are obtained by connecting the pixel points representing the center of gravity of the animal in all frames of each segmented video and performing curve fitting. In this embodiment, the least squares method is used for curve fitting.

[0135] The obtained thermal images and all trajectory curves are used to track and record the behavior of the animals.

[0136] It should be noted that the higher the temperature in the thermal image, the more active the behavior of the animal at this time is. Therefore, when analyzing the trajectory curves of different segmented videos, appropriate consideration should be given to the prediction of future behavior trajectories under the condition that the animal has active consciousness; the lower the temperature, the more likely the behavior of the animal at this time is an inertial action. Therefore, when analyzing the trajectory curves of different segmented videos, appropriate consideration should be given to the prediction of future behavior trajectories under the inertial state of the animal.

[0137] It should be understood that thermal images and trajectory curves can be used to reflect the analysis focus during animal behavior tracking. At the same time, during repeated experiments, by recording changes in parameters, the experimental environment and animal status can be analyzed, which can ensure that the experimenter can efficiently track and analyze animal behavior.

[0138] The various embodiments in this application are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments.

[0139] It should be noted that, unless otherwise specified and limited, terms such as "include", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a circuit structure, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such article or device. In the absence of further restrictions, the phrase "including a ..." defines an element, does not exclude the presence of other identical elements in the article or device including the element. In addition, the term "and\or" used herein includes any and all combinations of one or more related listed items.

[0140] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of the present application and include common knowledge or customary techniques in the art not invented herein.

[0141] It will be understood that the present application is not limited to the exact construction that has been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof.

Claims

1. A method for tracking and recording behavior during animal experiments, characterized in that: The method comprises the following steps: Step 1: Obtain experimental videos during animal experiments; Step 2: Segment the experimental video based on the animal's behavioral characteristics, predict the animal's position in the segmented video, and determine the animal's initiative based on the difference between the predicted position and the actual position; specifically: S1, obtaining representative pixel points of the animal region of each frame of the experimental video, for analyzing the behavior of the animals in the animal region to segment the experimental video, and obtaining a plurality of segmented videos; S2, for any segmented video, analyzes the changing characteristics of the representative pixels between all adjacent frames reflecting the animal's behavior, determines the predicted position of any representative pixel in each frame in the next frame; and determines the initiative of the animal in any segmented video based on the difference between the actual positions of the representative pixels; Step 3: Track and record the animal's behavior based on the animal's initiative in all segments of the experimental video and the change curve of the animal's center of gravity in all segments of the video; The method for segmenting the experimental video includes: S11, obtaining representative pixel points of the head, tail, center of gravity and limbs of the animal region in each frame of the image; S12, the straight-line distance from the head to the tail is recorded as the animal's body length , respectively obtain the angle between the two limbs on both sides of the body and the center of gravity 、 ; S13, combined with animal body length , angle 、 and the distance between heads in adjacent frames to determine the behavior of the animal in each frame; S14, merging consecutive frame images with the same behavior and recording them as a segmented video.

2. The method for tracking and recording behavior during an animal experiment as claimed in claim 1, wherein: In step S1, the animal region is obtained by applying the Otsu method to each frame of the image.

3. The method for tracking and recording behavior during an animal experiment as claimed in claim 1, wherein: In step S13, the method for determining the behavior of the animal in each frame of image is: The index of the i-th behavior in each frame image is marked as ,in, Represents walking index, Represents the scanning index, Stands for standing indicator; Among them, norm represents the normalization function, 、 Respectively represent the distance between the head representative pixel points of the animal area in the current frame, the previous frame, and the next frame; The mean index of all behaviors in each frame of image was calculated, and the behavior with the largest difference between the index in each frame of image and the mean index was taken as the behavior of the animal in each frame of image.

4. The method for tracking and recording behavior during an animal experiment as claimed in claim 3, wherein: In step S2, the method for determining the initiative of the animal in any segmented video includes: S21, when any segmented video is a segmented video of an animal walking or standing, analyzing the change distribution characteristics of the pixel points representing the animal's head between any frame image and the two frames before it, and constructing a state transition model for the segmented video; Wherein, the state transition model is composed of several representative vectors and their corresponding transition probabilities; S22, when any segmented video is a segmented video of an animal's scanning behavior, analyzing the distribution characteristics of changes in the angles between the pixel points representing the animal's two limbs and the line connecting the center of gravity between any frame image and the two frames before it, to obtain a state transition model for the segmented video; S23, for any frame image in any segmented video, using half the body length of the animal in any frame image as the radius, drawing a circle with the pixel representing the animal's head as the center, and evenly placing a preset number of particles in the circle; performing curve fitting on the pixel representing the animal's head within the time window of any frame image, obtaining the distance between any particle and the fitting point of the corresponding image on the fitting curve; and combining the curve distances between corresponding fitting points on the fitting curve between all adjacent frame images within the time window of any frame image, determining the degree to which any particle in any frame image conforms to the animal's movement trend; S24, taking any representative pixel point in any frame image as a starting point, plotting all representative vectors in the state transition model of the segmented video in which it is located, and obtaining the positions of the end points of the representative vectors; obtaining the minimum value of the distance between any particle and the positions of the end points of all representative vectors, using the end point position corresponding to the minimum value as the predicted position of the particle, and recording the minimum value as the distance between the particle and the predicted position; determining the predicted position of any representative pixel point in any frame image in the next frame image based on the degree to which any particle in any frame image conforms to the animal movement trend, the distance between any particle and its predicted position, the transition probability of any particle's predicted position, and the coordinates of any particle; S25, taking the accumulated distances between the predicted positions and the actual positions of all representative pixels in all frame images in any segmented video as the initiative of the animal in any segmented video.

5. The method for tracking and recording behavior during an animal experiment as claimed in claim 4, wherein: The method for constructing the state transition model in step S21 includes: S211, using the direction of the straight line connecting the pixel points representing the animal's head in the previous frame of any image and the previous frame of the previous frame as the polar axis of the polar coordinate; S212, taking a vector in polar coordinates of a straight line connecting a previous frame image of any frame image and a pixel point representing the animal's head in any frame image as a motion vector of the any frame image; S213, clustering all motion vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters; S214, taking the mean of all motion vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of motion vectors of the cluster to the number of all motion vectors in the segmented video, and using this ratio as the transition probability of the representative vector of the cluster; S215: All representative vectors and their corresponding transition probabilities constitute a state transition model for the segmented video. ;in, 、 are the 1st and tth representative vectors respectively, 、 are the transition probabilities of the 1st and tth representative vectors respectively.

6. The method for tracking and recording behavior during an animal experiment as claimed in claim 4, wherein: The method for constructing the state transition model in step S22 includes: S221, for any frame image and its previous frame image, respectively obtain the angles between the two limbs on both sides of the body and the line connecting the center of gravity 、 The arc tangent value of the difference between the ratio of any frame image and its previous frame image is used as the direction of the scanning feature vector of any frame image, and half of the body length of the animal in the frame image is used as the modulus of the scanning feature vector of the frame image; S222, clustering all scanned feature vectors of the segmented video using a region growing algorithm to obtain a plurality of clusters; S223, recording the mean of the scan feature vectors of each cluster as the representative vector of the cluster, obtaining the ratio of the number of scan feature vectors of the cluster to the number of all scan feature vectors in the segmented video as the transition probability of the representative vector of the cluster; S224: All representative vectors and their transition probabilities in the segmented video constitute a state transition model of the segmented video. ;in, 、 are the 1st and tth representative vectors respectively, 、 are the transition probabilities of the 1st and tth representative vectors respectively.

7. The method for tracking and recording behavior during an animal experiment as claimed in claim 4, wherein: The method for determining the degree to which any particle in any frame image conforms to the animal movement trend is as follows: The degree to which the zth particle in the jth frame image conforms to the animal's movement trend is recorded as , ; in, It indicates the degree to which the zth particle conforms to the animal's movement trend; represents the distance between the zth particle and the fitting point of the corresponding image in the fitting curve; exp represents the exponential function with the natural constant e as the base; M represents the size of the time window; 、 They respectively represent the curve distances between the corresponding fitting points on the fitting curve of the m-th frame and the m-1-th frame image in the time window of the j-th frame image and their previous frame image.

8. The method for tracking and recording behavior during an animal experiment as claimed in claim 4, wherein: The method for determining the predicted position of any representative pixel point in any frame image in the next frame image includes: The coordinates of the predicted position of the bth representative pixel in the jth frame image in the j+1th frame image are marked as , ; Where Z represents the number of particles in the j+1th frame image, is the coordinate of the zth particle; in, represents the weight of the z-th particle, ; It indicates the degree to which the zth particle conforms to the animal's movement trend; represents the transition probability of the predicted position of the zth particle; exp represents the exponential function with the natural constant e as the base; represents the distance between the zth particle and its predicted position.

9. The method for tracking and recording behavior during an animal experiment as claimed in claim 1, wherein: The method for tracking and recording the behavior of the animal comprises: Obtain the activity of all segmented videos in the experimental video, count the maximum and minimum values of all the activity, and perform proportional mapping on the preset temperature range based on the maximum and minimum values, where the maximum activity corresponds to the highest temperature and the minimum activity corresponds to the lowest temperature, to generate a thermal image; At the same time, trajectory curves are drawn for each segmented video of different behaviors, wherein the trajectory curves are obtained by connecting the pixel points representing the center of gravity of the animal in all frame images in each segmented video and performing curve fitting; The obtained thermal images and all trajectory curves are used to track and record the behavior of the animals.

Images