A method for analyzing rodent behavior based on event cameras

Abstract

The present invention discloses a rodent behavior analysis method based on an event camera, which relates to the technical field of animal behavior analysis, and includes the following steps: step one, setting up the experimental device and recording the animal behavior experiment process; step two, analyzing the collected data; step three, counting the data, using python code to count the event stream data of a single animal; step four, performing visual analysis, using a variety of visualization methods and inspection methods to compare the similarities and differences in activity between different animals and different groups of animals. The present invention adopts the above-mentioned rodent behavior analysis method based on an event camera, and the present invention objectively eliminates human intervention processes such as manual counting or threshold selection; the use of an event camera saves time compared to manually timing the immobility time; the event camera has a higher time resolution and low latency, and the gradual change process of the strength of the experimental animal activity can be seen at a high time resolution.

Description

Technical Field

[0001] The present invention relates to the technical field of animal behavior analysis, and in particular to a rodent behavior analysis method based on an event camera. Background Art

[0002] Animal behavior experiments (forced swimming, tail suspension test, open field, elevated cross and light-dark box, etc.) are important research methods in the field of neuroscience, and are also commonly used experiments in emotion research and antidepressant drug screening. Although the analysis system on the market currently meets the requirements of semi-automated and high-throughput analysis, the analysis of the results is overly dependent on the clarity of the behavioral video, and the accuracy of the results cannot be reliably guaranteed, which seriously restricts the testing conditions in the laboratory. In addition, the existing animal behavior experimental methods use the immobility time during the time period to be analyzed as the standard for measuring the emotional state of rodents, which is subjective to a certain extent. Different experimenters or even the same experimenter at different times will have different counts of the immobility time of rodents, which affects the accuracy and repeatability of the experimental results.

[0003] Forced swimming and tail suspension tests are classic experiments for evaluating depressive-like behaviors. There are three mainstream analysis methods for forced swimming and tail suspension tests. One is to manually count the immobility time. Currently, a large number of researchers still use manual timing methods. One is the threshold-based method represented by the above-mentioned commercial software companies. Most of them use a frame difference-based detection algorithm to determine whether the animal is in an immobile state through user-set thresholds, and then count the total immobility time. Another type is to detect key points of rodents through key point detection technology, and use the position information of key points to calculate the speed and acceleration of rodents, thereby identifying the immobility state of rodents and counting the immobility time of rodents. These methods are based on identifying the immobility state of rodents, counting the immobility time of rodents in forced swimming and tail suspension tests, and using the immobility time as a standard for measuring the emotional state of rodents. The first two methods are obviously subjective and require users to have relevant behavioral experimental analysis experience. The third method based on deep learning for key point recognition is easily affected by video quality and shooting angle. Monocular camera shooting will cause obvious occlusion of the animal's legs, which is not conducive to the accurate recognition of key points, thus affecting the next step of statistics on immobility time. Summary of the invention

[0004] The purpose of the present invention is to provide a rodent behavior analysis method based on an event camera, to provide an accurate, automatic and unbiased behavior analysis method to replace the analysis of rodent behavior based on ordinary videos by traditional video cameras. The method is simple to operate and can be used for behavioral experimental analyses such as forced swimming, tail suspension tests, open field, elevated cross and light and dark boxes, and can be widely promoted and applied in animal behavior experimental analysis systems of scientific research institutes.

[0005] To achieve the above object, the present invention provides a rodent behavior analysis method based on an event camera, comprising the following steps:

[0006] Step 1: Set up the experimental device and record the animal behavior experiment process;

[0007] Step 2: Process the collected data;

[0008] Step 3: Count the data and use Python code to count the event stream data of a single animal;

[0009] Step 4: Conduct visual analysis and use a variety of visualization methods and test methods to compare the similarities and differences in activity between different animals and different groups of animals.

[0010] Preferably, step two includes data denoising, data transfer and animal segmentation.

[0011] Preferably, during the data denoising process, the noise is filtered out by a noise filter in the event camera supporting software.

[0012] Preferably, k-means++ is used to cluster events in animal segmentation, and the number of cluster centers is the number of experimental animals.

[0013] Preferably, the animal segmentation comprises the following steps:

[0014] Step 1: Detect and divide animals in different areas, and regard each event as a sample point;

[0015] Step 2: The K-means++ algorithm randomly selects an event point as the initial cluster center;

[0016] Step 3: Calculate the shortest distance between each event and the current center and the probability of each event point being selected as the next cluster center;

[0017] Step 4: Determine k cluster centers, divide all event points into the nearest centroids and iterate until the termination condition;

[0018] Step 5: Use the midline of the lines connecting adjacent centroids as the dividing line to obtain the activity areas of different animals.

[0019] Preferably, events within 6 minutes of an animal are randomly sampled and 10% of the total data is used for cluster segmentation.

[0020] Preferably, in step 4, heat maps and bar graphs are used to visualize the results, and t-tests and Kolmogorov-Smirnow tests are used to perform hypothesis testing.

[0021] Preferably, the experimental device includes a laptop computer, an animal behavior experimental device, an event camera and data acquisition software.

[0022] Therefore, the present invention adopts a rodent behavior analysis method based on an event camera with the above structure, which has the following beneficial effects:

[0023] (1) The present invention objectively eliminates human intervention processes such as manual counting or threshold selection, making the analysis process more objective.

[0024] (2) The present invention uses an event camera, which saves time compared to manually timing the motionless time.

[0025] (3) The present invention uses heat maps, cumulative probability density maps, etc. to visually show the change process of the intensity of experimental animal activities over time and the distribution of the number of events per second, which is highly intuitive.

[0026] (4) Event cameras have higher temporal resolution and lower latency than ordinary cameras. They can observe the gradual changes in the intensity of experimental animals' activities at high temporal resolution, replacing the traditional binary division of "moving" and "still". They have the advantage of high precision, and the software supporting event cameras is easy to use and has low learning difficulty.

[0027] (5) The event camera has low requirements for the experimental environment and strong adaptability. The signal is stable in low light or high dynamic environment. Therefore, it has low requirements for the environment for animal experiments and is highly practical.

[0028] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 This is an analysis system diagram of a rodent behavior analysis method based on an event camera according to the present invention;

[0030] Figure 2 It is a diagram of the tail suspension test and forced swimming test apparatus of the present invention;

[0031] Figure 3 A heat map of the number of events in the forced swimming test in an embodiment of the present invention;

[0032] Figure 4A graph showing the total number of events in a forced swimming test according to an embodiment of the present invention;

[0033] Figure 5 This is a graph showing the change in the number of events over time in a forced swimming test in an embodiment of the present invention;

[0034] Figure 6 Graph showing the cumulative probability distribution of the number of events in the forced swimming test in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The technical solution of the present invention is further described below through the accompanying drawings and embodiments.

[0036] Unless otherwise defined, the technical terms or scientific terms used in the present invention should be understood by people with ordinary skills in the field to which the present invention belongs. The words "first", "second" and similar words used in the present invention do not indicate any order, quantity or importance, but are only used to distinguish different components. "Include" or "comprise" and similar words mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. "Connect" or "connected" and similar words are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0037] Example

[0038] See also Figure 1-6 The present invention provides a rodent behavior analysis method based on an event camera, comprising the following steps:

[0039] Step 1: Follow Figure 2 After setting up the experimental device, place the animals into the behavioral experimental device and immediately record the animal behavioral experimental process.

[0040] Step 2: Analyze the collected data. Because the structure of the event camera is very sensitive to changes in ambient brightness, the output asynchronous event stream contains a lot of noise interference, so the data is first denoised and the noise is filtered out using the noise filter in the event camera supporting software.

[0041] Convert the data in aedat4 format to csv format for easy processing. Use the module of Dynamic Vision Viewer software to export the captured aedat4 format video to csv event stream data.

[0042] When multiple animals are collected at the same time, k-means++ is used to cluster the events. The number of cluster centers is the number of experimental animals. The animals are segmented at the midpoint of the line connecting the cluster centers to obtain the experimental data of a single animal.

[0043] Step 3: Count the data and use Python code to count the event stream data of a single animal. Starting from the moment the rodent is placed, count the number of events per unit time within 6 minutes, and use the number of events per second of the rodent in the last 4 minutes to reflect the activity of the rodent. Use the average number of events per second of a single animal in 4 minutes to reflect the overall depression level of the animal.

[0044] Step 4: Perform visual analysis, using a variety of visualization methods and test methods to compare the similarities and differences in activity between different animals and different groups. Use a heat map to show the number of events per unit time of a group of animals over time, and intuitively show the gradual change of the intensity of animal activity; use the average number of events per second within 4 minutes as one of the indicators to measure the overall depression level of animals, and perform an inter-group t-test on this indicator to compare whether there are differences between groups; perform a Kolmogorov-Smirnov test (KS test) on the number of events per second within 4 minutes between different groups of animals to compare whether there are differences in the distribution of events between groups over a period of time.

[0045] Animal segmentation includes the following steps:

[0046] Step 1: Detect and divide the areas of different observed animals, use the location information of event data, regard each event as a sample point, and use K-means++ to cluster events within a period of time.

[0047] Step 2: Based on the number set by the user, the K-means++ algorithm randomly selects an event point as the initial cluster center.

[0048] Step 3: Calculate the shortest distance between each event and the current center, calculate the probability of each event point being selected as the next cluster center, and select the corresponding event point as the next center based on the probability.

[0049] Step 4: Determine k cluster centers, divide all event points into the nearest centroids and iterate to the termination condition.

[0050] Step 5: Use the midline of the lines connecting adjacent centroids as the dividing line to obtain the activity areas of different animals.

[0051] To increase computing speed and improve analysis efficiency, events within 6 minutes of the animals were randomly sampled, and 10% of all data were used for clustering.

[0052] The overall structure is as follows Figure 1 As shown in the figure, it includes four parts: a laptop, an animal behavior experimental device, an event camera, and data acquisition software, which can be divided into a hardware system and a software system. The hardware system mainly consists of three parts: a shooting device event camera, a behavioral experimental device (glass cylinder or hanging tail box), and a laptop. Use a data transmission cable to connect the event camera and the laptop, and use a tripod and other equipment to adjust the height and viewing angle of the event camera to ensure that the whole body activities of the animal in the behavioral experiment can be captured and recorded in real time. The event camera DAVIS346 is used to reduce the difficulty of algorithm development and reduce the complexity of the algorithm by taking advantage of its elimination of static background. Thanks to the advantage of the high dynamic range of the event camera, the system has no special requirements for the light intensity of the experimental environment, and it is sufficient to keep the light uniform and consistent during the experiment. In order to reduce the difficulty of experimenters to get started and facilitate the promotion of the system, the software system uses the Dynamic VisionViewer software supporting the event camera for data acquisition, denoising, and visualization. The captured raw data is saved in aedat4 format, and the data can be exported to csv format using the export module provided by the software. The performance requirement for the laptop is that it can install the corresponding software of the event camera. There is no GPU requirement. Depending on the number of animals photographed, at least 2G of storage space must be provided for the raw data recorded for each animal.

[0053] Taking the forced swimming and tail suspension tests as examples, the number of events per unit time is used to reflect the activity intensity of the experimental animals. Arrange the experimental device to ensure that the field of view of the event camera covers the entire behavioral experimental area, ensure that the event camera and computer are connected properly, open the video recording software, and select a video storage location with sufficient space.

[0054] Take the event camera analysis of the rodent forced swim test as an example:

[0055] Construction of depression animal model: 18 adult male C57BL / 6 mice were randomly divided into three groups. The first group was the intraperitoneal injection of lipopolysaccharide group (LPS group), with intraperitoneal injection of LPS 300ug / kg every day for 21 days; the second group was the chronic restraint stress group (CRS group), with restraint for 4.5h every day for 21 days; the third group was the control group without treatment.

[0056] Experimental device setup and behavioral experimental records: The forced swimming test data of the above 18 animals were collected. The swimming device uses a glass cylinder with a height of 25 cm and a diameter of 10 cm. The bottle is filled with about 16 cm of water and the water temperature is kept at about 25 degrees Celsius. In order to prevent the animal from touching the bottom with its hind legs, the depth of the water cannot be too shallow. The animal behavior is videotaped from the side and recorded using an event camera (DAVIS346, 346×260 pixels; USB3.0; iniVation). After the animal is placed in the water, the Record key is used to record immediately. After the experimental animal has been forced to swim for 6 minutes, the video recording is stopped, the experimental animal is taken out of the water, wiped dry with a paper towel, and put back in the cage. After each animal experiment, clean water should be replaced to avoid feces residue. If two animals are tested at the same time, a black acrylic plate is placed between the animals to prevent the animals from seeing each other.

[0057] Data processing and analysis: After all experiments are completed, the data captured by the event camera are denoised according to the actual situation, and the denoised data are exported to csv format, including the time, location and polarity of all event points. Ignoring the difference in positive and negative polarity, the number of events per second is counted from the moment the experimental animals are placed in the water to obtain the number of events for all animals within 6 minutes. The number of events for each group of experimental animals in the time period to be analyzed is visualized and statistically analyzed using tools familiar to researchers.

[0058] The experimental results were visualized using heat maps, cumulative probability density maps, etc. According to the heat map of the number of events, the number of events in the control group mice was greater than that in the depressed group mice. Figure 3 This is a heat map of the number of events for each mouse within 6 minutes. In the heat map, the animals in the depression group are darker in color than those in the control group, which means that the number of events per unit time is smaller and the activity is weaker. Figure 4 In order to use the total number of events of each mouse in the last 4 minutes to represent the overall activity intensity of the mouse, a t-test was performed to find that there was a significant difference between the mice in the depression group and the mice in the control group. The mice in the control group had more events in the last 4 minutes and were more active. For each group of mice, the number of events at the same time was averaged, which can be used as follows: Figure 5 The line graph shown plots the number of events in each group of mice. Figure 5 It can also be seen that the control mice were generally more active than the depressed mice in the last 4 minutes. Figure 6 The cumulative probability density plot of the mouse events was used to view the distribution of the number of mouse events, and the KS test was used to compare the differences between the groups. The experiment found that there was a significant difference between the control group and the experimental group, while there was no significant difference between the two experimental groups.

[0059] Therefore, the present invention adopts the above-mentioned rodent behavior analysis method based on event camera. The present invention objectively eliminates human intervention processes such as manual counting or threshold selection, making the analysis process more objective; the present invention uses event camera compared to manual timing of immobility time, which saves time; the present invention uses heat map, cumulative probability density map, etc. to intuitively see the change process of experimental animal activity strength over time and the distribution of event number per second, which is highly intuitive; event camera has higher time resolution and low latency than ordinary camera, and can see the gradual change process of experimental animal activity strength at high time resolution, replacing the traditional binary division method of "moving" and "immobile", and has the advantage of high precision, and the software matching the event camera is easy to use with low difficulty; it has low requirements for experimental environment, the event camera has strong adaptability, and the signal is stable in low light or high dynamic environment, so the environmental requirements for animal experiments are low and it is highly practical.

[0060] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

Claims

1. A rodent behavior analysis method based on event camera, characterized in that: The following steps are involved: Step 1: Set up the experimental device and record the animal behavior experiment process; Step 2: Process the collected data, including data denoising, data transfer and animal segmentation; Because the structure of the event camera is very sensitive to changes in ambient brightness, the output asynchronous event stream contains a lot of noise interference. Therefore, the data is first denoised and the noise is filtered out using the noise filter in the event camera supporting software. Convert the data in aedat4 format into a csv format that is easy to process, and use the module of Dynamic Vision Viewer software to export the captured aedat4 format video into csv event stream data; When multiple animals were collected at the same time, k-means++ was used to cluster the events. The number of cluster centers was the number of experimental animals. The animals were segmented at the midpoint of the line connecting the cluster centers to obtain the experimental data of a single animal. Step 3: Count the data and use Python code to count the event stream data of a single animal; Step 4: Conduct visual analysis and use a variety of visualization methods and test methods to compare the similarities and differences in activity between different animals and different groups of animals.

2. The method for analyzing rodent behavior based on an event camera according to claim 1, characterized in that: Animal segmentation includes the following steps: Step 1: Detect and divide animals in different areas, and regard each event as a sample point; Step 2: K-means++ algorithm randomly selects an event point as the initial cluster center; Step 3: Calculate the shortest distance between each event and the current center and the probability of each event point being selected as the next cluster center; Step 4: Determine k cluster centers, divide all event points into the nearest centroids and iterate until the termination condition; Step 5: Use the midline of the lines connecting adjacent centroids as the dividing line to obtain the activity areas of different animals.

3. The method for analyzing rodent behavior based on an event camera according to claim 2, characterized in that: Events within 6 min of each animal were randomly sampled and 10% of the total data was used for cluster segmentation.

4. The method for analyzing rodent behavior based on an event camera according to claim 1, characterized in that: In step 4, heat maps and bar charts were used to visualize the results, and t-test and Kolmogorov-Smirnow test were used to perform hypothesis testing.

5. The method for analyzing rodent behavior based on an event camera according to claim 1, characterized in that: The experimental setup included a laptop, an animal behavior experimental set-up, an event camera, and data acquisition software.

Images