AI-based zebra fish behavior analysis method and system
By using an AI-based zebrafish behavior analysis method, deep learning networks are used to automatically extract behavioral features and train a classification model, which solves the problems of low efficiency and insufficient accuracy of traditional methods and achieves efficient and accurate zebrafish behavior analysis.
Patent Information
- Application Number
- CN202511181705.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-08-22
- Publication Date
- 2025-11-21
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional zebrafish behavior analysis methods are inefficient, lack precision, and have poor adaptability, making it difficult to meet the needs for efficient, accurate, and flexible behavior analysis.
We employ an AI-based zebrafish behavior analysis method. By collecting video data, labeling behavioral features, constructing a dataset, and training a classification model, we use a deep learning network to extract behavioral features, thereby achieving automatic classification and real-time analysis.
It enables rapid and accurate analysis of zebrafish behavior, significantly reducing analysis time costs, improving detection sensitivity, and supporting analysis of various behavior types and experimental scenarios.
Smart Images
Figure CN120997907A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of zebrafish behavior analysis technology, specifically to an AI-based zebrafish behavior analysis method and system. Background Technology
[0002] Zebrafish, due to their high genomic similarity to humans, transparent development, and short reproductive cycle, have become important model organisms for behavioral, pharmacological, and toxicological research. In experimental studies, analyzing zebrafish swimming behavior, social behavior, and stress responses is crucial for investigating neurological diseases, drug effects, and environmental toxicity. Traditional zebrafish behavioral analysis methods rely on manual observation or semi-automated software, which have the following drawbacks: 1. Inefficient: Manually analyzing behavioral videos is time-consuming and highly subjective.
[0003] 2. Insufficient accuracy: Traditional analysis methods have limited ability to detect subtle behavioral changes.
[0004] 3. Poor adaptability: Existing analysis software is difficult to flexibly adapt to the needs of specific experimental designs or new behavioral parameters.
[0005] Therefore, it is necessary to develop an efficient, intelligent, and adaptable zebrafish behavior analysis system to meet the current requirements for zebrafish behavior analysis. Summary of the Invention
[0006] To address the aforementioned technical problems, this invention provides an AI-based zebrafish behavior analysis method and system, which enables rapid and accurate analysis of zebrafish behavior.
[0007] The technical solution of this invention is: an AI-based zebrafish behavior analysis method, comprising the following steps: 1.1 Collect video data of zebrafish behavior under different experimental scenarios; 1.2. Perform behavioral annotation on the video data and generate corresponding behavioral tags; 1.3 Extracting behavioral features from video data; 1.4 Construct a dataset based on behavioral labels and behavioral features, and divide the dataset into a training set, a validation set, and a test set; 1.5. Train a zebrafish behavior classification model using the training set. The zebrafish behavior classification model takes behavioral features as input and outputs behavior classification results. 1.6 Input the zebrafish activity video to be analyzed into the trained zebrafish behavior classification model for behavior classification.
[0008] Furthermore, the behavioral tags include at least one of the following: stationary, normal swimming, fast swimming, escape behavior, and spinning.
[0009] Furthermore, the behavioral characteristics include, Motion characteristics: including swimming speed, angle changes, and tail wagging frequency; Spatial features: including movement trajectory and position coordinates; Temporal characteristics: Includes the continuous temporal variation characteristics of behavior.
[0010] Furthermore, the behavioral features are automatically extracted using a deep learning network.
[0011] Furthermore, the behavior annotation in step 1.2 is implemented using manual annotation or automatic annotation.
[0012] Furthermore, the zebrafish behavior classification model can be any of the following: a convolutional neural network, a recurrent neural network, or a spatiotemporal network that combines CNN and LSTM.
[0013] An AI-based zebrafish analysis system includes a zebrafish behavior classification module, which receives zebrafish activity video data and outputs behavior classification results based on a zebrafish behavior classification model.
[0014] Furthermore, it also includes at least one of the following functional modules: The behavior trajectory output module is used to generate and display the movement trajectory of zebrafish within a preset time based on zebrafish activity videos; The heatmap output module is used to generate a heatmap of the dwell time in a region based on zebrafish location data; The key behavior detection module is used to extract swimming speed, pause duration, and group behavior indicators from video data; The stress response analysis module is used to compare behavioral changes before and after the application of stimuli and to analyze behavioral responses to light or chemical stimuli.
[0015] Furthermore, it also includes: The imaging and data acquisition module, including a high-speed camera and an infrared filter, is used to acquire video data of zebrafish under different lighting conditions. The control and interface module is equipped with an embedded main control chip and a touch interface, which supports real-time adjustment of experimental parameters and data transmission.
[0016] Furthermore, it also includes: a test chamber module equipped with an adjustable light source and a constant temperature circulating water system to provide a controllable experimental environment.
[0017] The beneficial technical effects of this invention are: This zebrafish behavior analysis method and system uses AI algorithms to collect video data of zebrafish in various experimental scenarios and train models. The trained zebrafish behavior classification model can automatically classify zebrafish behavior and perform real-time behavior analysis, which can significantly reduce analysis time costs. Moreover, the system can capture subtle behavioral features through deep learning technology, thereby improving detection sensitivity.
[0018] This analysis system also supports single-fish and multi-fish experiments, as well as the analysis of various behavioral types, which can better meet the requirements of zebrafish analysis experiments.
[0019] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the preferred embodiments of the present invention are described in detail below with reference to the accompanying drawings. Attached Figure Description
[0020] Figure 1 This is a flowchart of the zebrafish behavior analysis method according to an embodiment of the present invention; Figure 2 This is a structural block diagram of the zebrafish behavior analysis system according to an embodiment of the present invention; Figure 3 This is a behavioral trajectory diagram of a zebrafish according to an embodiment of the present invention; Figure 4 This is a thermal distribution diagram of zebrafish according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the analysis interface of the zebrafish behavior analysis system according to an embodiment of the present invention. Detailed Implementation
[0021] In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, the specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0022] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate for the embodiments of this application described herein.
[0023] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship described in the embodiments and shown in the accompanying drawings, or the orientation or positional relationship that the product of this invention is usually placed in during use. They are only for the convenience of describing this invention and simplifying the description, and are not intended to indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limiting this invention.
[0024] like Figure 1 As shown, this invention specifically relates to an AI-based zebrafish behavior analysis method, which includes the following steps: First, data collection is conducted, acquiring video data of zebrafish in various experimental scenarios. Data sources can include real-time laboratory data acquisition or publicly available zebrafish behavior analysis datasets. For real-time laboratory data acquisition, high-definition cameras, infrared cameras, and behavior monitoring systems can be used to continuously capture video data of zebrafish activity in different experimental scenarios. High-definition cameras (such as 120fps high frame rate cameras) should be used to record zebrafish behavior data under different conditions. The acquisition angle should be a single-view top view or side view. Different behavioral experimental scenarios should be collected to gather diverse behavioral data, including normal behavior (free swimming, foraging, resting); stimulus responses (light stimulation (changes in light intensity), sound stimulation (changes in frequency and intensity); drug experiments (observing behavioral changes after administering different drugs); environmental changes (behavior under the influence of temperature, pH, oxygen content, etc.); and pathological states (abnormal behavior under models of nerve damage and movement disorders). Public databases include zebrafish behavior video datasets released by some research institutions, such as the Zebrafish Behavioral Database and the mage Archive, as well as some publicly published papers that provide raw video data.
[0025] The above data typically needs to be preprocessed to provide a clean data source for subsequent AI algorithms to extract behavioral features. This process includes the following steps: 1) Background Removal: Eliminate fixed background and noise, retaining only the moving parts of the zebrafish.
[0026] 2) Object detection: Locate the individual zebrafish using AI object detection algorithms (such as YOLO and Mask R-CNN).
[0027] 3) Trajectory tracking: A multi-target tracking algorithm (such as Kalman filtering + SORT algorithm) is used to connect the positions of the fish in consecutive frames to generate trajectory data.
[0028] 4) Coordinate Transformation: Maps pixel coordinates to actual physical coordinates (e.g., millimeters) in the experimental space. Output: Formatted trajectory data (timestamp, X coordinate, Y coordinate).
[0029] Next, the collected video data is processed to construct a dataset for model training. First, the collected video data is labeled with different behavioral tags. The behavioral classification criteria are as follows: different behavioral categories are labeled, for example: 0: stationary, 1: normal swimming, 2: fast swimming, 3: escape behavior, 4: spinning. Labeling tools can be manual (VideoLabeler, BORIS), or automatic labeling using keypoint detection tools such as DeepLabCut, OpenPose, etc., to achieve initial labeling. Behavioral tags are then mapped to video frames, generating .csv files or JSON format records. Next, behavioral features are extracted using a deep learning network to automatically extract behavioral features, including motion features, spatial features, and temporal features. The motion features include swimming speed, angle changes, and tail wagging frequency; the spatial features include the zebrafish's trajectory and position coordinates in space; and the temporal features include continuous changes in the zebrafish's behavior over time.
[0030] The extracted behavioral features are used to construct a dataset with their corresponding behavioral labels.
[0031] Next, a zebrafish behavior classification model is constructed and trained using a training set. Model selection depends on the nature of the behavioral data; different AI models can be chosen: Convolutional Neural Networks (CNNs) are suitable for behavior classification of static images, such as single-frame behavior recognition. Their structure is: input image → feature extraction convolutional layer → classification fully connected layer. Recurrent Neural Networks (RNNs / LSTMs) are suitable for temporal behavior classification (continuous time data). Their structure is: input behavioral features → LSTM unit → classification output. Spatiotemporal Networks (CNN+LSTM) fuse spatial features of images with temporal information, suitable for complex behavior classification. Their structure is: CNN extracts image features → LSTM analyzes temporal changes → classification output. The zebrafish behavior classification model is used to input behavioral features and output behavior classification.
[0032] When training the zebrafish behavior classification model, the data is first divided into a training set (70%), a validation set (15%), and a test set (15%).
[0033] 1. Training methods: Loss function: Cross-entropy loss.
[0034] Optimizers: Adam, SGD, etc.
[0035] Batch size: 16~64 (adjust according to dataset size).
[0036] Learning rate: Initial learning rate 0.001, using a dynamic adjustment strategy.
[0037] 2. Training platform: Deep learning frameworks: TensorFlow, PyTorch.
[0038] GPU support: Improves training efficiency (e.g., NVIDIA GPUs).
[0039] Model Validation and Evaluation After training, the model performance is evaluated using the validation and test sets: 3. Evaluation indicators: Accuracy: The overall classification accuracy of the model.
[0040] Precision, Recall, and F1 score: These measures the balance of a classification model.
[0041] Confusion matrix: Analyzes the classification effect of different behavior categories.
[0042] 4. Model Optimization and Deployment Model optimization: Data augmentation: Adds changes to video lighting, rotation, scaling, etc.
[0043] Hyperparameter tuning: learning rate, batch size, number of network layers, etc.
[0044] Model compression: Reduce model size through methods such as pruning and quantization to facilitate deployment.
[0045] System Deployment: The trained model is integrated into the zebrafish behavior analysis system to achieve real-time or batch behavior recognition.
[0046] Deployment platforms: such as embedded devices (NVIDIA Jetson Nano), cloud servers, or local PCs. 5. Algorithm details: Fish detection: Using a YOLO-based target detection algorithm, individual zebrafish in the video are quickly located.
[0047] Behavior classification: Combining LSTM model analysis of motion trajectory and time series features to classify behavior types.
[0048] Noise reduction: Water ripples and reflections are eliminated using image enhancement techniques.
[0049] Finally, after the zebrafish behavior classification model is trained, it is used to classify the behavior of the collected zebrafish activity videos. This zebrafish behavior classification model can be applied to multiple fields such as drug screening, neuroscience research, and environmental toxicology.
[0050] Drug screening: Efficient detection of the effects of drugs on zebrafish behavior, used for screening neuroactive or toxic drugs; Neuroscience research: analyzing zebrafish stress behavior, learning and memory abilities, etc.; Environmental toxicology: Studying the effects of chemical pollutants on zebrafish behavior.
[0051] like Figure 2 As shown, this invention also discloses an AI-based zebrafish analysis system, which includes a zebrafish behavior classification module. This module is used to classify the behavior of zebrafish based on collected zebrafish activity video information. The zebrafish behavior classification module is trained using the zebrafish behavior classification model described above.
[0052] In addition to the zebrafish behavior classification module mentioned above, the system also includes a behavior trajectory output module, a heat map output module, a key behavior detection module, and a stress response analysis module.
[0053] like Figure 3 As shown, the behavior trajectory output module presents X and Y coordinate data in a two-dimensional plane, displaying the movement trajectory of the zebrafish during the experimental time. Based on the exported data of the behavior trajectory, such as X coordinate (PX), Y coordinate (PX), speed (mm / s), and angle of change of direction, the following multi-dimensional analyses can be performed. For example, spatial distribution analysis: such as activity area (mm²), indicating the size of the area where the fish will move during the experiment; center offset (mm), indicating whether the zebrafish tends to stay in a specific location within the experimental area. Another example is motion characteristic analysis: average speed (mm / s), reflecting the overall activity level of the fish; frequency of changes of direction (times / s): determining whether the fish exhibits exploratory or anxious behavior.
[0054] Heatmap output module: Used to display a heatmap of zebrafish's preferred time zones based on collected zebrafish activity video information (e.g., Figure 4 (As shown).
[0055] Key Behavior Detection Module: Used to collect data on zebrafish swimming speed, pause times, and group behavior based on acquired zebrafish activity video information. Stress Response Analysis Module: Used to analyze behavioral changes in zebrafish in response to light or chemical stimuli.
[0056] As a specific embodiment of the present invention, the zebrafish behavior analysis system also includes an experimental chamber module, an imaging and data acquisition module, and a control and interface module. The experimental chamber module is made of sheet metal and equipped with an adjustable LED light source to support precise light intensity control. It is equipped with a circulating water system to provide a constant temperature environment (accuracy ±1℃). The imaging and data acquisition module includes a high-speed camera (60 frames / second or higher) with a resolution supporting 1080p. It is equipped with an infrared filter to adapt to different experimental scenarios. Its image sensor has high sensitivity and supports shooting in low-light environments. The control and interface module includes an embedded main control chip (such as an ARM Cortex-A series) responsible for data acquisition and processing. It provides USB, HDMI, and Wi-Fi interfaces to support external devices and remote data transmission; it also integrates a touch screen for real-time adjustment of experimental parameters.
[0057] In a preferred embodiment of the present invention, the zebrafish behavior analysis system further includes an operation interface module, which specifically includes the following modules: Experiment Management Module Experiment List: Displays experiment name, time, status, creator, and other information in card or table format. Create New Experiment: A pop-up window allows users to enter the experiment name and settings (zebrafish number, behavior monitoring metrics, etc.).
[0058] Operation buttons: Edit, copy, delete experiments, and support batch operations.
[0059] Real-time monitoring module Video monitoring area: Real-time display of zebrafish behavior trajectory video stream.
[0060] Data monitoring area: Displays real-time generated data charts, such as heatmaps of behavioral trajectories.
[0061] Control Panel: Provides options for pausing / resumpting monitoring and adjusting monitoring parameters (sampling frequency, behavior detection type, etc.).
[0062] Data Analysis Module Data visualization: Displaying zebrafish behavior data using heatmaps, histograms, line graphs, and other methods.
[0063] Data filtering: Users can filter and analyze data by time period and behavior type.
[0064] Advanced Analysis: Supports correlation analysis (such as the relationship between activity level and light intensity), trend prediction, and other functions. Data Export: Supports exporting analysis results as CSV, Excel, or image formats.
[0065] Report generation module Template selection: A variety of lab report templates are available for users to choose from or customize.
[0066] Data input: Automatically retrieves charts and text descriptions from the data analysis module.
[0067] Format Preview: Preview the generated report style in real time.
[0068] Export function: Supports exporting to PDF and Word documents.
[0069] Settings Module Experimental setup: Adjusting behavior detection sensitivity, monitoring parameters, etc.
[0070] User Settings: Manage user accounts, permissions, and preferences.
[0071] System settings: Configure data storage path, system updates, etc.
[0072] The above embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and are not intended to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the scope of the technology disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention.
Claims
1. An AI-based method for zebrafish behavior analysis, characterized in that, Includes the following steps: 1.1 Collect video data of zebrafish behavior under different experimental scenarios; 1.
2. Perform behavioral annotation on the video data and generate corresponding behavioral tags; 1.3 Extracting behavioral features from video data; 1.4 Construct a dataset based on behavioral labels and behavioral features, and divide the dataset into a training set, a validation set, and a test set; 1.
5. Train a zebrafish behavior classification model using the training set. The zebrafish behavior classification model takes behavioral features as input and outputs behavior classification results. 1.6 Input the zebrafish activity video to be analyzed into the trained zebrafish behavior classification model for behavior classification.
2. The AI-based zebrafish behavior analysis method according to claim 1, characterized in that, The behavioral tags include at least one of the following: stationary, normal swimming, fast swimming, escape behavior, and spinning.
3. The AI-based zebrafish behavior analysis method according to claim 1, characterized in that, The behavioral characteristics include, Motion characteristics: including swimming speed, angle changes, and tail wagging frequency; Spatial features: including movement trajectory and position coordinates; Temporal characteristics: Includes the continuous temporal variation characteristics of behavior.
4. The AI-based zebrafish behavior analysis method according to claim 1, characterized in that, The behavioral features are automatically extracted using a deep learning network.
5. The AI-based zebrafish behavior analysis method according to claim 1, characterized in that, The behavior annotation in step 1.2 can be implemented by manual annotation or automatic annotation.
6. The AI-based zebrafish behavior analysis method according to claim 1, characterized in that, The zebrafish behavior classification model is any one of the following: convolutional neural network, recurrent neural network, or a spatiotemporal network that combines CNN and LSTM.
7. An AI-based zebrafish analysis system, characterized in that, It includes a zebrafish behavior classification module, which receives zebrafish activity video data and outputs behavior classification results based on the zebrafish behavior classification model trained by the method described in any one of claims 1-6.
8. The AI-based zebrafish analysis system according to claim 7, characterized in that, It also includes at least one of the following functional modules: The behavior trajectory output module is used to generate and display the movement trajectory of zebrafish within a preset time based on zebrafish activity videos; The heatmap output module is used to generate a heatmap of the dwell time in a region based on zebrafish location data; The key behavior detection module is used to extract swimming speed, pause duration, and group behavior indicators from video data; The stress response analysis module is used to compare behavioral changes before and after the application of stimuli and to analyze behavioral responses to light or chemical stimuli.
9. The AI-based zebrafish analysis system according to claim 7, characterized in that, Also includes: The imaging and data acquisition module, including a high-speed camera and an infrared filter, is used to acquire video data of zebrafish under different lighting conditions. The control and interface module is equipped with an embedded main control chip and a touch interface, which supports real-time adjustment of experimental parameters and data transmission.
10. The AI-based zebrafish analysis system according to claim 7, characterized in that, Also includes: The test chamber module is equipped with an adjustable light source and a constant temperature circulating water system to provide a controllable experimental environment.