AI and deep learning-based mouse characteristic behavior analysis method

Abstract

The application relates to the technical field of animal behavior intelligent analysis, and discloses a rat characteristic behavior analysis method based on AI and deep learning. The method comprises the following steps: extracting joint coordinates and motion directions of rats from a video stream to form time sequence data; performing self-adaptive cutting on a time axis by detecting motion direction mutation and joint trajectory breakage to segment independent behavior event units; for each unit, constructing a standardized behavior template grid based on internal joint coordinates, and mapping a motion trajectory to grid nodes to form a grid-based behavior feature map; and performing classification and statistics according to the spatial configuration and node trajectory mode of the map to output a structured analysis result. The method realizes accurate segmentation and standardized representation of animal behavior, and improves the objectivity and consistency of behavior analysis.

Description

Technical Field

[0001] This invention relates to the field of intelligent animal behavior analysis technology, specifically to a method for analyzing the characteristic behaviors of mice and rats based on AI and deep learning. Background Technology

[0002] In preclinical studies in neuroscience, pharmacology, and other fields, precise and objective quantitative analysis of the behavior of model animals such as mice and rats is crucial. Traditional behavioral analysis methods mainly rely on manual observation and recording or automated systems based on simple image processing. Manual observation has inherent drawbacks such as strong subjectivity, low efficiency, and difficulty in capturing transient micro-behaviors. Common automated systems often use fixed-duration time windows to segment videos or set single motion intensity thresholds to define the start and end of behavioral events. They cannot accurately capture the instantaneous transitions in animal behavior, resulting in blurred boundaries of behavioral event units, incorrectly cutting or merging continuous and complex behavioral flows, and affecting the accuracy of subsequent behavioral classification and statistics.

[0003] In terms of behavioral feature representation, existing technologies mostly utilize raw coordinate sequences of animal body joints extracted from videos or simply derived speed and angle as analytical features. However, due to differences in animal size, posture, and position within the field of vision, direct comparison of raw coordinate sequences lacks a unified standard. Unstandardized feature representation methods make it difficult to effectively compare behavioral patterns between different experimental batches and individuals, limiting model generalization ability and insensitivity to subtle but potentially biologically significant behavioral differences, thus hindering the realization of high-throughput, high-precision behavioral phenotypic analysis. Summary of the Invention

[0004] The purpose of this invention is to provide a method for analyzing the characteristic behaviors of mice and rats based on AI and deep learning, so as to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, this invention provides a method for analyzing the characteristic behaviors of mice and rats based on AI and deep learning, the method comprising: Collect raw video streams containing information on mouse and rat activities, extract the set of trunk joint coordinates and motion direction vectors of mice and rats in each video frame, and generate joint motion time series data; The system receives the joint motion timing data and segments the time axis according to the abrupt changes in the motion direction vector and the break points of the continuous joint trajectory, thereby segmenting independent behavioral event units. For each behavior event unit, a standardized behavior template mesh is constructed based on its internal set of joint coordinates, and all motion trajectories within the behavior event unit are mapped to the corresponding nodes of the behavior template mesh to form a meshed behavior feature map. Based on the spatial configuration and node trajectory pattern of the behavior template grid, the gridded behavior feature map is classified and labeled, and the cumulative occurrence duration and average displacement of grid nodes of each labeled category within a preset observation period are statistically analyzed, and finally, structured behavior analysis results are output.

[0006] Preferably, the method further includes: performing an initial cleaning operation on the original video stream to remove environmental noise frames and invalid motion frames, specifically including: The original video stream is subjected to frame-by-frame pixel value analysis to identify frames with a pixel change rate lower than a preset static threshold, which are marked as invalid motion frames. At the same time, it is detected whether there are pseudo-motion regions within the frames that are highly consistent with the environmental background pattern, and frames containing such pseudo-motion regions are marked as environmental noise frames. On the timeline of the original video stream, the positions of the frames marked as invalid motion frames and environmental noise frames are recorded as elimination indices. According to the elimination indices, all corresponding frames are removed from the original video stream, and the remaining video frames are reassembled in their original time order to form a continuous original video stream.

[0007] Preferably, the joint motion timing data is received, and the time axis is segmented according to the abrupt changes in the motion direction vector and the break points of the continuous joint trajectory, thereby segmenting independent behavioral event units, including: The motion direction vectors of adjacent time points are sequentially read from the joint motion time series data, and the angle difference between each pair of adjacent vectors is calculated. When the angle difference exceeds a preset action change threshold, the adjacent time point is determined to be a motion direction change point, and its timestamp is recorded. At the same time, it is detected whether there are joint coordinate zero change segments in the joint motion time series data that exceed a preset duration threshold. The first and last time points of such segments are recorded as trajectory breakpoints. All motion direction change points and trajectory breakpoints are integrated, and the motion direction change points and trajectory breakpoints are used as cutting points on the time axis to divide the continuous joint motion time series data into multiple non-overlapping time segments. Each segment is defined as an independent behavioral event unit.

[0008] Preferably, for each behavior event unit, a standardized behavior template mesh is constructed based on its internal set of joint coordinates, and all motion trajectories within the behavior event unit are mapped to the corresponding nodes of the behavior template mesh to form a meshed behavior feature map, including: For each independent behavioral event unit, all joint coordinates it encompasses are extracted, and the distribution boundary of all joint coordinates in three-dimensional space is calculated. Based on the distribution boundary, a three-dimensional cubic virtual space covering the distribution boundary is established, and this virtual space is uniformly divided into a specified number of tiny cubes along three orthogonal coordinate axes. Each tiny cube is defined as a grid node, and the set of all nodes constitutes the behavioral template grid. The joint coordinates at each moment within the behavioral event unit are assigned to the corresponding grid node of the behavioral template grid according to their spatial location. The average motion trajectory of all coordinate points assigned to each grid node during the entire behavioral event unit is calculated, and the average motion trajectory is used as the feature vector of the grid node. The feature vectors of all grid nodes together constitute a gridded behavioral feature map.

[0009] Preferably, based on the spatial configuration and node trajectory pattern of the behavior template mesh, the meshed behavior feature map is classified and labeled, and the cumulative occurrence duration and average displacement of each labeled category within a preset observation period are statistically analyzed. Finally, structured behavior analysis results are output, including: The spatial distribution density and topological connectivity of non-empty grid nodes in the behavior template grid are analyzed to form a spatial configuration descriptor. Simultaneously, the trajectory morphology pattern of the feature vector of each grid node in the gridded behavior feature map is extracted to form a trajectory pattern descriptor. The spatial configuration descriptor and the trajectory pattern descriptor are combined and matched against a preset behavior category knowledge base to assign one or more behavior category labels to each behavior event unit. Within the entire preset observation period, all labeled behavior event units are categorized according to their behavior category labels. For each behavior category, the duration of all behavior event units under it is accumulated to obtain the cumulative occurrence duration of the behavior category. Simultaneously, the average displacement of all grid nodes in the corresponding gridded behavior feature map is calculated for all behavior event units belonging to the behavior category to obtain the average displacement of the grid nodes. The cumulative occurrence duration of all behavior categories and the average displacement data of the grid nodes are integrated to generate structured behavior analysis results.

[0010] Preferably, performing frame-by-frame pixel value analysis on the original video stream to identify frames with a pixel change rate lower than a preset static threshold includes: Read consecutive video frames sequentially from the original video stream and calculate the sum of the absolute differences between corresponding pixels in two adjacent frames; compare the calculated sum of absolute differences with the preset static threshold; if the sum of absolute differences is lower than the preset static threshold, then determine that the next frame is a frame with a pixel change rate lower than the preset static threshold.

[0011] Preferably, detecting whether there is a pseudo-motion region within the frame that is highly consistent with the environmental background pattern, and marking the frame containing the pseudo-motion region as an environmental noise frame, includes: A reference background model is established for the original video stream. The reference background model is obtained by statistically modeling the pixel values ​​of all video frames within an initial time period. For each subsequent video frame, it is divided into multiple image blocks, and the texture and color feature vectors of each image block are extracted. The similarity between the feature vector of each image block and the feature vector at the corresponding position in the reference background model is calculated. If the similarity of an image block is higher than a preset background similarity threshold, the image block is determined to be an environmental background mode region. The number of image blocks determined to be environmental background mode regions in a frame is counted. If the number of image blocks exceeds a preset region number threshold, the frame is determined to be an environmental noise frame containing pseudo-motion regions.

[0012] Preferably, the motion direction vectors at adjacent time points are read sequentially from the joint motion time series data, and the angle difference between each pair of adjacent vectors is calculated, including: In the joint motion time series data, the three-dimensional spatial coordinates of a specified joint at two consecutive sampling time points are extracted; based on the three-dimensional spatial coordinates, the displacement vector of the specified joint between these two time points is calculated, and the displacement vector is the motion direction vector; the motion direction vectors of two adjacent time periods are obtained sequentially, and the cosine value of the angle between these two vectors is calculated; based on the cosine value of the angle, the angle difference is obtained by inverse trigonometric function operation.

[0013] Preferably, for each independent behavioral event unit, all joint coordinates it encompasses are extracted, and the distribution boundary of all joint coordinates in three-dimensional space is calculated, including: Read all joint coordinate data generated by a behavior event unit throughout its entire time span, with each data point containing three-dimensional coordinate values; find the maximum and minimum values ​​of all data points in each of the three coordinate axes; define a three-dimensional spatial rectangular region with the minimum value as the lower bound and the maximum value as the upper bound in each coordinate axis direction, which is the distribution boundary.

[0014] Preferably, the spatial distribution density and topological connectivity of non-empty grid nodes in the behavioral template mesh are analyzed to form a spatial configuration descriptor, including: In the behavior template grid, grid nodes containing coordinate assignment records are selected and defined as non-empty grid nodes; Calculate the centroid position of the three-dimensional coordinate set of all non-empty mesh nodes; using the centroid position as the center, count the number of non-empty mesh nodes in spherical spaces of different radii to form a node density distribution spectrum. Simultaneously, the straight-line distance between each pair of non-empty grid nodes in three-dimensional space is checked. If the distance is less than a preset connection distance threshold, a connection edge is established between each pair of non-empty grid nodes. The number of connection edges and the average connection length between all non-empty grid nodes are counted to form a set of topological connection relationships. The node density distribution spectrum and the set of topological connection relationships are encoded together to generate the spatial configuration descriptor.

[0015] Compared with the prior art, the beneficial effects of the present invention are: By adaptively segmenting the timeline based on abrupt changes in joint motion direction vectors and breaks in continuous joint trajectories, the physical moments of natural shifts in animal behavior can be accurately identified. Sudden changes in motion direction or discontinuous jumps in motion trajectories correspond to shifts in the animal's intention or natural boundaries of behavioral segments. Segmenting based on these intrinsic kinematic characteristics replaces external, arbitrary, fixed time windows, making each segmented behavioral event unit physically more complete and independent. This lays an accurate temporal foundation for subsequent analysis of single, pure behavioral patterns.

[0016] A standardized behavior template mesh is dynamically constructed based on the set of joint coordinates within each behavioral event unit. All motion trajectories are mapped to mesh nodes to generate feature maps, realizing the transformation of behavioral features from the original image space to a normalized topological space. The mesh template is constructed based on the animal's actual posture in the current behavioral event, eliminating the influence of individual body size and absolute spatial position. The resulting mesh node trajectory sequence is an abstract feature decoupled from the specific experimental scenario, purely describing the motion pattern. This structured feature representation greatly enhances the direct comparability of behavioral patterns between different samples, allowing pattern-based classification algorithms to focus more on the topological and dynamic characteristics of the motion itself, improving the discriminative power for similar behaviors and the robustness of the classification model. Attached Figure Description Figure 1 This is a schematic diagram illustrating the working principle of the AI ​​and deep learning-based mouse and rat characteristic behavior analysis method described in this invention. Figure 2 A flowchart of the initial cleaning operation for the raw video stream; Figure 3 Flowchart for segmenting behavioral event units; Figure 4 Time-varying characteristics of average displacement of grid nodes for different behavioral categories of rats and mice; Figure 5 A comparison chart showing the processing efficiency and time consumption at each stage. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Please see Figure 1 This invention provides a method for analyzing the characteristic behavior of mice and rats based on AI and deep learning. The method includes: First, acquiring raw video streams containing mouse and rat activity information through an image acquisition device; processing each video frame using a pre-trained deep learning pose estimation algorithm to extract the coordinates of key joints in the mouse and rat trunk in two-dimensional or three-dimensional space, forming a set of joint coordinates; and calculating motion direction vectors based on the displacement of joint points between adjacent frames to generate complete joint motion temporal data. Next, this joint motion temporal data is sent to an event segmentation module. This module analyzes the changes in the motion direction vectors, identifies the time points of abrupt changes, and detects breaks in the joint motion trajectory such as prolonged periods of stillness or loss. Using these points as cutting points, the continuous time axis is divided into multiple independent time periods, each corresponding to a complete and relatively independent behavioral event unit. Then, for each segmented behavioral event unit, a standardized, uniform-scale three-dimensional behavioral template mesh is constructed based on the spatial distribution range of all joint coordinate points within the unit. The joint motion trajectories recorded at each moment within the event unit are mapped to the corresponding nodes of the behavioral template mesh according to their spatial coordinates. Through aggregation and calculation, a meshed behavioral feature map is formed, with mesh nodes and their motion trajectory feature vectors as elements. Finally, the classification module analyzes the spatial configuration features of the behavioral template mesh and the trajectory motion patterns of each node reflected in the meshed behavioral feature map according to predefined or learned rules, and matches them with known behavioral categories to perform classification labeling for each behavioral event unit. Within a preset observation period, the system counts the cumulative total duration of all event units under each behavioral category and calculates the average displacement of all nodes in the corresponding meshed feature map of these event units. The system integrates information such as category, duration, and average displacement, and outputs a structured behavioral analysis result report.

[0019] In one embodiment of the present invention, see [reference] Figure 2An initial cleaning operation is performed on the original video stream to remove environmental noise frames and invalid motion frames. Specifically, the original video stream is analyzed frame-by-frame to identify frames with a pixel change rate lower than a preset static threshold. This process involves sequentially reading consecutive video frames from the original video stream and calculating the sum of the absolute differences between corresponding pixels in adjacent frames. The calculated sum of absolute differences is compared to the preset static threshold. If the sum of absolute differences is lower than the preset static threshold, the subsequent frame is determined to have a pixel change rate lower than the preset static threshold and is marked as an invalid motion frame. Simultaneously, the system detects whether there are pseudo-motion regions within the frame that are highly consistent with the environmental background pattern, establishing a reference background model for the original video stream. This reference background model is obtained by statistically modeling the pixel values ​​of all video frames within an initial time period. For each subsequent video frame, it is divided into multiple image blocks, and the texture and color feature vectors of each image block are extracted. The similarity between the feature vector of each image block and the feature vector at the corresponding position in the reference background model is calculated. If the similarity of an image block is higher than a preset background similarity threshold, the image block is determined to be an environmental background pattern region. The number of image blocks in a frame that are determined to be environmental background pattern regions is counted. If the number of image blocks exceeds a preset region number threshold, the frame is determined to be an environmental noise frame containing pseudo-motion regions. On the timeline of the original video stream, the frame positions marked as invalid motion frames and environmental noise frames are recorded as removal indices. Based on the removal indices, all corresponding frames are removed from the original video stream, and the remaining video frames are reassembled in their original chronological order to form a continuous, cleaned original video stream.

[0020] In some embodiments, a frame-by-frame pixel value analysis is performed on the original video stream to identify frames with a pixel change rate lower than a preset static threshold. This process involves sequentially reading consecutive video frames from the original video stream. For two adjacent frames, the absolute difference between the grayscale values ​​or color intensity values ​​of all corresponding pixels is calculated, and these absolute differences are summed to obtain the "sum of absolute differences," which characterizes the overall change between frames. This calculation process can be expressed by the following formula: ; in: This represents the sum of the calculated absolute differences. This indicates the absolute value operation, where W and H represent the width and height of the video frame, respectively. and These represent the coordinates located at the current time t and the previous adjacent time t-1 in the video frame. The pixel values. Then, the sum of the calculated absolute differences. The sum of the absolute differences is compared with a pre-set static threshold used to distinguish between motion and stillness. If the pixel change rate is lower than the preset static threshold, the video frame corresponding to time point t is determined to be a frame whose pixel change rate is lower than the preset static threshold, and this frame is marked as an invalid motion frame.

[0021] It is understandable that detecting the existence of pseudo-motion regions within a frame that are highly consistent with the environmental background pattern is another parallel step in the cleaning operation. The implementation includes establishing a reference background model for the original video stream. The establishment of the reference background model relies on statistical modeling of the pixel values ​​of all video frames within an initial time period of the original video stream, for example, by calculating the mean and variance of the color or intensity at each pixel location within that time period. For each video frame to be processed after the initial time period, the processing system spatially divides the video frame into multiple non-overlapping image blocks. For each image block, a feature vector characterizing its texture features and color distribution is extracted. In specific implementations, the similarity between the feature vector of each image block and the feature vector corresponding to the same spatial location region in the reference background model is calculated. Similarity calculation can use methods such as cosine similarity or Euclidean distance comparison. If the similarity calculation result of an image block is higher than a preset background similarity threshold, then the image block is determined to be an environmental background pattern region. The processing system then counts the number of image blocks in the entire video frame that are determined to be environmental background pattern regions. When the number of image blocks exceeds a preset region number threshold, the video frame is determined to be an environmental noise frame containing pseudo-motion regions.

[0022] In some embodiments, the system performing the initial cleaning operation records the frame position indices of all frames marked as invalid motion frames and environmental noise frames in a list of indices to be removed from the timeline of the original video stream. Optionally, the system removes all video frame data corresponding to the index positions from the data sequence of the original video stream according to the list of indices to be removed. In a specific implementation, after the removal operation is completed, the system reassembles and serializes the remaining video frame data according to their original time order, thereby forming a continuous, initially cleaned original video stream.

[0023] It is understandable that the preset static threshold, background similarity threshold, and region number threshold are all configurable parameters. Optionally, these thresholds can be set based on prior knowledge of the lighting stability of the experimental environment, camera noise level, and the contrast between mice / large animals and the background, or adaptively calculated by analyzing a clean environmental video stream without mouse / large animal activity during the initial system deployment. The initial cleaning operation effectively filters out a large amount of invalid data generated by slight fluctuations in ambient lighting, camera shake, or irrelevant minor disturbances in the background.

[0024] In one embodiment of the present invention, see [reference] Figure 3The system receives joint motion time-series data and segments the time axis based on abrupt changes in motion direction vectors and breaks in continuous joint trajectories, thereby dividing the data into independent behavioral event units. Specifically, the implementation includes: sequentially reading motion direction vectors from adjacent time points in the joint motion time-series data; extracting the three-dimensional spatial coordinates of a specified joint at two consecutive sampling time points from the joint motion time-series data; calculating the displacement vector of the specified joint between these two time points based on the three-dimensional spatial coordinates, which is the motion direction vector; sequentially acquiring the motion direction vectors of two adjacent time periods and calculating the cosine of the angle between these two vectors; and obtaining the angle difference using an inverse trigonometric function based on the cosine of the angle. When the angle difference exceeds a preset action abrupt change threshold, the adjacent time point is determined as a motion direction abrupt change point, and its timestamp is recorded. Simultaneously, the system detects whether there are joint coordinate zero-change segments in the joint motion time-series data exceeding a preset duration threshold, and records the beginning and end time points of such segments as trajectory breakpoints. By integrating all abrupt changes in motion direction and trajectory breakpoints, and using these points as cutting points on the time axis, continuous joint motion time series data is divided into multiple non-overlapping time segments, each segment being defined as an independent behavioral event unit.

[0025] In some embodiments, motion direction vectors at adjacent time points are sequentially read from joint motion time series data, and the angle difference between each pair of adjacent vectors is calculated. This process involves processing the coordinate data of a specified joint. The system extracts the three-dimensional spatial coordinates of a specified joint, such as the tip of the nose or the center of the trunk in a mouse, at two consecutive sampling time points from the joint motion time series data. The coordinates are expressed as follows: and This indicates that the displacement vector of the specified joint between these two time points is calculated based on the three-dimensional spatial coordinates. This is defined as the motion direction vector of the current time interval. The system sequentially acquires the motion direction vectors of two adjacent time intervals. and Calculate the cosine of the angle between two vectors. The calculation can be performed according to the formula: ; in: Representing vectors and dot product, and Representing vectors respectively and The length of the mold, This is the angle difference to be determined. The system uses the calculated cosine of the included angle and inverse trigonometric functions to obtain the precise angle difference. .

[0026] It is understandable that after calculating the angle difference between continuous motion direction vectors, the system compares the angle difference with a preset action mutation threshold. When the angle difference exceeds the preset action mutation threshold, the system determines the adjacent time point that generated the angle difference as the motion direction mutation point and records the timestamp corresponding to the motion direction mutation point in the event segmentation list. Simultaneously, the system checks in parallel whether there are joint coordinate zero-change segments in the joint motion time series data that exceed a preset duration threshold. A joint coordinate zero-change segment refers to a segment in which the three-dimensional spatial coordinate values ​​of a specified joint remain unchanged or change less than the minimum resolution at multiple consecutive sampling time points. The system identifies such segments and records the start and end time points of the segments as trajectory breakpoints. In specific implementation, all identified motion direction mutation points and trajectory breakpoints are integrated, and the positions of these points on the time axis are used as cutting points to segment the originally continuous joint motion time series data into multiple independent time segments that do not overlap in time. Each independent time segment is defined as an independent behavioral event unit.

[0027] In some embodiments, the preset action mutation threshold and the preset duration threshold are key configuration parameters. Optionally, the preset action mutation threshold can be set in the range of 30 degrees to 90 degrees to capture sudden changes or pauses in animal behavior; the preset duration threshold can be set in the range of 0.5 seconds to 2.0 seconds to identify natural intervals or cessation of behavior.

[0028] In one embodiment of the present invention, for each behavioral event unit, a standardized behavioral template mesh is constructed based on its internal set of joint coordinates, and all motion trajectories within the behavioral event unit are mapped to the corresponding nodes of the behavioral template mesh to form a meshed behavioral feature map. Specific implementation includes: for each independent behavioral event unit, extracting all joint coordinates it encompasses and calculating the distribution boundary of all joint coordinates in three-dimensional space. This process involves reading all joint coordinate data generated by a behavioral event unit throughout its entire time span, with each data point containing three-dimensional coordinate values; identifying the maximum and minimum values ​​of all data points along the three coordinate axes; defining a three-dimensional rectangular region with the minimum value as the lower bound and the maximum value as the upper bound along each coordinate axis, which is the distribution boundary. Based on the distribution boundary, a three-dimensional cubic virtual space covering the distribution boundary is established, and this virtual space is uniformly divided into a specified number of tiny cubes along the three orthogonal coordinate axes. Each tiny cube is defined as a mesh node, and the set of all nodes constitutes the behavioral template mesh. The joint coordinates at each moment within the behavioral event unit are assigned to the corresponding mesh nodes of the behavioral template mesh according to their spatial location. For each grid node, the average motion trajectory of all coordinate points assigned to it during the entire behavior event unit is calculated, and this average motion trajectory is used as the feature vector of the grid node. The feature vectors of all grid nodes together constitute the gridded behavior feature map.

[0029] In some embodiments, for each independent behavioral event unit, all joint coordinates it encompasses are extracted and the distribution boundary of all joint coordinates in three-dimensional space is calculated. Specifically, the system reads all joint coordinate data generated by a behavioral event unit throughout its entire time span. These data come from multiple predefined joint points on the torso of a mouse or rat, and each data point contains the coordinate values ​​of the joint point in three-dimensional space. The system iterates through the values ​​of all data points along the X-axis, Y-axis, and Z-axis, respectively, and finds the maximum value of all data points along the X-axis. and minimum value The maximum value in the Y-axis direction and minimum value and the maximum value in the Z-axis direction. and minimum value The minimum value in each coordinate axis direction. , , As the lower bound, with the maximum value , , As the upper bound, the system defines a three-dimensional rectangular region in three-dimensional space enclosed by these six boundary values. This three-dimensional rectangular region is determined as the distribution boundary of all joint coordinates of the behavior event unit.

[0030] Understandably, the process of constructing a standardized behavioral template mesh based on the distribution boundary then unfolds. The system establishes a three-dimensional cubic virtual space that precisely covers the aforementioned distribution boundary, with the three sides of this virtual space being equal to... , and In practical implementation, the system uniformly divides this three-dimensional cubic virtual space along three orthogonal coordinate axes, namely the X-axis, Y-axis, and Z-axis. The number of divisions along each coordinate axis can be configured to the same value N or different values, thereby dividing the entire virtual space into... Each tiny cubic unit is defined as a grid node, and the collection of all grid nodes together constitutes the behavior template grid for that behavior event unit. The system records each joint coordinate point at each sampling time within the behavior event unit, based on the spatial position of the joint coordinate point. Assignment is determined by assigning the user to a unique grid node in the behavior template grid that corresponds to its spatial location.

[0031] In some embodiments, a feature vector is calculated for each grid node to form a gridded behavior feature map. For each grid node in the behavior template grid, the system checks the historical records of all joint coordinate points assigned to it during the behavior event unit. If a grid node is assigned to at least one coordinate point, the system calculates the average motion trajectory for that grid node across all coordinate points assigned to it throughout the entire behavior event unit. The calculation of the average motion trajectory can be expressed as follows: ; in: The feature vector representing a grid node is the average motion trajectory, and K represents the total number of coordinate points assigned to that grid node. This represents the displacement vector of the k-th coordinate point assigned to this grid node relative to its coordinates at the previous time step. Optionally, the displacement vector can be calculated based on a fixed time window or adjacent frames. The system uses the calculated average motion trajectory... As the feature vectors of the corresponding grid nodes, the feature vectors of all grid nodes are organized according to their spatial topological relationships, and together they form a gridded behavioral feature map that completely describes the motion mode of the behavioral event unit.

[0032] In one embodiment of the present invention, based on the spatial configuration and node trajectory pattern of the behavior template mesh, the meshed behavior feature map is classified and labeled, and the cumulative occurrence duration and average displacement of each labeled category within a preset observation period are statistically analyzed, ultimately outputting a structured behavior analysis result. Specific implementation includes: analyzing the spatial distribution density and topological connectivity of non-empty mesh nodes in the behavior template mesh to form a spatial configuration descriptor. The process involves: selecting mesh nodes containing coordinate assignment records from the behavior template mesh and defining them as non-empty mesh nodes; calculating the centroid position of the three-dimensional coordinate set of all non-empty mesh nodes; counting the number of non-empty mesh nodes in spherical spaces of different radii centered on the centroid position to form a node density distribution spectrum; simultaneously, checking the straight-line distance between each pair of non-empty mesh nodes in three-dimensional space; if the distance is less than a preset connection distance threshold, establishing a connection edge between each pair of non-empty mesh nodes; statistically analyzing the number of connection edges and the average connection length between all non-empty mesh nodes to form a topological connectivity set; and encoding the node density distribution spectrum and the topological connectivity set to generate a spatial configuration descriptor. Simultaneously, the trajectory morphology pattern of the feature vector of each grid node in the gridded behavior feature map is extracted to form a trajectory pattern descriptor. Combining the spatial configuration descriptor and the trajectory pattern descriptor, a comparison and matching is performed with a pre-defined behavior category knowledge base, assigning one or more behavior category labels to each behavior event unit. Throughout the entire pre-defined observation period, all labeled behavior event units are categorized according to their behavior category labels. For each behavior category, the duration of all behavior event units under it is accumulated to obtain the cumulative occurrence duration of that behavior category; simultaneously, the average displacement of all grid nodes in the corresponding gridded behavior feature map is calculated among all behavior event units belonging to that behavior category to obtain the average displacement of the grid nodes. Integrating the cumulative occurrence duration of all behavior categories and the average displacement data of the grid nodes generates structured behavior analysis results.

[0033] In some embodiments, the spatial distribution density and topological connectivity of non-empty grid nodes in the behavioral template mesh are analyzed to form a spatial configuration descriptor. Specifically, in the behavioral template mesh, the system filters out grid nodes containing coordinate assignment records; these filtered grid nodes are defined as non-empty grid nodes. The system calculates the centroid position of the three-dimensional coordinate set of all non-empty grid nodes. The calculation of the centroid position involves the arithmetic mean of the coordinates of all non-empty grid nodes. Using the calculated centroid position as the center, the system sets a series of incremental radius values. In each radius The number of non-empty grid nodes falling into the defined spherical space is counted. All radii and the number of their corresponding nodes The system constructs a node density distribution spectrum to describe node aggregation. Simultaneously, it checks the straight-line distance between each pair of non-empty mesh nodes in 3D space. If the straight-line distance between a pair of non-empty mesh nodes is less than a preset connection distance threshold, the system establishes a virtual connection edge between these pairs. The system counts the total number E of connection edges established between all non-empty mesh nodes and calculates the average geometric length of all connection edges. The number of connected edges E and the average connection length Together they form a set of topological connections.

[0034] Understandably, the system also needs to extract the trajectory morphology pattern of the feature vector of each grid node in the gridded behavioral feature map to form a trajectory pattern descriptor. The extraction of the trajectory pattern descriptor focuses on the average motion trajectory vector carried by each non-empty grid node. The system analyzes the direction, amplitude, and time-series variation patterns of behavioral events. Combining spatial configuration descriptors and trajectory pattern descriptors, the system compares and matches these with a pre-defined behavior category knowledge base. This knowledge base stores reference ranges or feature models of spatial configuration descriptors and trajectory pattern descriptors corresponding to different typical behavior categories. The system assigns one or more behavior category labels to each processed behavioral event unit by calculating similarity or applying a classifier, such as "grooming," "climbing," "exploring," "stillness," and "scratching."

[0035] In some embodiments, the system categorizes and statistically analyzes all marked behavioral event units within a preset observation period. For each behavioral category, the system accumulates the durations of all behavioral event units belonging to that category to obtain the cumulative duration of that behavioral category. Simultaneously, the system calculates the average displacement of all grid nodes within the corresponding gridded behavior feature map for all behavior event units belonging to this behavior category. The calculation of the average displacement can be expressed as: ; in: This represents the average displacement of the grid nodes, and M represents the total number of non-empty grid nodes in all gridded behavioral feature maps belonging to this behavioral category. This represents the average motion trajectory vector of the m-th non-empty grid node. This represents the magnitude of the vector, i.e., the displacement amplitude. The system integrates the cumulative occurrence duration of all behavior categories. Average displacement of grid nodes The data is used to generate structured behavioral analysis results, which can be clearly presented in tabular form. See Table 1.

[0036] Table 1: Statistical Table of Behavioral Categories within the Preset Observation Period Behavior category tag Cumulative duration (seconds) Average displacement of grid nodes (pixels / second) Grooming 45.2 1.8 Climbing 120.5 15.3 explore 312.7 8.4 still 86.1 0.2 Scratching 38.6 4.7 Optionally, the structured behavioral analysis results may also include the frequency of occurrence of each behavioral category, sequence information, or description of its temporal correlation with other categories, thereby comprehensively characterizing the behavioral patterns of mice and rats during the observation period.

[0037] See Figure 4 The study presents the dynamic variation of the average displacement of grid nodes over time (0-300 seconds) for different behavior categories (climbing, exploring, and remaining stationary) in Phase 6, while also showcasing the contrast between the original data and the smoothed curve. Specifically, the blue series of curves in the figure represent the displacement characteristics of "climbing behavior": its raw data (light blue) shows significant high-frequency fluctuations, with an average displacement peak close to 20 pixels / second, while the smooth curve (dark blue) reflects the overall high dynamism of the grid node displacement under this behavior (average of about 15 pixels / second), which is highly consistent with the average displacement statistics of the "climbing" category in the table (15.3 pixels / second); the green series of curves correspond to "exploration behavior," with the raw data (light green) showing lower fluctuations than climbing behavior, and the average displacement of the smooth curve (dark green) remaining stable in the 8-9 pixel / second range, matching the statistics of the "exploration" category (8.4 pixels / second); the red series of curves represent "stationary behavior," with the displacement values ​​of both its raw data and the smooth curve approaching 0 pixels / second, consistent with the statistics of the "stationary" category (0.2 pixels / second), reflecting the extremely low activity of node movement under this behavior. By analyzing the fluctuation characteristics of the raw data and the trend features of the smooth curve, the time-varying differences in grid node displacement under different behavior patterns are intuitively depicted, providing a visualized quantitative basis for the dynamic identification and pattern analysis of behavior categories.

[0038] In one embodiment of the present invention, a frame-by-frame pixel value analysis is performed on the original video stream to identify frames with a pixel change rate lower than a preset static threshold. Specifically, consecutive video frames are read sequentially from the original video stream, and the sum of the absolute differences between corresponding pixels in two adjacent frames is calculated. The calculated sum of absolute differences is compared with a preset static threshold. If the sum of absolute differences is lower than the preset static threshold, the next frame is determined to be a frame with a pixel change rate lower than the preset static threshold. The method detects whether there are pseudo-motion regions within a frame that are highly consistent with the environmental background pattern, and marks frames containing pseudo-motion regions as environmental noise frames. Specifically, it establishes a reference background model for the original video stream, which is obtained by statistically modeling the pixel values ​​of all video frames within an initial time period. For each subsequent video frame, it is divided into multiple image blocks, and the texture and color feature vectors of each image block are extracted. The similarity between the feature vector of each image block and the feature vector at the corresponding position in the reference background model is calculated. If the similarity of an image block is higher than a preset background similarity threshold, the image block is determined to be an environmental background pattern region. The number of image blocks in a frame that are determined to be environmental background pattern regions is counted. If the number of image blocks exceeds a preset region number threshold, the frame is determined to be an environmental noise frame containing pseudo-motion regions.

[0039] In some embodiments, a frame-by-frame pixel value analysis is performed on the original video stream to identify frames with a pixel change rate lower than a preset static threshold. The system reads consecutive video frames sequentially from the original video stream. For two adjacent frames, the absolute difference in color or brightness values ​​of all corresponding pixels between them is calculated, and the absolute difference values ​​of all pixels are summed to obtain a cumulative difference value representing the overall change between frames. This calculation process can be expressed in the following form: ; in: This represents the calculated cumulative difference, where P and Q represent the width and height of the video frame in pixels, respectively. This indicates the absolute value operation. and These represent the coordinates in the current frame and the previous frame, respectively. The intensity value of each pixel. The system will calculate the accumulated difference. The difference is compared with a preset static threshold used to determine whether the image is still. If the pixel change rate is lower than the preset static threshold, the current video frame is determined to be a frame with a pixel change rate lower than the preset static threshold.

[0040] It is understandable that detecting whether there are pseudo-motion regions within a frame that are highly consistent with the environmental background pattern is the key to identifying environmental noise frames. Its implementation includes establishing a reference background model for the original video stream. The establishment of the reference background model is obtained by statistically learning the pixel values ​​of all video frames in an initial period of the original video stream, for example, using a Gaussian mixture model to generate a stable background image representation. In specific implementation, the method of establishing a reference background model using a Gaussian mixture model includes: First, the system selects an initial time period from the beginning of the original video stream, during which there is no activity of large or small animals or only minimal movement, ensuring a clean background; then, for each pixel position in each video frame within the initial time period, the system collects the pixel intensity value sequence at different time points; next, the system independently fits a Gaussian mixture model for each pixel position, which is composed of a linear combination of multiple Gaussian distribution components, each component representing a background state; the model parameters, including the mean, variance, and weight of each Gaussian component, are iteratively optimized using the expectation-maximization algorithm, so that the model can probabilistically describe the multimodal distribution of pixel intensity; after the model training is completed, for each pixel in subsequent video frames, its current intensity value is compared with the probability density calculated by the Gaussian mixture model at that pixel position. If the probability is lower than an adaptive threshold, the pixel is determined to belong to foreground motion; otherwise, it belongs to the background; finally, the background models of all pixels together constitute a dynamically updated reference background representation. For each subsequent video frame requiring processing, the system spatially divides the video frame into multiple rectangular image blocks of uniform size. For each image block, a feature vector representing its texture features and color statistical histogram is extracted. In practice, the system calculates the similarity between the feature vector of each image block and the feature vector corresponding to the same spatial location in the reference background model. The similarity calculation can use the histogram intersection method or the correlation coefficient method. If the similarity calculation result of an image block is higher than a preset background similarity threshold, the image block is determined to be a region highly consistent with the environmental background pattern, i.e., a pseudo-motion region. The system then counts the total number of image blocks identified as pseudo-motion regions in the entire video frame. When the total number of image blocks exceeds a preset region number threshold, the video frame is determined to be an environmental noise frame containing pseudo-motion regions.

[0041] In some embodiments, the pixel change rate analysis module and the environmental noise detection module operate in parallel, outputting an invalid motion frame marker list and an environmental noise frame marker list, respectively. The system integrates these two lists, recording the position indices of all marked frames on the timeline of the original video stream in a set of indices to be removed. Optionally, the system removes all video frame data blocks corresponding to the index positions from the data buffer of the original video stream based on the set of indices to be removed. In a specific implementation, after the removal operation is completed, the system reassembles and serializes the remaining video frame data blocks according to their original time order and timestamp information, thereby forming a continuous, noise-filtered original video stream for subsequent processing.

[0042] See Figure 5 In the entire process of analyzing the characteristics and behaviors of mice and rats, the relationship between processing efficiency (%) and processing time (minutes) at different stages of the analysis was presented using a dual-axis visualization method. Specifically, the processing efficiency of the original video frame preprocessing stage was approximately 92%, with a processing time of 1.25 minutes; the processing efficiency of the joint coordinate extraction stage dropped to 88%, but the processing time increased to 3.6 minutes, showing an inverse fluctuation characteristic between efficiency and time; the processing efficiency of the behavior event segmentation stage rebounded to 95%, with the processing time stabilizing at 2.0 minutes, reflecting the high resource utilization efficiency of this stage; the processing efficiency of the mesh feature construction stage dropped back to 90%, with the processing time remaining at 2.0 minutes, reflecting the resource consumption characteristics of feature mapping operations; the processing efficiency of the behavior classification and labeling stage dropped to 85%, with the processing time increasing to 3.2 minutes, which is related to the increased complexity of classification matching; the processing efficiency of the results statistical analysis stage rebounded to 93%, with the processing time reaching 3.5 minutes, corresponding to the increased resource requirements of statistical operations.

[0043] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0044] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for analyzing the characteristic behaviors of mice and rats based on AI and deep learning, characterized in that: The method includes: Collect raw video streams containing information on mouse and rat activities, extract the set of trunk joint coordinates and motion direction vectors of mice and rats in each video frame, and generate joint motion time series data; The system receives the joint motion timing data and segments the time axis according to the abrupt changes in the motion direction vector and the break points of the continuous joint trajectory, thereby segmenting independent behavioral event units. For each behavior event unit, a standardized behavior template mesh is constructed based on its internal set of joint coordinates, and all motion trajectories within the behavior event unit are mapped to the corresponding nodes of the behavior template mesh to form a meshed behavior feature map. Based on the spatial configuration and node trajectory pattern of the behavior template grid, the gridded behavior feature map is classified and labeled, and the cumulative occurrence duration and average displacement of grid nodes of each labeled category within a preset observation period are statistically analyzed, and finally, structured behavior analysis results are output.

2. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 1, characterized in that, The method further includes: performing an initial cleaning operation on the original video stream to remove environmental noise frames and invalid motion frames, specifically including: The original video stream is subjected to frame-by-frame pixel value analysis to identify frames with a pixel change rate lower than a preset static threshold, which are marked as invalid motion frames. At the same time, it is detected whether there are pseudo-motion regions within the frames that are highly consistent with the environmental background pattern, and frames containing such pseudo-motion regions are marked as environmental noise frames. On the timeline of the original video stream, the positions of the frames marked as invalid motion frames and environmental noise frames are recorded as elimination indices. According to the elimination indices, all corresponding frames are removed from the original video stream, and the remaining video frames are reassembled in their original time order to form a continuous original video stream.

3. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 1, characterized in that, The system receives the joint motion timing data and, based on abrupt changes in the motion direction vector and breaks in the continuous joint trajectory, segments the time axis to divide it into independent behavioral event units, including: The motion direction vectors of adjacent time points are sequentially read from the joint motion time series data, and the angle difference between each pair of adjacent vectors is calculated. When the angle difference exceeds a preset action change threshold, the adjacent time point is determined to be a motion direction change point, and its timestamp is recorded. At the same time, it is detected whether there are joint coordinate zero change segments in the joint motion time series data that exceed a preset duration threshold. The first and last time points of such segments are recorded as trajectory breakpoints. All motion direction change points and trajectory breakpoints are integrated, and the motion direction change points and trajectory breakpoints are used as cutting points on the time axis to divide the continuous joint motion time series data into multiple non-overlapping time segments. Each segment is defined as an independent behavioral event unit.

4. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 1, characterized in that, For each behavior event unit, a standardized behavior template mesh is constructed based on its internal set of joint coordinates, and all motion trajectories within the behavior event unit are mapped to the corresponding nodes of the behavior template mesh to form a meshed behavior feature map, including: For each independent behavioral event unit, all joint coordinates it encompasses are extracted, and the distribution boundary of all joint coordinates in three-dimensional space is calculated. Based on the distribution boundary, a three-dimensional cubic virtual space covering the distribution boundary is established, and this virtual space is uniformly divided into a specified number of tiny cubes along three orthogonal coordinate axes. Each tiny cube is defined as a grid node, and the set of all nodes constitutes the behavioral template grid. The joint coordinates at each moment within the behavioral event unit are assigned to the corresponding grid node of the behavioral template grid according to their spatial location. The average motion trajectory of all coordinate points assigned to each grid node during the entire behavioral event unit is calculated, and the average motion trajectory is used as the feature vector of the grid node. The feature vectors of all grid nodes together constitute a gridded behavioral feature map.

5. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 4, characterized in that, Based on the spatial configuration and node trajectory patterns of the behavior template mesh, the meshed behavior feature map is classified and labeled, and the cumulative occurrence duration and average displacement of each labeled category within a preset observation period are statistically analyzed. Finally, structured behavior analysis results are output, including: The spatial distribution density and topological connectivity of non-empty grid nodes in the behavior template grid are analyzed to form a spatial configuration descriptor. Simultaneously, the trajectory morphology pattern of the feature vector of each grid node in the gridded behavior feature map is extracted to form a trajectory pattern descriptor. The spatial configuration descriptor and the trajectory pattern descriptor are combined and matched against a preset behavior category knowledge base to assign one or more behavior category labels to each behavior event unit. Within the entire preset observation period, all labeled behavior event units are categorized according to their behavior category labels. For each behavior category, the duration of all behavior event units under it is accumulated to obtain the cumulative occurrence duration of the behavior category. Simultaneously, the average displacement of all grid nodes in the corresponding gridded behavior feature map is calculated for all behavior event units belonging to the behavior category to obtain the average displacement of the grid nodes. The cumulative occurrence duration of all behavior categories and the average displacement data of the grid nodes are integrated to generate structured behavior analysis results.

6. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 2, characterized in that, Perform frame-by-frame pixel value analysis on the original video stream to identify frames with a pixel change rate lower than a preset static threshold, including: Read consecutive video frames sequentially from the original video stream and calculate the sum of the absolute differences between corresponding pixels in two adjacent frames; compare the calculated sum of absolute differences with the preset static threshold; if the sum of absolute differences is lower than the preset static threshold, then determine that the next frame is a frame with a pixel change rate lower than the preset static threshold.

7. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 6, characterized in that, Detecting whether there are pseudo-motion regions within a frame that are highly consistent with the environmental background pattern, and marking frames containing such pseudo-motion regions as environmental noise frames, includes: A reference background model is established for the original video stream. The reference background model is obtained by statistically modeling the pixel values ​​of all video frames within an initial time period. For each subsequent video frame, it is divided into multiple image blocks, and the texture and color feature vectors of each image block are extracted. The similarity between the feature vector of each image block and the feature vector at the corresponding position in the reference background model is calculated. If the similarity of an image block is higher than a preset background similarity threshold, the image block is determined to be an environmental background mode region. The number of image blocks determined to be environmental background mode regions in a frame is counted. If the number of image blocks exceeds a preset region number threshold, the frame is determined to be an environmental noise frame containing pseudo-motion regions.

8. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 3, characterized in that, The motion direction vectors at adjacent time points are sequentially read from the joint motion time series data, and the angle difference between each pair of adjacent vectors is calculated, including: In the joint motion time series data, the three-dimensional spatial coordinates of a specified joint at two consecutive sampling time points are extracted; based on the three-dimensional spatial coordinates, the displacement vector of the specified joint between these two time points is calculated, and the displacement vector is the motion direction vector; the motion direction vectors of two adjacent time periods are obtained sequentially, and the cosine value of the angle between these two vectors is calculated; based on the cosine value of the angle, the angle difference is obtained by inverse trigonometric function operation.

9. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 4, characterized in that, For each independent behavior event unit, extract all joint coordinates it encompasses, and calculate the distribution boundary of all joint coordinates in three-dimensional space, including: Read all joint coordinate data generated by a behavior event unit throughout its entire time span, with each data point containing three-dimensional coordinate values; find the maximum and minimum values ​​of all data points in each of the three coordinate axes; define a three-dimensional spatial rectangular region with the minimum value as the lower bound and the maximum value as the upper bound in each coordinate axis direction, which is the distribution boundary.

10. The method for analyzing the characteristic behavior of mice and rats based on AI and deep learning according to claim 5, characterized in that, Analyze the spatial distribution density and topological connectivity of non-empty grid nodes in the behavioral template mesh to form a spatial configuration descriptor, including: In the behavior template grid, grid nodes containing coordinate assignment records are selected and defined as non-empty grid nodes; Calculate the centroid position of the three-dimensional coordinate set of all non-empty mesh nodes; using the centroid position as the center, count the number of non-empty mesh nodes in spherical spaces of different radii to form a node density distribution spectrum. Simultaneously, the straight-line distance between each pair of non-empty grid nodes in three-dimensional space is checked. If the distance is less than a preset connection distance threshold, a connection edge is established between each pair of non-empty grid nodes. The number of connection edges and the average connection length between all non-empty grid nodes are counted to form a set of topological connection relationships. The node density distribution spectrum and the set of topological connection relationships are encoded together to generate the spatial configuration descriptor.

Images