An animal behavior analysis system fusing spatial structure recognition and trajectory decoding

Abstract

The application discloses an animal behavior analysis system, equipment and storage medium fusing space structure recognition and trajectory decoding, and belongs to the technical fields of experimental animal behavior analysis, computer vision, digital image processing and intelligent scientific research software. The method constructs a unified decoding link around "video import-background extraction-experimental area recognition-mouse centroid tracking-behavior partition generation-index quantification-visualization output-batch management", and is compatible with two common behavior videos of open field and Y maze. The application can realize stable trajectory reconstruction and behavior index calculation under the conditions of uneven illumination, shadow reflection and unstable site boundary by combining bottom border structure recognition, blue ROI auxiliary correction, background difference, morphological denoising, missing frame filling, center / limb automatic segmentation and batch result export, and improves analysis efficiency, repeatability and traceability. The specified figure 2 is an abstract drawing.

Description

Technical Field

[0001] This invention relates to the interdisciplinary fields of experimental animal behavior analysis, computer vision, digital image processing, intelligent scientific research software, and biomedical engineering. Specifically, it relates to an automatic analysis method, system, device, and storage medium for mouse open field and Y-maze behavior based on structural constraint region recognition, background difference tracking, automatic generation of behavioral partitions, and visualization output of results. Background Technology

[0002] The Open Field Test (OFT) and the Y-Maze test are among the most commonly used behavioral paradigms in mouse studies, including neuroscience, pharmacology, stroke, and other pathophysiological research. They are widely applied to assess phenotypes such as spontaneous activity, exploratory behavior, anxiety-like behavior, spatial working memory, arm preference, and spontaneous alternation behavior. With the increasing scale of behavioral experiments and the continuous accumulation of video data, the use of computer vision methods for the automatic tracking, decoding, and quantitative analysis of behavioral videos has gradually become an important technical direction in behavioral research.

[0003] Current video analysis workflows for mouse behavior still rely heavily on manual observation, trajectory recording, or manual counting of entry times and dwell times. This is not only time-consuming and labor-intensive but also susceptible to factors such as differences in the experience of raters, subjective judgment bias, and insufficient consistency in repeated statistical analysis, making it difficult to meet the needs of large-scale, standardized, and highly reproducible scientific research analysis. To address these issues, some automated analysis software and video tracking algorithms have been used in open field and Y-maze experiments, but they still have many limitations in real-world scientific research scenarios.

[0004] First, existing automated analysis methods lack accuracy in extracting Regions of Interest (ROIs). Traditional thresholding methods typically rely on stable light-dark contrast, but in real-world experimental environments, open field substrates often contain shadows, reflections, cage wall textures, dirt noise, and slight camera tilt, making it difficult to accurately identify the boundaries of the effective activity area. Especially for the bottom four-sided bounded regions commonly found in open field videos, their boundaries do not always exhibit stable and clear grayscale differences. Relying solely on global thresholding, simple edge detection, or manual polygon drawing easily leads to pixel-level jagged edges, boundary breaks, or local overflow at the region edges, thus affecting subsequent trajectory constraints and statistical results.

[0005] Secondly, existing methods have weak anti-interference capabilities. Many algorithms heavily rely on fixed background assumptions, global brightness stability, or absolute frame difference methods for target detection. When affected by box reflections, shadows cast by upright animals, changes in ambient lighting, background texture interference, or local occlusion, they are prone to problems such as broken target contours, centroid drift, target loss, or incorrect tracking, thus reducing the accuracy and robustness of trajectory reconstruction and behavior quantification. Thirdly, for the Y-maze experiment, automatic segmentation between the central intersection area and the three arm regions remains a significant technical challenge. The Y-maze is an irregular geometric structure with relatively complex boundaries between its central area and the arms, and is easily affected by maze deformation, camera viewpoint shifts, local occlusion, and threshold fluctuations. Existing methods often encounter problems such as unclear boundaries between the central area and arm regions, misjudgment of trajectory crossing areas, and repeated or missed counts of arm entry events when determining arm regions. This leads to unstable statistical results for arm entry sequences, spontaneous alternation rates, and spatial working memory-related indicators, affecting the reliability of experimental conclusions.

[0006] Furthermore, existing technologies also have shortcomings in terms of analysis links and result output. On the one hand, different experimental paradigms usually rely on different scripts or different software for processing, lacking a unified video decoding framework suitable for open field experiments and Y-maze experiments. This makes it difficult to achieve batch processing, unified parameter management, and structured result archiving, hindering the formation of standardized and traceable behavioral analysis processes. On the other hand, many existing software programs only output a few macroscopic indicators such as total distance, average speed, or total dwell time. The data dimensions are relatively simple, lacking systematic analysis of dwell time in the central / peripheral areas, number of entries into each zone, trajectory maps, heatmaps, arm entry sequences, transition matrices, and higher-order state dynamics characteristics. This makes it difficult to meet the needs of complex behavioral phenotypic studies and large-scale cohort analyses.

[0007] Therefore, there is an urgent need for an automated video decoding method suitable for open field and Y-maze experiments. This method should be able to robustly identify the effective experimental area, accurately track mouse movement trajectories, and uniformly analyze and batch output multiple behavioral indicators under conditions of uneven lighting, unstable boundaries, complex backgrounds, and local interference. This would improve the automation, accuracy, repeatability, and traceability of behavioral experimental analysis. Summary of the Invention

[0008] To overcome the shortcomings of existing technologies, such as unstable experimental area recognition, broken trajectory, fragmented analysis links between open field and Y-maze, and insufficient traceability of results, this invention provides an animal behavior analysis system that integrates spatial structure recognition and trajectory decoding. This system enables unified import of two types of behavioral videos, robust recognition of experimental areas, reconstruction of mouse trajectories, regional behavior statistics, and batch output of results.

[0009] To achieve the above objectives, the present invention adopts the following technical solution.

[0010] Step S1: Video import and experimental information archiving.

[0011] Obtain the video file to be analyzed. The video can be a single video or a batch collection of multiple videos in a folder. Simultaneously read or create metadata such as experimental group, animal number, sampling time point, experimental date and experimental type to form a three-level index of video-animal-time point.

[0012] Step S2: Background extraction and frame preprocessing.

[0013] For each video, after a preset start skip time, multiple video frames are extracted, and median fusion is used to obtain the background image; and grayscale conversion, frame rate detection, analysis duration truncation, and pixel size to centimeter ratio conversion are performed as needed.

[0014] Step S3: Experimental area identification.

[0015] For open field videos, priority is given to identifying the effective motion region enclosed by the bottom four borders; for Y-maze videos, priority is given to identifying the overall Y-shaped maze outline; when the automatic identification results are unstable or the confidence level is insufficient, the ROI mask generated by the manually marked blue image is called for correction; finally, an effective motion region mask with the same size as the video frame is obtained.

[0016] Step S4: Mouse foreground detection and centroid tracking.

[0017] The current video frame is subtracted from the background image, and candidate foreground contours are extracted after thresholding and morphological processing. Mouse targets are then selected based on contour area and connectivity, and their centroid coordinates are calculated. The centroid coordinates in consecutive frames are concatenated to form a trajectory. When no target is detected in a certain frame, the position of the previous valid frame is used for filling or smoothing.

[0018] Step S5: Behavior partitions are generated automatically.

[0019] For open field videos, a central area mask is generated based on the center of the outer rectangle of the effective motion area according to a preset ratio, and the remaining areas are defined as peripheral areas. For Y-maze videos, a central intersection area mask is first constructed based on the maximum distance transformation value or the convexity defect of the contour, and then three arm areas are obtained through connected region analysis and automatically numbered according to the angular relationship.

[0020] Step S6: Quantify behavioral indicators.

[0021] Based on the trajectory coordinates, the total distance, average speed, session duration, dwell time in each area, and number of entries are calculated; among them, the open field experiment further outputs the proportion of the central area and the number of central entries; the Y maze experiment further outputs the arm entry sequence, the percentage of spontaneous alternation (SAP), and the inter-arm transition matrix.

[0022] Step S7: Visualize and export the results.

[0023] The motion trajectory is overlaid on the region outline with a red trajectory line, and the start and end points are marked as S and E respectively; a heat map is generated based on the coordinate density, a session statistics chart is generated based on the velocity and region sequence, and the results are exported as CSV files, image files, and report pages.

[0024] Step S8: Batch processing and result management.

[0025] The above steps are automatically repeated for multiple videos, and summary results are generated for different experimental groups and time points. At the same time, the ROI source, parameter configuration, result file path and analysis timestamp are recorded to enable result traceability and subsequent review.

[0026] Beneficial effects

[0027] 1. This invention provides a unified processing framework for two types of behavioral videos: open field and Y-maze. Compared with using different scripts or manual recording, it is more conducive to data consistency management in batch experiments.

[0028] 2. This invention combines structural constraint region identification with blue ROI-assisted correction, which can significantly reduce region misjudgment caused by uneven lighting, border reflection, viewing angle shift and background texture changes.

[0029] 3. This invention, through background median modeling, differential segmentation, morphological denoising, and missing frame padding, can maintain trajectory continuity and improve the stability of total distance and partition statistics even when the target is lost for a short time.

[0030] 4. This invention automatically provides statistics on the central and peripheral regions of an open field, and automatically provides statistics on the central region and the three arm regions of a Y-maze. It can also output the percentage of spontaneous alternation and the transition matrix, resulting in richer results.

[0031] 5. This invention can simultaneously output trajectory diagrams, heatmaps, zonal adjustment diagrams, session statistics graphs, and CSV result tables, facilitating paper graphing, result review, and standardized laboratory archiving.

[0032] 6. This invention can be extended to software systems, electronic devices, and computer-readable storage media, facilitating deployment in WSL, Linux workstations, laboratory servers, or graphical platforms. Attached Figure Description

[0033] The accompanying drawings described in this specification are only used to illustrate the technical solutions of the present invention. The structure, proportions, and layout in the drawings do not constitute a limitation on the scope of protection of the present invention. Equivalent substitutions, partial adjustments, and formal changes made to the drawings without affecting the technical effects and objectives of the present invention shall be deemed to fall within the scope of protection of the present invention.

[0034] Figure 1 This is an overall architecture diagram of an embodiment of the present invention, illustrating the relationship between video import, background extraction, experimental area identification, mouse tracking, behavior partitioning generation, index calculation, visualization output, and result management.

[0035] Figure 2 This is a flowchart of the experimental area recognition process according to an embodiment of the present invention, showing the recognition of the four borders at the bottom of the open field, the recognition of the outer contour of the Y-shaped maze, and the blue ROI-assisted correction process;

[0036] Figure 3 This is a flowchart of mouse foreground detection and centroid tracking according to an embodiment of the present invention, showing the processes of background subtraction, threshold segmentation, morphological denoising, contour screening, and missing frame filling;

[0037] Figure 4 This is a schematic diagram of the open field behavior decoding results according to an embodiment of the present invention, showing the division of the central / peripheral region, the red trajectory diagram, the heat map, and the velocity-region time series diagram;

[0038] Figure 5 This is a schematic diagram of the Y-maze behavior decoding results according to an embodiment of the present invention, showing the division of the central region and the three arm regions, the arm entry sequence, the spontaneous alternation statistics, and the transition matrix;

[0039] Figure 6 This is a schematic diagram of the batch aggregation and result export interface of an embodiment of the present invention, showing a CSV result table, grouped statistical charts, and experimental result traceability information;

[0040] Figure 7 The accompanying drawings, which illustrate the software deployment and summary of this invention, show the operating interface and multi-folder batch decoding process in a laboratory workstation or server environment. Detailed Implementation

[0041] The present invention will be further described below with reference to the accompanying drawings and specific embodiments. The following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. Based on the concept of the present invention, various modifications, substitutions, and improvements can be made by those skilled in the art without departing from the spirit and essence of the present invention, and all such modifications, substitutions, and improvements should fall within the scope of protection of the present invention.

[0042] Refer to the instruction manual appendix Figure 1-7To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly described below. The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited to the following embodiments.

[0043] like Figure 1 As shown, this embodiment provides an automatic behavioral video decoding system suitable for mouse open field and Y-maze experiments. The system generally includes a behavioral video file input module, a video import module, a background extraction module, an experimental area identification module, a mouse tracking module, a behavioral partitioning generation module, an index calculation module, a visualization output module, and a result management module, ultimately outputting structured behavioral results. These modules are sequentially connected along the video processing and behavioral decoding flow, forming a complete automated analysis chain from raw video input to behavioral quantification result output.

[0044] The system's input section includes behavioral video file input, used to receive raw video data acquired during experiments. The behavioral video file can be a single video file or a batch of video files from multiple folders; it can include behavioral video data from open field experiments, Y-maze experiments, or a mixture of both. The video files can use common video formats such as mp4, avi, and mov. This input section provides the raw data source for subsequent automatic identification of experiment types, extraction of background information, tracking of mouse trajectories, and calculation of behavioral indicators.

[0045] (1) The video import module is used to read, decode, and extract basic information from the input behavioral video file. The basic information may include frame rate, resolution, total number of frames, video duration, file name, file path, acquisition time, and experiment number. The function of this module is to convert the raw video data into a frame sequence that can be processed by subsequent algorithms, and to complete the registration and management of video metadata. Its implementation may include calling the video decoding library to read frame-by-frame images, building a video index, performing frame sampling, and detecting abnormal frames.

[0046] (2) Background extraction module, used to extract static background images from the input video for subsequent separation of moving targets from the background. This module generates the background by statistically modeling multiple time-point frames in the video, such as using median background, mean background, or other robust background estimation methods, to reduce the interference of the moving mouse target on the background modeling. The role of this module is to generate a background reference map that does not contain or weakens the foreground information of the mouse as much as possible, providing a basis for subsequent background subtraction, foreground detection, and trajectory extraction. Its implementation may include random frame median fusion, time-series background update, block background estimation, or adaptive background modeling methods.

[0047] (3) Experimental area identification module, used to identify the actual effective experimental activity area in the video, and call the open field area identification submodule or the Y-maze area identification submodule according to the experimental paradigm. The role of this module is to establish the spatial boundary of the effective movement area, avoid the interference of the mouse trajectory with the external background, shadows, edge clutter and non-experimental areas of the video, and improve the accuracy of subsequent behavior partitioning and index calculation.

[0048] The open field region identification submodule is used to identify the effective activity region in open field experiments. This submodule can perform boundary detection, contour fitting, and region filling on rectangular or near-rectangular restricted regions at the bottom of the video or in the main field of view to obtain the effective experimental region of the open field. For situations in actual experiments where there are bottom four-sided borders, uneven lighting, shadow interference, or unstable boundary brightness, this submodule can also combine edge enhancement, contour filtering, geometric constraints, and manual ROI correction to improve the robustness of region identification. Its function is to generate effective region masks suitable for open field trajectory constraints and center / periphery partitioning analysis.

[0049] The Y-maze region identification submodule is used to identify the effective activity regions in Y-shaped maze experiments. Based on the overall maze outline, the three-arm structure, and the geometric relationship of the central intersection area, this submodule performs contour extraction, region closure, and structure fitting to obtain a mask of the entire Y-shaped maze region. For cases where the boundaries between the central intersection area and the three arm regions are unclear, there is local occlusion, or threshold fluctuations, this submodule can further combine template constraints, contour regularization, and manual correction information to improve the accuracy of region segmentation. Its role is to provide a reliable spatial partitioning basis for arm entry determination, sequence statistics, spontaneous alternation calculation, and region transfer analysis.

[0050] In some implementations, the experimental region identification module may also receive pre-drawn auxiliary ROI information, such as a blue ROI mask or manually corrected boundaries, to supplement constraints, clip boundaries, or make local corrections to the automatically identified experimental regions, thereby improving the identification accuracy in complex scenes.

[0051] (4) Mouse tracking module, used to detect, locate, and reconstruct the trajectory of the mouse foreground target within the effective experimental area. This module can generate a candidate foreground region based on the difference between the current video frame and the background image, and then determine the mouse's position coordinates in each frame through steps such as threshold segmentation, morphological denoising, contour screening, and centroid extraction. If there is a temporary missing target, broken contour, or occlusion, this module can also perform missing frame filling, trajectory interpolation, or smoothing of adjacent frames to ensure the continuity and integrity of the trajectory. The role of this module is to obtain the continuous movement trajectory of the mouse over time, providing a basic coordinate sequence for subsequent behavior region determination and index calculation. Its implementation methods may include background subtraction, binary segmentation, connected component analysis, contour area screening, centroid estimation, trajectory smoothing, and interpolation compensation.

[0052] (5) The behavioral partitioning generation module is used to spatially divide the mouse's movement area in a behavioral sense based on the experimental type and the results of experimental area identification. For the open field experiment, this module can further divide the identified rectangular effective area into a central area and a peripheral area to analyze indicators such as the mouse's area preference, central area dwell time, and central area entry frequency in anxiety-related behaviors. For the Y maze experiment, this module can structurally divide the central intersection area and the three arm areas to form behavioral areas such as the central area, arm A, arm B, and arm C. The role of this module is to map simple positional trajectories into a sequence of regional states with behavioral significance, which facilitates subsequent statistics on the mouse's dwell time, entry frequency, and state transition relationships in different areas.

[0053] (6) The index calculation module is used to calculate various behavioral indicators based on mouse trajectory and behavioral partitioning results, and to generate structured data output. For the open field experiment, this module can calculate indicators such as total movement distance, average speed, stationary time, dwell time in the central area, dwell time in the peripheral area, number of times entering the central area, and dwell ratio in the central area; for the Y maze experiment, this module can calculate the number of times each arm enters, arm dwell time, arm entry sequence, number of spontaneous alternations, spontaneous alternation rate, region transition frequency, and other indicators related to spatial working memory and exploration preferences. The role of this module is to transform the spatiotemporal motion information in the video into standardized, comparable, and statistically analyzable behavioral quantification results. Its implementation methods may include distance calculation based on inter-frame displacement, state determination based on region mask, alternating behavior recognition based on sequence rules, and state transition matrix calculation based on adjacency statistics.

[0054] (7) Visualization output module, used to graphically display the calculated trajectory and behavioral indicators. In this embodiment, this module includes a trajectory graph submodule, a heatmap submodule, a regional statistical graph submodule, and a sequence analysis graph submodule. The trajectory graph submodule is used to plot the mouse's movement path within the effective experimental area as a trajectory graph, for example, using red trajectory lines to represent the movement route and marking the start and end points; the heatmap submodule is used to generate a heat distribution map based on the frequency or duration of the mouse's stay in different spatial locations, to intuitively reflect the activity preference areas; the regional statistical graph submodule is used to display indicators such as stay time, number of entries, and speed changes in the central / peripheral area or the central / three-arm area using bar charts, line charts, or other statistical graphs; the sequence analysis graph submodule is used to display results such as the arm entry sequence, spontaneous alternation events, regional state sequences, or state transition matrices in the Y maze. The purpose of this module is to transform abstract numerical results into visual outputs that are easy for researchers to understand and verify, improving the intuitiveness of the analysis process and the interpretability of the results. Its implementation can include image drawing, heat mapping, statistical chart generation, and sequence relationship graph generation.

[0055] (8) Result Management Module, used for exporting, storing, tracing, and batch summarizing the behavioral decoding results. In this embodiment, this module includes a CSV export submodule, a result tracing submodule, and a batch summarization submodule. The CSV export submodule exports behavioral indicators corresponding to a single video or multiple videos into a structured table file, facilitating subsequent statistical analysis and data archiving. The result tracing submodule records video source, parameter configuration, algorithm version, ROI version, processing time, and operator information to ensure traceability of the analysis process and verifiability of the results. The batch summarization submodule integrates the results of multiple videos, multiple experimental groups, or multiple batches, and outputs a summary table, grouped statistical results, or a batch analysis report. The purpose of this module is to achieve standardized management of behavioral video analysis results, improving the practicality and auditability of this invention in large-scale experiments, repeated experiments, and cross-batch experiments. Its implementation methods can include table file export, database archiving, log recording, batch statistical scripts, and version management strategies.

[0056] Through the collaborative work of the aforementioned modules, this invention can complete a full processing flow from behavioral video input, background extraction, experimental area identification, automatic mouse tracking, behavioral partition generation, index calculation to visualization output and result management, ultimately outputting structured behavioral results. These structured behavioral results can be trajectory maps, heatmaps, and index tables for a single experimental animal, or aggregated statistical results from multiple experimental subjects in the same or different batches. Compared to traditional analysis methods that rely on manual observation and counting, this invention offers advantages such as high automation, good adaptability to complex backgrounds and lighting changes, applicability to both open field and Y-maze experiments, richer result dimensions, and ease of batch processing and traceability.

[0057] In some implementations, the system can be further expanded with parameter configuration functions to set the number of background extraction frames, foreground threshold, contour area range, center area ratio, arm area division rules, interpolation strategies, and result export formats, etc., to adapt to different experimental equipment, different video qualities, and different scientific research task requirements. Through modular design, embodiments of the present invention can be deployed not only on laboratory workstations, but also on server environments or integrated into behavioral analysis software platforms, thereby achieving batch decoding of multiple folders, unified management, and continuous iterative updates.

[0058] like Figure 2 As shown, the experimental region identification process of this invention includes steps such as original video frame input, preprocessing and contour extraction, open field region identification branch, Y maze region identification branch, blue ROI auxiliary correction, and effective experimental region output. It is used to automatically extract the actual effective experimental activity region from complex background behavioral videos and provide a spatial constraint basis for subsequent mouse tracking and behavioral partitioning.

[0059] In this embodiment, the original video frame input is a single-frame image or a sequence of sampled frames read from a behavioral video file. This frame image can be a color image or it can be grayscale processed before being used as input for subsequent region recognition algorithms. The original video frame typically contains the main body of the experimental setup, the background desktop, lighting and shadows, edge reflections, cage wall textures, instrument markings, and other non-target areas. Therefore, it is necessary to preprocess and extract the contours to initially shape the image. Preprocessing may include grayscale transformation, smoothing filtering, contrast enhancement, edge enhancement, local threshold adjustment, noise suppression, or morphological pre-cleaning; contour extraction may include edge detection, binarization, connected component analysis, closing operation filling, contour tracking, and candidate region selection. The purpose of this step is to preserve as much of the experimental setup's boundary information as possible while suppressing background clutter from interfering with subsequent contour recognition.

[0060] After preprocessing, the system enters different region recognition branches based on the morphological characteristics of the experimental setup. For open field experiments, it enters the open field region recognition branch; for Y-maze experiments, it enters the Y-maze region recognition branch. In some implementations, the system can automatically or semi-automatically determine whether the current video belongs to an open field experiment or a Y-maze experiment based on video file naming rules, experimental batch labels, aspect ratio features, contour geometric features, or preset experiment type parameters.

[0061] For the open field region recognition branch, the system first performs boundary analysis on candidate contours to detect the bottom four-bordered confinement region located within the main video area. Since the bottom boundary of the open field often manifests as four relatively obvious but not entirely stable border lines in actual experiments, the system can combine edge detection results, rectangular geometric constraints, aspect ratio filtering, boundary closure degree, and connected region area to identify the rectangular or near-rectangular contours most likely to constitute the effective experimental region. Subsequently, the system performs boundary smoothing, corner correction, and region filling on the identified rectangular contours to generate an effective open field experimental region mask. This step ensures that even under conditions of shadow, reflection, unstable boundary contrast, or slight camera tilt, the system can still robustly identify the mouse's actual activity area, preventing the trajectory from exceeding the effective field or mistakenly falling into the background area.

[0062] For the Y-maze region identification branch, the system first detects candidate regions with Y-shaped topological features. Then, through contour closure, main branch direction analysis, central intersection location, and the contour extension relationship of the three arms, the overall shape of the Y-maze is identified. Subsequently, the system performs regularization processing on the identified Y-shaped contour, including boundary smoothing, local gap closure, internal filling, and region normalization, to form a complete and effective experimental region mask for the Y-maze. This step addresses the problem of unstable extraction of maze boundaries in Y-maze experiments due to factors such as shooting angle, contour blur, local occlusion, and lighting fluctuations, providing a reliable foundation for subsequent division of the central region and the three arm regions.

[0063] In this embodiment, Figure 2 The blue ROI-assisted correction step shown at the bottom is used to manually correct or constrain the automatic recognition results. Specifically, researchers can pre-draw the blue ROI region based on the outline of the experimental setup as the external boundary or reference template for the automatic recognition area. The system can then perform intersection calculations, boundary corrections, local cropping, or edge compensation between the automatic recognition results and the blue ROI to improve the accuracy and stability of the effective experimental area. This step is particularly suitable for scenarios with complex backgrounds, incomplete boundaries, obvious light spots, or local defects in the experimental setup outline.

[0064] Finally, the system outputs a valid experimental region mask. For open field experiments, the mask typically represents a regular or near-regular rectangular region; for Y-maze experiments, the mask typically represents a Y-shaped region with a central intersection and three extending arms. This output mask can be directly used as a spatial constraint for subsequent mouse foreground detection, or as the basic input for the behavior partitioning generation module. Through the above experimental region identification process, this embodiment of the invention can achieve unified and robust identification of open field and Y-maze experimental regions under complex background conditions, improving the reliability and repeatability of subsequent trajectory analysis and behavior quantification results.

[0065] like Figure 3 As shown, the mouse foreground detection and centroid tracking process in this embodiment of the invention includes steps such as current video frame input, background image input, background difference, threshold segmentation, morphological denoising, contour screening, mouse target region determination, centroid extraction, missing frame filling, and continuous trajectory sequence output, which are used to achieve automatic localization and trajectory reconstruction of mouse moving targets within the effective experimental area.

[0066] In this embodiment, the current video frame is the experimental image at the moment to be analyzed, and the background image is a static background reference image pre-constructed by the background extraction module. The system first performs background subtraction on the current video frame and the background image to highlight the foreground target area that has changed position relative to the static background. Since the mouse is the main moving target in the video, the background subtraction result can usually reflect the pixel change area of ​​the mouse's location well. The purpose of this step is to separate the mouse from the complex but relatively static experimental background and reduce the influence of fixed boundaries, substrate texture, and uniformly illuminated areas on target localization.

[0067] The background subtraction results proceed to the thresholding segmentation step. The system converts the subtracted image into a binary image based on a preset threshold, an adaptive threshold, or a local statistical threshold, clearly distinguishing the foreground target region from the background region. In practical applications, since noise points, boundary fragments, or residual reflections may still exist after background subtraction, the thresholding segmentation results are usually not entirely ideal. Therefore, further morphological denoising is required. Morphological denoising may include operations such as opening, closing, erosion, dilation, small region removal, hole filling, or edge smoothing to remove isolated noise points and enhance the coherence of the mouse's main outline.

[0068] After morphological denoising, the system performs contour filtering on the remaining foreground connected components. Specifically, the system can filter multiple candidate contours based on contour area, aspect ratio, roundness, position range, proximity to the centroid of the previous frame, or spatial relationship with the boundary of the effective experimental region, to determine the target contour most likely corresponding to the mouse subject. This step aims to avoid misidentifying shadows, reflective spots, edge interference blocks, or other non-mouse foreground elements as targets, thereby improving the stability of centroid localization.

[0069] After the target region for the mouse is determined, the system extracts the centroid of the target contour to obtain the mouse's position coordinates for that frame. The centroid can be the geometric center of the contour, the center of the smallest bounding rectangle, the centroid of the region, or other representative points. In some implementations, the system can also simultaneously record the target contour area, contour bounding box, instantaneous velocity, and orientation information to provide auxiliary parameters for subsequent behavioral analysis. The centroid coordinates of consecutive frames are then concatenated in temporal order to form the basic sequence of the mouse's movement trajectory.

[0070] In actual video analysis, a small number of frames may fail to consistently detect the mouse outline. This could be due to decreased contrast with the background caused by the mouse's brief stillness, or incomplete target outlines caused by local lighting changes, occlusion, or edge attachment. Therefore, this embodiment also includes a missing frame completion step. Specifically, when the system detects a missing target in a frame or a few consecutive frames, it can perform linear interpolation, adjacent frame extension, local smoothing compensation, or motion continuity-based estimation to restore the temporal continuity of the trajectory. This step aims to reduce the errors caused by trajectory breaks in distance calculation, velocity estimation, and region state determination.

[0071] Finally, the system outputs a continuous trajectory sequence. This trajectory sequence can be represented as a set of position coordinates ordered by frame number or timestamp, or it can be further converted into a trajectory graph, velocity time series, and region state sequence. Through the aforementioned foreground detection and centroid tracking process, this embodiment of the invention can robustly reconstruct the continuous movement trajectory of mice under conditions of complex backgrounds, local noise, boundary interference, and a small number of missing targets, providing reliable underlying positional information for unified automatic decoding of open field and Y-maze experiments.

[0072] like Figure 4 As shown, the open field behavior decoding results of this embodiment of the invention include a schematic diagram of the division of the central / peripheral area, a red trajectory diagram, a heat map, and a velocity-region time series diagram, which are used to comprehensively display the spatial exploration behavior, activity distribution characteristics, and temporal dynamics of mice in the open field experiment.

[0073] In this embodiment, a central and peripheral area is first generated based on the identified effective experimental area of ​​the open field. Specifically, the system can generate a rectangular or near-rectangular central area in the central region according to a preset ratio based on the length and width dimensions of the overall open field area, and the remaining area around it is defined as the peripheral area. In some implementations, the ratio of the central area can be set according to experimental needs, such as being generated according to a fixed ratio of the overall area, or it can be personalized according to user settings. The purpose of this step is to map the spatial position of the mouse to a regional state with behavioral significance, thereby enabling the analysis of anxiety-related phenotypes and exploration preferences.

[0074] Based on the above partitioning results, the system can generate a red trajectory map. This trajectory map uses the effective experimental area of ​​the open field as a background and draws the mouse's movement trajectory lines using the continuous centroid coordinates of the mouse in each frame. To enhance readability, the trajectory lines can be displayed in red, with "S" marked at the start position and "E" at the end position. This trajectory map visually displays the distribution of the mouse's movement paths, path density, and exploration preferences in the central / peripheral areas throughout the entire experimental period. This section aims to provide researchers with easily understandable and manually verifiable visualizations of movement paths.

[0075] The system further generates heatmaps based on the cumulative frequency or duration of mouse stays at various spatial locations. These heatmaps divide the open area into multiple spatial grids, statistically analyzing the cumulative time spent, number of visits, or location density of mice in each grid, and displaying this data using different grayscale or heat intensities. The purpose of the heatmaps is to reflect the mouse's activity hotspots, avoidance areas, and overall exploration bias from a spatial statistical perspective. For example, if a mouse mainly moves in the peripheral area, the peripheral area will show higher heat intensity; if a mouse frequently enters the central area, the central area will show higher heat intensity.

[0076] In this embodiment, the system can also generate a velocity-region time series graph. This graph uses time as the horizontal axis and velocity or displacement intensity as the vertical axis, while overlaying the mouse's regional state information at the corresponding time point, such as whether it is in the central region or the peripheral region. Specifically, a line graph can be used to represent velocity changes, and different colored bands, shading, or symbols can be used to identify regional states within the graph. The purpose of this graph is to reflect the dynamic changes in mouse activity intensity over time and to reveal the coupling relationship between velocity changes and regional preferences. For example, it can analyze whether the mouse slows down when entering the central region and whether it exhibits strong edge-adhering behavior during the rapid movement phase in the periphery.

[0077] Based on the aforementioned trajectory and zoning information, the index calculation module in this embodiment can output multiple structured indicators related to the open field experiment, including but not limited to total movement distance, average speed, maximum speed, stationary time, movement time, dwell time in the central area, dwell time in the peripheral area, dwell ratio in the central area, number of entries into the central area, number of entries into the peripheral area, and time-segmented behavioral statistics. In some implementations, the system can further output movement distance changes by time bins, speed distribution by region, or exploration uniformity indicators based on spatial grids. Figure 4 The various result representation methods shown in this invention enable a unified display of motion trajectories, spatial preferences, and temporal dynamic behaviors in open field experiments, thereby improving the completeness and interpretability of behavioral decoding results.

[0078] like Figure 5 As shown, the Y-maze behavior decoding results of this embodiment include a schematic diagram of the division of the central region and three arm regions, arm entry sequence, spontaneous alternation statistics, and transition matrix, which are used to quantify the spatial working memory, exploration order, and regional state transition behavior of mice in the Y-maze experiment.

[0079] In this embodiment, the system first performs structured behavioral partitioning based on the identified effective experimental area of ​​the Y-maze. Specifically, the central intersection of the Y-maze is defined as the central region, and the maze passages extending in three directions are defined as arms A, B, and C, respectively. The boundaries between the central region and each arm region can be determined based on the geometric center, arm axis direction, arm width range, and the topological relationship of the intersection area. In some implementations, the length and width of each arm region can be calibrated according to the actual size of the maze or a normalized ratio. The purpose of this step is to convert continuous position trajectories into discrete region state sequences, providing a spatial basis for arm entry determination and alternating behavior statistics.

[0080] After region segmentation, the system generates an arm entry sequence based on the mouse's position coordinates over time. Specifically, the system records an arm entry event when the mouse's centroid enters a specific arm region and meets preset entry criteria. For example, when the mouse's centroid enters arm A from the central region and remains there for more than a set threshold, it is recorded as an arm A entry; subsequent arm B and arm C entry events are generated similarly. By arranging each arm entry event in chronological order, the system can form a complete arm entry sequence, such as "A→B→C→A→C→B". This step serves to provide a basic event sequence for subsequent spontaneous alternation analysis and region transfer analysis.

[0081] Spontaneous alternation statistics are calculated based on the arm entry sequence. In this embodiment, three consecutive entry events corresponding to three different arms are counted as one valid spontaneous alternation. For example, "ABC", "BCA", and "CAB" are all considered valid alternations, while "ABA" and "ACC" are not counted as valid alternations due to the presence of duplicate arms. The system further calculates the spontaneous alternation rate based on the number of valid alternations and the total number of entries. In a typical implementation, the spontaneous alternation rate can be calculated as "number of valid alternations / (total number of entries - 2) × 100%". This indicator reflects the mouse's spatial working memory level and continuous exploration strategy in the Y maze.

[0082] In addition to the spontaneous alternation rate, the system can also generate a region transition matrix to statistically analyze the frequency or probability of mouse transitions between different regions. The transition matrix uses the central region, A-arm, B-arm, and C-arm as state nodes, with each cell representing the number or proportion of transitions from one region to another. This transition matrix helps to more systematically characterize the mouse's state transition patterns, re-entry preferences, and exploration path structure in the Y-maze. In some implementations, the transition matrix can be further normalized into a transition probability matrix and used for subsequent Markov chain analysis, behavioral state modeling, or inter-group comparisons.

[0083] In this embodiment, Figure 5 The system can further visualize the trajectory lines superimposed on the Y-maze region to show the mouse's round-trip path between the three arms. The system can output structured Y-maze metrics including, but not limited to, the number of entries into each arm, the dwell time in each arm, the dwell time in the central region, the total number of entries, the number of effective alternations, the spontaneous alternation rate, the arm preference index, the number of repeated entries, and the region transition matrix. Figure 5 The comprehensive decoding results shown in this embodiment of the invention can more completely reflect the exploration order, spatial preference and memory-related behavioral characteristics of mice in the Y maze experiment, and improve the automation level and data richness of traditional manual statistical methods.

[0084] like Figure 6 As shown in the figure, the batch summary and result export interface of this invention includes a batch task list area, a CSV result table area, a grouped statistical chart area, and an experimental result traceability information area, which are used to support batch processing of multiple videos, export of structured results, statistical display between groups, and full-process traceability management.

[0085] In this embodiment, the batch task list area is used to display multiple video files that are currently pending, being processed, or have been processed. These video files may come from one or more folders and may include video data from multiple mouse samples, multiple experimental batches, and different experimental paradigms. Each file item can display the file name, experiment type, processing status, processing progress, or completion indicator. This area supports unified scheduling and batch automatic decoding of large-scale behavioral videos, improving the laboratory's processing efficiency for multiple animals and batches of experimental data. In some implementations, the system can support task queue management through methods such as "add folder," "batch import," "start analysis," and "pause task." The CSV results table area is used to display and export structured behavioral results. The system can organize the main analytical indicators corresponding to each video into table rows. The table header may include sample number, experiment type, total movement distance, average speed, central zone dwell time, peripheral zone dwell time, number of central zone entries, number of arm entries, arm entry sequence, spontaneous alternation rate, and other custom indicators. This area standardizes the behavioral results of each video and summarizes them into a software table output that facilitates statistical analysis. In some implementations, the system supports exporting the results as CSV, TXT, Excel, or other structured data files, facilitating further analysis using statistical software, graphing software, or machine learning tools.

[0086] The grouped statistical chart area is used to visualize and compare behavioral indicators across multiple samples, groups, or batches. This area can generate bar charts, box plots, line graphs, scatter plots, or error bar graphs to show differences between experimental groups in indicators such as total distance, time spent in the central zone, and spontaneous alternation rate. The purpose of this area is to further summarize structured indicators at the individual sample level into group-level statistical results, facilitating inter-group comparisons and experimental effect assessments by researchers. In some implementations, the system can also support stratified statistics by experimental date, treatment batch, treatment conditions, or folder origin.

[0087] The experimental results traceability information area is used to record metadata and process information related to the results of this analysis. This traceability information may include the experiment date, video path, video frame rate, algorithm version, parameter configuration, ROI version, background extraction settings, processing time, operator information, and output directory. The purpose of this area is to ensure that the analysis process is recordable, the results are verifiable, and the parameters are reproducible, thereby meeting the traceability requirements for behavioral science experimental data management and patent implementation. In some implementations, the system may further record log files, error messages, abnormal frame information, and analysis status history to enhance the system's auditing capabilities and engineering practicality.

[0088] pass Figure 6The batch aggregation and result export mechanism shown in this embodiment of the invention can not only complete the automatic decoding of a single video, but also support the unified processing, result aggregation and full-process traceability of multiple videos, multiple experimental groups and multiple batches of data, thereby significantly improving the standardization, automation and batch processing capabilities of behavioral experimental analysis.

[0089] like Figure 7 As shown in the accompanying drawings, the software deployment and abstract of this invention include a laboratory workstation or server environment, a software interface, a multi-folder batch decoding process, and result output, which are used to illustrate the deployment method and overall operation mechanism of this invention in a real scientific research environment.

[0090] In this embodiment, the system can be deployed in a laboratory workstation, desktop computer, graphics workstation, server environment, or other electronic device with video processing capabilities. The device may include a processor, memory, display terminal, and software program for executing the method of the present invention. The workstation or server environment provides computational resources for video reading, background modeling, region recognition, foreground detection, trajectory tracking, behavior partitioning, index calculation, and graphics output. In some implementations, the system can run independently in a local environment or be deployed on a local area network server or cloud node to achieve centralized behavioral data processing.

[0091] Figure 7 The software interface shown may include a video file list area, an experimental area preview area, a trajectory plot preview area, a results table area, and an operation control area. The video file list area displays the videos to be analyzed and their status; the experimental area preview area displays the automatically identified open field or Y-maze effective area; the trajectory plot preview area displays visualizations such as mouse movement trajectories and heatmaps; the results table area displays key behavioral indicators; and the operation control area may include function buttons for starting analysis, stopping analysis, exporting results, and parameter settings. The purpose of this software interface is to provide researchers with an intuitive and operable analysis platform, enabling the method of this invention to be used in the form of a software tool.

[0092] Regarding the multi-folder batch decoding process, the system can receive behavioral videos from multiple source folders and automatically perform experimental type identification, background extraction, mouse tracking, behavioral decoding, and result export. Specifically, the system can first read multiple open field videos and Y-maze videos according to the folder structure or file naming rules, then perform experimental region identification and behavioral decoding processes for each video one by one, and finally export the video results in a unified format as structured files and image results. The purpose of this process is to achieve automatic batch processing of large-scale video data, reduce the workload of manually processing videos one by one, and improve the efficiency of experimental data processing.

[0093] Regarding output, the system can output various formats of result files, including trajectory plots, heatmaps, statistical charts, CSV tables, sequence analysis results, and summary reports. Output results can be saved locally or uploaded to a server database or project management system for centralized management. In some implementations, the system can also automatically generate experimental batch reports based on the output folder and archive the analysis parameters, algorithm version, and result files together to ensure operability for subsequent review and reanalysis.

[0094] because Figure 7 This software combines software deployment diagrams and abstract figures, thus its overall structure emphasizes the core workflow of "multi-folder input - automatic experiment type identification - background extraction and mouse tracking - behavioral indicator decoding - result export." This figure summarizes the complete engineering implementation of this invention from raw video to structured behavioral results. Figure 7 The deployment scheme shown in this invention allows it to run stably in a real scientific research environment as a software system, enabling unified automatic decoding, batch analysis, and result management of open field and Y-maze experiment videos. It has good engineering applicability and promotional value.

[0095] Combination Figures 1 to 7 The system structure, experimental area identification process, mouse tracking process, open field behavior decoding results, Y-maze behavior decoding results, batch aggregation and result export mechanism, and software deployment method shown in this embodiment of the invention construct a unified automatic behavioral video decoding method and system suitable for mouse open field experiments and Y-maze experiments. This method and system can achieve robust analysis and structured output of mouse behavioral videos under conditions of complex backgrounds, unstable boundaries, uneven lighting, and batch video processing, thereby significantly improving the automation level, accuracy, data richness, and traceability in behavioral research.

[0096] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An animal behavior analysis system integrating spatial structure recognition and trajectory decoding, characterized in that, The process includes the following steps: S1, acquiring the behavioral videos of mice to be analyzed and establishing an association index between video files, experimental groups, animal numbers, time points, and experimental types; S2, performing frame sampling on the videos, extracting background images, and completing grayscale conversion, time clipping, and frame rate correction; S3, identifying effective motion regions according to the experimental type, wherein for open field videos, the experimental region defined by the bottom four borders is identified, and for Y-maze videos, the entire Y-shaped maze region is identified, and manually labeled blue ROIs are used for correction when the automatic identification confidence is insufficient; S4, extracting the mouse foreground within the effective motion region based on background subtraction, threshold segmentation, morphological processing, and contour filtering, calculating the centroid coordinates of each frame, and constructing a continuous trajectory; S5, automatically generating behavioral partitions according to the experimental type, wherein a central region and a peripheral region are generated for open field videos, and a central region and three arm regions are generated for Y-maze videos; S6, calculating behavioral indicators based on the continuous trajectory and behavioral partitions, including at least one or more of the following: total movement distance, average speed, dwell time in each partition, and number of entries. S7 outputs trajectory graphs, heatmaps, partition diagrams, session statistics graphs, and result tables, and marks the starting point as S and the ending point as E; S8 performs batch processing and result aggregation on multiple videos, generating a traceable experiment report.

2. The method according to claim 1, characterized in that, The background image in step S2 is obtained by extracting video frames from multiple moments after a preset skip duration from the start of the video and performing median fusion; the preset skip duration is used to avoid the initial segments where the experimenter placed the animal or the scene has not yet stabilized; the frame rate correction is used to replace the original frame rate with a default frame rate when the original frame rate of the video is abnormal or missing.

3. The method according to claim 1, characterized in that, The effective motion region identification in step S3 employs at least one or more of the following methods: identifying the four-sided border structure at the bottom of the open field based on edge detection and line detection; identifying the outer contour of the Y-shaped maze based on contour extraction, convex hull analysis, or connected component analysis; reading the debug image containing blue annotations and extracting the blue ROI mask through color space thresholding; performing morphological closing operations or size resampling on the extracted mask to obtain an effective motion region consistent with the video frame.

4. The method according to claim 1, characterized in that, The mouse foreground extraction in step S4 includes: converting the current video frame into a grayscale image, performing a difference operation with the background image, performing threshold binarization on the difference result, performing an opening operation or denoising operation within the effective motion area, and filtering the target contour according to the contour area threshold; when no effective target is detected in the current frame but there are effective centroid coordinates in the previous frame, the effective centroid coordinates of the previous frame are used to fill in the position of the current frame to reduce trajectory breakage.

5. The method according to claim 1, characterized in that, In step S5, when generating behavioral partitions for the open field video, the bounding rectangle of the effective motion area is first obtained. The center rectangle area is generated according to the preset center ratio with its geometric center as the center point, and the area outside the center rectangle area within the effective motion area is defined as the peripheral area. In step S6, the behavioral indicators calculated for the open field video include at least the total distance, average speed, dwell time in the center area, dwell time in the peripheral area, proportion of the center area, and number of times the center area is entered.

6. The method according to claim 1, characterized in that, In step S5, when generating behavioral partitions for the Y-maze video, a central region mask is constructed based on the location of the maximum distance transformation or based on the convex defects of the main contour. Then, after deducting the central region from the overall Y-shaped maze region, multiple connected regions are obtained. The three connected regions with the largest areas are selected as the three arm regions, and they are sorted and marked according to the angular relationship between the centroid of each arm region and the central region. In step S6, the behavioral indicators calculated for the Y-maze video include at least the central region dwell time, the dwell time of each arm region, the number of times each arm region is entered, the percentage of spontaneous alternation (SAP), the arm entry sequence, and the inter-arm transition matrix.

7. The method according to claim 1, characterized in that, The trajectory map described in step S7 uses red trajectory lines to display the mouse's movement path, and the starting point S and the ending point E are marked with text in a background color; the heatmap is generated by a two-dimensional position histogram and Gaussian smoothing; the result table can be exported in the form of CSV, image file, web interface or report document.

8. An animal behavior analysis system integrating spatial structure recognition and trajectory decoding, characterized in that, include: The system comprises a video and experimental information import module, a background extraction and frame preprocessing module, an experimental region identification module, a mouse foreground detection and centroid tracking module, a behavior partitioning generation module, an index calculation module, a visualization output module, and a result management module; wherein each module collaboratively executes the method described in any one of claims 1 to 7.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images