A method and system for mapping social behavior and neural circuit of zebrafish based on spatiotemporal dynamic graph and hidden variable causal inference
Patent Information
- Application Number
- CN202611378991.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-07
- Publication Date
- 2026-10-27
AI Technical Summary
[0042]突破单鱼行为局限，实现群体高阶拓扑量化：通过引入时空动态图注意力网络，将多鱼交互从简单的“距离计算”升维到“图拓扑计算”，能够精准捕捉斑马鱼群体中复杂的动态社会层级和交互模式，为精神类药物筛选提供更丰富的行为学表型指标。
Smart Images
Figure CN122988016A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of neural information processing technology, specifically a method and system for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference. Background Technology
[0002] Zebrafish, due to their high genetic homology with humans and the transparent bodies of their juveniles, have become a core model organism for high-throughput screening of neuroscience and psychotropic drugs. Research typically requires simultaneous observation of both macroscopic social behaviors (such as gathering, chasing, and escaping) and microscopic neural activities (such as whole-brain calcium imaging signals) in zebrafish to establish a mapping relationship between neural circuits and behavior.
[0003] Existing technical solutions mainly include the following three categories:
[0004] 1. Currently, mainstream commercial software (such as EthoVision and ZebraLab) or open-source tools (such as idtracker.ai and SLEAP) mainly extract individual macroscopic kinematic parameters of zebrafish, such as centroid coordinates, swimming speed, and turning angle, based on background subtraction, blob detection, or deep learning pose estimation. For multi-fish interactions, they usually only calculate simple "nearest neighbor distance" or "number of times entering a specific area".
[0005] The technical solution has the following drawbacks: low feature dimensionality, loss of topological information, reducing complex social behavior to simple distance and speed statistics, and failing to quantify high-order social topological features such as "group hierarchy" and "information transmission direction"; lack of spatiotemporal context, as traditional methods are mostly frame-by-frame analysis or simple sliding window statistics, which cannot capture the dynamic evolution patterns of social behavior over a longer time scale.
[0006] 2. After acquiring whole-brain calcium imaging and behavioral data, existing studies typically calculate the Pearson correlation coefficient between the fluorescence signal of each neuron (or brain region) and behavioral parameters, or use a linear decoder / support vector machine to predict behavior. More advanced methods use Granger causality tests or transitivity to analyze the temporal relationships of the data.
[0007] This technical solution has the following drawbacks: "Correlation does not equal causation." Pearson correlation and machine learning decoding can only prove that "neural activity and behavior occur simultaneously," but cannot prove that "the neural activity caused the behavior." Granger causation is essentially "predictive in time" rather than true physical / biological causation, and it has extremely poor robustness to nonlinear relationships and unobserved variables. There are serious false positives. Because latent variables (such as fish fatigue) are not considered, traditional correlation analysis will misjudge "spurious correlations" caused by latent variables as "direct causation," leading to a completely wrong analysis of the neural mechanism.
[0008] 3. In recent years, constraint-based methods (such as the PC algorithm), score-based methods (such as the GES algorithm), and continuous optimization-based methods (such as NOTEARS) have begun to be used to construct gene regulatory networks or causal networks of brain waves.
[0009] The technical solution has the following drawbacks: it cannot handle latent variables. Classic causal discovery algorithms are all based on the "causal sufficiency" assumption (i.e., the assumption that there are no latent variables). However, this assumption does not hold in live animal experiments, and direct application will produce a large number of erroneous causal edges. It is difficult to integrate high-dimensional heterogeneous data. Neural signals (high frequency, high dimension, continuous calcium imaging time series) and social behaviors (low frequency, low dimension, discrete or continuous kinematic features) have huge differences in data distribution and sampling rate. Traditional causal algorithms are difficult to directly handle such cross-modal and cross-scale data alignment and joint modeling. Summary of the Invention
[0010] The purpose of this invention is to overcome or at least partially solve the above problems by proposing a method and system for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference.
[0011] To achieve the above objectives, the present invention adopts the following technical solution: The present invention provides a method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference, comprising the following steps:
[0012] S1: Synchronous acquisition and preprocessing of multimodal spatiotemporal data
[0013] Macroscopic behavioral videos and microscopic neural signal videos of multi-tailed zebrafish are acquired, and the macroscopic behavioral videos and microscopic neural signal videos are time-stamp aligned and spatially registered. The microscopic neural signal videos are then subjected to neuron ROI segmentation, calcium signal extraction, and dimensionality reduction or filtering to obtain microscopic neural signal feature sequences.
[0014] Specifically, macroscopic behavioral videos of multiple zebrafish in an experimental tank were acquired using a multi-view high-speed camera array; simultaneously, microscopic neural signal videos of whole-brain calcium imaging of the target zebrafish were acquired using a two-photon microscope or a light sheet microscope; the two types of videos were time-stamped and spatially registered to extract the coordinate sequence of key skeletal points and the time sequence of fluorescence intensity of whole-brain neurons from the multiple fish.
[0015] S2: Social Behavioral Feature Extraction Based on Spatiotemporal Dynamic Graph Attention Network
[0016] Based on the macroscopic behavioral videos, a multi-fish spatiotemporal dynamic graph is constructed, and a high-order social behavior feature sequence is extracted using a graph neural network.
[0017] When constructing the spatiotemporal dynamic graph, multiple zebrafish are abstracted as nodes of the graph, and the physical and visual interactions between the fish are abstracted as edges of the graph. The edge weights are calculated by combining relative distance, orientation angle, and velocity difference.
[0018] Preferably, the edge weights of the spatiotemporal dynamic graph are calculated according to the following formula:
[0019]
[0020] in, , They are time steps Shi Yu and fish Location, For time steps Shi Yu Facing the fish The included angle, , They are time steps Shi Yu and fish speed, For maximum speed, , , These are learnable weight coefficients. This is the distance attenuation parameter.
[0021] The constructed dynamic graph sequence is input into a spatiotemporal dynamic graph attention network (ST-GAT). In the spatial dimension, the graph attention mechanism (GAT) is used to aggregate the interaction features of neighboring nodes, capturing local social topologies such as "leader-follower" and "attack-escape". In the temporal dimension, temporal convolution (TCN) or a Transformer encoder is used to capture the temporal evolution of behavioral patterns. The output is a sequence of high-order social behavior feature vectors after dimensionality reduction.
[0022] S3: Neuro-behavioral causal mapping based on latent variable variational causality discovery
[0023] The high-order social behavior feature sequence and the microscopic neural signal feature sequence are input into the latent variable variational causal discovery model. The model infers the posterior distribution of unobservable latent variables and performs joint optimization by combining the continuity constraint of the directed acyclic graph, outputting a neural-behavioral causal topology graph that removes confounding biases of latent variables.
[0024] Specifically, constructing a system that includes observed variables (neural signal features) Behavioral characteristics ) and continuous latent variables Structural causal models (SCM) that represent unobservable internal states, such as stress and motivation, assume latent variables. and causal adjacency matrix Jointly generate observed variables and .
[0025] Preferably, the total loss function of the latent variable variational causal discovery model is:
[0026]
[0027] in, For observed variables and For neural signal characteristics, For behavioral characteristics, It is a causal adjacency matrix. As a latent variable, This is a reconstruction loss term used to measure the loss given latent variables. and causal adjacency matrix Time decoder reconstructs observation data The error, and the decoder introduces a structural equation model. ,make sure What is captured is culling. The direct causal effects after the impact; For the posterior distribution of latent variables With prior distribution The KL divergence penalty term between them constrains the posterior distribution of the latent variables to approximate the standard normal prior; This is an L1 sparsity penalty term for the causal adjacency matrix, which ensures that the causal graph remains sparse, consistent with the physiological characteristics of biological neural circuits. For the continuity constraint term of the directed acyclic graph, specifically, ,in Represents the trace of a matrix. Matrix Hadamard product, For variable dimensions, , This is the regularization coefficient.
[0028] The network parameters and the causal adjacency matrix are jointly optimized using gradient descent. For the optimized causal adjacency matrix Threshold truncation is performed to output a directional causal topology graph of "neural circuit → social behavior" or "social behavior → neural feedback".
[0029] Preferably, the unobservable latent variables biologically characterize the internal physiological or psychological states of zebrafish that cannot be directly observed, including at least one of hunger, fatigue, stress level, and internal motivation.
[0030] S4: Drug Screening Evaluation and Closed-Loop Regulation
[0031] The causal effect value is calculated based on the causal topology graph and used for drug screening and evaluation or to generate closed-loop regulatory instructions.
[0032] Specifically, based on the causal topology graph output in step S3, the causal effect value of a specific neural circuit to a specific social behavior (such as social avoidance behavior in an autism model) before and after drug intervention is calculated. If the causal effect value is significantly reversed, the drug is deemed effective.
[0033] Preferably, when applied to optogenetic closed-loop regulation, the evaluation and closed-loop control module calculates the current behavioral characteristics in real time. To the target neural circuit The causal effect value is determined; if the causal effect value is lower than a set threshold (e.g., zebrafish exhibit social withdrawal), a control signal is automatically generated to adjust the wavelength, power, and pulse frequency of the optogenetic laser to target the neural circuit. Light stimulation was applied until the causal effect value returned to the normal baseline range.
[0034] This invention also provides a zebrafish social behavior and neural circuit mapping system based on spatiotemporal dynamic graphs and latent variable causal inference, comprising:
[0035] The multimodal synchronous acquisition module includes a multi-view high-speed camera array and a microscopic imaging device, used to simultaneously acquire macroscopic behavioral videos and microscopic neural signal videos of multi-tailed zebrafish;
[0036] The dynamic graph behavior analysis module is used to construct a multi-fish spatiotemporal dynamic graph based on the macro-behavioral video, and extract high-order social behavior feature sequences through a graph neural network.
[0037] The latent variable causal inference module is used to input the high-order social behavior feature sequence and the micro-neural signal feature sequence into the latent variable variational causal discovery model. The model infers the posterior distribution of unobservable latent variables and performs joint optimization by combining the continuity constraint of the directed acyclic graph, and outputs a neural-behavioral causal topology graph that removes the confounding bias of latent variables.
[0038] The evaluation and closed-loop control module is used to calculate the causal effect value based on the causal topology diagram and output control signals for drug screening evaluation or closed-loop regulation.
[0039] Preferably, the multimodal synchronous acquisition module further includes a hardware synchronization trigger, which is connected to the multi-view high-speed camera array and the microscopic imaging device via hardware cables to achieve physical timestamp alignment of the multimodal data.
[0040] The present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the above-described method.
[0041] Compared with the prior art, the present invention has the following beneficial effects:
[0042] Breaking through the limitations of single-fish behavior, achieving high-order topology quantification of groups: By introducing a spatiotemporal dynamic graph attention network, the interaction of multiple fish is upgraded from simple "distance calculation" to "graph topology calculation", which can accurately capture the complex dynamic social hierarchy and interaction patterns in zebrafish groups, providing richer behavioral phenotypic indicators for the screening of psychotropic drugs.
[0043] Eliminating confounding biases from latent variables and revealing the true causal mechanism: Addressing the challenge of unobservable internal states in zebrafish, this study innovatively combines variational inference with continuous causal discovery. This is achieved by explicitly modeling latent variables. It effectively isolates "spurious correlations" caused by internal states (such as fatigue and hunger), and the output causal links have extremely high biological explanatory power, significantly reducing the false positive rate in drug screening.
[0044] Achieving joint optimization of heterogeneous data across scales: Behavioral feature extraction and causal structure inference are not separate sequential pipelines, but rather end-to-end joint optimizations achieved by sharing a latent variable space and objective function. The sparsity constraint of the causal graph can inversely guide the graph attention network to filter out noisy interaction edges without causal meaning, thereby achieving feature purification.
[0045] Achieving a leap from "observation" to "closed-loop regulation": This system is not only used for post-hoc analysis, but its precise causal link output can directly guide the closed-loop control of optogenetic or microfluidic drug delivery systems, realizing a technological upgrade from open-loop observation to closed-loop intervention in "neuro-behavioral" research. Attached Figure Description
[0046] Figure 1 This is a flowchart of the method of the present invention;
[0047] Figure 2 This is a system block diagram of the present invention. Detailed Implementation
[0048] The present invention will be further described in detail below with reference to the accompanying drawings.
[0049] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this description, those skilled in the art can make creative modifications to this embodiment as needed, but as long as they are within the scope of the claims of the present invention, they are protected by patent law.
[0050] This invention provides a method and system for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference, which solves the technical problems in the prior art. The overall idea is as follows:
[0051] Example 1
[0052] Please see Figure 1 A method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference includes the following steps:
[0053] Step S1: Synchronous acquisition and preprocessing of multimodal spatiotemporal data.
[0054] Macroscopic behavioral videos of multiple zebrafish in an experimental tank were acquired using a multi-view high-speed camera array; simultaneously, microscopic neural signal videos of whole-brain calcium imaging of the target zebrafish were acquired using a light-sheet microscope. Hardware synchronization triggers were used to align the timestamps and spatially register the two types of videos, extracting the skeletal keypoint coordinate sequences and the time series of whole-brain neuronal fluorescence intensity from the multiple fish.
[0055] Step S2: Extraction of social behavioral features based on spatiotemporal dynamic graph attention network.
[0056] A spatiotemporal dynamic graph of multiple fish is constructed based on macroscopic behavioral videos. Multiple zebrafish are abstracted as nodes in the graph, and the interactions between fish are abstracted as edges. At each time step... ,fish and fish Edge weights between Calculate using the following formula:
[0057] in, , They are time steps Shi Yu and fish Location, For time steps Shi Yu Facing the fish The included angle, , They are time steps Shi Yu and fish speed, For maximum speed, , , These are learnable weight coefficients. This is the distance attenuation parameter.
[0058] The constructed dynamic graph sequence is input into a spatiotemporal dynamic graph attention network (ST-GAT). In the spatial dimension, the graph attention mechanism (GAT) is used to aggregate the interaction features of neighboring nodes, capturing local social topologies such as "leader-follower" and "attack-escape". In the temporal dimension, a temporal convolutional network (TCN) or a Transformer encoder is used to capture the temporal evolution of behavioral patterns. The output is a sequence of high-order social behavior feature vectors after dimensionality reduction.
[0059] Step S3: Neural-behavioral causal mapping based on latent variable variational causality discovery.
[0060] Sequence of higher-order social behavior characteristics With microscopic neural signal characteristic sequences Input into the latent variable variational causal discovery model. Construct a model containing observed variables. and continuous latent variables Structural causal models (representing unobservable internal states such as stress, motivation, fatigue, and hunger). Unobservable latent variables in biology characterize the internal physiological or psychological states of zebrafish that cannot be directly observed.
[0061] The total loss function of the latent variable variational causal discovery model is:
[0062]
[0063] in, For observed variables and For neural signal characteristics, For behavioral characteristics, It is a causal adjacency matrix. As a latent variable, This is a reconstruction loss term used to measure the loss given latent variables. and causal adjacency matrix Time decoder reconstructs observation data The error, and the decoder introduces a structural equation model. ,make sure What is captured is culling. The direct causal effects after the impact; For the posterior distribution of latent variables With prior distribution The KL divergence penalty term between them constrains the posterior distribution of the latent variables to approximate the standard normal prior; This is an L1 sparsity penalty term for the causal adjacency matrix, which ensures that the causal graph remains sparse, consistent with the physiological characteristics of biological neural circuits. For the continuity constraint term of the directed acyclic graph, specifically, ,in Represents the trace of a matrix. Representation matrix Hadamard product, For variable dimensions, , This is the regularization coefficient.
[0064] The network parameters and the causal adjacency matrix are jointly optimized using gradient descent. For the optimized causal adjacency matrix Threshold truncation is performed to output a directional causal topology graph of "neural circuit → social behavior" or "social behavior → neural feedback".
[0065] Step S4: Drug screening and evaluation and closed-loop regulation.
[0066] Causal effect values are calculated based on causal topology graphs for drug screening and evaluation or to generate closed-loop regulatory instructions. Based on the causal topology graph output in step S3, the causal effect values from specific neural circuits to specific social behaviors (such as social avoidance behavior in an autism model) before and after drug intervention are calculated; if the causal effect value is significantly reversed, the drug is deemed effective.
[0067] When applied to optogenetic closed-loop regulation, the evaluation and closed-loop control module calculates current behavioral characteristics in real time. To the target neural circuit The causal effect value is determined; if the causal effect value is lower than a set threshold (e.g., zebrafish exhibit social withdrawal), a control signal is automatically generated to adjust the wavelength, power, and pulse frequency of the optogenetic laser to target the neural circuit. Light stimulation was applied until the causal effect value returned to the normal baseline range.
[0068] Example 2
[0069] Please see Figure 2 This embodiment provides a zebrafish social behavior and neural circuit mapping system based on spatiotemporal dynamic graphs and latent variable causal inference, including a multimodal synchronous acquisition module, a dynamic graph behavior analysis module, a latent variable causal inference module, and an evaluation and closed-loop control module.
[0070] The multimodal synchronous acquisition module includes a multi-view high-speed camera array, a light-sheet microscope (or two-photon microscope), and a hardware synchronization trigger. The hardware synchronization trigger connects the multi-view high-speed camera array and the microscopic imaging device via hardware cables, achieving timestamp alignment of the multimodal data through physical hardware synchronization. The multi-view high-speed camera array is aimed at the zebrafish experimental tank to acquire macroscopic behavioral videos of multiple zebrafish. The objective lens of the light-sheet microscope is immersed in the tank and aimed at the whole brain of a target juvenile zebrafish with a fixed head, acquiring microscopic neural signal videos of whole-brain calcium imaging. The other companion fish swim freely in the experimental tank to maintain a social interaction environment.
[0071] The dynamic graph behavior analysis module is equipped with a GPU computing unit to construct multi-fish spatiotemporal dynamic graphs in real time based on macroscopic behavioral videos, and extracts high-order social behavior feature sequences through a spatiotemporal dynamic graph attention network (ST-GAT). The edge weights of the spatiotemporal dynamic graph are calculated by weighting the relative distance, the cosine of the relative orientation angle, and the velocity difference.
[0072] The latent variable causal inference module is equipped with a deep learning inference engine, used to infer high-order social behavioral feature sequences. With microscopic neural signal characteristic sequences The input is fed into a latent variable variational causal discovery model. The model infers the posterior distribution of unobservable latent variables and performs joint optimization by incorporating the continuity constraint of the directed acyclic graph, outputting a neural-behavioral causal topology graph that removes confounding biases from latent variables.
[0073] The evaluation and closed-loop control module is used to calculate causal effect values based on the causal topology graph and output control signals for drug screening evaluation or closed-loop regulation. When applied to optogenetic closed-loop regulation, it calculates current behavioral characteristics in real time. To the target neural circuit If the causal effect value is lower than a set threshold, a control signal is automatically generated to adjust the wavelength, power, and pulse frequency of the optogenetic laser to target the neural circuit. Light stimulation was applied until the causal effect value returned to the normal baseline range.
[0074] The above description of the embodiments is provided to facilitate understanding and use of the present invention by those skilled in the art. It is obvious to those skilled in the art that various modifications can be made to the embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the protection scope of the present invention.
Claims
1. A method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference, characterized in that, Includes the following steps: S1. Acquire macroscopic behavioral videos and microscopic neural signal videos of multi-tailed zebrafish, and perform time stamp alignment and spatial registration on the macroscopic behavioral videos and microscopic neural signal videos. Perform neuron ROI segmentation, calcium signal extraction and dimensionality reduction or filtering on the microscopic neural signal videos to obtain microscopic neural signal feature sequences. S2. Construct a multi-fish spatiotemporal dynamic map based on the macroscopic behavioral video, and extract high-order social behavior feature sequences through a graph neural network; S3. Input the high-order social behavior feature sequence and the micro neural signal feature sequence into the latent variable variational causal discovery model. The model infers the posterior distribution of unobservable latent variables and performs joint optimization by combining the continuity constraint of the directed acyclic graph, and outputs a neural-behavioral causal topology graph that removes the confounding bias of latent variables. S4. Calculate the causal effect value based on the causal topology diagram, which can be used for drug screening and evaluation or to generate closed-loop regulatory instructions.
2. The method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference as described in claim 1, characterized in that, The edge weights of the spatiotemporal dynamic graph described in step S2 are calculated according to the following formula:
3. Among them, , They are time steps Shi Yu and fish Location, For time step Shi Yu Facing the fish The included angle, , They are time steps Shi Yu and fish speed, For maximum speed, , , These are learnable weight coefficients. This is the distance attenuation parameter.
4. A method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference, as described in claim 1 or 2, characterized in that: The graph neural network mentioned in step S2 is a spatiotemporal dynamic graph attention network ST-GAT, which includes a spatial graph attention layer and a time series encoding layer. The time series encoding layer is a temporal convolutional network TCN or a Transformer encoder.
5. The method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference as described in claim 1, characterized in that: The total loss function of the latent variable variational causal discovery model described in step S3 is:
6. Among them, For observed variables and For neural signal characteristics, For behavioral characteristics, It is a causal adjacency matrix. As a latent variable, This is a reconstruction loss term used to measure the loss given latent variables. and causal adjacency matrix Time decoder reconstructs observation data The error, and the decoder introduces a structural equation model. ; For the posterior distribution of latent variables With prior distribution KL divergence penalty term between; This is the L1 sparsity penalty term for the causal adjacency matrix; For the continuity constraint term of the directed acyclic graph, specifically, ,in Represents the trace of a matrix. Matrix Hadamard product, For variable dimensions, , is the regularization coefficient.
7. A method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference, as described in claim 1 or 4, characterized in that: The unobservable latent variables are biologically characterized by the internal physiological or psychological states of zebrafish that cannot be directly observed, including at least one of hunger, fatigue, stress level, and internal motivation.
8. The method for mapping zebrafish social behavior and neural circuits based on spatiotemporal dynamic graphs and latent variable causal inference according to claim 1, characterized in that: In step S4, when applied to optogenetic closed-loop regulation, the causal effect value from the current behavioral feature to the target neural circuit is calculated in real time. If the causal effect value is lower than a set threshold, a control signal is automatically generated to adjust the wavelength, power, and pulse frequency of the optogenetic laser to provide light stimulation to the target neural circuit until the causal effect value returns to the normal baseline range.
9. A zebrafish social behavior and neural circuit mapping system based on spatiotemporal dynamic graphs and latent variable causal inference, characterized in that, include: The multimodal synchronous acquisition module includes a multi-view high-speed camera array and a microscopic imaging device, used to simultaneously acquire macroscopic behavioral videos and microscopic neural signal videos of multi-tailed zebrafish; The dynamic graph behavior analysis module is used to construct a multi-fish spatiotemporal dynamic graph based on the macro-behavioral video, and extract high-order social behavior feature sequences through a graph neural network. The latent variable causal inference module is used to input the high-order social behavior feature sequence and the micro-neural signal feature sequence into the latent variable variational causal discovery model. The model infers the posterior distribution of unobservable latent variables and performs joint optimization by combining the continuity constraint of the directed acyclic graph, and outputs a neural-behavioral causal topology graph that removes the confounding bias of latent variables. The evaluation and closed-loop control module is used to calculate the causal effect value based on the causal topology diagram and output control signals for drug screening evaluation or closed-loop regulation.
10. A zebrafish social behavior and neural circuit mapping system based on spatiotemporal dynamic graphs and latent variable causal inference according to claim 7, characterized in that: The multimodal synchronous acquisition module also includes a hardware synchronization trigger, which is connected to the multi-view high-speed camera array and the microscopic imaging device via hardware cables to achieve physical timestamp alignment of the multimodal data.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.