A psychological sand table intelligent analysis method based on a dynamic heterogeneous graph

Abstract

This invention relates to the field of artificial intelligence-assisted diagnosis and treatment technology, specifically a method for intelligent analysis of psychological sandplay based on dynamic heterogeneous graphs. First, it acquires sandplay operation logs and constructs a dynamic heterogeneous graph network with sandplay objects as nodes and interaction relationships as edges. A gating mechanism adaptively fuses the basic physical features and deep semantic features of the sandplay objects. During the sandplay construction process, changes in sandplay object attributes and the environment are monitored in real time to update the graph structure. Based on preset rules, snapshots of the heterogeneous graph and the subject's state at key moments are captured to generate a time-slice sequence reflecting the dynamic evolution of the psychological state. Finally, the structural features of the formed sandplay graph, the temporal features of the time-slice sequence, global environmental information, and the subject's multimodal behavioral data are aggregated and comprehensively processed using a pre-trained psychological analysis model to output a psychological state assessment result. This invention significantly improves the depth, accuracy, and objectivity of auxiliary diagnosis of mental illnesses.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence-assisted diagnosis and treatment technology, specifically to a psychological sandplay intelligent analysis method based on dynamic heterogeneous graphs. Background Technology

[0002] Sandplay therapy, also known as sandplay therapy, is a psychotherapeutic technique developed based on Jungian analytical psychology and incorporating Eastern philosophical thought. In sandplay, clients, accompanied by a therapist, freely select objects from a shelf and arrange them in a specially designed box filled with sand and water, creating scenes that reflect their inner world. According to Jung's theory of imagery, each object, each shape, and the spatial arrangement within the sand tray is seen as a projection and expression of unconscious archetypes. Through this non-verbal creative process, individuals can bring unconscious content into consciousness, thereby integrating personality and promoting psychological growth and healing.

[0003] However, traditional sandplay analysis relies heavily on the therapist's personal experience, professional intuition, and subjective inferences. In current clinical practice, therapists often rely solely on memory or simple notes to record the sandplay creation process. The analysis is usually limited to the final static image of the sandplay, neglecting details such as the order of the objects, the duration of hesitation, and repeated modifications that contain rich psychological dynamics. While existing digital attempts include reconstructing sandplays using 3D scanning technology or simulating them through virtual reality, these technologies mostly focus on the visual reconstruction of the scene, lacking the automated exploration of the deep semantic relationships behind the sandplay elements and failing to systematically quantify the fleeting dynamic psychological changes during the sandplay construction process.

[0004] Therefore, existing sand table analysis techniques suffer from technical problems such as difficulty in capturing details of the dynamic construction process, lack of objective interpretation of the deep semantic relationships between sand table objects, and excessive reliance on human subjective judgment, resulting in low analysis efficiency and standardization. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides an intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs. By transforming the sandplay construction process into a dynamic heterogeneous graph structure and combining it with multimodal data analysis, the method achieves dynamic capture and deep semantic interpretation of the entire psychological sandplay process.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a psychological sandplay intelligent analysis method based on dynamic heterogeneous graphs, comprising the following steps:

[0007] S1: Obtain real-time operation logs during the creation of the psychological sandplay, the operation logs including operation type, operation timestamp, and attribute information of the operation object;

[0008] S2: Construct a dynamic heterogeneous graph network based on the operation log, wherein sand toys are defined as graph nodes, the interaction relationships between sand toys are defined as graph edges, and basic feature attributes and semantic feature attributes are assigned to different types of sand toys, and spatial relationship features and semantic relationship features are assigned to the connections between nodes.

[0009] S3: During the sand table creation process, the changes in sand table attributes and environmental conditions are monitored in real time, the dynamic heterogeneous diagram is updated, and key time points are captured according to preset rules. The local state diagram of the sand table, global information and subject status at that time point are extracted to generate a time slice sequence containing the time dimension.

[0010] S4: Combine the structural features of the final sand table model, the temporal features of the time slice sequence, and the global information of the sand table, input them into a pre-trained psychological analysis model for processing, and output the psychological state assessment results of the subjects.

[0011] Preferably, the specific process of constructing the dynamic heterogeneous graph network and assigning attributes includes:

[0012] The basic feature attributes of the sandplay tool are set as a multi-dimensional vector including orientation, three-dimensional coordinate components, sandplay tool size, operation statistics, and emotional score.

[0013] The semantic feature attributes of the sand figurines are set as semantic symbol vectors and name vectors transformed by a pre-trained language model;

[0014] The basic feature attributes and semantic feature attributes are weighted and merged using a gating fusion mechanism, and the gating weights are generated by the Sigmoid function to adaptively balance the proportion of the two types of features in the node representation.

[0015] When constructing the graph edges, the Euclidean distance between nodes is calculated, and the continuous distance values ​​are mapped into high-dimensional distance feature vectors using radial basis functions. The semantic similarity between sand objects is then combined to form edge features.

[0016] Preferably, the step of updating the dynamic heterogeneous graph specifically includes:

[0017] When a sand object is added, deleted, or its attributes are adjusted, the feature vector of the corresponding node is updated in real time.

[0018] Based on the semantic changes of the manipulated node, the semantic relationship features of other nodes that are connected to the manipulated node are updated through the graph network structure.

[0019] When operations such as filling sand, digging sand, injecting water, pumping water, or leveling terrain are detected, update the terrain features and corresponding environmental semantic symbols in the global information of the sand table.

[0020] When a change in the weather environment settings of the sandbox is detected, the theme symbol attributes of the entire sandbox are updated synchronously.

[0021] Preferably, the step of generating a time-slice sequence containing a time dimension includes:

[0022] An adaptive window capture or graph attention mechanism is used to identify special states during the sand table creation process. These special states include operation pause time exceeding a threshold, high-frequency continuous operation, repeated modification of the same sand table, and moments when the subject's facial expressions and speech show significant fluctuations.

[0023] At each identified specific state moment, a snapshot of the current dynamic heterogeneous graph is captured and associated with the subject's video frame image and global environmental parameters at that moment to form an independent time slice;

[0024] By linking multiple independent time slices in chronological order, a time slice sequence reflecting the dynamic construction process of the sand table is constructed.

[0025] Preferably, the processing procedure of the psychoanalysis model includes:

[0026] The dynamic heterogeneous graph is processed using a graph neural network module, and neighbor node information is aggregated through a message passing mechanism. Specifically, for spatial angle features, the angle information is expanded into a high-dimensional feature vector using a spherical harmonic function.

[0027] The time-slice sequence is processed using a temporal convolutional network or a recurrent neural network to extract evolutionary features during the sand table construction process;

[0028] The facial expression video data and speech audio data of the subjects were encoded through a visual analysis network and an acoustic feature extraction network, respectively.

[0029] Multimodal fusion of graph structure features, temporal evolution features, visual features, and acoustic features is performed and mapped to the mental state classification space through a fully connected layer to calculate the predicted probability of various mental states or mental illnesses.

[0030] Preferably, the emotional score includes positive, neutral, and negative emotional scores for the sandplay object; the operation statistics cover the number of creations, deletions, rotations, moves, scaling, and depth adjustments; the graph edge construction strategy includes prioritizing the retention of a preset number of edges with the closest Euclidean distance, and using dynamic thresholds to remove connections that are too far away, ensuring that each node maintains the minimum number of connections to guarantee the connectivity of the graph.

[0031] Preferably, the method further includes:

[0032] At the start of sand table creation, video and audio acquisition devices are activated simultaneously to record the nonverbal behavior information of the subjects.

[0033] After the sandplay is created, the audio of the conversation between the therapist and the subject about the theme of the work is recorded, and the semantic meaning of the final confirmed sandplay theme is extracted from it.

[0034] The extracted non-verbal behavioral information and dialogue topic semantics are used as auxiliary features and incorporated into the input layer of the psychoanalysis model.

[0035] This invention also provides a psychological sandplay intelligent analysis system based on dynamic heterogeneous graphs, the system comprising:

[0036] The data acquisition module is configured to record sand table operation logs, subject video streams, and audio streams.

[0037] The graph construction and update module is configured to build and maintain a dynamic heterogeneous graph with sand tools as nodes and relationships as edges in real time based on the operation log, and to perform dynamic calculation of node and edge features.

[0038] The time-series capture module is configured to identify key nodes in the operation process and generate time-slice sequences containing graph snapshots and environmental information.

[0039] The multimodal analysis module is configured to load pre-trained deep learning models, integrate heterogeneous graph features, temporal features, and audiovisual features, and output a psychological state analysis report.

[0040] This invention provides an intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs. It has the following beneficial effects:

[0041] 1. This invention constructs a dynamic heterogeneous graph containing sand table nodes and relational edges, and uses a gating fusion mechanism to adaptively combine the basic physical features and deep semantic features of the sand table. This enables the model to simultaneously understand the spatial layout logic and psychological symbolic meaning of the sand table, thereby solving the technical problem that traditional methods are difficult to quantify and analyze the complex semantic relationships between sand table elements, and greatly improving the depth and accuracy of sand table interpretation.

[0042] 2. This invention, by monitoring operation logs in real time and generating time-slice sequences containing key time points, can completely record the evolution of the sand table from scratch, and capture dynamic information such as the subject's hesitation, modification, and emotional fluctuations during the creation process. This analysis method based on dynamic data throughout the entire process makes up for the shortcomings of only performing static analysis on the final image, and provides a rich chain of evidence covering the time dimension for psychological assessment.

[0043] 3. This invention introduces radial basis functions to encode the distance between nodes in a high dimension and combines a gating mechanism to process edge features, enabling the model to sensitively perceive the strong correlation between close sand table objects while reasonably suppressing the interference of distant nodes. This refined spatial relationship processing method effectively improves the robustness and feature extraction capability of graph neural networks when processing complex sand table spatial structures.

[0044] 4. This invention constructs a comprehensive psychological analysis model by integrating sand table structure, time series information, and multimodal data such as video and audio of the subjects. This collaborative analysis mechanism of multi-source heterogeneous data can cross-validate psychological cues from different dimensions, thereby significantly reducing the risk of misjudgment that may be caused by single-modal analysis and improving the reliability of auxiliary diagnosis of mental illness. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the intelligent analysis method of the present invention;

[0046] Figure 2 This is a detailed flowchart of the sand table dynamic information extraction and updating mechanism of the present invention;

[0047] Figure 3 This is a schematic diagram of the basic model structure used in the first experiment of this invention;

[0048] Figure 4 This is a graph showing the changes in loss and accuracy of the model in this invention on the training and validation sets. Figure 4 a represents the average loss curves on the training and validation sets. Figure 4 b represents the accuracy curves on the training and validation sets;

[0049] Figure 5 This is a schematic diagram of the deep model structure for the optimized feature fusion and edge processing mechanism of this invention. Detailed Implementation

[0050] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0051] Example:

[0052] See Figure 1 This embodiment provides a psychological sandplay intelligent analysis method based on dynamic heterogeneous graphs. The method mainly includes four core stages: data acquisition, construction and updating of dynamic heterogeneous graphs, extraction of time slice sequences, and multimodal intelligent analysis.

[0053] S1. Data Acquisition and Global Information Recording

[0054] When the subject begins creating the sandplay, the system immediately activates a comprehensive recording mechanism. Specifically, the system turns on the video recording device to capture nonverbal information such as changes in facial expressions (e.g., blinking eyes, frowning) and body movements (e.g., adjusting posture, hand tremors); at the same time, it turns on high-fidelity voice recording to record the subject's self-talk during the creation process and the vocal characteristics (e.g., speech rate, tone, pauses) when conversing with the therapist after the creation is complete.

[0055] In addition, the system maintains a global information set of the sand table in real time. This global information is a dynamically changing data set, including but not limited to: the currently set weather conditions of the sand table, the terrain features of the sand (such as hills, rivers, and plains), the total number and distribution density of sand objects on the field, the cumulative number of various operations (moving, rotating, deleting), and the current overall semantic symbolic theme of the sand table.

[0056] S2. Construction and Feature Definition of Dynamic Heterogeneous Graphs

[0057] This invention views the sand table as a dynamically evolving graph structure network.

[0058] a. Node Definition and Feature Construction: Nodes are represented by sandplay figures. To accommodate complex psychological analysis needs, this embodiment designs multi-dimensional feature attributes for the nodes.

[0059] The basic features include the three-dimensional coordinates, orientation angle, overall size of the sandplay figure, statistical data on the manipulations performed in this sandplay (number of creations, number of moves, number of rotations, number of deletions, etc.), and emotional score (positive, neutral, negative score).

[0060] Semantic features are derived by using a pre-trained multilingual model to transform the names and inherent symbolic meanings of sand figurines into high-dimensional vectors. To balance the importance of physical and semantic attributes, this embodiment introduces a gating fusion mechanism. Let the basic feature be h. base The semantic feature is h semantic First, map them to a unified dimension, then calculate the gating weights z = sigmoid(W·[h base ,h semantic The final node feature is represented as h. final =z⋅h base +(1−z)·h semantic .

[0061] b-edge definition and relation construction

[0062] Edges are primarily constructed based on Euclidean distance, prioritizing connections to the nearest nodes and setting dynamic thresholds to remove weak connections that are too far apart. At the same time, each node is guaranteed to have at least a certain number of edges to maintain the connectivity of the graph.

[0063] Radial basis functions are introduced to handle distance features. Continuous distance values ​​d are mapped to K-dimensional vectors (e.g., 6-dimensional), as shown by the formula e. k (d)=exp(−γ k ∣∣d−μ k || 2 This encoding method enables the model to distinguish the different weights of near and far distance on psychological association. Furthermore, edge features also incorporate semantic similarity in name and symbolic meaning between connected nodes.

[0064] S3. Dynamic Updates and Time Slice Sequence Generation

[0065] See Figure 2 The system achieves dynamic evolution of the graph by parsing the operation log.

[0066] a. Sandplay Item Operation Response: When the log records an addition, deletion, or movement operation of a sandplay item, the system locates the corresponding node in the heterogeneous graph and updates its coordinates, angle, and operation statistics. Simultaneously, utilizing the message passing mechanism of a graph neural network, based on the semantic changes of this node, the system updates the semantic relationship state of other connected nodes, potentially triggering a change in global semantic symbolism.

[0067] b. Environmental Operation Response: When the log records environmental operations such as sand filling, sand digging, water injection, water pumping, or weather changes, the terrain features and environmental semantic fields in the global information of the sand table are updated directly.

[0068] c. Key Time Slice Capture: The system does not record every frame, but uses intelligent algorithms to capture key moments. Judgment criteria include: excessively long operation intervals (indicating hesitation), drastic changes in facial expression due to emotional fluctuations, and repeated compulsive modifications to a specific area. Once a judgment is triggered, the system packages the current heterogeneous graph state, the subject's video frames, and global information into a single time slice.

[0069] d-sequence generation: Arrange all time slices in the entire process of sand table construction in chronological order to form a time slice sequence, which contains the evolution trajectory of psychological state over time.

[0070] S4. Multimodal Intelligent Analysis Model

[0071] After the sand table is completed, the system will input the collected multimodal data into the deep learning model for comprehensive analysis.

[0072] a) Graph Feature Extraction: Heterogeneous graphs are processed using relational graph convolutional networks or graph isomorphic networks with extended layers. When aggregating neighbor node information, this embodiment innovatively introduces angle information, calculating triple angles and using spherical harmonic functions to expand them into high-dimensional features to capture fine spatial structure information.

[0073] b. Temporal and Multimodal Fusion: For time-slice sequences, a temporal model is used to extract evolution features; for video and audio data, convolutional neural networks and acoustic feature extraction networks are used for encoding, respectively.

[0074] c. Final prediction: All the above feature vectors (graph structure features, temporal features, visual features, acoustic features, global information) are concatenated or weighted and fused, and mapped to the mental health assessment space through a multilayer perceptron to output the type of mental illness that the subject may have or the corresponding risk score.

[0075] Experimental verification

[0076] To verify the effectiveness of this method, the inventors conducted comparative experiments.

[0077] In the first experiment (see Figure 3 , Figure 4 Using only basic 9-dimensional features and a simple graph convolutional network, the model showed some convergence on the training and validation sets, but it had limitations when dealing with complex semantics.

[0078] In the second experiment (see Figure 5 The optimized model employs the 74-dimensional composite features, gating fusion mechanism, RBF distance encoding, and spherical harmonic angle features detailed in the above embodiments. Experimental results show that the optimized model can more accurately capture subtle psychological cues in the sandbox, with significant improvements in classification accuracy and robustness, verifying the feasibility and superiority of combining dynamic heterogeneous graphs with deep semantic analysis.

[0079] The technical solutions provided by this invention are not limited to the specific feature dimensions or network layers mentioned above. Any equivalent substitutions and improvements made by those skilled in the art in accordance with the spirit of this invention should be included within the protection scope of this invention.

[0080] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A psychological sandplay intelligent analysis method based on dynamic heterogeneous graphs, characterized in that, Includes the following steps: S1: Obtain real-time operation logs during the creation of the psychological sandplay, the operation logs including operation type, operation timestamp, and attribute information of the operation object; S2: Construct a dynamic heterogeneous graph network based on the operation log, wherein sand toys are defined as graph nodes, the interaction relationships between sand toys are defined as graph edges, and basic feature attributes and semantic feature attributes are assigned to different types of sand toys, and spatial relationship features and semantic relationship features are assigned to the connections between nodes. S3: During the sand table creation process, the changes in sand table attributes and environmental conditions are monitored in real time, the dynamic heterogeneous diagram is updated, and key time points are captured according to preset rules. The local state diagram of the sand table, global information and subject status at that time point are extracted to generate a time slice sequence containing the time dimension. S4: Combine the structural features of the final sand table model, the temporal features of the time slice sequence, and the global information of the sand table, input them into a pre-trained psychological analysis model for processing, and output the psychological state assessment results of the subjects.

2. The intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs according to claim 1, characterized in that, The specific process of constructing a dynamic heterogeneous graph network and assigning attributes includes: The basic feature attributes of the sandplay tool are set as a multi-dimensional vector including orientation, three-dimensional coordinate components, sandplay tool size, operation statistics, and emotional score. The semantic feature attributes of the sand figurines are set as semantic symbol vectors and name vectors transformed by a pre-trained language model; The basic feature attributes and semantic feature attributes are weighted and merged using a gating fusion mechanism, and the gating weights are generated by the Sigmoid function to adaptively balance the proportion of the two types of features in the node representation. When constructing the graph edges, the Euclidean distance between nodes is calculated, and the continuous distance values ​​are mapped into high-dimensional distance feature vectors using radial basis functions. The semantic similarity between sand objects is then combined to form edge features.

3. The intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs according to claim 1, characterized in that, The steps for updating the dynamic heterogeneous graph specifically include: When a sand object is added, deleted, or its attributes are adjusted, the feature vector of the corresponding node is updated in real time. Based on the semantic changes of the manipulated node, the semantic relationship features of other nodes that are connected to the manipulated node are updated through the graph network structure. When operations such as filling sand, digging sand, injecting water, pumping water, or leveling terrain are detected, update the terrain features and corresponding environmental semantic symbols in the global information of the sand table. When a change in the weather environment settings of the sandbox is detected, the theme symbol attributes of the entire sandbox are updated synchronously.

4. The intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs according to claim 1, characterized in that, The step of generating a time slice sequence containing a time dimension includes: An adaptive window capture or graph attention mechanism is used to identify special states during the sand table creation process. These special states include operation pause time exceeding a threshold, high-frequency continuous operation, repeated modification of the same sand table, and moments when the subject's facial expressions and speech show significant fluctuations. At each identified specific state moment, a snapshot of the current dynamic heterogeneous graph is captured and associated with the subject's video frame image and global environmental parameters at that moment to form an independent time slice; By linking multiple independent time slices in chronological order, a time slice sequence reflecting the dynamic construction process of the sand table is constructed.

5. The intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs according to claim 1, characterized in that, The processing steps of the psychological analysis model include: The dynamic heterogeneous graph is processed using a graph neural network module, and neighbor node information is aggregated through a message passing mechanism. Specifically, for spatial angle features, the angle information is expanded into a high-dimensional feature vector using a spherical harmonic function. The time-slice sequence is processed using a temporal convolutional network or a recurrent neural network to extract evolutionary features during the sand table construction process; The facial expression video data and speech audio data of the subjects were encoded through a visual analysis network and an acoustic feature extraction network, respectively. Multimodal fusion of graph structure features, temporal evolution features, visual features, and acoustic features is performed and mapped to the mental state classification space through a fully connected layer to calculate the predicted probability of various mental states or mental illnesses.

6. The intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs according to claim 2, characterized in that, The emotional score includes positive, neutral, and negative emotional scores for the sandplay object; the operation statistics cover the number of creations, deletions, rotations, moves, scaling, and depth adjustments; the graph edge construction strategy includes prioritizing the retention of a preset number of edges with the closest Euclidean distance, and using dynamic thresholds to remove connections that are too far away, ensuring that each node maintains the minimum number of connections to guarantee the connectivity of the graph.

7. The intelligent analysis method for psychological sandplay based on dynamic heterogeneous graphs according to claim 1, characterized in that, The method further includes: At the start of sand table creation, video and audio acquisition devices are activated simultaneously to record the nonverbal behavior information of the subjects. After the sandplay is created, the audio of the conversation between the therapist and the subject about the theme of the work is recorded, and the semantic meaning of the final confirmed sandplay theme is extracted from it. The extracted non-verbal behavioral information and dialogue topic semantics are used as auxiliary features and incorporated into the input layer of the psychoanalysis model.

8. A psychological sandplay intelligent analysis system based on dynamic heterogeneous graphs, characterized in that, The system includes: The data acquisition module is configured to record sand table operation logs, subject video streams, and audio streams. The graph construction and update module is configured to build and maintain a dynamic heterogeneous graph with sand tools as nodes and relationships as edges in real time based on the operation log, and to perform dynamic calculation of node and edge features. The time-series capture module is configured to identify key nodes in the operation process and generate time-slice sequences containing graph snapshots and environmental information. The multimodal analysis module is configured to load pre-trained deep learning models, fuse heterogeneous graph features, temporal features, and audiovisual features, and output a psychological state analysis report. A memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images