Heterogeneous graph-based user mental health monitoring system and method
Through a heterogeneous graph-based user mental health monitoring system, using social media data and physiological monitoring, combined with a neural network model of the two-layer attention mechanism, the identification and tracking problems in traditional mental health diagnosis and treatment methods are solved, and the user is achieved Comprehensive monitoring and evaluation of mental health.
Patent Information
- Application Number
- CN202411057971.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-02
AI Technical Summary
Traditional mental health diagnosis and treatment methods require face-to-face communication, limiting the possibility of potential patients’ identification, and economic, resource, social discrimination and privacy protection issues make it difficult for patients to monitor and track mental health.
A user mental health monitoring system based on heterogeneous graphs collects social media data through network crawlers, uses a heterogeneous graph neural network model with a hierarchical aggregation method and a two-layer attention mechanism, and combines SAM scales and physiological monitoring to analyze user emotions and factors. , create a three-dimensional reference table for normal psychology and optimize the results of mental health monitoring.
It has achieved comprehensive monitoring and evaluation of users' mental health, improved the effectiveness of mental health monitoring, and overcomes the difficulties in identifying and tracking of traditional methods.
Smart Images

Figure CN119092113A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of psychological monitoring technology, and in particular to a user psychological health monitoring system and method based on a heterogeneous graph. Background Art
[0002] Heterogeneous graphs are also called heterogeneous graphs, heterogeneous information networks, and heterogeneous information networks, which are different from homogeneous graphs. They are graph data structures with multiple node types or multiple edge types, which are used to describe complex heterogeneous objects and their interactions. They have rich semantic information and provide an effective modeling tool and analysis method for graph data mining. Using heterogeneous graphs to model this type of rich and complex interactive data can retain more comprehensive semantic and structural information. With the increasing pressure of life and work in modern society, people of all ages have frequent psychological problems. For example, people who are troubled by mental health problems such as depression and stress are getting younger and younger, especially adolescents. Mental health problems are becoming more and more serious.
[0003] However, traditional mental health diagnosis and treatment methods require face-to-face communication with doctors. This contact-based approach limits the possibility of identifying potential patients. In addition, due to issues such as economic level, lack of resources, social discrimination, and privacy protection, even if patients realize that they may have a mental health disease, they will not go to the hospital or clinic for identification. As a result, previous treatment plans often make it difficult to monitor and track people's psychological or mental states in large sample situations.
[0004] Therefore, the existing needs are not met, and we propose a user mental health monitoring system and method based on heterogeneous graphs. Summary of the invention
[0005] The purpose of the present invention is to provide a user mental health monitoring system and method based on a heterogeneous graph. By collecting user social profile information, a hierarchical aggregation method is used to aggregate and classify images and texts to obtain multiple emotion nodes and adjacent node parameters; a heterogeneous graph neural network model based on a double-layer attention mechanism is constructed to aggregate different emotion nodes again, and then the adjacent nodes are fused to obtain a more comprehensive current user emotion aggregation result, which is convenient for judging the user's multiple emotions and the related factors causing the emotions; a normal psychological three-dimensional reference table is created, and the current user's emotions and the related factors are checked according to the psychological three dimensions. By analyzing the pleasure, arousal and controllability of the current user's emotions, a more comprehensive user mental health monitoring result is judged, thereby optimizing the effectiveness of the existing mental health monitoring method and solving the problems raised in the above background technology.
[0006] To achieve the above object, the present invention provides the following technical solutions:
[0007] The user mental health monitoring system based on heterogeneous graph includes:
[0008] The first information collection unit is used to
[0009] Collect user information and user social media data from multiple social platforms through web crawler technology, extract user portrait information, text information, and topic information from social media, and form a heterogeneous graph of users based on this;
[0010] The second information collection unit is used to
[0011] Use the SAM scale method to ask users questions about different topics and get their responses. While asking users questions, monitor their blood pressure, heart rate, and body language in real time, and record their physiological information in the corresponding answer list.
[0012] Association analysis unit, used for
[0013] A hierarchical aggregation method is used to aggregate and analyze user information, user social media data, and user reply information under different topic information, and the text semantic information in the heterogeneous graph is introduced to obtain different user emotion nodes and their associated neighbor information;
[0014] Aggregate processing unit for
[0015] Create a heterogeneous graph neural network model, introduce a two-layer attention mechanism, use semantic-level attention and node-level attention to perform semantic-level aggregation and node-level aggregation respectively, and obtain the aggregated representation of adjacent nodes; and fuse the aggregated representations of adjacent nodes to obtain a more comprehensive aggregated representation;
[0016] Psychometric unit for
[0017] The user's image information and semantic information are imported into a heterogeneous graph neural network model based on a double-layer attention mechanism for testing. The user's current mental health status is judged based on the aggregation results output by the heterogeneous graph neural network model.
[0018] Furthermore, the first information collection unit includes
[0019] The portrait extraction module is used to analyze the micro-expressions of the portrait information in the user's social media data, segment the portrait information according to the different micro-expressions, and set emotional labels for the micro-expressions based on the semantic content of the portrait information;
[0020] The text extraction module is used to perform text summarization on the short texts generated in the user's social media data and extract the topic information and factors in the short texts;
[0021] The text segmentation module is used to segment the short text generated in the user's social media data according to the text context and expression emotions, and obtain the semantic content of the user's expression under different emotions;
[0022] The information association module is used to match and associate the semantic content expressed by the user under different emotions with the micro-expressions of the user portrait, and obtain the emotional facial features of the user under different themes.
[0023] Furthermore, the first information collection unit also includes
[0024] A real-time monitoring module, used for real-time monitoring of the data processing time of the image extraction module, the text extraction module, the text segmentation module and the information association module;
[0025] A time acquisition module, used for acquiring the end time of data processing of the image extraction module, the text extraction module, the text segmentation module and the information association module for the same image information according to the data processing time of the image extraction module, the text extraction module, the text segmentation module and the information association module;
[0026] A time difference period acquisition module is used to sort the data processing end times of the portrait extraction module, the text extraction module, the text segmentation module and the information association module for the same portrait information in chronological order, and to acquire the time difference period between two adjacent times;
[0027] The data processing synchronization coefficient acquisition module is used to obtain the data processing synchronization coefficient according to the time difference period between the data processing end time of the same portrait information by the portrait extraction module, the text extraction module, the text segmentation module and the information association module; wherein the data processing synchronization coefficient is obtained by the following formula:
[0028]
[0029] Where P represents the data processing synchronization coefficient; n represents the number of time difference periods; T di represents the duration corresponding to the ith time difference period; T dmax Indicates the maximum duration of n time difference periods; T dmin represents the minimum duration of n time difference periods; m represents the number of image extraction modules, text extraction modules, text segmentation modules and information association modules, that is, m = 4; T si represents the average data processing time corresponding to the i-th module; T zi represents the preset data processing time reference value corresponding to the i-th module; λ represents the adjustment coefficient, and the adjustment coefficient is obtained by the following formula:
[0030]
[0031] Among them, λ represents the adjustment coefficient; T 01si T represents the data processing time of the current portrait information of the module preceding the data processing end time corresponding to the i-th time difference period; 01zi T represents the theoretical data processing time of the current image information of the module preceding the data processing end time corresponding to the i-th time difference period; 02si T represents the data processing time of the current portrait information of the module whose data processing end time is later than the i-th time difference period; 02zi Indicates the theoretical data processing time of the current portrait information of the module whose data processing end time is later than the i-th time difference period;
[0032] The synchronous operation determination module is used to determine the data processing synchronous operation quality of the first information collection unit according to the data processing synchronization coefficient corresponding to each portrait information processing.
[0033] Furthermore, the synchronous operation determination module includes:
[0034] A data processing synchronization coefficient extraction module is used to extract the data processing synchronization coefficient corresponding to each portrait information processing;
[0035] A coefficient difference acquisition module is used to acquire the coefficient difference between each two adjacent data processing synchronization coefficients according to the data processing synchronization coefficient corresponding to each portrait information processing;
[0036] The synchronization evaluation parameter acquisition module is used to obtain the synchronization evaluation parameter by using the coefficient difference between each two adjacent data processing synchronization coefficients; wherein the synchronization evaluation parameter is obtained by the following formula:
[0037]
[0038] Where Q represents the synchronization evaluation parameter; k represents the number of processed image information; P i represents the data processing synchronization coefficient corresponding to the i-th image information; P i-1 represents the data processing synchronization coefficient corresponding to the i-1th image information; P i-2 represents the data processing synchronization coefficient corresponding to the i-2th image information; P f represents the data processing synchronization coefficient corresponding to the first portrait information; P f Indicates the data processing synchronization coefficient corresponding to the nth portrait information;
[0039] A comparison module, used for comparing the synchronization evaluation parameter with a preset parameter threshold to obtain a comparison result;
[0040] The abnormality determination module is used to determine that the data processing synchronous operation quality is abnormal and issue an abnormality alarm when the comparison result shows that the synchronization evaluation parameter is lower than a preset parameter threshold.
[0041] Furthermore, the second information collection unit includes
[0042] The emotion acquisition module is used to measure the user's emotions through the SAM scale to obtain the user's emotional pleasure, arousal and control;
[0043] The physiological acquisition module is a wearable heart rate monitor that is used to measure the user's emotions, collect the user's heart rate and blood pressure values in real time, and judge the user's current emotional fluctuations based on the fluctuations of the heart rate and blood pressure values;
[0044] The behavior acquisition module is a high-definition camera that is used to measure the user's emotions, capture the user's body movements in real time, and judge the user's current emotional fluctuations based on the amplitude of the body movements.
[0045] Furthermore, the second information collection unit also includes
[0046] The text recording module is used to record the user's emotional pleasure, arousal and control, the user's real-time heart rate and blood pressure values, and the user's body movements in a table to form a complete user physiological data monitoring table.
[0047] Furthermore, the association analysis unit comprises
[0048] The node aggregation module is used to find the neighbor factor information of the user's emotion under a certain relationship for a node, and aggregate the neighbor factor information to obtain the representation of a certain emotion node under a certain relationship;
[0049] The semantic aggregation module is used to fuse neighbor node representations under multiple relationships at the semantic level to obtain a more comprehensive representation of a certain emotion node.
[0050] Furthermore, the aggregation processing unit includes
[0051] The data processing module is used to clean and uniformly format the first information and the second information of the user to obtain pre-processed user data; the user data is then divided into a training set and a test set as a basis for optimizing the parameters of the heterogeneous graph neural network model;
[0052] The model optimization module is used to import the training set into the heterogeneous graph neural network model based on the double-layer attention mechanism for training, output the aggregated training results, and establish the attention feature mechanism of the heterogeneous graph neural network model; then import the test set into the heterogeneous graph neural network model based on the double-layer attention mechanism for testing, and output the aggregated test results; after the training results are consistent with the test results, the optimal heterogeneous graph neural network model based on the double-layer attention mechanism is obtained.
[0053] Furthermore, the psychological determination unit includes
[0054] The result analysis module is used to analyze and judge the aggregated results of the current user's emotions output by the heterogeneous graph neural network model to obtain the current user's emotional status and factors;
[0055] The condition verification module is used to preset a normal psychological three-dimensional reference table based on the SAM scale, and verify the current user's emotional condition and related factors with the psychological three dimensions in the table to obtain a more comprehensive user mental health monitoring result.
[0056] The method for monitoring user mental health based on heterogeneous graphs comprises the following steps:
[0057] S1. Collect and organize user-related social images and text data, and use the SAM scale to conduct a preliminary emotional assessment of users;
[0058] S2. Use hierarchical aggregation method to aggregate and analyze user social data and SAM scale response information to obtain different user emotion nodes and adjacent nodes;
[0059] S3. Create a heterogeneous graph neural network model based on a double-layer attention mechanism, perform node aggregation at the semantic level and node level, and then fuse adjacent nodes;
[0060] S4. Importing the training set and the test set into the heterogeneous graph neural network model based on the double-layer attention mechanism in sequence, and optimizing the heterogeneous graph neural network model based on the double-layer attention mechanism;
[0061] S5, after preprocessing the current user heterogeneous graph information, import it into the heterogeneous graph neural network model based on the double-layer attention mechanism for testing, and output the aggregation result, which includes: the user's current emotion and its factors;
[0062] S6. Create a normal psychological three-dimensional reference table based on the SAM scale, compare the current aggregation results with the psychological three dimensions in the table, and obtain the final user mental health monitoring results.
[0063] Compared with the prior art, the present invention has the following beneficial effects:
[0064] The present invention optimizes the data diversity of mental health monitoring by collecting user social third-party image data and text information to form a heterogeneous graph; adopts a hierarchical aggregation method to aggregate and classify image data and various text information to obtain multiple emotion nodes and adjacent node parameters; constructs a heterogeneous graph neural network model based on a double-layer attention mechanism to aggregate different emotion nodes again, and then fuses adjacent nodes to obtain a more comprehensive current user emotion aggregation result, thereby facilitating the judgment of the user's multiple emotions and the relationship factors causing the emotions; creates a normal psychological three-dimensional reference table, checks the current user's emotions and the related factors according to the psychological three dimensions, and judges a more comprehensive user mental health monitoring result by analyzing the pleasure, arousal and controllability of the current user's emotions, thereby optimizing the effectiveness of existing mental health monitoring methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0065] Figure 1 It is a composition diagram of the user mental health monitoring system based on heterogeneous graph of the present invention;
[0066] Figure 2 This is a flow chart of the user mental health monitoring method based on heterogeneous graphs of the present invention. DETAILED DESCRIPTION
[0067] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0068] In order to solve the problem of traditional mental health diagnosis and treatment, face-to-face communication with doctors is required. This contact-based approach limits the possibility of identifying potential patients. In addition, due to economic level, lack of resources, social discrimination, privacy protection and other issues, even if patients realize that they may have mental health diseases, they will not go to hospitals or clinics for identification. As a result, previous treatment plans are often difficult to monitor and track people's psychological or mental states in large sample situations. For technical issues, please refer to Figure 1-Figure 2 , this embodiment provides the following technical solutions:
[0069] The user mental health monitoring system based on heterogeneous graph includes:
[0070] The first information collection unit is used to
[0071] Through web crawler technology, user information and user social media data are collected from multiple social platforms, and the user's portrait information, text information, and topic information in social media are extracted, based on which a heterogeneous graph of users is formed; specifically, with the user's authorization, web crawler technology is used to obtain various information texts and image information published on the user's social platform to form a heterogeneous graph data combining text and images.
[0072] The first information collection unit includes:
[0073] The portrait extraction module is used to analyze the micro-expressions of the portrait information in the user's social media data, segment the portrait information according to the different micro-expressions, and establish emotional labels for the micro-expressions in combination with the semantic content of the portrait information; specifically, by obtaining multiple user personal image information, and dividing the multiple user images according to different micro-expressions, and segmenting each image, cutting the background picture, retaining the main page of the micro-expression, and setting emotional labels for different micro-expressions; such as: cheerfulness, joy, depression, pain, etc.; in a specific embodiment, if the current user social platform image information is less or no, emotional instructions can be issued to the user, such as: when the user encounters a certain happy thing, by observing and photographing what kind of micro-expression the user will produce at the moment, this can be used as alternative image information.
[0074] The text extraction module is used to summarize the short texts generated in the user's social media data and extract the subject information and factors in the short texts. Specifically, by analyzing the text content posted on the user's social platform, a summary is made based on each text content to obtain the subject elements of each text content, thereby simplifying the amount of text information and obtaining the most effective text information.
[0075] The text segmentation module is used to perform text segmentation on short texts generated in the user's social media data according to the text context and the expressed emotions, so as to obtain the semantic content expressed under different emotions of the user; specifically, by performing sentiment analysis on the simplified text information, the emotional features covered by the semantic content expressed in the text information are determined, and multiple text information are segmented based on the different emotional features, thereby obtaining the semantic content expressed under different emotions of the user.
[0076] The information association module is used to match and associate the semantic content expressed by the user under different emotions with the micro-expressions of the user portrait, so as to obtain the emotional facial features of the user under different themes; specifically, by associating the segmented image information and semantic content with the same attributes, the semantic content that is consistent with the expression content is filtered out for the micro-expression label content in the image information, so as to enrich the emotional atmosphere of the semantic content and the emotional background of the image information.
[0077] Specifically, the first information collection unit also includes
[0078] A real-time monitoring module, used for real-time monitoring of the data processing time of the image extraction module, the text extraction module, the text segmentation module and the information association module;
[0079] A time acquisition module, used for acquiring the end time of data processing of the image extraction module, the text extraction module, the text segmentation module and the information association module for the same image information according to the data processing duration of the image extraction module, the text extraction module, the text segmentation module and the information association module;
[0080] A time difference period acquisition module is used to sort the data processing end times of the portrait extraction module, the text extraction module, the text segmentation module and the information association module for the same portrait information in chronological order, and to acquire the time difference period between two adjacent times;
[0081] The data processing synchronization coefficient acquisition module is used to obtain the data processing synchronization coefficient according to the time difference period between the data processing end time of the same portrait information by the portrait extraction module, the text extraction module, the text segmentation module and the information association module; wherein the data processing synchronization coefficient is obtained by the following formula:
[0082]
[0083] Where P represents the data processing synchronization coefficient; n represents the number of time difference periods; T di represents the duration corresponding to the ith time difference period; T dmax Indicates the maximum duration of n time difference periods; T dmin represents the minimum duration of n time difference periods; m represents the number of image extraction modules, text extraction modules, text segmentation modules and information association modules, that is, m = 4; T si represents the average data processing time corresponding to the i-th module; T zi represents the preset data processing time reference value corresponding to the i-th module; λ represents the adjustment coefficient, and the adjustment coefficient is obtained by the following formula:
[0084]
[0085] Among them, λ represents the adjustment coefficient; T 01si T represents the data processing time of the current portrait information of the module preceding the data processing end time corresponding to the i-th time difference period; 01zi T represents the theoretical data processing time of the current image information of the module preceding the data processing end time corresponding to the i-th time difference period; 02siT represents the data processing time of the current portrait information of the module whose data processing end time is later than the i-th time difference period; 02zi Indicates the theoretical data processing time of the current portrait information of the module whose data processing end time is later than the i-th time difference period;
[0086] The synchronous operation determination module is used to determine the data processing synchronous operation quality of the first information collection unit according to the data processing synchronization coefficient corresponding to each portrait information processing.
[0087] The technical effect of the above technical solution is: through the real-time monitoring module, the system can track the data processing time of key modules such as image extraction, text extraction, text segmentation and information association in real time. This real-time monitoring mechanism helps to timely discover performance bottlenecks or abnormal processing situations, and provide data support for subsequent optimization and adjustment.
[0088] The time acquisition module and the time difference period acquisition module record and compare the end time of processing of the same portrait information by each module, calculate the time difference period between them, and thus evaluate the data processing synchronization between the modules. This synchronization evaluation is the key to ensure the overall operation efficiency and quality of the system.
[0089] The calculation of the data processing synchronization coefficient (P) comprehensively considers multiple factors such as the time difference period, the difference between the average data processing time of each module and the reference value, and the adjustment coefficient, providing a quantitative indicator to evaluate the degree of data processing synchronization between the modules of the system. This coefficient not only reflects the difference in processing speed between modules, but also further considers the mutual influence of the processing time between the previous and next modules through the adjustment coefficient, making the evaluation result more comprehensive and accurate.
[0090] The introduction of the adjustment coefficient makes the calculation of the data processing synchronization coefficient more flexible and adaptive. By considering the difference between the actual value and the theoretical value of the processing time of the front and back modules, the system can more accurately evaluate the rationality of the current processing status and adjust the optimization strategy as needed.
[0091] The synchronization operation determination module determines the data processing synchronization operation quality of the first information collection unit based on the data processing synchronization coefficient corresponding to each portrait information processing. This helps the system to identify and respond to possible synchronization problems in a timely manner, ensuring the efficiency and stability of the overall operation.
[0092] The entire technical solution provides a powerful tool for system performance optimization and decision support. Through real-time monitoring and evaluation of data processing synchronization, system administrators or algorithm optimizers can promptly discover and solve potential performance problems, and formulate targeted optimization strategies based on the evaluation results to further improve the overall performance of the system and user satisfaction.
[0093] Specifically, the synchronous operation determination module includes:
[0094] A data processing synchronization coefficient extraction module is used to extract the data processing synchronization coefficient corresponding to each portrait information processing;
[0095] A coefficient difference acquisition module is used to acquire the coefficient difference between each two adjacent data processing synchronization coefficients according to the data processing synchronization coefficient corresponding to each portrait information processing;
[0096] The synchronization evaluation parameter acquisition module is used to obtain the synchronization evaluation parameter by using the coefficient difference between each two adjacent data processing synchronization coefficients; wherein the synchronization evaluation parameter is obtained by the following formula:
[0097]
[0098] Where Q represents the synchronization evaluation parameter; k represents the number of processed image information; P i represents the data processing synchronization coefficient corresponding to the i-th image information; P i-1 represents the data processing synchronization coefficient corresponding to the i-1th image information; P i-2 represents the data processing synchronization coefficient corresponding to the i-2th image information; P f represents the data processing synchronization coefficient corresponding to the first portrait information; P f Indicates the data processing synchronization coefficient corresponding to the nth portrait information;
[0099] A comparison module, used to compare the synchronization evaluation parameter with a preset parameter threshold to obtain a comparison result;
[0100] The abnormality determination module is used to determine that the data processing synchronous operation quality is abnormal and issue an abnormality alarm when the comparison result shows that the synchronization evaluation parameter is lower than a preset parameter threshold.
[0101] The technical effect of the above technical solution is: through the data processing synchronization coefficient extraction module, the system can continuously track the data processing synchronization coefficient in each image information processing process, thereby realizing continuous monitoring of the system synchronization operation status. This continuous monitoring helps to timely discover problems or potential risks in synchronization operation.
[0102] The coefficient difference acquisition module and the synchronization evaluation parameter acquisition module provide a method for dynamically evaluating the quality of system synchronization operation by calculating the coefficient difference between each two adjacent data processing synchronization coefficients and calculating the synchronization evaluation parameter (Q) based on these differences. This method not only considers the synchronization of single image information, but also evaluates the synchronization stability of the entire processing process through cumulative effects.
[0103] The calculation formula of the synchronization evaluation parameter takes into account both the synchronization coefficients of the current and adjacent data processing (Pi, Pi-1, Pi-2) and the synchronization coefficients of the first and last image information in the processing process (Pf, Pn), thus comprehensively considering the synchronous operation status of the system during the entire processing cycle. This comprehensive consideration helps to more accurately evaluate the synchronous operation quality of the system.
[0104] The comparison module and the abnormality judgment module compare the synchronization evaluation parameters through the preset parameter threshold. When the synchronization evaluation parameters are found to be lower than the threshold, the data processing synchronization operation quality is automatically judged to be abnormal and an abnormality alarm is issued. This automated mechanism can quickly respond to synchronization problems, reduce delays and errors caused by manual intervention, and improve the reliability and stability of the system.
[0105] Through the above technical solutions, system administrators can more clearly understand the status and trend of system synchronization operation, and timely identify and locate synchronization problems. This helps system administrators formulate targeted optimization strategies, improve the maintainability and optimization capabilities of the system, and ensure long-term stable operation of the system.
[0106] By promptly discovering and resolving synchronization issues, the system can reduce user waiting time and error feedback caused by synchronization delays or errors, thereby improving user experience. This is especially important for systems that need to process large amounts of data efficiently and accurately.
[0107] The second information collection unit is used to
[0108] Use the SAM scale method to ask users questions about different topics and get their responses. While asking users questions, monitor their blood pressure, heart rate, and body language in real time, and record their physiological information in the corresponding answer list.
[0109] The second information collection unit includes:
[0110] The emotion acquisition module is used to measure the user's emotions through the SAM scale to obtain the user's emotional pleasure, arousal and control. The SAM scale is a tool for measuring emotions. It contains three main scales: pleasure, arousal and control. Each scale has 9 levels of scoring options, from 1 for the lowest degree to 9 for the highest degree, so that the SAM scale can capture the user's emotional state in a specific situation in detail. It can be understood as follows: the pleasure scale measures the degree of pleasure or unpleasantness felt by an individual; the arousal scale measures the individual's tension or relaxation, and the degree of excitement or lack of excitement; the control scale measures the individual's inhibition or lack of inhibition, which can be understood as the individual's active or passive experience in emotion.
[0111] In this embodiment, for example, each dimension includes 5 puppet pictures and 9 level options. The first dimension is emotional valence, that is, pleasantness, which mainly measures positive and negative emotions, 1 represents "very pleasant", 5 represents "neutral", and 9 represents "very unhappy"; the second dimension is emotional arousal, that is, arousal, which represents the degree to which emotions are aroused, and mainly measures the strength of emotions, 1 represents "very excited", 5 represents "neutral", and 9 represents "very calm"; the third dimension is emotional dominance, that is, dominance, which represents the degree of control an individual has over the emotion, and mainly measures people's ability to control their emotional reactions, 1 represents "completely controlled", 5 represents "neutral", and 9 represents "completely controlled"; questions can be asked to the user in a paper-and-pencil manner, and the user can obtain the user's emotional evaluation form by selecting the corresponding puppet picture.
[0112] The physiological acquisition module is a wearable heart rate monitor, which is used to measure the user's emotions, collect the user's heart rate and blood pressure values in real time, and judge the user's current emotional fluctuations based on the fluctuations in heart rate and blood pressure values. Specifically, by monitoring the user's heart rate and blood pressure, it is mainly used to observe whether the user's inner physiological fluctuations are abnormal when answering different questions, and to check the severity of the abnormal situation; so as to judge the user's ability to regulate his or her emotions based on the degree of abnormality.
[0113] The behavior acquisition module is a high-definition camera that is used to measure the user's emotions, capture the user's body movements in real time, and judge the user's current emotional fluctuations based on the amplitude of the body movements. Specifically, by recording the user's body movements, it is determined whether the user's body reactions are abnormal. For example, more or fewer body movements are both manifestations of psychological abnormalities.
[0114] The text recording module is used to record the user's emotional pleasure, arousal and control, the user's real-time heart rate and blood pressure values, and the user's body movements in a table to form a complete user physiological data monitoring table.
[0115] Association analysis unit, used for
[0116] A hierarchical aggregation method is used to aggregate and analyze user information, user social media data, and user reply information under different topic information. The text semantic information in the heterogeneous graph is introduced to obtain different user emotion nodes and their associated neighbor information.
[0117] Association analysis unit, including:
[0118] The node aggregation module is used to find the neighbor factor information of the user's emotion in a certain relationship under a certain emotion, and aggregate the neighbor factor information to obtain the representation of a certain emotion node under a certain relationship; specifically, by introducing various types of nodes in the heterogeneous graph, the information contained in the subsequent heterogeneous graph neural network model is enriched; homogeneous graphs are often constructed from a single data source, but by collecting a large amount of user social data and their image information to form a heterogeneous graph, it can provide various aspects of information for research users from different angles; so as to effectively integrate this information together, conduct unified analysis, and improve the effect of mental health monitoring.
[0119] The semantic aggregation module is used to fuse the neighbor node representations under multiple relationships at the semantic level to obtain a more comprehensive representation of a certain emotion node; specifically, by introducing semantic nodes into emotion nodes, the semantics of a certain emotion node and its adjacent nodes or neighbor nodes with corresponding relationships are enriched, thereby providing belonging factors and outward emotion representations for a certain emotion node.
[0120] Aggregate processing unit for
[0121] Create a heterogeneous graph neural network model, introduce a two-layer attention mechanism, use semantic-level attention and node-level attention to perform semantic-level aggregation and node-level aggregation respectively, and obtain the aggregated representation of adjacent nodes; and fuse the aggregated representations of adjacent nodes to obtain a more comprehensive aggregated representation;
[0122] Aggregation processing unit, including:
[0123] The data processing module is used to clean and uniformly format the user's first information and second information to obtain pre-processed user data; then divide the user data into a training set and a test set as a basis for optimizing the parameters of the heterogeneous graph neural network model; specifically, by cleaning various information, it is mainly used to eliminate erroneous data and duplicate data in existing data to ensure the validity of the data; uniformly formatting various information is mainly used to optimize the source of information data and ensure the uniformity of various information data sources to facilitate their classification and use.
[0124] The model optimization module is used to import the training set into the heterogeneous graph neural network model based on the double-layer attention mechanism for training, output the aggregated training results, and establish the attention feature mechanism of the heterogeneous graph neural network model; then import the test set into the heterogeneous graph neural network model based on the double-layer attention mechanism for testing, and output the aggregated test results; after the training results are consistent with the test results, the optimal heterogeneous graph neural network model based on the double-layer attention mechanism is obtained; specifically, by training the heterogeneous graph neural network model, it is mainly used to establish the feature points of the heterogeneous graph neural network model, so as to facilitate the subsequent extraction of user information features; the heterogeneous graph neural network model is tested, which is mainly used to optimize the accuracy of the heterogeneous graph neural network model and ensure the accuracy of the current output results of the heterogeneous graph neural network model.
[0125] Psychometric unit for
[0126] The user's image information and semantic information are imported into a heterogeneous graph neural network model based on a double-layer attention mechanism for testing, and the user's current mental health status is judged based on the aggregation results output by the heterogeneous graph neural network model. Specifically, after the accuracy of the operation of the heterogeneous graph neural network model is established, the current user's heterogeneous graph data is preprocessed as described above, and then the current user's heterogeneous graph data is imported into the optimized heterogeneous graph neural network model for calculation, thereby outputting the current user's emotion aggregation result, namely: the current user's emotion category, which includes: cheerfulness, joy, depression, pain, etc., as well as the possible inducing factors of the current emotion and the outward representation of the emotion.
[0127] Psychometric unit, including:
[0128] The result analysis module is used to analyze and judge the aggregation results of the current user's emotions output by the heterogeneous graph neural network model to obtain the current user's emotional status and belonging factors; specifically, after outputting the current user's emotional aggregation results, the output results are analyzed.
[0129] Continuing with the above embodiment, for example: the current user emotion is: joy, the possible inducing factor is: corresponding achievements in work, and the outward expression of emotion is: dancing; based on the above results, it can be seen that: the current user has achieved corresponding achievements in work, and the emotion is relatively joyful and accompanied by body movements without harm awareness; for example: the output inducing factor is: corresponding achievements in work, the current user emotion is: depression, and the outward expression of emotion is: no body movements; based on the above results, it can be seen that: the current user has achieved corresponding achievements in work, and the emotion is not to produce corresponding joy, and the expression is accompanied by depression, and there are no body movements.
[0130] The status verification module is used to preset a normal psychological three-dimensional reference table based on the SAM scale, and verify the current user's emotional status and related factors with the psychological three dimensions in the table to obtain a more comprehensive user mental health monitoring result; specifically, the human body will have different responses to the entity when receiving different transactions, such as: emotions, body; it is well known that under normal conditions, achieving corresponding achievements in work is a positive event, the normal reaction of the human body is happiness or joy, and the human body should have certain emotional reactions.
[0131] Continuing with the above embodiment, for example, the current user has achieved corresponding achievements in work, and is in a relatively happy mood and accompanied by non-harmful body movements; the current user's reaction result is compared with the first dimension in the SAM scale, which is emotional valence, that is, pleasure, and it is concluded that the current user's pleasure should be 1, that is, representing "very happy", which indicates that the current user's pleasure is in a normal state; the current user's reaction result is then compared with the second dimension in the SAM scale, which is emotional arousal, that is, arousal, and it is concluded that the current user's arousal should be 1, that is, representing "very excited", which indicates that the current user's arousal is in a normal state; the current user's reaction result is then compared with the third dimension in the SAM scale, which is emotional dominance, that is, dominance, and it is concluded that the current user's dominance should be 9, that is, "complete control", which indicates that the current user's dominance is in a normal state, and thus it is concluded that the current user's mental health is in a normal state; for another example, the current user has achieved corresponding achievements in work, and is in a relatively depressed mood, and has no body movements; The current user response result is compared with the first dimension of the SAM scale, which is emotional valence, namely, pleasantness, and it is concluded that the current user's pleasantness should be 9, which means "very unhappy", indicating that the current user's pleasantness is in an abnormal state; then the current user response result is compared with the second dimension of the SAM scale, which is emotional arousal, namely, arousal, and it is concluded that the current user's arousal should be 9, which means "very calm", indicating that the current user's arousal is in an abnormal state, and the user's emotions are difficult to be aroused; then the current user response result is compared with the third dimension of the SAM scale, which is emotional dominance, namely, dominance, and it is concluded that the current user's dominance should be 1, namely: "completely controlled", indicating that the current user's dominance is in an abnormal state, and the user's emotions are completely controlled by depression, and it is difficult to arouse positive emotions through external events, so it is concluded that the current user's mental health is in an abnormal state; thereby completing the process of user mental health monitoring and optimizing the effectiveness of existing user mental health monitoring methods.
[0132] In order to better demonstrate the operation process of the user psychological health monitoring system based on heterogeneous graph, the present invention provides a method for user psychological health monitoring based on heterogeneous graph, comprising the following steps:
[0133] S1. Collect and organize user-related social images and text data, and use the SAM scale to conduct a preliminary emotional assessment of users;
[0134] S2. Use hierarchical aggregation method to aggregate and analyze user social data and SAM scale response information to obtain different user emotion nodes and adjacent nodes;
[0135] S3. Create a heterogeneous graph neural network model based on a double-layer attention mechanism, perform node aggregation at the semantic level and node level, and then fuse adjacent nodes;
[0136] S4. Importing the training set and the test set into the heterogeneous graph neural network model based on the double-layer attention mechanism in sequence, and optimizing the heterogeneous graph neural network model based on the double-layer attention mechanism;
[0137] S5, after preprocessing the current user heterogeneous graph information, import it into the heterogeneous graph neural network model based on the double-layer attention mechanism for testing, and output the aggregation result, which includes: the user's current emotion and its factors;
[0138] S6. Create a normal psychological three-dimensional reference table based on the SAM scale, compare the current aggregation results with the psychological three dimensions in the table, and obtain the final user mental health monitoring results.
[0139] The beneficial effects achieved by the above content are as follows: Through the above method, the data diversity of mental health monitoring is optimized; the hierarchical aggregation method is used to aggregate and classify image data and various text information, and a heterogeneous graph neural network model based on a double-layer attention mechanism is constructed to aggregate different emotional nodes again, so as to obtain a more comprehensive aggregation result of the current user's emotions, thereby facilitating the judgment of the user's multiple emotions and the related factors causing the emotions; a normal psychological three-dimensional reference table is created, and the current user's emotions and their related factors are checked according to the psychological three dimensions. By analyzing the pleasure, arousal and controllability of the current user's emotions, a more comprehensive user mental health monitoring result is judged, thereby optimizing the effectiveness of existing mental health monitoring methods.
[0140] Working principle: By collecting user social information, the hierarchical aggregation method is used to aggregate and classify images and texts to obtain parameters of multiple emotional nodes and adjacent nodes; a heterogeneous graph neural network model based on a double-layer attention mechanism is constructed to aggregate different emotional nodes again, and then the adjacent nodes are fused to obtain a more comprehensive aggregation result of the current user's emotions, which is convenient for judging the user's multiple emotions and the related factors that cause the emotions; a normal psychological three-dimensional reference table is created, and the current user's emotions and their related factors are checked according to the psychological three dimensions. By analyzing the pleasure, arousal and control of the current user's emotions, a more comprehensive user mental health monitoring result is determined.
[0141] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.
[0142] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the invention.
Claims
1. A user mental health monitoring system based on heterogeneous graphs, including: The first information collection unit is used to Collect user information and user social media data from multiple social platforms through web crawler technology, extract user portrait information, text information, and topic information from social media, and form a heterogeneous graph of users based on this; The second information collection unit is used to Use the SAM scale method to ask users questions about different topics and get their responses. While asking users questions, monitor their blood pressure, heart rate, and body language in real time, and record their physiological information in the corresponding answer list. Association analysis unit, used for A hierarchical aggregation method is used to aggregate and analyze user information, user social media data, and user reply information under different topic information, and the text semantic information in the heterogeneous graph is introduced to obtain different user emotion nodes and their associated neighbor information; Aggregate processing unit for Create a heterogeneous graph neural network model, introduce a two-layer attention mechanism, use semantic-level attention and node-level attention to perform semantic-level aggregation and node-level aggregation respectively, and obtain the aggregated representation of adjacent nodes; and fuse the aggregated representations of adjacent nodes to obtain a more comprehensive aggregated representation; Psychometric unit for The user's image information and semantic information are imported into a heterogeneous graph neural network model based on a double-layer attention mechanism for testing. The user's current mental health status is judged based on the aggregation results output by the heterogeneous graph neural network model.
2. The user psychological health monitoring system based on heterogeneous graph according to claim 1 is characterized by: The first information collection unit includes The portrait extraction module is used to analyze the micro-expressions of the portrait information in the user's social media data, segment the portrait information according to the different micro-expressions, and set emotional labels for the micro-expressions based on the semantic content of the portrait information; The text extraction module is used to perform text summarization on the short texts generated in the user's social media data and extract the topic information and factors in the short texts; The text segmentation module is used to segment the short text generated in the user's social media data according to the text context and expression emotions, and obtain the semantic content of the user's expression under different emotions; The information association module is used to match and associate the semantic content expressed by the user under different emotions with the micro-expressions of the user portrait, and obtain the emotional facial features of the user under different themes.
3. The user psychological health monitoring system based on heterogeneous graph according to claim 2 is characterized by: The first information collection unit also includes A real-time monitoring module, used for real-time monitoring of the data processing time of the image extraction module, the text extraction module, the text segmentation module and the information association module; A time acquisition module, used for acquiring the end time of data processing of the image extraction module, the text extraction module, the text segmentation module and the information association module for the same image information according to the data processing duration of the image extraction module, the text extraction module, the text segmentation module and the information association module; A time difference period acquisition module is used to sort the data processing end times of the portrait extraction module, the text extraction module, the text segmentation module and the information association module for the same portrait information in chronological order, and to acquire the time difference period between two adjacent times; The data processing synchronization coefficient acquisition module is used to obtain the data processing synchronization coefficient according to the time difference period between the data processing end time of the same portrait information by the portrait extraction module, the text extraction module, the text segmentation module and the information association module; wherein the data processing synchronization coefficient is obtained by the following formula: Where P represents the data processing synchronization coefficient; n represents the number of time difference periods; T di represents the duration corresponding to the ith time difference period; T dmax Indicates the maximum duration of n time difference periods; T dmin represents the minimum duration of n time difference periods; m represents the number of image extraction modules, text extraction modules, text segmentation modules and information association modules, that is, m = 4; T si represents the average data processing time corresponding to the i-th module; T zi represents the preset data processing time reference value corresponding to the i-th module; λ represents the adjustment coefficient, and the adjustment coefficient is obtained by the following formula: Where, λ represents the adjustment coefficient; T 01si T represents the data processing time of the current portrait information of the module preceding the data processing end time corresponding to the i-th time difference period; 01zi T represents the theoretical data processing time of the current image information of the module preceding the data processing end time corresponding to the i-th time difference period; 02si T represents the data processing time of the current portrait information of the module whose data processing end time is later than the i-th time difference period; 02zi Indicates the theoretical data processing time of the current portrait information of the module whose data processing end time is later than the i-th time difference period; The synchronous operation determination module is used to determine the data processing synchronous operation quality of the first information collection unit according to the data processing synchronization coefficient corresponding to each portrait information processing.
4. The user psychological health monitoring system based on heterogeneous graph according to claim 3 is characterized by: The synchronous operation determination module includes: A data processing synchronization coefficient extraction module is used to extract the data processing synchronization coefficient corresponding to each portrait information processing; A coefficient difference acquisition module is used to acquire the coefficient difference between each two adjacent data processing synchronization coefficients according to the data processing synchronization coefficient corresponding to each portrait information processing; The synchronization evaluation parameter acquisition module is used to obtain the synchronization evaluation parameter by using the coefficient difference between each two adjacent data processing synchronization coefficients; wherein the synchronization evaluation parameter is obtained by the following formula: Where Q represents the synchronization evaluation parameter; k represents the number of processed image information; P i represents the data processing synchronization coefficient corresponding to the i-th image information; P i-1 represents the data processing synchronization coefficient corresponding to the i-1th image information; P i-2 represents the data processing synchronization coefficient corresponding to the i-2th image information; P f represents the data processing synchronization coefficient corresponding to the first portrait information; P f Indicates the data processing synchronization coefficient corresponding to the nth portrait information; A comparison module, used for comparing the synchronization evaluation parameter with a preset parameter threshold to obtain a comparison result; The abnormality determination module is used to determine that the data processing synchronous operation quality is abnormal and issue an abnormality alarm when the comparison result shows that the synchronization evaluation parameter is lower than a preset parameter threshold.
5. The user psychological health monitoring system based on heterogeneous graph according to claim 1 is characterized by: The second information collection unit includes The emotion acquisition module is used to measure the user's emotions through the SAM scale to obtain the user's emotional pleasure, arousal and control; The physiological acquisition module is a wearable heart rate monitor that is used to measure the user's emotions, collect the user's heart rate and blood pressure values in real time, and judge the user's current emotional fluctuations based on the fluctuations of the heart rate and blood pressure values; The behavior acquisition module is a high-definition camera that is used to measure the user's emotions, capture the user's body movements in real time, and judge the user's current emotional fluctuations based on the amplitude of the body movements.
6. The user psychological health monitoring system based on heterogeneous graph according to claim 5 is characterized by: The second information collection unit also includes The text recording module is used to record the user's emotional pleasure, arousal and control, the user's real-time heart rate and blood pressure values, and the user's body movements in a table to form a complete user physiological data monitoring table.
7. The user psychological health monitoring system based on heterogeneous graph according to claim 1 is characterized by: The association analysis unit comprises The node aggregation module is used to find the neighbor factor information of the user's emotion under a certain relationship for a node, and aggregate the neighbor factor information to obtain the representation of a certain emotion node under a certain relationship; The semantic aggregation module is used to fuse neighbor node representations under multiple relationships at the semantic level to obtain a more comprehensive representation of a certain emotion node.
8. The user psychological health monitoring system based on heterogeneous graph according to claim 1 is characterized by: The aggregation processing unit comprises The data processing module is used to clean and uniformly format the first information and the second information of the user to obtain pre-processed user data; the user data is then divided into a training set and a test set as a basis for optimizing the parameters of the heterogeneous graph neural network model; The model optimization module is used to import the training set into the heterogeneous graph neural network model based on the double-layer attention mechanism for training, output the aggregated training results, and establish the attention feature mechanism of the heterogeneous graph neural network model; then import the test set into the heterogeneous graph neural network model based on the double-layer attention mechanism for testing, and output the aggregated test results; after the training results are consistent with the test results, the optimal heterogeneous graph neural network model based on the double-layer attention mechanism is obtained.
9. The user psychological health monitoring system based on heterogeneous graph according to claim 1 is characterized by: The psychometric unit comprises The result analysis module is used to analyze and judge the aggregated results of the current user's emotions output by the heterogeneous graph neural network model to obtain the current user's emotional status and factors; The condition verification module is used to preset a normal psychological three-dimensional reference table based on the SAM scale, and verify the current user's emotional condition and related factors with the psychological three dimensions in the table to obtain a more comprehensive user mental health monitoring result.
10. A method for monitoring user psychological health based on a heterogeneous graph, implemented based on the user psychological health monitoring system based on a heterogeneous graph according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1. Collect and organize user-related social images and text data, and use the SAM scale to conduct a preliminary emotional assessment of users; S2. Use hierarchical aggregation method to aggregate and analyze user social data and SAM scale response information to obtain different user emotion nodes and adjacent nodes; S3. Create a heterogeneous graph neural network model based on a double-layer attention mechanism, perform node aggregation at the semantic level and node level, and then fuse adjacent nodes; S4. Importing the training set and the test set into the heterogeneous graph neural network model based on the double-layer attention mechanism in sequence, and optimizing the heterogeneous graph neural network model based on the double-layer attention mechanism; S5, after preprocessing the current user heterogeneous graph information, import it into the heterogeneous graph neural network model based on the double-layer attention mechanism for testing, and output the aggregation result, which includes: the user's current emotion and its factors; S6. Create a normal psychological three-dimensional reference table based on the SAM scale, compare the current aggregation results with the psychological three dimensions in the table, and obtain the final user mental health monitoring results.
Citation Information
Patent Citations
Psychological crisis early warning method based on text and image information joint calculation
CN112687374A
Social recommendation method based on multi-feature heterogeneous graph neural network
CN113254803A
Heterogeneous graph-based user mental health monitoring method and system
CN113345590A
Social user depression tendency detection method based on heterogeneous graph attention network
CN114628008A
Method for improving Bert performance in sentiment analysis based on aspects
CN114722799A
Cited By
Mental health assessment system based on big data
CN120260938A
