Mental health prediction method and device, electronic device, and storage medium

Abstract

The disclosed embodiments provide a mental health prediction method and device, an electronic device, and a storage medium, and relate to the field of artificial intelligence technology. The mental health prediction method includes: obtaining sample data; performing data preprocessing on the sample data to obtain a multimodal hyperedge set; using the multimodal hyperedge set as an initial hypergraph structure and inputting it into a preset original neural network model to generate an initial neural network model; performing model construction processing on the initial neural network model to obtain a first neural network model; training the first neural network model to obtain a trained second neural network model; inputting the multimodal hyperedge set corresponding to the sample data into the second neural network model; predicting the sample data through the second neural network model to obtain a mental health prediction result. The technical solution provided by the disclosed embodiments can improve the accuracy of mental health prediction and enhance the monitoring effect of mental health conditions.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a mental health prediction method and device, electronic equipment, and storage medium. Background Art

[0002] Monitoring and intervention of adolescent mental health is an important part of mental health defense work. Testing and screening of adolescent mental health can help us understand their current mental health status, so that we can give timely attention or take corresponding measures to avoid the expansion or aggravation of adolescent psychological problems, help adolescents with serious psychological problems recover their health, and realize real-time monitoring and timely intervention of adolescent mental health.

[0003] The main current methods for detecting adolescent mental health, such as psychological questionnaires and psychological counseling, are effective for more serious psychological problems or mental illnesses. However, for many adolescents with mild psychological problems, such as psychological imbalance or mild mental unhealthiness, the above traditional detection methods lack specificity and are one-sided. They cannot provide comprehensive and rapid screening test results, resulting in low prediction accuracy of mental health conditions and poor monitoring effects. Summary of the invention

[0004] The main purpose of the embodiments of the present disclosure is to propose a mental health prediction method and device, an electronic device, and a storage medium, which can improve the accuracy of mental health prediction and enhance the monitoring effect of mental health conditions.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present disclosure proposes a mental health prediction method, comprising:

[0006] Acquire sample data; the sample data includes video data, text data, voice data, and physiological indicator data;

[0007] Performing data preprocessing on the sample data to obtain a multimodal hyperedge set;

[0008] The multimodal hyperedge set is used as an initial hypergraph structure and input into a preset original neural network model to generate an initial neural network model; the original neural network model is a dynamic hypergraph neural network model;

[0009] Performing model building processing on the initial neural network model to obtain a first neural network model;

[0010] Training the first neural network model according to preset model training data to obtain a trained second neural network model;

[0011] Inputting the multimodal hyperedge set corresponding to the sample data into the second neural network model;

[0012] The sample data is predicted by the second neural network model to obtain a mental health prediction result.

[0013] In some embodiments, performing data preprocessing on the sample data to obtain a multimodal hyperedge set includes:

[0014] Performing feature extraction processing on the sample data to obtain initial features;

[0015] Performing vector representation processing on the initial features to obtain feature vectors;

[0016] constructing a hyperedge set according to the feature vector to obtain a unimodal hyperedge set;

[0017] The unimodal hyperedge set is subjected to hyperedge set fusion to obtain the multimodal hyperedge set.

[0018] In some embodiments, the step of performing feature extraction processing on the sample data to obtain initial features includes:

[0019] If the sample data is the video data, extracting video features according to a preset first convolutional recurrent neural network;

[0020] If the sample data is the text data, extracting text features according to a preset Chinese sentence sentiment polarity analysis network;

[0021] If the sample data is the speech data, extracting speech features according to a preset second convolutional recurrent neural network;

[0022] If the sample data is the physiological indicator data, the physiological indicator features are extracted according to a preset threshold function.

[0023] In some embodiments, the performing model building processing on the initial neural network model to obtain a first neural network model includes:

[0024] Performing a hypergraph construction process on the initial neural network model to obtain a first sub-neural network model;

[0025] Performing hypergraph convolution processing on the first sub-neural network model to obtain intermediate features;

[0026] The first sub-neural network model is subjected to hypergraph iterative construction processing according to the intermediate features to obtain the first neural network model.

[0027] In some embodiments, the performing of a hypergraph construction process on the initial neural network model to obtain a first sub-neural network model includes:

[0028] For each node in the initial neural network model, calculate adjacent nodes according to a preset proximity algorithm, and obtain a first hyperedge according to the adjacent nodes;

[0029] Clustering the nodes according to a preset clustering algorithm to obtain cluster nodes, and obtaining a second hyperedge according to the cluster nodes;

[0030] A hypergraph is constructed according to the first hyperedge and the second hyperedge to obtain the first sub-neural network model.

[0031] In some embodiments, performing hypergraph convolution processing on the first sub-neural network model to obtain intermediate features includes:

[0032] Performing node convolution processing on the first sub-neural network model to obtain a first sub-feature;

[0033] The first sub-feature is subjected to hyper-edge convolution processing according to a preset multi-layer perceptron to obtain the intermediate feature.

[0034] In some embodiments, performing node convolution processing on the first sub-neural network model to obtain the first sub-feature includes:

[0035] Performing feature learning on each node in the first sub-neural network model to obtain second sub-features, and constructing a transformation matrix;

[0036] Modeling the relationship between nodes in the same hyperedge according to the second sub-feature and the transformation matrix to obtain a modeling result;

[0037] Perform dimension compression processing on the modeling result to obtain the first sub-feature.

[0038] To achieve the above-mentioned purpose, the second aspect of the present disclosure proposes a mental health prediction device, comprising:

[0039] A data acquisition module, used to acquire sample data; the sample data includes video data, text data, voice data, and physiological index data;

[0040] A preprocessing module, used for performing data preprocessing on the sample data to obtain a multimodal hyperedge set;

[0041] An initial model generation module, used to input the multimodal hyperedge set as an initial hypergraph structure into a preset original neural network model to generate an initial neural network model; the original neural network model is a dynamic hypergraph neural network model;

[0042] A model building module, used to perform model building processing on the initial neural network model to obtain a first neural network model;

[0043] A model training module, used to train the first neural network model according to preset model training data to obtain a trained second neural network model;

[0044] A data input module, used for inputting the multimodal hyperedge set corresponding to the sample data into the second neural network model;

[0045] The mental health prediction module is used to predict the sample data through the second neural network model to obtain a mental health prediction result.

[0046] To achieve the above objective, a third aspect of the present disclosure provides an electronic device, including:

[0047] at least one memory;

[0048] at least one processor;

[0049] at least one program;

[0050] The program is stored in a memory, and the processor executes at least one program to implement the method of the present disclosure as described in the first aspect above.

[0051] To achieve the above objective, a fourth aspect of the present disclosure provides a storage medium, which is a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute:

[0052] As the method according to the first aspect above.

[0053] The mental health prediction method and device, electronic device, and storage medium proposed in the embodiments of the present disclosure first obtain sample data, and perform data preprocessing on the sample data to obtain a multimodal hyperedge set, and then use the multimodal hyperedge set as the initial hypergraph structure and input it into a preset original neural network model to generate an initial neural network model, and then perform model construction processing on the initial neural network model, and train the model according to the model training data, and finally input the multimodal hyperedge set corresponding to the sample data into the trained model, predict the sample data, and obtain a mental health prediction result. The technical solution provided by the embodiments of the present disclosure can improve the accuracy of mental health prediction and enhance the monitoring effect of mental health conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of the mental health prediction method provided by the embodiment of the present disclosure.

[0055] Figure 2 yes Figure 1 Flowchart of step S120 in .

[0056] Figure 3 yes Figure 2 Flowchart of step S210 in .

[0057] Figure 4 yes Figure 1 Flowchart of step S140 in .

[0058] Figure 5 yes Figure 4 Flowchart of step S410 in .

[0059] Figure 6 yes Figure 4 Flowchart of step S420 in .

[0060] Figure 7 yes Figure 6 Flowchart of step S610 in .

[0061] Figure 8 It is a module block diagram of the mental health prediction device provided in an embodiment of the present disclosure.

[0062] Fig. 9 It is a schematic diagram of the hardware structure of the electronic device provided by the embodiment of the present disclosure.

[0063] Figure numerals: data acquisition module 810, preprocessing module 820, initial model generation module 830, model building module 840, model training module 850, data input module 860, mental health prediction module 870, processor 901, memory 902, input / output interface 903, communication interface 904, bus 905. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] It should be noted that, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. The terms "first", "second", etc. in the specification, claims and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0067] Monitoring and intervention of adolescent mental health is an important part of mental health defense work. Testing and screening of adolescent mental health can help us understand their current mental health status, so that we can give timely attention or take corresponding measures to avoid the expansion or aggravation of adolescent psychological problems, help adolescents with serious psychological problems recover their health, and realize real-time monitoring and timely intervention of adolescent mental health.

[0068] The main current methods for detecting adolescent mental health, such as psychological questionnaires and psychological counseling, are effective for more serious psychological problems or mental illnesses. However, for many adolescents with mild psychological problems, such as psychological imbalance or mild mental unhealthiness, the above traditional detection methods lack specificity and are one-sided. They cannot provide comprehensive and rapid screening test results, resulting in low prediction accuracy of mental health conditions and poor monitoring effects.

[0069] Therefore, it is an urgent need to use advanced neural network technology to invent a method that can comprehensively and quickly detect the mental health of adolescents.

[0070] Based on this, the embodiments of the present disclosure provide a mental health prediction method and device, an electronic device, and a storage medium. Sample data is first acquired and preprocessed to obtain a multimodal hyperedge set. The multimodal hyperedge set is then used as an initial hypergraph structure and input into a preset original neural network model to generate an initial neural network model. The initial neural network model is then subjected to model construction processing and the model is trained according to the model training data. Finally, the multimodal hyperedge set corresponding to the sample data is input into the trained model, the sample data is predicted, and a mental health prediction result is obtained. The technical solution provided by the embodiments of the present disclosure can improve the accuracy of mental health prediction and enhance the monitoring effect of mental health conditions.

[0071] Specifically, the disclosed embodiments provide a method for predicting mental health, which is based on neural network and hypergraph technology, extracts and integrates multi-modal sample data of adolescent testers, extracts features of each modality as unit sample features through neural network, constructs hyperedge sets for each modal feature of the sample, and then merges these single-modal hyperedge sets into a multi-modal hyperedge set; the multi-modal hyperedge set is the preprocessing result of the sample data, and will subsequently be input into the network as an initial hypergraph structure; the dynamic hypergraph neural network is constructed and trained and optimized using the labeled sample data; the data of the adolescent mental health testers is then input into it, and through multi-layer network evolution and feature clustering, the prediction result is finally output as the mental health test result of the adolescent tester, thereby realizing screening and detection of adolescent mental health.

[0072] The mental health prediction method provided in the embodiment of the present disclosure is a method for screening and detecting the mental health of adolescents using a dynamic hypergraph neural network framework (DHGNN).

[0073] Among them, the dynamic hypergraph neural network is a new type of hypergraph neural network model, which includes two parts: dynamic hypergraph construction (DHG) and hypergraph convolution (HGC). Specifically, the dynamic hypergraph construction is used to dynamically update the hypergraph structure of each layer; while the hypergraph convolution uses node convolution and hyperedge convolution to collect point and edge information.

[0074] The advantages of using dynamic hypergraph neural networks for mental health prediction include at least:

[0075] (1) Compared with other hypergraph neural networks, dynamic hypergraph construction can dynamically update the hypergraph structure, better represent the distribution of multimodal data at a deep level, and mine the high-order relationships between different modal features of youth detection samples;

[0076] (2) Hypergraph convolution uses a fixed-size, weight-sharing learnable convolution kernel for feature extraction, which can better aggregate information and perform more reliable feature clustering for adolescent detection samples;

[0077] (3) Compared with traditional psychological testing methods, dynamic hypergraph neural networks are faster and can be used to achieve faster, more accurate and reliable adolescent mental health testing.

[0078] In a specific embodiment, when DHGNN is used for adolescent mental health screening, a node represents an adolescent test sample, and a hyperedge represents a common attribute between the samples represented by all nodes contained in this edge. Dynamically updating the hypergraph structure can continuously generate new embedded features as the network goes deeper, capturing the high-order relationship of the embedded features; the hypergraph convolution operation can extract the local and global relationships between the features of each modality of the sample.

[0079] The mechanisms of adolescent psychological problems are complex and diverse, and the phenotypes overlap. DHGNN can obtain more comprehensive and reliable prediction results by learning multiple modal features.

[0080] The embodiments of the present disclosure provide a mental health prediction method and device, an electronic device, and a storage medium, which are specifically described through the following embodiments. First, the mental health prediction method in the embodiments of the present disclosure is described.

[0081] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0082] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0083] The mental health prediction method provided by the embodiments of the present disclosure relates to the field of artificial intelligence technology, and in particular to the field of data prediction technology. The mental health prediction method provided by the embodiments of the present disclosure can be applied to a terminal, or to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, or a smart watch, etc.; the server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms; the software can be an application that implements the mental health prediction method, etc., but is not limited to the above forms.

[0084] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0085] The disclosed embodiment proposes a method for predicting mental health, including: obtaining sample data; the sample data includes video data, text data, voice data, and physiological indicator data; performing data preprocessing on the sample data to obtain a multimodal hyperedge set; using the multimodal hyperedge set as an initial hypergraph structure, and inputting it into a preset original neural network model to generate an initial neural network model; the original neural network model is a dynamic hypergraph neural network model; performing model construction processing on the initial neural network model to obtain a first neural network model; training the first neural network model according to preset model training data to obtain a trained second neural network model; inputting the multimodal hyperedge set corresponding to the sample data into the second neural network model; predicting the sample data through the second neural network model to obtain a mental health prediction result.

[0086] Figure 1 is an optional flowchart of the mental health prediction method provided by the embodiment of the present disclosure, Figure 1 The method may include but is not limited to steps S110 to S170, specifically including:

[0087] S110, obtaining sample data;

[0088] S120, performing data preprocessing on the sample data to obtain a multimodal hyperedge set;

[0089] S130, inputting the multimodal hyperedge set as an initial hypergraph structure into a preset original neural network model to generate an initial neural network model;

[0090] S140, performing model building processing on the initial neural network model to obtain a first neural network model;

[0091] S150, training the first neural network model according to preset model training data to obtain a trained second neural network model;

[0092] S160, inputting the multimodal hyperedge set corresponding to the sample data into the second neural network model;

[0093] S170, predicting the sample data through the second neural network model to obtain a mental health prediction result.

[0094] In step S110, the sample data is data of multiple modalities, including but not limited to video data, text data, voice data, and physiological indicator data, wherein the physiological indicator data includes but is not limited to heart rate, blood pressure, heart rate variability, heart rate signal-to-noise ratio, breathing frequency, tension value, relaxation value and other indicators.

[0095] In step S120, the purpose of data preprocessing is to fuse sample data of multiple modalities to obtain a multimodal hyperedge set for use in subsequent model building and prediction processes to improve the accuracy of prediction.

[0096] In step S130, the original neural network model is a dynamic hypergraph neural network model. Therefore, subsequent models obtained through various means using the original neural network model are all dynamic hypergraph neural network models, such as the initial neural network model, the first neural network model, and the second neural network model.

[0097] In step S140, the first neural network model is a network model obtained after model construction. At this time, the structure of the model is relatively complete, and model training can be performed to achieve model prediction.

[0098] In step S150, the preset model training data is the data used for model training. Specifically, DHGNN is built using python, and the labeled data is used as a training set and a test set respectively to complete the training and optimization of the network model.

[0099] In steps S160 to S170, the multimodal hyperedge set corresponding to the sample data is input into the trained network model, and after multiple layers of hypergraph construction and iteration of the hypergraph convolutional network, the mental health prediction result is finally obtained. The mental health prediction result includes the output label of the model, including health, compulsion, depression, paranoia, anxiety, etc.

[0100] The mental health prediction method proposed in the embodiment of the present disclosure first obtains sample data and performs data preprocessing on the sample data to obtain a multimodal hyperedge set, and then uses the multimodal hyperedge set as the initial hypergraph structure and inputs it into a preset original neural network model to generate an initial neural network model, and then performs model construction processing on the initial neural network model, and trains the model according to the model training data, and finally inputs the multimodal hyperedge set corresponding to the sample data into the trained model, predicts the sample data, and obtains a mental health prediction result. The technical solution provided by the embodiment of the present disclosure can improve the accuracy of mental health prediction and enhance the monitoring effect of mental health conditions.

[0101] In some embodiments, data preprocessing is performed on sample data to obtain a multimodal hyperedge set, including: performing feature extraction processing on the sample data to obtain initial features; performing vector representation processing on the initial features to obtain feature vectors; constructing hyperedge sets based on the feature vectors to obtain unimodal hyperedge sets; and performing hyperedge set fusion on unimodal hyperedge sets to obtain a multimodal hyperedge set.

[0102] Figure 2 is a flowchart of step S120 in some embodiments, Figure 2The illustrated step S120 includes but is not limited to steps S210 to S240:

[0103] S210, performing feature extraction processing on the sample data to obtain initial features;

[0104] S220, performing vector representation processing on the initial features to obtain feature vectors;

[0105] S230, constructing a hyperedge set according to the feature vector to obtain a unimodal hyperedge set;

[0106] S240, performing super-edge set fusion on the single-modal super-edge sets to obtain a multi-modal super-edge set.

[0107] In step S210, data of different modalities are processed using respective appropriate neural network methods to extract initial features corresponding to sample data of each modality.

[0108] In step S220, the data feature classification result of each modality is represented by an n-dimensional feature vector, and the feature vector corresponding to the sample data of each modality is obtained.

[0109] In step S230, a hyperedge set is constructed for each modal feature of the sample, and a unimodal hyperedge set corresponding to the sample data of each modality is obtained.

[0110] In step S240, these unimodal hyperedge sets are merged into a multimodal hyperedge set.

[0111] In some embodiments, feature extraction processing is performed on sample data to obtain initial features, including: if the sample data is video data, video features are extracted according to a preset first convolutional recurrent neural network; if the sample data is text data, text features are extracted according to a preset Chinese sentence sentiment polarity analysis network; if the sample data is speech data, speech features are extracted according to a preset second convolutional recurrent neural network; if the sample data is physiological indicator data, physiological indicator features are extracted according to a preset threshold function.

[0112] Figure 3 is a flowchart of step S210 in some embodiments, Figure 3 The illustrated step S210 includes but is not limited to steps S310 to S340:

[0113] S310, if the sample data is video data, extracting video features according to a preset first convolutional recurrent neural network;

[0114] S320, if the sample data is text data, extracting text features according to a preset Chinese sentence sentiment polarity analysis network;

[0115] S330, if the sample data is speech data, extracting speech features according to a preset second convolutional recurrent neural network;

[0116] S340: If the sample data is physiological indicator data, extract physiological indicator features according to a preset threshold function.

[0117] In step S310, for the video data, a two-dimensional convolutional recurrent neural network (CRNN) is used to extract video features and classify them. The first convolutional recurrent neural network is a 2D CRNN network.

[0118] In step S320, for text data, different data sets are selected to perform polarity classification of different sentiment sentences, pre-training is performed through a public sentiment dataset, and the output text features of the trained Chinese sentence sentiment polarity analysis network are used.

[0119] In step S330, for the speech data, a spectrogram is obtained, and the time domain features and frequency domain features are extracted respectively using the CRNN algorithm, and finally the attention pooling technology is used for classification output, wherein the first convolutional recursive neural network is a CRNN network.

[0120] In step S340, for the physiological indicator data, the physiological indicator is classified by outputting an indicator through a threshold function to obtain a physiological indicator feature.

[0121] In some embodiments, the initial neural network model is subjected to model building processing to obtain a first neural network model, including: performing hypergraph building processing on the initial neural network model to obtain a first sub-neural network model; performing hypergraph convolution processing on the first sub-neural network model to obtain intermediate features; and performing hypergraph iterative construction processing on the first sub-neural network model according to the intermediate features to obtain the first neural network model.

[0122] Figure 4 is a flowchart of step S140 in some embodiments, Figure 4 The illustrated step S140 includes but is not limited to steps S410 to S430:

[0123] S410, performing hypergraph construction processing on the initial neural network model to obtain a first sub-neural network model;

[0124] S420, performing hypergraph convolution processing on the first sub-neural network model to obtain intermediate features;

[0125] S430, performing hypergraph iterative construction processing on the first sub-neural network model according to the intermediate features to obtain a first neural network model.

[0126] In step S410, the first sub-neural network model is a model obtained after the hypergraph construction process is performed.

[0127] In step S420, the intermediate feature is to extract features by performing a convolution operation on the hypergraph structure.

[0128] In step S430, the first neural network model is a completed network model obtained after iterative construction.

[0129] In steps S410 to S430, the model construction work of DHGNN includes: constructing a hypergraph structure through the relationship between nodes, then performing convolution operations on the hypergraph structure to extract features, and then constructing the hypergraph structure again based on the newly extracted features, thereby achieving the update of the hypergraph structure.

[0130] It should be noted that steps S410 to S430 represent an evolution of a hypergraph network. This evolution process can be iterated multiple times, so that the relationship between nodes is dynamically adjusted multiple times. The number of iterations can be appropriately adjusted according to the actual construction situation, required accuracy and other conditions.

[0131] In some embodiments, a hypergraph is constructed on the initial neural network model to obtain a first sub-neural network model, including: for each node in the initial neural network model, calculating adjacent nodes according to a preset proximity algorithm, and obtaining a first hyperedge based on the adjacent nodes; clustering the nodes according to a preset clustering algorithm to obtain cluster nodes, and obtaining a second hyperedge based on the cluster nodes; and constructing a hypergraph based on the first hyperedge and the second hyperedge to obtain the first sub-neural network model.

[0132] Figure 5 is a flowchart of step S410 in some embodiments, Figure 5 The illustrated step S410 includes but is not limited to steps S510 to S530:

[0133] S510, for each node in the initial neural network model, calculating adjacent nodes according to a preset adjacent algorithm, and obtaining a first hyperedge according to the adjacent nodes;

[0134] S520, clustering the nodes according to a preset clustering algorithm to obtain cluster nodes, and obtaining a second hyperedge according to the cluster nodes;

[0135] S530, construct a hypergraph according to the first hyperedge and the second hyperedge to obtain a first sub-neural network model.

[0136] In step S510, the hypergraph is represented in the form of an adjacency matrix, and the matrix size is |N|*|E|, which represents the number of nodes and the number of edges respectively, wherein the related nodes and edges are represented as h(n,e)=1, and the rest are represented as h(n,e)=0. First, for each node, the KNN (KNN, K-NearestNeighbor) method is used to find the n nodes that are most similar to the node (i.e., the closest in the feature space) to form a hyperedge, thus obtaining |N| hyperedges.

[0137] Among them, the first hyperedge represents the hyperedge formed by adjacent nodes found by the KNN method; the neighboring algorithm is also called the K nearest neighbor classification algorithm.

[0138] In step S520, a clustering method such as kmeans is then used to circle K center points in the nodes. For each node, it is assigned to the nearest S center nodes to form a hyperedge, and K new hyperedges are obtained.

[0139] The second hypertable represents the hyperedges obtained by using the nodes circled by the clustering algorithm; the Kmeans algorithm believes that if most of the most similar samples of a sample in the feature space belong to a certain category, then the sample also belongs to this category.

[0140] In step S530, the first hyperedge and the second hyperedge are used together to construct the model, so that the hypergraph construction process combines the KNN and Kmeans methods, thereby improving the prediction accuracy of the model.

[0141] In some embodiments, hypergraph convolution processing is performed on the first sub-neural network model to obtain intermediate features, including: performing node convolution processing on the first sub-neural network model to obtain first sub-features; and performing hyperedge convolution processing on the first sub-features according to a preset multi-layer perceptron to obtain intermediate features.

[0142] Figure 6 is a flowchart of step S420 in some embodiments, Figure 6 The illustrated step S420 includes but is not limited to steps S610 to S620:

[0143] S610, performing node convolution processing on the first sub-neural network model to obtain a first sub-feature;

[0144] S620, performing hyper-edge convolution processing on the first sub-feature according to a preset multi-layer perceptron to obtain an intermediate feature.

[0145] In a specific embodiment, the hypergraph convolution process includes a node convolution in step S610 and a hyperedge convolution in step S620. The node convolution aggregates node features to a hyperedge, and then the hyperedge convolution aggregates adjacent hyperedge features to a central node.

[0146] In step S610, node convolution aggregates node features to hyperedges, wherein the first sub-feature is the feature of the node obtained by the convolution operation, which is used to generate the final intermediate feature.

[0147] In step S620, the hyperedge convolution uses the first sub-feature MLP (Multilayer Perceptron) obtained in step S610 to calculate its weight. For each central node, the weighted sum of all the edges associated with it is calculated to obtain the feature of the central node.

[0148] The calculation process of the intermediate features in step S620 is as follows:

[0149] w = softmax(x e W+b) (Formula 1)

[0150]

[0151] Where |E| is the size of the adjacent hyperedge set, x e is the adjacent hyperedge feature, X n is the central node feature, W and b are learnable parameters.

[0152] In some embodiments, node convolution processing is performed on the first sub-neural network model to obtain a first sub-feature, including: performing feature learning on each node in the first sub-neural network model to obtain a second sub-feature, and constructing a transformation matrix; modeling the relationship between nodes in the same hyperedge according to the second sub-feature and the transformation matrix to obtain a modeling result; and performing dimensionality compression processing on the modeling result to obtain the first sub-feature.

[0153] Figure 7 is a flowchart of step S610 in some embodiments, Figure 7 The illustrated step S610 includes but is not limited to steps S710 to S730:

[0154] S710, performing feature learning on each node in the first sub-neural network model to obtain a second sub-feature and construct a transformation matrix;

[0155] S720, modeling the relationship between nodes in the same hyperedge according to the second sub-feature and the transformation matrix to obtain a modeling result;

[0156] S730, performing dimension compression processing on the modeling result to obtain a first sub-feature.

[0157] In step S710, the node convolution first performs feature learning on the node to construct a transformation matrix, which is used for feature arrangement and weighting. The second sub-feature represents the node feature obtained in step S710.

[0158] In step S720, the mutual relationship between nodes in the same hyperedge is modeled and represented by multiplying the transformation matrix and the second sub-feature.

[0159] In step S730, a convolution operation is performed to compress the dimension to obtain a first sub-feature, where the first sub-feature represents the feature of the edge containing the node in step S720.

[0160] The calculation process of the first sub-feature in steps S710 to S730 is:

[0161] T=MLP(X n ) (Formula 3)

[0162] x e =conv(T·MLP(X n )) (Formula 4)

[0163] Among them, x e is the adjacent hyperedge feature, X n The central node feature.

[0164] The disclosed embodiment proposes a mental health prediction device, including: a data acquisition module, used to acquire sample data; the sample data includes video data, text data, voice data, and physiological indicator data; a preprocessing module, used to perform data preprocessing on the sample data to obtain a multimodal hyperedge set; an initial model generation module, used to input the multimodal hyperedge set as an initial hypergraph structure into a preset original neural network model to generate an initial neural network model; the original neural network model is a dynamic hypergraph neural network model; a model construction module, used to perform model construction processing on the initial neural network model to obtain a first neural network model; a model training module, used to train the first neural network model according to preset model training data to obtain a trained second neural network model; a data input module, used to input the multimodal hyperedge set corresponding to the sample data into the second neural network model; a mental health prediction module, used to predict the sample data through the second neural network model to obtain a mental health prediction result.

[0165] See also Figure 8 , Figure 8The diagram illustrates a mental health prediction device of an embodiment, which includes: a data acquisition module 810, a preprocessing module 820, an initial model generation module 830, a model construction module 840, a model training module 850, a data input module 860, and a mental health prediction module 870. The data acquisition module 810 is connected to the preprocessing module 820, the preprocessing module 820 is connected to the initial model generation module 830, the initial model generation module 830 is connected to the model construction module 840, the model construction module 840 is connected to the model training module 850, the model training module 850 is connected to the data input module 860, and the data input module 860 is connected to the mental health prediction module 870.

[0166] The specific implementation of the mental health prediction device of this embodiment is basically consistent with the specific implementation of the mental health prediction method described above, and belongs to the same inventive concept, and will not be repeated here.

[0167] The present disclosure also provides an electronic device, including:

[0168] at least one memory;

[0169] at least one processor;

[0170] at least one program;

[0171] The program is stored in the memory, and the processor executes the at least one program to implement the mental health prediction method disclosed in the present invention. The electronic device can be any intelligent terminal including a mobile phone, a tablet computer, a personal digital assistant (PDA), a car computer, etc.

[0172] See also Fig. 9 , Fig. 9 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0173] The processor 901 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present disclosure;

[0174] The memory 902 can be implemented in the form of ROM (Read Only Memory), static storage device, dynamic storage device or RAM (Random Access Memory). The memory 902 can store operating systems and other applications. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program codes are stored in the memory 902, and the processor 901 is used to call and execute the mental health prediction method of the embodiment of the present disclosure;

[0175] Input / output interface 903, used to implement information input and output;

[0176] Communication interface 904, used to realize communication interaction between the device and other devices, which can be realized by wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WI FI, Bluetooth, etc.); and

[0177] A bus 905 that transmits information between various components of the device (e.g., the processor 901, the memory 902, the input / output interface 903, and the communication interface 904);

[0178] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0179] The embodiment of the present disclosure also provides a storage medium, which is a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the above-mentioned mental health prediction method.

[0180] The mental health prediction method and device, electronic device, and storage medium proposed in the embodiments of the present disclosure first obtain sample data, and perform data preprocessing on the sample data to obtain a multimodal hyperedge set, and then use the multimodal hyperedge set as the initial hypergraph structure and input it into a preset original neural network model to generate an initial neural network model, and then perform model construction processing on the initial neural network model, and train the model according to the model training data, and finally input the multimodal hyperedge set corresponding to the sample data into the trained model, predict the sample data, and obtain a mental health prediction result. The technical solution provided by the embodiments of the present disclosure improves the accuracy of mental health prediction and enhances the monitoring effect of mental health conditions.

[0181] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely disposed relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0182] The embodiments described in the embodiments of the present disclosure are intended to more clearly illustrate the technical solutions of the embodiments of the present disclosure and do not constitute a limitation on the technical solutions provided by the embodiments of the present disclosure. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present disclosure are also applicable to similar technical problems.

[0183] It can be understood by those skilled in the art that Figure 1-7 The technical solutions shown in the figure do not constitute a limitation on the embodiments of the present disclosure, and may include more or fewer steps than those shown in the figure, or a combination of certain steps, or different steps.

[0184] The device embodiments described above are merely illustrative, and the units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0185] Those skilled in the art will appreciate that all or some of the steps in the methods disclosed above, and the functional modules / units in the systems and devices may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0186] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0187] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0188] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0189] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0190] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0191] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including multiple instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as ROM), random access memory (Random Access Memory, referred to as RAM), disk or optical disk and other media that can store programs.

[0192] The preferred embodiments of the present disclosure are described above with reference to the accompanying drawings, but the scope of the rights of the present disclosure is not limited thereto. Any modification, equivalent substitution and improvement made by those skilled in the art without departing from the scope and essence of the present disclosure should be within the scope of the rights of the present disclosure.

Claims

1. A method for predicting mental health, characterized in that: include: Acquire sample data; the sample data includes video data, text data, voice data, and physiological indicator data; Performing data preprocessing on the sample data to obtain a multimodal hyperedge set; The multimodal hyperedge set is used as an initial hypergraph structure and input into a preset original neural network model to generate an initial neural network model; the original neural network model is a dynamic hypergraph neural network model; Performing model building processing on the initial neural network model to obtain a first neural network model; Training the first neural network model according to preset model training data to obtain a trained second neural network model; Inputting the multimodal hyperedge set corresponding to the sample data into the second neural network model; The sample data is predicted by the second neural network model to obtain a mental health prediction result.

2. The method according to claim 1, characterized in that The performing data preprocessing on the sample data to obtain a multimodal hyperedge set includes: Performing feature extraction processing on the sample data to obtain initial features; Performing vector representation processing on the initial features to obtain feature vectors; constructing a hyperedge set according to the feature vector to obtain a unimodal hyperedge set; The unimodal hyperedge set is subjected to hyperedge set fusion to obtain the multimodal hyperedge set.

3. The method according to claim 2, characterized in that The step of performing feature extraction processing on the sample data to obtain initial features includes: If the sample data is the video data, extracting video features according to a preset first convolutional recurrent neural network; If the sample data is the text data, extracting text features according to a preset Chinese sentence sentiment polarity analysis network; If the sample data is the speech data, extracting speech features according to a preset second convolutional recurrent neural network; If the sample data is the physiological indicator data, the physiological indicator features are extracted according to a preset threshold function.

4. The method according to claim 1, characterized in that The performing model building processing on the initial neural network model to obtain a first neural network model includes: Performing a hypergraph construction process on the initial neural network model to obtain a first sub-neural network model; Performing hypergraph convolution processing on the first sub-neural network model to obtain intermediate features; The first sub-neural network model is subjected to hypergraph iterative construction processing according to the intermediate features to obtain the first neural network model.

5. The method according to claim 4, characterized in that The hypergraph construction process is performed on the initial neural network model to obtain a first sub-neural network model, including: For each node in the initial neural network model, calculate adjacent nodes according to a preset adjacent algorithm, and obtain a first hyperedge according to the adjacent nodes; Clustering the nodes according to a preset clustering algorithm to obtain cluster nodes, and obtaining a second hyperedge according to the cluster nodes; A hypergraph is constructed according to the first hyperedge and the second hyperedge to obtain the first sub-neural network model.

6. The method according to claim 4, characterized in that The step of performing hypergraph convolution processing on the first sub-neural network model to obtain intermediate features includes: Performing node convolution processing on the first sub-neural network model to obtain a first sub-feature; The first sub-feature is subjected to hyper-edge convolution processing according to a preset multi-layer perceptron to obtain the intermediate feature.

7. The method according to claim 6, characterized in that The performing node convolution processing on the first sub-neural network model to obtain the first sub-feature includes: Performing feature learning on each node in the first sub-neural network model to obtain second sub-features, and constructing a transformation matrix; Modeling the relationship between nodes in the same hyperedge according to the second sub-feature and the transformation matrix to obtain a modeling result; Perform dimension compression processing on the modeling result to obtain the first sub-feature.

8. A mental health prediction device, characterized in that: include: A data acquisition module, used to acquire sample data; the sample data includes video data, text data, voice data, and physiological index data; A preprocessing module, used for performing data preprocessing on the sample data to obtain a multimodal hyperedge set; An initial model generation module, used to input the multimodal hyperedge set as an initial hypergraph structure into a preset original neural network model to generate an initial neural network model; the original neural network model is a dynamic hypergraph neural network model; A model building module, used for performing model building processing on the initial neural network model to obtain a first neural network model; A model training module, used to train the first neural network model according to preset model training data to obtain a trained second neural network model; A data input module, used for inputting the multimodal hyperedge set corresponding to the sample data into the second neural network model; The mental health prediction module is used to predict the sample data through the second neural network model to obtain a mental health prediction result.

9. An electronic device, characterized in that: include: at least one memory; at least one processor; at least one program; The program is stored in the memory, and the processor executes at least one of the programs to implement: A method as claimed in any one of claims 1 to 7.

10. A storage medium, the storage medium being a computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to cause a computer to execute: A method as claimed in any one of claims 1 to 7.

Images