Psychological state prediction method and system based on geometric mean and graph structure

Abstract

The application discloses a psychological state prediction method and system based on geometric mean and graph structure, and the method comprises the following steps: acquiring multi-modal physiological data of a patient object; performing normalization processing on the multi-modal physiological data based on a geometric mean algorithm to obtain normalized data; performing feature extraction on the normalized data based on a graph network construction algorithm and a graph neural network to obtain graph psychological features; inputting the graph psychological features into a pre-trained LSTM neural network to obtain future psychological state parameters corresponding to the patient object; and the LSTM neural network is trained through a training data set comprising a plurality of training psychological state sequences and corresponding graph psychological feature labels. It can be seen that the application can realize cross-modal mapping from physiological data to psychological trend, and improve the continuity, timeliness and accuracy of psychological state prediction based on multi-modal physiological data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a psychological state prediction method and system based on geometric mean and graph structure. BACKGROUND

[0002] In the prior art, the prediction of psychological state usually relies on static scale data or single physiological signal for feature extraction and classification judgment, which is difficult to accurately reflect the psychological change trend of individuals in actual life scenes. Some schemes attempt to combine multi-modal physiological data such as heart rate, electroencephalogram, skin electricity, etc., but most of them use simple splicing or traditional statistical methods for processing, ignoring the complex relationship between modal data, resulting in limited feature fusion effect and insufficient model prediction ability. In addition, the existing methods mostly use rule matching or linear modeling means in time series modeling, lacking the mining of long-term dependence information, and it is difficult to realize the accurate dynamic prediction of psychological state. It can be seen that the existing technology has defects and needs to be solved. SUMMARY

[0003] The technical problem to be solved by the present application is to provide a psychological state prediction method and system based on geometric mean and graph structure, which can realize cross-modal mapping of physiological data to psychological trend, and improve the continuity, timeliness and accuracy of psychological state prediction based on multi-modal physiological data.

[0004] To solve the above technical problems, the first aspect of the present application discloses a psychological state prediction method based on geometric mean and graph structure, the method comprising:

[0005] obtaining multi-modal physiological data of a patient object;

[0006] normalizing the multi-modal physiological data based on a geometric mean algorithm to obtain normalized data;

[0007] performing feature extraction on the normalized data based on a graph network construction algorithm and a graph neural network to obtain graph psychological features;

[0008] inputting the graph psychological features into a pre-trained LSTM neural network to obtain future psychological state parameters corresponding to the patient object; the LSTM neural network is trained by a training data set comprising a plurality of training psychological state sequences and corresponding graph psychological feature labels.

[0009] As an optional implementation, in the first aspect of the present application, the multi-modal physiological data includes at least one of brain wave EEG data, heart rate data, respiratory rate data, text emotion score data, voice data and expression feature data.

[0010] As an optional implementation, in the first aspect of the present application, the geometric mean algorithm is used to normalize the multi-modal physiological data to obtain normalized data, including:

[0011] The multi-modal physiological data with the same time interval are grouped into a data group to obtain a plurality of data groups;

[0012] For each data group, at least one data type corresponding to the normalized data group parameter is determined based on the normalization algorithm and the geometric mean algorithm corresponding to different data types.

[0013] All normalized data group parameters corresponding to all data groups are determined as normalized data.

[0014] As an optional implementation, in the first aspect of the present application, the normalization algorithm and the geometric mean algorithm corresponding to different data types are used to determine at least one data type corresponding to the normalized data group parameter of the data group, including:

[0015] The physiological data of different data types in the data group are grouped to obtain at least one data type corresponding to the type data group of the data group;

[0016] For each type data group, the geometric mean of the data values of all physiological data in the type data group is calculated to obtain the data group parameter corresponding to the type data group;

[0017] For the data group parameters corresponding to different data types of the data group, the normalization algorithm corresponding to the data type is used to normalize to obtain at least one data type corresponding to the normalized data group parameter of the data group.

[0018] As an optional implementation, in the first aspect of the present application, the graph network construction algorithm and the graph neural network are used to extract features from the normalized data to obtain graph psychological features, including:

[0019] Each time interval is determined as a graph node, and all normalized data group parameters corresponding to the data group are determined as graph node parameters corresponding to the graph node;

[0020] For any two graph nodes, the adjacency feature parameter between the two graph nodes is calculated based on the normalized data group parameter;

[0021] All graph nodes and adjacency feature parameters are determined as a graph network corresponding to the normalized data.

[0022] input the graph network into a trained graph neural network to obtain a graph psychological feature corresponding to the normalized data; the graph neural network is obtained by training a training data set including a plurality of training graph networks and corresponding psychological feature labels.

[0023] As an optional implementation, in the first aspect of the present application, the calculation of the adjacency feature parameter between the two graph nodes based on the normalized data set parameters comprises:

[0024] determining the intersection between the data types corresponding to all the normalized data set parameters of the two graph nodes to obtain a plurality of same data types;

[0025] calculating the weighted sum average of the difference between the normalized data set parameters corresponding to all the same data types corresponding to the two graph nodes to obtain the adjacency feature parameter corresponding to the two graph nodes; wherein the calculation weight corresponding to the difference corresponding to each same data type is proportional to the data amount of the data set corresponding to the same data type.

[0026] As an optional implementation, in the first aspect of the present application, the LSTM neural network and the graph neural network are trained based on a preset common training data set; the common training data set includes a plurality of training psychological state sequences and corresponding graph psychological feature labels and multi-modal data graph network labels corresponding to each psychological state data.

[0027] As an optional implementation, in the first aspect of the present application, the method further comprises:

[0028] based on a preset data threshold rule, determining whether the future psychological state parameter is a dangerous psychological state to obtain a first determination result;

[0029] when the first determination result is yes, obtaining the future psychological state parameter corresponding to a plurality of time points corresponding to the patient object to obtain a psychological state parameter sequence;

[0030] according to a preset data change rule, determining whether the psychological state parameter sequence satisfies a preset psychological state aggravation condition to obtain a second determination result;

[0031] when the second determination result is no, sending an alarm message to a first level alarm terminal;

[0032] when the second determination result is yes, sending an alarm message to a second level alarm terminal; the second level is higher than the first level in the severity level.

[0033] The second aspect of the embodiment of the present application discloses a psychological state prediction system based on geometric mean and graph structure, the system comprises:

[0034] An acquisition module is configured to acquire multi-modal physiological data of a patient object.

[0035] A processing module is configured to perform normalization processing on the multi-modal physiological data based on a geometric mean algorithm to obtain normalized data.

[0036] An extraction module is configured to perform feature extraction on the normalized data based on a graph network construction algorithm and a graph neural network to obtain graph psychological features.

[0037] A prediction module is configured to input the graph psychological features into a pre-trained LSTM neural network to obtain future psychological state parameters corresponding to the patient object, wherein the LSTM neural network is trained by a training data set comprising a plurality of training psychological state sequences and corresponding graph psychological feature labels.

[0038] As an optional implementation, in the second aspect of the present application, the multi-modal physiological data comprises at least one of brain wave EEG data, heart rate data, respiratory rate data, text emotion score data, voice data and expression feature data.

[0039] As an optional implementation, in the second aspect of the present application, the processing module performs normalization processing on the multi-modal physiological data based on a geometric mean algorithm to obtain normalized data, and the specific manner comprises:

[0040] Grouping a plurality of physiological data with the same time interval in the multi-modal physiological data into a data group to obtain a plurality of data groups.

[0041] For each data group, determining a normalized data group parameter corresponding to at least one data type corresponding to the data group based on a normalization processing algorithm and a geometric mean algorithm corresponding to the different data types.

[0042] Determining all the normalized data group parameters corresponding to all the data groups as the normalized data.

[0043] As an optional implementation, in the second aspect of the present application, the processing module determines a normalized data group parameter corresponding to at least one data type corresponding to the data group based on a normalization processing algorithm and a geometric mean algorithm corresponding to the different data types, and the specific manner comprises:

[0044] Grouping physiological data of different data types in the data group to obtain a type data group corresponding to at least one data type corresponding to the data group.

[0045] For each of the types of data groups, a geometric mean of data values of all physiological data in the type of data group is calculated to obtain a data group parameter corresponding to the type of data group;

[0046] For the data group parameters corresponding to different data types corresponding to the data group, normalization processing is performed based on a normalization processing algorithm corresponding to the data type to obtain a normalized data group parameter corresponding to at least one data type corresponding to the data group.

[0047] As an optional implementation, in the second aspect of the present application, the extraction module extracts features from the normalized data based on a graph network construction algorithm and a graph neural network to obtain a specific manner of the graph psychological features, which includes:

[0048] Each of the acquisition time intervals is determined as a graph node, and all of the normalized data group parameters corresponding to the data group corresponding to the graph node are determined as graph node parameters corresponding to the graph node;

[0049] For any two of the graph nodes, an adjacency feature parameter between the two graph nodes is calculated based on the normalized data group parameters;

[0050] All of the graph nodes and the adjacency feature parameters are determined as a graph network corresponding to the normalized data;

[0051] The graph network is input into a trained graph neural network to obtain a graph psychological feature corresponding to the normalized data; the graph neural network is obtained by training a training data set including a plurality of training graph networks and corresponding psychological feature labels.

[0052] As an optional implementation, in the second aspect of the present application, the extraction module calculates the adjacency feature parameter between the two graph nodes based on the normalized data group parameters in a specific manner, which includes:

[0053] An intersection between data types corresponding to all of the normalized data group parameters corresponding to the two graph nodes is determined to obtain a plurality of same data types;

[0054] A weighted sum average value of differences between the normalized data group parameters corresponding to all of the same data types corresponding to the two graph nodes is calculated to obtain an adjacency feature parameter corresponding to the two graph nodes; wherein a calculation weight corresponding to each of the differences corresponding to the same data type is proportional to a data amount of the data group corresponding to the same data type.

[0055] As an optional implementation, in the second aspect of the present application, the LSTM neural network and the graph neural network are trained based on a preset common training data set; the common training data set includes a plurality of training psychological state sequences and corresponding graph psychological feature annotations and multimodal data graph network annotations corresponding to each psychological state data.

[0056] As an optional implementation, in the second aspect of the present application, the system is further used to perform the following steps:

[0057] Based on a preset data threshold rule, it is judged whether the future psychological state parameter is a dangerous psychological state, and a first judgment result is obtained;

[0058] When the first judgment result is yes, the future psychological state parameters corresponding to a plurality of time points corresponding to the patient object are obtained, and a psychological state parameter sequence is obtained;

[0059] According to a preset data change rule, it is judged whether the psychological state parameter sequence meets a preset psychological state aggravation condition, and a second judgment result is obtained;

[0060] When the second judgment result is no, an alarm message is sent to a first-level alarm terminal;

[0061] When the second judgment result is yes, an alarm message is sent to a second-level alarm terminal; the second level is higher than the first level in the severity level.

[0062] The third aspect of the present application discloses another psychological state prediction system based on geometric mean and graph structure, the system comprises:

[0063] A memory storing executable program codes;

[0064] A processor coupled with the memory;

[0065] The processor invokes the executable program codes stored in the memory to execute part or all of the steps of the psychological state prediction method based on geometric mean and graph structure disclosed in the first aspect of the present application.

[0066] The fourth aspect of the present application discloses a computer storage medium storing computer instructions, when the computer instructions are invoked, part or all of the steps of the psychological state prediction method based on geometric mean and graph structure disclosed in the first aspect of the present application are executed.

[0067] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0068] The present application firstly acquires multi-modal physiological data of a patient object, and normalizes the multi-modal physiological data through a geometric mean algorithm to construct a unified feature scale, then models the normalized data into a graph structure based on a graph network construction algorithm, extracts graph psychological features reflecting psychological correlations in the graph structure by means of a graph neural network, further inputs the graph psychological features into a pre-trained LSTM neural network to complete dynamic prediction of a future psychological state, so that cross-modal mapping of physiological data to psychological trends can be realized, and continuity, timeliness and accuracy of psychological state prediction can be improved based on multi-modal physiological data. BRIEF DESCRIPTION OF DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0070] Figure 1 is a flow diagram of a psychological state prediction method based on geometric mean and graph structure disclosed by the embodiments of the present application.

[0071] Figure 2 is a structural diagram of a psychological state prediction system based on geometric mean and graph structure disclosed by the embodiments of the present application.

[0072] Figure 3 is another structural diagram of a psychological state prediction system based on geometric mean and graph structure disclosed by the embodiments of the present application. DETAILED DESCRIPTION

[0073] In order to make the technical personnel in the art better understand the present application scheme, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0074] The terms "first", "second", etc. in the specification and claims of the present application and the above drawings are used to distinguish different objects, not to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, device, product or equipment including a series of steps or units is not limited to the listed steps or units, but can optionally include steps or units not listed, or can optionally include other steps or units inherent to the process, method, product or equipment.

[0075] Reference to“an embodiment” or“the embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase“in one embodiment” or“in at least one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are they necessarily all referring to a particular embodiment that is separate from other embodiments. It is explicitly contemplated that embodiments described herein can be combined with each other in their individual aspects.

[0076] The application discloses a psychological state prediction method and system based on geometric mean and graph structure. First, the multi-modal physiological data of a patient object is obtained, and normalized by a geometric mean algorithm to construct a unified feature scale. Then, the normalized data is modeled into a graph structure based on a graph network construction algorithm, and graph psychological features reflecting psychological associations are extracted by a graph neural network. The graph psychological features are further input into a pre-trained LSTM neural network to complete dynamic prediction of future psychological states, so that cross-modal mapping of physiological data to psychological trends is realized, and the continuity, timeliness and accuracy of psychological state prediction are improved based on multi-modal physiological data. The following will be described in detail.

[0077] Embodiment one

[0078] Please refer to Figure 1 , Figure 1 is a flowchart of a psychological state prediction method based on geometric mean and graph structure disclosed by the embodiment of the application. Among them, Figure 1 The psychological state prediction method based on geometric mean and graph structure described above can be applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As Figure 1 shown, the psychological state prediction method based on geometric mean and graph structure can include the following operations:

[0079] 101. Obtain multi-modal physiological data of a patient object.

[0080] 102. Normalize the multi-modal physiological data based on a geometric mean algorithm to obtain normalized data.

[0081] 103. Extract features from the normalized data based on a graph network construction algorithm and a graph neural network to obtain graph psychological features.

[0082] 104. Input the graph psychological features into a pre-trained LSTM neural network to obtain future psychological state parameters corresponding to the patient object.

[0083] Optionally, the LSTM neural network is trained by a training data set including a plurality of training psychological state sequences and corresponding graph psychological feature labels.

[0084] It can be seen that the above embodiment of the application first acquires multi-modal physiological data of a patient object, and performs normalization processing through a geometric mean algorithm to construct a unified feature scale, then models the normalized data into a graph structure based on a graph network construction algorithm, and extracts graph psychological features reflecting psychological correlations by means of a graph neural network, further inputs the graph psychological features into a pre-trained LSTM neural network to complete dynamic prediction of future psychological states, so as to realize cross-modal mapping of physiological data to psychological trends, and improve continuity, timeliness and accuracy of psychological state prediction based on multi-modal physiological data.

[0085] As an optional embodiment, in the above step, the multi-modal physiological data includes at least one of brain wave EEG data, heart rate data, respiratory rate data, text emotion score data, voice data and expression feature data.

[0086] It can be seen that through the above optional embodiment, the content of the multi-modal physiological data is limited to comprehensively represent multi-modal physiological features of the patient object, to assist in realizing cross-modal mapping of physiological data to psychological trends, and to improve continuity, timeliness and accuracy of psychological state prediction based on multi-modal physiological data.

[0087] As an optional embodiment, in the above step, the normalization processing of the multi-modal physiological data based on the geometric mean algorithm obtains normalized data, including:

[0088] Grouping a plurality of physiological data with the same acquisition time interval in the multi-modal physiological data into a data group to obtain a plurality of data groups;

[0089] For each data group, determining a normalized data group parameter corresponding to at least one data type corresponding to the data group based on a normalization processing algorithm and a geometric mean algorithm corresponding to different data types;

[0090] Determining all normalized data group parameters corresponding to all data groups as normalized data.

[0091] It can be seen that through the above optional embodiment, the data with the same acquisition time interval in the multi-modal physiological data is divided into a plurality of data groups, and different types of physiological data in each data group are processed by using corresponding normalization processing algorithms combined with the geometric mean algorithm to obtain normalized data group parameters, and all normalized parameters corresponding to all data groups are integrated into normalized data, so as to realize structured and normalized expression of physiological data at different time points and different modalities, and effectively improve the accuracy and stability of subsequent psychological feature extraction and psychological state prediction.

[0092] As an optional embodiment, in the above step, the determining of the normalized data set parameter corresponding to the at least one data type corresponding to the data set based on the normalization processing algorithm corresponding to the different data types and the geometric mean algorithm comprises:

[0093] grouping the physiological data of different data types in the data set to obtain a type data set corresponding to the at least one data type corresponding to the data set;

[0094] For each type data set, calculating the geometric mean of the data values of all physiological data in the type data set to obtain a data set parameter corresponding to the type data set;

[0095] For the data set parameters corresponding to different data types corresponding to the data set, performing normalization processing based on the normalization processing algorithm corresponding to the data type to obtain the normalized data set parameter corresponding to the at least one data type corresponding to the data set.

[0096] It can be seen that through the above optional embodiment, the different types of physiological data in each data set are respectively divided into corresponding type data sets, and the geometric mean of all physiological data in each type data set is calculated to obtain a type data set parameter. Then, according to the normalization processing algorithm corresponding to each data type, the type data set parameter is normalized to obtain multiple normalized data set parameters corresponding to each data set, realizing the standardized expression of physiological data of different data types in a unified time period, and improving the consistency of data features and the effectiveness of subsequent analysis.

[0097] As an optional embodiment, in the above step, the feature extraction of the normalized data based on the graph network construction algorithm and the graph neural network to obtain the graph psychological feature comprises:

[0098] Each acquisition time interval is determined as a graph node, and all normalized data set parameters corresponding to the corresponding data set are determined as graph node parameters corresponding to the graph node;

[0099] For any two graph nodes, the adjacent feature parameter between the two graph nodes is calculated based on the normalized data set parameter;

[0100] All graph nodes and adjacent feature parameters are determined as a graph network corresponding to the normalized data;

[0101] The graph network is input into the trained graph neural network to obtain a graph psychological feature corresponding to the normalized data; the graph neural network is obtained by training a training data set comprising a plurality of training graph networks and corresponding psychological feature labels.

[0102] It can be seen that, by the above optional embodiments, each acquisition time interval is mapped to a graph node, and all normalized data set parameters in the data set corresponding to the time interval are set as graph node parameters, the adjacency feature parameters between any two graph nodes are calculated based on the normalized data set parameters, a graph network containing all graph nodes and their adjacency relationships is constructed, and the graph network is input into the trained graph neural network to extract graph psychological features, thereby realizing the structured expression of multi-modal physiological data in different time intervals and the deep learning extraction of psychological state features through unified modeling and feature extraction, and significantly improving the accuracy of psychological state prediction and individualized analysis capability.

[0103] As an optional embodiment, in the above step, the adjacency feature parameters between the two graph nodes are calculated based on the normalized data set parameters, including:

[0104] determining the intersection between the data types corresponding to all normalized data set parameters of the two graph nodes, obtaining a plurality of same data types;

[0105] calculating the weighted sum average of the difference values between the normalized data set parameters corresponding to all same data types corresponding to the two graph nodes, obtaining the adjacency feature parameters corresponding to the two graph nodes; wherein the calculation weight corresponding to each same data type corresponding difference value is proportional to the data amount of the data set corresponding to the same data type.

[0106] It can be seen that, for any two graph nodes, the intersection of the data types in the normalized data set parameters corresponding to the two graph nodes is determined, a plurality of same data types are obtained, and the adjacency feature parameters between the two graph nodes are obtained based on the normalized parameter difference corresponding to each data type in the intersection, and the data amount corresponding to the data type is weighted and summed to obtain the adjacency feature parameters between the two graph nodes, by establishing a difference measurement basis on the same data type and considering the weight distribution factor of the data amount, thereby enhancing the semantic expression ability of the edge weight in the graph network and improving the feature extraction accuracy of the graph neural network for multi-modal time series data.

[0107] As an optional embodiment, in the above step, the LSTM neural network and the graph neural network are obtained by common training based on a preset common training data set; the common training data set includes a plurality of training psychological state sequences and corresponding graph psychological feature labels and multi-modal data graph network labels corresponding to each psychological state data.

[0108] It can be seen that through the above optional embodiments, by jointly training two neural networks, the efficient representation of the graph neural network to the multi-modal graph structure and the accurate modeling of the LSTM to the psychological state time evolution are realized, the prediction accuracy of the model to the future psychological state of the patient and the construction ability of the correlation between the multi-modal features are improved, the cross-modal mapping from physiological data to psychological trend is assisted to be realized, and the continuity, timeliness and accuracy of the psychological state prediction based on multi-modal physiological data are improved.

[0109] As an optional embodiment, in the above steps, the method further comprises:

[0110] Based on the preset data threshold rule, it is judged whether the future psychological state parameter is a dangerous psychological state, and a first judgment result is obtained;

[0111] When the first judgment result is yes, the future psychological state parameters corresponding to the multiple time points corresponding to the patient object are obtained, and a psychological state parameter sequence is obtained;

[0112] According to the preset data change rule, it is judged whether the psychological state parameter sequence meets the preset psychological state aggravation condition, and a second judgment result is obtained;

[0113] When the second judgment result is no, an alarm message is sent to the first level alarm terminal;

[0114] When the second judgment result is yes, an alarm message is sent to the second level alarm terminal; the second level is higher than the first level in the severity level.

[0115] It can be seen that through the above optional embodiments, it is first judged whether the future psychological state parameter is in a dangerous state based on the preset data threshold rule, after recognizing the dangerous psychological state, the parameter sequence is further constructed by combining the psychological state parameters of multiple time points, and it is judged whether there is a psychological state aggravation trend according to the preset data change rule, and then the hierarchical warning is realized according to the judgment result, the first level alarm is triggered when there is no obvious aggravation trend, and the second level alarm of a higher level is triggered if there is a continuous aggravation sign, so as to realize the accurate identification and dynamic hierarchical response to the potential psychological crisis, and enhance the practical value and timeliness of the prediction result in the clinical warning.

[0116] Embodiment two

[0117] Please refer to Figure 2 , Figure 2 is a structure diagram of a psychological state prediction system based on geometric mean and graph structure disclosed by the embodiment of the application. Wherein, Figure 2The described geometric mean and graph structure-based mental state prediction system can be applied in a data processing system / data processing device / data processing server (wherein the server includes a local processing server or a cloud processing server). As shown in Figure 2 The geometric mean and graph structure-based mental state prediction system can include:

[0118] The acquisition module 201 is configured to acquire multi-modal physiological data of a patient object.

[0119] The processing module 202 is configured to perform normalization processing on the multi-modal physiological data based on a geometric mean algorithm to obtain normalized data.

[0120] The extraction module 203 is configured to perform feature extraction on the normalized data based on a graph network construction algorithm and a graph neural network to obtain graph mental features.

[0121] The prediction module 204 is configured to input the graph mental features into a pre-trained LSTM neural network to obtain future mental state parameters corresponding to the patient object.

[0122] Optionally, the LSTM neural network is trained by a training data set including a plurality of training mental state sequences and corresponding graph mental feature labels.

[0123] As can be seen, the above embodiment first acquires multi-modal physiological data of a patient object, and performs normalization processing by a geometric mean algorithm to construct a unified feature scale, then models the normalized data as a graph structure based on a graph network construction algorithm, and extracts graph mental features reflecting psychological associations by means of a graph neural network, further inputs the graph mental features into a pre-trained LSTM neural network to complete dynamic prediction of future mental states, so as to realize cross-modal mapping of physiological data to psychological trends, and improve the continuity, timeliness and accuracy of mental state prediction based on multi-modal physiological data.

[0124] As an optional embodiment, the multi-modal physiological data includes at least one of brain wave EEG data, heart rate data, respiratory rate data, text emotion score data, voice data and expression feature data.

[0125] As can be seen, by the above optional embodiment, the content of the multi-modal physiological data is limited to comprehensively represent the multi-modal physiological features of the patient object, to assist in realizing cross-modal mapping of physiological data to psychological trends, and to improve the continuity, timeliness and accuracy of mental state prediction based on multi-modal physiological data.

[0126] As an optional embodiment, the processing module performs normalization processing on the multi-modal physiological data based on a geometric mean algorithm, and the specific manner of obtaining the normalized data includes:

[0127] grouping the plurality of physiological data with the same acquisition time interval in the multi-modal physiological data into one data group to obtain a plurality of data groups;

[0128] For each data group, based on the normalization processing algorithm and the geometric mean algorithm corresponding to different data types, determining the normalized data group parameter corresponding to at least one data type corresponding to the data group;

[0129] All normalized data group parameters corresponding to all data groups are determined as normalized data.

[0130] It can be seen that through the above optional embodiments, the data with the same acquisition time interval in the multi-modal physiological data is divided into a plurality of data groups, and the corresponding normalization processing algorithm is used for processing the physiological data of different types in each data group in combination with the geometric mean algorithm to obtain the normalized data group parameter, and then all the normalized parameters corresponding to all the data groups are integrated into the normalized data, thereby realizing the structured and normalized expression of the physiological data of different time points and different modalities, and effectively improving the accuracy and stability of subsequent psychological feature extraction and psychological state prediction.

[0131] As an optional embodiment, the specific manner in which the processing module determines the normalized data group parameter corresponding to at least one data type corresponding to the data group based on the normalization processing algorithm and the geometric mean algorithm corresponding to different data types includes:

[0132] Grouping the physiological data of different data types in the data group to obtain at least one data type corresponding to the data group corresponding to the type data group;

[0133] For each type data group, calculating the geometric mean of the data values of all physiological data in the type data group to obtain the data group parameter corresponding to the type data group;

[0134] For the data group parameters corresponding to different data types corresponding to the data group, based on the normalization processing algorithm corresponding to the data type, the normalization processing is performed to obtain the normalized data group parameter corresponding to at least one data type corresponding to the data group.

[0135] It can be seen that through the above optional embodiments, the different types of physiological data in each data group are respectively divided into corresponding type data groups, and the geometric mean of all physiological data in each type data group is calculated to obtain the type data group parameter, and then the type data group parameter is normalized according to the normalization processing algorithm corresponding to each data type to obtain a plurality of normalized data group parameters corresponding to each data group, thereby realizing the standardized expression of the physiological data of different data types in a unified time period and improving the data feature consistency and the effectiveness of subsequent analysis.

[0136] As an optional embodiment, the extraction module extracts features from the normalized data based on a graph network construction algorithm and a graph neural network to obtain the specific manner of obtaining the graph psychological features, including:

[0137] Each acquisition time interval is determined as a graph node, and all normalized data set parameters corresponding to the corresponding data set are determined as graph node parameters corresponding to the graph node;

[0138] For any two graph nodes, the adjacent feature parameters between the two graph nodes are calculated based on the normalized data set parameters;

[0139] All graph nodes and adjacent feature parameters are determined as a graph network corresponding to the normalized data;

[0140] The graph network is input into the trained graph neural network to obtain the graph psychological features corresponding to the normalized data; the graph neural network is trained by a training data set including a plurality of training graph networks and corresponding psychological feature labels.

[0141] As can be seen, by the above optional embodiment, each acquisition time interval is mapped to a graph node, and all normalized data set parameters in the data set corresponding to the time interval are set as graph node parameters, the adjacent feature parameters between any two graph nodes are calculated based on the normalized data set parameters, a graph network containing all graph nodes and their adjacent relationships is constructed, and the graph network is input into the trained graph neural network to extract the graph psychological features, thereby realizing the structured expression of multi-modal physiological data in different time intervals and the deep learning extraction of psychological state features through unified modeling and feature extraction, and significantly improving the accuracy of psychological state prediction and individualized analysis ability.

[0142] As an optional embodiment, the extraction module calculates the adjacent feature parameters between the two graph nodes based on the normalized data set parameters, and the specific manner includes:

[0143] Determine the intersection between the data types corresponding to all normalized data set parameters of the two graph nodes to obtain a plurality of same data types;

[0144] Calculate the weighted sum average of the difference between the normalized data set parameters corresponding to all same data types corresponding to the two graph nodes to obtain the adjacent feature parameters corresponding to the two graph nodes; wherein the calculation weight corresponding to each same data type corresponding to the difference is proportional to the data amount of the data set corresponding to the same data type.

[0145] It can be seen that, through the above optional embodiments, for any two graph nodes, the intersection of the data types in the corresponding normalized data set parameters of the two is determined, a plurality of same data types are obtained, and based on the normalized parameter difference corresponding to each data type in the intersection, the data amount corresponding to the data type is weighted and summed and averaged to obtain the adjacency feature parameter between the two graph nodes. By establishing a difference measurement basis on the same data type and considering the data amount factor to allocate weights, the semantic expression ability of the edge weight in the graph network is enhanced, and the feature extraction accuracy of the graph neural network for multi-modal time series data is improved.

[0146] As an optional embodiment, the LSTM neural network and the graph neural network are trained based on a preset common training data set; the common training data set includes a plurality of training psychological state sequences and corresponding graph psychological feature labels and multi-modal data graph network labels corresponding to each psychological state data.

[0147] It can be seen that, through the above optional embodiments, by jointly training the two neural networks, efficient representation of the graph neural network for multi-modal graph structure and accurate modeling of the psychological state time series evolution by the LSTM are realized, the prediction accuracy of the model for the future psychological state of the patient and the construction ability of the correlation between multi-modal features are improved, the cross-modal mapping from physiological data to psychological trend is assisted to be realized, and the continuity, timeliness and accuracy of the psychological state prediction based on multi-modal physiological data are improved.

[0148] As an optional embodiment, the system is further used to perform the following steps:

[0149] Based on a preset data threshold rule, it is determined whether the future psychological state parameter is a dangerous psychological state, and a first determination result is obtained;

[0150] When the first determination result is yes, a plurality of time point corresponding future psychological state parameters corresponding to the patient object are obtained, and a psychological state parameter sequence is obtained;

[0151] According to a preset data change rule, it is determined whether the psychological state parameter sequence satisfies a preset psychological state aggravation condition, and a second determination result is obtained;

[0152] When the second determination result is no, an alarm message is sent to a first level alarm terminal;

[0153] When the second determination result is yes, an alarm message is sent to a second level alarm terminal; the second level is higher than the first level in the severity level.

[0154] It can be seen that, through the above optional embodiments, whether the future psychological state parameter is in a dangerous state is first judged based on the preset data threshold rule, after the dangerous psychological state is identified, the parameter sequence is further constructed in combination with the psychological state parameters at multiple time points, and whether there is a trend of psychological state aggravation and deterioration is judged according to the preset data change rule, and then the graded early warning is realized according to the judgment result, the first level alarm is triggered when there is no obvious deterioration trend, and the second level alarm of a higher level is triggered if there is a continuous aggravation sign, so as to realize accurate identification and dynamic grading response of potential psychological crisis, and enhance the practical value and timeliness of the prediction result in clinical early warning.

[0155] Embodiment three

[0156] Please refer to Figure 3 , Figure 3 The application also discloses a psychological state prediction system based on a geometric mean and a graph structure. Figure 3 The psychological state prediction system based on the geometric mean and the graph structure is applied to a data processing system / data processing device / data processing server (wherein the server comprises a local processing server or a cloud processing server). As shown in Figure 3 The psychological state prediction system based on the geometric mean and the graph structure can comprise:

[0157] a memory 301 storing executable program codes;

[0158] a processor 302 coupled with the memory 301;

[0159] The processor 302 invokes the executable program codes stored in the memory 301, and is configured to execute the steps of the psychological state prediction method based on the geometric mean and the graph structure described in the embodiment one.

[0160] Embodiment four

[0161] The application discloses a computer readable storage medium storing a computer program for electronic data exchange, wherein the computer program causes a computer to execute the steps of the psychological state prediction method based on the geometric mean and the graph structure described in the embodiment one.

[0162] Embodiment five

[0163] The application discloses a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute the steps of the psychological state prediction method based on the geometric mean and the graph structure described in the embodiment one.

[0164] The above-described embodiments of the present specification are described with reference to particular embodiments. Other embodiments are within the scope of the following claims. In some cases, actions recited in the claims can be performed in a different order and still achieve desirable results. Additionally, the processes depicted in the accompanying figures do not necessarily require the particular order shown or sequential order to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0165] The systems, apparatuses, modules, or units illustrated by the above-described embodiments can be specifically realized by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0166] For the convenience of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of the units can be implemented in the same or multiple software and / or hardware in the implementation of the present specification.

[0167] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, the embodiments of the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0168] The present specification is described with reference to flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present specification. It should be understood that each flow and / or block in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus produce a result for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 The functions specified in the flow or multiple flows and / or blocks Figure 1 The apparatuses specified in the flow or multiple flows and / or blocks

[0169] These computer program instructions can also be stored in a computer- readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0170] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0171] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0172] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, non-volatile memory, such as read-only memory (ROM), EPROM, and / or flash memory, etc. The memory is an example of computer readable media.

[0173] Computer readable media includes permanent and non-permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile discs (DVDs) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0174] It should also be noted that the terms "comprises", "comprising", or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without further constraints, exclude the presence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0175] The specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0176] The various embodiments in the specification are described in progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the difference from other embodiments. In particular, for the system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the part of the method embodiments.

[0177] Finally, it should be noted that the psychological state prediction method and system based on geometric mean and graph structure disclosed by the embodiments of the present application are only the preferred embodiments of the present application, and are only used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalent ones. The modification or replacement does not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for predicting psychological states based on geometric mean and graph structure, characterized in that: The method comprises: Acquiring multimodal physiological data of a patient subject; Normalizing the multimodal physiological data based on a geometric mean algorithm to obtain normalized data includes: Grouping multiple physiological data with the same acquisition time interval in the multimodal physiological data into one data group to obtain multiple data groups; For each of the data groups, determining normalized data group parameters corresponding to at least one data type corresponding to the data group based on normalization processing algorithms and geometric mean algorithms corresponding to different data types; Determining all the normalized data group parameters corresponding to all the data groups as normalized data; Based on the graph network construction algorithm and graph neural network, feature extraction is performed on the normalized data to obtain graph psychological features, including: Determine each of the acquisition time intervals as a graph node, and determine all of the normalized data group parameters corresponding to the corresponding data group as graph node parameters corresponding to the graph node; For any two of the graph nodes, determining the intersection of the data types corresponding to all the normalized data group parameters corresponding to the two graph nodes to obtain a plurality of identical data types; Calculating the weighted sum average of the differences between the normalized data group parameters corresponding to all the same data types corresponding to the two graph nodes to obtain the adjacency feature parameters corresponding to the two graph nodes; wherein the calculation weight corresponding to each difference corresponding to the same data type is proportional to the data volume of the data group corresponding to the same data type; Determine all the graph nodes and the adjacency feature parameters as a graph network corresponding to the normalized data; Inputting the graph network into a trained graph neural network to obtain graph psychological features corresponding to the normalized data; the graph neural network is trained by a training dataset including a plurality of training graph networks and corresponding psychological feature annotations; The graph psychological features are input into a pre-trained LSTM neural network to obtain the future psychological state parameters corresponding to the patient object; the LSTM neural network is trained by a training data set including multiple training psychological state sequences and corresponding graph psychological feature annotations.

2. The method for predicting psychological state based on geometric mean and graph structure according to claim 1, characterized in that: The multimodal physiological data includes at least one of EEG data, heart rate data, respiratory rate data, text emotion score data, voice data, and expression feature data.

3. The method for predicting psychological state based on geometric mean and graph structure according to claim 1, characterized in that: The determining of normalized data group parameters corresponding to at least one data type corresponding to the data group based on the normalization processing algorithms and geometric mean algorithms corresponding to different data types includes: Grouping physiological data of different data types in the data group to obtain a type data group corresponding to at least one data type corresponding to the data group; For each data group of the type, calculating the geometric mean of the data values ​​of all physiological data in the data group of the type, and obtaining the data group parameter corresponding to the data group of the type; The data group parameters corresponding to different data types corresponding to the data group are normalized based on a normalization processing algorithm corresponding to the data type to obtain normalized data group parameters corresponding to at least one data type corresponding to the data group.

4. The method for predicting psychological state based on geometric mean and graph structure according to claim 1, characterized in that: The LSTM neural network and the graph neural network are jointly trained based on a preset common training data set; the common training data set includes multiple training psychological state sequences and corresponding graph psychological feature annotations and multimodal data graph network annotations corresponding to each psychological state data.

5. The method for predicting psychological state based on geometric mean and graph structure according to claim 1, characterized in that: The method further comprises: Based on a preset data threshold rule, determining whether the future mental state parameter is a dangerous mental state, and obtaining a first determination result; When the first judgment result is yes, obtaining the future mental state parameters corresponding to multiple time points corresponding to the patient object to obtain a mental state parameter sequence; determining, according to a preset data change rule, whether the mental state parameter sequence satisfies a preset mental state aggravation condition, and obtaining a second judgment result; When the second judgment result is no, sending an alarm message to the first-level alarm terminal; When the second judgment result is yes, an alarm message is sent to a second-level alarm terminal; the second level is more serious than the first level.

6. A psychological state prediction system based on geometric mean and graph structure, characterized in that: The system comprises: an acquisition module, for acquiring multimodal physiological data of a patient subject; A processing module, configured to perform normalization processing on the multimodal physiological data based on a geometric mean algorithm to obtain normalized data, comprising: Grouping multiple physiological data with the same acquisition time interval in the multimodal physiological data into one data group to obtain multiple data groups; For each of the data groups, determining normalized data group parameters corresponding to at least one data type corresponding to the data group based on normalization processing algorithms and geometric mean algorithms corresponding to different data types; Determining all the normalized data group parameters corresponding to all the data groups as normalized data; The extraction module is used to construct an algorithm and a graph neural network based on the graph network, extract features from the normalized data, and obtain graph psychological features, including: Determine each of the acquisition time intervals as a graph node, and determine all of the normalized data group parameters corresponding to the corresponding data group as graph node parameters corresponding to the graph node; For any two of the graph nodes, determining the intersection of the data types corresponding to all the normalized data group parameters corresponding to the two graph nodes to obtain a plurality of identical data types; Calculating the weighted sum average of the differences between the normalized data group parameters corresponding to all the same data types corresponding to the two graph nodes to obtain the adjacency feature parameters corresponding to the two graph nodes; wherein the calculation weight corresponding to each difference corresponding to the same data type is proportional to the data volume of the data group corresponding to the same data type; Determine all the graph nodes and the adjacency feature parameters as a graph network corresponding to the normalized data; Inputting the graph network into a trained graph neural network to obtain graph psychological features corresponding to the normalized data; the graph neural network is trained by a training dataset including a plurality of training graph networks and corresponding psychological feature annotations; A prediction module is used to input the graph psychological features into a pre-trained LSTM neural network to obtain future psychological state parameters corresponding to the patient subject; the LSTM neural network is trained by a training data set including multiple training psychological state sequences and corresponding graph psychological feature annotations.

7. A psychological state prediction system based on geometric mean and graph structure, characterized in that: The system comprises: a memory storing executable program code; a processor coupled to the memory; The processor calls the executable program code stored in the memory to execute the psychological state prediction method based on geometric mean and graph structure as described in any one of claims 1 to 5.

Images