Method and device for psychological assessment, medium and program product

Abstract

The invention provides a psychological assessment method and device, a computer readable medium and a computer program product. The method comprises the steps of obtaining multi-modal data of a to-be-evaluated object, wherein the multi-modal data comprises physiological and behavior data from a plurality of data sources; performing feature extraction processing on the obtained multi-modal data to obtain corresponding feature information; fusing the feature information corresponding to the multi-modal data to obtain corresponding fused feature information; and a trained target model is used to output a corresponding psychological assessment result based on the fused feature information, and the target model is automatically iteratively updated based on user data and a model verification result. According to the method, psychological assessment is carried out by obtaining the target data of the multiple modes, mutual verification and complementation can be carried out among the data of the modes, and therefore the accuracy and reliability of the psychological assessment result are remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, the field of mental health or medical health management, and particularly to a method, device, computer-readable medium, and computer program product for conducting psychological assessments. Background Art

[0002] Psychological assessment is a process of testing and evaluating a person's mental state. Objective and accurate psychological assessment is of great significance for the prevention, diagnosis, and treatment of psychological problems and mental illnesses. Traditional psychological assessment methods mostly rely on questionnaire self-report or interview assessment, and their results highly depend on the subjective feedback of the test takers. Although such methods are easy to operate, the data source is single, and they are easily interfered by subjective factors such as individual cognitive biases and social desirability effects, making it difficult to comprehensively reflect the true mental state.

[0003] As a complex system of biological-psychological-social multi-factor interaction, a single data source is difficult to comprehensively characterize its dynamic characteristics. Existing technologies have not effectively integrated multi-source information such as subjective reports, physiological signals, and behavioral data, and the complementarity and synergy effects between data have not been fully explored, which has become the core obstacle restricting the improvement of assessment accuracy. Summary of the Invention

[0004] Multiple aspects of the present application provide a method, device, computer-readable medium, and computer program product for conducting psychological assessments.

[0005] In one aspect of the present application, a method for conducting psychological assessment is provided, wherein the method includes: Obtaining multi-modal data of the object to be evaluated, where the multi-modal data includes physiological and behavioral data from multiple data sources; Performing feature extraction processing on the obtained multi-modal data to obtain corresponding feature information; Performing fusion processing on the feature information corresponding to the multi-modal data to obtain corresponding fused feature information; Using the trained target model to output corresponding psychological assessment results based on the fused feature information, wherein the target model is automatically iteratively updated based on user data and model verification results.

[0006] In one aspect of the present application, a device for conducting psychological assessment is provided, wherein the device includes: A device for obtaining multi-modal data of the object to be evaluated, where the multi-modal data includes physiological and behavioral data from multiple data sources; A device for performing feature extraction processing on the obtained multi-modal data to obtain corresponding feature information; A device for performing fusion processing on the feature information corresponding to the multi-modal data to obtain corresponding fused feature information; A device for outputting corresponding psychological assessment results based on the fused feature information by using a trained target model, wherein the target model is automatically iteratively updated based on user data and model verification results.

[0007] On the other hand, the present application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to execute the method of the embodiments of the present application.

[0008] On the other hand, the present application provides a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the method of the embodiments of the present application.

[0009] On the other hand, the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method of the embodiments of the present application.

[0010] In the solution provided by the embodiments of the present application, by obtaining target data of multiple modalities and using a trained target model to output corresponding psychological assessment results based on the fused features of the multiple-modal data, this multi-modal data fusion method can comprehensively depict the complexity of the psychological state, reduce the limitations of a single data source, and the various modal data can be mutually verified and supplemented, thereby significantly improving the accuracy and reliability of the psychological assessment results; the model of the embodiments of the present application is an intelligent classification and evaluation model of psychological state based on machine learning. After model verification, a self-learning algorithm is used to automatically optimize the training of the intelligent model, so that the intelligent classification and evaluation model of psychological state is automatically iteratively updated according to user data and model verification results. This dynamic optimization mechanism enables the model to adapt to the differences and dynamic changes of individual psychological states, thereby realizing the personalization and dynamics of the assessment and further improving the prediction accuracy of the model; by associating other types of data with biometric and biochemical data and psychological state data, data features highly correlated with biometric and biochemical characteristics and psychological state characteristics are selected for training in pre-training, significantly improving the pertinence and training efficiency of model training and improving the prediction accuracy of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0012] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments read in conjunction with the accompanying drawings: Figure 1 FIG. 3 shows a schematic flowchart of a method for psychological assessment provided by an embodiment of the present application; Figure 2 FIG. 4 shows a schematic diagram of exemplary data collection and transmission according to an embodiment of the present application; Figure 3 FIG. 5 shows a schematic diagram of an exemplary intelligent psychological assessment system according to an embodiment of the present application; Figure 4 FIG. 6 shows a schematic structural diagram of a device for psychological assessment provided by an embodiment of the present application; Figure 5 FIG. 7 shows a schematic structural diagram of a device suitable for implementing the solution in an embodiment of the present application.

[0013] Like or similar reference numerals in the drawings represent like or similar components. Detailed Embodiments

[0014] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the scope of protection of the present application.

[0015] In a typical configuration of the present application, devices of a terminal and a service network both include one or more processors (CPUs), an input / output interface, a network interface, and a memory.

[0016] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM), and / or non-volatile memory in the form of, for example, read-only memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.

[0017] A computer-readable medium includes permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be computer program instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD), or other optical storage, magnetic cassette tapes, magnetic disk storage, or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible by a computing device.

[0018] Figure 1 The flowchart shows a method for psychological assessment provided by an embodiment of the present application. The method at least includes steps S101, S102, S103, and S104.

[0019] In an actual scenario, the execution subject of this method can be an intelligent hardware device, or it can also be an application program running on the intelligent hardware device. The intelligent hardware device includes a user device or a network device. The user device includes, but is not limited to, various terminal devices such as a computer, a mobile phone, a tablet computer, a smart watch, a smart bracelet, etc. The network device includes, but is not limited to, a network host, a single network server, a set of multiple network servers, or a computer set based on cloud computing. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Among them, cloud computing is a type of distributed computing, which consists of a virtual computer composed of a group of loosely coupled computers.

[0020] Figure 2 The schematic diagram shows an exemplary collection and transmission of data according to an embodiment of the present application. Refer to Figure 2 , the tester interacts with the intelligent hardware device to complete the assessment task, or the tester wears a corresponding physiological assessment device and interacts with the intelligent hardware device to complete the assessment task. Among them, the physiological assessment device includes wearable physiological sensors, smart watches, smart bracelets, physiological polygraphs, electroencephalogram acquisition devices, eye trackers, smart headphones, smart clothes, cameras, video cameras, voice recorders, and tape recorders, etc. Optionally, the solution of the embodiment of the present application collects the biological sample information of the tester for biochemical tests and transmits the obtained data to the intelligent hardware device for calculation.

[0021] Different from traditional psychological assessment methods, the method according to the embodiments of the present application obtains target data of multiple modalities, and uses a trained target model to output corresponding psychological assessment results based on the fusion features of multiple modality data, realizing comprehensive consideration of the influence of multiple factors on psychological assessment results from multiple aspects such as subjective reports, physiological signals, and behavioral data when conducting psychological assessments, making the psychological assessment results obtained by the method according to the embodiments of the present application more accurate and reliable.

[0022] Referring to Figure 1 , in step S101, multi-modal data of the object to be evaluated is obtained, and the multi-modal data includes physiological and behavioral data from multiple data sources.

[0023] Wherein, the object to be evaluated refers to the object that needs to conduct a psychological assessment.

[0024] Wherein, the multi-modal data includes various data that can reflect the psychological state of the object to be evaluated.

[0025] According to one embodiment, the multi-modal data includes physiological-related data, cognitive-behavioral data, bio-chemical data, device usage data, self-assessment report data, and medical treatment data. The meanings and data collection methods of each type of data will be described below respectively.

[0026] 1) Physiological-related data; The physiological data includes basic physiological data and other physiological data. The basic physiological data includes electroencephalogram (EEG), electrocardiogram (ECG), electrodermal activity (EDA), respiratory rhythm, blood oxygen, pulse, eye movement, movement, and sleep-related data. The other physiological data includes facial image data, facial video data, and voice data.

[0027] For the basic physiological data, it is collected through physiological assessment devices or wearable devices. For example, an electrocardiogram signal is collected through a smart watch or a physiological polygraph, an electroencephalogram signal is collected through a smart brain ring and the artifacts are removed by independent component analysis (ICA), and an electrodermal activity signal is collected through a smart watch, a smart bracelet, a smart earphone, etc. and smoothed by the moving average method.

[0028] For the other physiological data, it is automatically collected by the smart device when the object to be evaluated interacts with the smart device and completes the preset tasks in the smart device. For example, the facial expression recorded by the camera is obtained (feature points are extracted by OpenCV Dlib), and the acoustic features (MFCC, fundamental frequency, etc.) are collected by the voice device. Wherein, the smart device may be the smart hardware device according to the embodiments of the present application, or the smart device is other devices independent of the smart hardware device according to the embodiments of the present application. The smart hardware device according to the embodiments of the present application can automatically collect target data, or can actively or passively collect target data by interacting with the evaluation object.

[0029] 2) Cognitive behavior data; The cognitive behavior data includes basic cognitive behavior data, advanced cognitive behavior data, and social cognitive behavior data. Among them, the basic cognitive behavior data includes data on sensation, perception, attention, alertness, consciousness, memory, cognitive control, and executive function cognitive components; the advanced cognitive behavior data includes data on calculation, speech, imagination, logic, reasoning, problem-solving, creativity, and decision-making cognitive components; the social cognitive behavior data includes data on self-awareness, self-boundary, interpersonal interaction, attribution, attitude, and implicit social cognitive components.

[0030] For cognitive behavior data, it is automatically collected by the intelligent device when the object to be evaluated interacts with the intelligent device and completes the corresponding cognitive component test tasks.

[0031] 3) Biological and biochemical data; The biological and biochemical data includes, but is not limited to, the biological and biochemical indicators and data results of the blood, urine, saliva, feces, and spinal fluid samples of the object to be evaluated.

[0032] For biological and biochemical data, by collecting the blood, urine, saliva, feces, and spinal fluid samples of the object to be evaluated and performing basic biochemical tests, genomics, transcriptomics, proteomics, and metabolomics analyses on the samples, the relevant biological and biochemical indicators and data results are obtained as corresponding features. For example, through metabolomics analysis of blood samples, the concentrations of inflammatory factors (IL-6, TNF-α) and the levels of metabolites (such as serotonin) are extracted.

[0033] 4) Device usage data; The intelligent device usage data includes general intelligent device usage data and intelligent application usage data. The general intelligent device usage data includes the usage duration, time period, and frequency of the intelligent device; the intelligent application usage data includes the usage type, usage duration, time period, and frequency of the intelligent application.

[0034] For device usage data, it is automatically collected by the intelligent device by obtaining the permissions of the intelligent device and applications used by the object to be evaluated.

[0035] 5) Self-evaluation report data; The self-evaluation report data includes demographic information data, self-evaluation data on emotional state, self-evaluation data on physical symptoms, and self-evaluation data on social personality. Among them, the demographic information data includes age, gender, education level, income, family size, and economic status; the self-evaluation data on emotional state includes anxiety, depression, fear, panic emotions, and attitudes towards self-harm and self-mutilation; the self-evaluation data on physical symptoms includes physical symptoms, sleep status, and physical illness conditions; the self-evaluation data on social personality includes life events, personality type, interpersonal relationships, self-awareness, coping styles, satisfaction, and dissimulation.

[0036] The self-evaluation report data is collected by the object to be evaluated through a self-evaluation questionnaire preset in the intelligent device, and is automatically collected by the intelligent device.

[0037] 6) Diagnostic data; The diagnostic data includes the test subject's diagnosis information, medication information, treatment information, prognosis information, examination and laboratory information, other-evaluation information of mental state, and clinical follow-up information.

[0038] The diagnostic data is collected by docking with the hospital's diagnostic management system (HIS system) where the test subject seeks medical treatment.

[0039] Optionally, the physiological-related data, cognitive behavior data, and device usage data can be collected synchronously.

[0040] It should be noted that the multi-modal data involved in the embodiments of the present application are all obtained with the consent or permission of the relevant objects, and the collection, use, and processing of the relevant target data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions. Moreover, the collection, processing, and preservation of the biochemistry data in the embodiments of the present application comply with the ethical review standards.

[0041] According to one embodiment, the data of the obtained multi-modal data is preprocessed.

[0042] Among them, the preprocessing includes, but is not limited to, various processes such as cleaning, standardizing, and transforming the obtained multi-modal data. Optionally, the preprocessing includes processes such as word segmentation and stop word removal for text data, cropping, scaling, or denoising for image data, noise reduction or normalization for audio data, and filtering or normalization for physiological signal data.

[0043] In step S102, feature extraction processing is performed on the obtained multi-modal data to obtain corresponding feature information.

[0044] Specifically, in step S102, the type of the multi-modal data to be processed is determined, and a feature extraction operation matching this type is adopted.

[0045] The feature extraction operations for different types of target data are described below.

[0046] 1) Physiological-related data; For the basic physiological data, first perform signal preprocessing, and then extract the corresponding time-domain features, frequency-domain features, and / or non-linear features of the basic physiological data.

[0047] Specifically, through signal preprocessing, the EEG / ECG data were filtered using a bandpass filter (0.5-49 Hz) and independent component analysis (ICA) was used to remove artifacts such as eye movements, body movements, and electromyography. The skin electricity was smoothed using the moving average method (window length 5 s), and the respiratory rhythm was standardized to a sampling frequency of 10 Hz using cubic spline interpolation.

[0048] Among them, the time domain features mainly calculate the mean, variance, peak-to-peak value, and waveform complexity of each signal within a specified time; the frequency domain features calculate the Fourier transform (FFT), energy ratio of different frequency bands, power spectrum density integral of different frequency bands, and centroid frequency of each signal; the nonlinear features calculate the sample entropy (SampEn(m,r,N)= -ln[A / B]), Lyapunov index (calculated based on the maximum exponent), and multifractal spectrum (based on the wavelet transform modulus maximum method).

[0049] For facial image data, features corresponding to the facial image data are obtained by performing facial key point extraction, facial motion analysis, and spatiotemporal feature extraction.

[0050] For example, facial image data is extracted through the OpenCV Dlib library. For dynamic data, the optical flow method is used to calculate the facial motion vector (V(x,y,t)=[I_xI_t, I_yI_t] / [I_x²+I_y²]), and the 3D-CNN network is used to extract spatiotemporal features for micro-expression recognition.

[0051] For speech data, the fundamental frequency feature and emotional feature extraction are used to analyze the emotional and intonation information of the speech signal. Specifically, the multidimensional coefficients are extracted by the Mel filter bank to calculate the MFCC features, the fundamental frequency is calculated by the autocorrelation function (F0=argmax(R(τ))), and the eGeMAPS parameter set is extracted using the opensmile toolkit to calculate the emotional features.

[0052] 2) cognitive behavioral data; For cognitive behavioral data, signal preprocessing was first performed to remove outliers, and then the mean reaction time, coefficient of variation of reaction time, and accuracy were calculated as characteristics reflecting the cognitive behavior of the subject to be evaluated.

[0053] For example, to calculate the reaction time and accuracy of the subject to be evaluated when performing cognitive behavioral responses, first remove the outliers (excluding trial data within ±3SD), then calculate the average reaction time and coefficient of variation of the reaction time, and at the same time calculate the accuracy of the correct response.

[0054] 3) Biological and biochemical data; For bio - biochemical data, through omics data analysis, extract gene variation characteristics, association analysis results and functional characteristics of genomics, protein expression levels, modifications and functional characteristics of proteomics, and metabolite concentrations, differential analysis and pathway characteristics of metabolomics.

[0055] The omics data analysis includes the analysis of genomics, proteomics, and metabolomics.

[0056] Among them, the analysis of genomics includes performing single - nucleotide polymorphism (SNP) association analysis, copy number variation (CNV) analysis, structural variation (SV) analysis, genome - wide association study (GWAS), gene set enrichment analysis (GSEA algorithm), and extracting gene variation characteristics (including SNP loci, allele frequencies, genotype quality scores (GQ), sequencing depths (DP)), association analysis results (including p - values, OR (odds ratio), 95% confidence intervals, Q - values (after multiple - test correction)), and functional characteristics (including gene pathway enrichment scores (such as KEGG pathway p - values), loss - of - function (LOF) variant numbers).

[0057] Among them, the analysis of proteomics includes performing protein quantitative analysis and protein function analysis, and extracting protein expression level characteristics, protein modification characteristics, and protein functional characteristics. Among them, the protein expression level characteristics include protein abundance (Intensity), Fold Change (FC), p - values, the protein modification characteristics include phosphorylation, ubiquitination sites and modification levels, and the protein functional characteristics include pathway activity scores (such as the total protein expression of the Wnt pathway) and interaction network node degrees (Degree).

[0058] Among them, the analysis of metabolomics includes performing metabolite identification analysis, biomarker screening analysis, and metabolic pathway analysis, and extracting metabolite characteristics (including metabolite concentrations (μM), mass - to - charge ratios (m / z), retention times (RT)), differential analysis results (including VIP values, p - values, Fold Change (log2FC)), and pathway characteristics (including pathway enrichment p - values, pathway impact values (based on topological analysis)).

[0059] For conventional detection indicators, extract the inflammation index and metabolic syndrome score.

[0060] 4) Device usage data; For device usage data, calculate the daily usage entropy value, application switching frequency, and temporal dependence characteristics as features for evaluating the user's device usage habits and behavior patterns.

[0061] For example, perform usage pattern analysis to extract the daily usage entropy value (in hourly segments) and the application switching frequency (number of application switches / usage duration); perform behavioral sequence modeling to extract temporal dependence features through an LSTM network.

[0062] 5) Self-evaluation report data; For the self-evaluation report data, calculate the total score and average score of the questionnaire and scale, and calculate the total score and average score of the multi-dimensional data included in the report data as the features corresponding to the report data.

[0063] 6) Diagnosis and treatment data; For the diagnosis and treatment data, convert it into categorical data by extracting diagnosis codes, calculate the comorbidity index to evaluate the comorbidity situation, and analyze the medication information to calculate the drug load index and treatment plan complexity, thereby completing the feature extraction of clinical diagnosis and treatment data.

[0064] For example, by extracting diagnosis codes and converting them into categorical data, calculate the weighted sum of the Charlson index to evaluate the comorbidity situation; by constructing and analyzing treatment features, calculate the drug load index and treatment plan complexity (number of drug types × dosing frequency).

[0065] Continue to refer to Figure 1 For description, in step S103, fuse the feature information corresponding to the multi-modal data to obtain the corresponding fused feature information.

[0066] Among them, the fused feature information can correspond to one or more modalities of data.

[0067] Among them, the fusion processing method is applicable to a variety of psychological assessment scenarios.

[0068] Among them, the fusion processing methods of the method include but are not limited to at least any one of the following: 1) Early Fusion; the early fusion includes but is not limited to feature splicing, feature transformation, and shared feature extractor. Among them, the feature splicing directly splices the features of the same modality into a new feature vector. The feature transformation transforms the features of different modalities into the same space through some mathematical transformations (such as Fourier transform, wavelet transform, etc.) and then fuses them. The shared feature extractor uses a shared feature extractor (such as a shared convolutional neural network layer) to extract features from different modalities of data, making the extracted features have a certain degree of consistency for subsequent fusion.

[0069] 2) Intermediate Fusion; the Intermediate Fusion includes, but is not limited to, attention mechanism fusion and circulant matrix fusion. The attention mechanism fusion uses the attention mechanism to weight the features of different modalities, highlighting the important feature parts. For example, design an attention layer with gated weighting to fuse visual information and text information. The circulant matrix fusion reshapes the feature vector into a circulant matrix and defines the interaction operation between the vector and the matrix to fully explore the interaction between different modality features.

[0070] 3) Late Fusion; the Late Fusion includes, but is not limited to, voting method and model fusion. Among them, the voting method performs weighted voting based on the prediction results of each modality. The model fusion inputs the features of different modalities into different models respectively, and finally fuses the output results of these models.

[0071] 4) Learning-based Fusion; the Learning-based Fusion includes, but is not limited to, multi-layer perceptron (MLP) fusion and deep learning model fusion. Among them, the multi-layer perceptron fusion inputs the features of different modalities into the multi-layer perceptron, and through its non-linear transformation, automatically learns the fusion method of the features. The deep learning model fusion uses deep learning models (such as convolutional neural networks, Transformers, etc.) to fuse multi-modal features. For example, use the method of combining Transformer and DenseNet to extract and fuse the features of images.

[0072] 5) Special Fusion Methods; including, but not limited to, consistent feature learning, cross-modal alignment perception, and feature diffusion model. The consistent feature learning compensates for single-modal features through a learnable modality dictionary to maintain the consistency between multi-modal features. The cross-modal alignment perception optimizes the fusion process by constructing a correlation matrix to describe the relationship between source image pixels. The feature diffusion model uses the feature diffusion model to enhance and classify features, eliminating unnecessary noise and patterns.

[0073] In step S104, use the trained target model to output the corresponding psychological assessment result based on the fused feature information.

[0074] According to one embodiment, the target model is an intelligent classification and evaluation model of mental state based on machine learning. After model verification, the method automatically optimizes the intelligent model training using a self-learning algorithm, so that the intelligent classification and evaluation model of mental state automatically iteratively updates according to user data and model verification results.

[0075] Optionally, the target model is constructed using various machine learning techniques such as deep belief network (DBN), convolutional neural network (CNN), artificial neural network (ANN), support vector machine (SVM), logistic regression, linear regression, Transformer algorithm, and ensemble machine learning algorithms. The target model is used to identify the types of psychological problems, such as anxiety, depression, stress, fear, psychosis, etc., and evaluate the severity of these psychological problems, such as mild, moderate, severe, etc.

[0076] Among them, the psychological assessment results include various information that can indicate the types of psychological problems and the severity of psychological problems of the object to be evaluated.

[0077] For example, the psychological assessment results may include the types of psychological problems (such as anxiety, depression), the severity of the problems (such as a score from 0 to 10), and cognitive function scores such as executive function or memory.

[0078] According to one embodiment, the method further includes step S105.

[0079] In step S105, based on the psychological assessment results output by the target model, corresponding result feedback information and intervention strategy information are provided.

[0080] Among them, the result feedback information includes feedback on the type of psychological problem, feedback on the severity of the psychological problem, feedback on the susceptibility and risk of the psychological problem, feedback on the cognitive function status, and summary feedback.

[0081] Among them, the intervention strategy information includes various intervention measures and suggestions for the psychological assessment results. The intervention measures and suggestions include, but are not limited to, emotional problem intervention measures, cognitive problem intervention measures, behavioral problem intervention measures, psychological adjustment and maintenance suggestions. For example, mindfulness breathing training is recommended for anxiety.

[0082] According to one embodiment, the method trains the target model through step S106.

[0083] In step S106, the target model is trained based on the multi-modal data of multiple sample objects, so that the target model learns how to output corresponding psychological assessment results based on the multi-modal data of the sample objects through training.

[0084] Among them, the target model dynamically updates parameters based on incremental learning.

[0085] In the embodiments of the present application, after preprocessing and enhancement processing of the multi-modal data, the model is initialized and pre-trained, the loss is calculated through forward propagation, the parameters are updated using an optimization algorithm, and at the same time, the model performance is monitored and an early stopping mechanism is applied. When new data accumulates to a certain extent, online incremental training is performed to improve the performance and generalization ability of the model.

[0086] Among them, the preprocessing is the same as or similar to the way of preprocessing the multi-modal data of the object to be evaluated mentioned above, which will not be elaborated here.

[0087] Among them, the enhancement processing includes, but is not limited to, processing such as synonym replacement or data augmentation for text data, processing such as rotation, flipping, adding noise or color adjustment for image data, processing such as speed adjustment or spectrum adjustment for audio data, and processing such as signal superposition or noise injection for physiological signal data.

[0088] According to one embodiment, step S106 includes steps S1061 to S1064.

[0089] In step S1061, target training data is obtained. Among them, the target training data includes multi-modal data corresponding to multiple sample objects and their annotation results.

[0090] Among them, the multi-modal data includes, but is not limited to, physiological-related data, cognitive behavior data, bio-chemical data, device usage data, self-assessment report data, and medical treatment data.

[0091] Among them, the annotation results include, but are not limited to, the types of psychological problems of the sample objects and information indicating the severity of psychological problems.

[0092] According to one embodiment, multi-modal training sample data including bio-chemical data, psychological state data, and other related data is loaded, and data preprocessing and data enhancement are performed. Then, the sample data is annotated, and the annotation content includes information for indicating the types of psychological problems and the severity of psychological problems of the sample objects. The annotated data set is divided into a training data set and a validation data set according to a certain ratio. Among them, in model training, dividing the data set into a training set and a validation set can monitor the accuracy of the validation set to prevent the model from overfitting.

[0093] In step S1062, feature extraction processing and feature fusion processing are performed on the multi-modal data of the multiple sample objects, and the obtained fusion features are used as input features for model training.

[0094] In step S1063, the psychological assessment result output by the target model based on the input features is obtained.

[0095] In step S1064, according to the difference between the psychological assessment result output by the target model and the actual psychological assessment result of the sample object, the target model is continuously updated until the difference meets the predetermined requirements. Specifically, through multiple iterative trainings, the loss between the psychological assessment result obtained in each iteration and the actual psychological assessment result is calculated, and the model parameters are updated based on the calculated loss.

[0096] Among them, the predetermined requirements include one or more conditions for determining whether the expected training effect is achieved.

[0097] Optionally, the predetermined requirements include at least any one of the following conditions: the value of the loss function between the predicted psychological assessment result and the actual psychological assessment result of the sample is lower than a preset threshold; the value of a predetermined performance index of the model reaches a preset threshold, and the performance index includes but is not limited to accuracy or recall, etc.; the number of iterations reaches a preset number threshold.

[0098] According to one embodiment, other types of data are associated with biochemistry data and mental state data, and data features highly associated with biochemistry characteristics and mental state characteristics are selected for training in pre-training.

[0099] Among them, the method uses a predetermined selection algorithm to select multiple key features.

[0100] Among them, the key features are features that have a high correlation with biochemistry characteristics and mental state characteristics, or features that have significant differences at different levels of biochemistry characteristics and different mental state characteristics, or key core node features related to the network connection of biochemistry characteristics and mental state characteristics.

[0101] Among them, the selection algorithm includes but is not limited to feature correlation analysis, feature difference analysis, network analysis, and feature selection algorithms.

[0102] Among them, through feature correlation analysis, for each pair of features in the fused feature matrix, calculate the correlation coefficient between them, such as Pearson correlation coefficient or Spearman rank correlation coefficient, to measure the strength of the linear or non-linear relationship between the features. According to the specific problem and data characteristics, set a correlation threshold. If the absolute value of the correlation coefficient between two features is greater than this threshold, it is considered that they have a high correlation. For other data features that have a high correlation with biochemistry characteristics and mental state characteristics, that is, features with an absolute value of the correlation coefficient greater than the threshold, they are selected as key features.

[0103] Among them, through feature difference analysis, according to different biochemistry characteristic levels and different mental state characteristics, the data set is divided into multiple groups. Then, perform a difference test on the fused features between different groups, such as t-test, analysis of variance, etc., and calculate the corresponding p-value. Then, determine a significance level (for example, 0.05). If the p-value of a certain feature on different groups is less than this significance level, it is determined that the feature has a significant difference at different levels or states, and it is selected as a key feature.

[0104] Among them, through network analysis, using biophysical and biochemical characteristics and psychological state characteristics as nodes, a network diagram can be constructed according to the correlation or other association metrics between them. Network analysis tools such as NetworkX can be used. Then, by calculating indicators such as degree centrality and betweenness centrality of the nodes, the characteristics of the key core nodes in the network are identified. These nodes have close connection relationships with many other important characteristics and have a key impact on the structure and function of the network. If a certain data characteristic is a key core node or has a close connection with these key nodes, it is selected as a key characteristic.

[0105] Among them, through a feature selection algorithm, according to specific requirements and data characteristics, a suitable feature selection algorithm is selected, such as recursive feature elimination (RFE), or model-based feature selection (such as calculating feature importance scores using models such as random forest and XGBoost), etc. The selected algorithm is used to analyze the fused features and calculate the importance score or ranking of each feature. Then, according to the feature importance score or ranking, the features with higher scores are selected as key features. At the same time, combined with the requirements of high correlation and high complementarity, the final set of key features is further screened out.

[0106] According to an embodiment, a cross-validation strategy and independent samples are used to verify and evaluate the trained target model, and the model is comprehensively evaluated through multiple evaluation metrics. Among them, the cross-validation strategy includes K-fold cross-validation, leave-one-user-out, time series segmentation, etc. The evaluation methods include but are not limited to AUC-ROC, F1-score (macro average), confusion matrix, RMSE, MAE, R² score, SHAP value analysis (feature importance ranking), and attention weight visualization for Transformer models.

[0107] According to an embodiment, the trained target model is hardware-accelerated and model-optimized through a distributed training and inference strategy. Among them, the distributed training includes but is not limited to multi-GPU data parallelism and mixed-precision training. The inference strategy includes but is not limited to TensorRT engine conversion.

[0108] According to the method of the present application, by obtaining target data of multiple modalities and using a trained target model to output corresponding psychological assessment results based on the fusion features of the multi-modal data, this multi-modal data fusion method can comprehensively depict the complexity of the psychological state, reduce the limitations of a single data source, and the various modality data can be mutually verified and supplemented, thus significantly improving the accuracy and reliability of the psychological assessment results; the model in the embodiments of the present application is an intelligent classification and evaluation model of psychological state based on machine learning. After model verification, a self-learning algorithm is used to automatically optimize the intelligent model training, enabling the intelligent classification and evaluation model of psychological state to automatically iterate and update according to user data and model verification results. This dynamic optimization mechanism enables the model to adapt to the differences and dynamic changes of individual psychological states, thereby realizing the personalization and dynamics of the assessment and further improving the prediction accuracy of the model; by associating other types of data with biochemistry data and psychological state data, data features highly correlated with biochemistry characteristics and psychological state characteristics are selected for training in pre-training, significantly improving the pertinence and training efficiency of model training and the prediction accuracy of the model.

[0109] The method of the embodiments of the present application will be described below with an example.

[0110] Referring to Figure 3 , Figure 3 shows a schematic diagram of an exemplary intelligent psychological assessment system according to an embodiment of the present application. The system includes: a multi-modal data acquisition module, a multi-modal data preprocessing module, a data feature extraction module, a psychological state classification and evaluation module, an assessment result feedback module, and a personalized psychological intervention module.

[0111] Among them, the multi-modal data acquisition module is used to obtain physiological-related data, cognitive behavior data, biochemistry data, device usage data, self-assessment report data, and medical treatment data of the testee through wearable devices, intelligent sensors, and biological sample analysis.

[0112] Among them, the multi-modal data preprocessing module is used to clean and standardize the multi-modal data.

[0113] Among them, the data feature extraction module is used for feature extraction and cross-modal fusion of the multi-modal data.

[0114] Among them, the psychological state classification and evaluation module is used to output the psychological state classification and severity evaluation results based on the input fusion features through a trained model. Among them, deep belief network (DBN), convolutional neural network (CNN), artificial neural network (ANN), support vector machine (SVM), logistic regression, linear regression algorithm, Transformer algorithm, and integrated machine learning algorithm are used to create the model. The created model can not only distinguish the types of psychological problems but also evaluate the severity of psychological problems. The model is trained in the manner of step S106 above, which will not be elaborated here. Among them, when the Transformer model is used in this example, the number of attention heads is 8, and the accuracy of the training set reaches 92%.

[0115] Among them, the test result feedback module is used to feedback the test results, including the feedback of the type of psychological problem, the feedback of the severity of the psychological problem, the feedback of the susceptibility and risk of the psychological problem, the feedback of the cognitive function status, and the summary feedback.

[0116] Among them, the personalized psychological intervention module is used to generate personalized intervention suggestions and feedback them through intelligent devices.

[0117] The intelligent psychological assessment system in this example uses multi-modal data for psychological assessment. The multi-modal data can be mutually verified and supplemented, making the assessment results output by the model more accurate and reliable. The AUC of this multi-modal data fusion method is 10% higher than that of a single data source.

[0118] Figure 4 The structural schematic diagram of a device for psychological assessment provided by an embodiment of the present application is shown. The device includes: a device for obtaining multi-modal data of an object to be evaluated (hereinafter referred to as "data acquisition device 101"), a device for performing feature extraction processing on the obtained multi-modal data to obtain corresponding feature information (hereinafter referred to as "feature acquisition device 102"), a device for fusing the feature information corresponding to the multi-modal data to obtain corresponding fusion feature information (hereinafter referred to as "feature fusion device 103"), and a device for using a trained target model to output corresponding psychological assessment results based on the fusion feature information (hereinafter referred to as "result prediction device 104").

[0119] Refer to Figure 4 , the data acquisition device 101 acquires the multi-modal data of the object to be evaluated, and the multi-modal data includes physiological and behavioral data from multiple data sources.

[0120] Among them, the object to be evaluated refers to the object that needs to undergo psychological assessment.

[0121] Among them, the multi-modal data includes various data that can reflect the psychological state of the object to be evaluated.

[0122] According to one embodiment, the multimodal data includes physiological-related data, cognitive behavior data, biochemical data, device usage data, self-assessment report data, and medical treatment data. The meanings and data collection methods of each type of data will be described below respectively.

[0123] 1) Physiological-related data; The meaning of the physiological-related data has been described above and will not be repeated here.

[0124] For basic physiological data, the data acquisition device 101 collects it through physiological assessment devices or wearable devices. For other physiological data, the data acquisition device 101 interacts with the intelligent device through the object to be evaluated, and when the preset tasks in the intelligent device are completed, the intelligent device automatically collects the data.

[0125] 2) Cognitive behavior data; The meaning of the cognitive behavior data has been described above and will not be repeated here.

[0126] For cognitive behavior data, the data acquisition device 101 interacts with the intelligent device through the object to be evaluated, and when the object to be evaluated completes the corresponding cognitive component test tasks, the intelligent device automatically collects the data.

[0127] 3) Biochemical data; The meaning has been described above and will not be repeated here.

[0128] For biochemical data, the data acquisition device 101 collects samples of blood, urine, saliva, feces, and spinal fluid from the object to be evaluated, and conducts basic biochemical tests, genomics, transcriptomics, proteomics, and metabolomics analyses on the samples to obtain relevant biochemical indicators and data results as corresponding features.

[0129] 4) Device usage data; The meaning of the device usage data has been described above and will not be repeated here.

[0130] For device usage data, the data acquisition device 101 obtains it by acquiring the permissions of the intelligent devices and applications used by the object to be evaluated, and the intelligent device automatically collects the data.

[0131] 5) Self-assessment report data; The meaning of the self-assessment report data has been described above and will not be repeated here.

[0132] For self-assessment report data, the data acquisition device 101 conducts it by having the object to be evaluated complete the self-assessment questionnaire preset in the intelligent device, and the intelligent device automatically collects the data.

[0133] 6) Medical treatment data; The meaning of the diagnosis and treatment data has been described above and will not be elaborated here.

[0134] For the diagnosis and treatment data, the data acquisition device 101 collects it by docking with the diagnosis and treatment management system (HIS system) of the hospital where the tester seeks medical treatment.

[0135] Optionally, the physiological-related data, cognitive behavior data, and device usage data can be collected synchronously.

[0136] It should be noted that the target data involved in the embodiments of the present application are all obtained with the consent or permission of the relevant parties, and the collection, use, and processing of the relevant target data need to comply with the relevant laws, regulations, and standards of the relevant countries and regions. Moreover, the collection, processing, and preservation of the biochemical data in the embodiments of the present application all comply with the ethical review standards.

[0137] According to one embodiment, the device includes a device for preprocessing the data of the acquired multimodal data.

[0138] Among them, the preprocessing includes but is not limited to various processes such as cleaning, standardizing, and transforming the obtained multimodal data. Optionally, the preprocessing includes processes such as word segmentation and stop word removal for text data, cropping, scaling, or denoising for image data, noise reduction or normalization for audio data, and filtering or normalization for physiological signal data.

[0139] The feature acquisition device 102 performs feature extraction processing on the acquired multimodal data to obtain corresponding feature information.

[0140] The feature acquisition device 102 determines the type of the multimodal data to be processed and adopts a feature extraction operation matching this type.

[0141] The feature extraction operations for different types of target data will be described below.

[0142] 1) Physiological-related data; For the basic physiological data, the feature acquisition device 102 first performs signal preprocessing, and then extracts the corresponding time-domain features, frequency-domain features, and / or non-linear features of the basic physiological data.

[0143] Specifically, through signal preprocessing, the feature acquisition device 102 filters electroencephalogram / electrocardiogram data using band-pass filtering (0.5 - 49 Hz) and removes artifacts such as eye movement, body movement, and electromyogram using independent component analysis (ICA), smooths the skin conductance using the moving average method (window length 5 s), and standardizes the sampling frequency of the respiratory rhythm to 10 Hz using cubic spline interpolation.

[0144] For facial image data, the feature acquisition device 102 acquires features corresponding to the facial image data by performing facial key point extraction, facial motion analysis and spatiotemporal feature extraction.

[0145] For speech data, the feature acquisition device 102 extracts fundamental frequency features and emotional features to analyze the emotional and intonation information of the speech signal. Specifically, the multidimensional coefficients are extracted by the Mel filter bank to calculate the MFCC features, the fundamental frequency is calculated by the autocorrelation function (F0=argmax(R(τ))), and the eGeMAPS parameter set is extracted by the opensmile toolkit to calculate the emotional features.

[0146] 2) cognitive behavioral data; For cognitive behavior data, the feature acquisition device 102 first performs signal preprocessing to remove abnormal values, and then calculates the average reaction time, the coefficient of variation of the reaction time and the accuracy rate as characteristics reflecting the cognitive behavior of the object to be evaluated.

[0147] For example, to calculate the reaction time and accuracy of the subject to be evaluated when performing cognitive behavioral responses, first remove the outliers (excluding trial data within ±3SD), then calculate the average reaction time and coefficient of variation of the reaction time, and at the same time calculate the accuracy of the correct response.

[0148] 3) Biological and biochemical data; For biological and biochemical data, the feature acquisition device 102 extracts gene variation features, association analysis results and functional features of genomics, protein expression, modification and functional features of proteomics, and metabolite concentration, differential analysis and pathway features of metabolomics through omics data analysis.

[0149] The omics data analysis includes genomics, proteomics, and metabolomics analysis. The analysis process of genomics, proteomics, and metabolomics has been described in the previous text and will not be repeated here.

[0150] For conventional detection indicators, the feature acquisition device 102 extracts the inflammation index and the metabolic syndrome score.

[0151] 4) Equipment usage data; For the device usage data, the feature acquisition device 102 calculates the daily usage entropy value, application switching frequency and timing dependency features as features for evaluating the user's device usage habits and behavior patterns.

[0152] For example, we can analyze usage patterns, extract daily usage entropy values ​​(in hours) and application switching frequency (number of application switches / usage duration), and perform behavior sequence modeling to extract timing dependency features through the LSTM network.

[0153] 5) Self-assessment report data; For the self - evaluation report data, the feature acquisition device 102 calculates the total scores and average scores of the questionnaires and scales, and calculates the total scores and average scores of the multi - dimensional data included in the report data as the features corresponding to the report data.

[0154] 6) Diagnostic data; For the diagnostic data, the feature acquisition device 102 extracts diagnostic codes and converts them into classification data, calculates the comorbidity index to evaluate the comorbidity situation, and analyzes the medication information to calculate the drug burden index and the complexity of the treatment plan, thereby completing the feature extraction of the clinical diagnostic data.

[0155] For example, by extracting diagnostic codes and converting them into classification data, the Charlson index is calculated by weighted summation to evaluate the comorbidity situation; by constructing and analyzing treatment features, the drug burden index and the complexity of the treatment plan (the number of drug types × dosing frequency) are calculated.

[0156] Continue to refer to Figure 4 for the following description. The feature fusion device 103 fuses the feature information corresponding to the multi - modal data to obtain the corresponding fused feature information.

[0157] Among them, the fused feature information can correspond to one or more modalities of data.

[0158] Among them, the fusion processing method is applicable to a variety of psychological assessment scenarios.

[0159] The way in which the feature fusion device 103 performs the fusion processing has been described in part of step S103 above and will not be elaborated here.

[0160] The result prediction device 104 uses the trained target model to output the corresponding psychological assessment result based on the fused feature information.

[0161] According to one embodiment, the target model is an intelligent classification and evaluation model of mental state based on machine learning. After model verification, the method automatically optimizes the intelligent model training using a self - learning algorithm, so that the intelligent classification and evaluation model of mental state automatically iteratively updates according to user data and model verification results.

[0162] Among them, the psychological assessment result includes various information that can indicate the type of psychological problems and the severity of psychological problems of the object to be evaluated. For example, the psychological assessment result can include the type of psychological problems (such as anxiety, depression), the severity of the problem (such as a score from 0 to 10), and cognitive function scores such as executive function or memory.

[0163] According to one embodiment, the device further includes a feedback intervention device.

[0164] The feedback intervention device provides corresponding result feedback information and intervention strategy information based on the psychological assessment results output by the target model.

[0165] Among them, the result feedback information includes feedback on the type of psychological problems, feedback on the severity of psychological problems, feedback on the susceptibility and risks of psychological problems, feedback on the cognitive function status, and summary feedback.

[0166] Among them, the intervention strategy information includes but is not limited to intervention measures and suggestions for the results, providing intervention measures for emotional problems, cognitive problems, behavioral problems, psychological adjustment and maintenance suggestions. For example, mindfulness breathing training is recommended for anxiety.

[0167] According to one embodiment, the device further includes a model training device.

[0168] The model training device trains the target model based on the multimodal data of multiple sample objects, so that the target model learns through training how to output corresponding psychological assessment results based on the multimodal data of the sample objects.

[0169] Among them, the preprocessing and enhancement processing have been described above and will not be elaborated here.

[0170] Among them, the target model dynamically updates parameters based on incremental learning.

[0171] In the embodiments of the present application, after preprocessing and enhancement processing of the multimodal data, the model is initialized and pre-trained, the loss is calculated through forward propagation, the parameters are updated using an optimization algorithm, and at the same time, the model performance is monitored and an early stopping mechanism is applied. When new data accumulates to a certain extent, online incremental training is performed to improve the performance and generalization ability of the model.

[0172] According to one embodiment, the model training device is used to perform the following operations.

[0173] The model training device obtains target training data. Among them, the target training data includes multimodal data corresponding to multiple sample objects and their annotation results.

[0174] Among them, the multimodal data includes but is not limited to physiological-related data, cognitive behavior data, biochemical data, device usage data, self-assessment report data, and diagnosis and treatment data.

[0175] Among them, the annotation results include but are not limited to the type of psychological problems of the sample objects and information indicating the severity of psychological problems.

[0176] According to one embodiment, the model training device loads multimodal training sample data, including biochemistry data, psychological state data, and other relevant data, and performs data preprocessing and data augmentation. Then, the sample data is labeled, and the labeling content includes information indicating the type and severity of the psychological problems of the sample object. The labeled data set is divided into a training data set and a validation data set according to a certain ratio. Among them, when dividing the data set into a training set and a validation set during model training, the accuracy of the validation set can be monitored to prevent the model from overfitting.

[0177] Then, the model training device performs feature extraction processing and feature fusion processing on the multimodal data of the multiple sample objects, and uses the obtained fusion features as the input features for model training.

[0178] Then, the model training device obtains the psychological assessment result output by the target model based on the input features.

[0179] Then, according to the difference between the psychological assessment result output by the target model and the actual psychological assessment result of the sample object, the target model is continuously updated until the difference meets the predetermined requirements. Specifically, through multiple iterative trainings, the loss between the psychological assessment result obtained in each iteration and the actual psychological assessment result is calculated, and the model parameters are updated based on the calculated loss.

[0180] Among them, the predetermined requirements include one or more conditions for judging whether the expected training effect is achieved.

[0181] Optionally, the predetermined requirements include but are not limited to at least any one of the following conditions: the value of the loss function between the predicted psychological assessment result and the actual psychological assessment result of the sample is lower than the preset threshold; the value of the predetermined performance index of the model reaches the preset threshold, and the performance index includes but is not limited to accuracy or recall, etc.; the number of iterations reaches the preset number threshold.

[0182] According to one embodiment, the model training device associates other types of data with biochemistry data and psychological state data, and selects data features highly correlated with biochemistry features and psychological state features for training during pre-training..

[0183] Among them, the model training device uses a predetermined selection algorithm to select multiple key features.

[0184] Among them, the key features are features highly correlated with biochemistry features and psychological state features, or features with significant differences in different levels of biochemistry features and different psychological state features, or key core node features related to the network connection of biochemistry features and psychological state features.

[0185] Among them, the process of selecting key features based on the selection algorithm has been described in the foregoing text and will not be elaborated here.

[0186] According to one embodiment, the model training device uses a cross-validation strategy and independent samples to verify and evaluate the trained target model, and comprehensively evaluates the model through a variety of evaluation metrics. Among them, the cross-validation strategy includes K-fold cross-validation, leave-one-user-out, time series segmentation, etc. The evaluation methods include but are not limited to AUC-ROC, F1-score (macro average), confusion matrix, RMSE, MAE, R² score, SHAP value analysis (feature importance ranking), and attention weight visualization for Transformer models.

[0187] According to one embodiment, the trained target model is hardware-accelerated and model-optimized through a distributed training and inference strategy. Among them, the distributed training includes but is not limited to multi-GPU data parallelism and mixed-precision training. The inference strategy includes but is not limited to TensorRT engine conversion.

[0188] According to the device of the present application, by obtaining target data of multiple modalities and using the trained target model to output corresponding psychological assessment results based on the fusion features of the multi-modal data, this multi-modal data fusion method can comprehensively depict the complexity of the psychological state, reduce the limitations of a single data source, and the various modal data can be mutually verified and supplemented, thereby significantly improving the accuracy and reliability of the psychological assessment results; the model of the embodiment of the present application is an intelligent classification and evaluation model of psychological state based on machine learning. After the model is verified, a self-learning algorithm is used to automatically optimize the intelligent model training, so that the intelligent classification and evaluation model of psychological state automatically iterates and updates according to the user data and model verification results. This dynamic optimization mechanism enables the model to adapt to the differences and dynamic changes of individual psychological states, thereby realizing the personalization and dynamics of the assessment and further improving the prediction accuracy of the model; by associating other types of data with biochemistry data and psychological state data, data features highly correlated with biochemistry characteristics and psychological state characteristics are selected for training in pre-training, significantly improving the pertinence and training efficiency of model training and the prediction accuracy of the model.

[0189] Based on the same inventive concept, an embodiment of the present application further provides an electronic device. The method corresponding to the electronic device may be the method for psychological assessment in the foregoing embodiments, and the principle of solving problems is similar to that of the method. The electronic device provided in the embodiment of the present application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of multiple embodiments of the present application described above.

[0190] The electronic device may be a user device, or a device formed by integrating a user device and a network device through a network, or may also be an application program running on the above device. The user device includes, but is not limited to, various terminal devices such as a computer, a mobile phone, a tablet computer, a smart watch, and a bracelet. The network device includes, but is not limited to, a network host, a single network server, a set of multiple network servers, or a computer set based on cloud computing, etc., and can be used to implement some processing functions when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing. Among them, cloud computing is a type of distributed computing, which consists of a virtual computer formed by a group of loosely coupled computer sets.

[0191] Figure 5 The structure of a device suitable for implementing the methods and / or technical solutions in the embodiments of the present application is shown. The device 1200 includes a central processing unit (CPU, Central Processing Unit) 1201, which can execute various appropriate actions and processes according to the program stored in the read-only memory (ROM, Read Only Memory) 1202 or the program loaded from the storage part 1208 into the random access memory (RAM, Random Access Memory) 1203. In the RAM 1203, various programs and data required for system operation are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204. The input / output (I / O, Input / Output) interface 1205 is also connected to the bus 1204.

[0192] The following components are connected to the I / O interface 1205: an input part 1206 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, etc.; an output part 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, etc. and a speaker; a storage part 1208 including one or more computer-readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, etc.; and a communication part 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication part 1209 performs communication processing via a network such as the Internet.

[0193] Specifically, the method and / or embodiments in the embodiments of the present application can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program codes for executing the method shown in the flowchart. When the computer program is executed by a central processing unit (CPU) 1201, the above functions defined in the method of the present application are executed.

[0194] Another embodiment of the present application also provides a computer-readable storage medium, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the method and / or technical solution of any one or more of the foregoing embodiments of the present application.

[0195] Specifically, this embodiment can adopt any combination of one or more computer-readable media. The computer-readable media can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the computer-readable storage medium include: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this document, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0196] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including - but not limited to - electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device.

[0197] The program code contained on a computer-readable medium may be transmitted using any appropriate medium, including - but not limited to - wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0198] The computer program code for performing the operations of this application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0199] The flowcharts or block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combination of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0200] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0201] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or page components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0202] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0203] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above integrated units can be implemented in the form of hardware or in the form of hardware plus software functional units.

[0204] The above integrated units implemented in the form of software functional units can be stored in a computer-readable storage medium. The above software functional units stored in a storage medium include several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in each embodiment of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0205] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present application.

[0206] In addition, it is obvious that the word "including" does not exclude other units or steps, and the singular does not exclude the plural. A plurality of units or devices stated in the device can also be implemented by one unit or device through software or hardware. The words such as first and second are used to represent names, rather than indicating any specific order.

Claims

1. A method for conducting psychological assessment, characterized in that, The method includes: Obtaining multimodal data of the object to be evaluated, where the multimodal data includes physiological and behavioral data from multiple data sources; Performing feature extraction processing on the obtained multimodal data to obtain corresponding feature information; Fusing the feature information corresponding to the multimodal data to obtain corresponding fused feature information; Using the trained target model to output corresponding psychological assessment results based on the fused feature information, where the target model is automatically iteratively updated based on user data and model verification results.

2. The method according to claim 1, characterized in that, The target model is an intelligent classification and evaluation model of mental state based on machine learning. After model verification, the method automatically optimizes the training of the intelligent model using a self-learning algorithm, enabling the intelligent classification and evaluation model of mental state to be automatically iteratively updated according to user data and model verification results.

3. The method according to claim 1, characterized in that, The method further includes: Providing corresponding result feedback information and intervention strategy information based on the psychological assessment results output by the target model.

4. The method according to claim 3, wherein The intervention strategy information includes intervention measures for emotional problems, cognitive problems, behavioral problems, psychological adjustment and maintenance suggestions.

5. The method according to claim 1, characterized in that The multimodal data includes physiological-related data, cognitive behavior data, bio-chemical data, device usage data, self-assessment report data, and medical treatment data. The physiological data includes basic physiological data and other physiological data.

6. The method according to claim 5, wherein The ways for the method to obtain multimodal data include: For basic physiological data, it is collected through physiological assessment devices or wearable devices; For other physiological data, it is automatically collected by the intelligent device when the object to be evaluated interacts with the intelligent device and completes the preset tasks in the intelligent device; For cognitive behavior data, it is automatically collected by the intelligent device when the object to be evaluated completes the corresponding cognitive component test tasks during the interaction with the intelligent device; For bio-chemical data, by collecting samples of the blood, urine, saliva, feces, and spinal fluid of the object to be evaluated and performing basic biochemical tests, genomics, transcriptomics, proteomics, and metabolomics analyses on the samples, relevant bio-chemical indicators and data results are obtained.

7. The method according to claim 1, characterized in that, The method further includes: Training the target model based on the multimodal data of multiple sample objects, enabling the target model to learn through training how to output corresponding psychological assessment results based on the multimodal data of the sample objects.

8. The method according to claim 7, wherein Training the target model based on the multimodal data of multiple sample objects includes: Obtaining target training data, where the target training data includes multimodal data corresponding to multiple sample objects and their annotation results; Performing feature extraction processing and feature fusion processing on the multimodal data of the multiple sample objects, and using the obtained fused features as input features for model training; Obtaining the psychological assessment results output by the target model based on the input features; Continuously updating the target model according to the difference between the psychological assessment results output by the target model and the actual psychological assessment results of the sample objects until the difference meets the predetermined requirements.

9. The method according to claim 7 or 8, characterized in that, The method further includes: Associate other types of data with biophysical and biochemical data and psychological state data, and select data features highly correlated with biophysical and biochemical features and psychological state features for training in the pre-training of the target model.

10. The method according to claim 7 or 8, characterized in that, The target model dynamically updates its parameters based on incremental learning.

11. The method according to claim 7 or 8, characterized in that, The method uses a cross-validation strategy and independent samples to verify and evaluate the trained target model.

12. The method according to claim 7 or 8, characterized in that The method performs hardware acceleration and optimization of the target model through a distributed training and inference strategy.

13. A device for performing psychological assessment, characterized in that, The device includes: A device for acquiring multimodal data of an object to be evaluated, where the multimodal data includes physiological and behavioral data from multiple data sources; A device for performing feature extraction processing on the acquired multimodal data to obtain corresponding feature information; A device for performing fusion processing on the feature information corresponding to the multimodal data to obtain corresponding fused feature information; A device for using the trained target model to output corresponding psychological assessment results based on the fused feature information, where the target model automatically iteratively updates its parameters based on user data.

14. An electronic device, the electronic device includes: At least one processor; And A memory communicatively connected to the at least one processor; characterized in that The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1 to 12.

15. A computer-readable medium, on which computer program instructions are stored, and the computer program instructions can be executed by a processor to implement the method according to any one of claims 1 to 12.

16. A computer program product, including a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 12.

Images