Mental disease category identification method and device, computer equipment and storage medium

Abstract

The invention relates to a mental disease category identification method and device, computer equipment and a storage medium. The method comprises the following steps: acquiring multi-modal data of a target patient; performing feature extraction on each type of modal data to obtain a feature vector of each type of modal data, and generating a multi-modal fusion feature based on the feature vector of each type of modal data; matching the multi-modal fusion feature with a pre-constructed informatization expression mode of pathological expressions of various mental diseases, and determining a target informatization expression mode; and determining the mental disease category of the target patient according to the target informatization expression mode. The mental disease category of the patient can be effectively identified, and meanwhile, various evaluation models do not need to be trained respectively.

Description

Technical Field

[0001] The present application relates to the interdisciplinary field of mental illness identification and artificial intelligence, and in particular to a method, apparatus, computer device and storage medium for identifying the category of mental illness. Background Art

[0002] Mental illness is a type of disease that seriously affects people's physical and mental health. In the past, the diagnosis and evaluation of mental illness mainly relied on the doctor's subjective judgment and experience, and lacked objective and accurate evaluation methods.

[0003] With the development of information technology, there are currently related technologies that use artificial intelligence technology to build intelligent evaluation models, and build corresponding evaluation systems based on the models. The above-mentioned evaluation models can be, for example, depression evaluation models, anxiety evaluation models, bipolar disorder evaluation models, etc., and the evaluation systems can be depression evaluation systems, anxiety evaluation systems, and bipolar disorder evaluation systems. The above-mentioned evaluation models can generate specific evaluation results based on the multidimensional data of the patient. For example, the evaluation results of the depression evaluation model can include information used to indicate whether the patient suffers from depression, and information used to indicate the degree of depression of the patient when the patient suffers from depression. However, the above-mentioned evaluation models can usually only be used to identify or evaluate a certain type of mental illness, and the generalization ability is weak, and it is not suitable for identifying or evaluating other mental illnesses. Taking the depression degree evaluation model as an example, it is a model trained using multimodal data of patients with depression to identify the degree of depression of patients. The model mainly learns and identifies the characteristics of depression. If the patient is not suffering from depression, but other mental illnesses, such as anxiety, then the model cannot accurately identify whether the patient suffers from other mental illnesses and the corresponding condition.

[0004] Since the generalization ability of the above-mentioned evaluation model is weak, if you want the evaluation system to be able to evaluate multiple mental illnesses, you usually need to train different evaluation models for different mental illnesses and integrate them into the evaluation system. However, training multiple evaluation models for different mental illnesses requires huge amounts of training data and computing resources, and will also increase the difficulty of managing the evaluation models in the later stages. Summary of the invention

[0005] In response to the above-mentioned deficiencies or shortcomings, the present application provides a method, apparatus, computer device and storage medium for identifying the category of mental illness. The embodiments of the present application can effectively identify the category of mental illness of a patient without the need to train multiple assessment models separately.

[0006] The present application provides a method for identifying a mental illness category according to a first aspect. In some embodiments, the method includes:

[0007] Acquire multimodal data of target patients;

[0008] Extract features from various modal data to obtain feature vectors of various modal data, and generate multimodal fusion features based on the feature vectors of various modal data;

[0009] Matching the multimodal fusion features with pre-constructed information expression patterns of pathological manifestations of various mental illnesses to determine the target information expression pattern;

[0010] Determine the target patient's mental illness category based on the target information expression pattern.

[0011] In some embodiments, feature extraction is performed on various types of modal data to obtain feature vectors of various types of modal data, and multimodal fusion features are generated based on the feature vectors of various types of modal data, including:

[0012] Inputting various modal data into a multimodal feature fusion model to obtain multimodal fusion features output by the multimodal feature fusion model;

[0013] The multimodal feature fusion model includes an input layer, multiple functional module layers for processing different modal data, and a feature fusion layer; the input layer is used to transmit various types of modal data to the corresponding functional module layers respectively; each functional module layer is used to extract feature vectors from the modal data input to this layer using the feature extraction network corresponding to this layer; the feature fusion layer is used to assign different weights to the feature vectors extracted by each functional module layer through a weighting mechanism or an attention mechanism, and generate multimodal fusion features based on the feature vectors extracted by each functional module layer and their corresponding weights.

[0014] In some embodiments, the multimodal data of the target patient is obtained by preprocessing the original multimodal data of the target patient using a non-parametric Bayesian model.

[0015] In some embodiments, the process of constructing the information expression patterns of the pathological manifestations of the multiple mental illnesses includes:

[0016] Obtain multimodal sample data from multiple patients;

[0017] Perform feature extraction on the multimodal sample data of each patient to obtain the multimodal features of each patient;

[0018] Clustering is performed based on the multimodal features of each patient to divide the multiple patients into multiple pathological clusters;

[0019] The Apriori algorithm was used to perform association rule analysis on each pathological cluster and an information expression model was constructed based on the analysis results to obtain information expression models of the pathological manifestations of various specified mental illnesses.

[0020] In some embodiments, the operation of using the Apriori algorithm to perform association rule analysis on each pathology cluster and constructing an information expression pattern based on the analysis results includes:

[0021] For each pathology cluster, the multimodal features of each patient in the pathology cluster are converted into feature item sets;

[0022] Use the Apriori algorithm to scan the feature item sets of all patients in the pathology cluster, determine the frequent item sets whose corresponding support thresholds meet the preset conditions, and mine association rules that reach the preset confidence thresholds from the frequent item sets;

[0023] An information-based expression pattern corresponding to the pathology cluster is constructed based on the mined association rules. The information-based expression pattern corresponding to the pathology cluster includes common symptoms of patients in the cluster and association rules between symptoms.

[0024] In some embodiments, after determining the mental illness category of the target patient according to the target information expression pattern, the method further includes:

[0025] A diagnostic report is generated based on the target information expression pattern, which includes the detected case manifestations and recommended treatment plans.

[0026] In some embodiments, the method further comprises:

[0027] The parameters and information expression mode of the multimodal feature fusion model are updated based on the doctor's feedback data and new patient's multimodal data.

[0028] According to a second aspect, the present application provides a device for identifying a mental illness category. In some embodiments, the device includes:

[0029] A data acquisition module, used to acquire multimodal data of a target patient;

[0030] The feature fusion module is used to extract features from various modal data, obtain feature vectors of various modal data, and generate multimodal fusion features based on the feature vectors of various modal data;

[0031] A pattern matching module is used to match the multimodal fusion feature with the pre-constructed information expression patterns of the pathological manifestations of various mental illnesses to determine the target information expression pattern;

[0032] The classification module is used to determine the mental illness category of the target patient based on the target information expression pattern.

[0033] According to a third aspect, the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for identifying the category of mental illness provided in any of the above embodiments are implemented.

[0034] According to a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for identifying the category of mental illness provided in any of the above embodiments.

[0035] In the above-mentioned embodiment of the present application, by acquiring the multimodal data of the target patient, feature extraction is performed on each type of modal data to obtain feature vectors of each type of modal data, and multimodal fusion features are generated based on the feature vectors of each type of modal data. Then, the multimodal fusion features are matched with the pre-constructed information expression patterns of the pathological manifestations of various mental illnesses to determine the target information expression pattern. Finally, the mental illness category of the target patient can be determined based on the target information expression pattern. This embodiment can effectively identify the patient's mental illness category without determining the patient's mental illness. At the same time, there is no need to train multiple evaluation models, which reduces the data, manpower, and computing resources required in the model training stage, and also reduces the difficulty of model management in the later stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 A flowchart of a method for identifying a mental illness category provided by the present application according to one or more embodiments;

[0037] Figure 2 A schematic diagram of the structure of a multimodal feature fusion model provided by the present application according to one or more embodiments;

[0038] Figure 3 A schematic diagram of a process for constructing an information-based expression model of pathological manifestations of mental illnesses according to one or more embodiments of the present application;

[0039] Figure 4 A structural block diagram of a device for determining a mental illness category provided by the present application according to one or more embodiments;

[0040] Figure 5 This is a diagram of the internal structure of a computer device provided by the present application according to one or more embodiments. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below in conjunction with the accompanying drawings. It should be clear that the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0042] When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. Instead, they are only examples of devices and methods consistent with some aspects of the present application as detailed in the attached claims.

[0043] In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, unless otherwise specified, "multiple" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0044] In response to the deficiencies or defects of related technologies, the present application provides a method for identifying the category of mental illness. The method can effectively identify the category of a patient's mental illness without being sure of the mental illness the patient suffers from. At the same time, the method does not require pre-training of multiple evaluation models during implementation, thereby reducing the data, manpower, and computing resources required in the model training phase, and also reduces the difficulty of managing the trained models.

[0045] In some embodiments, the method includes Figure 1 The steps shown are described in detail below.

[0046] S110: Acquire multimodal data of a target patient.

[0047] In this embodiment, the method can be applied to an intelligent evaluation system, which can be a computer device such as a personal computer or a server. It should be particularly noted that the intelligent evaluation system is not limited to a specific mental illness, and supports the evaluation of a variety of mental illnesses, such as depression, anxiety, drug addiction, etc. The system can dynamically identify the type of mental illness suffered by the patient by collecting data of different modalities of the patient and combining it with an artificial intelligence algorithm, so that it is suitable for auxiliary evaluation of a variety of mental and psychological diseases.

[0048] The target patients mentioned above refer to patients with the categories of mental illnesses to be identified. The system can collect multimodal data of target patients through a variety of information collection devices. The information collection equipment can include a variety of advanced equipment such as recording equipment, real-time cameras, 64-channel EEG recorders, near-infrared equipment and multi-channel bio-signal recording systems, which can comprehensively collect data of multiple modalities such as patients' voice, facial expressions and bio-signals. For example, the recording equipment can clearly record the patient's voice intonation, speech speed, volume and other characteristics, as well as the coherence and logic of the language; the real-time camera can capture subtle changes in the patient's facial expression, such as frowning, smiling, eye contact, etc., and these data can also be used to analyze the tension of the patient's facial muscles and the duration of the expression; the 64-channel EEG recorder can obtain the patient's brain electrical activity signals, including the frequency, amplitude, phase, etc. of brain waves, which can reflect the functional state of the patient's brain and the neural activity pattern; the near-infrared equipment and multi-channel bio-signal recording system can monitor the changes in the patient's heart rate, blood pressure, respiratory rate, skin galvanic response and other physiological indicators, as well as the fluctuations of these indicators in different situations.

[0049] Accordingly, multimodal data may include the patient's voice data, facial expression data, biosignal data and other data. Furthermore, multimodal data may also include data entered by the patient in the user operation interface through input devices (such as a mouse, keyboard, touchpad, etc.), such as information filled in a scale, etc. It should also be noted that before the system collects the patient's multimodal data, it is necessary to inform the patient and obtain the patient's authorization and consent before performing data collection operations through the information collection device. After completing the data collection operation, it is also necessary to pay attention to the security and privacy protection of the data to ensure that the patient's information is not leaked.

[0050] S120: extracting features from various types of modal data to obtain feature vectors of various types of modal data, and generating multimodal fusion features based on the feature vectors of various types of modal data.

[0051] Different methods can be used to extract features for data of different modalities, such as using ASR (auto-speech recognition) technology to process language data and using convolutional neural networks to process image data.

[0052] By extracting features from various modal data, multimodal features can be obtained, that is, feature vectors of various modal data. For example, speech features (such as speech speed, intonation, volume, speech coherence, emotional color of sound, etc.) can be extracted from speech data, facial expression features (such as facial muscle tension, eyebrow movement type, frequency and duration of various expressions, frequency and duration of eye contact, etc.) can be extracted from facial expression data, and physiological signal features (such as brain wave frequency, amplitude and phase information, regional differences in brain electrical activity, heart rate and heart rate variability, blood pressure level, respiratory rate, skin electrical level, etc.) can be extracted from physiological signal data. In this way, multimodal features of the target patient (including speech features, facial expression features, and physiological signal features) can be obtained.

[0053] After obtaining the multimodal features of the target patient, they can be fused through feature fusion technology (such as directly concatenating each feature vector into one feature vector) to obtain a comprehensive feature vector, namely, a multimodal fusion feature.

[0054] S130: Match the multimodal fusion feature with pre-constructed information expression patterns of pathological manifestations of various mental illnesses to determine the target information expression pattern.

[0055] Among them, the information expression pattern is used to describe the pathological manifestations of the corresponding mental illness. In this embodiment, the system will pre-build corresponding information expression patterns for the pathological manifestations of multiple specified mental illnesses through clustering algorithms and association rule mining technology. These information expression patterns can be used to assist in diagnosing the type of mental illness suffered by the patient.

[0056] S140: Determine the mental illness category of the target patient based on the target information expression pattern.

[0057] When using the system to assist in the diagnosis of target patients, the type of mental illness of the target patients can be identified through pre-built information expression patterns, thus providing doctors with diagnostic advice and treatment plans.

[0058] In some embodiments, features are extracted from various types of modal data to obtain feature vectors of various types of modal data, and multimodal fusion features are generated based on the feature vectors of various types of modal data, including: inputting various types of modal data into a multimodal feature fusion model, and obtaining multimodal fusion features output by the multimodal feature fusion model.

[0059] See also Figure 2 The multimodal feature fusion model includes an input layer, multiple functional module layers for processing different modal data, and a feature fusion layer; the input layer is used to transmit various types of modal data to the corresponding functional module layers respectively; each functional module layer is used to extract feature vectors from the modal data input to this layer using the feature extraction network corresponding to this layer; the feature fusion layer is used to assign different weights to the feature vectors extracted by each functional module layer through a weighting mechanism or an attention mechanism, and generate multimodal fusion features based on the feature vectors extracted by each functional module layer and their corresponding weights.

[0060] In this embodiment, the multimodal feature fusion model is specifically a FHF (Function & Hierarchy Framework) model, which is a framework model for multimodal information fusion. It integrates data from different modalities, such as speech, facial expressions, and physiological signals, through a hierarchical structure and functional modules. In this model, data from different modalities are processed at independent levels and weightedly fused according to preset functional rules to eventually form a comprehensive feature vector.

[0061] In the present embodiment, the input layer of the FHF model receives multimodal data, such as voice data, facial expression data and physiological signal data, and the input layer is connected to each functional module layer (Feature Extraction Layer), and different functional module layers are used to process different modal data. The input layer inputs the corresponding modal data into each functional module layer, so that different functional module layers are provided with different feature extraction networks, and the feature extraction network is used to extract modal features from modal data. The feature extraction network includes, for example, a convolutional neural network (for extracting features from facial expression data) and a recurrent neural network (for extracting features from voice data or physiological signal data). Each functional module layer is connected to a feature fusion layer (Intermediate Fusion Layer), and the modal features extracted by each layer are input into the feature fusion layer to combine the features of different modalities.

[0062] The feature fusion layer specifically assigns different weights to different modal features through a weighting mechanism or an attention mechanism, and then combines the modal features into a comprehensive feature vector based on the weights corresponding to each modal feature. Furthermore, a deep neural network can be used to perform nonlinear mapping on the combined comprehensive feature vector to obtain a multimodal fusion feature, and finally, the output layer will output the multimodal fusion feature.

[0063] The traditional feature fusion method mainly directly splices the features of each modality into a vector. However, this method is prone to information redundancy and noise when processing multimodal data. The early fusion or late fusion strategy either loses some information or limits the interaction between different modalities. In this embodiment, the FHF model is used to realize feature fusion. The FHF model adopts a hierarchical structure so that the data of each modality is fully processed in its own layer, avoiding the information loss that may be caused by direct fusion, and giving weights to each modal feature through a weighting mechanism and an attention mechanism, so that the model can dynamically adjust the weight of each modality, ensuring that important features can play a role in different situations, and has higher flexibility. In addition, by performing nonlinear mapping on the comprehensive feature vector, it is beneficial to capture the complex relationship between multimodal data, so as to adapt to the diagnosis of different mental illnesses. It is also worth noting that compared with simple feature splicing, the FHF model can better cope with the situation where some modal data is missing, and reduce the influence of noise and redundant information, and has stronger fault tolerance.

[0064] The following is an explanation of the training process of the FHF model, which involves joint learning of multimodal data and mainly includes the following steps:

[0065] (1) Data preprocessing: Standardize and denoise the data of each modality to ensure data quality and consistency.

[0066] (2) Single-modal feature learning: Use the data of each modality to train the model separately in order to extract key features. For example, use convolutional neural networks to extract facial expression features and use recurrent neural networks to analyze time series data of speech and physiological signals.

[0067] (3) Multimodal feature fusion: The features of each modality are input into the feature fusion layer of the model, and the features are weighted and combined according to the weight or attention mechanism.

[0068] (4) Model optimization and updating: Use a loss function (such as cross entropy loss) to evaluate the results of the output layer and continuously optimize the model parameters through the back propagation algorithm.

[0069] (5) Validation and testing: After training the model using the training set, use the validation set and test set to evaluate the performance of the model to prevent overfitting.

[0070] Furthermore, in some embodiments, the multimodal data of the target patient is obtained after information preprocessing is performed on the original multimodal data of the target patient using a non-parametric Bayesian model. That is, the system collects the original multimodal data of the target patient, and then uses the non-parametric Bayesian model to perform information preprocessing on the original multimodal data of the target patient, thereby obtaining the above-mentioned multimodal data of the target patient.

[0071] The non-parametric Bayesian model can automatically adjust the parameters of the model according to the distribution of the data, so as to better adapt to different data characteristics. In this embodiment, the non-parametric Bayesian model is used to process the original multimodal data, which can remove the noise and outliers in the original multimodal data and obtain a higher quality data set. In addition, the model can also smooth and complete the data of the time series signal type (such as physiological signal data) according to the data distribution, so as to facilitate the subsequent feature extraction and analysis of the data of the time series signal type.

[0072] In some embodiments, the process of constructing the information expression patterns of the pathological manifestations of the multiple mental illnesses is as follows: Figure 3 As shown, including:

[0073] S210: Acquire multimodal sample data of multiple patients.

[0074] Among them, multimodal sample data refers to the multimodal data of patients used to construct information expression models of pathological manifestations of different mental illnesses, which also includes data of multiple modalities, such as patients' voice data, facial expression data, physiological signal data, etc.

[0075] S220: Extract features from the multimodal sample data of each patient to obtain multimodal features of each patient.

[0076] The feature extraction process of this step may be parameterized by the feature extraction content introduced in the above embodiment, which will not be described in detail here.

[0077] S230: Clustering is performed based on the multimodal features of each patient to divide the multiple patients into multiple pathology clusters.

[0078] In this embodiment, clustering algorithms such as Kmeans (K means), KNN (K-Nearest Neighbor), T-SNE (t-Distributed Stochastic Neighbor Embedding) and the like may be used to implement clustering operations.

[0079] Taking the K-means algorithm as an example, K-means clustering can group data points (each data point represents a patient) into K clusters. In specific implementation, it can group patients with similar symptoms into the same group. For example, by analyzing the patient's symptom data, the patient can be divided into several different pathological clusters, each cluster representing a specific mental illness or pathological state.

[0080] S240: Apriori algorithm is used to perform association rule analysis on each pathological cluster and an information expression model is constructed based on the analysis results to obtain information expression models of pathological manifestations of multiple specified mental illnesses.

[0081] The Apriori algorithm is an association rule learning algorithm used to mine frequent itemsets and discover association rules from them. These rules can be expressed in the form of "if...then..." to describe patterns in data.

[0082] This embodiment uses the Apriori algorithm to implement the analysis of pathological manifestations of mental illness to find the correlation between different symptoms. Specifically, the Apriori algorithm can be applied to the symptom data in each pathological cluster to find the association rules between symptoms. The clustering results and the association rules are then combined to construct an information expression pattern corresponding to each pathological cluster. The information expression pattern corresponding to each pathological cluster can be used to determine the interaction between the pathological characteristics and symptoms of different mental illnesses.

[0083] In some embodiments, the operation of using the Apriori algorithm to perform association rule analysis on each pathology cluster and constructing an information expression pattern based on the analysis results includes:

[0084] For each pathology cluster, the multimodal features of each patient in the pathology cluster are converted into feature item sets;

[0085] Use the Apriori algorithm to scan the feature item sets of all patients in the pathology cluster, determine the frequent item sets whose corresponding support thresholds meet the preset conditions, and mine association rules that reach the preset confidence thresholds from the frequent item sets;

[0086] An information-based expression pattern corresponding to the pathology cluster is constructed based on the mined association rules. The information-based expression pattern corresponding to the pathology cluster includes common symptoms of patients in the cluster and association rules between symptoms.

[0087] In this embodiment, the multimodal features of each patient can be converted into an item set (i.e., a feature item set). For example, the multimodal features of a patient can be represented as {slow speech, reduced smile, increased heart rate}. Each item set has a corresponding support threshold, which is used to evaluate the reliability of the corresponding item set.

[0088] After completing the item set conversion operation, frequent item set mining needs to be performed. Specifically, the Apriori algorithm scans these item sets multiple times and finds frequent item sets that meet a certain support threshold. Then, in the frequent item sets, it mines association rules that meet the confidence threshold. For example: Rule: {slow speech, increased heart rate} → {anxiety}; meaning: patients with slow speech and increased heart rate are more likely to show anxiety.

[0089] In this embodiment, the information expression pattern refers to the association rules and frequent feature combinations constructed by the Apriori algorithm, which are used to describe the pathological manifestations of different mental illnesses. Its data structure can be as follows:

[0090] Combination of pathological manifestation characteristics: such as {frequent frowning, rapid breathing}.

[0091] Corresponding to the category of mental illness: such as depression or anxiety.

[0092] By way of example, the following are information expression patterns corresponding to several common mental illnesses.

[0093] (1) Depression:

[0094] Pattern: {slow speech, less smiling, lack of sleep} → depression

[0095] Explanation: Patients who display the above characteristics are more likely to suffer from depression.

[0096] (2) Anxiety disorder

[0097] Pattern: {increased breathing rate, increased heart rate, faster speech speed} → anxiety disorder

[0098] Explanation: People with anxiety disorders often experience an increase in breathing rate and heart rate.

[0099] (3) Bipolar disorder

[0100] Pattern: {Frequent mood swings, significant changes in speech speed} → Bipolar disorder

[0101] Explanation: People with bipolar disorder often fluctuate between periods of elation and depression.

[0102] In some embodiments, after determining the mental illness category of the target patient according to the target information expression pattern, the method also includes: generating a diagnosis report according to the target information expression pattern, the diagnosis report including the detected case manifestations and recommended treatment plans.

[0103] This embodiment can also provide targeted diagnostic reports for target patients, improve the degree of personalized diagnosis, and help patients receive better treatment.

[0104] In some embodiments, the method further includes: updating the parameters and information expression mode of the multimodal feature fusion model based on the doctor's feedback data and the new patient's multimodal data.

[0105] In this embodiment, the system will be regularly maintained and updated to ensure the accuracy and reliability of the multimodal feature fusion model and the information expression patterns of various mental illnesses.

[0106] It should be noted that, with respect to the various steps included in the method for identifying the category of mental illness provided in any of the above embodiments, unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of these steps may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0107] Based on the same inventive concept, the present application also provides a device for identifying the category of mental illness. Figure 4 As shown, the device includes the following modules:

[0108] A data acquisition module 110 is used to acquire multimodal data of a target patient;

[0109] The feature fusion module 120 is used to extract features from various types of modal data, obtain feature vectors of various types of modal data, and generate multi-modal fusion features based on the feature vectors of various types of modal data;

[0110] The pattern matching module 130 is used to match the multimodal fusion feature with pre-constructed information expression patterns of pathological manifestations of various mental illnesses to determine a target information expression pattern;

[0111] The classification module 140 is used to determine the mental illness category of the target patient based on the target information expression pattern.

[0112] In some embodiments, the feature fusion module 120 is specifically used to: input various types of modal data into the multimodal feature fusion model to obtain the multimodal fusion features output by the multimodal feature fusion model. The multimodal feature fusion model includes an input layer, multiple functional module layers for processing different modal data, and a feature fusion layer; the input layer is used to transmit various types of modal data to the corresponding functional module layers respectively; each functional module layer is used to extract feature vectors from the modal data input to this layer using the feature extraction network corresponding to this layer; the feature fusion layer is used to assign different weights to the feature vectors extracted by each functional module layer through a weighting mechanism or an attention mechanism, and generate multimodal fusion features based on the feature vectors extracted by each functional module layer and their corresponding weights.

[0113] In some embodiments, the device further includes a preprocessing module (not shown in the figure). The preprocessing module is used to perform information preprocessing on the original multimodal data of the target patient using a non-parametric Bayesian model to obtain the multimodal data of the target patient.

[0114] In some embodiments, the device further comprises an expression pattern building module (not shown in the figure).

[0115] Expression pattern building blocks, including:

[0116] A sample data acquisition submodule is used to acquire multimodal sample data of multiple patients;

[0117] A feature extraction submodule is used to extract features from the multimodal sample data of each patient to obtain the multimodal features of each patient;

[0118] A cluster analysis submodule, used for clustering based on the multimodal features of each patient, so as to divide the multiple patients into multiple pathological clusters;

[0119] The expression pattern construction submodule is used to perform association rule analysis on each pathological cluster using the Apriori algorithm and to construct an information-based expression pattern based on the analysis results, thereby obtaining information-based expression patterns of the pathological manifestations of various specified mental illnesses.

[0120] In some embodiments, the expression pattern construction submodule uses the Apriori algorithm to perform association rule analysis on each pathology cluster and constructs an information-based expression pattern based on the analysis results, including: for each pathology cluster, converting the multimodal features of each patient in the pathology cluster into a feature item set; using the Apriori algorithm to scan the feature item sets of all patients in the pathology cluster, determining the frequent item sets whose corresponding support thresholds meet preset conditions, and mining association rules that reach a preset confidence threshold from the frequent item sets; constructing an information-based expression pattern corresponding to the pathology cluster based on the mined association rules, and the information-based expression pattern corresponding to the pathology cluster includes the common symptoms of patients in the cluster and association rules between symptoms.

[0121] In some embodiments, the device further includes a report generation module (not shown in the figure). The report generation module is used to generate a diagnosis report according to the target information expression pattern after determining the mental illness category of the target patient according to the target information expression pattern, and the diagnosis report includes the detected case manifestations and recommended treatment plans.

[0122] In some embodiments, the device also includes an updating module (not shown in the figure), which is used to update the parameters and information expression mode of the multimodal feature fusion model based on the doctor's feedback data and new patient's multimodal data.

[0123] For the specific limitations of the mental illness category identification device, please refer to the limitations of the mental illness category identification method above, which will not be repeated here. The various modules in the above-mentioned mental illness category identification device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0124] The present application also provides a computer device. In some embodiments, the computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the clinical efficacy characteristic enhancement method provided in any of the above embodiments can be implemented.

[0125] Furthermore, in some embodiments, the internal structure diagram of the computer device may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as information expression patterns of pathological manifestations of various mental illnesses. The specific stored data can also be found in the definitions in the above method embodiments. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for identifying the category of mental illness is implemented.

[0126] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0127] The present application also provides a computer-readable storage medium. In some embodiments, a computer program is stored on the computer-readable storage medium. When the computer program is executed by a processor, the steps of the method for identifying the category of mental illness provided in any of the above embodiments are implemented.

[0128] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0129] Those skilled in the art can understand that all or part of the processes in the above method embodiments can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink), DRAM (SLDRAM), memory bus (Rambus), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0130] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0131] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for identifying mental illness categories, characterized in that: The method comprises: Acquire multimodal data of target patients; Extract features from various modal data to obtain feature vectors of various modal data, and generate multimodal fusion features based on the feature vectors of various modal data; Matching the multimodal fusion features with pre-constructed information expression patterns of pathological manifestations of various mental illnesses to determine a target information expression pattern; The mental illness category of the target patient is determined based on the target information expression pattern.

2. The method according to claim 1, characterized in that Feature extraction is performed on various modal data to obtain feature vectors of various modal data. Multimodal fusion features are generated based on the feature vectors of various modal data, including: Inputting various types of modal data into a multimodal feature fusion model to obtain multimodal fusion features output by the multimodal feature fusion model; The multimodal feature fusion model includes an input layer, multiple functional module layers for processing different modal data, and a feature fusion layer; the input layer is used to transmit each type of modal data to the corresponding functional module layer respectively; each functional module layer is used to extract feature vectors from the modal data input to the layer using the feature extraction network corresponding to the layer; the feature fusion layer is used to assign different weights to the feature vectors extracted by each functional module layer through a weighting mechanism or an attention mechanism, and generate multimodal fusion features based on the feature vectors extracted by each functional module layer and their corresponding weights.

3. The method according to claim 1 or 2, characterized in that The multimodal data of the target patient is obtained by preprocessing the original multimodal data of the target patient using a non-parametric Bayesian model.

4. The method according to claim 1 or 2, characterized in that: The process of constructing the information expression model of the pathological manifestations of the various mental illnesses includes: Obtain multimodal sample data from multiple patients; Perform feature extraction on the multimodal sample data of each patient to obtain the multimodal features of each patient; performing clustering based on the multimodal features of each patient to divide the multiple patients into multiple pathological clusters; The Apriori algorithm is used to perform association rule analysis on each of the pathological clusters and an information expression pattern is constructed based on the analysis results to obtain information expression patterns of the pathological manifestations of multiple specified mental illnesses.

5. The method according to claim 4, characterized in that The operation of using the Apriori algorithm to perform association rule analysis on each of the pathological clusters and constructing an information expression pattern based on the analysis results includes: For each of the pathology clusters, converting the multimodal features of each patient in the pathology cluster into a feature item set; Use the Apriori algorithm to scan the feature item sets of all patients in the pathology cluster, determine the frequent item sets whose corresponding support thresholds meet the preset conditions, and mine association rules that reach the preset confidence thresholds from the frequent item sets; An information-based expression pattern corresponding to the pathology cluster is constructed based on the mined association rules. The information-based expression pattern corresponding to the pathology cluster includes common symptoms of patients in the cluster and association rules between symptoms.

6. The method according to claim 1, characterized in that After determining the mental illness category of the target patient according to the target information expression pattern, the method further includes: A diagnosis report is generated according to the target information expression pattern, wherein the diagnosis report includes detected case manifestations and recommended treatment plans.

7. The method according to claim 2, characterized in that The method further comprises: The parameters of the multimodal feature fusion model and the information expression mode are updated based on the doctor's feedback data and the new patient's multimodal data.

8. A device for identifying the type of mental illness, characterized in that: The device comprises: A data acquisition module, used to acquire multimodal data of a target patient; The feature fusion module is used to extract features from various modal data, obtain feature vectors of various modal data, and generate multimodal fusion features based on the feature vectors of various modal data; A pattern matching module, used to match the multimodal fusion features with pre-constructed information expression patterns of pathological manifestations of various mental illnesses to determine a target information expression pattern; A classification module is used to determine the mental illness category of the target patient based on the target information expression pattern.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images