A balanced approach to enhance multimodal imbalanced data for mental illness

Abstract

The present invention discloses a method for balanced enhancement of multimodal unbalanced data of mental illness, which belongs to the technical field of deep learning for processing unbalanced data, and mainly oversamples multimodal time series unbalanced data of mental illness. First, the system performs multimodal preprocessing on mental illness data, combines adaptive balanced similarity and high-dimensional local outlier detection, captures complex boundary samples and assigns dynamic weights, and improves the extraction accuracy of boundary information in high-dimensional feature space by fusing multi-scale k-nearest neighbor and LOF (global outlier factor) algorithms. In the generation stage, the system constructs an evolutionary GAN model that integrates conditional generative adversarial networks (CGAN) and attention mechanisms to generate high-fidelity and diverse minority class samples. Through adaptive optimization and adversarial entropy regularization, the model convergence is accelerated and the diversity of generated samples is improved. Finally, through discriminant verification, the classification performance and data quality are significantly improved.

Description

Technical Field

[0001] The present invention relates to the technical field of deep learning data mining, and specifically to a method for balanced enhancement of multimodal mental illness imbalanced data. Background Art

[0002] In many practical applications of mental illnesses, such as depression diagnosis, anxiety detection, bipolar disorder monitoring, etc., the processing of time series data is not only crucial, but also a necessary data source for the auxiliary identification of mental disorders. These data record the changes in patients' symptoms at different time points and provide valuable information for diagnosis and treatment. However, these time series data often face the problem of imbalance, that is, the number of samples of certain symptoms or categories is much less than that of other categories. For example, in depression monitoring, there may be a large amount of daily normal emotional data, while the data of severe depressive episodes is relatively small. This imbalance will seriously affect the learning efficiency and generalization ability of the model, because the model tends to favor symptom categories with a larger number of samples, thereby ignoring fewer but perhaps equally important symptom categories. This will not only cause the model to perform poorly in identifying rare but important symptoms, but may also have serious consequences in clinical applications. Therefore, solving the imbalance problem of time series data is of great significance to improving the accuracy and reliability of mental illness detection models.

[0003] Current technologies for dealing with data category imbalance mainly include traditional oversampling methods and neural networks. Traditional methods: random oversampling and SMOTE, try to balance the data set by increasing the number of samples in the minority category, which can improve the performance of the model when dealing with imbalanced data. In recent years, neural networks, especially conditional generative adversarial networks (cGAN) and classification generative adversarial networks (CatGAN), have attracted attention for their ability to generate high-quality time series data samples. These methods can accurately generate new samples of specific categories by introducing conditional information, showing a new way to solve the problem of data imbalance.

[0004] Although the above methods have their own advantages, there are still many limitations in practical applications. Traditional oversampling techniques often ignore the correlation between adjacent variables when processing time series data, and may introduce noise when generating new samples, reducing the model's ability to recognize complex data patterns. Although GANs-related methods can generate high-quality data samples, in the application of time series oversampling, these methods do not fully utilize the boundaries between majority class samples and minority class samples, which is particularly important for improving subsequent classification performance. In addition, maintaining the logic and continuity of time series data is still a challenge when using GANs for oversampling. These problems limit the effectiveness and breadth of existing technologies in practical applications. Summary of the invention

[0005] In view of the deficiencies of the prior art, the present invention provides a method for balancing and enhancing unbalanced data of multimodal mental illnesses, which solves the problems raised in the above-mentioned background technology.

[0006] To achieve the above objectives, the present invention is implemented by the following technical scheme: a method for balancing and enhancing multimodal mental illness imbalance data, comprising the following steps:

[0007] S1. Perform multi-level and multi-modal preprocessing on the original mental illness dataset, integrate semantic analysis, sentiment mining and time series feature extraction techniques to generate an optimized dataset D; capture and analyze boundary samples in the dataset by introducing adaptive balanced similarity and high-dimensional space local outlier detection methods; assign dynamic weighting factors to the identified boundary samples, and adjust the boundary samples in real time through machine learning algorithms;

[0008] S2, adopt the fusion of multi-scale kNN and LOF algorithms to analyze each observation in the high-dimensional feature space; by introducing multi-scale analysis based on the kNN algorithm, the neighborhood structure changes of the observations at different scales are captured; at the same time, the LOF algorithm determines whether the sample is in the critical classification area from a global perspective;

[0009] S3, Generative Adversarial Network and Evolutionary Generative Adversarial Model of Generative Network Based on Attention Mechanism; The generator system accepts multi-source inputs, including Gaussian white noise vectors, extracted multimodal minority class boundary sample information and their category labels, where the minority class is set to 0 and the majority class is set to 1, and introduces the attention mechanism to dynamically select the optimal input features to generate minority class samples;

[0010] S4. Adopt an adaptive optimization strategy to dynamically adjust the learning rate, weight decay, and loss function structure of the generator and discriminator based on the adversarial training process.

[0011] Optionally, the preprocessing of the original mental illness dataset in step S1, the dataset includes time series data and corresponding text data;

[0012] The specific steps are as follows:

[0013] (1) Use wavelet transform to remove noise and irrelevant repeated information in time series data;

[0014] (2) Align the time series data and text data in the time dimension and connect them into a unified feature vector after feature extraction;

[0015] (3) Perform MinMaxScaler normalization on the feature vector;

[0016] (4) The target minority class is marked as 0 and the majority class is marked as 1.

[0017] Optionally, the adaptive balanced similarity boundary sample identification, local outlier detection and dynamic weight assignment in the data set D in step S1 are:

[0018] The dataset is ,in is the sample feature, is the corresponding category label; calculate the balanced similarity matrix S between samples; let Representation sample and The similarity between them is expressed as follows:

[0019]

[0020] in, is a hyperparameter to control the similarity;

[0021] Defining Adaptive Balanced Similarity , adjust the similarity by weighting the minority and majority class samples separately:

[0022] Among them, α and β are the weight adjustment factors of minority and majority class samples, respectively, which are set to the inverse of the number of class samples;

[0023] based on Matrix, using the k nearest neighbor method to identify the nearest neighbors of each sample; for a sample , calculate its k-nearest neighbor distance to the minority class and majority class samples. If the k-nearest neighbors of the sample contain both the minority class and the majority class, it is determined to be a boundary sample;

[0024] For each sample , calculate its k nearest neighbor set ,calculate The accessible density is , defined as the inverse of its k nearest neighbor distance:

[0025]

[0026] calculate The local outlier factor ; used to measure The degree of abnormality relative to its neighbors, if , it indicates For potential outliers:

[0027]

[0028] For each identified boundary sample , combining its balanced similarity and local outlier factor, and assigning a dynamic weight :

[0029] in, and A hyperparameter to adjust the balance between similarity and outlier factor.

[0030] Optionally, in step S2, each observation value is analyzed by fusing a multi-scale k-nearest neighbor and LOF algorithm;

[0031] The analysis process is as follows:

[0032] For the dataset , define multiple different scales , , , …, , where each represents a setting for the number of neighbors; for each scale , use the k nearest neighbor algorithm to search for its nearest neighbor sample set , and calculate the k nearest neighbor distance at each scale :

[0033]

[0034] Multi-scale importance evaluation, for each sample , calculate the total distance at different scales , defines the multi-scale importance score of the sample ;

[0035] For each sample ,Global anomaly factor calculation,The global anomaly factor is calculated to measure the degree of outlier of the sample in the global data distribution;

[0036] Calculate the global mean vector μ and covariance matrix Σ:

[0037]

[0038]

[0039] Calculation Sample The Mahalanobis distance is:

[0040]

[0041] Based on the Mahalanobis distance, calculate the sample The global anomaly factor is:

[0042]

[0043] in, is the median of the Mahalanobis distances of all samples;

[0044] like ,but is a potential global abnormal sample;

[0045] For each sample , combined with multi-scale importance scores and the global anomaly factor , define the boundary information index ; The dynamic weight is based on the boundary information index , used to construct a multimodal boundary information matrix , where each sample Corresponding to a matrix row:

[0046]

[0047] This matrix contains the boundary information of all samples, combined with the dynamic weights obtained in step S1 , based on the boundary information matrix and boundary information indicators , filter samples and select boundary information indicators As boundary samples, the selected boundary samples and their boundary information are input as the enhancement of minority class samples. Optionally, in step S3, the input and attention mechanism of the generator and the adaptive boundary information correction module of the discriminator are:

[0048] The generator input includes Gaussian white noise z, multimodal minority class boundary information B, and category label y, where the minority class is set to 0 and the majority class is set to 1;

[0049] Gaussian white noise z is distributed by standard normal distribution (0, 1) mid-sampling;

[0050] The multimodal minority class boundary information B comes from the boundary information matrix extracted in step S2 ;

[0051] The category label y is a binary vector; after fusing the above inputs, the input vector of the generator is ;

[0052] The attention mechanism is introduced into the generator model by learning the weight matrix , dynamically select the optimal input features, and the calculation formula is:

[0053]

[0054] in, is the attention weight of the i-th input feature, Represents the i-th element in the input vector, which is the final output sample after passing through the generator network ;

[0055] The discriminator will generate samples , category label y and boundary information B are concatenated into a vector and input into the neural network for processing; an adaptive boundary information correction module is introduced to adjust the influence of adaptive boundary information on classification in real time. represents the output of the discriminator, ,in is the weight matrix of the discriminator, The activation function is Sigmoid, and the discriminator outputs a scalar, which indicates the probability that the input sample is a true sample. At the same time, the generator is back-propagated according to the output value to update the parameters of the generator.

[0056] Optionally, the network model loss function of the generator is:

[0057]

[0058] The loss function of the discriminator is:

[0059]

[0060] Among them, pd, t represents the distribution of real data, and pG represents the distribution of data generated by the generator.

[0061] Optionally, the network models of the generator and the discriminator in step S3 are:

[0062] The network model of the generator:

[0063] (1) The initial input vector is input into a fully connected layer, which contains 128 neurons and uses the ReLU activation function;

[0064] (2) The output of the first layer is input to the second fully connected layer, which contains 256 neurons and uses the ReLU activation function;

[0065] (3) The output of the second layer is input to a third fully connected layer containing 512 neurons. The attention mechanism is introduced in this layer to learn the weight matrix The input features are weighted, and the activation function is ReLU;

[0066] (4) The output of the third layer is passed to the fourth fully connected layer, which contains 1024 neurons and uses the ReLU activation function;

[0067] (5) The output of the fourth layer is input to the output layer of the generator. The number of neurons in the output layer is consistent with the target output dimension, and an activation function is selected according to the target data type to generate the target sample.

[0068] The network model of the discriminator:

[0069] (1) The input vector is input to the first fully connected layer of the discriminator, which contains 1024 neurons and uses the LeakyRelu activation function;

[0070] (2) The output of the first layer is input to the second fully connected layer, which contains 512 neurons and uses the LeakyRelu activation function;

[0071] (3) The output of the second layer is passed to the third fully connected layer, which contains 256 neurons and uses the LeakyRelu activation function;

[0072] (4) The output of the third layer is input to the fourth fully connected layer, which contains 128 neurons and uses the LeakyRelu activation function;

[0073] (5) The output of the fourth layer is passed to the output layer of the discriminator, which contains a neuron for outputting the probability value of the input sample being a true sample.

[0074] Optionally, in step S4, dynamically adjusting weight decay and introducing entropy regularization are adopted;

[0075] Initialize the re-attenuation parameters ,in is the weight decay of the generator, The weight decay of the discriminator is adjusted according to the accuracy of the discriminator after each epoch. If the accuracy of the discriminator is above 92%, the weight decay is increased. ; Introduce adversarial entropy regularization and calculate the entropy value of the category probability distribution output by the discriminator for each generated sample:

[0076]

[0077] in is the predicted probability of the discriminator for category i, and C is the number of categories;

[0078] Adding the entropy regularization term to the generator's loss function, the updated generator loss function is:

[0079]

[0080] To control the weight parameter of entropy regularization, by adjusting , controls the influence of the regularization term on the generator.

[0081] The present invention provides a method for balancing and enhancing multimodal mental illness imbalance data, which has the following beneficial effects:

[0082] The system generates an optimized dataset D by performing multi-level and multi-modal preprocessing on the original mental illness dataset, integrating semantic analysis, sentiment mining and time series feature extraction techniques, and effectively improving the quality and information richness of the dataset. It uses a fusion of multi-scale k-nearest neighbor and LOF algorithms to deeply analyze samples in high-dimensional feature space, accurately identify and enhance boundary sample information, and significantly improve the detection ability of minority class samples. By constructing a fusion of conditional generative adversarial networks (CGAN) and an evolutionary GAN model based on an attention mechanism, the minority class samples generated by the system have high fidelity and diversity, effectively solving the problem of class imbalance. At the same time, adaptive optimization strategies and adversarial entropy regularization are introduced to ensure the rapid convergence and robustness of the generative adversarial model in complex data environments. Finally, through multiple discriminant verifications, the performance and data quality of the classifier when processing unbalanced datasets are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1 A flowchart of a method for balancing and enhancing imbalanced multimodal mental illness data provided in the invention;

[0084] Figure 2 A boundary information vector matrix calculation graph for the invention;

[0085] Figure 3 A specific network structure diagram for the invention;

[0086] Figure 4 It is a schematic diagram of the process of the invention. DETAILED DESCRIPTION

[0087] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0088] See also Figures 1 to 4 The present invention provides a technical solution: a method for balancing and enhancing multimodal mental illness imbalance data, including operations S1 to S4.

[0089] In this embodiment, the operation includes preprocessing the original mental illness data set in step S1, and the data set includes time series data (such as an electrocardiogram) and corresponding text data; first, wavelet transform is used to remove noise and irrelevant repeated information in the time series data; then, the time series data and text data are aligned in the time dimension and connected into a unified feature vector after feature extraction; then, the feature vector is normalized by MinMaxScaler to ensure the consistency of data of different modalities; finally, the target minority class is marked as 0 and the majority class is marked as 1, preparing for subsequent data processing and analysis.

[0090] In step S1, the boundary sample identification based on adaptive balanced similarity, local outlier detection and dynamic weight assignment in the data set D are:

[0091] The dataset is ,in is the sample feature, is the corresponding category label (0 represents the minority class, 1 represents the majority class); calculate the balanced similarity matrix S between samples, which is used to measure the similarity between samples while considering the balance of categories; set Representation sample and The similarity between them is as follows: is a hyperparameter that controls similarity):

[0092]

[0093] Defining Adaptive Balanced Similarity , adjust the similarity by weighting the minority and majority class samples separately:

[0094] Among them, α and β are the weight adjustment factors of minority and majority class samples, respectively, which are usually set to the inverse of the number of class samples;

[0095] based on The matrix uses the k nearest neighbor (k nearest neighbor) method to identify the nearest neighbors of each sample. , calculate its k-nearest neighbor distance to the minority class and majority class samples. If the k-nearest neighbors of the sample contain both the minority class and the majority class, it is determined to be a boundary sample;

[0096] For each sample , calculate its k nearest neighbor set ,calculate The accessible density is , defined as the inverse of its k nearest neighbor distance:

[0097]

[0098] calculate The local outlier factor ; used to measure The degree of abnormality relative to its neighbors, if , it indicates is a potential outlier, possibly located at the boundary of the data:

[0099]

[0100] For each identified boundary sample , combining its balanced similarity and local outlier factor, and assigning a dynamic weight : ;in, and These dynamic weighting factors will be used in the subsequent steps of generative adversarial model training to enhance the influence of minority class samples in the generation process.

[0101] Operation S2: The system performs in-depth analysis on each observation by integrating the multi-scale k-nearest neighbor and LOF algorithms. In the multi-scale k-nearest neighbor analysis, a multi-scale k-nearest neighbor search is first performed; for the data set , define multiple different scales , , , …, , where each represents a setting for the number of neighbors; for each scale Next, use the k nearest neighbor algorithm to search for its nearest neighbor sample set , and calculate the k nearest neighbor distance at each scale :

[0102]

[0103] Then a multi-scale importance evaluation is performed, for each sample , calculate the total distance at different scales , to reflect its importance at different scales; define the multi-scale importance score of the sample ;

[0104] Used to quantify the overall importance of samples at different scales;

[0105] The global anomaly factor is calculated for each sample , calculate its global anomaly factor to measure the degree of outlier of the sample in the global data distribution; first calculate the global mean vector μ and covariance matrix Σ:

[0106]

[0107]

[0108] Calculation Sample The Mahalanobis distance is:

[0109]

[0110] Based on the Mahalanobis distance, calculate the sample The global anomaly factor is:

[0111]

[0112] in, is the median of all sample Mahalanobis distances. If ,but is a potential global anomaly sample.

[0113] For each sample , combined with multi-scale importance scores and the global anomaly factor , define the boundary information index ;in and is a weight parameter used to balance the impact of multi-scale importance and global anomaly factors; The dynamic weight is based on the boundary information index , construct a multimodal boundary information matrix , where each sample Corresponding to a matrix row, it represents its boundary information vector:

[0114]

[0115] This matrix contains the boundary information of all samples, combined with the dynamic weights obtained in step S1 , which will be used as the input of the subsequent generative adversarial model; based on the boundary information matrix and boundary information indicators , filter the samples and select the boundary information index The samples with higher probability are used as boundary samples to strengthen the influence of these samples in the generative adversarial model. The screened boundary samples and their boundary information are used as the key input for minority class sample enhancement to ensure that the generated samples can accurately reflect the boundary characteristics.

[0116] Operation S3, the generator input and attention mechanism and the adaptive boundary information correction module of the discriminator are: The generator input includes three parts: Gaussian white noise z, multimodal minority class boundary information B, and category label y, where the minority class is set to 0 and the majority class is set to 1; specifically, Gaussian white noise z is usually obtained from a standard normal distribution (0, 1); the multimodal minority class boundary information B comes from the boundary information matrix extracted in step S2 ; The category label y is a binary vector that indicates the category of the sample. By fusing the above inputs together, the input vector of the generator is ;

[0117] The attention mechanism is introduced into the generator model by learning the weight matrix , dynamically select the optimal input features;

[0118] The calculation formula is:

[0119]

[0120] in, is the attention weight of the i-th input feature, Represents the i-th element in the input vector, which is the final output sample after passing through the generator network ;

[0121] In the discriminator, the discriminator will generate samples , the class label y and the boundary information B are taken as input, and after being concatenated into a vector, they are input into the neural network for processing; an adaptive boundary information correction module is introduced to adjust the influence of the adaptive boundary information on the classification in real time. represents the output of the discriminator, ,in is the weight matrix of the discriminator, is the activation function Sigmoid. The discriminator outputs a scalar, which indicates the probability that the input sample is a true sample. At the same time, the generator is back-propagated according to the output value to update the parameters of the generator. In each back-propagation, the discriminator adjusts the weight of the boundary information according to the current discrimination result.

[0122] The network model loss function of the generator is:

[0123]

[0124] The loss function of the discriminator is:

[0125]

[0126] Among them, pd, t represents the distribution of real data, and pG represents the distribution of data generated by the generator.

[0127] The network models of the generator and discriminator in step S3 are:

[0128] The network model of the generator:

[0129] (1) The initial input vector is input into a fully connected layer, which contains 128 neurons and uses the ReLU activation function;

[0130] (2) The output of the first layer is input to the second fully connected layer, which contains 256 neurons and uses the ReLU activation function;

[0131] (3) The output of the second layer is input to a third fully connected layer containing 512 neurons. The attention mechanism is introduced in this layer to learn the weight matrix The input features are weighted, and the activation function is ReLU;

[0132] (4) The output of the third layer is passed to the fourth fully connected layer, which contains 1024 neurons and uses the ReLU activation function;

[0133] (5) The output of the fourth layer is input to the output layer of the generator. The number of neurons in the output layer is consistent with the target output dimension, and a suitable activation function is selected according to the target data type to generate the target sample.

[0134] The network model of the discriminator:

[0135] (1) The input vector is input to the first fully connected layer of the discriminator, which contains 1024 neurons and uses the LeakyRelu activation function;

[0136] (2) The output of the first layer is input to the second fully connected layer, which contains 512 neurons and also uses the LeakyRelu activation function.

[0137] (3) The output of the second layer is passed to the third fully connected layer, which contains 256 neurons and continues to use the LeakyRelu activation function;

[0138] (4) The output of the third layer is input to the fourth fully connected layer, which contains 128 neurons and also uses the LeakyRelu activation function;

[0139] (5) The output of the fourth layer is passed to the output layer of the discriminator, which contains a neuron for outputting the probability value of the input sample being a true sample.

[0140] Operation S4 adopts dynamic adjustment of weight decay and introduces entropy regularization;

[0141] Initialize the re-attenuation parameters ,in is the weight decay of the generator, The weight decay of the discriminator is adjusted after each epoch according to the accuracy of the discriminator. If the accuracy of the discriminator remains above 92%, the weight decay is increased. ; Introduce adversarial entropy regularization and calculate the entropy value of the category probability distribution output by the discriminator for each generated sample:

[0142]

[0143] in is the predicted probability of the discriminator for category i, and C is the number of categories. The entropy regularization term is added to the loss function of the generator, and the updated generator loss function is:

[0144]

[0145] is the weight parameter that controls entropy regularization, by adjusting , controls the influence of the regularization term on the generator; increases It helps to improve the diversity of generated samples and reduce It helps to improve the accuracy of generated samples;

[0146] Train the generative adversarial network model: set the number of iterations to 60, and start training by taking boundary information, category labels, and Gaussian white noise as generator inputs; calculate the number of minority class samples required to be generated based on the difference between the number of minority class samples and the number of majority class samples in the data; use boundary information to generate suitable minority class samples in ABGAN, and then combine the synthesized minority class samples with the original data set to form a balanced data set.

[0147] The minority class samples and the original data set are combined to form a balanced data set.

[0148] In order to evaluate the effectiveness of the ABGAN method in dealing with imbalanced problems, 7 data sets were selected from the UEA public database, and the performance of the ABGAN method was compared with ROS, SMOTE, BL-SMOTE, ADASYN, INOS, CGAN, Cat GAN, and BFGAN, and then the nearest neighbor was used as the classifier; the multi-modal processing of the minority class was artificially synthesized.

[0149] The dataset is processed as follows:

[0150] 50words_3_4 means that the classes labeled 3 and 4 are selected to form the minority class, and the rest are the majority class. In the binary classification problem, the imbalance IR is obtained by dividing the number of majority classes by the number of minority classes.

[0151] The detailed information of the dataset is shown in Table 1:

[0152] Table 1 Data set details

[0153] Dataset Number of samples Number of minority classes / number of majority classes Imbalance rate Data Dimensions Aliases Electricity 45312 6970 / 38342 5.5 152 El Weather 45312 1370 / 16789 12.26 426 We Electrocardiogram 5000 1100 / 3900 3.5 96 E5000 Two-lead ECG 389 39 / 350 8.97 176 TE NATOPS 499 53 / 396 7.47 270 NA Racket Sports 599 80 / 479 5.99 131 Ra OSU Leaf_3_4 199 86 / 113 1.31 427 Osu

[0154] Table 2 Nearest neighbor classifier experimental results

[0155] Data set Baseline ROS SMOTE BL-SMOTE ADASYN INOS CGAN Cat GAN BFGAN ABGAN El 0.977 0.987 0.994 0.990 0.994 0.991 0.991 0.984 0.995 0.996 El 0.785 0.760 0.761 0.806 0.809 0.767 0.774 0.778 0.806 0.856 E5000 0.854 0.855 0.878 0.870 0.858 0.854 0.854 0.861 0.876 0.895 TE 0.802 0.899 0.908 0.914 0.935 0.933 0.925 0.931 0.913 0.913 NA 0.823 0.837 0.842 0.869 0.855 0.866 0.864 0.839 0.880 0.881 Ra 0.800 0.807 0.815 0.815 0.821 0.823 0.823 0.827 0.839 0.853 Osu 0.674 0.673 0.670 0.638 0.651 0.673 0.659 0.674 0.676 0.686 Mean 0.841 0.853 0.863 0.868 0.866 0.869 0.864 0.862 0.876 0.883 AverageRank 7.722 7.500 6.000 5.278 5.444 5.167 6.222 5.722 3.500 2.444

[0156] Note: The bold items are the optimal values

[0157] The experimental results show that the proposed method of balanced enhancement of multimodal mental illness imbalanced data can generate higher-quality minority class samples and more effectively improve the classifier performance. The specific experimental results are shown in Table 2. The higher the AUC value, the better the classifier performance.

[0158] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for balancing and enhancing multimodal mental illness imbalanced data, characterized in that: The following steps are involved: S1. Perform multi-level and multi-modal preprocessing on the original mental illness dataset, integrate semantic analysis, sentiment mining and time series feature extraction techniques to generate an optimized dataset D; capture and analyze boundary samples in the dataset by introducing adaptive balanced similarity and high-dimensional space local outlier detection methods; assign dynamic weighting factors to the identified boundary samples, and adjust the boundary samples in real time through machine learning algorithms; S2, adopt the fusion of multi-scale kNN and LOF algorithms to analyze each observation in the high-dimensional feature space; by introducing multi-scale analysis based on the kNN algorithm, the neighborhood structure changes of the observations at different scales are captured; at the same time, the LOF algorithm determines whether the sample is in the critical classification area from a global perspective; S3, Generative Adversarial Network and Evolutionary Generative Adversarial Model of Generative Network Based on Attention Mechanism; The generator system accepts multi-source inputs, including Gaussian white noise vectors, extracted multimodal minority class boundary sample information and their category labels, where the minority class is set to 0 and the majority class is set to 1, and introduces the attention mechanism to dynamically select the optimal input features to generate minority class samples; The input and attention mechanism of the generator and the adaptive boundary information correction module of the discriminator are: The generator input includes Gaussian white noise z, multimodal minority class boundary information B, and category label y, where the minority class is set to 0 and the majority class is set to 1; Gaussian white noise z is distributed by standard normal distribution (0, 1) mid-sampling; The multimodal minority class boundary information B comes from the boundary information matrix extracted in step S2 ; The category label y is a binary vector; after fusing the above inputs, the input vector of the generator is ; The attention mechanism is introduced into the generator model by learning the weight matrix , dynamically select the optimal input features, and the calculation formula is: in, is the attention weight of the i-th input feature, Represents the i-th element in the input vector, which is the final output sample after passing through the generator network ; The discriminator will generate samples , category label y and boundary information B are concatenated into a vector and input into the neural network for processing; an adaptive boundary information correction module is introduced to adjust the influence of adaptive boundary information on classification in real time. represents the output of the discriminator, ,in is the weight matrix of the discriminator, The activation function is Sigmoid. The discriminator outputs a scalar, which indicates the probability that the input sample is a true sample. At the same time, the generator is back-propagated according to the output value to update the parameters of the generator. The network models of the generator and discriminator are: The network model of the generator: (1) The initial input vector is input into a fully connected layer, which contains 128 neurons and uses the ReLU activation function; (2) The output of the first layer is input to the second fully connected layer, which contains 256 neurons and uses the ReLU activation function; (3) The output of the second layer is input to a third fully connected layer containing 512 neurons. The attention mechanism is introduced in this layer to learn the weight matrix The input features are weighted, and the activation function is ReLU; (4) The output of the third layer is passed to the fourth fully connected layer, which contains 1024 neurons and uses the ReLU activation function; (5) The output of the fourth layer is input to the output layer of the generator. The number of neurons in the output layer is consistent with the target output dimension, and an activation function is selected according to the target data type to generate the target sample. The network model of the discriminator: (1) The input vector is input to the first fully connected layer of the discriminator, which contains 1024 neurons and uses the LeakyRelu activation function; (2) The output of the first layer is input to the second fully connected layer, which contains 512 neurons and uses the LeakyRelu activation function; (3) The output of the second layer is passed to the third fully connected layer, which contains 256 neurons and uses the LeakyRelu activation function; (4) The output of the third layer is input to the fourth fully connected layer, which contains 128 neurons and uses the LeakyRelu activation function; (5) The output of the fourth layer is passed to the output layer of the discriminator, where the output layer includes a neuron for outputting the probability value of the input sample being a true sample; S4. Adopt an adaptive optimization strategy to dynamically adjust the learning rate, weight decay, and loss function structure of the generator and discriminator based on the adversarial training process.

2. The method for balancing and enhancing multimodal mental illness imbalanced data according to claim 1, characterized in that: Preprocessing of the original mental illness dataset in step S1, the dataset includes time series data and corresponding text data; The specific steps are as follows: (1) Use wavelet transform to remove noise and irrelevant repeated information in time series data; (2) Align the time series data and text data in the time dimension and connect them into a unified feature vector after feature extraction; (3) Perform MinMaxScaler normalization on the feature vector; (4) The target minority class is marked as 0 and the majority class is marked as 1.

3. The method for balancing and enhancing multimodal mental illness imbalanced data according to claim 1, characterized in that: In step S1, the boundary sample identification based on adaptive balanced similarity, local outlier detection and dynamic weight assignment in the data set D are: The dataset is ,in is the sample feature, is the corresponding category label; calculate the balanced similarity matrix S between samples; let Representation sample and The similarity between them is expressed as follows: in, is a hyperparameter to control the similarity; Defining Adaptive Balanced Similarity , adjust the similarity by weighting the minority and majority class samples separately: Among them, α and β are the weight adjustment factors of minority and majority class samples, respectively, which are set to the inverse of the number of class samples; based on Matrix, using the k nearest neighbor method to identify the nearest neighbors of each sample; for a sample , calculate its k-nearest neighbor distance to the minority class and majority class samples. If the k-nearest neighbors of the sample contain both the minority class and the majority class, it is determined to be a boundary sample; For each sample , calculate its k nearest neighbor set ,calculate The accessible density is , defined as the inverse of its k nearest neighbor distance: calculate The local outlier factor ; used to measure The degree of abnormality relative to its neighbors, if , it indicates For potential outliers: For each identified boundary sample , combining its balanced similarity and local outlier factor, and assigning a dynamic weight : in, and A hyperparameter to adjust the balance between similarity and outlier factor.

4. The method for balancing and enhancing multimodal mental illness imbalanced data according to claim 1, characterized in that: In step S2, each observation value is analyzed by fusing the multi-scale k-nearest neighbor and LOF algorithms; The analysis process is as follows: For the dataset , define multiple different scales , , , …, , where each represents a setting for the number of neighbors; for each scale , use the k nearest neighbor algorithm to search for its nearest neighbor sample set , and calculate the k nearest neighbor distance at each scale : Multi-scale importance evaluation, for each sample , calculate the total distance at different scales , defines the multi-scale importance score of the sample ; For each sample ,Global anomaly factor calculation,The global anomaly factor is calculated to measure the degree of outlier of the sample in the global data distribution; Calculate the global mean vector μ and covariance matrix Σ: Calculation Sample The Mahalanobis distance is: Based on the Mahalanobis distance, calculate the sample The global anomaly factor is: in, is the median of the Mahalanobis distances of all samples; like ,but is a potential global abnormal sample; For each sample , combined with multi-scale importance scores and the global anomaly factor , define the boundary information index ; The dynamic weight is based on the boundary information index , used to construct a multimodal boundary information matrix , where each sample Corresponding to a matrix row: This matrix contains the boundary information of all samples, combined with the dynamic weights obtained in step S1 , based on the boundary information matrix and boundary information indicators , filter samples and select boundary information indicators As boundary samples, the screened boundary samples and their boundary information are input as the enhancement of minority class samples.

5. The method for balancing and enhancing multimodal mental illness imbalanced data according to claim 1, characterized in that: The network model loss function of the generator is: The loss function of the discriminator is: Among them, pd, t represents the distribution of real data, and pG represents the distribution of data generated by the generator.

6. The method for balancing and enhancing multimodal mental illness imbalanced data according to claim 1, characterized in that: In step S4, dynamically adjusting weight decay and introducing entropy regularization are adopted; Initialize the re-attenuation parameters ,in is the weight decay of the generator, The weight decay of the discriminator is adjusted according to the accuracy of the discriminator after each epoch. If the accuracy of the discriminator is above 92%, the weight decay is increased. ; Introduce adversarial entropy regularization and calculate the entropy value of the category probability distribution output by the discriminator for each generated sample: in is the predicted probability of the discriminator for category i, and C is the number of categories; Adding the entropy regularization term to the generator's loss function, the updated generator loss function is: To control the weight parameter of entropy regularization, by adjusting , controls the influence of the regularization term on the generator.

Images