74 results about "Minority class" patented technology

The invention discloses an anti-fact generation method for processing class imbalance based on a real sample. The method comprises the following steps: preprocessing input data; performing causal feature selection by adopting a causal discovery algorithm; calculating causal feature tendency scores, and performing matching; carrying out anti-fact generation, forming a synthesized minority class set, and integrating the synthesized minority class set with original data to obtain an enhanced data set; and performing data cleaning on the enhanced data set to obtain a balanced data set. According to the method, a data set is effectively balanced by generating a high-quality and close-to-reality anti-fact sample, so that the performance of a downstream classifier on key indexes is remarkably improved; the feature values of the real instances are combined to ensure that the generated samples are located in a reasonable area of data distribution, so that the credibility and availability of the enhanced data are improved; the generated anti-fact sample is located in a boundary region between the majority class and the minority class, the decision region of the minority class is effectively expanded, and the unique post-cleaning avoids the influence of noise accumulation on model training.

The invention relates to a dynamic density guided generative adversarial network unbalanced data processing method (DAG-WGAN). The DAG-WGAN realizes unbalanced data processing through data preprocessing, dynamic density estimation and weight distribution, potential structure learning based on a variational auto-encoder (VAE), and density guide generation and dynamic feedback optimization based on WGAN-GP. The DAG-WGAN adaptively evaluates the sample density by using kernel density estimation (KDE) and a Gaussian kernel function, and allocates a weight for a generation process, thereby emphatically enhancing the low density and discriminating the sample generation of a difficult region. The VAE learns a potential manifold structure of a minority class of samples, realizes density-guided generation of a potential space under a WGAN-GP framework, and ensures diversity and manifold consistency of generated samples. In addition, a dynamic feedback mechanism is introduced, the weight and the gradient penalty coefficient are adaptively adjusted and generated, and the training stability and the sample generation robustness are improved.

The invention discloses a Tokamak plasma rupture prediction method based on cWGAN and TCN, which takes AUC as a measurement index to evaluate the effectiveness of a model, and adopts a conditional Wasserstein generative adversarial network (cWGAN)-based minority class sample generation method for solving the problem that the number of minority class samples is limited. Gradient penalty (GP) is introduced in a training stage to improve the quality of generated samples and training stability, and an inverse frequency mini-batch minority class equilibrium training strategy is adopted to enable a generator to fully learn distribution characteristics of broken samples, so that high-quality minority class samples are generated and a training set is expanded. A Transform structure is introduced on the basis of a time convolutional network (TCN), local time sequence features are efficiently extracted by using the TCN, and the Transform enhances the expression ability of a model for a long-time dependency relationship and global features, so that the prediction effect is improved.

The application discloses a network intrusion detection method based on LSTM and an attention mechanism, and comprises the following steps: 1) data preprocessing and target sample construction; 2) WGAN model establishment and new training sample merging; 3) network intrusion detection model establishment based on LSTM and the attention mechanism; 4) training in the obtained network intrusion detection model; and 5) model performance evaluation. The method can improve the detection accuracy and the detection rate of minority class samples.

The invention relates to the technical field of oil-immersed transformer fault diagnosis, in particular to an oil-immersed transformer fault diagnosis method and system based on feature enhancement and deep isomerism. The method comprises the steps that a minority class oversampling method is used for synthesizing minority fault sample data into new samples conforming to physical constraints; an enhanced input space is constructed based on deep features generated by an auto-encoder VAE, and deep features of fault gas in a new sample are fully extracted; based on the extracted deep features, using a heterogeneous integrated model to simulate a lightweight gradient elevator Light GBM to capture a shallow relationship of the deep features; using a convolutional neural network 1D-CNN to identify local features of the fault gas; establishing a global dependency relationship on the basis of a Transform model; the robustness of the model is remarkably enhanced through a dynamic weighting mode, and therefore it is guaranteed that reliable and stable diagnosis results are continuously output in various complex and high-uncertainty practical application scenes.

The application discloses a text sentiment analysis method based on class imbalance data. The method comprises the following steps: extracting semantic information of text by using a pre-trained classifier for a first data set, and then obtaining a sentiment classification boundary corresponding to each text, wherein the sentiment classification boundary is used to define a majority class and a minority class in the first data set; based on the classification boundary, generating pseudo samples of the minority class by using feature information in the majority class, and iteratively adjusting the classification boundary of the classifier by taking a set rejection function as an optimization target, wherein the rejection function is used to measure the quality of the generated pseudo samples; adding the generated pseudo samples to the first data set to form a second data set, and continuing to train the classifier by using the second data set. The application obtains hidden information of classification from data of the majority class, and performs oversampling on the minority class data according to the obtained classifier, so that a data class balancing strategy is realized.

The invention discloses an unbalanced classification method for electroencephalogram data in epilepsy detection, and relates to the technical field of data optimization and big data processing. According to the method, firstly, covariance matrixes of majority class samples and minority class samples are calculated, linear transformation is achieved through matrix decomposition, and the minority class samples inherit global distribution characteristics of the majority class samples; and then, in the transformed feature space, sorting samples based on mahalanobis distance and performing partition pairing, selecting sample pairs with large difference to generate convex combination synthesis samples, and ensuring sample diversity and boundary consistency. Experimental results show that on a CHB-MIT electroencephalogram data set, the method effectively solves the problems that a traditional oversampling technology is prone to expanding minority class decision boundaries and generated samples are lack of diversity, and the reliability of epileptic seizure detection is remarkably improved.

The invention discloses a user behavior analysis method and system.The method comprises the steps that on the basis of normal behavior data of a user, frequent item sets and rare item sets of normal behaviors are mined from mass unlabeled behavior data, and an accurate basis is provided for data classification; a rare item set is used as a standard to divide majority class and minority class data sets, and minority class data is anchored based on rare features, so that the processing accuracy is improved; performing N times of conversion on the majority class data set based on the rare item set, classifying the converted data into a minority class data set, dynamically balancing the two classes of data, and improving the sensitivity of the model to a minority class normal behavior mode; and constructing a user behavior graph and a graph neural network learning model based on the balanced training data, carrying out structured representation on a user complex behavior pattern, inputting graph neural network learning to obtain a graph structure, nodes and edge features of a normal behavior pattern, inputting actual user behavior data into the trained model, and outputting an accurate user behavior analysis result.

The application discloses a kind of class imbalance table data processing methods based on contrast constraint diffusion generation, belong to data processing technical field, it includes using binary classification table dataset to generate model training, obtain the generation model based on noise generation minority class's synthetic latent representation, generate model is randomly generated with several noise inputs, diffusion modeling branch is denoised to each noise and obtains the synthetic latent representation of minority class of each noise, using decoding module is decoded to each synthetic latent representation, obtains synthetic table sample;Data filtering is carried out to all synthetic table samples, then all synthetic table samples after filtering are merged with table dataset, and enhanced imbalance table data are obtained.The application introduces contrast learning branch in the process of generating minority class sample, effectively overcome the deficiency existing in class imbalance table data processing of existing oversampling method and general generation model.

An ensemble learning based method is for a binary classification on an imbalanced dataset. The imbalanced dataset has a minority class comprising positive samples and a majority class comprising negative samples. The method includes: generatively oversampling the imbalanced dataset by synthetically generating minority class examples, thereby generating a generated dataset; using the generated dataset to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers; and learning a weighted majority vote classifier by combining outputs of the base classifiers. Each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.

The invention discloses a credit data synthesis oversampling method and system based on noise-free loss distribution migration, and relates to the technical field of unbalanced classification in artificial intelligence and data mining, and the method comprises the steps: obtaining an original credit unbalanced data set; noise label sample filtering is carried out on the data set, the prediction probability of each sample in the data set is predicted through a model, and the noise-free loss value of each sample is calculated; dividing a plurality of loss intervals according to the value range of the noise-free loss value, distributing each sample to the corresponding loss interval, and determining noise-free loss distribution of majority-class and minority-class samples; migration is carried out based on noise-free loss distribution, synthetic sample distribution is determined, root samples and auxiliary samples are screened out, and minority class pseudo samples are synthesized through linear interpolation; and adding the minority class pseudo samples into the original credit unbalanced data set to obtain a class balanced credit data set. According to the optimized balanced data set, the recognition precision of majority-class samples and minority-class samples can be effectively improved.

Class imbalance in a training dataset may negatively impact the accuracy of a machine-learning model in classifying rare events that are underrepresented in the training dataset. Training datasets comprising time-series data present a unique challenge. Accordingly, resampling techniques for up-sampling and / or down-sampling a training dataset of time series are disclosed. The up-sampling may respect the temporal correlation of time samples in the time series, while generating synthetic time series that mimic the feature values of time series belonging to the minority class. Down-sampling may be used to fine-tune the ratio of time series belonging to the minority class to the time series belonging to the majority class.

The present disclosure relates to a credit evaluation model generation method, device, equipment and medium, wherein the method comprises: obtaining a majority class sample set, a minority class sample set and a reference credit evaluation model; using the reference credit evaluation model to perform credit prediction on each first sample and each second sample respectively, to determine and delete a first target sample from the majority class sample set according to the prediction result; clustering each second sample to obtain a plurality of class clusters, and generating at least one second target sample according to the plurality of class clusters, to add the second target sample to the minority class sample set; and generating a target credit evaluation model according to the updated majority class sample set and the updated minority class sample set. Thus, the overfitting of the target credit evaluation model can be avoided, and the generalization performance of the target credit evaluation model can be improved.

The invention discloses a self-adaptive oversampling method combining sample local density and position information, and belongs to the technical field of data balance. According to the method, minority class samples are divided into boundary minority class samples and safe minority class samples; then, aiming at the safe minority class samples, endowing the samples which are considered to be more important with higher weights so as to generate more new samples by utilizing the samples in a subsequent sample generation process; in the synthesis stage of the security sample, screening out neighbor samples meeting conditions, and generating a new sample near the security sample; finally, for the boundary minority class samples, adopting a sampling method combined with the majority class samples to synthesize samples for the boundary minority class samples; more emphasizes are put on important minority samples, adjacent samples are adaptively selected for the important minority samples, and the problem of determining where and how to generate new samples is solved.

The invention discloses a self-adaptive collaborative oversampling and unbalanced multi-classification method and system, and the method comprises the steps: selecting a minority class sample which is difficult to classify as a target sample for synthesis through the weight generated by AdaBoost before each iteration, and can avoid the synthesis of an invalid sample. In addition, a sample synthesis mode is changed, the phenomenon of uneven local interpolation is avoided, meanwhile, the interpolation position and range of the synthesized sample are adjusted in a self-adaptive mode along with the increase of iteration, and the diversity of minority class samples is increased. And finally integrating T classifier results so as to improve the multi-classification accuracy of the model for the unbalanced data set. Compared with the prior art, the method has the advantages of wide applicability, high classification accuracy of minority class boundary samples and the like.

This invention belongs to the field of data processing technology and relates to a synthetic minority class oversampling method for classifying multi-label imbalanced tabular data. The method can handle tabular data with mixed data types, and the designed two-stage weighted nearest neighbor algorithm is suitable for tabular data containing both discrete and continuous features. By introducing boundary samples and isolated samples, and employing a random selection followed by judgment approach, the requirements for computational cost and synthesis quality are balanced. This invention considers the impact of feature importance on the distance between calculated samples; the obtained nearest neighbor samples are close in distance to key features, which helps improve the selection quality of nearest neighbor samples, thereby improving the synthesis quality of new samples.

The invention belongs to the field of machine learning, and discloses a machine learning-oriented minority class sample enhancement method, system and device, and a storage medium, CTGAN accurately fits the joint distribution characteristics of minority class subject data through adversarial training, and guarantees the statistical rationality of generated samples; the large language model breaks through the limitation of traditional interpolation, explores a potential long-tail feature combination, and makes up the coverage blind area of distribution fitting. After the two are complementarily generated, the hard constraint module forcibly checks feature legality, type matching and cross-column logic consistency, and eliminates invalid samples; the unified scoring device anchors real distribution by using an original data set, retains high-confidence samples through threshold screening, and finally ensures the uniqueness of the samples through full-column duplicate removal. By adopting the method, the diversity and effectiveness of synthetic samples are remarkably improved, and the bottleneck of a single generation technology is broken through; dynamic balance of scale, quality and diversity is realized through a systematic quality control process, so that enhanced samples better meet downstream classification task requirements.

The application relates to a rotating machinery sample unbalanced fault diagnosis method based on a latent diffusion model, characterized by comprising the following steps: obtaining a vibration signal of a rotating machinery to form a sample set; constructing and training an attention variational autoencoder to perform feature learning on the sample set, obtain an encoder and a decoder, and map a time-frequency diagram sample to a latent space by using the encoder to obtain a latent feature representation; constructing a diffusion model in the latent space to learn the feature distribution of a minority class sample and generate a preset number of minority class synthetic samples; performing sample screening, including: performing principal component analysis on the real minority class samples to construct a principal component subspace, projecting the generated samples to the principal component subspace, and adopting Mahalanobis distance to perform distribution consistency measurement, and retaining the generated samples with Mahalanobis distance not exceeding a preset threshold; and constructing a class distribution balanced enhanced data set.

The present application relates to the technical field of oil-immersed transformer fault diagnosis, in particular to an oil-immersed transformer fault diagnosis method and system based on feature enhancement and deep heterogeneity. The method comprises the following steps: using a minority class oversampling method to synthesize new samples conforming to physical constraints from minority fault sample data; constructing an enhanced input space based on deep features generated by a self-encoder VAE to fully extract deep features of fault gases in the new samples; based on the extracted deep features, using a heterogeneous ensemble model to simulate a light gradient boosting machine LightGBM to capture shallow relationships of deep features; using a convolutional neural network 1D-CNN to identify local features of fault gases; establishing global dependency relationships based on a Transformer model; and significantly enhancing the robustness of the model through a dynamic weighting method, thereby ensuring that the model continuously outputs reliable and stable diagnosis results in various complex and highly uncertain actual application scenarios.

The invention discloses a teaching dialogue cognition level dynamic classification method and system based on time entropy, and belongs to the field of artificial intelligence education. In order to solve the problems that an existing method depends on static characteristics and ignores dynamic interaction and class imbalance, a new model with better performance on a known reference data set is provided. The scheme comprises the following steps: calculating teacher-student speaking time entropies to quantify interaction and divide teaching styles; semantic richness features are extracted; and constructing a dynamic weighted heterogeneous integrated model, fusing XGBoost and LightGBM, and dynamically adjusting the weight according to category distribution and minority class performance. Compared with a traditional method, the model achieves more accurate and robust classification performance on a reference data set, especially improves the recognition capability of minority classes, and provides an advanced modeling scheme for an intelligent teaching system. The abstract drawing is Figure 1.

The invention relates to a transformer operation state evaluation method and device, electronic equipment and a storage medium, and relates to the technical field of transformer fault diagnosis, and the method comprises the steps: carrying out the sample enhancement of a minority class fault sample set in a multi-dimensional feature system through employing an IMGOA algorithm, and constructing an enhanced sample set; performing visual evaluation on the enhanced sample set by adopting a principal component analysis method, and screening key feature subsets of which the cumulative contribution rate of the enhanced sample set which is qualified through evaluation reaches a preset threshold value based on a feature importance evaluation mechanism of a CatBoost classification model; and performing optimization on hyper-parameters of the CatBoost classification model by using a multi-strategy enhanced goose swarm optimization algorithm, and training the CatBoost classification model based on the key feature subset and the optimal hyper-parameter combination to construct a transformer operation state evaluation model. According to the invention, the diagnosis precision can be improved, the fault type can be judged more accurately, and the possibility of misjudgment or missed judgment is reduced.

The present invention relates to a mixed-type data sampling processing device and method. According to one embodiment of the present invention, the mixed-type data sampling processing device filters minority class sample data to be synthesized and generates, with respect to the minority class sample data to be synthesized, synthetic samples in consideration of both numerical variables and categorical variables, thereby enabling local oversampling to be implemented for mixed-type data.

Embodiments of the present application disclose a minority class sample processing method and device, equipment and a storage medium, and relate to the technical field of big data. In the embodiments, distance data between different original minority class samples in a minority class training sample set is determined; sampling weights of the original minority class samples are determined respectively according to the distance data; and synthetic minority class samples are generated respectively with the original minority class samples as root samples according to the sampling weights. The technical scheme of the embodiments of the present application introduces the sampling weights to generate the synthetic minority class samples, thereby increasing the overall number of the minority class samples and balancing the relative number of all original sample data. Meanwhile, the original minority class samples are screened according to the sampling weights, the minority class samples are synthesized under the premise that the sample distribution is unchanged, and the quality of the generated minority class samples is improved.

An ensemble learning based method is for a binary classification on an imbalanced dataset. The imbalanced dataset has a minority class comprising positive samples and a majority class comprising negative samples. The method includes: generatively oversampling the imbalanced dataset by synthetically generating minority class examples, thereby generating a generated dataset; using the generated dataset to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers; and learning a weighted majority vote classifier by combining outputs of the base classifiers. Each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.

The application discloses a landslide identification method and device based on a generative adversarial network data expansion strategy, and comprises the following steps: acquiring a multi-source data set and performing pretreatment, extracting landslide disaster factors, and constructing a landslide sample library after pretreatment and landslide label production are performed on the landslide disaster factors; after the pretreated landslide disaster factors and landslide labels are superimposed in layers, one-dimensional array data are produced through an image cube, and the one-dimensional array data are divided into a training set and a test set; landslide data in the training set are input into a network in batches for training, minority class sample expansion is completed, and the expanded samples are added to the original training set to form a balanced data set; the balanced data set is input into a back-end machine learning classification model for training, an optimal model generated by training is used to test the test set, and a landslide prediction result is obtained. The application solves the problem of imbalance between classes in the data set, and can make the landslide identification boundary more clear and complete, and improve the precision of landslide target identification.

The application discloses a multi-modal deep learning classification method and system for unbalanced data, relates to the fields of deep learning and reinforcement learning, and comprises the following steps: obtaining a sample set containing multi-modal data; for a minority class sample set, constructing a modal correlation learning model to extract multiple modal correlation features and form a feature set; constructing a reinforcement learning agent, taking classification accuracy as a reward signal, and dynamically selecting the weighted result of one or more optimal modal correlation features as a reference modal correlation feature through a reinforcement learning strategy. In the classification stage, the sample to be classified is input into the modal correlation learning model to obtain the modal correlation feature, and if the similarity with the reference modal correlation feature exceeds a preset threshold, the classification output probability weight is adjusted based on the similarity size, so that the recognition accuracy of the minority class sample is enhanced. The application can effectively identify the modal correlation features specific to the minority class, and significantly improves the classification accuracy and stability under the condition of unbalanced data.

The application discloses an uncertainty-guided fault-enhanced sampling method, system and device for improving minority class fault recognition rate. The method comprises the following steps: obtaining operation and maintenance samples of a target device; converting operation and maintenance sample data in the operation and maintenance samples into initial two-dimensional spectrum graphs one by one; establishing an association between all initial two-dimensional spectrum graphs and fault class labels according to fault class labels in operation and maintenance results of the operation and maintenance samples to obtain initial training samples; training a preset feature network according to the initial training samples to obtain a target feature network; determining minority class faults according to the fault class labels and the initial training samples; inputting all initial two-dimensional spectrum graphs into the target feature network to obtain target latent features; performing uncertainty-guided enhancement on feature components of the target latent features corresponding to the minority class faults to obtain minority class synthetic features; and inputting the minority class synthetic features into the target feature network for decoding to obtain enhanced samples.

An ensemble learning based method is for a binary classification on an imbalanced dataset. The imbalanced dataset has a minority class comprising positive samples and a majority class comprising negative samples. The method includes: generatively oversampling the imbalanced dataset by synthetically generating minority class examples, thereby generating a generated dataset; using the generated dataset to generate subsamples, and learning a base classifier on each of the subsamples to determine a plurality of base classifiers; and learning a weighted majority vote classifier by combining outputs of the base classifiers. Each of the base classifiers is assigned a weight in such a way that a diversity between the base classifiers on the positive samples is minimized.

The application relates to the technical field of Internet, and provides a precise marketing method, device and equipment based on user consumption data, which comprises the following steps: dividing a data set into a majority class data set and a minority class data set after normalizing the consumption data; sorting the Mahalanobis distance of each sample in the minority class data set and then dividing the sorted samples into several subsets to calculate the minimum Mahalanobis distance; taking the samples in different subsets as parent samples by using the MAHAKIL algorithm with a random parameter, iteratively generating offspring samples, and training a user group classifier by taking the offspring samples with a Mahalanobis distance greater than or equal to the minimum Mahalanobis distance from the parent samples as a new minority class data set to perform precise marketing on different user groups; and balancing the sample quantity relationship between the minority class data set and the majority class data set can make the classifier accurately identify the minority class users, avoid the bias towards a certain class of users caused by the quantity difference of the training data set, and improve the precision of marketing.