Semi-supervised multi-view depth representation learning method and system under class imbalance condition

Abstract

The invention belongs to the field of deep learning, and discloses a semi-supervised multi-view deep representation learning method and system under a class imbalance condition, and the method comprises the steps: taking each view data as a subset, independently learning a feature learning network for each subset, and converting the data of other views into missing views by using View Switch if the views are missing; a balance subset is constructed on the basis of View Switch; modeling information contained in each view into view shared information and view unique information, and fusing features learned by each subset by using similar loss and orthogonal loss to serve as final multi-view features. According to the method, deep learning classification is carried out on data with semi-supervised, multi-view and class imbalance characteristics at the same time, firstly, a semi-supervised multi-view learning algorithm based on View Switch is designed, and then a deep representation learning method based on View Switch and a balance subset is designed under the class imbalance condition. According to the invention, the research work of the deep representation learning theory and the application technology thereof is powerfully promoted.

Description

Technical Field

[0001] This invention belongs to, but is not limited to, the field of deep learning technology, and particularly relates to a semi-supervised multi-view deep representation learning method and system under imbalanced conditions. Background Technology

[0002] In many AI applications, data often exhibits characteristics such as semi-supervised analysis, multiple views, and class imbalance. For example, in computer-aided medical image analysis, medical experts need to identify lesions in medical images to create labeled data. Due to time and energy constraints, experts can often only label a small portion of the images. However, a large number of medical images collected and stored in hospitals can be used as unlabeled data, thus exhibiting semi-supervised characteristics. In computer vision applications, capturing faces from multiple angles yields images with various perspectives or poses. These observed images can also be described using descriptors such as color, shape, and texture, resulting in multiple-view characteristics for facial image data. In car insurance fraud detection, fraudulent customers accounted for only 7.4% of the total customers out of 108,000 samples, while the number of legitimate customers far exceeded that of fraudulent customers, demonstrating class imbalance.

[0003] Moreover, in many important industry application scenarios, the data often possesses two or even all three characteristics. For example, for petrochemical equipment fault signal data, the sample data corresponding to fault log records is limited (i.e., fault category labels exist), while most sample data lacks corresponding fault log records (i.e., no category labels). Each device can collect data from multiple measurement points, allowing for the construction of multi-view signal data to analyze whether the device is faulty. Petrochemical equipment fault signal data has multiple fault categories and normal categories, with each category exhibiting significant imbalance, with the normal category having the largest number of samples. Therefore, petrochemical equipment fault signal data typically possesses two or even all three characteristics of semi-supervised, multi-view, and class imbalance simultaneously, posing considerable difficulties and challenges to petrochemical equipment fault signal classification and diagnosis technology.

[0004] Deep learning combined with large-scale data is a key characteristic of current artificial intelligence technology development. Deep learning possesses strong data feature representation and perception capabilities, effectively learning complex, nonlinear, and abstract feature representations to achieve various tasks such as classification, clustering, prediction, and regression. Among these, deep representation learning is a fundamental task within deep learning, serving as a prerequisite for other learning tasks; the ability to learn deep representations significantly impacts the quality of subsequent task completion.

[0005] Designing effective deep representation learning methods to address the characteristics of semi-supervised, multi-view, and class-imbalanced data is of significant theoretical research value and practical application importance.

[0006] 1. Semi-supervised feature representation learning

[0007] Pseudo-labeling, a simple and effective semi-supervised learning method, was proposed in 2013. Its main idea is to first use a pre-trained model to give unlabeled data a pseudo-label, then transform the unsupervised data into a regularization term of the objective function based on the idea of ​​entropy regularization, and use cross-entropy to evaluate the error magnitude.

[0008] Encoder-based methods leverage the advantage of encoders, which are applicable to both labeled and unlabeled data, to extract data distribution information hidden in unlabeled data. Training strategy-based methods enhance the capabilities of semi-supervised learning through cleverly designed training strategies.

[0009] 2. Multi-view feature representation learning

[0010] Multi-view feature representation learning methods can generally be divided into three categories: methods based on collaborative learning, methods based on discriminative subspaces, and methods based on consistency and complementarity.

[0011] Collaborative learning-based methods utilize existing single-view classifiers to achieve feature learning and sample expansion for multi-view labeled data. Methods based on discriminative subspaces aim to learn a subspace from the multi-view data, where samples of the same class are close in distance, and samples of different classes are far apart. Methods based on consistency and complementarity aim to obtain better multi-view features by fully utilizing the consistency and complementarity information of the multi-view data.

[0012] 3. Class-Imbald Feature Representation Learning

[0013] Existing class imbalance feature representation learning methods mainly address the class imbalance problem from two aspects: data balancing and feature learning.

[0014] Data balancing: Data balancing methods aim to reduce the disparity between imbalanced classes at the data level, thereby mitigating the impact of class imbalance on classifiers. These methods typically use sampling to ensure that the number of samples for all classes in the dataset is consistent. Currently, mainstream data balancing techniques include undersampling, oversampling, and combined sampling.

[0015] Undersampling methods alleviate the problem of classifiers being biased towards the majority class by reducing the number of majority class samples.

[0016] Oversampling methods improve minority class classification performance by increasing the number of minority class samples. The classic SMOTE algorithm generates new, non-repeating minority class samples by randomly selecting nearest-neighbor samples of the same class for interpolation.

[0017] Combined sampling integrates undersampling and oversampling techniques to construct class-balanced datasets.

[0018] Feature learning methods aim to reduce the dissimilarity between imbalanced classes at the feature level, thereby mitigating the impact of class imbalance on the classifier. These methods can be categorized into cost-sensitive learning, ensemble learning, and deep learning.

[0019] Cost-sensitive learning transforms a classifier based on minimum error rate into a cost-sensitive classifier based on minimum cost by improving the internal structure of the classifier model.

[0020] Ensemble learning methods combine multiple weak classifiers to form a strong classifier.

[0021] Deep learning methods leverage their powerful feature learning capabilities to extract more effective classification features.

[0022] In summary, existing imbalanced methods have made progress in both data balancing and feature learning. However, these methods still have shortcomings: Regarding data balancing, while sampling-based methods can mitigate the impact of class imbalance to some extent, they lack theoretical guidance on the optimal number of samples. In feature learning, cost-sensitive methods struggle to set reasonable cost-sensitive parameters; ensemble learning methods fail to adequately utilize the differences among multiple classifiers; and deep learning methods still rely on traditional imbalance handling strategies, requiring further exploration.

[0023] 4. Fault Signal Classification

[0024] Fault signal classification methods can be divided into two categories: those based on control technology and those based on machine learning.

[0025] Fault signal classification based on control technology can be categorized into linear system fault signal classification methods and nonlinear system fault signal classification methods, depending on whether the fault signal is linearly separable. Linear system fault signal classification methods design linear analytical models for the faulty system, enabling real-time fault classification. These methods are mainly divided into observer methods and filter methods. Nonlinear system fault signal classification: Since fault signals are generally linearly inseparable in practical applications, nonlinear fault signal classification methods that can handle nonlinear fault signals are also commonly used.

[0026] Fault signal classification based on machine learning: Fault signal classification methods based on control technology are suitable for applications with relatively simple system models that can be accurately modeled. However, in real-world fault classification problems, there are still many systems for which accurate models cannot be established. Fault signal classification methods based on machine learning can handle fault classification problems where accurate models cannot be established. These methods mainly include signal statistical analysis methods, classifier design methods, and deep learning methods.

[0027] Signal statistical analysis methods utilize statistical information from fault signals for classification. Commonly used statistical analysis methods include principal component analysis, canonical correlation analysis, and multivariate analysis. Classifier design methods typically involve preprocessing the fault signals using feature extraction methods, followed by classifying the features using a classifier. Deep learning methods: Deep neural networks (DNNs) are a very promising tool for feature representation and classification decisions.

[0028] Based on the above analysis, the urgent technical problems that need to be solved in the existing technology are:

[0029] When data in real-world applications simultaneously exhibits the characteristics of semi-supervised learning, multiple views, and class imbalance, how to perform deep representation becomes a significant research question. It is necessary to consider how to conduct deep learning on semi-supervised multi-view data and perform effective balancing processing. Summary of the Invention

[0030] To address the problems existing in the prior art, this invention provides a semi-supervised multi-view deep representation learning method and system for imbalanced conditions.

[0031] This invention is implemented as follows: a semi-supervised multi-view depth representation learning method under class imbalance, characterized in that the method specifically includes:

[0032] S1: Treat each view data as a subset, and learn a feature learning network for each subset separately. If there are missing views, use View Switch to convert the data of other views into the missing views.

[0033] S2: Construct a balanced subset based on View Switch;

[0034] S3: Model the information contained in each view as shared information and view-specific information, and use similarity loss and orthogonal loss to fuse the features learned by each subset as the final multi-view features.

[0035] Furthermore, the View Switch network in this invention designs a View Switch network for all views to complete the mutual conversion between views. All views are labeled, and the training method borrows from StarGAN, utilizing view data and view label data for training. During View Switch training, a view feature association dictionary model is performed based on a multi-view attention mechanism. First, the features in the view are divided, and these features are treated as different words in different languages. Then, the multi-view attention mechanism is used to learn the association relationships between these features, forming an association dictionary, and storing the association weights between features in this dictionary. During the view conversion process, the association weights in the association dictionary are used to calculate the view conversion. Simultaneously, labeled samples are introduced into the View Switch training, and specific category views are generated by controlling the sample labels. By combining view labels and sample labels, View Switch can perform conversions between specific categories of specific views.

[0036] Furthermore, the construction of balanced subsets based on View Switch involves using View Switch to transform data from other views into minority class samples in the current view, thereby enriching the number of minority class samples in the current subset. For severely imbalanced data, View Switch can use other similar data in the current subset mixed with random noise to generate more minority class samples and finally construct a balanced subset.

[0037] Furthermore, the shared information is information shared by multiple views, and the unique information is information belonging only to the current view. The model first models the information contained in each view as view-shared information and view-unique information. Then, it uses similarity loss to semantically align the shared information obtained in each view. At the same time, orthogonal loss is introduced to reduce redundancy.

[0038] Another objective of this invention is to provide a semi-supervised multi-view deep representation learning system for imbalanced scenarios, the system specifically comprising:

[0039] The multi-view balanced subset construction module is used to transform data from other views into minority class sample data in the current view using View Switch;

[0040] The view completion module is used to convert data from other views into missing views when using View Switch;

[0041] The view encoding module is used to learn features for each view data.

[0042] The multi-view depth representation learning module is used for shared-unique multi-view representation learning.

[0043] Based on the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solution to be protected by this invention are as follows:

[0044] First, most existing methods only address a single problem. For example, semi-supervised learning only utilizes unlabeled samples to improve model performance, multi-view learning focuses on modeling complementary features between views, and class imbalance learning mainly relies on resampling or cost-sensitive strategies. This invention introduces a View Switch mechanism to achieve cross-view sample conversion and completion, enabling views that originally lack minority class samples to receive balanced enhancement, thus solving the problem that traditional methods cannot simultaneously take into account multi-dimensional characteristics.

[0045] In imbalanced scenarios, minority class samples are often scarce or even missing across different views, causing the model to favor the majority class. This invention addresses this by using View Switch to convert data from other views into minority class samples for the current view, thereby generating a balanced subset. This mechanism increases the availability of minority class samples without altering the original data distribution, achieving relative balance in inter-class distribution and thus improving the fairness and discriminative performance of classification tasks.

[0046] In practical applications, due to differences in acquisition equipment, data loss, or cost limitations, data from different views is often incomplete. This invention utilizes a view switch cross-view completion mechanism to reconstruct sample representations in missing views, thereby ensuring the integrity of the multi-view representation learning process. This not only enhances the robustness of the model but also improves data utilization efficiency, enabling the system to maintain good learning performance even in real-world environments with incomplete data.

[0047] By encoding each view and extracting representative feature vectors, and then integrating them within a shared-unique multi-view representation learning framework, the model can learn common information across views while retaining the uniqueness of each view. This representation method significantly improves the expressive power of multi-view data, providing a more accurate and stable feature foundation for subsequent classification and retrieval tasks.

[0048] Unlike existing methods that simply address imbalance through oversampling and undersampling, this invention deeply integrates semi-supervised learning with a multi-view transformation mechanism, maintaining discriminative ability even with a small number of labeled samples and a large number of unlabeled samples. Experiments demonstrate that this method effectively reduces interference from the majority class and improves the recognition accuracy of minority class samples, exhibiting significant technical advantages.

[0049] At the theoretical level, the multi-view balanced subset construction and sharing-unique deep representation framework proposed in this invention provides a new paradigm for solving the semi-supervised imbalanced learning problem and promotes the improvement of the deep representation learning method system. At the application level, this method can adapt to multi-source data scenarios such as remote sensing image classification, medical image diagnosis, and cross-modal retrieval, and helps to solve long-standing bottleneck problems such as data imbalance and view missingness, and has broad application prospects and promotional value.

[0050] Second, the "semi-supervised multi-view deep representation learning method under class imbalance" proposed in this invention can significantly improve data utilization and classification accuracy in practical applications such as petrochemical equipment fault diagnosis. Since the method simultaneously solves three complex data problems—semi-supervised, multi-view, and class imbalance—its commercial value lies in:

[0051] 1. Reduce labor costs: Make full use of a large amount of unlabeled data to reduce reliance on expert manual annotation.

[0052] 2. Enhance product performance: Improve the accuracy and robustness of fault diagnosis, fraud detection, and medical auxiliary diagnosis.

[0053] 3. Enhance industry competitiveness: Develop a universal algorithm framework applicable to multiple fields, facilitating integration with existing artificial intelligence platforms and big data systems.

[0054] Currently, there are separate studies in the fields of semi-supervised learning, multi-view learning, and class-imbalanced learning both domestically and internationally, but a unified deep representation learning framework capable of simultaneously solving all three types of problems is lacking. This invention, through an innovative scheme of "View Switch network + balanced subset construction + shared and unique information modeling," achieves the fusion processing of various complex data characteristics and proposes a deep learning method adaptable to the triple scenarios of semi-supervised, multi-view, and class-imbalanced learning. This technology performs deep learning classification on data simultaneously exhibiting semi-supervised, multi-view, and class-imbalanced characteristics, proposing a semi-supervised multi-view imbalanced deep representation learning method based on View Switch and balanced subsets.

[0055] In practical applications, data often simultaneously presents three challenges: "limited sample labeling, diverse perspectives / feature sources, and extreme class imbalance." Traditional methods typically address only one or two of these issues, failing to address all three while maintaining classification performance. The solution presented in this invention addresses these challenges through:

[0056] 1. The View Switch network enables the switching between views and compensates for missing data;

[0057] 2. Multi-view attention mechanism + feature dictionary modeling enhances the ability to model the correlation of information from different perspectives;

[0058] 3. The balanced subset construction method effectively alleviates the class imbalance problem;

[0059] 4. Shared-unique representation networks utilize both consistency and complementarity features to reduce redundancy.

[0060] This successfully solved the long-standing problem of "deep representation learning under triple complex data characteristics".

[0061] Traditional academia harbors a "technical bias," believing that semi-supervised learning, multi-view learning, and class-imbalanced learning must be studied independently and cannot be addressed within a unified framework. This invention breaks through this inherent mindset, proposing a deep integration of generative models (view switching), balanced sampling, and feature representation learning. It demonstrates that these three types of problems can not only be modeled simultaneously but also collaboratively optimized within the same framework. This method effectively eliminates over-reliance on "single perspectives or single strategies," pioneering a new paradigm for cross-task fusion learning. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of a semi-supervised multi-view learning algorithm based on View Switch provided in an embodiment of the present invention;

[0063] Figure 2 This is a schematic diagram of a semi-supervised multi-view imbalanced deep representation learning method based on View Switch and balanced subsets provided in an embodiment of the present invention;

[0064] Figure 3 This is a schematic diagram of a multi-view deep representation learning network based on shared and unique information provided in an embodiment of the present invention;

[0065] Figure 4 This is a module diagram of a semi-supervised multi-view deep representation learning system under class imbalance conditions provided in an embodiment of the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0067] This invention provides a semi-supervised multi-view deep representation learning method for imbalanced scenarios, the method specifically including:

[0068] This paper presents a semi-supervised multi-view learning algorithm based on View Switch. First, it designs the View Switch network architecture and its unsupervised training method. Second, to achieve better view switching performance, it models view feature association dictionaries based on a multi-view attention mechanism to enhance the ability to model relationships between views. Finally, it proposes a category-specific view switching method based on labeled data to enable more fine-grained view switching. Labeled samples are introduced into the training of View Switch, and views of specific categories are generated by controlling the sample labels.

[0069] This paper constructs a semi-supervised multi-view imbalance deep representation learning model based on View Switch and balanced subsets, designs a multi-set-based semi-supervised multi-view imbalance feature learning framework to effectively utilize the multi-set characteristics of multi-view data, constructs balanced subsets based on the View Switch multi-view balanced subset construction method, and proposes a method to solve the view missing problem in the case of semi-pairing.

[0070] A multi-view deep representation learning network based on shared and unique information is constructed. First, the information contained in each view is modeled as shared information and unique information. Then, a similarity loss is used to semantically align the shared information obtained in each view. Simultaneously, to address the redundancy issue in multi-view data, an orthogonal loss is introduced to reduce redundancy. With the help of similarity and orthogonal constraints, this framework can effectively utilize the consistency and complementarity of multi-view data while effectively reducing redundancy.

[0071] like Figure 1 As shown, the consistency of multi-view data manifests as a strong correlation between different views of the same multi-view sample. A View Switch is designed using pairwise information and correlations between views to achieve mutual conversion between views. Inspired by StarGAN, this invention designs a View Switch network for all views to complete the mutual conversion between views. Since the training of View Switch only requires view labels and not sample labels, this network can be trained unsupervised. Specifically, first, view labels are assigned to all views, and then the training can be performed using view data and view label data, similar to the training method of StarGAN.

[0072] Learning the relationships between features of different views is crucial for view transitions and is therefore key to ViewSwitch training. To better model these relationships, a view feature association dictionary modeling method based on a multi-view attention mechanism is proposed. In machine translation, attention mechanisms can learn the relationships between words in different languages. Inspired by this, this invention first divides the features in the view and treats these features as different words in different languages. Then, a multi-view attention mechanism is used to learn the relationships between these features, forming an association dictionary, and storing the association weights between features in this dictionary. During view transitions, the association weights in the association dictionary are used in the view transition calculation. This feature association dictionary learning effectively models the relationships between features of multiple views, thereby improving the view transition performance of the ViewSwitch.

[0073] The View Switch model described above is unsupervised training, thus it cannot perform category-specific view transitions. To achieve category-specific view transitions, labeled data is introduced into this View Switch network. In CatGAN, because sample labels are introduced into the generator, the category of the generated samples can be controlled by manipulating the sample labels. Inspired by CatGAN, this invention introduces labeled samples into the training of View Switch, generating category-specific views by controlling the sample labels. By combining view labels and sample labels, View Switch can perform category-specific view transitions.

[0074] In practical applications, the data we face is often semi-supervised multi-view data, and this data is frequently imbalanced. Based on the proposed View Switch network, a semi-supervised multi-view feature learning method applicable to imbalanced scenarios is proposed. The technical approach of this method is as follows: Figure 2 As shown.

[0075] Semi-supervised multi-view deep representation learning based on multiple balanced subsets: Constructing multiple balanced subsets is an effective method for handling imbalanced problems, and multi-view data naturally possesses multi-set properties. This invention first treats each view data as a subset, with each subset learning a separate feature learning network. Then, the features learned from each subset are fused and used as the final multi-view features. The fusion method employs... Figure 3 The shared-unique multi-view representation learning model is shown in the image.

[0076] A View Switch-Based Method for Constructing Balanced Subsets: Multi-set methods, used to solve imbalanced classification problems, require that samples from different classes within each subset be balanced. However, the subsets constructed in the above method are often class-imbalanced. This invention utilizes View Switch to construct balanced subsets from multiple views. Specifically, for a subset formed by a particular view, View Switch can be used to transform data from other views into minority class samples within the current view, thereby increasing the number of minority class samples in the subset. Furthermore, for severely imbalanced data, View Switch can use other similar data within the subset mixed with random noise to generate more minority class samples, ultimately constructing a balanced subset.

[0077] Strategies for handling partial pairings or missing views: In general, we assume that all views in multi-view data are complete. However, in actual data acquisition, missing views (partial pairings) are very common. To complete all views of the same training sample, this invention uses a View Switch to convert the data of other views into missing views, thereby resolving the issue of missing views.

[0078] Consistency and complementarity are two fundamental characteristics of multi-view data. In multi-view data, consistency and complementarity manifest as follows: each view contains information shared by multiple views simultaneously, while also containing information unique to that single view. Based on this characteristic of multi-view data, the information contained in each view is modeled as shared information (information shared by multiple views) and unique information (information belonging only to the current view). First, a multi-view deep representation learning network based on consistency and complementarity information is constructed, i.e., a multi-view deep representation learning network based on shared and unique information, such as... Figure 3 As shown.

[0079] like Figure 4 As shown in the figure, an embodiment of the present invention provides a semi-supervised multi-view deep representation learning system for imbalanced conditions, specifically comprising:

[0080] The multi-view balanced subset construction module is used to transform data from other views into minority class sample data in the current view using View Switch;

[0081] The view completion module is used to convert data from other views into missing views when using View Switch;

[0082] The view encoding module is used to learn features for each view data.

[0083] The multi-view depth representation learning module is used for shared-unique multi-view representation learning.

[0084] The semi-supervised multi-view deep representation learning system for class imbalance proposed in this invention is based on the core principle of achieving balancing, completion, and representation of multi-source heterogeneous data through multi-module collaboration. First, when faced with uneven data class distribution, the multi-view balanced subset construction module utilizes View Switch technology to map or convert sample data from other views into minority class samples in the target view, thereby dynamically expanding the number of samples in the scarce class and forming a balanced subset. This process is essentially a cross-view sample generation and transfer learning, ensuring the rationality of data augmentation by maintaining semantic consistency, thus mitigating the training bias problem caused by class imbalance.

[0085] Secondly, the view completion module specifically addresses the "partial pairing" problem that frequently occurs in the practical acquisition and application of multi-view data. Through the cross-view feature transformation capability of View Switch, existing view features are converted into fitted representations of missing views, thus completing the virtual samples. This mechanism ensures the credibility of the completed data by aligning semantics and maintaining distribution consistency in the feature space. This allows the model to still receive complete multi-view inputs when facing missing modalities, thereby avoiding performance degradation caused by incomplete information and enhancing the model's robustness and universality.

[0086] Finally, the view encoding module and the multi-view deep representation learning module form an overall information representation framework. The view encoding module learns features for each independent view, extracting local representations while preserving their structural characteristics. Subsequently, the multi-view deep representation learning module models shared-unique information, maintaining consistency of shared features across views while retaining the unique differences between each view, and reducing redundancy through similarity and orthogonality constraints. Ultimately, the model obtains a deep representation that reflects both the common semantics of multiple views and preserves individual differences, enabling the overall system to perform efficient, stable, and discriminative learning and inference even under class imbalance and missing view conditions.

[0087] This invention has a wide range of applications, spanning multiple fields. It can be used not only for accurate analysis and assisted diagnosis of multimodal patient data in healthcare scenarios, but also for multi-source sensor data monitoring and fault prediction in industrial intelligence and equipment maintenance. Furthermore, it has significant potential applications in computer vision and security, such as cross-camera identity recognition and abnormal behavior detection. This multi-view, multimodal deep representation learning method provides a unified and efficient data modeling approach for different industries, broadening the practical application pathways of artificial intelligence.

[0088] In experiments, the feature dictionary based on the multi-view attention mechanism established in this invention can intuitively present the feature correlations between different perspectives, giving the model interpretability. By introducing similarity constraints and orthogonality constraints in the modeling process, the decoupling of shared information and unique information is achieved, effectively reducing feature redundancy and making the final feature representation more sparse and discriminative. This provides sufficient evidence for the scientific validity of the method from both theoretical and empirical perspectives.

[0089] In semi-paired scenarios where viewpoint data is missing, the View Switch module designed in this invention can automatically fill in the missing viewpoints, thereby avoiding a significant performance drop due to incomplete data. This mechanism ensures that the model can maintain stable recognition and inference performance even when there are objective defects in data acquisition or limitations in application deployment, verifying its practicality in real-world applications.

[0090] Further experiments demonstrate that the present invention exhibits strong stability and robustness in the face of data noise and class imbalance. Compared with single-view or traditional feature extraction methods, this method maintains high accuracy while significantly reducing fluctuations, meaning that it still possesses reliability and anti-interference capabilities in complex, dynamic, and imperfect data environments.

[0091] This invention not only theoretically proposes a new mechanism for interpretability and information decoupling in multi-view deep representation learning, but also demonstrates advantages such as enhanced robustness, strong missing completion ability, and high feature discriminativeness in experimental verification, laying a solid foundation for its application in multiple fields such as medicine, industry, and computer vision security.

[0092] It should be noted that embodiments of the present invention can be implemented in hardware, software, or a combination of both. The hardware portion can be implemented using dedicated logic; the software portion can be stored in memory and executed by a suitable instruction execution system, such as a microprocessor or dedicated-design hardware. Those skilled in the art will understand that the above-described devices and methods can be implemented using computer-executable instructions and / or included in processor control code, for example, such code provided on a carrier medium such as a disk, CD, or DVD-ROM, a programmable memory such as read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The devices and modules of the present invention can be implemented by hardware circuitry such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, or programmable hardware devices such as field-programmable gate arrays, programmable logic devices, etc., or by software executed by various types of processors, or by a combination of the above-described hardware circuitry and software, such as firmware.

[0093] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications, equivalent substitutions, and improvements made by those skilled in the art within the scope of the technology disclosed in the present invention, and within the spirit and principles of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A semi-supervised multi-view deep representation learning method for imbalanced learning, characterized in that, include: S1: Treat each view data as a subset, train a feature learning network for each subset, and when a view is missing, use View Switch to convert the data of other views into the data of the missing view; S2, construct a balanced subset based on the View Switch to increase the number of minority class samples; S3 models the information in each view as shared information and unique information, uses similarity loss to semantically align the shared information, and uses orthogonal loss to reduce feature redundancy, thus obtaining the final multi-view features.

2. The method as described in claim 1, characterized in that, The View Switch is trained by introducing view labels, and a feature association dictionary is established using a multi-view attention mechanism to model the relationship between the features of each view. During the view transition process, the association weights in the dictionary are used to realize feature transformation.

3. The method as described in claim 1, characterized in that, The steps for constructing a balanced subset based on View Switch include: Convert samples from other views into minority class sample data in the target view via View Switch; For severely imbalanced data, new minority class samples are generated by combining similar samples with random noise, thereby constructing a balanced subset.

4. The method as described in claim 1, characterized in that, The shared information is information contained in multiple views simultaneously, and the unique information is information belonging only to the current view; similarity loss is used to semantically align the shared information of each view, and orthogonal loss is used to constrain the unique information to be orthogonal to the shared information.

5. A semi-supervised multi-view deep representation learning system for imbalanced learning, characterized in that, include: The multi-view balanced subset construction module is used to convert other view data into minority class sample data of the target view and construct a balanced subset using View Switch; The view completion module is used to convert data from other views into data from the missing view using the View Switch; The view encoding module is used to learn feature representations for each view data; The multi-view representation learning module is used to jointly model the shared and unique information of each view, and to perform feature fusion using similarity loss and orthogonal loss.

6. The system as described in claim 5, characterized in that, The multi-view balanced subset construction module further includes: The minority class sample augmentation unit is used to generate new minority class sample data by mixing similar samples with random noise in cases of severe class imbalance.

7. The system as described in claim 5, characterized in that, The view completion module establishes a feature association dictionary based on a multi-view attention mechanism, and uses the weight parameters in the dictionary to perform feature transformation between views in order to complete the missing view completion.

8. An apparatus for semi-supervised multi-view deep representation learning in class-imbalanced cases, characterized in that, include: processor; Memory is used to store executable instructions for a computer; When the instructions are executed by the processor, the device performs the method as described in any one of claims 1 to 4.

9. The apparatus as claimed in claim 8, characterized in that, The device further includes a graphics processing unit for accelerating multi-view feature transformation and balanced subset construction of the View Switch network during the training phase.

10. A computer-readable storage medium having a computer program stored thereon that, when executed by a processor, performs the method as described in any one of claims 1 to 4.

Images