Prototype learning based non-identically distributed multi-modal data mining method and system

Abstract

This invention discloses a non-independent, identically distributed multimodal data mining method and system based on prototype learning. First, missing modalities are handled through multimodal adaptation and masking mechanisms. Known coarse-grained risk labels are used as strong constraints to adaptively cluster within each dataset to extract fine-grained risk prototypes. Simultaneously, a coarse-grained predictor is constructed to assist feature semantic learning and support subsequent inference. Fine-grained risk prototypes from other nodes are introduced as supplementary samples, and weights are assigned according to the sample proportions of the source data, achieving knowledge sharing while protecting data privacy. A hierarchical contrastive loss consisting of positive samples, weakly negative samples, and negative samples is constructed based on the expanded dataset. This loss, combined with a fine-grained classification loss, is used to perform multi-task iterative optimization of the model, effectively improving the model's ability to mine unknown fine-grained risk categories. Through prior coarse label guidance and adaptive clustering mechanisms, the problem of fine-grained classification accuracy under data skew is solved.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and data mining technology, and more specifically, to a method and system for non-independent, identically distributed multimodal data mining based on prototype learning. Background Technology

[0002] In recent years, with the deepening of smart grid construction and the widespread application of power Internet of Things (IoT) technology, the power system has generated massive amounts of multi-source heterogeneous data during operation, marketing, and refined management. Utilizing data mining techniques to extract potential value from this massive data and uncover patterns in electricity consumption and settlement has become a core means of ensuring grid financial security, optimizing resource allocation, and improving operational efficiency. Among these, electricity bill collection risk prediction, as a crucial technical task in power marketing and risk control, aims to accurately predict whether customers have overdue electricity bills, payment channel risks, or potential money laundering risks based on multi-dimensional user and grid characteristics. The accuracy and generalization ability of risk classification prediction results directly determine the practical value of power system risk control in real-world industrial scenarios.

[0003] Traditional electricity bill collection risk prediction models typically build classification algorithms based on the independent and identically distributed (IID) assumption, meaning they assume that the training samples and the actual predicted user data follow the same probability distribution. However, in actual power system operation scenarios, due to the complexity of electricity users and the diverse characteristics of different industries, multi-source electricity data often exhibits strong non-independent and identically distributed (Non-IID) characteristics. Specifically, this manifests in two types of bias: first, feature distribution bias, where there are significant absolute differences in electricity consumption volume and load curves between different industries (such as high-energy-consuming manufacturing enterprises and ordinary residential users), and their operations and electricity consumption are significantly affected by external environmental factors such as climate and seasons; second, category distribution bias, where the credit status of electricity users in specific regions or industries may change abruptly due to external factors such as macroeconomic fluctuations or negative public opinion events, leading to uneven distribution of the probability of arrears risk in time and space. At the same time, among all network users, only a very small minority actually have high-risk arrears, resulting in significant sample imbalance in the data. Furthermore, the data required for power risk prediction has a high degree of multimodal characteristics, including not only structured numerical data (such as current, voltage, historical payment amount, and electricity consumption measured by smart meters), but also textual data (such as reports of negative public opinion events of enterprises, customer service communication logs, and litigation documents). Traditional single data mining models are difficult to effectively integrate and collaboratively process this kind of complex, non-independent, and identically distributed multimodal power data.

[0004] Currently, prototype-based learning techniques are often used to alleviate the non-independent and identically distributed problem in general machine learning. However, directly applying them to complex real-world power scenarios such as electricity bill collection risk prediction presents significant technical limitations. On one hand, traditional prototype learning heavily relies on fine-grained, high-quality labels to construct accurate risk prototypes. However, in actual power marketing management, limited by the cost of manual auditing and information acquisition channels, only coarse-grained status labels are often obtained (e.g., simply marking it as general "overdue" or "normal," without distinguishing whether the overdue payment is due to "short-term cash flow difficulties," "industry-wide systemic downturn," or "deliberate malicious non-payment"). This results in traditional models constructing risk prototypes that are too broad and fail to reflect the true and refined centers of power risk categories. On the other hand, in power scenarios where the samples are not independent and identically distributed and the high-risk samples are extremely unbalanced, traditional methods usually use fixed local prototypes, which are difficult to effectively borrow or expand risk features across different power-consuming industries or geographical distributions. This makes the constructed prototypes prone to deviating from the real risk distribution, and extremely sensitive to a few high-risk samples and random noise in the operation of the power grid. The generalization ability of the model is greatly reduced, which can easily lead to false alarms or false alarms, directly threatening the security of power cash flow and the sound operation of the power grid system.

[0005] Therefore, how to overcome the limitations of traditional prototype learning in complex power operation scenarios characterized by non-independent and identically distributed, multimodal, and only coarse-grained risk labels, effectively discover and align fine-grained technical features in multimodal power data, and thereby improve the accuracy and generalization of power classification models such as electricity bill collection risk prediction, is a technical problem that urgently needs to be solved in the field of power information and intelligent control. Summary of the Invention

[0006] The technical solution of this invention aims to address the significant limitations of existing prototype learning techniques in scenarios involving non-independent and identically distributed data and data imbalance. Specifically, it includes: First, addressing the problem of insufficient representational power of a single prototype due to coarse-grained status labels in electricity marketing: Solving the defect that traditional prototype learning, when only able to acquire coarse-grained risk labels such as "overdue / normal," cannot address the multi-dimensional risk causes within a class, leading to a deviation from the actual electricity consumption focus; Second, addressing the difficulty of cross-domain feature expansion in scenarios involving non-independent and identically distributed data across multiple industries in the power grid: Solving the problem of limited local prototypes caused by inconsistent electricity consumption volume and distribution among different electricity-consuming groups (such as high-energy-consuming industries and ordinary residents), breaking down distribution barriers, and enabling cross-dataset prototype feature borrowing and expansion; Third, addressing the problem of low reliability of minority class prototypes in extreme imbalance scenarios in power risk control: Alleviating the problem of weak risk prototype support caused by insufficient sample size for extreme abnormal events, and avoiding excessive sensitivity of the model to random fluctuation noise in the power grid; Fourth, addressing the difficulty of integrating multi-source heterogeneous power business data: Solving the problem that traditional data mining models struggle to collaboratively process multimodal data such as structured numerical data, images, and text from smart meters.

[0007] The present invention adopts the following technical solution.

[0008] In a first aspect, the present invention provides a method for mining non-independent, identically distributed multimodal data based on prototype learning, comprising the following steps: S1. Obtain multiple non-independent and identically distributed power business datasets containing coarse-grained risk labels; extract the features of samples within each power business dataset to obtain sample feature vectors, and perform clustering based on the sample feature vectors and the coarse-grained risk labels to determine the fine-grained risk categories and corresponding fine-grained risk prototypes under each coarse-grained category. S2. For any of the aforementioned power business datasets, the fine-grained risk prototypes of the remaining power business datasets are used as supplementary samples and assigned corresponding weights to construct an expanded dataset; based on the expanded dataset, the feature similarity between the samples and the fine-grained risk prototypes at different levels is calculated, and a hierarchical contrast loss is constructed in combination with the weights; the fine-grained risk prototypes at different levels include: positive sample prototypes of the same coarse and fine class, weak negative sample prototypes of the same coarse but different fine class, and negative sample prototypes of different coarse classes. S3. Calculate the fine-grained classification loss based on the expanded dataset, and fuse the fine-grained classification loss with the hierarchical contrast loss into a total loss function; update the model parameters based on the total loss function and iterate the fine-grained risk prototype to output the fine-grained risk classification result for the multimodal power business dataset.

[0009] Preferably, in S1, the samples in each of the power business datasets include data of multiple modalities, and the data of multiple modalities includes at least image data, text data and structured data; The structured data includes electricity consumption characteristics and historical payment data of users in different industries; the text data includes negative public opinion event data reflecting changes in users' credit status. Extracting features from samples within each of the power business datasets to obtain sample feature vectors includes: extracting features from the structured data using a fully connected network, extracting features from the text data using a pre-trained language model, and extracting features from the image data using a visual network model to obtain single-modal features corresponding to each modality; and aligning and concatenating the single-modal features to obtain the sample feature vectors.

[0010] Preferably, the single-modal features are dimensionally aligned and concatenated to obtain the sample feature vector, including: Each of the single-modal features is input into a unified projection layer, which includes a modality adaptation sublayer and a common projection layer. The modality adaptation sublayer maps each of the single-modal features to features with the same hidden layer dimension. The mapped modal features are concatenated and input into the common projection layer for dimension alignment to obtain the sample feature vector. Specifically, for missing data in the samples, zero vectors are used to fill the gaps during the splicing process, and during the forward propagation of the common projection layer, the feature weights at the corresponding positions of the zero vectors are reset to zero through a masking mechanism.

[0011] Preferably, clustering is performed based on the sample feature vector and the coarse-grained risk label to determine the fine-grained risk category and the corresponding fine-grained risk prototype under each coarse-grained category, including: Under each coarse-grained category, an initial fine-grained risk prototype is constructed based on the sample feature vector, and the distance from each sample feature vector to the initial fine-grained risk prototype is calculated to assign fine-grained labels. Calculate the distance difference between the nearest first risk prototype and the second risk prototype of the sample feature vector. If the distance difference is less than a preset conflict threshold, it is determined to be a conflict sample. If the conflict sample still cannot be absorbed by a single category after a preset number of iterations, the dynamic fission of the category is triggered, the number of fine-grained labels under the current coarse-grained category is increased, and the feature mean of the conflict sample is used as the fine-grained risk prototype corresponding to the newly added fine-grained risk category. A multidimensional risk evolution state is constructed for all sample feature vectors under the same fine-grained risk category. The multidimensional risk evolution state includes a risk center, a risk boundary, and a prototype stability weight. When iteratively updating the fine-grained risk prototype based on the sample feature vectors, the update magnitude of the risk center is constrained by the prototype stability weight to perform a time-series stable update until a preset convergence condition is met.

[0012] Preferably, fine-grained risk prototypes from other power business datasets are used as supplementary samples and assigned corresponding weights to construct an expanded dataset, including: The samples in the current power business dataset are used as a subset of the original samples, and each sample in the original subset is assigned a preset basic weight. The fine-grained risk prototypes of the remaining power business datasets are used as a subset of cross-dataset prototype samples; For any supplementary sample in the cross-dataset prototype sample subset, obtain the remaining power business dataset that provides the supplementary sample; calculate the proportion of samples in the remaining power business dataset that belong to the same coarse-grained category and the same fine-grained risk category as the supplementary sample, and use the proportion of the number of samples as the corresponding weight of the supplementary sample. The original sample subset is merged with the cross-dataset prototype sample subset to obtain the expanded dataset.

[0013] Preferably, the step of calculating the feature similarity between samples and different levels of fine-grained risk prototypes based on the expanded dataset, and constructing a hierarchical contrastive loss by combining the weights, includes: Calculate the cosine similarity between the sample feature vectors in the expanded dataset and the corresponding positive sample prototypes, weak negative sample prototypes, and negative sample prototypes; Based on a preset temperature coefficient, the calculated cosine similarities of each type are subjected to exponential mapping and summed to obtain positive sample loss term, weak negative sample loss term and negative sample loss term. Based on the preset balance parameters, the positive sample loss term is used as the numerator, and the sum of the positive sample loss term, the weighted weak negative sample loss term, and the weighted negative sample loss term is used as the denominator to calculate the contrast probability score for the sample feature vector. Based on the weights corresponding to each sample, the negative logarithmic values ​​of the contrast probability scores are weighted and summed to obtain the hierarchical contrast loss.

[0014] Preferably, a fine-grained classification loss is calculated based on the augmented dataset, and the fine-grained classification loss is fused with the hierarchical contrastive loss into a total loss function, including: Based on the fine-grained predicted probability and the corresponding true fine-grained label of each sample in the augmented dataset, the fine-grained classification loss is calculated using the cross-entropy loss function. Configure a balancing coefficient for the hierarchical contrast loss; sum the fine-grained classification loss with the hierarchical contrast loss incorporating the balancing coefficient to obtain the total loss function used for backpropagation.

[0015] Secondly, the present invention provides a non-independent, identically distributed (IAD) multimodal data mining system based on prototype learning, and the method for running the non-independent, identically distributed (IAD) multimodal data mining based on prototype learning includes: The feature extraction and clustering module is used to acquire multiple non-independent and identically distributed power business datasets containing coarse-grained risk labels; extract the features of samples within each power business dataset to obtain sample feature vectors, and perform clustering based on the sample feature vectors and the coarse-grained risk labels to determine the fine-grained risk categories and corresponding fine-grained risk prototypes under each coarse-grained category. The cross-dataset augmentation and comparison calculation module is used to construct an augmented dataset by taking fine-grained risk prototypes from other power business datasets as supplementary samples and assigning them corresponding weights for any of the aforementioned power business datasets; calculating the feature similarity between the samples and fine-grained risk prototypes at different levels based on the augmented datasets, and constructing a hierarchical comparison loss by combining the weights; the fine-grained risk prototypes at different levels include: positive sample prototypes of the same coarse and fine class, weak negative sample prototypes of the same coarse but different fine classes, and negative sample prototypes of different coarse classes. The model joint optimization and output module is used to calculate the fine-grained classification loss based on the expanded dataset, fuse the fine-grained classification loss with the hierarchical contrast loss into a total loss function, update the model parameters based on the total loss function and iterate the fine-grained risk prototype, and output the fine-grained risk classification result for the multimodal power business dataset.

[0016] Thirdly, the present invention provides a terminal, including a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method.

[0017] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the steps of the method.

[0018] The beneficial effects of this invention are compared with those of the prior art: 1. This invention achieves refined feature representation in coarse-grained risk labeling scenarios, improving the accuracy of electricity bill collection prediction. It overcomes the shortcomings of traditional prototype learning methods that rely on fine-grained labeled data. By performing cluster mining within coarse-grained categories, it obtains fine-grained risk prototypes representing specific risk causes. Simultaneously, for detached samples in complex power grid data, this invention dynamically assigns sudden abnormal electricity consumption samples that cannot be effectively converged during traditional clustering to newly added fine-grained risk categories. This avoids the feature center shift problem caused by forced classification in traditional algorithms, providing accurate feature support for model feature learning.

[0019] 2. This invention achieves privacy-preserving cross-domain feature augmentation, improving the problems of data silos and sample imbalance in non-independent and identically distributed scenarios. It eliminates the need for cross-industry transmission of raw privacy data. Instead, it utilizes a cross-dataset prototype weighted augmentation strategy to supplement features with fine-grained risk prototypes from different electricity-consuming groups. Simultaneously, it assigns corresponding weights to the supplementary prototypes based on the sample proportions of the source dataset, adapting to the non-independent and identically distributed data distribution characteristics of real-world power grid scenarios, effectively improving the model's generalization and risk warning performance.

[0020] 3. A hierarchical contrastive learning framework is constructed to optimize the semantic destruction problem of traditional contrastive learning. This invention establishes a three-level prototype contrastive mechanism including positive samples of equal coarseness and fineness, weak negative samples of equal coarseness but different fineness, and negative samples of different coarseness. By introducing weak negative sample prototypes, the model can accurately distinguish specific causes of default (such as distinguishing between climate-related defaults and business deterioration defaults) while preserving the semantic association of the macro-level "overdue" dimension, greatly alleviating the risk confusion problem in fine-grained electricity classification scenarios.

[0021] 4. Achieve adaptive fusion and robust handling of missing modalities in multimodal power data. Addressing the challenge of fusing heterogeneous data such as meter readings, real-world images, and public opinion texts, this invention employs modal-differentiated feature mapping and a unified dimension alignment method to adapt to the inherent characteristics of different modal data. To address the prevalent problem of missing modalities in real-world scenarios, a masking mechanism is introduced to zero out the feature weights at zero-vector filling positions, shielding against noise interference from missing data and improving the robustness of the multimodal model under complex power grid conditions. Attached Figure Description

[0022] Figure 1 This is an overall flowchart of the non-independent, identically distributed multimodal data mining method based on prototype learning provided by the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.

[0024] Example 1: like Figure 1 As shown, the non-independent, identically distributed multimodal data mining method based on prototype learning includes the following steps: S1. Obtain multiple non-independent and identically distributed power business datasets containing coarse-grained risk labels; extract the features of samples within each power business dataset to obtain sample feature vectors, and perform clustering based on the sample feature vectors and the coarse-grained risk labels to determine the fine-grained risk categories and corresponding fine-grained risk prototypes under each coarse-grained category.

[0025] S1.1: To implement step S1 above, the parameterization of the non-independent and identically distributed datasets is first defined. The sample and label information of all non-independent and identically distributed power business datasets is recorded, a power business dataset set is constructed, and the core parameters are defined: In the scenario of electricity bill collection risk prediction, the non-independent identically distributed (IOD) nature of the data primarily stems from differences in data distribution among different electricity-consuming groups or regions. Suppose there are a total of N non-independent identically distributed electricity business datasets, such as datasets corresponding to different groups like high-energy-consuming manufacturing industries, general commercial users, and residential electricity consumers, or to power grid marketing data from different climate regions. Because the electricity consumption volume and operating conditions of different groups are affected differently by climate or external economic factors, significant feature distribution and category distribution shifts occur between the datasets. Each dataset is represented as Where i is the dataset index and j is the index of a sample in the dataset and , For the first The total number of samples in each dataset.

[0026] When extracting features from samples within each power business dataset, the sample features contain multiple modalities of data. In a preferred embodiment of this invention, these include image data, text data, and structured data, represented as follows: .in For image data, For text data, The data is structured; specifically, in the context of electricity risk scenarios, image feature data includes on-site photos of paper overdue payment notices and business license photos; text feature data includes public opinion on corporate litigation, news reports, and customer service communication records; and structured feature data includes overdue electricity bill duration, smart meter sensor readings, and historical payment amounts.

[0027] The sample coarse-grained risk label is represented as ,in, For the first The total number of sample categories in each dataset; Fine-grained label definition: Fine-grained labels do not exist in the original dataset, but serve as intermediate quantities to assist the data mining process, used to characterize the specific risk causes hidden under coarse-grained states (such as simply labeled "overdue"). For example, fine-grained labels mined by clustering can represent "overdue payments due to climate-induced cash flow disruptions," "abnormal payment channels," or "potential money laundering risks." There is a many-to-one unique mapping between fine-grained labels and coarse-grained risk labels; for any fine-grained label... There is a unique coarse-grained risk label. Corresponding to this, that is = ,in This is a hierarchical mapping function from fine-grained labels to coarse-grained risk labels. For any coarse-grained risk label... ,exist A fine-grained label to be explored.

[0028] S1.2: Initial Model Construction. This involves building a basic feature extractor, a coarse-grained classification model, and a fine-grained classification model for each power business dataset, and defining learnable parameters. The specific steps are as follows: S1.2.1: Model Definition. For the... A power business dataset Construct a multimodal feature extractor Coarse-grained predictor Fine-grained predictor ,in For feature extractor parameters, For coarse-grained predictor parameters, These are the parameters for the fine-grained predictor. The feature extractor is specifically defined in S1.2.2.

[0029] In a preferred embodiment of the present invention, the coarse-grained predictor and the fine-grained predictor are multilayer perceptron (MLP) classification heads. The network structure shares the feature extractor across different datasets within the same modality. The coarse-grained and fine-grained predictors are modeled independently to avoid negative interference from fine-grained feature learning on coarse-grained features and reduce the risk of model overfitting.

[0030] S1.2.2: Multimodal Feature Mapping. To extract and fuse features from various modalities, the process includes the following stages: First, extract the single-modal features corresponding to each modality of data. This is done for the power business dataset. Samples in ,Will Before the structured data in the data is vectorized through a fully connected network (such as MLP), a normalization process based on industry benchmark electricity load is adopted to address the absolute physical magnitude differences in features such as smart meter sensor readings and historical payment amounts. This is to eliminate the dominant interference of absolute electricity consumption values ​​from extremely high-energy-consuming enterprises on the multimodal joint distribution, before vectorization mapping is performed. The text data in the dataset undergoes feature extraction using pre-trained language models (such as BERT). The image data is processed by visual network models (such as ResNet) to extract features, thereby obtaining unimodal features output by three feature extractors (structured, text, and image).

[0031] Next, the dimensionality of each single-modal feature is aligned and concatenated to obtain the final sample feature vector. Specifically, the extracted single-modal features are added to a unified projection layer. The unified projection layer includes a modality adaptation sublayer and a common projection layer. The specific network structure and masking logic are as follows: the modality adaptation sublayer is a fully connected layer, which maps each single-modal feature to a layer with the same hidden layer dimension. The features of each modality are then concatenated after mapping and input into the common projection layer for dimensional alignment. The common projection layer is a fully connected layer.

[0032] Specifically, for situations where certain modalities of data are missing in actual electricity marketing operations—for example, a company user with good credit and no negative news reports leading to missing text modalities; or a user who has never been contacted for offline collection leading to missing image modalities—zero vectors are used for padding during stitching. Furthermore, during the forward propagation of the common projection layer, a masking mechanism is used to strictly set the feature weights at the corresponding positions of the zero vectors to zero, thus avoiding computational errors and weight interference caused by the zero vectors. The input dimension of the common projection layer is... , where m is the number of input modes, and in this embodiment, m=3 is preferred, and the output dimension is a fixed common dimension.

[0033] After the above multimodal feature mapping, the final sample feature vector is obtained. This provides a feature base for the subsequent construction of fine-grained risk prototypes.

[0034] S1.3: Construction of Fine-Grained Risk Prototypes Based on Clustering and Dynamic Noise Processing. For each power business dataset, fine-grained labels are generated for each coarse-grained category, along with an initial prototype for each label, representing the core features of the fine-grained risk category. Specifically, under each coarse-grained category, an initial fine-grained risk prototype is constructed based on the sample feature vectors obtained in the above steps, and fine-grained labels are assigned using distance metrics. Simultaneously, for detached samples that are difficult to classify, a noise judgment and dynamic class expansion mechanism based on a distance threshold is introduced. This includes the following steps: S1.3.1: Initial Prototype Selection: In the first... For each coarse-grained category in the dataset... Corresponding sample feature vector set Random selection A set of unique feature vectors are used as the initial fine-grained risk prototype, denoted as . The corresponding fine-grained labels are as follows: .

[0035] S1.3.2 Sample Distance Calculation and Assignment: Calculate the distance from each sample feature vector to the initial fine-grained risk prototype to assign fine-grained labels. Specifically, for Each feature vector in Calculate its Euclidean distance to all initial prototypes, and assign it to the fine-grained risk category corresponding to the nearest prototype, i.e. ,in, This indicates that during the initial allocation, samples are assigned... Fine-grained tags.

[0036] S1.3.3 Noise Judgment and Dynamic Classification: For detached samples that are difficult to classify, this method breaks through the traditional single distance threshold judgment method and introduces a dynamic fission mechanism based on prototype conflict. The specific processing logic is as follows: First, calculate the distance of each sample to all prototypes. For samples whose distance to all prototypes exceeds the preset isolation threshold, they are judged as regular noise samples (in power business, this usually corresponds to occasional data acquisition errors or isolated events) and are not included in this round of calculation. Second, establish a prototype conflict detection mechanism. For sample feature vectors... If the distance difference between the nearest first risk prototype and the second risk prototype is less than the preset conflict threshold, that is, the sample is highly similar to two different risk causes at the same time, such as simultaneously exhibiting the dual characteristics of "abnormal climate" and "deteriorating business", then it is determined to be a conflict sample.

[0037] After the next round of valid sample mean update, if the conflicting samples still cannot be absorbed by a single category after a preset number of iterations, the prototype conflict event will automatically trigger dynamic category fission: increasing the number of fine-grained labels under the current coarse-grained category. Furthermore, noise samples are used as fine-grained risk prototypes corresponding to newly added fine-grained risk categories, such as "composite evolutionary risk," and these new prototypes are incorporated into the update process of this round, thereby achieving adaptive discovery of unknown or derived power default risks.

[0038] S1.3.4 Prototype Iterative Update and Convergence: The fine-grained risk prototype is iteratively updated based on the mean of the feature vectors of all samples under the same fine-grained risk category until a preset convergence condition is met. To accurately characterize the dynamic fluctuation characteristics of power risk, this step abandons the use of a single feature mean vector and instead constructs the fine-grained risk prototype as a multi-dimensional risk evolution state. This risk evolution state not only includes the risk center but also introduces risk boundaries and prototype stability constraints.

[0039] Specifically, for each fine-grained risk category First, the mean of all feature vectors in the category is calculated as the risk center for the new round. Second, the variance or maximum deviation of all samples in the category relative to the risk center is calculated as the risk boundary of the prototype (used to characterize the feature divergence of this type of power risk). Finally, the cosine similarity between the current round risk center and the previous round risk center is calculated, and the stability weight of the prototype is generated by combining it with the time decay function. In each iteration, the update magnitude of the risk center is strictly constrained by the stability weight, that is: if the prototype stability of the previous round is high, the center drift in this round is limited, forcing the formation of a time-series stable prototype, thereby avoiding the model from experiencing violent oscillations due to short-term extreme power data (such as sudden anomalies of individual high-energy-consuming enterprises). After completing the multidimensional state update of all fine-grained risk categories in this round, the updated risk evolution state is used as the prototype for the new round, and the process jumps to the aforementioned S1.3.2 to reassign samples. This iterative process is repeated until the change in the risk center and the convergence rate of the risk boundary simultaneously meet the preset conditions, at which point the iteration stops.

[0040] S1.3.5 Fine-grained risk prototype determination: After iterative convergence, determine each coarse-grained category. The final fine-grained risk categories and their corresponding fine-grained risk prototypes are denoted as follows: .

[0041] S2. For any of the aforementioned power business datasets, fine-grained risk prototypes from the remaining power business datasets are used as supplementary samples and assigned corresponding weights to construct an expanded dataset; based on the expanded dataset, the feature similarity between the samples and fine-grained risk prototypes at different levels is calculated, and a hierarchical contrast loss is constructed by combining the weights; the fine-grained risk prototypes at different levels include: positive sample prototypes of the same coarseness and fineness, weak negative sample prototypes of the same coarseness but different fineness, and negative sample prototypes of different coarseness. The specific steps are as follows: S2.1. Cross-dataset weighted augmentation. To address the lack of fine-grained samples in a single power business dataset, for example, rare high-risk events such as "potential money laundering" or "large-scale arrears due to extreme weather" rarely occur in a certain power grid area or general business group, resulting in a lack of corresponding fine-grained risk features for that area, fine-grained risk prototypes from other power business datasets (such as high-risk areas or other industry datasets) are added as supplementary samples to the feature vector set extracted from the current power business dataset, so as to achieve risk feature transfer and knowledge sharing among different electricity users.

[0042] For the i-th power business dataset, the samples in the current power business dataset are taken as a subset of the original samples, and each sample in this subset is assigned a preset base weight, for example, a default weight. Simultaneously, the fine-grained risk prototypes of the remaining power business datasets are used as a subset of cross-dataset prototype samples. For any supplementary sample in this subset, the remaining power business datasets that provide the supplementary sample are obtained; the proportion of samples in the remaining power business datasets that belong to the same coarse-grained category and the same fine-grained risk category as the supplementary sample is calculated, and the proportion of such samples is directly used as the corresponding weight of the supplementary sample.

[0043] The specific expansion rule is as follows: merge the original sample subset with the cross-dataset prototype sample subset to construct the expanded dataset. . No. augmentation sets of each dataset It consists of a subset of the original samples and a subset of prototype samples across the dataset, namely:

[0044] The former is a subset of the original samples; the latter is a subset of the prototype samples across datasets. , These represent the original sample data and its feature vector extracted by the model, respectively. , These represent the coarse-grained risk label and the fine-grained label corresponding to the original sample, respectively. This represents the total number of original samples in the i-th power business dataset; This represents the total number of power business datasets participating in the joint computation; This represents the total number of coarse-grained categories in the s-th power business dataset; This represents the total number of fine-grained risk categories under the corresponding coarse-grained category c; This represents the fine-grained risk prototype corresponding to the coarse-grained category c and the fine-grained risk category k in the s-th dataset; when stored in the augmented set, it comes with its own category label (c,k); This represents the corresponding weight of the cross-dataset prototype.

[0045] Cross-dataset prototype weight definition:

[0046] In the formula, the numerator represents the first... Coarse-grained categories within a power business dataset The finer-grained risk category is The number of samples, the denominator This represents the total number of samples in the s-th power business dataset. , Let represent the coarse-grained risk label and the fine-grained label corresponding to the j-th sample in the s-th power business dataset, respectively. This weighted proportion is used to objectively measure and balance the contribution of the prototype under different data distributions. That is, if a certain type of fine-grained risk (such as seasonal arrears in a specific industry) occurs more frequently (with a larger sample proportion) in the auxiliary dataset providing the prototype, then the prototype is given a higher confidence weight when borrowing features across datasets. For prototype samples, since they are already in vector form, the feature extraction step is skipped, and they are directly used as target feature vectors in subsequent calculations.

[0047] S2.2. Hierarchical Contrastive Learning Calculates Similarity and Loss. A contrastive loss is constructed by calculating the similarity of three types of sample pairs: positive, weakly negative, and negative, to achieve feature alignment between samples and prototypes. The "positive sample pairs" aim to bring samples with the same coarse-grained state and the same specific risk cause, such as both being "overdue due to climate influence." The "weakly negative sample pairs" aim to slightly separate samples with the same state but different causes, such as distinguishing between "overdue due to climate" and "overdue due to malicious default," improving the granularity of risk identification. The "negative sample pairs" aim to significantly separate samples with different coarse-grained states, such as strictly separating "overdue payers" from "paying customers."

[0048] Specifically, the following steps are included: S2.2.1 Define the similarity calculation method: use cosine similarity to calculate the similarity between feature vectors. The formula is:

[0049] Where a and b represent any two feature vectors (such as sample features or prototypes) whose similarity is to be calculated, and d is the dimension of the feature vector. and Let be the component values ​​of feature vectors a and b in the t-th dimension, respectively.

[0050] S2.2.2 Calculate the loss term for each type of sample: Calculate the cosine similarity between the sample feature vectors in the augmented dataset and the corresponding positive sample prototypes, weak negative sample prototypes, and negative sample prototype risk centers; based on the preset temperature coefficient... The calculated cosine similarities are then subjected to exponential mapping and summed to obtain positive sample loss terms, weak negative sample loss terms, and negative sample loss terms. For a sample u in the augmented dataset (whose corresponding feature vector is...) The coarse-grained category is Fine-grained risk category is The specific calculations are as follows: (1) Calculate the loss of positive sample pairs: Traverse the entire power business dataset Calculate the loss of positive sample pairs ( This refers to the similarity loss between sample features and their respective fine-grained risk prototypes:

[0051] in, For the sample eigenvectors, For the prototype of the positive sample in the s-th dataset, The preset temperature coefficient; (2) Calculate the loss of weak negative sample pairs ( For the sample Its weak negative sample pairs are coarse-grained categories within each dataset. However, the fine-grained risk category is not... The prototype. Traverse the entire dataset. Calculate the loss of weak negative sample pairs ( This refers to the similarity loss between sample features and other fine-grained risk prototypes at the same coarse-grained level:

[0052] (3) Calculate the loss of negative sample pairs ( For the sample Its negative sample pairs are within each dataset, and the coarse-grained category is not... The prototype. Traverse the entire dataset. Calculate the loss of negative sample pairs ( ): Similarity loss of sample features to all fine-grained risk prototypes at the coarse-grained level.

[0053] S2.2.3 Calculate the hierarchical contrast probability score: based on the preset balancing parameters, i.e., hyperparameters. The positive sample loss term is used as the numerator, and the sum of the positive sample loss term, the weighted weak negative sample loss term, and the weighted negative sample loss term is used as the denominator to calculate the contrast probability score for the feature vector of that sample. :

[0054] Among them, hyperparameters The preferred value range is (0,1), which is used to flexibly adjust the penalty intensity of the model for weak negative samples and negative samples.

[0055] In particular, to adapt the model to the physical characteristics of the power industry, hyperparameters... It is not a fixed value, but rather negatively correlated with the industry electricity consumption volatility of the dataset to which the current sample belongs. Specifically, for industries that are heavily affected by seasonality / climate and have large fluctuations in electricity consumption (such as agriculture or certain manufacturing industries), a smaller value is assigned. The value is used to reduce the penalty for weak negative samples, allowing for greater flexibility in the feature space; for industries with very stable applications, a larger value is assigned. Values ​​are used to distinguish subtle risk causes.

[0056] S2.3. Fine-grained classification loss calculation and total loss function construction. The hierarchical comparison loss and fine-grained classification loss are fused to form the total loss. Simultaneously, the alignment of risk characteristics across different electricity-consuming groups (such as high-energy-consuming industries and ordinary residents) and the classification accuracy of specific hidden risks (such as money laundering risk and abnormal payment channels) are optimized. Specifically, the following steps are included: S2.3.1 Calculate the hierarchical contrast loss ( The hierarchical contrast loss is obtained by weighting and summing the negative logarithms of the contrast probability scores based on the weights corresponding to each sample. Specifically, based on the augmented set... Calculate the weighted contrast loss:

[0057] in, The weights corresponding to sample u (the weights of the original samples are 1 by default, and the weights of the prototype samples are calculated from the proportions across the dataset). The calculated hierarchical contrast probability score. The total weight of all samples involved in the calculation is used for loss standardization; S2.3.2 Calculate the fine-grained classification loss ( Based on the fine-grained predicted probabilities and corresponding true fine-grained labels of each sample in the augmented dataset, the fine-grained classification loss is calculated using the cross-entropy loss function, as shown in the formula:

[0058] in, This represents the predicted probability distribution of the fine-grained predictor for sample u. The true fine-grained label corresponding to sample u; This represents the cross-entropy operation function, used to calculate the difference between the predicted probability distribution and the true label distribution. It primarily penalizes the model for classification errors in the causes of electricity bill defaults, such as underreporting minority class samples with potential money laundering risks, or misclassifying weather-related overdue payments as malicious arrears.

[0059] S2.3.3 Fusion of the Total Loss Function. The fine-grained classification loss and the hierarchical contrast loss are weighted and summed to obtain the total loss used for backpropagation to optimize the model parameters. :

[0060] in, The preset balance coefficient, and This is used to flexibly adjust the weight of the contrast loss in the total loss.

[0061] S2.4. Backpropagation to optimize model parameters. The model parameters are updated based on the total loss function obtained through fusion. Specifically, the parameters of the feature extractor and the fine-grained predictor are updated with the objective of minimizing the total loss: The optimization objective is to find the optimal network parameters for the current model, i.e., the optimal network parameters for the feature extractor. Optimal network parameters for fine-grained predictors That is, satisfying:

[0062] Optimization method: Employ optimizers such as Adam and SGD, calculate parameter gradients using the backpropagation algorithm, iteratively update parameters until the change in loss in a single iteration is less than a preset convergence threshold. This round of parameter optimization resulted in the trained multimodal electricity bill collection risk prediction model. Based on newly input user basic information, multimodal difference characteristics, and historical payment data, the model can automatically propagate forward and accurately output the user's electricity bill delinquency probability and specific fine-grained risk categories, thereby guiding the electricity marketing department to conduct targeted collection or risk prevention interventions.

[0063] S3. Calculate the fine-grained classification loss based on the expanded dataset, and fuse the fine-grained classification loss with the hierarchical contrast loss into a total loss function; update the model parameters based on the total loss function and iterate the fine-grained risk prototype to output the fine-grained risk classification result for the multimodal power business dataset.

[0064] S3.1 Data Augmentation and Entropy Filtering for Long-Tail Fine-Grained Risk Categories. For fine-grained risk categories with insufficient sample size, data augmentation is used to expand the sample size and filter out noisy samples to ensure the quality of the new samples. Specifically, this includes the following steps: S3.1.1 Small number of samples judgment: The preset small number of samples threshold is... For the extended set of the i-th power business dataset If the sample size of a certain fine-grained risk category meets the following conditions, then the category is determined to have a long-tail distribution problem. For example, in the actual operation of electricity marketing, high-risk fine-grained risks such as "deliberate money laundering" and "mass defaults caused by extreme weather" are often rare events, resulting in a sample size that is much smaller than that of ordinary overdue samples:

[0065] In the formula, : indicates the extended set Any target sample in the dataset; This indicates the coarse-grained risk label corresponding to sample u; This indicates a specific coarse-grained category that is currently being analyzed; This represents the fine-grained label corresponding to sample u; This indicates the k-th fine-grained risk category being analyzed under the corresponding coarse-grained category c; This indicates the total number of samples in the set that meet the conditions.

[0066] S3.1.2 Gaussian Noise Enhancement: Differentiated noise enhancement strategies are employed based on the characteristics of different modalities. For structured data, Gaussian noise is added to the structured data components of the original samples to generate new sample data. To simulate the minute fluctuations in sensor readings or reasonable fluctuations in user electricity consumption and payment amounts during actual power grid operation, the model's robustness to feature shifts is improved. The formula is as follows:

[0067] In the formula, This represents the structured data component of the j-th original sample in the i-th power business dataset; Let the introduced random Gaussian noise variable satisfy the following conditions: , This indicates that the mean is 0 and the variance is 0. Gaussian distribution; The enhanced feature vector extracted from the new sample is denoted as:

[0068] In the formula, This represents the enhanced sample feature vector obtained after the enhanced sample has passed through the above feature extractor. This represents the latest round of parameter optimization for the multimodal feature extractor and its network parameters. Furthermore, for the labels of enhanced samples, the coarse-grained risk labels are still strictly retained as those of the original samples. Fine-grained labels are still retained. .

[0069] For image data, Gaussian noise is added to each pixel in the image to generate new samples; for text data, Gaussian noise is added to the Embedding layer obtained by the word embedding model to generate new samples.

[0070] S3.1.3 Entropy Filtering: To prevent low-quality augmented samples from interfering with the model, once the model has completed half of the preset total training iterations and possesses a certain level of discriminative ability, the predicted entropy of newly added augmented samples is calculated:

[0071] in, Indicates newly added enhanced samples The prediction entropy is used to measure the uncertainty of the model in classifying it; This represents the total number of coarse-grained categories in the i-th power business dataset; This represents the total number of fine-grained risk categories under a specific coarse-grained category c; This indicates that the current fine-grained predictor is targeting this enhanced feature. The output is the probability value that it is predicted to be class (c,k).

[0072] After the calculation is completed, an entropy filtering threshold is pre-set for the i-th power business dataset. ,reserve The high-determinism samples are discarded, and the remaining high-quality augmented samples form the augmented sample subset, denoted as . ; S3.1.4 Expanding the Dataset Update: This will enhance the sample set. Add to the extension set The updated dataset is obtained:

[0073] in, This represents the operation of finding the union of two sets; This represents the final updated dataset set for expanded power business. This updated dataset will be fully utilized for subsequent rounds of model parameter optimization and fine-grained risk prototyping iterations.

[0074] S3.2 generates a new round of fine-grained risk prototypes based on the updated dataset. Recalculate the fine-grained risk prototype to improve the accuracy of the prototype's representation of category features: S3.2.1 Sample set partitioning: For the first... The updated set of datasets The sample set belonging to coarse-grained category c and fine-grained risk category k is selected and denoted as . The formula is expressed as:

[0075] S3.2.2 New Prototype Calculation: To maintain the consistency of the multidimensional evolutionary state of the prototype, the mean of the feature vectors of all samples in the set is used as the risk center of the new round of prototypes.

[0076] in, This represents the risk center vector, calculated based on the latest round of data features, belonging to the i-th power business dataset, and representing the new round of fine-grained risk prototype. Let u be the feature vector of sample u. This represents the total number of samples in the sample set. The risk boundary and prototype stability weights for this round are updated synchronously based on this sample set, completing the overall iteration of the multidimensional risk evolution state for this fine-grained risk category.

[0077] S3.3. Loop Training and Termination Judgment. The new prototype replaces the previous prototype, and the next optimization loop begins, continuing until the termination condition is met. This includes the following steps: S3.3.1 Loop Trigger: The newly calculated... Replace the initial prototype obtained in the initial stage (as described in step S1.3 above) or the one retained from the previous iteration. Each power business dataset will synchronously share its updated local fine-grained risk prototype for use by other power business datasets in the next round. Then, it will jump back to the cross-dataset weighted expansion step (i.e., the aforementioned step S2.1), repeating the cross-dataset expansion, loss calculation and parameter optimization, and prototype update iteration process. S3.3.2 Termination condition: When the number of iterations reaches the preset maximum number of iterations. or the total loss of three consecutive iterations. The changes were all less than the preset convergence threshold. When the time comes, stop training; S3.3.3 Output Results: Output and save the final optimal model parameters. (i.e., the parameters of the trained feature extractor network) and (i.e., the trained fine-grained predictor network parameters) complete the training of a fine-grained classification model for non-independent and identically distributed data. The trained model can be directly deployed in the power grid marketing control system or risk control cloud platform. When new users' multimodal power business data (such as new meter data, the latest corporate public opinion and on-site photos) are accessed, feature extraction and classification reasoning can be performed in real time, and the potential fine-grained electricity fee collection risk warning results of the user can be automatically output, thereby providing reliable technical support for power grid financial security and intelligent dispatch.

[0078] Example 2: This invention provides a non-independent, identically distributed (i.e., MIDD) multimodal data mining system based on prototype learning, and the method for running the non-independent, identically distributed (i.e., MIDD) multimodal data mining based on prototype learning includes: The feature extraction and clustering module is used to acquire multiple non-independent and identically distributed power business datasets containing coarse-grained risk labels; extract the features of samples within each power business dataset to obtain sample feature vectors, and perform clustering based on the sample feature vectors and the coarse-grained risk labels to determine the fine-grained risk categories and corresponding fine-grained risk prototypes under each coarse-grained category. The cross-dataset augmentation and comparison calculation module is used to construct an augmented dataset by taking fine-grained risk prototypes from other power business datasets as supplementary samples and assigning them corresponding weights for any of the aforementioned power business datasets; calculating the feature similarity between the samples and fine-grained risk prototypes at different levels based on the augmented datasets, and constructing a hierarchical comparison loss by combining the weights; the fine-grained risk prototypes at different levels include: positive sample prototypes of the same coarse and fine class, weak negative sample prototypes of the same coarse but different fine classes, and negative sample prototypes of different coarse classes. The model joint optimization and output module is used to calculate the fine-grained classification loss based on the expanded dataset, fuse the fine-grained classification loss with the hierarchical contrast loss into a total loss function, update the model parameters based on the total loss function and iterate the fine-grained risk prototype, and output the fine-grained risk classification result for the multimodal power business dataset.

[0079] This example uses an electricity bill collection risk prediction scenario to accurately predict whether a customer faces the risk of overdue electricity bills by integrating multi-dimensional data, while also identifying potential risks such as payment channel risks and money laundering risks. This scenario requires a focus on the differences in data distribution across different industries and user groups, such as electricity consumption volume (which varies across industries); climate impact (different industries are affected differently by climate in terms of electricity consumption and business conditions); and external influences (such as negative public opinion events leading to changes in user credit status and the risk of overdue payments). These differences need to be comprehensively analyzed based on basic user information, risk difference characteristics, and historical payment data to address the insufficient adaptability of traditional models to non-independent and identically distributed data.

[0080] The specific application implementation steps are as follows: A. Preprocessing stage: Define a non-independent, identically distributed dataset: Divide into three datasets by industry: Industrial (D1), Commercial (D2), and Residential (D3). Each sample represents a single user; for example, D1 contains 800 factories, D2 contains 5000 shops, and D3 contains 100,000 households. Features include three modalities: structured data (e.g., "average monthly electricity consumption," "equipment rated power"), text data (e.g., "negative public opinion text about shops," "litigation records"), and image data (e.g., "on-site equipment photos," "business license images"). Labels are divided into two levels: coarse-grained (1 = risk of overdue payments, 0 = no risk of overdue payments); and fine-grained labels representing the implicit specific risk causes. For example, the coarse-grained "1 = risk of overdue payments" is further subdivided into: 11 = deteriorating business type overdue payments (based on sudden changes in textual public opinion), 12 = deliberate electricity fraud / money laundering type anomalies (based on mismatches between images and structured electricity consumption), and 13 = climate change type default (based on seasonal surges in electricity consumption leading to a break in the cash flow).

[0081] The initial model is constructed by building a multimodal feature extractor, a coarse-grained predictor, and a fine-grained predictor for each dataset. The feature extractor uses fully connected layers to process structured data, BERT to process text, and ResNet to process images. For users lacking relevant information or images, a unified projection layer's masking mechanism resets the zero-vector weights to zero, resulting in a unified feature vector output. The coarse-grained predictor takes the feature vector as input and outputs the probability of "having / not having overdue risk" through a sigmoid layer. The fine-grained predictor outputs the probability of further subdivided risks through a softmax layer.

[0082] For each dataset's "risk of overdue payments" samples (coarse-grained 1), three feature vectors are randomly selected as initial high / medium / low-risk prototypes. Samples are then assigned to the nearest prototypes, and the mean is used for updating. Specifically, if some samples are found, such as factories experiencing sudden and extremely high power consumption accompanied by severe negative public opinion, and are found to be far from the existing high / medium / low-risk prototypes (exceeding the distance threshold), they are classified as noise samples. If they still cannot be classified after iteration, they are dynamically classified independently, for example, by adding "14 = extreme abnormal risk," and this is used as the prototype for the new fine-grained risk category.

[0083] B Prototype Optimization Phase: Data augmentation across datasets: For fine-grained risk categories with scarce samples (e.g., only 20 high-risk business samples in D2), high-risk fine-grained risk prototypes from D1 and D3 are added to D2 as "supplementary samples" to construct an augmented set; at the same time, prototype weights are set according to the number of fine-grained samples in each industry (e.g., D1 has many high-risk samples, so the weight is set to 0.6; D3 is set to 0.4) to balance the data contribution.

[0084] Hierarchical contrastive learning calculates the loss: Three types of sample pairs are lost using cosine similarity. Positive sample pairs, such as high-risk commercial user features → high-risk prototypes in the D2 augmentation set, bring the target closer; weak negative sample pairs, such as high-risk commercial user features → medium-risk prototypes in D2, moderately increase the target's distance; negative sample pairs, such as high-risk commercial user features → risk-free prototypes in D2, maximize the target's distance. Combining prototype weights, the three types of losses are fused to obtain the hierarchical contrastive loss.

[0085] Total Loss and Parameter Optimization: The hierarchical contrastive loss and fine-grained classification loss, i.e., cross-entropy loss, are fused to calculate the error in predicting the specific risk cause of a user. A balance coefficient α = 0.3 is set to obtain the total loss. The Adam optimizer is then used for backpropagation to update... With the newly added fine-grained predictor parameters Until the change in loss per instance is less than 0.0001.

[0086] C update and iteration phase: Small sample augmentation and screening: Set the sample threshold T=300. If a certain fine-grained risk category (such as D1 with only 250 low-risk industrial samples) satisfies the condition that the number of samples < T, add Gaussian noise (noise variance 0.1) to the original samples to generate new samples to simulate small fluctuations in sensor data. Calculate the prediction entropy of the new samples (the lower the entropy value, the higher the certainty). Keep the samples with entropy values ​​less than 0.5 and add them to the augmented set to form the updated dataset.

[0087] Update the fine-grained risk prototype: Based on the updated dataset, divide the samples according to fine-grained risk categories, and use the mean of the feature vectors of samples of the same category as the new risk prototype to replace the original prototype.

[0088] Looping and Termination: Substitute the new prototype into the above Phase B process and repeat. Continue until the number of iterations reaches 100 rounds, or the total loss change is <0.0001 for 3 consecutive rounds. Finally, output the trained and converged multimodal classification model of electricity bill collection risk and its optimal network parameters, and deploy it in the actual power grid business system to achieve high-precision prediction of electricity bill overdue risk and intelligent diagnosis of causes for non-independent and identically distributed users.

[0089] Example 3: This invention provides a terminal, including a processor and a storage medium; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method.

[0090] Example 4: The present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method.

[0091] This disclosure can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of this disclosure.

[0092] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0093] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0094] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.

Claims

1. A multimodal data mining method based on prototype learning for non-independent and identically distributed data, characterized in that: Includes the following steps: S1. Obtain multiple non-independent and identically distributed power business datasets containing coarse-grained risk labels; extract the features of samples within each power business dataset to obtain sample feature vectors, and perform clustering based on the sample feature vectors and the coarse-grained risk labels to determine the fine-grained risk categories and corresponding fine-grained risk prototypes under each coarse-grained category. S2. For any of the aforementioned power business datasets, fine-grained risk prototypes from the remaining power business datasets are used as supplementary samples and assigned corresponding weights to construct an expanded dataset. Based on the expanded dataset, the feature similarity between the sample and the fine-grained risk prototypes at different levels is calculated, and the hierarchical contrast loss is constructed by combining the weights. The different levels of fine-grained risk prototypes include: positive sample prototypes of the same coarse and fine class, weak negative sample prototypes of the same coarse but different fine class, and negative sample prototypes of different coarse classes. S3. Calculate the fine-grained classification loss based on the expanded dataset, and fuse the fine-grained classification loss with the hierarchical contrast loss into a total loss function; update the model parameters based on the total loss function and iterate the fine-grained risk prototype to output the fine-grained risk classification result for the multimodal power business dataset.

2. The non-independent, identically distributed multimodal data mining method based on prototype learning according to claim 1, characterized in that, In S1, the samples in each of the power business datasets include data of multiple modalities, which include at least image data, text data, and structured data; The structured data includes electricity consumption characteristics and historical payment data of users in different industries; the text data includes negative public opinion event data reflecting changes in users' credit status. Extracting features from samples within each of the aforementioned power business datasets to obtain sample feature vectors includes: for the structured data, after normalization processing based on industry benchmark electricity load, extracting features from the structured data using a fully connected network, extracting features from the text data using a pre-trained language model, and extracting features from the image data using a visual network model to obtain single-modal features corresponding to each modality; and aligning and concatenating the dimensions of each single-modal feature to obtain the sample feature vector.

3. The non-independent, identically distributed multimodal data mining method based on prototype learning according to claim 2, characterized in that, The single-modal features are dimensionally aligned and concatenated to obtain the sample feature vector, including: Each of the single-modal features is input into a unified projection layer, which includes a modality adaptation sublayer and a common projection layer. The modality adaptation sublayer maps each of the single-modal features to features with the same hidden layer dimension. The mapped modal features are concatenated and input into the common projection layer for dimension alignment to obtain the sample feature vector. Specifically, for missing data in the samples, zero vectors are used to fill the gaps during the splicing process, and during the forward propagation of the common projection layer, the feature weights at the corresponding positions of the zero vectors are reset to zero through a masking mechanism.

4. The non-independent and identically distributed multimodal data mining method based on prototype learning according to claim 1, characterized in that, Clustering is performed based on the sample feature vectors and the coarse-grained risk labels to determine the fine-grained risk categories and corresponding fine-grained risk prototypes under each coarse-grained category, including: Under each coarse-grained category, an initial fine-grained risk prototype is constructed based on the sample feature vector, and the distance from each sample feature vector to the initial fine-grained risk prototype is calculated to assign fine-grained labels. Calculate the distance difference between the nearest first risk prototype and the second risk prototype of the sample feature vector. If the distance difference is less than a preset conflict threshold, it is determined to be a conflict sample. If the conflict sample still cannot be absorbed by a single category after a preset number of iterations, the dynamic fission of the category is triggered, the number of fine-grained labels under the current coarse-grained category is increased, and the feature mean of the conflict sample is used as the fine-grained risk prototype corresponding to the newly added fine-grained risk category. A multidimensional risk evolution state is constructed for all sample feature vectors under the same fine-grained risk category. The multidimensional risk evolution state includes a risk center, a risk boundary, and a prototype stability weight. When iteratively updating the fine-grained risk prototype based on the sample feature vectors, the update magnitude of the risk center is constrained by the prototype stability weight to perform a time-series stable update until a preset convergence condition is met.

5. The non-independent and identically distributed multimodal data mining method based on prototype learning according to claim 1, characterized in that, Fine-grained risk prototypes from other power business datasets are used as supplementary samples and assigned corresponding weights to construct an expanded dataset, including: The samples in the current power business dataset are used as a subset of the original samples, and each sample in the original subset is assigned a preset basic weight. The fine-grained risk prototypes of the remaining power business datasets are used as a subset of cross-dataset prototype samples; For any supplementary sample in the cross-dataset prototype sample subset, obtain the remaining power business dataset that provides the supplementary sample; calculate the proportion of samples in the remaining power business dataset that belong to the same coarse-grained category and the same fine-grained risk category as the supplementary sample, and use the proportion of the number of samples as the corresponding weight of the supplementary sample; The original sample subset is merged with the cross-dataset prototype sample subset to obtain the expanded dataset.

6. The non-independent and identically distributed multimodal data mining method based on prototype learning according to claim 1, characterized in that, The step of calculating the feature similarity between samples and different levels of fine-grained risk prototypes based on the expanded dataset, and constructing a hierarchical contrastive loss by combining the weights, includes: Calculate the cosine similarity between the sample feature vectors in the expanded dataset and the corresponding positive sample prototypes, weak negative sample prototypes, and negative sample prototypes; Based on a preset temperature coefficient, the calculated cosine similarities of each type are subjected to exponential mapping and summed to obtain positive sample loss term, weak negative sample loss term and negative sample loss term. Based on the preset balance parameters, the positive sample loss term is used as the numerator, and the sum of the positive sample loss term, the weighted weak negative sample loss term, and the weighted negative sample loss term is used as the denominator to calculate the contrast probability score for the sample feature vector. Based on the weights corresponding to each sample, the negative logarithmic values ​​of the contrast probability scores are weighted and summed to obtain the hierarchical contrast loss.

7. The non-independent and identically distributed multimodal data mining method based on prototype learning according to claim 1, characterized in that, Based on the augmented dataset, a fine-grained classification loss is calculated, and the fine-grained classification loss is fused with the hierarchical contrastive loss into a total loss function, including: Based on the fine-grained predicted probability and the corresponding true fine-grained label of each sample in the augmented dataset, the fine-grained classification loss is calculated using the cross-entropy loss function. Configure a balancing coefficient for the hierarchical contrast loss; sum the fine-grained classification loss with the balancing coefficient for the hierarchical contrast loss to obtain the total loss function used for backpropagation.

8. A non-independent, identically distributed (IOD) multimodal data mining system based on prototype learning, running the non-independent, identically distributed (IOD) multimodal data mining method based on prototype learning as described in any one of claims 1-7, characterized in that, include: The feature extraction and clustering module is used to acquire multiple non-independent and identically distributed power business datasets containing coarse-grained risk labels; extract the features of samples within each power business dataset to obtain sample feature vectors, and perform clustering based on the sample feature vectors and the coarse-grained risk labels to determine the fine-grained risk categories and corresponding fine-grained risk prototypes under each coarse-grained category. The cross-dataset augmentation and comparison calculation module is used to construct an augmented dataset by taking fine-grained risk prototypes from other power business datasets as supplementary samples and assigning them corresponding weights for any of the aforementioned power business datasets. Based on the expanded dataset, the feature similarity between the sample and the fine-grained risk prototypes at different levels is calculated, and the hierarchical contrast loss is constructed by combining the weights. The different levels of fine-grained risk prototypes include: positive sample prototypes of the same coarse and fine class, weak negative sample prototypes of the same coarse but different fine class, and negative sample prototypes of different coarse classes. The model joint optimization and output module is used to calculate the fine-grained classification loss based on the expanded dataset, fuse the fine-grained classification loss with the hierarchical contrast loss into a total loss function, update the model parameters based on the total loss function and iterate the fine-grained risk prototype, and output the fine-grained risk classification result for the multimodal power business dataset.

9. A terminal, comprising a processor and a storage medium; characterized in that: The storage medium is used to store instructions; The processor is configured to operate according to the instructions to perform the steps of the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1-7.

Images