Image classification method and system for training data set imbalance

Abstract

The invention discloses an image classification method and system for training data set imbalance, and belongs to the field of unbalanced training data set image classification. The method specifically comprises the steps of collecting an unbalanced image data set, and performing preliminary training on a first feature extraction network and a first classifier; collecting a pure image data set, extracting an initial feature data set based on the first feature extraction network after preliminary training and the pure image data set, and calculating a real feature center of the tail category; obtaining an updated feature data set based on the real feature center, combining the obtained balanced feature data set, adjusting the first classifier after preliminary training, collecting a to-be-classified unbalanced image data set, and inputting the to-be-classified unbalanced image data set into the first feature extraction network after preliminary training and the adjusted first classifier to obtain a classification result; according to the method, the prediction accuracy of the model on the tail category is improved, and the classification stability of the model under the unbalanced data set is enhanced.

Description

Technical Field

[0001] This invention belongs to the field of image classification of imbalanced training datasets, and specifically relates to an image classification method and system for imbalanced training datasets. Background Technology

[0002] In industrial manufacturing and equipment maintenance scenarios, deep learning-based image classification models are commonly used to assess the quality and identify defect types in images captured by production line cameras or inspection cameras. However, the training performance of such industrial image classification models is closely related to the category distribution of the training dataset. In actual production lines, there are ample samples of common defects and normal samples, while rare defect samples occur infrequently, are costly to collect, and are difficult to cover the actual range of appearance variations. This results in a long-tailed distribution in the training dataset, affecting the model's stability and defect recall rate after deployment on-site. Therefore, effectively training image classification models for imbalanced industrial image datasets is a crucial problem that urgently needs to be solved in the implementation of industrial vision quality inspection.

[0003] In existing technologies, methods for handling imbalanced datasets in industrial image classification tasks typically involve data resampling to balance the number of samples, introducing class weighting into the loss function, or utilizing transfer learning to improve the feature representation ability of tail categories. However, existing image classification methods for imbalanced training datasets suffer from the following problems: they ignore the fact that tail categories, trained on long-tailed data, experience a shift in their training set feature centers, resulting in significant deviations from the true feature centers. This leads to low prediction accuracy for rare defect categories in industrial settings, making it prone to missed or misclassified tail defects. Summary of the Invention

[0004] The purpose of this invention is to provide an image classification method and system for imbalanced training datasets, in order to solve the problems of low prediction accuracy in existing technologies.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: An image classification method for imbalanced training datasets includes: An imbalanced image dataset is collected, and the first feature extraction network and the first classifier are initially trained using the imbalanced image dataset. Collect a clean image dataset, extract an initial feature dataset based on the first feature extraction network after preliminary training and the clean image dataset, and calculate the true feature center of the tail category; An updated feature dataset for the tail category is generated based on the real feature centers. The first classifier after initial training is adjusted by merging the updated feature dataset and the initial feature dataset to obtain a balanced feature dataset. An imbalanced image dataset to be classified is collected, and the dataset is input into a first feature extraction network that has been pre-trained and a first classifier that has been adjusted to obtain the classification result.

[0006] Specifically, an imbalanced image dataset is collected, and the first feature extraction network and the first classifier are initially trained using the imbalanced image dataset, including: An imbalanced image dataset with a long-tailed distribution is collected, and data augmentation is performed on the imbalanced image dataset to obtain an augmented dataset, wherein each sample in the imbalanced image dataset has a corresponding true class label; The augmented dataset is input into the first feature extraction network and the first classifier to obtain the predicted probability distribution of each sample in the augmented dataset for each category. Based on the predicted probability distribution of each sample and the target distribution corresponding to the true class label, the cross-entropy loss is calculated, and the parameters of the first feature extraction network and the first classifier are iteratively updated until the parameters after iteration meet the preset iteration termination condition. The iteration is then completed, and the first feature extraction network and the first classifier after preliminary training are obtained.

[0007] Specifically, a clean image dataset is collected. Based on the pre-trained first feature extraction network and the clean image dataset, an initial feature dataset is extracted, and the true feature centers of the tail category are calculated, including: Collect a clean image dataset and input the clean image dataset into the first feature extraction network after preliminary training to obtain the initial feature datasets of each category in the clean image dataset. The tail category is modeled based on the initial feature dataset to obtain each sub-center, and the weight of each sub-center in the tail category is determined. Based on the sub-centers and their weights, the tail categories are merged and optimized to obtain the true feature centers of the tail categories.

[0008] Specifically, the tail category is modeled based on the initial feature dataset to obtain each sub-center, and the weight of each sub-center in the tail category is determined, including: Based on the initial feature dataset, the mean vector and covariance matrix of each head category in the clean image dataset are calculated to obtain the center of each head category; Clustering is performed on the initial feature dataset of the tail category to divide the initial feature dataset of the tail category into sample sets, thereby obtaining the sub-centers of the tail category, and the sub-centers and sample sets correspond one-to-one. Based on the center of each head category, calculate the weight of each sub-center in the tail category.

[0009] Specifically, based on the center of each head category, the weights of each sub-center in the tail category are calculated, including: Based on the sample set corresponding to each subcenter, calculate the mean vector, covariance matrix and cluster weight of each subcenter; Calculate the similarity between the mean vector of each sub-center and the mean vector of each head category center, and filter the head categories based on the similarity to obtain the reference head categories; The weights of each subcenter in the tail category are calculated based on the cluster weights, covariance matrix, and similarity between each subcenter and the reference head category.

[0010] Specifically, based on each sub-center and its weight, the tail categories are merged and optimized to obtain the true feature centers of the tail categories, including: Based on each subcenter and its weight, the tail categories are initially merged, and the initial mean vector and covariance matrix are calculated to obtain the initial merging result. Based on the preliminary merging results, the preliminary covariance matrix is ​​subjected to morphological constraints to obtain the constrained merging results. The merging parameters in the initial merging process and the morphological constraint process are iteratively updated until the constraint merging result meets the preset iteration termination condition. The iteration is then completed, and the true feature center of the tail category is obtained.

[0011] Specifically, based on each subcenter and its weight, the tail categories are initially merged, and a preliminary mean vector and a preliminary covariance matrix are calculated to obtain the preliminary merging results, including: Based on the weights of each subcenter, a weighted average is performed on the mean vectors of each subcenter to obtain a preliminary mean vector; Calculate the difference between the mean vector of each subcenter and the preliminary mean vector. Based on the difference and the weight of each subcenter, perform weighted merging of the covariance matrices of each subcenter to obtain the preliminary covariance matrix. By combining the initial mean vector and the initial covariance matrix, a preliminary merging result is obtained.

[0012] Specifically, based on the preliminary merging results, morphological constraints are applied to the preliminary covariance matrix to obtain constrained merging results, including: The initial covariance matrix is ​​mapped to the space of a symmetric positive definite matrix manifold to obtain the mapped covariance matrix; In the symmetric positive definite matrix manifold space, each subcenter of the tail category is mapped to a causal node. Based on the preset causal relationship rules, the neighbor set of each causal node is determined, and causal edges are constructed between the causal nodes. Based on the mapping covariance matrix and the neighbor set, the preliminary influence coefficient of each causal edge is calculated. The preliminary influence coefficient is then normalized based on the causal flow constraint to obtain the preliminary causal weight. Based on the initial causal weights, a causal adjustment function is constructed to obtain the adjustment operator. The mapping covariance matrix is ​​iteratively updated through the adjustment operator until the mapping covariance matrix meets the preset update termination condition. The iteration is completed, and the optimized covariance matrix is ​​obtained. The optimized covariance matrix is ​​mapped back to the initial space to obtain the constrained covariance matrix. The constrained covariance matrix and the initial mean vector are combined to obtain the constrained merging result.

[0013] Specifically, based on the initial causal weights, a causal regulation function is constructed, resulting in a regulation operator, including: Construct a tail-centered causal graph based on causal nodes and causal edges; Based on the mapping covariance matrix, extract the mapping covariance submatrix corresponding to each pair of subcenters in the causal graph of the tail subcenters; Based on the sample set corresponding to each subcenter and the mean vector of each subcenter, the sparsity of each subcenter is calculated. Based on the mapping covariance submatrix, sparsity, and preliminary causal weights, calculate the causal regulation subfunction for each pair of subcenters; Based on the causal regulation function, the weights and mean vectors of each subcenter, the preliminary regulation matrix for each pair of subcenters is calculated, and the preliminary regulation matrix is ​​constrained to obtain the constrained regulation matrix. The constraint regulator matrix is ​​topologically ordered and combined based on the tail subcenter causal graph. The causal regulation function is obtained by weighted accumulation and used as the regulation operator.

[0014] An image classification system for imbalanced training datasets includes: The preliminary training module is used to collect an imbalanced image dataset and use the imbalanced image dataset to perform preliminary training on the first feature extraction network and the first classifier. The center determination module is used to collect a clean image dataset. Based on the first feature extraction network after preliminary training and the clean image dataset, the initial feature dataset is extracted, and the true feature center of the tail category is calculated. The classification adjustment module generates an updated feature dataset for the tail category based on the true feature centers, and adjusts the first classifier after initial training by merging the updated feature dataset and the initial feature dataset to obtain a balanced feature dataset. The image classification module is used to collect an imbalanced image dataset to be classified, and input the imbalanced image dataset to be classified into the first feature extraction network after preliminary training and the first classifier after adjustment to obtain the classification result.

[0015] Compared with existing technologies, the beneficial effects of this invention include: using an imbalanced image dataset for initial model training; modeling tail categories using multiple subcenters based on a clean image dataset to obtain each subcenter and its weight; constructing a causal graph of tail subcenters, combining causal relationship rules to impose morphological constraints on the covariance matrix, constructing an adjustment operator through causal weights, iteratively updating the mapped covariance matrix to obtain an optimized covariance matrix, and determining the true feature centers of tail categories; expanding the feature samples of tail categories based on the true feature centers, constructing a balanced feature dataset, and adjusting the classifier; inputting the imbalanced image dataset to be classified into the adjusted network and classifier, and outputting the classification results; solving the problem in existing technologies where tail category feature centers shift when training with imbalanced image datasets with long-tail distributions, resulting in low tail category classification accuracy, improving the model's prediction accuracy of the true distribution of tail categories, and enhancing the model's classification stability under long-tail data. Attached Figure Description

[0016] Figure 1 This invention provides a flowchart of an image classification method for imbalanced training datasets. Figure 2 The flowchart for generating true feature centers for tail categories provided by this invention; Figure 3 A flowchart illustrating the preliminary covariance matrix morphological constraints provided by this invention; Figure 4 This invention provides a structural diagram of an image classification system for imbalanced training datasets. Detailed Implementation

[0017] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0018] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0019] Example 1 Please see Figures 1-3 The present invention provides an embodiment of an image classification method for imbalanced training datasets, comprising the following specific steps: Step S1: Collect an imbalanced image dataset and use the imbalanced image dataset to perform preliminary training on the first feature extraction network and the first classifier.

[0020] The specific steps of step S1 are as follows: Step S101: Collect an imbalanced image dataset with a long-tailed distribution, and perform data augmentation on the imbalanced image dataset to obtain an augmented dataset.

[0021] In this embodiment, each sample in the imbalanced image dataset has a true class label. It should be noted that after collecting the imbalanced image dataset with a long tail distribution, the data needs to be preprocessed. The preprocessing specifically includes: outlier handling, data standardization, missing value imputation, time alignment, and feature smoothing.

[0022] In the data augmentation process, the tail class samples of the imbalanced image dataset are oversampled to make the tail class samples appear more frequently during training, resulting in tail class expansion samples. The tail class expansion samples and the imbalanced image dataset are then merged to obtain the augmented dataset.

[0023] Step S102: Input the augmented dataset into the first feature extraction network and the first classifier to obtain the predicted probability distribution of each sample in the augmented dataset for each category.

[0024] In this embodiment, the first feature extraction network is a convolutional neural network, the first classifier is a probability-calibrated support vector machine, and the predicted probability distribution represents the probability that each sample belongs to each category.

[0025] Step S103: Based on the predicted probability distribution of each sample and the target distribution corresponding to the true category label, calculate the cross-entropy loss, iteratively update the parameters of the first feature extraction network and the first classifier until the parameters after iteration meet the preset iteration termination condition, and the iteration is completed, thus obtaining the first feature extraction network and the first classifier after preliminary training.

[0026] In this embodiment, the difference between the predicted probability distribution of each sample and the target distribution corresponding to the true category label is used as the loss function, and cross-entropy loss is used for calculation. The parameters of the first feature extraction network and the first classifier are iteratively updated based on the loss value.

[0027] The iteration termination conditions include: the loss function value is less than or equal to a preset loss threshold, or the number of iteration rounds reaches a preset first round number. The loss threshold and the first round number are set by those skilled in the art according to the actual situation.

[0028] Step S2: Collect a clean image dataset. Based on the first feature extraction network after initial training and the clean image dataset, extract the initial feature dataset and calculate the true feature center of the tail category.

[0029] The specific steps of step S2 are as follows: Step S201: Collect a clean image dataset and input the clean image dataset into the first feature extraction network after preliminary training to obtain the initial feature datasets of each category in the clean image dataset.

[0030] In this embodiment, a clean image dataset represents a dataset where each sample has a clear sample label and is free from noise or pseudo-label interference. It should be noted that after collecting a clean image dataset, the data needs to be preprocessed. The preprocessing specifically includes: outlier handling, data standardization, missing value imputation, time alignment, and feature smoothing.

[0031] Step S202: Model the tail category based on the initial feature dataset, obtain each sub-center, and determine the weight of each sub-center in the tail category.

[0032] The specific steps of step S202 are as follows: Step S2021: Based on the initial feature dataset, calculate the mean vector and covariance matrix of each head category in the clean image dataset to obtain the center of each head category.

[0033] In this embodiment, based on the initial feature dataset, the feature vectors of all samples in each head category are obtained. The feature vectors of all samples in each head category are weighted and averaged to obtain the mean vector of each head category. The difference term between the feature vectors of all samples in each head category and the corresponding mean vector is calculated to obtain the covariance matrix of each head category. Based on the mean vector and the covariance matrix, the distribution center of each head category in the feature space is determined, i.e., the center of each head category.

[0034] Step S2022: Perform clustering on the initial feature dataset of the tail category, divide the initial feature dataset of the tail category into sample sets, and obtain the sub-centers of the tail category.

[0035] In this embodiment, a subset of initial feature data belonging to the tail category is selected from the initial feature dataset, and the subset of initial feature data is divided into n1 sample sets. The sub-distribution corresponding to each sample set is used as the sub-center of the tail category.

[0036] Step S2023: Calculate the weight of each sub-center in the tail category based on the center of each head category.

[0037] The specific steps of step S2023 are as follows: Step S20231: Based on the sample set corresponding to each sub-center, calculate the mean vector, covariance matrix and cluster weight of each sub-center.

[0038] In this embodiment, a weighted average is performed on all sample feature vectors of each sub-center to obtain the mean vector of each sub-center. The difference term between all sample feature vectors of each sub-center and the mean vector of each sub-center is calculated to obtain the covariance matrix of each sub-center. Based on the proportion of the number of samples in the sample set of each sub-center to the total number of samples in the initial feature vector subset of the tail category, the cluster weight of each sub-center is calculated.

[0039] Step S20232: Calculate the similarity between the mean vector of each sub-center and the mean vector of each head category center, and filter the head categories based on the similarity to obtain the reference head categories.

[0040] In this embodiment, a first similarity threshold is preset. When the similarity between the mean vector of the sub-center and the mean vector of the head category center is greater than or equal to the first similarity threshold, the head category is used as the reference head category of the sub-center, and n2 reference head categories are selected for each sub-center of the tail category.

[0041] Step S20233: Calculate the weight of each sub-center in the tail category based on the cluster weight, covariance matrix, and similarity between each sub-center and the reference head category.

[0042] In this embodiment, the cluster weight of each subcenter is used as the basic weight. The distribution stability factor of each subcenter is calculated based on the covariance matrix of each subcenter. The distribution stability factor is multiplied by the basic weight to obtain the covariance correction weight.

[0043] The similarity between each subcenter and the reference head category is normalized to obtain the similarity factor of each subcenter. The similarity factor and the covariance correction weight are multiplied to obtain the comprehensive weight of each subcenter. The comprehensive weight of each subcenter is normalized to obtain the weight of each subcenter in the tail category.

[0044] Step S203: Based on the sub-centers and their weights, merge and optimize the tail categories to obtain the true feature centers of the tail categories.

[0045] The specific steps of step S203 are as follows: Step S2031: Based on each subcenter and its weight, perform preliminary merging of the tail categories, calculate the preliminary mean vector and the preliminary covariance matrix, and obtain the preliminary merging result.

[0046] The specific steps of step S2031 are as follows: Step S20311: Based on the weights of each sub-center, perform a weighted average of the mean vectors of each sub-center to obtain a preliminary mean vector.

[0047] In this embodiment, the mean vectors of each sub-center are weighted and summed based on their respective weights to obtain a weighted result. The weighted result is then divided by the sum of the weights of each sub-center to obtain the preliminary mean vector of the tail category.

[0048] Step S20312: Calculate the difference term between the mean vector of each sub-center and the preliminary mean vector. Based on the difference term and the weight of each sub-center, perform weighted merging of the covariance matrices of each sub-center to obtain the preliminary covariance matrix. The preliminary covariance matrix represents the overall center position of the tail category in the feature space.

[0049] In this embodiment, the covariance matrix of each subcenter is offset and corrected based on the difference term and the weight of each subcenter to obtain the corrected covariance matrix.

[0050] Based on the weights of each subcenter, the corrected covariance matrices are weighted and merged to obtain the preliminary covariance matrix of the tail category. The preliminary covariance matrix represents the degree of dispersion of the overall distribution of the tail category.

[0051] Step S20313: Combine the preliminary mean vector and the preliminary covariance matrix to obtain the preliminary merged result.

[0052] In this embodiment, the preliminary mean vector and the preliminary covariance matrix are combined to obtain a preliminary merging result, which can represent the central position and distribution state of the tail category in the feature space.

[0053] like Figure 3 As shown, step S2032: Based on the preliminary merging results, morphological constraints are applied to the preliminary covariance matrix to obtain the constrained merging results.

[0054] The specific steps of step S2032 are as follows: Step S20321: Map the initial covariance matrix to the symmetric positive definite matrix manifold space to obtain the mapped covariance matrix.

[0055] In this embodiment, the initial covariance matrix is ​​mapped from Euclidean space to a symmetric positive definite matrix manifold space through matrix logarithmic mapping to obtain the corresponding mapped covariance matrix. The mapped covariance matrix is ​​used to ensure the positive definiteness of the initial covariance matrix and to provide a more geometrically constrained metric basis in the manifold space.

[0056] Step S20322: In the symmetric positive definite matrix manifold space, each subcenter of the tail category is mapped to a causal node. Based on the preset causal relationship rules, the neighbor set of each causal node is determined, and causal edges are constructed between the causal nodes.

[0057] In this embodiment, each sub-center of the tail category is mapped to the corresponding mapping covariance matrix, and each sub-center is mapped to a causal node in the manifold space. Based on the preset causal relationship rules, the mutual dependence between each causal node is calculated, the neighbor set of each causal node is determined, and a causal edge is established between any two causal nodes that have a causal dependency relationship. The causal relationship rules are set by those skilled in the art according to the actual situation.

[0058] Step S20323: Calculate the preliminary influence coefficient of each causal edge based on the mapping covariance matrix and the neighbor set. Normalize the preliminary influence coefficient based on the causal flow constraint to obtain the preliminary causal weight.

[0059] In this embodiment, based on the mapping covariance matrix corresponding to the causal nodes at both ends of each causal edge, and combined with the distribution relationship of the neighbor set, the preliminary influence coefficient of the causal edge is calculated. The preliminary influence coefficient represents the importance of each causal edge in the causal propagation process.

[0060] The initial influence coefficients of all causal edges are treated as a whole, and the initial influence coefficients of each causal edge are normalized based on the causal flow constraint to obtain the initial causal weights corresponding to each causal edge. The causal flow constraint is based on the covariance matrix and the neighbor set, and sets the maximum distributable influence for each causal edge. According to the weight of the sub-center, the distributable influence is evenly distributed among the outgoing edges at each causal node while maintaining quota conservation.

[0061] Step S20324: Based on the initial causal weights, construct the causal adjustment function to obtain the adjustment operator. Iterate and update the mapping covariance matrix through the adjustment operator until the mapping covariance matrix meets the preset update termination condition. The iteration is completed, and the optimized covariance matrix is ​​obtained.

[0062] The specific steps of step S20324 are as follows: Step S203241: Construct a tail-center causal graph based on causal nodes and causal edges.

[0063] In this embodiment, causal nodes and corresponding causal edges are combined to form a topological structure containing all causal nodes and corresponding connections, and a tail sub-center causal graph is constructed. The tail sub-center causal graph represents the overall distribution state of tail category sub-centers under causal relationship constraints.

[0064] Step S203242: Based on the mapping covariance matrix, extract the mapping covariance submatrix corresponding to each pair of subcenters in the causal graph of the tail subcenter.

[0065] In this embodiment, the row and column index positions of each pair of subcenters in the tail subcenter causal graph are determined in the mapping covariance matrix, and based on the index of each pair of subcenters, the corresponding local submatrix is ​​extracted from the mapping covariance matrix to obtain a set of mapping covariance submatrices corresponding to the tail subcenter causal graph structure.

[0066] Step S203243: Calculate the sparsity of each sub-center based on the sample set corresponding to each sub-center and the mean vector of each sub-center.

[0067] In this embodiment, based on the sample set corresponding to each sub-center and the mean vector of each sub-center, the difference between all sample features and the mean vector in the sample set of each sub-center is calculated. The sum of squares of the differences is statistically analyzed to obtain the variance measure of each sub-center in the feature space, and the variance measure is used as sparsity.

[0068] Step S203244: Based on the mapping covariance submatrix, sparsity, and preliminary causal weights, calculate the causal regulation subfunction for each pair of subcenters.

[0069] In this embodiment, based on the mapping covariance submatrix of each pair of subcenters, the difference measure between the covariance matrices corresponding to each pair of subcenters is calculated, and the difference measure is normalized to obtain the normalized difference measure of the pair of subcenters.

[0070] Based on the sparsity corresponding to each pair of sub-centers, the sparsity difference of each pair of sub-centers is calculated. The sparsity difference and the normalized difference measure are input into the hyperbolic tangent function to obtain the sparsity adjustment factor of the pair of sub-centers.

[0071] Multiply the sparsity adjustment factor and the corresponding initial causal weight to obtain the adjusted causal weight of the pair of subcenters, and use the adjusted causal weight as the causal adjustment subfunction of each pair of subcenters.

[0072] Step S203245: Based on the causal adjustment sub-function, the weights and mean vectors of each sub-center, calculate the preliminary adjustment sub-matrix for each pair of sub-centers, and perform constraint processing on the preliminary adjustment sub-matrix to obtain the constrained adjustment sub-matrix.

[0073] In this embodiment, the causal regulation function of each pair of subcenters is multiplied with the weight of the corresponding subcenter to obtain the regulation strength. An outer product matrix is ​​constructed using the mean vector of the corresponding subcenter. The outer product function is weighted based on the regulation strength to obtain the preliminary regulation sub-matrix of each pair of subcenters.

[0074] Step S203246: Based on the tail subcenter causal graph, the constraint adjustment submatrix is ​​topologically ordered and combined, and the causal adjustment function is obtained by weighted accumulation, which serves as the adjustment operator.

[0075] In this embodiment, based on the tail subcenter causal graph, the causal nodes and corresponding causal edges are topologically sorted to obtain the combination order of each constraint adjustment submatrix.

[0076] Based on the combination order, the corresponding constraint adjustment sub-matrices are selected sequentially, and each constraint adjustment sub-matrix is ​​weighted by the preliminary causal weights corresponding to each causal edge. All weighted constraint adjustment sub-matrices are summed to obtain the causal adjustment function, which is then used as the adjustment operator.

[0077] Step S203247: Iteratively update the mapping covariance matrix by adjusting the operator until the mapping covariance matrix meets the preset update termination condition. The iteration is completed, and the optimized covariance matrix is ​​obtained.

[0078] In this embodiment, the mapped covariance matrix after iteration is used as the optimized covariance matrix. The preset update termination conditions include the difference metric between the mapped covariance matrices before and after the update being less than a preset first update threshold, or the number of iterations reaching a preset first iteration threshold. The first update threshold and the first iteration threshold are set by those skilled in the art according to the actual situation.

[0079] Step S20325: Map the optimized covariance matrix back to the initial space to obtain the constraint covariance matrix. Combine the constraint covariance matrix and the initial mean vector to obtain the constraint merging result.

[0080] In this embodiment, a matrix exponential mapping operation is performed on the optimized covariance matrix to map the optimized covariance matrix back from the symmetric positive definite matrix manifold space to the initial Euclidean space, thereby obtaining the constrained covariance matrix.

[0081] Step S2033: Iteratively update the merging parameters in the preliminary merging process and the morphological constraint process until the constraint merging result meets the preset iteration termination condition, and the iteration is completed, obtaining the true feature center of the tail category.

[0082] In this embodiment, the iteration termination conditions include convergence threshold conditions, loss function conditions, and iteration round number conditions, which can be set by those skilled in the art according to the actual situation.

[0083] Step S3: Generate an updated feature dataset for the tail category based on the true feature centers, and adjust the first classifier after preliminary training by merging the updated feature dataset and the initial feature dataset to obtain a balanced feature dataset.

[0084] In this embodiment, the tail category samples are used to generate and expand features by using the true feature centers of the tail category to obtain an updated feature dataset for the tail category. The updated feature dataset and the initial feature dataset are then merged to obtain a balanced feature dataset.

[0085] The balanced feature dataset is input into the first classifier after initial training, and the first classifier is adjusted and retrained.

[0086] Step S4: Collect the imbalanced image dataset to be classified, input the imbalanced image dataset to be classified into the first feature extraction network after preliminary training and the first classifier after adjustment, and obtain the classification result.

[0087] In this embodiment, after collecting the imbalanced image dataset to be classified, the data needs to be preprocessed. The preprocessing specifically includes: outlier handling, data standardization, missing value imputation, time alignment, and feature smoothing.

[0088] Example 2 Please see Figure 4 The present invention provides an embodiment of an image classification system for imbalanced training datasets, comprising: a preliminary training module, a center determination module, a classification adjustment module, and an image classification module.

[0089] The preliminary training module is used to collect an imbalanced image dataset and use the imbalanced image dataset to perform preliminary training on the first feature extraction network and the first classifier.

[0090] The center determination module is used to collect a clean image dataset, extract an initial feature dataset based on the first feature extraction network after preliminary training and the clean image dataset, and calculate the true feature center of the tail category.

[0091] The classification adjustment module generates an updated feature dataset for the tail category based on the true feature centers, and adjusts the first classifier after initial training by merging the updated feature dataset and the initial feature dataset to obtain a balanced feature dataset.

[0092] The image classification module is used to collect an imbalanced image dataset to be classified, and input the imbalanced image dataset to be classified into the first feature extraction network after preliminary training and the first classifier after adjustment to obtain the classification result.

[0093] In addition, the parts of the technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art have not been described in detail, so as to avoid excessive elaboration.

[0094] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. An image classification method for imbalanced training datasets, characterized in that, include: An imbalanced image dataset is collected, and the first feature extraction network and the first classifier are initially trained using the imbalanced image dataset. A clean image dataset is collected. Based on the first feature extraction network after preliminary training and the clean image dataset, an initial feature dataset is extracted, and the true feature center of the tail category is calculated. The clean image dataset is divided into head category and tail category according to the number of samples in each category. The true feature center is obtained by modeling the tail category with multiple sub-centers and constructing a causal graph of tail sub-centers, and morphological constraints are applied to the covariance matrix. An updated feature dataset for the tail category is generated based on the real feature centers. The first classifier after initial training is adjusted by merging the updated feature dataset and the initial feature dataset to obtain a balanced feature dataset. An imbalanced image dataset to be classified is collected, and the dataset is input into a first feature extraction network that has been pre-trained and a first classifier that has been adjusted to obtain the classification result.

2. The image classification method for imbalanced training datasets according to claim 1, characterized in that, The process of acquiring an imbalanced image dataset and using it to perform preliminary training on a first feature extraction network and a first classifier includes: An imbalanced image dataset with a long-tailed distribution is collected, and data augmentation is performed on the imbalanced image dataset to obtain an augmented dataset, wherein each sample in the imbalanced image dataset has a corresponding true class label; The augmented dataset is input into the first feature extraction network and the first classifier to obtain the predicted probability distribution of each sample in the augmented dataset for each category. Based on the predicted probability distribution of each sample and the target distribution corresponding to the true class label, the cross-entropy loss is calculated, and the parameters of the first feature extraction network and the first classifier are iteratively updated until the parameters after iteration meet the preset iteration termination condition. The iteration is then completed, and the first feature extraction network and the first classifier after preliminary training are obtained.

3. The image classification method for imbalanced training datasets according to claim 2, characterized in that, The collected clean image dataset, based on the pre-trained first feature extraction network and the clean image dataset, extracts an initial feature dataset and calculates the true feature centers of the tail category, including: Collect a clean image dataset and input the clean image dataset into the first feature extraction network after preliminary training to obtain the initial feature datasets of each category in the clean image dataset. The tail category is modeled based on the initial feature dataset to obtain each sub-center, and the weight of each sub-center in the tail category is determined. Based on the sub-centers and their weights, the tail categories are merged and optimized to obtain the true feature centers of the tail categories.

4. The image classification method for imbalanced training datasets according to claim 3, characterized in that, The process of modeling the tail category based on the initial feature dataset to obtain each sub-center and determining the weight of each sub-center in the tail category includes: Based on the initial feature dataset, the mean vector and covariance matrix of each head category in the clean image dataset are calculated to obtain the center of each head category; Clustering is performed on the initial feature dataset of the tail category to divide the initial feature dataset of the tail category into sample sets, thereby obtaining the sub-centers of the tail category, and the sub-centers and sample sets correspond one-to-one. Based on the center of each head category, calculate the weight of each sub-center in the tail category.

5. The image classification method for imbalanced training datasets according to claim 4, characterized in that, The calculation of the weights of each sub-center in the tail category based on the center of each head category includes: Based on the sample set corresponding to each subcenter, calculate the mean vector, covariance matrix and cluster weight of each subcenter; Calculate the similarity between the mean vector of each sub-center and the mean vector of each head category center, and filter the head categories based on the similarity to obtain the reference head categories; The weights of each subcenter in the tail category are calculated based on the cluster weights, covariance matrix, and similarity between each subcenter and the reference head category.

6. The image classification method for imbalanced training datasets according to claim 5, characterized in that, Based on the sub-centers and their weights, the tail categories are merged and optimized to obtain the true feature centers of the tail categories, including: Based on each subcenter and its weight, the tail categories are initially merged, and the initial mean vector and covariance matrix are calculated to obtain the initial merging result. Based on the preliminary merging results, the preliminary covariance matrix is ​​subjected to morphological constraints to obtain the constrained merging results. The merging parameters in the initial merging process and the morphological constraint process are iteratively updated until the constraint merging result meets the preset iteration termination condition. The iteration is then completed, and the true feature center of the tail category is obtained.

7. The image classification method for imbalanced training datasets according to claim 6, characterized in that, The process involves preliminary merging of tail categories based on each sub-center and its weight, calculating a preliminary mean vector and a preliminary covariance matrix to obtain preliminary merging results, including: Based on the weights of each subcenter, a weighted average is performed on the mean vectors of each subcenter to obtain a preliminary mean vector; Calculate the difference between the mean vector of each subcenter and the preliminary mean vector. Based on the difference and the weight of each subcenter, perform weighted merging of the covariance matrices of each subcenter to obtain the preliminary covariance matrix. By combining the initial mean vector and the initial covariance matrix, a preliminary merging result is obtained.

8. The image classification method for imbalanced training datasets according to claim 7, characterized in that, The preliminary covariance matrix is ​​subjected to morphological constraints based on the preliminary merging results to obtain constrained merging results, including: The initial covariance matrix is ​​mapped to the space of a symmetric positive definite matrix manifold to obtain the mapped covariance matrix; In the symmetric positive definite matrix manifold space, each subcenter of the tail category is mapped to a causal node. Based on the preset causal relationship rules, the neighbor set of each causal node is determined, and causal edges are constructed between the causal nodes. Based on the mapping covariance matrix and the neighbor set, the preliminary influence coefficient of each causal edge is calculated. The preliminary influence coefficient is then normalized based on the causal flow constraint to obtain the preliminary causal weight. Based on the initial causal weights, a causal adjustment function is constructed to obtain the adjustment operator. The mapping covariance matrix is ​​iteratively updated through the adjustment operator until the mapping covariance matrix meets the preset update termination condition. The iteration is completed, and the optimized covariance matrix is ​​obtained. The optimized covariance matrix is ​​mapped back to the initial space to obtain the constrained covariance matrix. The constrained covariance matrix and the initial mean vector are combined to obtain the constrained merging result.

9. The image classification method for imbalanced training datasets according to claim 8, characterized in that, The process of constructing a causal adjustment function based on preliminary causal weights to obtain an adjustment operator includes: Construct a tail-centered causal graph based on causal nodes and causal edges; Based on the mapping covariance matrix, extract the mapping covariance submatrix corresponding to each pair of subcenters in the causal graph of the tail subcenters; Based on the sample set corresponding to each subcenter and the mean vector of each subcenter, the sparsity of each subcenter is calculated. Based on the mapping covariance submatrix, sparsity, and preliminary causal weights, calculate the causal regulation subfunction for each pair of subcenters; Based on the causal regulation function, the weights and mean vectors of each subcenter, the preliminary regulation matrix for each pair of subcenters is calculated, and the preliminary regulation matrix is ​​constrained to obtain the constrained regulation matrix. The constraint regulator matrix is ​​topologically ordered and combined based on the tail subcenter causal graph. The causal regulation function is obtained by weighted accumulation and used as the regulation operator.

10. An image classification system for imbalanced training datasets, used to implement the image classification method for imbalanced training datasets according to any one of claims 1-9, characterized in that, include: The preliminary training module is used to collect an imbalanced image dataset and use the imbalanced image dataset to perform preliminary training on the first feature extraction network and the first classifier. The center determination module is used to collect a clean image dataset. Based on the first feature extraction network after preliminary training and the clean image dataset, the initial feature dataset is extracted, and the true feature center of the tail category is calculated. The classification adjustment module is used to generate an updated feature dataset for the tail category based on the true feature centers, and to adjust the first classifier after initial training by merging the updated feature dataset and the initial feature dataset to obtain a balanced feature dataset. The image classification module is used to collect an imbalanced image dataset to be classified, and input the imbalanced image dataset to be classified into the first feature extraction network after preliminary training and the first classifier after adjustment to obtain the classification result.

Images