Method and device for eliminating long tail effect of electric power inspection image data
By determining the category similarity in the power inspection image data and weighting the loss function, the problem of long-tail effect in the power inspection image data is solved, and the recognition accuracy and generalization ability of the model are improved.
Patent Information
- Application Number
- CN202410851736.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2024-11-01
AI Technical Summary
There is a long tail effect in the power inspection image data, resulting in poor performance of the model in a few categories, affecting its generalization ability and overall performance.
By obtaining the long-tail image training dataset, the similarity of each category is determined, and the loss function weight is calculated based on the similarity, and the parameters of the power inspection image recognition model are updated to ensure that the model pays more attention to the small number of samples and is easily predicted during the training process. The wrong category is trained through the weighted loss function to eliminate the long-tail effect.
The model's recognition accuracy and overall recognition accuracy of a few categories are improved, the model's generalization ability is enhanced, and the performance imbalance caused by the long-tail effect is solved.
Smart Images

Figure CN118887487A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for eliminating the long-tail effect of power inspection image data. Background Art
[0002] Power inspections often generate a large amount of image data, including images of normal and abnormal conditions. By classifying power inspection images through image classification and recognition models, the specific conditions of the power inspection can be quickly identified.
[0003] However, traditional image classification models usually require a large number of labeled samples for training, but in practical applications, it is often difficult to obtain a large number of labeled samples, especially in power inspection scenarios. This leads to the small sample problem, that is, insufficient training data, which limits the performance of the model. Small sample incremental learning aims to solve this problem. It allows the model to continue learning with a small number of new samples, but existing methods often face challenges such as overfitting and forgetting old knowledge.
[0004] Among them, the long tail effect is an important issue in small sample incremental learning. In real-world datasets, the number of samples in each category is often unbalanced. The minority category may have only a few samples, while the majority category has a large number of samples. This imbalance leads to poor performance of the model on the minority category, which is the long tail effect. The existence of the long tail effect seriously affects the generalization ability and overall performance of the model.
[0005] Therefore, how to eliminate the long-tail effect in image data is a technical problem that needs to be solved in power inspection image classification and recognition. Summary of the invention
[0006] The embodiment of the present invention provides a method and device for eliminating the long tail effect of power inspection image data, so as to solve the problem of eliminating the long tail effect in image data.
[0007] In a first aspect, an embodiment of the present invention provides a method for eliminating the long-tail effect of power inspection image data, comprising:
[0008] Obtain a long-tail image training data set; wherein the long-tail image training data set includes power inspection image samples of multiple categories;
[0009] Determine the similarity of power inspection image samples of each category based on the power inspection image recognition model after small sample incremental learning;
[0010] Based on the similarity of power inspection image samples of each category, the loss function weight corresponding to each category is determined;
[0011] Based on the long-tail image training data set, the loss function weight and loss function corresponding to each category, the parameters of the power inspection image recognition model are updated to obtain the power inspection image recognition model that eliminates the long-tail effect.
[0012] In a possible implementation, the power inspection image recognition model includes a feature extractor and a classifier; determining the similarity of power inspection image samples of each category based on the power inspection image recognition model after small sample incremental learning includes:
[0013] The sample features of each power inspection image sample in the long-tail image training data set are extracted by a feature extractor;
[0014] For each category, determine the central feature of the category among the sample features of the category;
[0015] Based on the similarity measurement algorithm, the similarity between each central feature is calculated, and each central feature is sorted and numbered to form a central feature set so that the similarity between adjacent central features is the highest, and the number of each central feature is used as the similarity index of the category corresponding to the central feature.
[0016] In a possible implementation, for each category, a central feature of the category is determined from each sample feature of the category, including:
[0017] Calculate the average value of each sample feature of the category as the central feature of the category.
[0018] In a possible implementation, based on the similarity of power inspection image samples of each category, the loss function weight corresponding to each category is determined, including:
[0019] Taking the serial number of the first category as the center, a sliding window is used to select multiple local impact categories in the central feature set, and based on the number of power inspection image samples of each local impact category, the number of power inspection image samples of the first category, and the similarity between each local impact category and the first category, the local impact of the first category is calculated; wherein the first category is any category;
[0020] Determine the global impact on the first category based on the maximum number of power inspection image samples of each category and the number of power inspection image samples of the first category;
[0021] A loss function weight of the first category is determined based on the local impact and the global impact suffered by the first category.
[0022] In a possible implementation, the loss function weight corresponding to the first category is:
[0023]
[0024] in, For Category The corresponding loss function weight is, is the number of the first category, n max is the maximum number of samples of power inspection images of each category, For Category The number of power inspection image samples, W is the sliding window size, j is the number of local impact categories, n j is the number of power inspection image samples of category j, is the class j and the class The similarity between them, a is a hyperparameter.
[0025] In a possible implementation, based on the long-tail image training data set, the loss function weight and the loss function corresponding to each category, the parameters of the power inspection image recognition model are updated to obtain the power inspection image recognition model that eliminates the long-tail effect, including:
[0026] The loss function is weighted based on the weights corresponding to each category to obtain a weighted loss function;
[0027] Based on the long-tail image training data set and the weighted loss function, the parameters of the power inspection image recognition model are updated to obtain a power inspection image recognition model that eliminates the long-tail effect.
[0028] In one possible implementation, the weighted loss function is:
[0029]
[0030] Among them, Loss swr is the loss function value, N is the number of power inspection image samples in the long-tail image training dataset, For Category The weight of The power inspection image recognition model is The predicted output vector of the power inspection image sample i, z i,j For vector The jth component in , C is the number of categories.
[0031] In a possible implementation manner, it is characterized in that before determining the similarity of power inspection image samples of each category based on the power inspection image recognition model that has undergone small sample increment learning, the method further includes:
[0032] Obtaining an initial power inspection image recognition model;
[0033] The initial power inspection image recognition model is trained based on the base class training set to obtain a preliminarily trained power inspection image recognition model;
[0034] Based on the small sample class training set, small sample incremental learning is performed on the initially trained power inspection image recognition model to obtain a power inspection image recognition model that has undergone small sample incremental learning.
[0035] In a second aspect, an embodiment of the present invention provides a device for eliminating the long-tail effect of power inspection image data, comprising:
[0036] An acquisition module is used to acquire a long-tail image training data set; wherein the long-tail image training data set includes power inspection image samples of multiple categories;
[0037] A comparison module, used for determining the similarity of power inspection image samples of various categories based on a power inspection image recognition model that has undergone small sample incremental learning;
[0038] A weighting module is used to determine the loss function weight corresponding to each category based on the similarity of power inspection image samples of each category;
[0039] The updating module is used to update the parameters of the power inspection image recognition model based on the long-tail image training data set, the loss function weight and loss function corresponding to each category, so as to obtain the power inspection image recognition model that eliminates the long-tail effect.
[0040] The embodiment of the present invention provides a method and device for eliminating the long-tail effect of power inspection image data. On the basis of small sample incremental learning, the similarity of power inspection image samples of each category in the long-tail image training data set is determined in view of the characteristic that the number of samples of a minority category in the long-tail image training data set is far less than that of the majority category. The weight value is determined by calculating the ratio of the number of samples of each category to the overall number of samples and combining the similarity between categories to ensure that the model pays more attention to those categories with a small number of samples and prone to misprediction during the training process, so that the model can learn the features of different categories more balanced, improve the overall recognition accuracy, and eliminate the long-tail effect in the model. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1 It is a flow chart of a method for eliminating the long-tail effect of power inspection image data provided by an embodiment of the present invention;
[0043] Figure 2 is a flow chart of a method for eliminating the long-tail effect of power inspection image data provided by another embodiment of the present invention;
[0044] Figure 3 It is a structural schematic diagram of a device for eliminating the long-tail effect of power inspection image data provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0045] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present invention. However, it should be clear to those skilled in the art that the present invention may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present invention.
[0046] In order to make the purpose, technical solutions and advantages of the present invention more clear, specific embodiments will be described below in conjunction with the accompanying drawings.
[0047] Figure 1 The implementation flow chart of the method for eliminating the long tail effect of power inspection image data provided by the embodiment of the present invention is described in detail as follows:
[0048] Step 101, obtaining a long-tail image training data set; wherein the long-tail image training data set includes power inspection image samples of multiple categories.
[0049] In this embodiment, the long-tail image training data set D is represented by D={(xi, yi)|i{1, 2, ..., N}}. Wherein, N represents the total number of samples contained in the data set, x represents the sample, and y represents the category, i.e., the label. i Denotes the i-th sample, with y i Indicates the label corresponding to the th sample, y ∈ {1, 2, ..., C}.
[0050] Generally, for a long-tail image dataset, the category subscript y is usually sorted in ascending order from the largest to the smallest number of samples, that is, n1≥n y ≥n C .
[0051] Step 102 : determining the similarity of power inspection image samples of various categories based on the power inspection image recognition model that has undergone small sample incremental learning.
[0052] In this embodiment, small sample incremental learning is a key step, which allows the image recognition model to continue learning when a small number of samples are added. The present invention improves the limitations of small sample incremental learning in dealing with the long tail effect in image classification.
[0053] The core idea of small sample incremental learning is to allow the model to continue learning with only a small number of new category samples while avoiding forgetting old knowledge. In this embodiment, we have adopted a series of technical means to ensure the learning effect of the model. First, we introduced an adaptive learning rate adjustment strategy to dynamically adjust the learning rate according to the training status of the model to avoid overfitting and forgetting problems. Secondly, we used memory replay technology to consolidate the model's old knowledge memory by replaying historical sample data. The combination of these technologies enables the model to maintain the ability to recognize old categories while learning new categories.
[0054] Step 103: Determine the loss function weight corresponding to each category based on the similarity of the power inspection image samples of each category.
[0055] In this embodiment, in the long-tail data distribution, the number of samples in the minority categories is far less than that in the majority categories, resulting in insufficient attention of the model to these minority categories during the training process. In order to solve this problem, this embodiment proposes a dynamic adjustment mechanism for category weights. Categories with high weights will receive more attention in subsequent model training, thereby improving the model's ability to recognize long-tail categories. Specifically, the weight value is determined by calculating the ratio of the number of samples in each category to the total number of samples, and combining the similarity between categories. This method can ensure that the model pays more attention to categories with a small number of samples and prone to prediction errors during the training process.
[0056] Step 104 , based on the long-tail image training data set, the loss function weight and the loss function corresponding to each category, update the parameters of the power inspection image recognition model to obtain the power inspection image recognition model that eliminates the long-tail effect.
[0057] In this embodiment, the similarity between samples of different categories is evaluated to determine the importance of each category. According to the evaluation results, a weight is assigned to each long-tail effect category, and the size of the weight is positively correlated with the importance of the long-tail effect category. In combination with the long-tail image training data set, the loss function weight corresponding to each category and the loss function, the parameters of the power inspection image recognition model are updated to ensure that the model pays more attention to those categories with a small number of samples and prone to prediction errors during the training process.
[0058] Based on small sample incremental learning, the embodiment of the present invention determines the similarity of power inspection image samples of each category in the long-tail image training data set, targeting the characteristic that the number of samples of minority categories in the long-tail image training data set is far less than that of majority categories. The weight value is determined by calculating the ratio of the number of samples in each category to the overall number of samples and combining the similarity between categories, ensuring that the model pays more attention to categories with small sample numbers and prone to misprediction during the training process, so that the model can learn the features of different categories more balancedly, improve the overall recognition accuracy, and eliminate the long-tail effect in the model.
[0059] In a possible implementation, the power inspection image recognition model includes a feature extractor and a classifier; determining the similarity of power inspection image samples of each category based on the power inspection image recognition model after small sample incremental learning includes:
[0060] The sample features of each power inspection image sample in the long-tail image training data set are extracted by a feature extractor;
[0061] For each category, determine the central feature of the category among the sample features of the category;
[0062] Based on the similarity measurement algorithm, the similarity between each central feature is calculated, and each central feature is sorted and numbered to form a central feature set so that the similarity between adjacent central features is the highest, and the number of each central feature is used as the similarity index of the category corresponding to the central feature.
[0063] In this embodiment, the similarity between samples of different categories is evaluated to determine the importance of each category. According to the evaluation results, a weight is assigned to each long-tail effect category, and the size of the weight is positively correlated with the importance of the long-tail effect category. The specific implementation steps are shown below, based on the sliding window and the features of the samples, the feature similarity is modeled, so as to perform subsequent weight calculations:
[0064] Input: long-tail distribution training dataset D, sliding window size w, hyperparameter a
[0065] Output: Model parameters
[0066] 1: The first step is feature similarity modeling
[0067] 2: Train the initial model based on D and the cross entropy loss function, and use the feature extractor of the model to extract the feature representation of all samples in D to obtain the set S of sample features
[0068] 3: Determine the set V of cluster centers based on the set S
[0069] 4: Reorder V according to similarity
[0070] 5: The second step is weight calculation
[0071] 6: Renumber class labels
[0072] 7: Calculate the weight of each class
[0073] 8: Construct a weighted loss function to update model parameters
[0074] The first step of feature similarity modeling is to train an initial power inspection image recognition model based on the long-tail image training dataset D to represent the sample features in the dataset D.
[0075] Then, the feature extractor in the initial model is used to calculate the feature representation of all samples in D, and a set S = {s1, s2, ..., sN} containing the features of all samples in the training set can be obtained.
[0076] The second step of feature similarity modeling is to sort all categories according to similarity. Since each class has more than one sample feature, all sample features of a class are averaged to obtain the class center that can represent all samples of the class. This calculation is performed once for all categories to obtain a set V containing all class centers, expressed as:
[0077]
[0078] In the formula, (v y , y) represents the class center of the yth category and its corresponding class label, N represents the total number of samples contained in the data set, s represents the feature output of the initial model for the input x, and n y Indicates the number of samples contained in the y-th class.
[0079] After obtaining the set V, the elements in V are sorted according to the similarity measurement algorithm so that the similarity between the two closest class centers is the highest, and the similarity between the two class centers that are farther apart is lower.
[0080] By modeling feature similarity, a set of reordered class labels can be obtained. These class labels are renumbered from 1 to C, and y is used to represent the renumbered labels. For example, the 5th class after renumbering may be the 10th class before renumbering. After renumbering the labels, the 1st class has the lowest similarity with the Cth class, and when y∈{2, 3, ..., C-1}, the yth class has the highest similarity with the y-1th class and the +1th class. At this time, the difference in label numbers can be used as an evaluation indicator of the similarity between classes.
[0081] Since the model tends to predict categories with fewer samples to classes with similar visual appearance and more samples, in order to make the model pay more attention to these categories with fewer samples and easy to be predicted incorrectly, you can assign larger weights to these categories during model training to get more attention from the model.
[0082] In a possible implementation, for each category, a central feature of the category is determined from each sample feature of the category, including:
[0083] Calculate the average value of each sample feature of the category as the central feature of the category.
[0084] In this embodiment, the second step of feature similarity modeling is to sort all categories according to similarity. Since each class has more than one sample feature, all sample features of a category are averaged to obtain a central feature that can represent all samples of the category.
[0085] In a possible implementation, based on the similarity of power inspection image samples of each category, the loss function weight corresponding to each category is determined, including:
[0086] Taking the serial number of the first category as the center, a sliding window is used to select multiple local impact categories in the central feature set, and based on the number of power inspection image samples of each local impact category, the number of power inspection image samples of the first category, and the similarity between each local impact category and the first category, the local impact of the first category is calculated; wherein the first category is any category;
[0087] Determine the global impact on the first category based on the maximum number of power inspection image samples of each category and the number of power inspection image samples of the first category;
[0088] A loss function weight of the first category is determined based on the local impact and the global impact suffered by the first category.
[0089] In this embodiment, the weight is designed by combining the number of samples within a class with the similarity between classes. When calculating the weight of a class, the position of the class in the label space is used as the starting point, and the influence of other classes on the class is calculated one by one from both sides at the same time. The influence is evaluated by the ratio of the number of samples of other classes to the class and the feature similarity between the two classes, while the feature similarity between the two classes is measured by the difference in label numbers.
[0090] The larger the sample number ratio, the smaller the sample number of this class is compared with other classes. The smaller the label number difference, the higher the similarity between this class and other classes. At this time, this class is often more likely to be misclassified into other classes, and a larger weight should be given to this class.
[0091] When calculating the influence of other categories on this category, the influence of distant categories on this category can be ignored, thereby reducing unnecessary calculations. Therefore, in this embodiment, a sliding window is used to limit the scope of objects where similarity works. Since only the local influence of other categories within the window on this category is used at this time, the global influence of the entire data set on this category is not considered. Therefore, the global influence is applied by combining the ratio of the number of samples of the class with the largest number of samples in the class in the entire data set to the number of samples of this class.
[0092] This embodiment first extracts the sample features of the data set, and uses a similarity measurement method to model the similarity of these sample features, and renumbers the labels so that two categories with similar label numbers have greater visual feature similarity. Then, the weight of each class is calculated based on the renumbered labels, and finally the calculated weight is combined with the cross entropy loss function for model training. This weight calculation method can effectively solve the problem that minority class samples tend to be predicted as head class samples with similar visual features, thereby improving the model's recognition accuracy for minority classes and the overall recognition accuracy.
[0093] In a possible implementation, the loss function weight corresponding to the first category is:
[0094]
[0095] in, For Category The corresponding loss function weight is, is the number of the first category, n max is the maximum number of samples of power inspection images of each category, For Category The number of power inspection image samples, W is the sliding window size, j is the number of local impact categories, n j is the number of power inspection image samples of category j, is the class j and the class The similarity between them, a is a hyperparameter.
[0096] In this embodiment, the weight is designed by combining the number of samples within a class with the similarity between classes. When calculating the weight of a class, the position of the class in the label space is used as the starting point, and the influence of other classes on the class is calculated one by one from both sides at the same time. The influence is evaluated by the ratio of the number of samples of other classes to the class and the feature similarity between the two classes, while the feature similarity between the two classes is measured by the difference in label numbers, and the above formula can be obtained.
[0097] In a possible implementation, based on the long-tail image training data set, the loss function weight and the loss function corresponding to each category, the parameters of the power inspection image recognition model are updated to obtain the power inspection image recognition model that eliminates the long-tail effect, including:
[0098] The loss function is weighted based on the weights corresponding to each category to obtain a weighted loss function;
[0099] Based on the long-tail image training data set and the weighted loss function, the parameters of the power inspection image recognition model are updated to obtain a power inspection image recognition model that eliminates the long-tail effect.
[0100] In this embodiment, the power inspection image recognition model can be trained and updated based on the conventional cross entropy method. The loss of each sample in the cross entropy method can be expressed as:
[0101]
[0102] In the formula, z i Represents the model for the i-th input sample x i The predicted output vector, z i ∈R C , z i,j Represents vector z i The j-th component in , j∈{1, 2, ..., C}.
[0103] Combining the softmax cross entropy loss function with dynamically adjusted category weights, a weighted loss function can be constructed. This function adjusts the training loss according to the weights of each category during model training, so that the model pays more attention to categories with higher importance during training. Specifically, when calculating the loss function, the weight value of each category is multiplied by the corresponding loss term to obtain a weighted total loss. By optimizing this weighted loss function, the model can learn the features of different categories more evenly and improve the overall recognition accuracy.
[0104] In one possible implementation, the weighted loss function is:
[0105]
[0106] Among them, Loss swr is the loss function value, N is the number of power inspection image samples in the long-tail image training dataset, For Category The weight of The power inspection image recognition model is The predicted output vector of the power inspection image sample i, z i,j For vector The jth component in , C is the number of categories.
[0107] In this embodiment, after obtaining the weight of each category, the weight is combined with the softmax cross entropy loss function to obtain the above-mentioned weighted loss function.
[0108] The basic model is optimized using the weighted loss function, and new knowledge is continuously learned from a small number of new samples. The weight distribution of the basic model is updated until the expected performance indicators are achieved, thereby improving the recognition ability of the basic model on long-tail category samples.
[0109] In a possible implementation manner, it is characterized in that before determining the similarity of power inspection image samples of each category based on the power inspection image recognition model that has undergone small sample increment learning, the method further includes:
[0110] Obtaining an initial power inspection image recognition model;
[0111] The initial power inspection image recognition model is trained based on the base class training set to obtain a preliminarily trained power inspection image recognition model;
[0112] Based on the small sample class training set, small sample incremental learning is performed on the initially trained power inspection image recognition model to obtain a power inspection image recognition model that has undergone small sample incremental learning.
[0113] In this embodiment, the image recognition model can be any deep learning model suitable for image recognition, such as a convolutional neural network (CNN), etc., which is not specifically limited in this embodiment of the application. The deep learning model is preliminarily trained to obtain a basic model that can be used for image recognition.
[0114] Then, the preliminarily trained image recognition model is subjected to small sample incremental learning. A small number of long-tail category samples are collected, which appear less frequently in the data set. These small samples are used to perform incremental learning on the image recognition model, that is, on the basis of the existing model, the image recognition model learns new knowledge by continuing training. During the incremental learning process, an appropriate learning rate adjustment strategy is adopted to avoid catastrophic forgetting of the original knowledge by the image recognition model.
[0115] Specifically, a small sample incremental learning framework is used to initialize and train the image recognition model.
[0116] The basic model for incremental learning of small sample classes is defined as:
[0117] p=Cω(fθ(x))
[0118] Among them, the image recognition model obtained by preliminary training is composed of feature extractor f θ(·) and a classifier Cω(·), θ and ω represent the parameters of the feature extractor and the classifier respectively, x represents the image to be recognized, and f θ (x) represents the feature extractor f θ (·) The features extracted from image x, Cω(f θ (x)) represents the feature f of the classifier Cω(·) for image x θ (x) is the recognition result.
[0119] In the setting of incremental learning of small-shot classes, there is a series of labeled training sets {D1, D2, …, DN}. Where DN = {(x i ,y i )}i=1 |DN| .x i Represents a sample, and the corresponding category label is y i |DN| represents the total number of samples in the training set DN.
[0120] The first training set D1 is a pre-acquired base class training set, which consists of base classes, and each class has a large number of training samples. The remaining training sets DN, N>1, consist of new classes, i.e., small sample classes, where each new class has only a few training samples. CN represents the category set of the Nth training set DN. For any training set, their category sets do not intersect, i.e.
[0121] The image recognition model needs to be learned on all datasets {D1, D2, …, DN}. Moreover, during the Nth learning, only the current training set DN is available, and the previously learned datasets will not be accessible. After the model is trained on DN, it needs to evaluate all categories {C1∪C2, …, CN-1∪CN} encountered in {D1, D2, …, DN}. When training on new categories, the Cway, K-shot small sample learning setting is adopted, that is, each training set DN, N>1 contains C categories, and each category has only K training data.
[0122] In the small sample class incremental learning problem, the training set of the new class is not available at the beginning, but gradually becomes available for training over time. At the same time, since the number of samples contained in the new class is too small, it is difficult to obtain good feature representation by directly training the model. Therefore, small sample class incremental learning usually includes the following two steps of learning process:
[0123] (1) In the first learning process, the image recognition model will first learn on the existing base class training set D1 = {(xi, yi)} | D1 | i = 1 to obtain a feature extractor and base class classifier with good performance. At this stage, each base class (assuming class b) contains a large number of training samples.
[0124] (2) In the Nth, N>1 learning process, the image recognition model incrementally learns on the gradually emerging small sample class datasets {D2, D3, …, DN}, and continuously fine-tunes and updates the entire model. At this stage, the number of training samples for each new category (assuming n categories) contained in DN (N>1) is very small, generally not more than 20.
[0125] The ultimate goal of incremental learning of small sample classes is that the model will not forget the memory of the old classes it has learned while constantly learning new classes. That is, after learning n new classes, the model can correctly identify b+n class images at the same time.
[0126] In a specific embodiment, the method for eliminating the long tail effect of image data provided by the present application is described in a specific application scenario. Figure 2 , the specific implementation process includes:
[0127] Step 201: Data preparation.
[0128] Select an image dataset with a long tail effect. The dataset contains 1000 categories, and the number of samples in each category ranges from 10 to 10000.
[0129] Step 202: Model initialization.
[0130] Choose a deep learning model suitable for image recognition, such as a convolutional neural network (CNN), and perform preliminary training on it.
[0131] Step 203: small sample incremental learning.
[0132] 100 samples are randomly selected from the long-tail category and the model is incrementally learned with a learning rate of 0.01.
[0133] Step 204: Category importance evaluation.
[0134] Evaluate the similarity between each category in the dataset and assign a weight to each category based on the similarity.
[0135] Step 205: Weighted redistribution.
[0136] During the training of the model, the loss function is weighted according to the assigned weights, with the class with the highest weight accounting for 50% of the loss.
[0137] Step 206: Model optimization and iteration.
[0138] Use the weighted loss function to optimize the model and repeat the above steps until the recognition accuracy of the model on the long-tail category reaches a satisfactory level.
[0139] From the above, it can be seen that the embodiment of the present invention effectively solves the long-tail effect problem in image recognition and classification tasks through the strategy of small sample incremental learning and weighted redistribution, and improves the accuracy and stability of the model. The model is continuously updated using a small number of samples to improve the generalization ability of the model. By weighted adjustment of the loss function, the model pays more attention to minority class samples, thereby improving the recognition ability of minority class samples. In this way, better model performance and generalization ability can be achieved under the conditions of small samples and imbalanced categories. This invention not only has important theoretical value, but also has broad application prospects in practical applications. It can be widely used in various scenarios related to image recognition, and promote the development and application of image classification technology.
[0140] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.
[0141] The following is an embodiment of the device of the present invention. For details not described in detail therein, reference may be made to the corresponding method embodiment described above.
[0142] Figure 3 The structure diagram of the device for eliminating the long tail effect of power inspection image data provided by an embodiment of the present invention is shown. For the convenience of explanation, only the part related to the embodiment of the present invention is shown, which is described in detail as follows:
[0143] like Figure 3 As shown, the device 3 for eliminating the long tail effect of power inspection image data includes:
[0144] An acquisition module 31 is used to acquire a long-tail image training data set; wherein the long-tail image training data set includes power inspection image samples of multiple categories;
[0145] A comparison module 32, used to determine the similarity of power inspection image samples of various categories based on the power inspection image recognition model learned through small sample increments;
[0146] A weighting module 33, used to determine the loss function weight corresponding to each category based on the similarity of the power inspection image samples of each category;
[0147] The updating module 34 is used to update the parameters of the power inspection image recognition model based on the long-tail image training data set, the loss function weight and loss function corresponding to each category, and obtain the power inspection image recognition model that eliminates the long-tail effect.
[0148] In a possible implementation, the power inspection image recognition model includes a feature extractor and a classifier; the comparison module 32 is specifically used for:
[0149] The sample features of each power inspection image sample in the long-tail image training data set are extracted by a feature extractor;
[0150] For each category, determine the central feature of the category among the sample features of the category;
[0151] Based on the similarity measurement algorithm, the similarity between each central feature is calculated, and each central feature is sorted and numbered to form a central feature set so that the similarity between adjacent central features is the highest, and the number of each central feature is used as the similarity index of the category corresponding to the central feature.
[0152] In a possible implementation, the comparison module 32 is specifically configured to:
[0153] Calculate the average value of each sample feature of the category as the central feature of the category.
[0154] In a possible implementation, the weighting module 33 is specifically used to:
[0155] Taking the serial number of the first category as the center, a sliding window is used to select multiple local impact categories in the central feature set, and based on the number of power inspection image samples of each local impact category, the number of power inspection image samples of the first category, and the similarity between each local impact category and the first category, the local impact of the first category is calculated; wherein the first category is any category;
[0156] Determine the global impact on the first category based on the maximum number of power inspection image samples of each category and the number of power inspection image samples of the first category;
[0157] A loss function weight of the first category is determined based on the local impact and the global impact suffered by the first category.
[0158] In a possible implementation, the loss function weight corresponding to the first category is:
[0159]
[0160] in, For Category The corresponding loss function weight is, is the number of the first category, n max is the maximum number of samples of power inspection images of each category, For Category The number of power inspection image samples, W is the sliding window size, j is the number of local impact categories, n j is the number of power inspection image samples of category j, is the class j and the class The similarity between them, a is a hyperparameter.
[0161] In a possible implementation, the updating module 34 is specifically used for:
[0162] The loss function is weighted based on the weights corresponding to each category to obtain a weighted loss function;
[0163] Based on the long-tail image training data set and the weighted loss function, the parameters of the power inspection image recognition model are updated to obtain a power inspection image recognition model that eliminates the long-tail effect.
[0164] In one possible implementation, the weighted loss function is:
[0165]
[0166] Among them, Loss swr is the loss function value, N is the number of power inspection image samples in the long-tail image training dataset, For Category The weight of The power inspection image recognition model is The predicted output vector of the power inspection image sample i, z i,j For vector The jth component in , C is the number of categories.
[0167] In a possible implementation, it is characterized in that the comparison module 32 is further used for:
[0168] Before determining the similarity of power inspection image samples of each category based on the power inspection image recognition model that has undergone small sample increment learning, obtaining an initial power inspection image recognition model;
[0169] The initial power inspection image recognition model is trained based on the base class training set to obtain a preliminarily trained power inspection image recognition model;
[0170] Based on the small sample class training set, small sample incremental learning is performed on the initially trained power inspection image recognition model to obtain a power inspection image recognition model that has undergone small sample incremental learning.
[0171] Based on small sample incremental learning, the embodiment of the present invention determines the similarity of power inspection image samples of each category in the long-tail image training data set, targeting the characteristic that the number of samples of minority categories in the long-tail image training data set is far less than that of majority categories. The weight value is determined by calculating the ratio of the number of samples in each category to the overall number of samples and combining the similarity between categories, ensuring that the model pays more attention to categories with small sample numbers and prone to misprediction during the training process, so that the model can learn the features of different categories more balancedly, improve the overall recognition accuracy, and eliminate the long-tail effect in the model.
[0172] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0173] Those of ordinary skill in the art will appreciate that the templates, units, and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0174] If the module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned embodiments of the method for eliminating the long tail effect of the power inspection image data. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium.
[0175] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.
Claims
1. A method for eliminating the long tail effect of power inspection image data, characterized in that: include: Acquire a long-tail image training data set; wherein the long-tail image training data set includes power inspection image samples of multiple categories; Determine the similarity of power inspection image samples of each category based on the power inspection image recognition model after small sample incremental learning; Based on the similarity of power inspection image samples of each category, the loss function weight corresponding to each category is determined; Based on the long-tail image training data set, the loss function weight and loss function corresponding to each category, the parameters of the power inspection image recognition model are updated to obtain a power inspection image recognition model that eliminates the long-tail effect.
2. The method for eliminating the long tail effect of power inspection image data according to claim 1, characterized in that: The power inspection image recognition model includes a feature extractor and a classifier; The similarity of power inspection image samples of various categories is determined based on the power inspection image recognition model after small sample incremental learning, including: Extracting sample features of each power inspection image sample in the long-tail image training data set by the feature extractor; For each category, determine the central feature of the category among the sample features of the category; Based on the similarity measurement algorithm, the similarity between each central feature is calculated, and each central feature is sorted and numbered to form a central feature set so that the similarity between adjacent central features is the highest, and the number of each central feature is used as the similarity index of the category corresponding to the central feature.
3. The method for eliminating the long tail effect of power inspection image data according to claim 2, characterized in that: For each category, determining the central feature of the category from each sample feature of the category includes: Calculate the average value of each sample feature of the category as the central feature of the category.
4. The method for eliminating the long tail effect of power inspection image data according to claim 2, characterized in that: The method of determining the loss function weight corresponding to each category based on the similarity of the power inspection image samples of each category includes: Taking the serial number of the first category as the center, a sliding window is used to select multiple local impact categories from the central feature set, and based on the number of power inspection image samples of each local impact category, the number of power inspection image samples of the first category, and the similarity between each local impact category and the first category, the local impact of the first category is calculated; wherein the first category is any category; Determine the global impact on the first category based on the maximum value of the number of power inspection image samples of each category and the number of power inspection image samples of the first category; A loss function weight of the first category is determined based on the local impact and the global impact suffered by the first category.
5. The method for eliminating the long tail effect of power inspection image data according to claim 4, characterized in that: The loss function weight corresponding to the first category is: in, For Category The corresponding loss function weight is, is the number of the first category, n max is the maximum number of samples of power inspection images of each category, For Category The number of power inspection image samples, W is the sliding window size, j is the number of local impact categories, n j is the number of power inspection image samples of category j, is the class j and the class The similarity between them, a is a hyperparameter.
6. The method for eliminating the long tail effect of power inspection image data according to claim 1, characterized in that: The method of updating the parameters of the power inspection image recognition model based on the long-tail image training data set, the loss function weight and the loss function corresponding to each category to obtain the power inspection image recognition model that eliminates the long-tail effect includes: The loss function is weighted based on the weights corresponding to each category to obtain a weighted loss function; Based on the long-tail image training data set and the weighted loss function, the parameters of the power inspection image recognition model are updated to obtain a power inspection image recognition model that eliminates the long-tail effect.
7. The method for eliminating the long tail effect of power inspection image data according to claim 6, characterized in that: The weighted loss function is: Among them, Loss swr is the loss function value, N is the number of power inspection image samples in the long-tail image training dataset, For Category The weight of The power inspection image recognition model is The predicted output vector of the power inspection image sample i, z i,j For vector The jth component in , C is the number of categories.
8. The method for eliminating the long tail effect of power inspection image data according to any one of claims 1 to 7, characterized in that: Before determining the similarity of power inspection image samples of various categories based on the power inspection image recognition model after small sample increment learning, the method further includes: Obtaining an initial power inspection image recognition model; The initial power inspection image recognition model is trained based on the base class training set to obtain a preliminarily trained power inspection image recognition model; Based on the small sample class training set, small sample incremental learning is performed on the initially trained power inspection image recognition model to obtain a power inspection image recognition model that has undergone small sample incremental learning.
9. A device for eliminating the long tail effect of power inspection image data, characterized in that: include: An acquisition module is used to acquire a long-tail image training data set; wherein the long-tail image training data set includes power inspection image samples of multiple categories; A comparison module, used for determining the similarity of power inspection image samples of various categories based on a power inspection image recognition model that has undergone small sample incremental learning; A weighting module is used to determine the loss function weight corresponding to each category based on the similarity of power inspection image samples of each category; An updating module is used to update the parameters of the power inspection image recognition model based on the long-tail image training data set, the loss function weight and loss function corresponding to each category, so as to obtain a power inspection image recognition model that eliminates the long-tail effect.
10. The device for eliminating the long tail effect of power inspection image data according to claim 9, characterized in that: The power inspection image recognition model includes a feature extractor and a classifier; the comparison module is specifically used for: Extracting sample features of each power inspection image sample in the long-tail image training data set by the feature extractor; For each category, determine the central feature of the category among the sample features of the category; Based on the similarity measurement algorithm, the similarity between each central feature is calculated, and each central feature is sorted and numbered so that the similarity between adjacent central features is the highest, and the number of each central feature is used as the similarity index of the category corresponding to the central feature.
Citation Information
Cited By
Image recognition method and device based on AR glasses, equipment and medium
CN122290116A
Method and apparatus for recognizing inspection image of power system, device, and storage medium
WO2026137894A1
