An image classification method and device, electronic equipment, and readable storage medium

Abstract

The disclosure provides an image classification method and device, electronic equipment and readable storage medium, and belongs to the technical field of machine learning. The method comprises: updating the weight of a target classifier based on a first classification category. The quantity ratio of the sample categories in a first training set and the sample categories in a second training set is determined, and the loss function value of an image classification model corresponding to the target classifier is calculated based on the quantity ratio. The first training set is a training set corresponding to the first classification category, the second training set is a historical training set of the image classification model, and the second training set is a training set corresponding to a second classification category. The second classification category is different from the first classification category. The backbone network of the image classification model is updated based on the loss function value, the first training set and the second training set until the image classification model converges. Image classification is performed based on the converged image classification model. The image classification method provided by the disclosure can reduce the influence of the data imbalance problem on the training performance of the image classification model and improve the generalization ability of the model.

Description

Technical Field

[0001] This disclosure belongs to the field of machine learning technology, and more specifically, relates to an image classification method and apparatus, electronic device, and readable storage medium. Background Technology

[0002] Currently, deep learning-based AI models have achieved excellent performance in tasks such as image classification, speech recognition, and machine translation. However, most of these models are trained on static, identically distributed datasets, making it impossible for them to adapt or scale their behavior over time. Incremental learning aims to enable trained machine learning models to solve multiple tasks at different times. It deals with learning tasks involving data with inconsistent distributions at multiple points in time, requiring the model to learn different specific tasks at different times with a single set of parameters, while not forgetting previous knowledge.

[0003] However, the drawback of playback-based incremental learning is that it requires additional computational resources and storage space to recall old knowledge. As the number of task types increases, the limited amount of playback data means that the number of old class samples is far less than the number of new class samples. This leads to data imbalance in each incremental learning stage, and the degree of data imbalance gradually intensifies over time, resulting in a decline in the performance of image training models. Therefore, an image classification method is urgently needed. Summary of the Invention

[0004] The purpose of this disclosure is to provide an image classification method, apparatus, electronic device, and readable storage medium to solve the problem of performance degradation of image training models caused by imbalanced training data and improve the generalization of training models.

[0005] A first aspect of this disclosure provides an image classification method, comprising: The weights of the target classifier are updated based on the first classification category.

[0006] The ratio of sample categories in the first training set to the number of sample categories in the second training set is determined, and the loss function value of the image classification model corresponding to the target classifier is calculated based on the ratio. The first training set is the training set corresponding to the first classification category, and the second training set belongs to the historical training set of the image classification model and is the training set corresponding to the second classification category. The second classification category is different from the first classification category.

[0007] Based on the loss function value, the first training set, and the second training set, the parameters of the backbone network of the image classification model are updated until the image classification model converges.

[0008] Image classification is performed based on the converged image classification model.

[0009] A second aspect of this disclosure provides an image classification apparatus, comprising: The weight update module is used to update the weights of the target classifier based on the first classification category.

[0010] The loss function calculation module is used to determine the ratio of sample categories in the first training set to sample categories in the second training set, and calculate the loss function value of the image classification model corresponding to the target classifier based on the ratio. The first training set is the training set corresponding to the first classification category, and the second training set belongs to the historical training set of the image classification model and is the training set corresponding to the second classification category. The second classification category is different from the first classification category.

[0011] The parameter update module is used to update the parameters of the backbone network of the image classification model based on the loss function value, the first training set, and the second training set until the image classification model converges.

[0012] The image classification module is used to classify images based on the converged image classification model.

[0013] A third aspect of this disclosure provides an electronic device including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the image classification method described above.

[0014] A fourth aspect of this disclosure provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the image classification method described above.

[0015] The beneficial effects of the image classification method and apparatus, electronic device, and readable storage medium provided in this disclosure are as follows: First, this embodiment can use the ratio of the number of the first classification category to the number of the second classification category as prior information to dynamically adjust the relative margin between the features of the first classification category and the second classification category, thereby reducing the impact of data imbalance on the training performance of the image classification model and improving the generalization ability of the model.

[0016] Second, this embodiment can reset the classification category by updating the weights of the target classifier, ensuring that the features corresponding to different sample classification categories are evenly distributed in the feature space, avoiding confusion between sample classification categories, and ensuring that the distance between features corresponding to the same sample category is much smaller than the distance between features of different sample categories, thus reserving sufficient feature space for new sample classification categories that may appear in the future.

[0017] Third, this embodiment does not modify the backbone network structure or the playback data sampling method of the image classification model. Therefore, this embodiment can be used in conjunction with other incremental learning methods based on playback and dynamic structures to improve the applicability of the model. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of an image classification method provided in an embodiment of the present disclosure; Figure 2 This is a schematic diagram of an image classification model structure provided in an embodiment of the present disclosure; Figure 3 This is a structural block diagram of an image classification device provided in an embodiment of the present disclosure; Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of the present disclosure. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, so as to provide a thorough understanding of the embodiments of this disclosure. However, those skilled in the art will understand that this disclosure may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this disclosure with unnecessary detail.

[0021] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following description will be provided in conjunction with the accompanying drawings and specific embodiments.

[0022] Please refer to Figure 1 , Figure 1 This is a flowchart illustrating an image classification method provided in an embodiment of the present disclosure. The method includes: S101: Update the weights of the target classifier based on the first classification category.

[0023] In this embodiment, the first training set is the image dataset to be trained, the first classification category refers to all sample classification categories contained in the first training set, and the target classifier refers to the classifier in the image classification model trained on the first training set. The target classifier can classify all sample classification categories learned at different stages, and the target classifier can be an unbiased single-layer linear classifier.

[0024] In the target classifier, each category corresponds to a different weight. Whenever training on the first training set, the weights of the target classifier are updated based on the first category to update its corresponding category. This allows for dynamic adjustment of the model network to adapt to new training tasks. The model network can also be expanded to learn new training tasks, for example, by using more neurons or network layers. As the number of training tasks increases, the model network grows larger, and the feature dimensions extracted by the backbone network of the image classification model also increase.

[0025] S102: Determine the ratio of the number of sample categories in the first training set to the number of sample categories in the second training set, and calculate the loss function value of the image classification model corresponding to the target classifier based on the ratio. The first training set is the training set corresponding to the first classification category, and the second training set belongs to the historical image training set of the image classification model, and is the training set corresponding to the second classification category. The second classification category is different from the first classification category.

[0026] In this embodiment, the first training set is the training set corresponding to the first classification category, and the second training set is the historical training set of the image classification model, that is, the training set corresponding to the second classification category, which is different from the first classification category. The ratio of the number of sample categories in the first and second training sets can be determined based on this ratio, and the loss function value of the image classification model corresponding to the target classifier can be calculated based on this ratio. This loss function value can be calculated using a bounded uniform prototype contrastive loss model, or it can be calculated using other loss functions.

[0027] S103: Update the parameters of the backbone network of the image classification model based on the loss function value, the first training set, and the second training set until the image classification model converges.

[0028] In this embodiment, based on the calculated loss function value, the parameters of the backbone network of the image classification model are updated through the training process of the first training set and the second training set. The parameter update process of the backbone network can be realized through relevant deep learning algorithms until the image classification model converges.

[0029] This embodiment provides a reference example, such as using the SGD (Stochastic Gradient Descent) algorithm to update the backbone network parameters of the image classification model. First, a sample is randomly selected from the first training set. The loss function value and its corresponding gradient of the backbone network parameters are calculated. The gradient value and learning rate are then used to update the backbone network parameters. The update formula is as follows: ,in, This represents the backbone network parameters at stage t+1; This represents the backbone network parameters at stage t; Indicates the gradient value; This represents the learning rate. By setting the convergence criteria or the number of iterations required for the image classification model, if the model reaches the convergence criteria or the specified number of iterations after t stages of updates, the updates stop, indicating that the current image classification model has converged. If the model fails to reach the convergence criteria or the specified number of iterations after t stages of updates, the next stage of updates is performed until the image classification model reaches the convergence criteria or the specified number of iterations.

[0030] S104: Image classification based on the converged image classification model.

[0031] In this embodiment, by continuously updating the backbone network parameters of the image classification model until the image classification model converges, the image classification process can be performed based on the converged image classification model.

[0032] As can be seen from the above, this embodiment can dynamically adjust the image classification model by updating the weights of the target classifier to update its corresponding classification category without changing the original network structure of the image classification model. This allows the model to adapt to new training tasks or expand to new learning tasks, thereby mitigating the impact of data imbalance on the training performance of the image classification model and improving the model's generalization ability.

[0033] In one embodiment of this disclosure, updating the weights of the target classifier based on a first classification category includes: Initialize N first weights. N is the number of first categories, and each first weight corresponds to one first category.

[0034] Orthogonalize the N first weights and multiple second weights to make the first weights orthogonal to each other and the first and second weights orthogonal to each other. Each second weight corresponds to a second classification category.

[0035] The first and second weights after orthogonalization are determined as the weights of the target classifier to update the target classifier weights.

[0036] In this embodiment, the first weight is the weight corresponding to the first classification category, and each first weight corresponds to one first classification category; the second weight is the weight corresponding to the second classification category. Updating the weights of the target classifier based on the first classification category can be detailed as follows: First, N primary weights are initialized, where N represents the number of primary categories. Next, the initialized N primary weights are pairwise orthogonalized, and then each of the initialized N primary weights is pairwise orthogonalized with multiple secondary weights. Finally, all the orthogonalized primary and secondary weights are normalized so that their magnitudes are all 1. These steps update the weights of the target classifier.

[0037] Specifically, relevant algorithms can be used to orthogonalize N first weights and N first weights and multiple second weights. This embodiment provides a reference example, such as using the Gram-Schmidt algorithm to achieve orthogonalization between different weights. Gram-Schmidt orthogonalization is a fundamental theory in linear algebra, providing a method for orthogonalizing linearly independent vector groups based on Euclidean space. Let a set of linearly independent vectors be defined. Find an orthogonal basis corresponding to the linearly independent vector. .choose As the first basis vector, i.e. For each i > 1, calculate the vector With all the orthogonal basis vectors already obtained The projection, and from Subtracting these projections from the original vectors yields new orthogonal basis vectors. That is, for each j < i, calculate the projection. ,in, This allows us to obtain an orthogonal basis corresponding to a set of linearly independent vectors.

[0038] As can be seen from the above, this embodiment can update the weights of the target classifier through orthogonalization processing, thereby improving the scientificity and applicability of the image classification model.

[0039] In one embodiment of this disclosure, the image classification method further includes: In response to a change in the feature dimension of the image classification model backbone network, the weights of the target classifier are updated based on the number of first classification categories.

[0040] In this embodiment, before the image classification model trains on the first training set, it is necessary to first detect the feature dimensions of the backbone network of the image classification model to clarify the weight update process of the current target classifier. If it is detected that the feature dimensions of the backbone network of the image classification model have changed, the weights of the target classifier can be updated based on the number of the first classification categories, so that the weights of the classifier are greater than or equal to the number of the first classification categories.

[0041] This embodiment provides a reference example, where the first training set is set to a dataset containing 100 first classification categories, and the second training set is set to a dataset containing 90 second classification categories. Before training on the first training set, the feature dimensions of the backbone network of the image classification model are detected. If the current feature dimensions are insufficient to directly train the first training set, the weights of the target classifier can be updated by padding with zeros to match the number of first classification categories.

[0042] As can be seen from the above, this embodiment can expand the number of weights of the target classifier by detecting the feature dimensions of the backbone network of the image classification model, thereby reserving space for new classification categories that may appear in the future and improving the model's learning ability.

[0043] In one embodiment of this disclosure, calculating the loss function value of the image classification model corresponding to the target classifier based on the quantity ratio includes: The loss function value of the image classification model corresponding to the target classifier is calculated using the first formula. The first formula is:

[0044] in, This represents the value of the loss function. This represents the feature corresponding to the i-th category. This indicates the first weight. This indicates the number of first-weighted values. This indicates the second weight. This indicates the number of second weights. It represents a feature set. This represents the set of weights. This represents a metric function. This represents a temperature hyperparameter. Indicates quantity ratio.

[0045] In this embodiment, the loss function value of the image classification model can be calculated using a first formula, which can be expressed as:

[0046] in, It is a set containing features corresponding to multiple classification categories. That is, the feature corresponding to the i-th category. It is a set containing multiple weights. The weight corresponding to the i-th category is the second weight. The number of the second weights, Indicates the first weight. The number of items with the first weight.

[0047] in, This is a metric function, which can be represented differently for different vectors. For two features... and This metric function can be expressed as For features and weight This metric function can be expressed as ,in It is an additive margin. This refers to the ratio of the number of sample categories in the first training set to the number of sample categories in the second training set. It is a temperature hyperparameter that allows adjustment of convergence speed and generalization, and can be customized.

[0048] As can be seen from the above, this embodiment determines the convergence degree of the image classification model by calculating the loss function value of the image classification model, thereby better evaluating the training performance of the image classification model, improving the accuracy of image classification, and enhancing the model's generalization ability.

[0049] In one embodiment of this disclosure, the image classification method further includes: The second formula establishes the correspondence between the classification categories and weights of the target classifier.

[0050] The second formula is: .in, This represents the distance between the category and the weight. This indicates the number of categories. Indicates the position of the weight. Indicates the location of the feature center.

[0051] In this embodiment, each category of the target classifier has a corresponding weight. After weight updates, the target classifier needs to be reassigned, thus establishing a correspondence between the target classifier's categories and weights. Weight assignment can be achieved using an improved Hungarian algorithm, a combinatorial optimization algorithm that solves the task assignment problem in polynomial time. The basic steps of this algorithm include: (1) Initialization: Create a bipartite graph where each edge has a non-negative cost, and find the perfect matching with the lowest cost.

[0052] (2) Row operations: Perform operations on each row of the matrix, subtracting the smallest element in each row from each element in that row, thus ensuring that each row contains at least one zero.

[0053] (3) Column operations: Perform operations on each column of the matrix, subtracting the smallest element in each column from each element in that column, so that each column contains at least one zero.

[0054] (4) Find zero elements: Find as many zero elements as possible so that each row and each column contains only one selected zero element.

[0055] (5) Test: If the number of zero elements found is equal to the number of rows (or columns) of the matrix, that is, an optimal solution has been found and the algorithm ends.

[0056] (6) Modify the matrix: If no optimal solution is found, the matrix needs to be modified, and step (4) is repeated. Find the smallest element in all uncovered rows and columns, subtract the element from these rows, and add it to the covered columns. Repeat this process until an optimal solution is found.

[0057] The core of this algorithm is to establish a correspondence between the feature center and the weight corresponding to each classification category by calculating the distance between them. Its core formula is the second formula, which can be expressed as:

[0058] in, This represents the distance between the category and the weight. This indicates the number of categories. Indicates the position of the weight. Indicates the location of the feature center.

[0059] As can be seen from the above, this embodiment can establish a correspondence between the classification categories and weights of the target classifier, assign corresponding weights to semantically similar classification categories, thereby eliminating the impact of data imbalance and increasing the generalization ability of the model.

[0060] In one embodiment of this disclosure, the target classifier is used for: The image category corresponding to the first classification category is determined based on the first training set and the weights of the updated target classifier.

[0061] In this embodiment, the first training set is input into the image classification model. Based on the updated weights of the target classifier, the image category corresponding to each sample category in the first training set can be determined. Finally, the image classification model outputs the image category corresponding to the first classification category, completing the image classification process.

[0062] As can be seen from the above, this embodiment can expand the learning task of the image classification model and improve the incremental learning efficiency by implementing the image classification process of the first training set.

[0063] In one embodiment of this disclosure, Figure 2 yes Figure 2 This is a schematic diagram of an image classification model structure provided in an embodiment of the present disclosure, as shown below. Figure 2 As shown, the target classifier is specifically used for: The third formula is used to calculate the cosine similarity between the features corresponding to the first category and the weights of each target classifier.

[0064] The third formula is: .in, This represents the feature corresponding to the first category. This represents the weights of the target classifier.

[0065] The category corresponding to the weight with the highest cosine similarity is taken as the image category corresponding to the first category.

[0066] In this embodiment, the image category corresponding to the first classification category is determined based on the first training set and the weights of the updated target classifier, which can be described in detail as follows: In image classification, the backbone network of the image classification model transforms the input sample category into a feature, then calculates the cosine similarity between this feature and each weight, and takes the category corresponding to the weight with the highest cosine similarity as the output image classification result. The cosine similarity between the feature corresponding to the sample category in the first training set and the weight of each target classifier can be calculated using a third formula, which can be expressed as: .in, This represents the feature corresponding to the first category. This represents the weights of the target classifier. By calculating cosine similarity, the category corresponding to the weight with the highest cosine similarity in the calculation results is taken as the image category corresponding to the first category. Based on the calculation results of this cosine similarity, the image classification model can output the image category corresponding to each first category.

[0067] During the training of the image classification model, there is no overlap between the image categories in any two stages. During the training of stage t, the image classification model can only be trained using the first and second training sets corresponding to the current stage. After the training of stage t is completed, the image classification model can classify all sample categories in stage t and previous stages.

[0068] This embodiment provides a reference example. For instance, this image classification model is applicable to scenarios such as supermarket shelf classification, new product introduction, old product removal, and category segmentation. Various product images are input into the image classification model as the first training set. The model identifies and classifies the products, outputting the corresponding product categories for each image. Given that product categories are constantly changing and increasing, whenever a new category appears, the image classification model can dynamically update its parameters, incrementally learning and training new category product images based on existing categories, mitigating the impact of data imbalance, and maintaining its ability to classify all product images.

[0069] As can be seen from the above, this embodiment can achieve incremental learning training on a small amount of sample data, reduce the impact of data imbalance on the performance of image classification models, and can also be used in conjunction with other incremental learning methods to improve the efficiency and generalization of the model.

[0070] Corresponding to the image classification method in the above embodiments, Figure 3 This is a structural block diagram of an incremental image classification device based on uniform prototypes and dynamic decision boundaries, provided as an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiment of this disclosure are shown. References Figure 3 The incremental image classification device 20 based on uniform prototype and dynamic decision boundary includes: a weight update module 21, a loss function calculation module 22, a parameter update module 23, and an image classification module 24.

[0071] The weight update module 21 is used to update the weights of the target classifier based on the first classification category.

[0072] The loss function calculation module 22 is used to determine the ratio of the number of sample categories in the first training set to the number of sample categories in the second training set, and calculates the loss function value of the image classification model corresponding to the target classifier based on the ratio. The first training set is the training set corresponding to the first classification category, and the second training set belongs to the historical training set of the image classification model and is the training set corresponding to the second classification category. The second classification category is different from the first classification category.

[0073] The parameter update module 23 is used to update the parameters of the backbone network of the image classification model based on the loss function value, the first training set, and the second training set until the image classification model converges.

[0074] Image classification module 24 is used for image classification based on the converged image classification model.

[0075] In one embodiment of this disclosure, the weight update module 21 is specifically used for: Initialize N first weights. N is the number of first categories, and each first weight corresponds to one first category.

[0076] Orthogonalize the N first weights and multiple second weights to make the first weights orthogonal to each other and the first and second weights orthogonal to each other. Each second weight corresponds to a second classification category.

[0077] The first and second weights after orthogonalization are determined as the weights of the target classifier to update the target classifier weights.

[0078] In one embodiment of this disclosure, the weight update module 21 is further configured to: In response to a change in the feature dimension of the image classification model backbone network, the weights of the target classifier are updated based on the number of first classification categories.

[0079] In one embodiment of this disclosure, the loss function calculation module 22 is specifically used for: The loss function value of the image classification model corresponding to the target classifier is calculated using the first formula. The first formula is:

[0080] in, This represents the value of the loss function. This represents the feature corresponding to the i-th category. This indicates the first weight. This indicates the number of first-weighted values. This indicates the second weight. This indicates the number of second weights. It represents a feature set. This represents the set of weights. This represents a metric function. This represents a temperature hyperparameter. Indicates quantity ratio.

[0081] In one embodiment of this disclosure, the weight update module 21 is further configured to: The second formula establishes the correspondence between the classification categories and weights of the target classifier.

[0082] The second formula is: .in, This represents the distance between the category and the weight. This indicates the number of categories. Indicates the position of the weight. Indicates the location of the feature center.

[0083] In one embodiment of this disclosure, the image classification module 24 is further configured to: The image category corresponding to the first classification category is determined based on the first training set and the weights of the updated target classifier.

[0084] In one embodiment of this disclosure, the image classification module 24 is specifically used for: The third formula is used to calculate the cosine similarity between the features corresponding to the first category and the weights of each target classifier.

[0085] The third formula is: .in, This represents the feature corresponding to the first category. This represents the weights of the target classifier.

[0086] The category corresponding to the weight with the highest cosine similarity is taken as the image category corresponding to the first category.

[0087] See Figure 4 , Figure 4 This is a schematic block diagram of an electronic device provided according to an embodiment of the present disclosure. Figure 4 The electronic device 300 in this embodiment may include one or more processors 301, one or more input devices 302, one or more output devices 303, and one or more memories 304. The processors 301, input devices 302, output devices 303, and memories 304 communicate with each other via a communication bus 305. The memories 304 store computer programs, including program instructions. The processors 301 execute the program instructions stored in the memories 304. Specifically, the processors 301 are configured to invoke the program instructions to perform the functions of the modules in the aforementioned device embodiments, for example... Figure 3 The functions of modules 21 to 24 are shown.

[0088] It should be understood that, in the embodiments of this disclosure, the processor 301 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0089] Input device 302 may include a touchpad, a fingerprint sensor (for collecting the user's fingerprint information and fingerprint orientation information), a microphone, etc., and output device 303 may include a display (LCD, etc.), a speaker, etc.

[0090] The memory 304 may include read-only memory and random access memory, and provides instructions and data to the processor 301. A portion of the memory 304 may also include non-volatile random access memory. For example, the memory 304 may also store device type information.

[0091] In specific implementations, the processor 301, input device 302, and output device 303 described in the embodiments of this disclosure can execute the implementation methods described in the first and second embodiments of the image classification method provided in the embodiments of this disclosure, or they can execute the implementation methods of the electronic devices described in the embodiments of this disclosure, which will not be repeated here.

[0092] In another embodiment of this disclosure, a computer-readable storage medium is provided. This computer-readable storage medium stores a computer program, which includes program instructions. When executed by a processor, the program instructions implement all or part of the processes in the methods described above. The computer program can also instruct related hardware to complete the process. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0093] The computer-readable storage medium can be an internal storage unit of the electronic device in any of the foregoing embodiments, such as a hard disk or memory of the electronic device. The computer-readable storage medium can also be an external storage device of the electronic device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the electronic device. Furthermore, the computer-readable storage medium can include both internal and external storage units of the electronic device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0094] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0095] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the electronic devices and units described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0096] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces or units, or it may be an electrical, mechanical, or other form of connection.

[0097] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of this disclosure, depending on actual needs.

[0098] Furthermore, the functional units in the various embodiments of this disclosure can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0099] The above are merely specific embodiments of this disclosure, but the scope of protection of this disclosure is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this disclosure, and such modifications or substitutions should all be covered within the scope of protection of this disclosure. Therefore, the scope of protection of this disclosure should be determined by the scope of the claims.

Claims

1. An image classification method, characterized in that, include: Initialize N first weights; N is the number of first categories, and each first weight corresponds to one first category; Orthogonalize the N first weights and multiple second weights to make the first weights orthogonal to each other and the first and second weights orthogonal to each other; each second weight corresponds to a second classification category. The first and second weights after orthogonalization are determined as the weights of the target classifier to update the target classifier weights; Determine the ratio of sample categories in the first training set to the number of sample categories in the second training set, and calculate the loss function value of the image classification model corresponding to the target classifier using a first formula; the first formula is: ； in, This represents the value of the loss function; This represents the feature corresponding to the i-th category; Indicates the first weight; Indicates the number of first-weighted items; Indicates the second weight; Indicates the number of second weights; Represents a feature set; Represents the set of weights; Represents a metric function; This represents a temperature hyperparameter; The first training set is the training set corresponding to the first classification category, and the second training set belongs to the historical training set of the image classification model and is the training set corresponding to the second classification category; the second classification category is different from the first classification category. Based on the loss function value, the first training set, and the second training set, the parameters of the backbone network of the image classification model are updated until the image classification model converges. Image classification is performed based on the converged image classification model.

2. The image classification method as described in claim 1, characterized in that, Also includes: In response to a change in the feature dimension of the image classification model backbone network, the weights of the target classifier are updated based on the number of the first classification categories.

3. The image classification method as described in claim 1, characterized in that, Also includes: The correspondence between the classification categories and weights of the target classifier is established based on the second formula; The second formula is: ;in, This represents the distance between the category and its weight. Indicates the number of categories; Indicates the weight position; Indicates the location of the feature center.

4. The image classification method as described in claim 1, characterized in that, The target classifier is used for: The image category corresponding to the first classification category is determined based on the input image and the weights of the updated target classifier.

5. The image classification method as described in claim 4, characterized in that, The target classifier is specifically used for: The cosine similarity between the features corresponding to the first classification category and the weights of each target classifier is calculated using the third formula. The third formula is: ;in, This represents the feature corresponding to the first category; Indicates the weights of the target classifier; The category corresponding to the weight with the highest cosine similarity is taken as the image category corresponding to the first category.

6. An image classification device, characterized in that, include: The weight update module is used to initialize N initial weights; N is the number of first categories, and each first weight corresponds to one first category; Orthogonalize the N first weights and multiple second weights to make the first weights orthogonal to each other and the first and second weights orthogonal to each other; each second weight corresponds to a second classification category. The first and second weights after orthogonalization are determined as the weights of the target classifier to update the target classifier weights; The loss function calculation module is used to determine the ratio of the number of sample categories in the first training set to the number of sample categories in the second training set, and to calculate the loss function value of the image classification model corresponding to the target classifier using a first formula; the first formula is: ； in, This represents the value of the loss function; This represents the feature corresponding to the i-th category; Indicates the first weight; Indicates the number of first-weighted items; Indicates the second weight; Indicates the number of second weights; Represents a feature set; Represents the set of weights; Represents a metric function; This represents a temperature hyperparameter; The first training set is the training set corresponding to the first classification category, and the second training set belongs to the historical training set of the image classification model and is the training set corresponding to the second classification category; the second classification category is different from the first classification category. The parameter update module is used to update the parameters of the backbone network of the image classification model based on the loss function value, the first training set, and the second training set until the image classification model converges. The image classification module is used to classify images based on the converged image classification model.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images