An image classification method and device based on continuous learning

Abstract

This application provides an image classification method and apparatus based on continuous learning, comprising: determining the image category of a target image based on an image classification model; the image classification model is trained through the following steps: for any model training task, during each round of training of the initial image classification model according to the model training task: determining whether the task information of the model training task in that round of training has entered a preset confidence region; if it has entered, updating the projection matrix corresponding to each layer obtained in the previous round of training in the initial image classification model according to a first predetermined method; if it has not entered, updating the projection matrix corresponding to each layer obtained in the previous round of training according to a second predetermined method; and finally determining the projection matrix obtained in that round of training. This improves the accuracy and stability of the trained image classification model when applied to identify images of unknown categories.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to an image classification method and image classification device based on continuous learning. Background Technology

[0002] Recently, with the development of deep learning, neural network models have been widely applied in fields such as speech recognition, image classification, and object detection. In image classification, in particular, neural network models have achieved significant research results, in many cases reaching near-human recognition speeds and accuracy.

[0003] However, unlike humans, neural network models do not possess continuous learning capabilities. They cannot apply knowledge from one task to another, nor can they forget knowledge from previous tasks while learning a new one. When learning a new task, neural network models forget knowledge learned from previous tasks. This causes the accuracy of trained image classification models to be unstable when applied to identifying images of unknown categories, resulting in very low accuracy for some image categories and ultimately, classification failure. Furthermore, during training, neural network models continuously learn the mapping relationship between input and output data. In the early stages of learning, the knowledge learned by the model is not accurate enough, introducing misleading information that is persistently stored in the model. This leads to poor performance of the trained image classification model and affects the accuracy of image classification. Summary of the Invention

[0004] In view of this, the purpose of this application is to provide an image classification method and device based on continuous learning. By adding an appropriate projection matrix to adjust the model parameters, the continuous learning ability of the model can be improved, and the accuracy and stability of the trained image classification model can be enhanced when applied to identify images of unknown categories. In addition, by setting the projection matrix to be updated after the task information of the model training enters the preset trust region, misleading information generated by the training task when it is untrusted can be avoided in the model, thereby further improving the performance of the image classification model and the accuracy of image classification.

[0005] This application provides an image classification method based on continuous learning, the method comprising:

[0006] The target image to be classified is input into a pre-trained image classification model to determine the image category of the target image; wherein, the image classification model is trained through the following steps:

[0007] Sequentially acquire multiple model training tasks under continuous learning conditions;

[0008] The initial image classification model is trained multiple times according to each model training task to obtain the trained image classification model; wherein, for any model training task, during each round of training of the initial image classification model according to that model training task:

[0009] Determine whether the task information of the projection matrix corresponding to each layer of the initial image classification model has entered the preset confidence region during the training round of the model;

[0010] If the task information of the projection matrix corresponding to the layer in the initial image classification model has entered the preset reliable region during this round of training, then according to the first predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training; wherein, the initial projection matrix corresponding to each layer is an identity matrix with the same matrix size as the model parameter size of the initial image classification model; the projection matrix is ​​used to update the model parameters of the initial image classification model during backpropagation.

[0011] Furthermore, the method also includes:

[0012] If the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset reliable region during this round of training, then according to the second predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

[0013] Furthermore, each model training task includes a preset total number of training rounds and a preset round threshold; the step of determining whether the task information of the projection matrix corresponding to each layer of the initial image classification model has entered the preset reliable region during the training round includes:

[0014] Obtain the task information of the projection matrix corresponding to each layer of the model training task in this round of training; the task information includes the number of training rounds corresponding to this round of training.

[0015] If the difference between the number of training rounds corresponding to the current training round and the preset total number of training rounds is less than the preset number of rounds threshold, then it is determined that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset reliable region during the current training round.

[0016] Furthermore, each model training task includes a preset loss threshold corresponding to the model training task or a preset gradient threshold corresponding to each layer in the model training task; the step of determining whether the task information of the projection matrix corresponding to each layer in the initial image classification model during this round of training has entered the preset reliable region includes:

[0017] Obtain task information for the model training task, specifically the projection matrix of each layer during this training round; the task information includes the loss value corresponding to this training round or the gradient matrix of each layer obtained through backpropagation.

[0018] If the loss value corresponding to the training round is less than the preset loss value threshold, it is determined that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset reliable region during the training round.

[0019] Alternatively, if the sum of the absolute values ​​of each element in the gradient matrix of each layer is less than the preset gradient threshold corresponding to that layer, then it is determined that the task information of the projection matrix corresponding to that layer has entered the preset reliable region during the training round of the model.

[0020] Furthermore, each model training task includes a threshold for the ratio of the gradient norms corresponding to each layer in the model training task; the step of determining whether the task information of the projection matrix corresponding to each layer in the initial image classification model during this round of training has entered the preset reliable region includes:

[0021] For each layer in the initial image classification model, the task information of the projection matrix corresponding to that layer during the current training round is obtained. The task information includes the ratio of the first norm and the second norm of that layer during the current training round. The first norm is the norm of the product of the projection matrix corresponding to that layer after projection and the current gradient. The second norm is the norm of the current gradient.

[0022] If the ratio is less than the threshold of the ratio of the gradient norm of the corresponding layer, it is determined that the task information of the projection matrix of the corresponding layer has entered the preset reliable region during the training round of the model.

[0023] Furthermore, each model training task also includes a sample image set and image labels corresponding to that model training task; the step of determining the projection matrix corresponding to the layer obtained in the current training round according to the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training, in a first predetermined manner, includes:

[0024] The sample image set corresponding to the training task of this model is shuffled and divided into multiple sample image subsets;

[0025] For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images; wherein, when the subset of sample images is the first subset of sample images in the plurality of subsets of sample images, the projection matrix obtained in the previous round of training or the initial projection matrix is ​​used as the projection matrix updated using the previous subset of sample images;

[0026] The projection matrix obtained by updating using the last sample image subset from the plurality of sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training.

[0027] Furthermore, when each model training task includes a threshold for the ratio of the gradient norm corresponding to that model training task, if the task information of the projection matrix corresponding to that layer in the initial image classification model does not enter the preset confidence region during this round of training, then according to the second predetermined method, the projection matrix corresponding to that layer obtained in this round of training is determined based on the projection matrix corresponding to that layer in the initial image classification model obtained in the previous round of training, including:

[0028] The sample image set corresponding to the training task of this model is shuffled and divided into multiple sample image subsets;

[0029] For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images.

[0030] The projection matrix obtained by updating using the last sample image subset from the multiple sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training.

[0031] In the process of determining the projection matrix corresponding to the layer updated using the sample image subset, if the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset reliable region during the training round, then for each image data of the input projection matrix corresponding to the layer, the target ratio of the first norm and the second norm corresponding to the layer is calculated using each image data.

[0032] The target ratios calculated using each image data are sorted in ascending order, and the m image data corresponding to the top m target ratios are selected, where m is a positive integer;

[0033] Based on the m image data, and using the projection matrix corresponding to the layer updated with the previous sample image subset, the projection matrix corresponding to the layer updated using the sample image subset is determined by the following formula:

[0034]

[0035] Where i represents the layer number corresponding to the projection matrix, k represents the index of the sample image subset, and i and k are positive integers; Pi _k Pi represents the projection matrix corresponding to the i-th layer obtained by updating using a subset of sample images; _k-1 This represents the projection matrix corresponding to the i-th layer obtained by updating using the previous subset of sample images; xi ′ _k xi represents the average value of the m image data in the projection matrix corresponding to the i-th layer; ′ _ kT represents xi ′ _k The transpose of ; α represents a hyperparameter and α>0.

[0036] This application embodiment also provides an image classification device based on continuous learning, the device comprising:

[0037] The classification module is used to input the target image to be classified into a pre-trained image classification model to determine the image category of the target image;

[0038] The training module is used to train the image classification model through the following steps:

[0039] Sequentially acquire multiple model training tasks under continuous learning conditions;

[0040] The initial image classification model is trained multiple times according to each model training task to obtain the trained image classification model; wherein, for any model training task, during each round of training of the initial image classification model according to that model training task:

[0041] It is determined whether the task information of the projection matrix corresponding to each layer of the initial image classification model has entered the preset confidence region during the training round; wherein, the initial projection matrix corresponding to each layer is an identity matrix with the same matrix size as the model parameter size of the initial image classification model; the projection matrix is ​​used to update the model parameters of the initial image classification model during backpropagation;

[0042] If the task information of the projection matrix corresponding to the layer in the initial image classification model has entered the preset reliable region during this round of training, then according to the first predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

[0043] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of the image classification method based on continuous learning as described above.

[0044] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the image classification method based on continuous learning described above.

[0045] This application provides an image classification method and apparatus based on continuous learning, comprising: inputting a target image to be classified into a pre-trained image classification model to determine the image category of the target image; wherein the image classification model is trained through the following steps: sequentially acquiring multiple model training tasks under continuous learning conditions; sequentially performing multiple rounds of training on an initial image classification model according to each model training task to obtain a trained image classification model; wherein, for any model training task, during each round of training of the initial image classification model according to the model training task: determining whether the task information of the projection matrix corresponding to each layer in the initial image classification model has entered a preset confidence region during the training round; if the task information of the projection matrix corresponding to that layer in the initial image classification model has entered the preset confidence region during the training round, then determining the projection matrix corresponding to that layer obtained in the training round according to a first predetermined method based on the projection matrix corresponding to that layer in the initial image classification model obtained in the previous round of training; wherein the initial projection matrix corresponding to each layer is an identity matrix with the same matrix size as the model parameter size of the initial image classification model; the projection matrix is ​​used to update the model parameters of the initial image classification model during backpropagation.

[0046] By adjusting the model parameters with an appropriate projection matrix, the model's continuous learning ability can be improved, enhancing the accuracy and stability of the trained image classification model when applied to identify images of unknown categories. Furthermore, updating the projection matrix after the training task information enters a preset trust region can avoid misleading information generated by untrusted training tasks in the model, thereby further improving the performance and accuracy of the image classification model.

[0047] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 One of the flowcharts for training an image classification model provided in an embodiment of this application is shown;

[0050] Figure 2 The second flowchart illustrates a training method for an image classification model provided in an embodiment of this application;

[0051] Figure 3 A schematic diagram of the structure of a training device for an image classification model provided in an embodiment of this application is shown;

[0052] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0054] Research has shown that with the development of deep learning, neural network models have been widely applied in fields such as speech recognition, image classification, and object detection. In image classification, in particular, neural network models have achieved significant research results, in many cases reaching near-human recognition speeds and accuracy.

[0055] However, unlike humans, neural network models do not possess continuous learning capabilities. They cannot apply knowledge from one task to another, nor can they forget knowledge from previous tasks while learning a new one. When learning a new task, neural network models forget knowledge learned from previous tasks. This causes the accuracy of trained image classification models to be unstable when applied to identifying images of unknown categories, resulting in very low accuracy for some image categories and ultimately, classification failure. Furthermore, during training, neural network models continuously learn the mapping relationship between input and output data. In the early stages of learning, the knowledge learned by the model is not accurate enough, introducing misleading information that is persistently stored in the model. This leads to poor performance of the trained image classification model and affects the accuracy of image classification.

[0056] Based on this, embodiments of this application provide an image classification method and apparatus based on continuous learning. By adding an optimized and more suitable projection matrix to adjust model parameters, the continuous learning capability of the model is improved, thereby enhancing the accuracy and stability of the trained image classification model when applied to identify images of unknown categories. Furthermore, the projection matrix is ​​updated after the task information of the model training enters a preset trust region to avoid misleading information generated in the model by untrusted training tasks, thereby further improving the performance of the image classification model and the accuracy of image classification.

[0057] The image classification method based on continuous learning provided in this application includes: inputting a target image to be classified into a pre-trained image classification model to determine the image category of the target image.

[0058] Here, the pre-trained image classification model can be a neural network model, such as a convolutional neural network (CNN) or a recurrent neural network (RNN). The image classification model can extract feature information at different levels from the input target image through a complex multi-layered network structure composed of neurons, and then determine the image category of the target image; in this case, each layer of the neural network model is equipped with a projection matrix whose projected gradient is orthogonal to the input space of that layer, i.e., the orthogonal projection matrix OWM algorithm (orthogonal weights modification).

[0059] In practical applications, image classification models can recognize handwritten digit images and identify the digits within them. They can also be used to identify captured images and determine the species of animals in the images, such as cats, dogs, and birds. Furthermore, they can more accurately identify the specific breed of each animal, such as whether the dog in the image is a Corgi, a Golden Retriever, or a Labrador.

[0060] Furthermore, the image classification model training method provided in this application embodiment may include:

[0061] The first step is to sequentially obtain multiple model training tasks under continuous learning conditions.

[0062] It should be noted that under continuous learning conditions, when an image classification model is trained based on a later model training task, the data from the previous model training task is often lost or overwritten, causing the neural network model to forget the knowledge learned from previous tasks when learning new tasks.

[0063] The second step is to train the initial image classification model in multiple rounds according to each model training task to obtain the trained image classification model.

[0064] In practice, different model training tasks can be used to train the image classification model to recognize images of different categories. For example, model training task 1 can be used to train the image classification model to recognize images of cats, and model training task 2 can be used to train the image classification model to recognize images of dogs.

[0065] It should be noted that multiple model training tasks are often stored in the model training queue and executed sequentially. After the initial image classification model has completed multiple rounds of training according to a certain model training task Taskn, it will continue to be trained according to another model training task Taskn, and so on, until all model training tasks in the model training queue have been completed, and a trained image classification model is obtained.

[0066] In existing technologies, image classification models sometimes experience a "catastrophic forgetting" problem when transitioning from an old training task to a new one. This is because the model may suddenly forget the knowledge learned in the previous task. Furthermore, in the early stages of a new training task, the model is not yet adapted to the new data, and the learned knowledge is inaccurate. This inaccurate knowledge often introduces misleading information into the neural network, such as incorrectly altering neuron weights. This misleading information is persistently stored in the model. All of these factors contribute to poor performance in trained image classification models, impacting the accuracy and stability of image classification.

[0067] The training method of the image classification model provided in the embodiments of this application will be described in detail below.

[0068] Please see Figure 1 , Figure 1 This is one of the flowcharts for a training method of an image classification model provided in an embodiment of this application. For example... Figure 1As shown in , for any model training task, the process of each round of training the initial image classification model according to this model training task may include:

[0069] S101. Determine whether the task information of the projection matrix corresponding to each layer in the initial image classification model has entered the preset trusted region during this round of training of this model training task.

[0070] Among them, the model parameters of the initial image classification model are initialized as w i_0 (generally following a Gaussian distribution), and the initial projection matrix P is initialized i_0 , where i represents the layer number corresponding to this projection matrix; the gradient after projection of the initial projection matrix is orthogonal to the input space of the initial image classification model; the initial projection matrix is an identity matrix with the same matrix scale as the model parameter scale of the initial image classification model (the number of rows of the initial projection matrix is the same as the number of rows of the model parameter matrix), that is, P i_0 =I i ; the projection matrix is used to update the model parameters of the initial image classification model during the backpropagation stage of the model training process.

[0071] In the first possible implementation manner, when each model training task includes the preset total number of training rounds and the preset round threshold corresponding to this model training task, then step S101 may include:

[0072] S1011. Obtain the task information of the projection matrix corresponding to each layer during this round of training of this model training task.

[0073] Among them, the task information includes the number of training rounds corresponding to this round of training.

[0074] S1012. If the difference between the number of training rounds corresponding to this round of training and the preset total number of training rounds is less than the preset round threshold, then determine that the task information of the projection matrix corresponding to each layer in this model training task has entered the preset trusted region.

[0075] That is, assuming that this round of training is the epoch-th round of training, the preset total number of training rounds is N, and the preset round threshold is r, if N - epoch < r, it means that the task information in the epoch-th round of training of this model training task has entered the preset trusted region.

[0076] It should be understood that generally, in the early stage of neural network training, the knowledge learned by the network is usually not accurate enough. As the training progresses, the neural network can gradually learn the mapping rules of real data. Through this implementation manner, the image classification model can be made immune to inaccurate knowledge within a certain number of training rounds.

[0077] In a second possible implementation, when each model training task includes a preset loss threshold corresponding to the model training task or a preset gradient threshold corresponding to each layer in the model training task, step S101 may include:

[0078] S1013. Obtain the task information of the projection matrix corresponding to each layer of the model training task during this round of training.

[0079] The task information includes the loss value corresponding to this round of training or the gradient matrix of each layer obtained through backpropagation.

[0080] S1014. If the loss value corresponding to the training round is less than the preset loss value threshold, it is determined that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset confidence region during the training round; or, if the sum of the absolute values ​​of each element in the gradient matrix of each layer (the L1 norm of the gradient matrix) is less than the preset gradient threshold corresponding to that layer, it is determined that the task information of the projection matrix corresponding to that layer of the model training task has entered the preset confidence region during the training round.

[0081] That is, assuming the preset loss threshold for this training round is L, the loss value obtained by forward propagation of the current input data x is loss; if loss ≤ L, then the task information of the projection matrix corresponding to each layer of the model training task in this training round has entered the preset confidence region. Alternatively, assuming the gradient matrix of the i-th layer is Δw when the input data is x. i_x The preset gradient threshold for the i-th layer during this training round is W. i If the sum of the absolute values ​​of each element in the gradient matrix of the i-th layer (the L1 norm of the gradient matrix) satisfies:

[0082] ||Δw i_x || L1 ≤W i

[0083] Then, during this training round, the task information of the projection matrix corresponding to the i-th layer of the model training task has entered the preset confidence region.

[0084] It should be understood that in the early stages of neural network training, the loss value or the L1 norm of the gradient matrix is ​​usually relatively large. Then, as the neural network continues to train, the loss value or the L1 norm of the gradient matrix will rapidly decrease to a certain range and fluctuate within a small range. At this point, the output of each layer of the network is closer to the optimal output for overcoming catastrophic forgetting. Through this implementation, when forward propagating to a batch of data, if the calculated loss value or the L1 norm of the gradient matrix is ​​not higher than the set threshold, then this batch of data will be recorded by the projection matrix. This discards a lot of inaccurate information, making the projection matrix more realistic. Since the value of the loss function fluctuates throughout the training process, even in the later stages of training, there will still be some inaccurate information corresponding to high L1 norms of the loss value or gradient matrix. Therefore, through this implementation, even in the later stages of network training, the method provided in this application can adaptively discard some inaccurate information, which is more conducive to obtaining an image classification model with good classification performance, thereby improving the accuracy and stability of image classification when applied to identifying images of unknown categories.

[0085] In a third possible implementation, when each model training task includes a threshold for the ratio of the gradient norms corresponding to each layer in the model training task, step S101 may include:

[0086] S1015. For the projection matrix corresponding to each layer in the initial image classification model, obtain the task information of the projection matrix corresponding to the layer in this round of training for the model training task.

[0087] The task information includes the ratio of the first norm and the second norm of the layer in this round of training; the first norm is the norm of the product of the projection matrix of the layer after projection and the current gradient; the second norm is the norm of the current gradient.

[0088] S1016. If the ratio is less than the ratio threshold of the gradient norm of the corresponding layer, it is determined that the task information of the projection matrix of the corresponding layer has entered the preset reliable region during the training round of the model.

[0089] It should be noted that, unlike the first two implementations, where a single judgment can determine whether the task information of the projection matrix corresponding to each layer has entered the preset trusted region, the third implementation requires judging layer by layer whether the task information of the projection matrix corresponding to each layer has entered the preset trusted region.

[0090] Specifically, assuming the threshold for the ratio of the gradient norm corresponding to the training task of the model is β, when the ratio of the first norm and the second norm corresponding to the layer in this round of training satisfies the following formula (1), it is determined that the task information of the projection matrix corresponding to the layer in this round of training has entered the preset confidence region:

[0091]

[0092] In the formula, Δw i_k-1 P′ represents the current gradient; i_k The projection matrix corresponding to this layer after projection. Denotes the first norm; This represents the second norm.

[0093] It is worth noting that the above three implementation methods can determine whether the task information of the projection matrix corresponding to each layer has entered the preset trusted region separately, or they can be combined to determine whether the task information of the projection matrix corresponding to each layer has entered the preset trusted region. For example, the first and second implementation methods can be combined. When both conditions are met—the difference between the number of training rounds corresponding to this round of training and the preset total number of training rounds is less than the preset round number threshold, and the loss value corresponding to this round of training is less than the preset loss value threshold—it is determined that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset trusted region during this round of training. Another example is that the first and second implementation methods can be combined. When either of the above two conditions is met, it is determined that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset trusted region during this round of training. Yet another example is that the first and / or the second implementation methods can be combined with the third implementation method. That is, after determining that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset trusted region during this round of training according to the first and / or the second implementation methods, the task information of the projection matrix corresponding to each layer of the model training task has entered the preset trusted region layer by layer according to the third implementation method.

[0094] S102. If the task information of the projection matrix corresponding to the layer in the initial image classification model has entered the preset reliable region during this round of training, then according to the first predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

[0095] In this way, by introducing the concept of a reliable region and setting the projection matrix to be updated after the task information of the model training enters the preset reliable region, we can avoid misleading information generated by the training task when it is unreliable, thus achieving immunity to the training task when it is unreliable. We can also avoid using the projection matrix to update the model parameters too early when the reliability of the model parameters of the image classification model is low, thereby improving the convergence accuracy of the image classification model, improving the performance of the image classification model and the accuracy of image classification, achieving the technical effect of "discarding the dross and selecting the essence", which is in line with the general law of human brain to acquire knowledge.

[0096] Furthermore, each model training task also includes the sample image set and image labels corresponding to that model training task. Please refer to the following. Figure 2 , Figure 2 This is a second flowchart illustrating a training method for an image classification model provided in an embodiment of this application. Figure 2 As shown, determining the projection matrix corresponding to the current layer obtained in the first predetermined method, based on the projection matrix corresponding to the current layer in the initial image classification model obtained in the previous training round, may include:

[0097] S201. Shuffle and divide the sample image set corresponding to the training task of the model into multiple sample image subsets.

[0098] In this step, multiple sample images in the sample image set corresponding to the model training task are randomly shuffled and divided into multiple mini-batch subsets. For each round of training for this model training task, the image classification model is trained sequentially using multiple subsets of sample images in subsequent steps until the training of the image classification model is completed using the last subset of sample images, at which point the training round of this model training task is considered complete. Each subset of sample images includes at least one sample image. Multiple sample images corresponding to each model training task have the same image label, while different model training tasks correspond to different image labels.

[0099] S202. For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images.

[0100] Wherein, when the sample image subset is the first sample image subset among the plurality of sample image subsets, the projection matrix obtained from the previous training round (if this training round is not the first training round) or the initial projection matrix (if this training round is the first training round) is used as the projection matrix updated using the previous sample image subset.

[0101] In practical implementation, the projection matrix corresponding to the layer updated using the sample image subset can be determined using the following formula, based on the sample image subset and the projection matrix corresponding to the layer updated using the previous sample image subset:

[0102]

[0103] Where i represents the layer number corresponding to the projection matrix, k represents the index of the subset of sample images, and i and k are positive integers; P i_k This indicates that the projection matrix corresponding to the i-th layer is updated using a subset of sample images, specifically the k-th sample subset; P i_k-1 This indicates that the projection matrix corresponding to the i-th layer is updated using the previous sample image subset, i.e., the (k-1)-th sample subset; x i_k This represents the average value of multiple input data in the projection matrix corresponding to the i-th layer. x represents i_k The transpose of ; α represents a hyperparameter and α>0.

[0104] Here, when the subset of sample images includes multiple sample images, x i_k It can be obtained by performing matrix operations on the image data of multiple sample images and taking the average value.

[0105] S203. The projection matrix obtained by updating using the last sample image subset in the plurality of sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training.

[0106] Furthermore, after S201 shuffles and divides the sample image set corresponding to the model training task into multiple sample image subsets, the process of training each round of the initial image classification model according to the model training task also includes:

[0107] Step 1: For each subset of sample images, input the subset of sample images into the image classification model trained iteratively using the previous subset of sample images, and determine the image classification result of the subset of sample images.

[0108] In this round of training, if the sample image subset is the first sample image subset among multiple sample image subsets, the image classification model obtained from the previous round of training (this round of training is not the first round of training) or the initial image classification model set in the initialization (this round of training is the first round of training) can be used as the image classification model obtained by iterative training using the previous sample image subset.

[0109] In this step, for each subset of sample images, the subset of sample images is input into the image classification model that has been iteratively trained using the previous subset of sample images. Through the forward propagation process of the model, the image classification result of the subset of sample images output by the image classification model can be obtained.

[0110] Step 2: Based on the image classification results and image labels of the sample image subset, determine the loss value corresponding to the sample image subset.

[0111] In this step, the loss value corresponding to the sample image subset can be determined based on the image classification results and the image labels of the sample image subset using any method in the prior art. For example, the image classification results and the image labels can be substituted into the cross-entropy loss function to determine the loss value.

[0112] Step 3: Determine the current gradient of the loss value corresponding to the subset of sample images in the current initial image classification model.

[0113] Step 4: Based on the current model parameters of the initial image classification model, the current gradient of the loss value corresponding to the sample image subset on the current initial image classification model, and the projection matrix obtained by updating using the sample image subset, update the model parameters of the initial image classification model, and re-determine the image classification model obtained after updating the model parameters as the initial image classification model, so as to complete the training of the initial image classification model using the sample image subset in this round of training.

[0114] In this step, gradient descent and backpropagation are used to correct the gradient direction of the initial image classification model based on the projection matrix orthogonal to the input space during backpropagation, thereby updating the model parameters. This helps to overcome the "catastrophic forgetting problem," improve the model's continuous learning ability, and enhance the accuracy and stability of the trained image classification model when applied to identify images of unknown categories.

[0115] In practical implementation, based on the current model parameters of the initial image classification model, the current gradient of the loss value corresponding to the sample image subset on the initial image classification model, and the projection matrix updated using the sample image subset, the model parameters of the initial image classification model can be updated using the following formula:

[0116] w i_k =w i_k-1 -γ·P i_k ·Δw i_k-1 ;

[0117] Where i represents the layer number corresponding to the projection matrix, k represents the index of the subset of sample images, and k is a positive integer; P i_kw represents the projection matrix of the i-th layer obtained by updating using a subset of the sample images; i_k This represents the model parameter matrix of the i-th layer of the initial image classification model after model parameter updates; w i_k-1 The model parameter matrix of the current i-th layer of the initial image classification model before updating the model parameters; Δw i_k-1 This represents the current gradient matrix of the i-th layer of the initial image classification model, corresponding to the loss value of the subset of sample images; γ represents the learning rate.

[0118] Please return to the reference. Figure 1 S103. If the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset reliable region during this round of training, then according to the second predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

[0119] In this step, for the first and second implementation methods in S101 above, if the task information during this round of training does not enter the preset trusted region, then P i_k =P i_k-1 , where i represents the layer number corresponding to the projection matrix. Accordingly, when this training round is the first round of a multi-round training, the initial projection matrix P0 can be used as the projection matrix obtained from the previous round of training.

[0120] It should be noted that even if the projection matrix obtained in the previous training round is not updated, the projection matrix obtained in this training round is still used to update the model parameters of the initial image classification model during the backpropagation phase of the model training process in this round.

[0121] Furthermore, for the third implementation method in S101 above, step S103 may include:

[0122] S1031. Shuffle and divide the sample image set corresponding to the training task of the model into multiple sample image subsets.

[0123] S1032. For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images.

[0124] S1033. The projection matrix obtained by updating using the last sample image subset in the plurality of sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training.

[0125] Here, the descriptions of S1031 to S1033 can refer to the descriptions of S201 to S203 above, and can achieve the same technical effect, so they will not be repeated here.

[0126] In the process of determining the projection matrix corresponding to the layer updated using a subset of sample images, if the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset confidence region during this round of training, then:

[0127] (1) For each image data of the projection matrix corresponding to the input layer, calculate the target ratio of the first norm and the second norm of the layer using each image data.

[0128] (2) Sort the target ratios calculated using each image data in ascending order, and filter out the m image data corresponding to the first m target ratios, where m is a positive integer.

[0129] (3) Based on the m image data, and using the projection matrix corresponding to the layer updated with the previous sample image subset, the projection matrix corresponding to the layer updated using the sample image subset is determined by the following formula:

[0130]

[0131] Where i represents the layer number corresponding to the projection matrix, k represents the index of the subset of sample images, and i and k are positive integers; P i_k P represents the projection matrix corresponding to the i-th layer obtained by updating using a subset of sample images; i_k-1 This represents the projection matrix corresponding to the i-th layer, updated using the previous subset of sample images; x′ i_k x′ represents the average value of the m image data in the projection matrix corresponding to the i-th layer of the input; i_k T Represents x′ i_k The transpose of ; α represents a hyperparameter and α>0.

[0132] This implementation method can adjust the similarity of the projection matrix input data between training tasks or between sample image subsets (batches). Any abrupt data will not participate in the update of the projection matrix. When training a new task, the method provided in this application will remove abrupt data before applying backpropagation, so that the input data for the projection matrix is ​​as balanced as possible. Then, the gradient of the new sample is projected onto the data space perpendicular to the previously learned data. This method effectively improves the capacity of the neural network while minimizing interference with previously learned knowledge.

[0133] In one experiment, an embodiment of this application designed an image classification model based on the orthogonal weights modification (OWM) algorithm. The first nine layers of this image classification model consist of alternating combinations of convolutional layers, dropout layers, and max pooling layers. The three convolutional layers contain 64, 128, and 256 filters, respectively, with a uniform kernel size of 2×2. The convolutional layers are mainly used for feature extraction. The three max pooling layers have a uniform window size of 2×2. The purpose of the pooling layers is to reduce the dimensionality of the feature map, maintain the feature invariance of the image, and prevent overfitting to some extent. The last three layers of the image classification model consist of dense layers with a corresponding number of neurons of [1000-1000-10]. The image classification model uses a cross-entropy loss function with L2 regularization, and the ReLU activation function is used with weight initialization using the Xavier method. Experimental results show that the image classification method based on continuous learning provided in this embodiment of the application demonstrates a significant improvement in image classification performance.

[0134] This application provides an image classification method based on continuous learning. The method includes: inputting a target image to be classified into a pre-trained image classification model to determine the image category of the target image; wherein the image classification model is trained through the following steps: sequentially acquiring multiple model training tasks under continuous learning conditions; sequentially performing multiple rounds of training on an initial image classification model according to each model training task to obtain a trained image classification model; wherein, for any model training task, during each round of training of the initial image classification model according to the model training task: determining whether the task information of the projection matrix corresponding to each layer in the initial image classification model has entered a preset confidence region at the time of the training round; wherein, the initial projection matrix corresponding to each layer is an identity matrix with the same matrix size as the model parameter size of the initial image classification model; the projection matrix is ​​used to update the model parameters of the initial image classification model during backpropagation; if the task information of the projection matrix corresponding to the layer in the initial image classification model has entered the preset confidence region at the time of the training round, then, according to a first predetermined method, determining the projection matrix corresponding to the layer obtained in the training round based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

[0135] By adjusting the model parameters with an appropriate projection matrix, the model's continuous learning ability can be improved, enhancing the accuracy and stability of the trained image classification model when applied to identify images of unknown categories. Furthermore, updating the projection matrix after the training task information enters a preset trust region can avoid misleading information generated by untrusted training tasks in the model, thereby further improving the performance and accuracy of the image classification model.

[0136] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of an image classification device based on continuous learning, provided in an embodiment of this application. Figure 3 As shown, the device 300 includes:

[0137] The classification module 310 is used to input the target image to be classified into a pre-trained image classification model to determine the image category of the target image;

[0138] Training module 320 is used to train the image classification model through the following steps:

[0139] Sequentially acquire multiple model training tasks under continuous learning conditions;

[0140] The initial image classification model is trained multiple times according to each model training task to obtain the trained image classification model; wherein, for any model training task, during each round of training of the initial image classification model according to that model training task:

[0141] It is determined whether the task information of the projection matrix corresponding to each layer of the initial image classification model has entered the preset confidence region during the training round; wherein, the initial projection matrix corresponding to each layer is an identity matrix with the same matrix size as the model parameter size of the initial image classification model; the projection matrix is ​​used to update the model parameters of the initial image classification model during backpropagation;

[0142] If the task information of the projection matrix corresponding to the layer in the initial image classification model has entered the preset reliable region during this round of training, then according to the first predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

[0143] Furthermore, the training module 320 is also used for:

[0144] If the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset reliable region during this round of training, then according to the second predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

[0145] Furthermore, each model training task includes a preset total number of training rounds and a preset round threshold; when the training module 320 determines whether the task information of the projection matrix corresponding to each layer of the initial image classification model has entered the preset confidence region during the training round, the training module 320 is used to:

[0146] Obtain the task information of the projection matrix corresponding to each layer of the model training task in this round of training; the task information includes the number of training rounds corresponding to this round of training.

[0147] If the difference between the number of training rounds corresponding to the current training round and the preset total number of training rounds is less than the preset number of rounds threshold, then it is determined that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset reliable region during the current training round.

[0148] Furthermore, each model training task includes a preset loss threshold corresponding to the model training task or a preset gradient threshold corresponding to each layer in the model training task; when the training module 320 determines whether the task information of the projection matrix corresponding to each layer in the initial image classification model during this round of training has entered the preset confidence region, the training module 320 is used to:

[0149] Obtain task information for the model training task, specifically the projection matrix of each layer during this training round; the task information includes the loss value corresponding to this training round or the gradient matrix of each layer obtained through backpropagation.

[0150] If the loss value corresponding to the training round is less than the preset loss value threshold, it is determined that the task information of the projection matrix corresponding to each layer of the model training task has entered the preset reliable region during the training round.

[0151] Alternatively, if the sum of the absolute values ​​of each element in the gradient matrix of each layer is less than the preset gradient threshold corresponding to that layer, then it is determined that the task information of the projection matrix corresponding to that layer has entered the preset reliable region during the training round of the model.

[0152] Furthermore, each model training task includes a threshold for the ratio of the gradient norms corresponding to each layer in the model training task; when the training module 320 determines whether the task information of the projection matrix corresponding to each layer in the initial image classification model has entered the preset confidence region during this round of training, the training module 320 is used to:

[0153] For each layer in the initial image classification model, the task information of the projection matrix corresponding to that layer during the current training round is obtained. The task information includes the ratio of the first norm and the second norm of that layer during the current training round. The first norm is the norm of the product of the projection matrix corresponding to that layer after projection and the current gradient. The second norm is the norm of the current gradient.

[0154] If the ratio is less than the threshold of the ratio of the gradient norm of the corresponding layer, it is determined that the task information of the projection matrix of the corresponding layer has entered the preset reliable region during the training round of the model.

[0155] Furthermore, each model training task also includes a sample image set and image labels corresponding to that model training task; when the training module 320 determines the projection matrix corresponding to the layer obtained in the current training round according to the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training in a first predetermined manner, the training module 320 is used to:

[0156] The sample image set corresponding to the training task of this model is shuffled and divided into multiple sample image subsets;

[0157] For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images; wherein, when the subset of sample images is the first subset of sample images in the plurality of subsets of sample images, the projection matrix obtained in the previous round of training or the initial projection matrix is ​​used as the projection matrix updated using the previous subset of sample images;

[0158] The projection matrix obtained by updating using the last sample image subset from the plurality of sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training.

[0159] Furthermore, when each model training task includes a threshold for the ratio of the gradient norm corresponding to that model training task, the training module 320, when used to determine the projection matrix corresponding to the layer obtained in this training round according to the second predetermined method based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous training round if the task information of the projection matrix corresponding to that layer in the initial image classification model does not enter the preset confidence region during this round of training, the training module 320 is used to:

[0160] The sample image set corresponding to the training task of this model is shuffled and divided into multiple sample image subsets;

[0161] For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images.

[0162] The projection matrix obtained by updating using the last sample image subset from the multiple sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training.

[0163] In the process of determining the projection matrix corresponding to the layer updated using the sample image subset, if the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset reliable region during the training round, then for each image data of the input projection matrix corresponding to the layer, the target ratio of the first norm and the second norm corresponding to the layer is calculated using each image data.

[0164] The target ratios calculated using each image data are sorted in ascending order, and the m image data corresponding to the top m target ratios are selected, where m is a positive integer;

[0165] Based on the m image data, and using the projection matrix corresponding to the layer updated with the previous sample image subset, the projection matrix corresponding to the layer updated using the sample image subset is determined by the following formula:

[0166]

[0167] Where i represents the layer number corresponding to the projection matrix, k represents the index of the subset of sample images, and i and k are positive integers; P i_k P represents the projection matrix corresponding to the i-th layer obtained by updating using a subset of sample images; i_k-1 This represents the projection matrix corresponding to the i-th layer, updated using the previous subset of sample images; x′ i_k x′ represents the average value of the m image data in the projection matrix corresponding to the i-th layer of the input; i_k T Represents x′i_k The transpose of ; α represents a hyperparameter and α>0.

[0168] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 4 As shown, the electronic device 400 includes a processor 410, a memory 420, and a bus 430.

[0169] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, they can perform the operations described above. Figure 1 as well as Figure 2 The steps of an image classification method based on continuous learning shown in the method embodiment are described in detail in the method embodiment, and will not be repeated here.

[0170] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 as well as Figure 2 The steps of an image classification method based on continuous learning shown in the method embodiment are described in detail in the method embodiment, and will not be repeated here.

[0171] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0172] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0173] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0174] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0175] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0176] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image classification method based on continuous learning, characterized in that, The method includes: The target image to be classified is input into a pre-trained image classification model to determine the image category of the target image; wherein, the image classification model is trained through the following steps: Sequentially acquire multiple model training tasks under continuous learning conditions; The initial image classification model is trained multiple times according to each model training task to obtain the trained image classification model; wherein, for any model training task, during each round of training of the initial image classification model according to that model training task: It is determined whether the task information of the projection matrix corresponding to each layer of the initial image classification model has entered the preset confidence region during the training round; wherein, the initial projection matrix corresponding to each layer is an identity matrix with the same matrix size as the model parameter size of the initial image classification model; the projection matrix is ​​used to update the model parameters of the initial image classification model during backpropagation; If the task information of the projection matrix corresponding to the layer in the initial image classification model has entered the preset reliable region during the training round, then the projection matrix corresponding to the layer obtained in the training round is determined according to the first predetermined method based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous training round. Each model training task includes a threshold for the ratio of gradient norms corresponding to each layer in the model training task; the determination of whether the task information of the projection matrix corresponding to each layer in the initial image classification model during this round of training has entered the preset reliable region includes: For each layer in the initial image classification model, the task information of the projection matrix corresponding to that layer during the current training round is obtained. The task information includes the ratio of the first norm and the second norm of that layer during the current training round. The first norm is the norm of the product of the projection matrix corresponding to that layer after projection and the current gradient. The second norm is the norm of the current gradient. If the ratio is less than the threshold of the ratio of the gradient norm of the corresponding layer, it is determined that the task information of the projection matrix of the corresponding layer has entered the preset reliable region during the training round of the model.

2. The method according to claim 1, characterized in that, The method further includes: If the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset reliable region during this round of training, then according to the second predetermined method, the projection matrix corresponding to the layer obtained in this round of training is determined based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training.

3. The method according to claim 1, characterized in that, Each model training task also includes a sample image set and image labels corresponding to the model training task; the step of determining the projection matrix corresponding to the layer obtained in the current training round according to the projection matrix corresponding to the layer in the initial image classification model obtained in the previous round of training, according to the first predetermined method, includes: The sample image set corresponding to the training task of this model is shuffled and divided into multiple sample image subsets; For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images; wherein, when the subset of sample images is the first subset of sample images in the plurality of subsets of sample images, the projection matrix obtained in the previous round of training or the initial projection matrix is ​​used as the projection matrix updated using the previous subset of sample images; The projection matrix obtained by updating using the last sample image subset from the plurality of sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training.

4. The method according to claim 2, characterized in that, When each model training task includes a gradient norm ratio threshold corresponding to that model training task, if the task information of the projection matrix corresponding to that layer in the initial image classification model does not enter the preset confidence region during this round of training, then according to the second predetermined method, the projection matrix corresponding to that layer obtained in this round of training is determined based on the projection matrix corresponding to that layer in the initial image classification model obtained in the previous round of training, including: The sample image set corresponding to the training task of this model is shuffled and divided into multiple sample image subsets; For any subset of sample images, based on the subset of sample images and the projection matrix corresponding to the layer updated using the previous subset of sample images, determine the projection matrix corresponding to the layer updated using the subset of sample images. The projection matrix obtained by updating using the last sample image subset from the multiple sample image subsets is determined as the projection matrix corresponding to the layer obtained in this round of training. In the process of determining the projection matrix corresponding to the layer updated using the sample image subset, if the task information of the projection matrix corresponding to the layer in the initial image classification model does not enter the preset reliable region during the training round, then for each image data of the input projection matrix corresponding to the layer, the target ratio of the first norm and the second norm corresponding to the layer is calculated using each image data. The target ratios calculated using each image data are sorted in ascending order, and the m image data corresponding to the top m target ratios are selected, where m is a positive integer; Based on the m image data, and using the projection matrix corresponding to the layer updated with the previous sample image subset, the projection matrix corresponding to the layer updated using the sample image subset is determined by the following formula: ； in, This indicates the layer number corresponding to the projection matrix. This indicates the index corresponding to the subset of sample images. and It is a positive integer; This indicates that the first image obtained by updating using a subset of the sample images is... The projection matrix corresponding to the layer; This indicates that the first sample image is updated using a subset of the previous sample images. The projection matrix corresponding to the layer; Indicates the input number The average value of m image data in the projection matrix corresponding to the layer; express Transpose of; Describe a hyperparameter and .

5. An image classification device based on continuous learning, characterized in that, The device includes: The classification module is used to input the target image to be classified into a pre-trained image classification model to determine the image category of the target image; The training module is used to train the image classification model through the following steps: Sequentially acquire multiple model training tasks under continuous learning conditions; The initial image classification model is trained multiple times according to each model training task to obtain the trained image classification model; wherein, for any model training task, during each round of training of the initial image classification model according to that model training task: It is determined whether the task information of the projection matrix corresponding to each layer of the initial image classification model has entered the preset confidence region during the training round; wherein, the initial projection matrix corresponding to each layer is an identity matrix with the same matrix size as the model parameter size of the initial image classification model; the projection matrix is ​​used to update the model parameters of the initial image classification model during backpropagation; If the task information of the projection matrix corresponding to the layer in the initial image classification model has entered the preset reliable region during the training round, then the projection matrix corresponding to the layer obtained in the training round is determined according to the first predetermined method based on the projection matrix corresponding to the layer in the initial image classification model obtained in the previous training round. Each model training task includes a threshold for the ratio of gradient norms corresponding to each layer in the model training task; when the training module determines whether the task information of the projection matrix corresponding to each layer in the initial image classification model has entered the preset confidence region during this round of training, the training module is used to: For each layer in the initial image classification model, the task information of the projection matrix corresponding to that layer during the current training round is obtained. The task information includes the ratio of the first norm and the second norm of that layer during the current training round. The first norm is the norm of the product of the projection matrix corresponding to that layer after projection and the current gradient. The second norm is the norm of the current gradient. If the ratio is less than the threshold of the ratio of the gradient norm of the corresponding layer, it is determined that the task information of the projection matrix of the corresponding layer has entered the preset reliable region during the training round of the model.

6. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of a continuous learning-based image classification method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of an image classification method based on continuous learning as described in any one of claims 1 to 4.

Images